build
Usage
prism build
combines the functionality of prism agent apply
and prism agent run
. That is, it constructs your agent using the information in your agent's configuration YML and then immediately executes your project using your newly-built agent.
Usage: prism agent build [OPTIONS]
Build your agent using a configuration YML and then run your project on the newly created agent.
Examples:
โข prism agent build -f ./ec2.yml
โข prism agent build -f /Users/docker.yml --context '{"HI": 1}'
โญโ Options โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ * --file -f TEXT Path to agent configuration YML โ
โ [required] โ
โ --module -m TEXT Modules to execute. You can specify multiple modules with as follows: -m โ
โ <your_first_module> -m <your_second_module>. โ
โ --all-downstream Execute all tasks downstream of modules specified with --module โ
โ --all-upstream Execute all tasks upstream of modules specified with --module โ
โ --log-level -l [info|warn|error|debug] Set the log level โ
โ --full-tb Show the full traceback when an error occurs โ
โ --vars -v TEXT Variables as key value pairs. These overwrite variables in prism_project.py. โ
โ All values are intepreted as strings. โ
โ --context TEXT Context as a dictionary. Must be a valid JSON. These overwrite variables in โ
โ prism_project.py โ
โ --help Show this message and exit. โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
Important: when developing, we recommend using prism agent build
over individual calls to prism agent apply
and prism agent run
. This ensures that your agent always has your most up-to-date code.
Example
Here's what the output looks like in Terminal (using a Docker agent as an example):
$ prism agent build -f ./docker_agent.yml
--------------------------------------------------------------------------------
<HH:MM:DD> | INFO | Creating agent...
<HH:MM:DD> | INFO | Did not find `server_url` in configuration...defaulting to unix://var/run/docker.sock
etl-project-docker-agent:1.1[build] | Step 1/7 : FROM python:3.10.8-slim-bullseye
etl-project-docker-agent:1.1[build] | ---> db19fdf3337d
etl-project-docker-agent:1.1[build] | Step 2/7 : COPY etl_project/requirements.txt ./etl_project/requirements.txt
etl-project-docker-agent:1.1[build] | ---> Using cache
etl-project-docker-agent:1.1[build] | ---> 94b39b4230b9
etl-project-docker-agent:1.1[build] | Step 3/7 : RUN pip install --upgrade pip && pip install -r ./etl_project/requirements.txt
etl-project-docker-agent:1.1[build] | ---> Using cache
etl-project-docker-agent:1.1[build] | ---> 354ee738fadb
etl-project-docker-agent:1.1[build] | Step 4/7 : COPY etl_project ./etl_project
etl-project-docker-agent:1.1[build] | ---> 46d1a5fc033c
etl-project-docker-agent:1.1[build] | Step 5/7 : WORKDIR ./etl_project
etl-project-docker-agent:1.1[build] | ---> Running in dfef365c56da
etl-project-docker-agent:1.1[build] | Removing intermediate container dfef365c56da
etl-project-docker-agent:1.1[build] | ---> a86e524ab13d
etl-project-docker-agent:1.1[build] | Step 6/7 : ENV AWS_ACCESS_KEY_ID=*****************
etl-project-docker-agent:1.1[build] | ---> Running in afc0f4cae407
etl-project-docker-agent:1.1[build] | Removing intermediate container afc0f4cae407
etl-project-docker-agent:1.1[build] | ---> 9a5cfa367c08
etl-project-docker-agent:1.1[build] | Step 7/7 : ENV AWS_SECRET_ACCESS_KEY=**********************************
etl-project-docker-agent:1.1[build] | ---> Running in 11a198388a11
etl-project-docker-agent:1.1[build] | Removing intermediate container 11a198388a11
etl-project-docker-agent:1.1[build] | ---> ef1ea0d753fb
etl-project-docker-agent:1.1[build] | Successfully built ef1ea0d753fb
etl-project-docker-agent:1.1[build] | Successfully tagged etl-project-docker-agent:1.1
<HH:MM:DD> | INFO | Streaming agent logs...
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | Running with prism v0.2.0rc1...
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | Found project directory at /etl_project
etl-project-docker-agent:1.0[run] |
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | RUNNING EVENT 'parsing prism_project.py'................................................ [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | FINISHED EVENT 'parsing prism_project.py'............................................... [DONE in 1.42s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | RUNNING EVENT 'module DAG'.............................................................. [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | FINISHED EVENT 'module DAG'............................................................. [DONE in 0.01s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | RUNNING EVENT 'creating pipeline, DAG executor'......................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | FINISHED EVENT 'creating pipeline, DAG executor'........................................ [DONE in 0.01s]
etl-project-docker-agent:1.0[run] |
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | ======================= tasks 'finicky-macaw-JaEvjyMWtb' =======================
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | 1 of 3 RUNNING EVENT 'extract.py'....................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | 1 of 3 FINISHED EVENT 'extract.py'...................................................... [DONE in 120.01s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | 2 of 3 RUNNING EVENT 'transform.py'..................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | 2 of 3 FINISHED EVENT 'transform.py'.................................................... [DONE in 791.38s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | 3 of 3 RUNNING EVENT 'load.py'.......................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | 3 of 3 FINISHED EVENT 'load.py'......................................................... [DONE in 7.84s]
<HH:MM:DD> | INFO | Done streaming agent logs...
Last updated