build
Usage
prism build combines the functionality of prism agent apply and prism agent run. That is, it constructs your agent using the information in your agent's configuration YML and then immediately executes your project using your newly-built agent.
Usage: prism agent build [-h] -f FILE [--modules [X.py Y.py ...]] [--all-upstream] [--all-downstream] [--full-tb] [-l] [--vars [A=a B=b ...]] [--context '{}']
Build your agent using a configuration YML and then run your project on the newly-built agent
Options:
-h, --help show this help message and exit
Subcommand Options:
-f, --file FILE Path to agent configuration YML.
Run Command Options:
--modules [X.py Y.py ...]
Path to script(s) that you want to run; if not specified, all modules in project are run
--all-upstream Run all modules upstream of --modules
--all-downstream Run all modules downstream of --modules
General Options:
--full-tb Display the full traceback for errors in the project; default is False
-l, --log-level Log level, must be one of `info`, `warn`, `error`, or `debug`. Default is `info`
--vars [A=a B=b ...] Prism variables as key-value pairs `key=value`. These overwrite any variable definitions in `prism_project.py`. All values are read as strings.
--context '{}' Prism variables as JSON. Cannot co-exist with --vars. These overwrite any variable definitions in `prism_project.py`.Important: when developing, we recommend using prism agent build over individual calls to prism agent apply and prism agent run. This ensures that your agent always has your most up-to-date code.
Example
Here's what the output looks like in Terminal (using a Docker agent as an example):
$ prism agent build -f ./docker_agent.yml
--------------------------------------------------------------------------------
<HH:MM:DD> | INFO | Creating agent...
<HH:MM:DD> | INFO | Did not find `server_url` in configuration...defaulting to unix://var/run/docker.sock
etl-project-docker-agent:1.1[build] | Step 1/7 : FROM python:3.10.8-slim-bullseye
etl-project-docker-agent:1.1[build] | ---> db19fdf3337d
etl-project-docker-agent:1.1[build] | Step 2/7 : COPY etl_project/requirements.txt ./etl_project/requirements.txt
etl-project-docker-agent:1.1[build] | ---> Using cache
etl-project-docker-agent:1.1[build] | ---> 94b39b4230b9
etl-project-docker-agent:1.1[build] | Step 3/7 : RUN pip install --upgrade pip && pip install -r ./etl_project/requirements.txt
etl-project-docker-agent:1.1[build] | ---> Using cache
etl-project-docker-agent:1.1[build] | ---> 354ee738fadb
etl-project-docker-agent:1.1[build] | Step 4/7 : COPY etl_project ./etl_project
etl-project-docker-agent:1.1[build] | ---> 46d1a5fc033c
etl-project-docker-agent:1.1[build] | Step 5/7 : WORKDIR ./etl_project
etl-project-docker-agent:1.1[build] | ---> Running in dfef365c56da
etl-project-docker-agent:1.1[build] | Removing intermediate container dfef365c56da
etl-project-docker-agent:1.1[build] | ---> a86e524ab13d
etl-project-docker-agent:1.1[build] | Step 6/7 : ENV AWS_ACCESS_KEY_ID=*****************
etl-project-docker-agent:1.1[build] | ---> Running in afc0f4cae407
etl-project-docker-agent:1.1[build] | Removing intermediate container afc0f4cae407
etl-project-docker-agent:1.1[build] | ---> 9a5cfa367c08
etl-project-docker-agent:1.1[build] | Step 7/7 : ENV AWS_SECRET_ACCESS_KEY=**********************************
etl-project-docker-agent:1.1[build] | ---> Running in 11a198388a11
etl-project-docker-agent:1.1[build] | Removing intermediate container 11a198388a11
etl-project-docker-agent:1.1[build] | ---> ef1ea0d753fb
etl-project-docker-agent:1.1[build] | Successfully built ef1ea0d753fb
etl-project-docker-agent:1.1[build] | Successfully tagged etl-project-docker-agent:1.1
<HH:MM:DD> | INFO | Streaming agent logs...
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | Running with prism v0.1.9rc2...
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | Found project directory at /etl_project
etl-project-docker-agent:1.0[run] |
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | RUNNING EVENT 'parsing prism_project.py'................................................ [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | FINISHED EVENT 'parsing prism_project.py'............................................... [DONE in 1.42s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | RUNNING EVENT 'module DAG'.............................................................. [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | FINISHED EVENT 'module DAG'............................................................. [DONE in 0.01s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | RUNNING EVENT 'creating pipeline, DAG executor'......................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | FINISHED EVENT 'creating pipeline, DAG executor'........................................ [DONE in 0.01s]
etl-project-docker-agent:1.0[run] |
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | ======================= tasks 'finicky-macaw-JaEvjyMWtb' =======================
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | 1 of 3 RUNNING EVENT 'extract.py'....................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | 1 of 3 FINISHED EVENT 'extract.py'...................................................... [DONE in 120.01s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | 2 of 3 RUNNING EVENT 'transform.py'..................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | 2 of 3 FINISHED EVENT 'transform.py'.................................................... [DONE in 791.38s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | 3 of 3 RUNNING EVENT 'load.py'.......................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO | 3 of 3 FINISHED EVENT 'load.py'......................................................... [DONE in 7.84s]
<HH:MM:DD> | INFO | Done streaming agent logs...Last updated