build

Usage

prism build combines the functionality of prism agent apply and prism agent run. That is, it constructs your agent using the information in your agent's configuration YML and then immediately executes your project using your newly-built agent.

Usage: prism agent build [OPTIONS]                                                                                                  
                                                                                                                                     
 Build your agent using a configuration YML and then run your project on the newly created agent.                                    
                                                                                                                                     
 Examples:                                                                                                                           
                                                                                                                                     
  • prism agent build -f ./ec2.yml                                                                                                   
  • prism agent build -f /Users/docker.yml --context '{"HI": 1}'                                                                     
                                                                                                                                     
╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ *  --file            -f  TEXT                     Path to agent configuration YML                                                 │
│                                                   [required]                                                                      │
│    --module          -m  TEXT                     Modules to execute. You can specify multiple modules with as follows: -m        │
│                                                   <your_first_module> -m <your_second_module>.                                    │
│    --all-downstream                               Execute all tasks downstream of modules specified with --module                 │
│    --all-upstream                                 Execute all tasks upstream of modules specified with --module                   │
│    --log-level       -l  [info|warn|error|debug]  Set the log level                                                               │
│    --full-tb                                      Show the full traceback when an error occurs                                    │
│    --vars            -v  TEXT                     Variables as key value pairs. These overwrite variables in prism_project.py.    │
│                                                   All values are intepreted as strings.                                           │
│    --context             TEXT                     Context as a dictionary. Must be a valid JSON. These overwrite variables in     │
│                                                   prism_project.py                                                                │
│    --help                                         Show this message and exit.                                                     │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Important: when developing, we recommend using prism agent build over individual calls to prism agent apply and prism agent run. This ensures that your agent always has your most up-to-date code.

Example

Here's what the output looks like in Terminal (using a Docker agent as an example):

$ prism agent build -f ./docker_agent.yml
--------------------------------------------------------------------------------
<HH:MM:DD> | INFO  | Creating agent...
<HH:MM:DD> | INFO  | Did not find `server_url` in configuration...defaulting to unix://var/run/docker.sock

etl-project-docker-agent:1.1[build] | Step 1/7 : FROM python:3.10.8-slim-bullseye
etl-project-docker-agent:1.1[build] |  ---> db19fdf3337d
etl-project-docker-agent:1.1[build] | Step 2/7 : COPY etl_project/requirements.txt ./etl_project/requirements.txt
etl-project-docker-agent:1.1[build] |  ---> Using cache
etl-project-docker-agent:1.1[build] |  ---> 94b39b4230b9
etl-project-docker-agent:1.1[build] | Step 3/7 : RUN pip install --upgrade pip && pip install -r ./etl_project/requirements.txt
etl-project-docker-agent:1.1[build] |  ---> Using cache
etl-project-docker-agent:1.1[build] |  ---> 354ee738fadb
etl-project-docker-agent:1.1[build] | Step 4/7 : COPY etl_project ./etl_project
etl-project-docker-agent:1.1[build] |  ---> 46d1a5fc033c
etl-project-docker-agent:1.1[build] | Step 5/7 : WORKDIR ./etl_project
etl-project-docker-agent:1.1[build] |  ---> Running in dfef365c56da
etl-project-docker-agent:1.1[build] | Removing intermediate container dfef365c56da
etl-project-docker-agent:1.1[build] |  ---> a86e524ab13d
etl-project-docker-agent:1.1[build] | Step 6/7 : ENV AWS_ACCESS_KEY_ID=*****************
etl-project-docker-agent:1.1[build] |  ---> Running in afc0f4cae407
etl-project-docker-agent:1.1[build] | Removing intermediate container afc0f4cae407
etl-project-docker-agent:1.1[build] |  ---> 9a5cfa367c08
etl-project-docker-agent:1.1[build] | Step 7/7 : ENV AWS_SECRET_ACCESS_KEY=**********************************
etl-project-docker-agent:1.1[build] |  ---> Running in 11a198388a11
etl-project-docker-agent:1.1[build] | Removing intermediate container 11a198388a11
etl-project-docker-agent:1.1[build] |  ---> ef1ea0d753fb
etl-project-docker-agent:1.1[build] | Successfully built ef1ea0d753fb
etl-project-docker-agent:1.1[build] | Successfully tagged etl-project-docker-agent:1.1

<HH:MM:DD> | INFO  | Streaming agent logs...

etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | Running with prism v0.2.2...
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | Found project directory at /etl_project
etl-project-docker-agent:1.0[run] |
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'parsing prism_project.py'................................................ [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'parsing prism_project.py'............................................... [DONE in 1.42s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'task DAG'................................................................ [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'task DAG'............................................................... [DONE in 0.01s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'creating pipeline, DAG executor'......................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'creating pipeline, DAG executor'........................................ [DONE in 0.01s]
etl-project-docker-agent:1.0[run] |
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | ======================= tasks 'finicky-macaw-JaEvjyMWtb' =======================
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 1 of 3 RUNNING EVENT 'extract.Extract'...................................................[RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 1 of 3 FINISHED EVENT 'extract.Extract'................................................. [DONE in 120.01s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 2 of 3 RUNNING EVENT 'transform.Transform'.............................................. [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 2 of 3 FINISHED EVENT 'transform.Transform'............................................. [DONE in 791.38s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 3 of 3 RUNNING EVENT 'load.Load'........................................................ [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 3 of 3 FINISHED EVENT 'load.Load'....................................................... [DONE in 7.84s]

<HH:MM:DD> | INFO  | Done streaming agent logs...