run

Usage

prism agent run executes your project using the most recent version of the agent. For example, for Docker agents, prism run creates a transient Docker container for your custom Docker image and executes the inputted command.

Usage: prism agent run [-h] -f FILE [--modules [X.py Y.py ...]] [--all-upstream] [--all-downstream] [--full-tb] [-l] [--vars [A=a B=b ...]] [--context '{}']

Run your project using an agent

Options:
  -h, --help            show this help message and exit

Subcommand Options:
  -f, --file FILE       Path to agent configuration YML

Run Command Options:
  --modules [X.py Y.py ...]
                        Path to script(s) that you want to run; if not specified, all modules in project are run
  --all-upstream        Run all modules upstream of --modules
  --all-downstream      Run all modules downstream of --modules

General Options:
  --full-tb             Display the full traceback for errors in the project; default is False
  -l, --log-level       Log level, must be one of `info`, `warn`, `error`, or `debug`. Default is `info`
  --vars [A=a B=b ...]  Prism variables as key-value pairs `key=value`. These overwrite any variable definitions in `prism_project.py`. All values are read as strings.
  --context '{}'        Prism variables as JSON. Cannot co-exist with --vars. These overwrite any variable definitions in `prism_project.py`.

Important: prism agent run only executes your Prism project. It does not rebuild your agent. Prior to calling prism agent run, it's always a good idea to rebuild your agent so that your agent has your most up-to-date code.

Example

Here's what the output looks like in Terminal (using a Docker agent as an example):

$ prism agent run -f ./docker_agent.yml
--------------------------------------------------------------------------------
<HH:MM:DD> | INFO  | Streaming agent logs...

etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | Running with prism v0.1.9...
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | Found project directory at /etl_project
etl-project-docker-agent:1.0[run] |
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'parsing prism_project.py'................................................ [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'parsing prism_project.py'............................................... [DONE in 1.42s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'module DAG'.............................................................. [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'module DAG'............................................................. [DONE in 0.01s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'creating pipeline, DAG executor'......................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'creating pipeline, DAG executor'........................................ [DONE in 0.01s]
etl-project-docker-agent:1.0[run] |
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | ======================= tasks 'finicky-macaw-JaEvjyMWtb' =======================
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 1 of 3 RUNNING EVENT 'extract.py'....................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 1 of 3 FINISHED EVENT 'extract.py'...................................................... [DONE in 120.01s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 2 of 3 RUNNING EVENT 'transform.py'..................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 2 of 3 FINISHED EVENT 'transform.py'.................................................... [DONE in 791.38s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 3 of 3 RUNNING EVENT 'load.py'.......................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 3 of 3 FINISHED EVENT 'load.py'......................................................... [DONE in 7.84s]

<HH:MM:DD> | INFO  | Done streaming agent logs...
--------------------------------------------------------------------------------