Prism
v0.2.2
v0.2.2
  • 👋Welcome to Prism!
  • Getting Started
    • Installation
    • Creating your first project
    • Why Prism?
  • Fundamentals
    • Tasks
      • tasks
        • tasks.ref()
      • hooks
        • hooks.sql
        • hooks.spark
        • hooks.dbt_ref
        • hooks.get_connection
        • hooks.get_cursor
    • Targets
      • Multiple targets
    • Config files
      • prism_project.py
        • RUN_ID / SLUG
        • SYS_PATH_CONF
        • THREADS
        • PROFILE_YML_PATH / PROFILE
        • PRISM_LOGGER
        • TRIGGERS_YML_PATH / TRIGGERS
      • Profile YML
      • Triggers YML
    • Jinja
      • __file__ and Path
      • prism_project
      • wkdir
      • parent_dir
      • concat
      • env
  • Adapters
    • Overview
    • sql
      • BigQuery
      • Postgres
      • Redshift
      • Snowflake
      • Trino
      • Presto
    • PySpark
    • dbt
  • Agents
    • Overview
    • Docker
    • EC2
  • CLI
    • Command Line Interface
    • agent
      • apply
      • run
      • build
      • delete
    • compile
    • connect
    • create
      • agent
      • task
      • trigger
    • graph
    • init
    • run
    • spark-submit
  • Advanced features
    • Concurrency
    • Logging
    • Triggers
    • Retries
    • Python Client
  • API Reference
    • prism.task.PrismTask
    • @task(...)
    • @target(...)
    • @target_iterator(...)
    • TaskManager
      • tasks.ref(...)
    • PrismHooks
      • hooks.sql(...)
      • hooks.dbt_ref(...)
      • hooks.get_connection(...)
      • hooks.get_cursor(...)
    • prism.target.PrismTarget
  • Use Cases
    • Analytics on top of dbt
    • Machine Learning
  • Wiki
    • DAGs
Powered by GitBook
On this page
  • Usage
  • Example
  1. CLI
  2. agent

run

Usage

prism agent run executes your project using the most recent version of the agent. For example, for Docker agents, prism run creates a transient Docker container for your custom Docker image and executes the inputted command.

Usage: prism agent run [OPTIONS]                                                                                                    
                                                                                                                                     
 Run your project using an agent.                                                                                                    
                                                                                                                                     
 Examples:                                                                                                                           
                                                                                                                                     
  • prism agent run -f ./ec2.yml --module module01.py --module module02.py                                                           
  • prism agent run -f ./docker.yml --m module01                                                                                     
  • prism agent run -f /Users/docker.yml --vars VAR1=VALUE1                                                                          
                                                                                                                                     
╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ *  --file            -f  TEXT                     Path to agent configuration YML                                                 │
│                                                   [required]                                                                      │
│    --module          -m  TEXT                     Modules to execute. You can specify multiple modules with as follows: -m        │
│                                                   <your_first_module> -m <your_second_module>.                                    │
│    --all-downstream                               Execute all tasks downstream of modules specified with --module                 │
│    --all-upstream                                 Execute all tasks upstream of modules specified with --module                   │
│    --log-level       -l  [info|warn|error|debug]  Set the log level                                                               │
│    --full-tb                                      Show the full traceback when an error occurs                                    │
│    --vars            -v  TEXT                     Variables as key value pairs. These overwrite variables in prism_project.py.    │
│                                                   All values are intepreted as strings.                                           │
│    --context             TEXT                     Context as a dictionary. Must be a valid JSON. These overwrite variables in     │
│                                                   prism_project.py                                                                │
│    --help                                         Show this message and exit.                                                     │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Important: prism agent run only executes your Prism project. It does not rebuild your agent. Prior to calling prism agent run, it's always a good idea to rebuild your agent so that your agent has your most up-to-date code.

Example

Here's what the output looks like in Terminal (using a Docker agent as an example):

$ prism agent run -f ./docker_agent.yml
--------------------------------------------------------------------------------
<HH:MM:DD> | INFO  | Streaming agent logs...

etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | Running with prism v0.2.2...
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | Found project directory at /etl_project
etl-project-docker-agent:1.0[run] |
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'parsing prism_project.py'................................................ [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'parsing prism_project.py'............................................... [DONE in 1.42s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'task DAG'................................................................ [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'task DAG'............................................................... [DONE in 0.01s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'creating pipeline, DAG executor'......................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'creating pipeline, DAG executor'........................................ [DONE in 0.01s]
etl-project-docker-agent:1.0[run] |
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | ======================= tasks 'finicky-macaw-JaEvjyMWtb' =======================
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 1 of 3 RUNNING EVENT 'extract.Extract'...................................................[RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 1 of 3 FINISHED EVENT 'extract.Extract'................................................. [DONE in 120.01s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 2 of 3 RUNNING EVENT 'transform.Transform'.............................................. [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 2 of 3 FINISHED EVENT 'transform.Transform'............................................. [DONE in 791.38s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 3 of 3 RUNNING EVENT 'load.Load'........................................................ [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 3 of 3 FINISHED EVENT 'load.Load'....................................................... [DONE in 7.84s]

<HH:MM:DD> | INFO  | Done streaming agent logs...
--------------------------------------------------------------------------------
PreviousapplyNextbuild

Last updated 1 year ago