apply

Usage

prism agent apply constructs your agent using the information in your agent's configuration YML. For example, for Docker agents, prism apply constructs a custom Docker image containing your project data.

Usage: prism agent apply [OPTIONS]                                                                                                  
                                                                                                                                     
 Build your agent using a configuration YML.                                                                                         
                                                                                                                                     
 Examples:                                                                                                                           
                                                                                                                                     
  • prism agent apply -f ./ec2.yml                                                                                                   
  • prism agent apply -f /Users/docker.yml                                                                                           
                                                                                                                                     
╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ *  --file       -f  TEXT                     Path to agent configuration YML                                                      │
│                                              [required]                                                                           │
│    --log-level  -l  [info|warn|error|debug]  Set the log level                                                                    │
│    --full-tb                                 Show the full traceback when an error occurs                                         │
│    --help                                    Show this message and exit.                                                          │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Here's what prism agent apply does under the hood:

1. Parses the project's prism_project.py file and identifies all paths containing project related data (e.g., SYS_PATH_CONF, PROFILE_YML_PATH, TRIGGERS_YML_PATH, and so on).

2. Securely builds the agent computing environment with all the data identified in step 1. For example:

   • For Docker agents, Prism constructs a transient build context and uses Docker's Python SDK to create an image.

   • For EC2 instances / EMR clusters (coming soon!), Prism uses the SSH protocol to securely copy all project data into the virtual machine.

Example

Here's what the output looks like in Terminal (using a Docker agent as an example):

$ prism agent apply -f ./docker_agent.yml
--------------------------------------------------------------------------------
<HH:MM:DD> | INFO  | Creating agent...
<HH:MM:DD> | INFO  | Did not find `server_url` in configuration...defaulting to unix://var/run/docker.sock

etl-project-docker-agent:1.0[build] | Step 1/7 : FROM python:3.10.8-slim-bullseye
etl-project-docker-agent:1.0[build] |  ---> db19fdf3337d
etl-project-docker-agent:1.0[build] | Step 2/7 : COPY etl_project/requirements.txt ./etl_project/requirements.txt
etl-project-docker-agent:1.0[build] |  ---> Using cache
etl-project-docker-agent:1.0[build] |  ---> 94b39b4230b9
etl-project-docker-agent:1.0[build] | Step 3/7 : RUN pip install --upgrade pip && pip install -r ./etl_project/requirements.txt
etl-project-docker-agent:1.0[build] |  ---> Using cache
etl-project-docker-agent:1.0[build] |  ---> 354ee738fadb
etl-project-docker-agent:1.0[build] | Step 4/7 : COPY etl_project ./etl_project
etl-project-docker-agent:1.0[build] |  ---> 46d1a5fc033c
etl-project-docker-agent:1.0[build] | Step 5/7 : WORKDIR ./etl_project
etl-project-docker-agent:1.0[build] |  ---> Running in dfef365c56da
etl-project-docker-agent:1.0[build] | Removing intermediate container dfef365c56da
etl-project-docker-agent:1.0[build] |  ---> a86e524ab13d
etl-project-docker-agent:1.0[build] | Step 6/7 : ENV AWS_ACCESS_KEY_ID=*****************
etl-project-docker-agent:1.0[build] |  ---> Running in afc0f4cae407
etl-project-docker-agent:1.0[build] | Removing intermediate container afc0f4cae407
etl-project-docker-agent:1.0[build] |  ---> 9a5cfa367c08
etl-project-docker-agent:1.0[build] | Step 7/7 : ENV AWS_SECRET_ACCESS_KEY=**********************************
etl-project-docker-agent:1.0[build] |  ---> Running in 11a198388a11
etl-project-docker-agent:1.0[build] | Removing intermediate container 11a198388a11
etl-project-docker-agent:1.0[build] |  ---> ef1ea0d753fb
etl-project-docker-agent:1.0[build] | Successfully built ef1ea0d753fb
etl-project-docker-agent:1.0[build] | Successfully tagged prism-docker-etl-project-docker-agent:1.0