# Docker

## Prerequisites

To run Docker agents locally, you must have the Docker engine installed on your machine. More information can be found [here](https://docs.docker.com/engine/install/).

## Configuring your Docker agent

Docker agents are configured using a YML file:

```yaml
# docker_agent.yml

agent:
  type: docker
  image: <base image>
  server_url:  # URL to the docker server
  requirements:  # path to requirements.txt, relative to this file
  env:
    <env var 1>: 'env var 1 value'
```

The YML file has a single top-level key, `agent`. This key contains all the configurations needed for your agent. Specifically:

* `type`: this will always be `docker` for Docker agents
* `image`: the base image for your custom, Prism Docker image. If this is not specified, then Prism defaults to [`python:3.10.8-slim-bullseye`](https://hub.docker.com/_/python).
* `server_url`: URL to the Docker server. If this is not specified, then Prism defaults to `unix://var/run/docker.sock`&#x20;
* `requirements`: path to the `requirements.txt` file. This path must be *relative to the path of the agent YML file.*
* `env`: `{key,value}` pairs representing environment variables to add to your Docker image

## Example agent

For the remainder of this guide, let's assume that we have the following project.

```
etl_project/
  ├── prism_project.py
  ├── modules/
  │   ├── extract.py
  │   ├── transfrom.py
  │   └── load.py
  ├── docker_agent.yml
  ├── requirements.txt
  └── triggers.yml
```

And, let's assume that we defined our `docker_agent.yml` as follows:

```yaml
# docker_agent.yml

agent:
  type: docker
  requirements: ./requirements.txt
  env:
    AWS_ACCESS_KEY_ID: "{{ env('AWS_ACCESS_KEY_ID') }}"
    AWS_SECRET_ACCESS_KEY: "{{ env('AWS_SECRET_ACCESS_KEY') }}"

```

Note the way we specified the `requirements` key. We can do this because `docker_agent.yml` and `requirements.txt` are in the same directory.

## Creating your Docker agent

You can build your Docker agent using the command: `prism agent apply -f <path to agent YML>`:

```
$ prism agent apply -f ./docker_agent.yml
--------------------------------------------------------------------------------
<HH:MM:DD> | INFO  | Creating agent...
<HH:MM:DD> | INFO  | Did not find `server_url` in configuration...defaulting to unix://var/run/docker.sock

etl-project-docker-agent:1.0[build] | Step 1/7 : FROM python:3.10.8-slim-bullseye
etl-project-docker-agent:1.0[build] |  ---> db19fdf3337d
etl-project-docker-agent:1.0[build] | Step 2/7 : COPY etl_project/requirements.txt ./etl_project/requirements.txt
etl-project-docker-agent:1.0[build] |  ---> Using cache
etl-project-docker-agent:1.0[build] |  ---> 94b39b4230b9
etl-project-docker-agent:1.0[build] | Step 3/7 : RUN pip install --upgrade pip && pip install -r ./etl_project/requirements.txt
etl-project-docker-agent:1.0[build] |  ---> Using cache
etl-project-docker-agent:1.0[build] |  ---> 354ee738fadb
etl-project-docker-agent:1.0[build] | Step 4/7 : COPY etl_project ./etl_project
etl-project-docker-agent:1.0[build] |  ---> 46d1a5fc033c
etl-project-docker-agent:1.0[build] | Step 5/7 : WORKDIR ./etl_project
etl-project-docker-agent:1.0[build] |  ---> Running in dfef365c56da
etl-project-docker-agent:1.0[build] | Removing intermediate container dfef365c56da
etl-project-docker-agent:1.0[build] |  ---> a86e524ab13d
etl-project-docker-agent:1.0[build] | Step 6/7 : ENV AWS_ACCESS_KEY_ID=*****************
etl-project-docker-agent:1.0[build] |  ---> Running in afc0f4cae407
etl-project-docker-agent:1.0[build] | Removing intermediate container afc0f4cae407
etl-project-docker-agent:1.0[build] |  ---> 9a5cfa367c08
etl-project-docker-agent:1.0[build] | Step 7/7 : ENV AWS_SECRET_ACCESS_KEY=**********************************
etl-project-docker-agent:1.0[build] |  ---> Running in 11a198388a11
etl-project-docker-agent:1.0[build] | Removing intermediate container 11a198388a11
etl-project-docker-agent:1.0[build] |  ---> ef1ea0d753fb
etl-project-docker-agent:1.0[build] | Successfully built ef1ea0d753fb
etl-project-docker-agent:1.0[build] | Successfully tagged prism-docker-etl-project-docker-agent:1.0
```

This commands builds a custom Docker image. In the example above, this image is called `etl-project-docker-agent:1.0`.

Multiple `prism agent apply` calls will increment the tag associated with the custom Docker image. For example, if we were to call `prism agent apply` again, then Prism would re-build the Docker image and name it `etl-project-docker-agent:1.1`.

## Running your Docker agent

There are two commands you can use to run your Docker agent.

<details>

<summary>prism agent run</summary>

The `prism agent run` command runs a transient container with your Docker image. This container is automatically removed on daemon side when the container’s process exits.

```
$ prism agent run -f ./docker_agent.yml
--------------------------------------------------------------------------------
<HH:MM:DD> | INFO  | Streaming agent logs...

etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | Running with prism v0.2.0rc1...
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | Found project directory at /etl_project
etl-project-docker-agent:1.0[run] |
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'parsing prism_project.py'................................................ [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'parsing prism_project.py'............................................... [DONE in 1.42s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'module DAG'.............................................................. [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'module DAG'............................................................. [DONE in 0.01s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'creating pipeline, DAG executor'......................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'creating pipeline, DAG executor'........................................ [DONE in 0.01s]
etl-project-docker-agent:1.0[run] |
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | ======================= tasks 'finicky-macaw-JaEvjyMWtb' =======================
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 1 of 3 RUNNING EVENT 'extract.py'....................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 1 of 3 FINISHED EVENT 'extract.py'...................................................... [DONE in 120.01s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 2 of 3 RUNNING EVENT 'transform.py'..................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 2 of 3 FINISHED EVENT 'transform.py'.................................................... [DONE in 791.38s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 3 of 3 RUNNING EVENT 'load.py'.......................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 3 of 3 FINISHED EVENT 'load.py'......................................................... [DONE in 7.84s]

<HH:MM:DD> | INFO  | Done streaming agent logs...
--------------------------------------------------------------------------------
```

</details>

<details>

<summary>prism agent build</summary>

The `prism agent build` command *re-builds your Docker image* and then runs a transient container with your newly built image. As with `prism agent run`, this container is automatically removed on daemon side when the container’s process exits.

```
$ prism agent build -f ./docker_agent.yml
--------------------------------------------------------------------------------
<HH:MM:DD> | INFO  | Creating agent...
<HH:MM:DD> | INFO  | Did not find `server_url` in configuration...defaulting to unix://var/run/docker.sock

etl-project-docker-agent:1.1[build] | Step 1/7 : FROM python:3.10.8-slim-bullseye
etl-project-docker-agent:1.1[build] |  ---> db19fdf3337d
etl-project-docker-agent:1.1[build] | Step 2/7 : COPY etl_project/requirements.txt ./etl_project/requirements.txt
etl-project-docker-agent:1.1[build] |  ---> Using cache
etl-project-docker-agent:1.1[build] |  ---> 94b39b4230b9
etl-project-docker-agent:1.1[build] | Step 3/7 : RUN pip install --upgrade pip && pip install -r ./etl_project/requirements.txt
etl-project-docker-agent:1.1[build] |  ---> Using cache
etl-project-docker-agent:1.1[build] |  ---> 354ee738fadb
etl-project-docker-agent:1.1[build] | Step 4/7 : COPY etl_project ./etl_project
etl-project-docker-agent:1.1[build] |  ---> 46d1a5fc033c
etl-project-docker-agent:1.1[build] | Step 5/7 : WORKDIR ./etl_project
etl-project-docker-agent:1.1[build] |  ---> Running in dfef365c56da
etl-project-docker-agent:1.1[build] | Removing intermediate container dfef365c56da
etl-project-docker-agent:1.1[build] |  ---> a86e524ab13d
etl-project-docker-agent:1.1[build] | Step 6/7 : ENV AWS_ACCESS_KEY_ID=*****************
etl-project-docker-agent:1.1[build] |  ---> Running in afc0f4cae407
etl-project-docker-agent:1.1[build] | Removing intermediate container afc0f4cae407
etl-project-docker-agent:1.1[build] |  ---> 9a5cfa367c08
etl-project-docker-agent:1.1[build] | Step 7/7 : ENV AWS_SECRET_ACCESS_KEY=**********************************
etl-project-docker-agent:1.1[build] |  ---> Running in 11a198388a11
etl-project-docker-agent:1.1[build] | Removing intermediate container 11a198388a11
etl-project-docker-agent:1.1[build] |  ---> ef1ea0d753fb
etl-project-docker-agent:1.1[build] | Successfully built ef1ea0d753fb
etl-project-docker-agent:1.1[build] | Successfully tagged etl-project-docker-agent:1.1

<HH:MM:DD> | INFO  | Streaming agent logs...

etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | Running with prism v0.2.0rc1...
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | Found project directory at /etl_project
etl-project-docker-agent:1.0[run] |
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'parsing prism_project.py'................................................ [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'parsing prism_project.py'............................................... [DONE in 1.42s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'module DAG'.............................................................. [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'module DAG'............................................................. [DONE in 0.01s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | RUNNING EVENT 'creating pipeline, DAG executor'......................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | FINISHED EVENT 'creating pipeline, DAG executor'........................................ [DONE in 0.01s]
etl-project-docker-agent:1.0[run] |
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | ======================= tasks 'finicky-macaw-JaEvjyMWtb' =======================
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 1 of 3 RUNNING EVENT 'extract.py'....................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 1 of 3 FINISHED EVENT 'extract.py'...................................................... [DONE in 120.01s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 2 of 3 RUNNING EVENT 'transform.py'..................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 2 of 3 FINISHED EVENT 'transform.py'.................................................... [DONE in 791.38s]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 3 of 3 RUNNING EVENT 'load.py'.......................................................... [RUN]
etl-project-docker-agent:1.0[run] | <HH:MM:DD> | INFO  | 3 of 3 FINISHED EVENT 'load.py'......................................................... [DONE in 7.84s]

<HH:MM:DD> | INFO  | Done streaming agent logs...
```

Notice that this command re-built the image: now, the image tag is `1.1`, not `1.0`.

</details>

More information on these commands can be found in the [CLI](https://docs.runprism.com/v0.2.0rc1/cli) section.