task

Usage

prism create task is used to create tasks in your project.

Usage: prism create task [-h] [--type TYPE] [-n N] [--name NAME] [--dir DIR] [--full-tb] [-l]

Create new tasks for your project

Options:
  -h, --help         show this help message and exit

Subcommand Options:
  --type TYPE        Task type. One of `python`,`pyspark`. Default is `python`
  -n, --number N     Number of tasks to create. Default is 1.
  --name NAME        Task name. If only a single task is requested, then the task will be named `<task_name>.py`. If multiple tasks are requested, then the tasks will be named `<task_name>_<number>.py`. Tasks
                     should have short, all-lowercase names. Underscores can be used in the module name if it improves readability.
  --dir DIR          Folder within the `modules` directory in which the new tasks should live. If not specified, then new tasks will be dumpted into `modules/`

General Options:
  --full-tb          Display the full traceback for errors in the project; default is False
  -l, --log-level    Log level, must be one of `info`, `warn`, `error`, or `debug`. Default is `info`

This command is fairly straightforward — it creates modules for the desired task type from a standard task template. It also names the file (and the corresponding Prism task class) according to the name argument.

Example

Here's what the output will look like in Terminal (if there are no errors in the project):

$ prism create task --type python --name extract_data
--------------------------------------------------------------------------------
<HH:MM:SS> | INFO  | Running with prism v0.1.9rc1...
<HH:MM:SS> | INFO  | Found project directory at /Users/my_first_project
 
<HH:MM:SS> | INFO  | RUNNING EVENT 'parsing prism_project.py'................................................ [RUN]
<HH:MM:SS> | INFO  | FINISHED EVENT 'parsing prism_project.py'............................................... [DONE in 0.03s]
 
<HH:MM:SS> | INFO  | Creating tasks...
 
<HH:MM:SS> | INFO  | Done!
--------------------------------------------------------------------------------

After running this command, your project directory will look like this:

my_first_project/
  ├── data/
  ├── dev/
  │   └── dev.ipynb
  ├── output/
  ├── modules/
  │   ├── module01.py
  │   ├── module02.py
  │   └── extract_data.py
  ├── prism_project.py
  ├── profile.yml
  └── triggers.yml

Moreover, the extract_data.py task will look like:

# modules/extract_data.py

import prism.task
import prism.target

class ExtractData(prism.task.PrismTask):
    
    def run(self, tasks, hooks):
        # TODO: implement