task

Usage

prism create task is used to create tasks in your project.

Usage: prism create task [OPTIONS]                                                                                                  
                                                                                                                                     
 Create new tasks for your project.                                                                                                  
                                                                                                                                     
 Examples:                                                                                                                           
                                                                                                                                     
  • prism create task --type python                                                                                                  
  • prism create task --n 3 --name extract_step                                                                                      
                                                                                                                                     
╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --type       -t  [python|pyspark]         Task type. Default is python                                                            │
│ --decorated                               If specified, the task will be a decorated function                                     │
│ --number         INTEGER                  Number of tasks to create. Default is 1.                                                │
│ --name           TEXT                     Task name. If only a single task is requested, then the task will be named              │
│                                           <task_name>.py. If multiple tasks are requested, then the tasks will be named           │
│                                           <task_name>_<number>.py. Tasks should have short, all-lowercase names. Underscores can  │
│                                           be used in the module name if it improves readability.                                  │
│ --dir            TEXT                     Folder within the modules directory in which the new tasks should live. If not          │
│                                           specified, then new tasks will be dumpted into modules/                                 │
│ --log-level  -l  [info|warn|error|debug]  Set the log level                                                                       │
│ --full-tb                                 Show the full traceback when an error occurs                                            │
│ --help                                    Show this message and exit.                                                             │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

This command is fairly straightforward — it creates modules for the desired task type from a standard task template. It also names the file (and the corresponding Prism task class) according to the name argument.

Example

Here's what the output will look like in Terminal (if there are no errors in the project):

$ prism create task --type python --name extract_data
--------------------------------------------------------------------------------
<HH:MM:SS> | INFO  | Running with prism v0.2.3...
<HH:MM:SS> | INFO  | Found project directory at /Users/my_first_project
 
<HH:MM:SS> | INFO  | RUNNING EVENT 'parsing prism_project.py'................................................ [RUN]
<HH:MM:SS> | INFO  | FINISHED EVENT 'parsing prism_project.py'............................................... [DONE in 0.03s]
 
<HH:MM:SS> | INFO  | Creating tasks...
 
<HH:MM:SS> | INFO  | Done!
--------------------------------------------------------------------------------

After running this command, your project directory will look like this:

my_first_project/
  ├── data/
  ├── dev/
  │   └── dev.ipynb
  ├── output/
  ├── modules/
  │   ├── module01.py
  │   ├── module02.py
  │   └── extract_data.py
  ├── prism_project.py
  ├── profile.yml
  └── triggers.yml

Moreover, the extract_data.py task will look like:

# modules/extract_data.py

import prism.task
import prism.target

class ExtractData(prism.task.PrismTask):
    
    def run(self, tasks, hooks):
        # TODO: implement