Instantiate a Prism DAG object
Parameters:
- project_dir: Prism project directory
- log_level: one of info, warn, error, or critical
PrismDAG.connect(connection_type: str)
Create a connection to an adapter. This replicates the functionality of the prism connect --type CLI command.
Parameters:
- connection_type: Adapter type. Valid adapters are listed here.
PrismDAG.compile(
modules: Optional[List[str]] = None,
all_downstream: bool = True
)
Compile the Prism project. This replicates the functionality of the prism compile.
Parameters:
- modules: A list of modules to compile. If this value is None, then all modules are compiled.
- all_downstream: Whether to compile modules downstream of those in modules. The default is True.
PrismDAG.run(
modules: Optional[List[str]] = None,
all_upstream: bool = True,
all_downstream: bool = True,
full_tb: bool = True,
user_context: Dict[Any, Any] = None
)
Run the Prism project. This replicates the functionality of the prism run.
Parameters:
- modules: A list of modules to compile. If this value is None, then all modules are compiled.
- all_upstream: Whether to compile modules upstream of those in modules. The default is True.
- all_downstream: Whether to compile modules downstream of those in modules. The default is True.
- full_tb: Display full traceback if errors arise at any stage of the pipeline. The default is True.
- user_context: dictionary with variables to overwrite prism_project.py.
PrismDAG.get_task_output(
module_path: pathlib.Path,
bool_run: bool = False,
**kwargs
)
Get output of the task at module_path. If bool_run == True, then run the project first. Note that if bool_run == False, then only tasks with a target can be parsed.
Parameters:
- module_path: path to module (relative to modules/ folder)
- bool_run: boolean indicating whether to run project first; default is False
- **kwargs: keyword arguments for running
Returns:
- task output
PrismDAG.get_pipeline_output(
bool_run: bool = False,
)
Get pipeline output, defined as the output associated with the last task in the project
Parameters:
- bool_run: boolean indicating whether to run project first; default is False
Returns:
- output associated with last task in the project
