Prism
v0.2.5
v0.2.5
  • 👋Welcome to Prism!
  • Getting Started
    • Installation
    • Creating your first project
    • Why Prism?
  • Fundamentals
    • Tasks
      • tasks
        • tasks.ref()
      • hooks
        • hooks.sql
        • hooks.spark
        • hooks.dbt_ref
        • hooks.get_connection
        • hooks.get_cursor
    • Targets
      • Multiple targets
    • Config files
      • prism_project.py
        • RUN_ID / SLUG
        • SYS_PATH_CONF
        • THREADS
        • PROFILE_YML_PATH / PROFILE
        • PRISM_LOGGER
        • TRIGGERS_YML_PATH / TRIGGERS
      • Profile YML
      • Triggers YML
    • Jinja
      • __file__ and Path
      • prism_project
      • wkdir
      • parent_dir
      • concat
      • env
  • Adapters
    • Overview
    • sql
      • BigQuery
      • Postgres
      • Redshift
      • Snowflake
      • Trino
      • Presto
    • PySpark
    • dbt
  • Agents
    • Overview
    • Docker
    • EC2
  • CLI
    • Command Line Interface
    • agent
      • apply
      • run
      • build
      • delete
    • compile
    • connect
    • create
      • agent
      • task
      • trigger
    • graph
    • init
    • run
    • spark-submit
  • Advanced features
    • Concurrency
    • Logging
    • Triggers
    • Retries
    • Python Client
    • Skipping tasks
  • API Reference
    • prism.task.PrismTask
    • @task(...)
    • @target(...)
    • @target_iterator(...)
    • TaskManager
      • tasks.ref(...)
    • PrismHooks
      • hooks.sql(...)
      • hooks.dbt_ref(...)
      • hooks.get_connection(...)
      • hooks.get_cursor(...)
    • prism.target.PrismTarget
  • Use Cases
    • Analytics on top of dbt
    • Machine Learning
  • Wiki
    • DAGs
Powered by GitBook
On this page
  • Ref-ing tasks in different modules
  • One task in ref'd module
  • Multiple tasks in ref'd module
  • Ref-ing tasks in the same module
  1. Fundamentals
  2. Tasks
  3. tasks

tasks.ref()

PrevioustasksNexthooks

tasks.ref(...) allows users to reference and access other tasks' outputs and/or . This is the most important function in Prism. It is impossible to build even moderately complex workflows without it.

tasks.ref(...) takes two arguments:

  • task: str: the name of the task whose output you'd like to retrieve. This name should be in the form <module_name>, or <module_name>.<task_name>, where <task_name> is the name of the Prism task class / function.

  • local: bool: whether task lives in the same module. The default is False.

Let's look at some examples.

Ref-ing tasks in different modules

One task in ref'd module

In most Prism projects, each module in tasks/ will contain a single task. Let's see how we'd use tasks.ref() in this situation.

# tasks/hello_world.py

import prism.task
import prism.target

class HelloWorld(prism.task.PrismTask):
    
    def run(self, tasks, hooks):
        test_str = "Hello, world!"
        return test_str
# tasks/second_task.py

import prism.task
import prism.target

class SecondTask(prism.task.PrismTask):
    
    def run(self, tasks, hooks):
        hello_world_str = tasks.ref("hello_world")
        additional_details = "\n" + "This is a Prism project"!
        return hello_world_str + additional_details
# tasks/hello_world.py

from prism.decorators import task

@task()
def hello_world(tasks, hooks):
    test_str = "Hello, world!"
    return test_str
# tasks/second_task.py

import prism.decorators import task

@task()
def second_task(tasks, hooks):
    hello_world_str = tasks.ref("hello_world")
    additional_details = "\n" + "This is a Prism project"!
    return hello_world_str + additional_details

Our tasks.ref() is located in the second task. We simply use the name of the module containing the task whose output we want to retrieve. Note that we exclude the .py suffix from the module name.

Important: you could specify the task name using <module_name>.<task_name>, i.e., hello_world.HelloWorld. But this isn't necessary, since hello_world.py only contains a single task.

Multiple tasks in ref'd module

Now, let's look at a situation where the ref'd module has multiple tasks and you want to retrieve the output of a specific task.

The basic usage is still the same. However, since the ref'd module has multiple tasks, you need to specify your task like <module_name>.<task_name>. Here, task_name refers to name of the Prism task class or function.

# tasks/hello_world.py

import prism.task
import prism.target

class HelloWorld(prism.task.PrismTask):
    
    def run(self, tasks, hooks):
        test_str = "Hello, world!"
        return test_str
        
class FooBar(prism.task.PrismTask):
    
    def run(self, tasks, hooks):
        return "foo, bar"
# tasks/second_task.py

import prism.task
import prism.target

class SecondTask(prism.task.PrismTask):
    
    def run(self, tasks, hooks):
        # Get the output of HelloWorld task
        hello_world_str = tasks.ref("hello_world.HelloWorld")
        
        # Get the output of FooBar task
        foo_bar_str = tasks.ref("hello_world.FooBar")
        
        return hello_world_str + foo_bar_str
# tasks/hello_world.py

from prism.decorators import task

@task()
def hello_world(tasks, hooks):
    test_str = "Hello, world!"
    return test_str


@task()
def foo_bar(tasks, hooks):
    return "foo, bar"
# tasks/second_task.py

import prism.decorators import task

@task()
def second_task(tasks, hooks):
    # Get the output of HelloWorld task
    hello_world_str = tasks.ref("hello_world.hello_world")
        
    # Get the output of FooBar task
    foo_bar_str = tasks.ref("hello_world.foo_bar")
    
    return hello_world_str + foo_bar_str

Ref-ing tasks in the same module

Now, let's look at an example where we want to retrieve the output of a task in the same module. This is where the local argument becomes important.

To ref a task in the same module, simply use the task_name (i.e., either the class name or function name), and set local = True. Here's what that looks like:

# tasks/hello_world.py

import prism.task
import prism.target

class HelloWorld(prism.task.PrismTask):
    
    def run(self, tasks, hooks):
        test_str = "Hello, world!"
        return test_str


class FooBar(prism.task.PrismTask):
    
    def run(self, tasks, hooks):
        hello_world_str = tasks.ref("HelloWorld", local=True)
        return hello_world_str + "\n" + "foo, bar"
# tasks/hello_world.py

from prism.decorators import task

@task()
def hello_world(tasks, hooks):
    test_str = "Hello, world!"
    return test_str


@task()
def foo_bar(tasks, hooks):
    hello_world_str = tasks.ref("hello_world", local=True)
    return hello_world_str + "\n" + "foo, bar"

And that's it!

Don't worry if you don't remember all of these details — Prism will give you ample warnings and direction so that you ref your tasks perfectly.

targets