BigQueryConnector

Configuration

The required BigQueryConnector arguments are:

• id:

• creds: the path to the Google authentication credentials. The default is the environment variable GOOGLE_APPLICATION_CREDENTIALS.

bigquery_connector = BigQueryConnector(
    id="bigquery_connector_id",
    creds="/example_path/creds.json"
)

Under the hood, prism interacts with the BigQuery Python API to create the SQL engine. For more information, see here the Google BigQuery documentation.

execute_sql

You can run queries against the BigQuery engine using the execute_sql function:

from prism.decorators import task
from prism.runtime import CurrentRun

@task()
def bigquery_task(self):
    conn = CurrentRun.conn("bigquery_connector_id")
    data = conn.execute_sql(
        sql="SELECT * FROM table"
    )

Note that when return_type = None, the result will be a list of Row objects containing the query data.