• Home
• airflow.providers.google
• airflow.providers.google.cloud
• airflow.providers.google.cloud.operators
• airflow.providers.google.cloud.operators.dataform

airflow.providers.google.cloud.operators.dataform

Module Contents

Classes

DataformCreateCompilationResultOperator	Creates a new CompilationResult in a given project and location.
DataformGetCompilationResultOperator	Fetches a single CompilationResult.
DataformCreateWorkflowInvocationOperator	Creates a new WorkflowInvocation in a given Repository.
DataformGetWorkflowInvocationOperator	Fetches a single WorkflowInvocation.
DataformCancelWorkflowInvocationOperator	Requests cancellation of a running WorkflowInvocation.

class airflow.providers.google.cloud.operators.dataform.DataformCreateCompilationResultOperator(project_id, region, repository_id, compilation_result, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', delegate_to=None, impersonation_chain=None, *args, **kwargs)

Bases: airflow.models.BaseOperator

Creates a new CompilationResult in a given project and location.

Parameters

• project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.

• region (str) – Required. The ID of the Google Cloud region that the task belongs to.

• repository_id (str) – Required. The ID of the Dataform repository that the task belongs to.

• compilation_result (CompilationResult | dict) – Required. The compilation result to create.

• retry (Retry | _MethodDefault) – Designation of what errors, if any, should be retried.

• timeout (float | None) – The timeout for this request.

• metadata (Sequence[tuple[str, str]]) – Strings which should be sent along with the request as metadata.

• gcp_conn_id (str) – The connection ID to use when fetching connection info.

• delegate_to (str | None) – The account to impersonate, if any. For this to work, the service accountmaking the request must have domain-wide delegation enabled.

• impersonation_chain (str | Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

execute(context)

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataform.DataformGetCompilationResultOperator(project_id, region, repository_id, compilation_result_id, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', delegate_to=None, impersonation_chain=None, *args, **kwargs)

Bases: airflow.models.BaseOperator

Fetches a single CompilationResult.

Parameters

• project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.

• region (str) – Required. The ID of the Google Cloud region that the task belongs to.

• repository_id (str) – Required. The ID of the Dataform repository that the task belongs to.

• compilation_result_id (str) – The Id of the Dataform Compilation Result

• retry (Retry | _MethodDefault) – Designation of what errors, if any, should be retried.

• timeout (float | None) – The timeout for this request.

• metadata (Sequence[tuple[str, str]]) – Strings which should be sent along with the request as metadata.

• gcp_conn_id (str) – The connection ID to use when fetching connection info.

• delegate_to (str | None) – The account to impersonate, if any. For this to work, the service accountmaking the request must have domain-wide delegation enabled.

• impersonation_chain (str | Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields = ['repository_id', 'compilation_result_id', 'delegate_to', 'impersonation_chain']

execute(context)

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataform.DataformCreateWorkflowInvocationOperator(project_id, region, repository_id, workflow_invocation, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', delegate_to=None, impersonation_chain=None, asynchronous=False, wait_time=10, *args, **kwargs)

Bases: airflow.models.BaseOperator

Creates a new WorkflowInvocation in a given Repository.

Parameters

• project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.

• region (str) – Required. The ID of the Google Cloud region that the task belongs to.

• repository_id (str) – Required. The ID of the Dataform repository that the task belongs to.

• workflow_invocation (WorkflowInvocation | dict) – Required. The workflow invocation resource to create.

• retry (Retry | _MethodDefault) – Designation of what errors, if any, should be retried.

• timeout (int | None) – The timeout for this request.

• metadata (Sequence[tuple[str, str]]) – Strings which should be sent along with the request as metadata.

• gcp_conn_id (str) – The connection ID to use when fetching connection info.

• delegate_to (str | None) – The account to impersonate, if any. For this to work, the service accountmaking the request must have domain-wide delegation enabled.

• impersonation_chain (str | Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

• asynchronous (bool) – Flag to return workflow_invocation_id from the Dataform API. This is useful for submitting long running workflows and waiting on them asynchronously using the DataformWorkflowInvocationStateSensor

• wait_time (int) – Number of seconds between checks

template_fields = ['workflow_invocation', 'delegate_to', 'impersonation_chain']

operator_extra_links

execute(context)

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataform.DataformGetWorkflowInvocationOperator(project_id, region, repository_id, workflow_invocation_id, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', delegate_to=None, impersonation_chain=None, *args, **kwargs)

Bases: airflow.models.BaseOperator

Fetches a single WorkflowInvocation.

Parameters

• project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.

• region (str) – Required. The ID of the Google Cloud region that the task belongs to.

• repository_id (str) – Required. The ID of the Dataform repository that the task belongs to.

• workflow_invocation_id (str) – the workflow invocation resource’s id.

• retry (Retry | _MethodDefault) – Designation of what errors, if any, should be retried.

• timeout (float | None) – The timeout for this request.

• metadata (Sequence[tuple[str, str]]) – Strings which should be sent along with the request as metadata.

• gcp_conn_id (str) – The connection ID to use when fetching connection info.

• delegate_to (str | None) – The account to impersonate, if any. For this to work, the service accountmaking the request must have domain-wide delegation enabled.

• impersonation_chain (str | Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields = ['repository_id', 'workflow_invocation_id', 'delegate_to', 'impersonation_chain']

operator_extra_links

execute(context)

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataform.DataformCancelWorkflowInvocationOperator(project_id, region, repository_id, workflow_invocation_id, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', delegate_to=None, impersonation_chain=None, *args, **kwargs)

Bases: airflow.models.BaseOperator

Requests cancellation of a running WorkflowInvocation.

Parameters

• project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.

• region (str) – Required. The ID of the Google Cloud region that the task belongs to.

• repository_id (str) – Required. The ID of the Dataform repository that the task belongs to.

• workflow_invocation_id (str) – the workflow invocation resource’s id.

• retry (Retry | _MethodDefault) – Designation of what errors, if any, should be retried.

• timeout (float | None) – The timeout for this request.

• metadata (Sequence[tuple[str, str]]) – Strings which should be sent along with the request as metadata.

• gcp_conn_id (str) – The connection ID to use when fetching connection info.

• delegate_to (str | None) – The account to impersonate, if any. For this to work, the service accountmaking the request must have domain-wide delegation enabled.

• impersonation_chain (str | Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields = ['repository_id', 'workflow_invocation_id', 'delegate_to', 'impersonation_chain']

operator_extra_links

execute(context)

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

Previous Next

Was this entry helpful?