airflow.providers.google.cloud.sensors.cloud_storage_transfer_service

This module contains a Google Cloud Transfer sensor.

Classes

CloudDataTransferServiceJobStatusSensor

Waits for at least one operation belonging to the job to have the expected status.

Module Contents

class airflow.providers.google.cloud.sensors.cloud_storage_transfer_service.CloudDataTransferServiceJobStatusSensor(*, job_name, expected_statuses, project_id=PROVIDE_PROJECT_ID, gcp_conn_id='google_cloud_default', impersonation_chain=None, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), **kwargs)[source]

Bases: airflow.sensors.base.BaseSensorOperator

Waits for at least one operation belonging to the job to have the expected status.

See also

For more information on how to use this operator, take a look at the guide: CloudDataTransferServiceJobStatusSensor

Parameters:

  • job_name (str) – The name of the transfer job

  • expected_statuses (set[str] | str) – The expected state of the operation. See: https://cloud.google.com/storage-transfer/docs/reference/rest/v1/transferOperations#Status

  • project_id (str) – (Optional) the ID of the project that owns the Transfer Job. If set to None or missing, the default project_id from the Google Cloud connection is used.

  • gcp_conn_id (str) – The connection ID used to connect to Google Cloud.

  • impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

  • deferrable (bool) – Run sensor in deferrable mode

template_fields: collections.abc.Sequence[str] = ('job_name', 'impersonation_chain')[source]
job_name[source]
expected_statuses[source]
project_id = None[source]
gcp_cloud_conn_id = 'google_cloud_default'[source]
impersonation_chain = None[source]
deferrable = True[source]
poke(context)[source]

Override when deriving this class.

execute(context)[source]

Run on the worker and defer using the triggers if deferrable is set to True.

execute_complete(context, event)[source]

Act as a callback for when the trigger fires.

This returns immediately. It relies on trigger to throw an exception, otherwise it assumes execution was successful.

Was this entry helpful?