airflow.providers.amazon.aws.hooks.glue_databrew

Module Contents

Classes

GlueDataBrewHook

Interact with AWS DataBrew.
class airflow.providers.amazon.aws.hooks.glue_databrew.GlueDataBrewHook(*args, **kwargs)[source]

Bases: airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook

Interact with AWS DataBrew.

Additional arguments (such as aws_conn_id) may be specified and are passed down to the underlying AwsBaseHook.

See also

job_completion(job_name, run_id, delay=10, max_attempts=60)[source]

Wait until Glue DataBrew job reaches terminal status.

Parameters

  • job_name (str) – The name of the job being processed during this run.

  • run_id (str) – The unique identifier of the job run.

  • delay (int) – Time in seconds to delay between polls

  • maxAttempts – Maximum number of attempts to poll for completion

Returns

job status

Return type

str

get_job_state(job_name, run_id)[source]

Get the status of a job run.

Parameters

  • job_name (str) – The name of the job being processed during this run.

  • run_id (str) – The unique identifier of the job run.

Returns

State of the job run. ‘STARTING’|’RUNNING’|’STOPPING’|’STOPPED’|’SUCCEEDED’|’FAILED’|’TIMEOUT’

Return type

str

Was this entry helpful?