airflow.providers.amazon.aws.links.emr

Module Contents

Classes

EmrClusterLink

Helper class for constructing Amazon EMR Cluster Link.

EmrLogsLink

Helper class for constructing Amazon EMR Logs Link.

EmrServerlessLogsLink

Helper class for constructing Amazon EMR Serverless link to Spark stdout logs.

EmrServerlessDashboardLink

Helper class for constructing Amazon EMR Serverless Dashboard Link.

EmrServerlessS3LogsLink

Helper class for constructing link to S3 console for Amazon EMR Serverless Logs.

EmrServerlessCloudWatchLogsLink

Helper class for constructing link to CloudWatch console for Amazon EMR Serverless Logs.

Functions

get_serverless_log_uri(*, s3_log_uri, application_id, ...)

Retrieve the S3 URI to EMR Serverless Job logs.

get_serverless_dashboard_url(*[, aws_conn_id, ...])

Retrieve the URL to EMR Serverless dashboard.

get_log_uri(*[, cluster, emr_client, job_flow_id])

Retrieve the S3 URI to the EMR Job logs.

Bases: airflow.providers.amazon.aws.links.base_aws.BaseAwsLink

Helper class for constructing Amazon EMR Cluster Link.

name = 'EMR Cluster'[source]
key = 'emr_cluster'[source]
format_str[source]

Bases: airflow.providers.amazon.aws.links.base_aws.BaseAwsLink

Helper class for constructing Amazon EMR Logs Link.

name = 'EMR Cluster Logs'[source]
key = 'emr_logs'[source]
format_str[source]

Format AWS Service Link.

Some AWS Service Link should require additional escaping in this case this method should be overridden.

airflow.providers.amazon.aws.links.emr.get_serverless_log_uri(*, s3_log_uri, application_id, job_run_id)[source]

Retrieve the S3 URI to EMR Serverless Job logs.

Any EMR Serverless job may have a different S3 logging location (or none), which is an S3 URI. The logging location is then {s3_uri}/applications/{application_id}/jobs/{job_run_id}.

airflow.providers.amazon.aws.links.emr.get_serverless_dashboard_url(*, aws_conn_id=None, emr_serverless_client=None, application_id, job_run_id)[source]

Retrieve the URL to EMR Serverless dashboard.

The URL is a one-use, ephemeral link that expires in 1 hour and is accessible without authentication.

Either an AWS connection ID or existing EMR Serverless client must be passed. If the connection ID is passed, a client is generated using that connection.

airflow.providers.amazon.aws.links.emr.get_log_uri(*, cluster=None, emr_client=None, job_flow_id=None)[source]

Retrieve the S3 URI to the EMR Job logs.

Requires either the output of a describe_cluster call or both an EMR Client and a job_flow_id..

Bases: airflow.providers.amazon.aws.links.base_aws.BaseAwsLink

Helper class for constructing Amazon EMR Serverless link to Spark stdout logs.

name = 'Spark Driver stdout'[source]
key = 'emr_serverless_logs'[source]

Format AWS Service Link.

Some AWS Service Link should require additional escaping in this case this method should be overridden.

Bases: airflow.providers.amazon.aws.links.base_aws.BaseAwsLink

Helper class for constructing Amazon EMR Serverless Dashboard Link.

name = 'EMR Serverless Dashboard'[source]
key = 'emr_serverless_dashboard'[source]

Format AWS Service Link.

Some AWS Service Link should require additional escaping in this case this method should be overridden.

Bases: airflow.providers.amazon.aws.links.base_aws.BaseAwsLink

Helper class for constructing link to S3 console for Amazon EMR Serverless Logs.

name = 'S3 Logs'[source]
key = 'emr_serverless_s3_logs'[source]
format_str[source]

Format AWS Service Link.

Some AWS Service Link should require additional escaping in this case this method should be overridden.

Bases: airflow.providers.amazon.aws.links.base_aws.BaseAwsLink

Helper class for constructing link to CloudWatch console for Amazon EMR Serverless Logs.

This is a deep link that filters on a specific job run.

name = 'CloudWatch Logs'[source]
key = 'emr_serverless_cloudwatch_logs'[source]
format_str[source]

Format AWS Service Link.

Some AWS Service Link should require additional escaping in this case this method should be overridden.

Was this entry helpful?