airflow.providers.amazon.aws.hooks.base_aws
¶
This module contains Base AWS Hook.
See also
For more information on how to use this hook, take a look at the guide: Amazon Web Services Connection
Module Contents¶
Classes¶
Base AWS Session Factory class. |
|
Generic class for interact with AWS. |
|
Base class for interact with AWS. |
Functions¶
Resolve custom SessionFactory class. |
Attributes¶
- class airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory(conn, region_name=None, config=None)[source]¶
Bases:
airflow.utils.log.logging_mixin.LoggingMixin
Base AWS Session Factory class.
This handles synchronous and async boto session creation. It can handle most of the AWS supported authentication methods.
User can also derive from this class to have full control of boto3 session creation or to support custom federation.
Note
Not all features implemented for synchronous sessions are available for async sessions.
See also
- class airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook(aws_conn_id=default_conn_name, verify=None, region_name=None, client_type=None, resource_type=None, config=None)[source]¶
Bases:
airflow.hooks.base.BaseHook
,Generic
[BaseAwsConnection
]Generic class for interact with AWS.
This class provide a thin wrapper around the boto3 Python library.
- Parameters
aws_conn_id (str | None) – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).
verify (bool | str | None) – Whether or not to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html
region_name (str | None) – AWS region_name. If not specified then the default boto3 behaviour is used.
client_type (str | None) – Reference to
boto3.client service_name
, e.g. ‘emr’, ‘batch’, ‘s3’, etc. Mutually exclusive withresource_type
.resource_type (str | None) – Reference to
boto3.resource service_name
, e.g. ‘s3’, ‘ec2’, ‘dynamodb’, etc. Mutually exclusive withclient_type
.config (botocore.config.Config | dict[str, Any] | None) – Configuration for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html
- property config: botocore.config.Config[source]¶
Configuration for botocore client read-only property.
- property verify: bool | str | None[source]¶
Verify or not SSL certificates boto3 client/resource read-only property.
- property conn_client_meta: botocore.client.ClientMeta[source]¶
Get botocore client metadata from Hook connection (cached).
- property conn_partition: str[source]¶
Get associated AWS Region Partition from Hook connection (cached).
- get_session(region_name=None, deferrable=False)[source]¶
Get the underlying boto3.session.Session(region_name=region_name).
- get_client_type(region_name=None, config=None, deferrable=False)[source]¶
Get the underlying boto3 client using boto3 session.
- get_resource_type(region_name=None, config=None)[source]¶
Get the underlying boto3 resource using boto3 session.
- conn()[source]¶
Get the underlying boto3 client/resource (cached).
- Returns
boto3.client or boto3.resource
- Return type
BaseAwsConnection
- get_conn()[source]¶
Get the underlying boto3 client/resource (cached).
Implemented so that caching works as intended. It exists for compatibility with subclasses that rely on a super().get_conn() method.
- Returns
boto3.client or boto3.resource
- Return type
BaseAwsConnection
- get_credentials(region_name=None)[source]¶
Get the underlying botocore.Credentials object.
This contains the following authentication attributes: access_key, secret_key and token. By use this method also secret_key and token will mask in tasks logs.
- expand_role(role, region_name=None)[source]¶
Get the Amazon Resource Name (ARN) for the role.
If IAM role is already an IAM role ARN, the value is returned unchanged.
- static retry(should_retry)[source]¶
Repeat requests in response to exceeding a temporary quote limit.
- test_connection()[source]¶
Test the AWS connection by call AWS STS (Security Token Service) GetCallerIdentity API.
- get_waiter(waiter_name, parameters=None, deferrable=False, client=None)[source]¶
Get a waiter by name.
First checks if there is a custom waiter with the provided waiter_name and uses that if it exists, otherwise it will check the service client for a waiter that matches the name and pass that through.
If deferrable is True, the waiter will be an AIOWaiter, generated from the client that is passed as a parameter. If deferrable is True, client must be provided.
- Parameters
waiter_name (str) – The name of the waiter. The name should exactly match the name of the key in the waiter model file (typically this is CamelCase).
parameters (dict[str, str] | None) – will scan the waiter config for the keys of that dict, and replace them with the corresponding value. If a custom waiter has such keys to be expanded, they need to be provided here.
deferrable (bool) – If True, the waiter is going to be an async custom waiter. An async client must be provided in that case.
client – The client to use for the waiter’s operations
- class airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook(aws_conn_id=default_conn_name, verify=None, region_name=None, client_type=None, resource_type=None, config=None)[source]¶
Bases:
AwsGenericHook
[Union
[boto3.client
,boto3.resource
]]Base class for interact with AWS.
This class provide a thin wrapper around the boto3 Python library.
- Parameters
aws_conn_id (str | None) – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).
verify (bool | str | None) – Whether or not to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html
region_name (str | None) – AWS region_name. If not specified then the default boto3 behaviour is used.
client_type (str | None) – Reference to
boto3.client service_name
, e.g. ‘emr’, ‘batch’, ‘s3’, etc. Mutually exclusive withresource_type
.resource_type (str | None) – Reference to
boto3.resource service_name
, e.g. ‘s3’, ‘ec2’, ‘dynamodb’, etc. Mutually exclusive withclient_type
.config (botocore.config.Config | dict[str, Any] | None) – Configuration for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html