Microsoft Azure Connection

The Microsoft Azure connection type enables the Azure Integrations.

Authenticating to Azure

There are five ways to connect to Azure using Airflow.

  1. Use token credentials i.e. add specific credentials (client_id, secret, tenant) and subscription id to the Airflow connection.

  2. Use a JSON file

  3. Use a JSON dictionary i.e. add a key config directly into the Airflow connection.

  4. Use managed identity by setting managed_identity_client_id, workload_identity_tenant_id (under the hook, it uses DefaultAzureCredential with these arguments)

  5. Fallback on DefaultAzureCredential This includes a mechanism to try different options to authenticate: Managed System Identity, environment variables, authentication through Azure CLI and etc. subscriptionId is required in this authentication mechanism.

Only one authorization method can be used at a time. If you need to manage multiple credentials or keys then you should configure multiple connections.

Default Connection IDs

All hooks and operators related to Microsoft Azure Container Instances use azure_default by default.

Configuring the Connection

Login (optional)

Specify the client_id used for the initial connection. This is only needed for token credentials authentication mechanism. It can be left out to fall back on DefaultAzureCredential.

Password (optional)

Specify the secret used for the initial connection. This is only needed for token credentials authentication mechanism. It can be left out to fall back on DefaultAzureCredential.

Extra (optional)

Specify the extra parameters (as json dictionary) that can be used in Azure connection. The following parameters are all optional:

  • tenantId: Specify the tenant to use. This is only needed for token credentials authentication mechanism.

  • subscriptionId: Specify the subscription id to use. This is only needed for token credentials and DefaultAzureCredential authentication mechanism.

  • key_path: If set, it uses the JSON file authentication mechanism. It specifies the path to the json file that contains the authentication information.

  • key_json: If set, it uses the JSON dictionary authentication mechanism. It specifies the json that contains the authentication information.

  • managed_identity_client_id: The client ID of a user-assigned managed identity. If provided with workload_identity_tenant_id, they’ll pass to DefaultAzureCredential.

  • workload_identity_tenant_id: ID of the application’s Microsoft Entra tenant. Also called its “directory” ID. If provided with managed_identity_client_id, they’ll pass to DefaultAzureCredential.

The entire extra column can be left out to fall back on DefaultAzureCredential.

When specifying the connection in environment variable you should specify it using URI syntax.

Note that all components of the URI should be URL-encoded.

For example:

export AIRFLOW_CONN_AZURE_DEFAULT='azure://?key_path=%2Fkeys%2Fkey.json'

Was this entry helpful?