airflow.providers.microsoft.azure.hooks.wasb

This module contains integration with Azure Blob Storage.

It communicate via the Window Azure Storage Blob protocol. Make sure that a Airflow connection of type wasb exists. Authorization can be done by supplying a login (=Storage account name) and password (=KEY), or login and SAS token in the extra field (see connection wasb_default for an example).

Module Contents

airflow.providers.microsoft.azure.hooks.wasb.BlockBlobService[source]
class airflow.providers.microsoft.azure.hooks.wasb.WasbHook(wasb_conn_id: str = default_conn_name)[source]

Bases: airflow.hooks.base.BaseHook

Interacts with Azure Blob Storage through the wasb:// protocol.

These parameters have to be passed in Airflow Data Base: account_name and account_key.

Additional options passed in the 'extra' field of the connection will be passed to the BlockBlockService() constructor. For example, authenticate using a SAS token by adding {"sas_token": "YOUR_TOKEN"}.

Parameters

wasb_conn_id (str) -- Reference to the wasb connection.

conn_name_attr = wasb_conn_id[source]
default_conn_name = wasb_default[source]
conn_type = wasb[source]
hook_name = Azure Blob Storage[source]
get_conn(self)[source]

Return the BlockBlobService object.

check_for_blob(self, container_name, blob_name, **kwargs)[source]

Check if a blob exists on Azure Blob Storage.

Parameters

  • container_name (str) -- Name of the container.

  • blob_name (str) -- Name of the blob.

  • kwargs (object) -- Optional keyword arguments that BlockBlobService.exists() takes.

Returns

True if the blob exists, False otherwise.

Return type

bool

check_for_prefix(self, container_name: str, prefix: str, **kwargs)[source]

Check if a prefix exists on Azure Blob storage.

Parameters

  • container_name (str) -- Name of the container.

  • prefix (str) -- Prefix of the blob.

  • kwargs (object) -- Optional keyword arguments that BlockBlobService.list_blobs() takes.

Returns

True if blobs matching the prefix exist, False otherwise.

Return type

bool

get_blobs_list(self, container_name: str, prefix: str, **kwargs)[source]

Return a list of blobs from path defined in prefix param

Parameters

  • container_name (str) -- Name of the container.

  • prefix (str) -- Prefix of the blob.

  • kwargs (object) -- Optional keyword arguments that BlockBlobService.list_blobs() takes (num_results, include, delimiter, marker, timeout)

Returns

List of blobs.

Return type

list(azure.storage.common.models.ListGenerator)

load_file(self, file_path: str, container_name: str, blob_name: str, **kwargs)[source]

Upload a file to Azure Blob Storage.

Parameters

  • file_path (str) -- Path to the file to load.

  • container_name (str) -- Name of the container.

  • blob_name (str) -- Name of the blob.

  • kwargs (object) -- Optional keyword arguments that BlockBlobService.create_blob_from_path() takes.

load_string(self, string_data: str, container_name: str, blob_name: str, **kwargs)[source]

Upload a string to Azure Blob Storage.

Parameters

  • string_data (str) -- String to load.

  • container_name (str) -- Name of the container.

  • blob_name (str) -- Name of the blob.

  • kwargs (object) -- Optional keyword arguments that BlockBlobService.create_blob_from_text() takes.

get_file(self, file_path: str, container_name: str, blob_name: str, **kwargs)[source]

Download a file from Azure Blob Storage.

Parameters

  • file_path (str) -- Path to the file to download.

  • container_name (str) -- Name of the container.

  • blob_name (str) -- Name of the blob.

  • kwargs (object) -- Optional keyword arguments that BlockBlobService.create_blob_from_path() takes.

read_file(self, container_name: str, blob_name: str, **kwargs)[source]

Read a file from Azure Blob Storage and return as a string.

Parameters

  • container_name (str) -- Name of the container.

  • blob_name (str) -- Name of the blob.

  • kwargs (object) -- Optional keyword arguments that BlockBlobService.create_blob_from_path() takes.

delete_file(self, container_name: str, blob_name: str, is_prefix: bool = False, ignore_if_missing: bool = False, **kwargs)[source]

Delete a file from Azure Blob Storage.

Parameters

  • container_name (str) -- Name of the container.

  • blob_name (str) -- Name of the blob.

  • is_prefix (bool) -- If blob_name is a prefix, delete all matching files

  • ignore_if_missing (bool) -- if True, then return success even if the blob does not exist.

  • kwargs (object) -- Optional keyword arguments that BlockBlobService.create_blob_from_path() takes.

Was this entry helpful?