airflow.providers.microsoft.azure.operators.container_instances¶
Module Contents¶
-
airflow.providers.microsoft.azure.operators.container_instances.DEFAULT_ENVIRONMENT_VARIABLES:Dict[str, str][source]¶
-
airflow.providers.microsoft.azure.operators.container_instances.DEFAULT_SECURED_VARIABLES:Sequence[str] = [][source]¶
-
airflow.providers.microsoft.azure.operators.container_instances.DEFAULT_VOLUMES:Sequence[Volume] = [][source]¶
-
class
airflow.providers.microsoft.azure.operators.container_instances.AzureContainerInstancesOperator(*, ci_conn_id: str, registry_conn_id: Optional[str], resource_group: str, name: str, image: str, region: str, environment_variables: Optional[dict] = None, secured_variables: Optional[str] = None, volumes: Optional[list] = None, memory_in_gb: Optional[Any] = None, cpu: Optional[Any] = None, gpu: Optional[Any] = None, command: Optional[List[str]] = None, remove_on_error: bool = True, fail_if_exists: bool = True, tags: Optional[Dict[str, str]] = None, os_type: str = 'Linux', restart_policy: str = 'Never', ip_address: Optional[IpAddress] = None, ports: Optional[List[ContainerPort]] = None, **kwargs)[source]¶ Bases:
airflow.models.BaseOperatorStart a container on Azure Container Instances
- Parameters
ci_conn_id (str) -- connection id of a service principal which will be used to start the container instance
registry_conn_id (Optional[str]) -- connection id of a user which can login to a private docker registry. For Azure use Azure connection id
resource_group (str) -- name of the resource group wherein this container instance should be started
name (str) -- name of this container instance. Please note this name has to be unique in order to run containers in parallel.
image (str) -- the docker image to be used
region (str) -- the region wherein this container instance should be started
environment_variables (Optional[dict]) -- key,value pairs containing environment variables which will be passed to the running container
secured_variables (Optional[str]) -- names of environmental variables that should not be exposed outside the container (typically passwords).
volumes (list[<conn_id, account_name, share_name, mount_path, read_only>]) -- list of
Volumetuples to be mounted to the container. Currently only Azure Fileshares are supported.memory_in_gb (double) -- the amount of memory to allocate to this container
cpu (double) -- the number of cpus to allocate to this container
gpu (azure.mgmt.containerinstance.models.GpuResource) -- GPU Resource for the container.
command (Optional[List[str]]) -- the command to run inside the container
container_timeout (datetime.timedelta) -- max time allowed for the execution of the container instance.
tags (Optional[dict[str, str]]) -- azure tags as dict of str:str
os_type (str) -- The operating system type required by the containers in the container group. Possible values include: 'Windows', 'Linux'
restart_policy (str) -- Restart policy for all containers within the container group. Possible values include: 'Always', 'OnFailure', 'Never'
ip_address (IpAddress) -- The IP address type of the container group.
Example:
AzureContainerInstancesOperator( ci_conn_id = "azure_service_principal", registry_conn_id = "azure_registry_user", resource_group = "my-resource-group", name = "my-container-name-{{ ds }}", image = "myprivateregistry.azurecr.io/my_container:latest", region = "westeurope", environment_variables = {"MODEL_PATH": "my_value", "POSTGRES_LOGIN": "{{ macros.connection('postgres_default').login }}", "POSTGRES_PASSWORD": "{{ macros.connection('postgres_default').password }}", "JOB_GUID": "{{ ti.xcom_pull(task_ids='task1', key='guid') }}" }, secured_variables = ['POSTGRES_PASSWORD'], volumes = [("azure_container_instance_conn_id", "my_storage_container", "my_fileshare", "/input-data", True),], memory_in_gb=14.0, cpu=4.0, gpu=GpuResource(count=1, sku='K80'), command=["/bin/echo", "world"], task_id="start_container" )