airflow.providers.cncf.kubernetes.pod_generator_deprecated

Backwards compatibility for Pod generation.

This module provides an interface between the previous Pod API and outputs a kubernetes.client.models.V1Pod. The advantage being that the full Kubernetes API is supported and no serialization need be written.

Module Contents

Classes

PodDefaults

Static defaults for Pods.

PodGenerator

Contains Kubernetes Airflow Worker configuration logic.

Functions

make_safe_label_value(string)

Normalize a provided label to be of valid length and characters.

Attributes

MAX_POD_ID_LEN

MAX_LABEL_LEN

airflow.providers.cncf.kubernetes.pod_generator_deprecated.MAX_POD_ID_LEN = 253[source]
airflow.providers.cncf.kubernetes.pod_generator_deprecated.MAX_LABEL_LEN = 63[source]
class airflow.providers.cncf.kubernetes.pod_generator_deprecated.PodDefaults[source]

Static defaults for Pods.

XCOM_MOUNT_PATH = '/airflow/xcom'[source]
SIDECAR_CONTAINER_NAME = 'airflow-xcom-sidecar'[source]
XCOM_CMD = 'trap "exit 0" INT; while true; do sleep 30; done;'[source]
VOLUME_MOUNT[source]
VOLUME[source]
SIDECAR_CONTAINER[source]
airflow.providers.cncf.kubernetes.pod_generator_deprecated.make_safe_label_value(string)[source]

Normalize a provided label to be of valid length and characters.

Valid label values must be 63 characters or less and must be empty or begin and end with an alphanumeric character ([a-z0-9A-Z]) with dashes (-), underscores (_), dots (.), and alphanumerics between.

If the label value is greater than 63 chars once made safe, or differs in any way from the original value sent to this function, then we need to truncate to 53 chars, and append it with a unique hash.

class airflow.providers.cncf.kubernetes.pod_generator_deprecated.PodGenerator(image=None, name=None, namespace=None, volume_mounts=None, envs=None, cmds=None, args=None, labels=None, node_selectors=None, ports=None, volumes=None, image_pull_policy=None, restart_policy=None, image_pull_secrets=None, init_containers=None, service_account_name=None, resources=None, annotations=None, affinity=None, hostnetwork=False, tolerations=None, security_context=None, configmaps=None, dnspolicy=None, schedulername=None, extract_xcom=False, priority_class_name=None)[source]

Contains Kubernetes Airflow Worker configuration logic.

Represents a kubernetes pod and manages execution of a single pod. Any configuration that is container specific gets applied to the first container in the list of containers.

Parameters

  • image (str | None) – The docker image

  • name (str | None) – name in the metadata section (not the container name)

  • namespace (str | None) – pod namespace

  • volume_mounts (list[kubernetes.client.models.V1VolumeMount | dict] | None) – list of kubernetes volumes mounts

  • envs (dict[str, str] | None) – A dict containing the environment variables

  • cmds (list[str] | None) – The command to be run on the first container

  • args (list[str] | None) – The arguments to be run on the pod

  • labels (dict[str, str] | None) – labels for the pod metadata

  • node_selectors (dict[str, str] | None) – node selectors for the pod

  • ports (list[kubernetes.client.models.V1ContainerPort | dict] | None) – list of ports. Applies to the first container.

  • volumes (list[kubernetes.client.models.V1Volume | dict] | None) – Volumes to be attached to the first container

  • image_pull_policy (str | None) – Specify a policy to cache or always pull an image

  • restart_policy (str | None) – The restart policy of the pod

  • image_pull_secrets (str | None) – Any image pull secrets to be given to the pod. If more than one secret is required, provide a comma separated list: secret_a,secret_b

  • init_containers (list[kubernetes.client.models.V1Container] | None) – A list of init containers

  • service_account_name (str | None) – Identity for processes that run in a Pod

  • resources (kubernetes.client.models.V1ResourceRequirements | dict | None) – Resource requirements for the first containers

  • annotations (dict[str, str] | None) – annotations for the pod

  • affinity (dict | None) – A dict containing a group of affinity scheduling rules

  • hostnetwork (bool) – If True enable host networking on the pod

  • tolerations (list | None) – A list of kubernetes tolerations

  • security_context (kubernetes.client.models.V1PodSecurityContext | dict | None) – A dict containing the security context for the pod

  • configmaps (list[str] | None) – Any configmap refs to read configmaps for environments from. If more than one configmap is required, provide a comma separated list configmap_a,configmap_b

  • dnspolicy (str | None) – Specify a dnspolicy for the pod

  • schedulername (str | None) – Specify a schedulername for the pod

  • pod – The fully specified pod. Mutually exclusive with path_or_string

  • extract_xcom (bool) – Whether to bring up a container for xcom

  • priority_class_name (str | None) – priority class name for the launched Pod

gen_pod()[source]

Generate pod.

static add_sidecar(pod)[source]

Add sidecar.

static from_obj(obj)[source]

Convert to pod from obj.

static make_unique_pod_id(dag_id)[source]

Generate a unique Pod name.

Kubernetes pod names must be <= 253 chars and must pass the following regex for validation ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$

Parameters

dag_id – a dag_id with only alphanumeric characters

Returns

str valid Pod name of appropriate length

Was this entry helpful?