• Home
• airflow.providers.cncf.kubernetes
• airflow.providers.cncf.kubernetes.utils
• airflow.providers.cncf.kubernetes.utils.pod_manager

airflow.providers.cncf.kubernetes.utils.pod_manager

Launches PODs

Module Contents

Classes

PodPhase	Possible pod phases
PodOperatorHookProtocol	Protocol to define methods relied upon by KubernetesPodOperator
PodLoggingStatus	Used for returning the status of the pod and last log time when exiting from fetch_container_logs
PodManager	Helper class for creating, monitoring, and otherwise interacting with Kubernetes pods

Functions

should_retry_start_pod(exception)	Check if an Exception indicates a transient error and warrants retrying
get_container_status(pod, container_name)	Retrieves container status
container_is_running(pod, container_name)	Examines V1Pod pod to determine whether container_name is running.
container_is_terminated(pod, container_name)	Examines V1Pod pod to determine whether container_name is terminated.
get_container_termination_message(pod, container_name)

exception airflow.providers.cncf.kubernetes.utils.pod_manager.PodLaunchFailedException

Bases: airflow.exceptions.AirflowException

When pod launching fails in KubernetesPodOperator.

airflow.providers.cncf.kubernetes.utils.pod_manager.should_retry_start_pod(exception)

Check if an Exception indicates a transient error and warrants retrying

class airflow.providers.cncf.kubernetes.utils.pod_manager.PodPhase

Possible pod phases See https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-phase.

PENDING = 'Pending'

RUNNING = 'Running'

FAILED = 'Failed'

SUCCEEDED = 'Succeeded'

terminal_states

class airflow.providers.cncf.kubernetes.utils.pod_manager.PodOperatorHookProtocol

Bases: airflow.typing_compat.Protocol

Protocol to define methods relied upon by KubernetesPodOperator

Subclasses of KubernetesPodOperator, such as GKEStartPodOperator, may use hooks that don’t extend KubernetesHook. We use this protocol to document the methods used by KPO and ensure that these methods exist on such other hooks.

property core_v1_client: kubernetes.client.CoreV1Api

Get authenticated CoreV1Api object.

property is_in_cluster: bool

Expose whether the hook is configured with load_incluster_config or not

get_pod(name, namespace)

Read pod object from kubernetes API.

get_namespace()

Returns the namespace that defined in the connection

airflow.providers.cncf.kubernetes.utils.pod_manager.get_container_status(pod, container_name)

Retrieves container status

airflow.providers.cncf.kubernetes.utils.pod_manager.container_is_running(pod, container_name)

Examines V1Pod pod to determine whether container_name is running. If that container is present and running, returns True. Returns False otherwise.

airflow.providers.cncf.kubernetes.utils.pod_manager.container_is_terminated(pod, container_name)

Examines V1Pod pod to determine whether container_name is terminated. If that container is present and terminated, returns True. Returns False otherwise.

airflow.providers.cncf.kubernetes.utils.pod_manager.get_container_termination_message(pod, container_name)

class airflow.providers.cncf.kubernetes.utils.pod_manager.PodLoggingStatus

Used for returning the status of the pod and last log time when exiting from fetch_container_logs

running: bool

last_log_time: DateTime | None

class airflow.providers.cncf.kubernetes.utils.pod_manager.PodManager(kube_client)

Bases: airflow.utils.log.logging_mixin.LoggingMixin

Helper class for creating, monitoring, and otherwise interacting with Kubernetes pods for use with the KubernetesPodOperator

run_pod_async(pod, **kwargs)

Runs POD asynchronously

delete_pod(pod)

Deletes POD

create_pod(pod)

Launches the pod asynchronously.

await_pod_start(pod, startup_timeout=120)

Waits for the pod to reach phase other than Pending

Parameters

• pod (kubernetes.client.models.v1_pod.V1Pod) –

• startup_timeout (int) – Timeout (in seconds) for startup of the pod (if pod is pending for too long, fails task)

Returns

Return type

None

follow_container_logs(pod, container_name)

fetch_container_logs(pod, container_name, *, follow=False, since_time=None, post_termination_timeout=120)

Follows the logs of container and streams to airflow logging. Returns when container exits.

await_container_completion(pod, container_name)

Waits for the given container in the given pod to be completed

Parameters

• pod (kubernetes.client.models.v1_pod.V1Pod) – pod spec that will be monitored

• container_name (str) – name of the container within the pod to monitor

await_pod_completion(pod)

Monitors a pod and returns the final state

Parameters

pod (kubernetes.client.models.v1_pod.V1Pod) – pod spec that will be monitored

Returns

tuple[State, str | None]

Return type

kubernetes.client.models.v1_pod.V1Pod

parse_log_line(line)

Parse K8s log line and returns the final state

Parameters

line (str) – k8s log line

Returns

timestamp and log message

Return type

tuple[DateTime | None, str]

container_is_running(pod, container_name)

Reads pod and checks if container is running

container_is_terminated(pod, container_name)

Reads pod and checks if container is terminated

read_pod_logs(pod, container_name, tail_lines=None, timestamps=False, since_seconds=None, follow=True, post_termination_timeout=120)

Reads log from the POD

read_pod_events(pod)

Reads events from the POD

read_pod(pod)

Read POD information

await_xcom_sidecar_container_start(pod)

extract_xcom(pod)

Retrieves XCom value and kills xcom sidecar container

Previous Next

Was this entry helpful?