airflow.providers.daskexecutor.executors.dask_executor

DaskExecutor.

See also

For more information on how the DaskExecutor works, take a look at the guide: Dask Executor

Module Contents

Classes

DaskExecutor

DaskExecutor submits tasks to a Dask Distributed cluster.
class airflow.providers.daskexecutor.executors.dask_executor.DaskExecutor(cluster_address=None)[source]

Bases: airflow.executors.base_executor.BaseExecutor

DaskExecutor submits tasks to a Dask Distributed cluster.

supports_pickling: bool = False[source]
start()[source]

Executors may need to get things started.

execute_async(key, command, queue=None, executor_config=None)[source]

Execute the command asynchronously.

Parameters

  • key (airflow.models.taskinstancekey.TaskInstanceKey) – Unique key for the task instance

  • command (airflow.executors.base_executor.CommandType) – Command to run

  • queue (str | None) – name of the queue

  • executor_config (Any | None) – Configuration passed to the executor.

sync()[source]

Sync will get called periodically by the heartbeat method.

Executors should override this to perform gather statuses.

end()[source]

Wait synchronously for the previously submitted job to complete.

terminate()[source]

Get called when the daemon receives a SIGTERM.

Was this entry helpful?