`airflow.models.dagrun`¶

Module Contents¶

class airflow.models.dagrun.TISchedulingDecision[source]¶

Bases: typing.NamedTuple

Type of return for DagRun.task_instance_scheduling_decisions

tis :List[TI][source]¶

schedulable_tis :List[TI][source]¶

changed_tis :bool[source]¶

unfinished_tasks :List[TI][source]¶

finished_tasks :List[TI][source]¶

class airflow.models.dagrun.DagRun(dag_id: Optional[str] = None, run_id: Optional[str] = None, queued_at: Optional[datetime] = __NO_VALUE, execution_date: Optional[datetime] = None, start_date: Optional[datetime] = None, external_trigger: Optional[bool] = None, conf: Optional[Any] = None, state: Optional[DagRunState] = None, run_type: Optional[str] = None, dag_hash: Optional[str] = None, creating_job_id: Optional[int] = None)[source]¶

Bases: airflow.models.base.Base, airflow.utils.log.logging_mixin.LoggingMixin

DagRun describes an instance of a Dag. It can be created by the scheduler (for regular runs) or by an external trigger

__tablename__ = dag_run[source]¶

__NO_VALUE[source]¶

id[source]¶

dag_id[source]¶

queued_at[source]¶

execution_date[source]¶

start_date[source]¶

end_date[source]¶

_state[source]¶

run_id[source]¶

creating_job_id[source]¶

external_trigger[source]¶

run_type[source]¶

conf[source]¶

last_scheduling_decision[source]¶

dag_hash[source]¶

dag[source]¶

__table_args__[source]¶

task_instances[source]¶

DEFAULT_DAGRUNS_TO_EXAMINE[source]¶

state[source]¶

is_backfill[source]¶

__repr__(self)[source]¶

get_state(self)[source]¶

set_state(self, state: DagRunState)[source]¶

refresh_from_db(self, session: Session = None)[source]¶

Reloads the current dagrun from the database

Parameters: session (Session) -- database session

classmethod next_dagruns_to_examine(cls, state: DagRunState, session: Session, max_number: Optional[int] = None)[source]¶

Return the next DagRuns that the scheduler should attempt to schedule.

This will return zero or more DagRun rows that are row-level-locked with a "SELECT ... FOR UPDATE" query, you should ensure that any scheduling decisions are made in a single transaction -- as soon as the transaction is committed it will be unlocked.

Return type: list[airflow.models.DagRun]

static find(dag_id: Optional[Union[str, List[str]]] = None, run_id: Optional[str] = None, execution_date: Optional[datetime] = None, state: Optional[DagRunState] = None, external_trigger: Optional[bool] = None, no_backfills: bool = False, run_type: Optional[DagRunType] = None, session: Session = None, execution_start_date: Optional[datetime] = None, execution_end_date: Optional[datetime] = None)[source]¶

Returns a set of dag runs for the given search criteria.

Parameters

dag_id (str or list[str]) -- the dag_id or list of dag_id to find dag runs for
run_id (str) -- defines the run id for this dag run
run_type (airflow.utils.types.DagRunType) -- type of DagRun
execution_date (datetime.datetime or list[datetime.datetime]) -- the execution date
state (DagRunState) -- the state of the dag run
external_trigger (bool) -- whether this dag run is externally triggered
no_backfills (bool) -- return no backfills (True), return all (False). Defaults to False
session (sqlalchemy.orm.session.Session) -- database session
execution_start_date (datetime.datetime) -- dag run that was executed from this date
execution_end_date (datetime.datetime) -- dag run that was executed until this date

static generate_run_id(run_type: DagRunType, execution_date: datetime)[source]¶: Generate Run ID based on Run Type and Execution Date

get_task_instances(self, state: Optional[Iterable[TaskInstanceState]] = None, session=None)[source]¶: Returns the task instances for this dag run

get_task_instance(self, task_id: str, session: Session = None)[source]¶

Returns the task instance specified by task_id for this dag run

Parameters

task_id (str) -- the task id
session (Session) -- Sqlalchemy ORM Session

get_dag(self)[source]¶

Returns the Dag associated with this DagRun.

Returns: DAG

get_previous_dagrun(self, state: Optional[DagRunState] = None, session: Session = None)[source]¶: The previous DagRun, if there is one

get_previous_scheduled_dagrun(self, session: Session = None)[source]¶: The previous, SCHEDULED DagRun, if there is one

update_state(self, session: Session = None, execute_callbacks: bool = True)[source]¶

Determines the overall state of the DagRun based on the state of its TaskInstances.

Parameters

session (Session) -- Sqlalchemy ORM Session
execute_callbacks (bool) -- Should dag callbacks (success/failure, SLA etc) be invoked directly (default: true) or recorded as a pending request in the callback property

Returns

Tuple containing tis that can be scheduled in the current loop & callback that needs to be executed

task_instance_scheduling_decisions(self, session: Session = None)[source]¶

_get_ready_tis(self, scheduleable_tasks: List[TI], finished_tasks: List[TI], session: Session)[source]¶

_are_premature_tis(self, unfinished_tasks: List[TI], finished_tasks: List[TI], session: Session)[source]¶

_emit_true_scheduling_delay_stats_for_finished_state(self, finished_tis)[source]¶: This is a helper method to emit the true scheduling delay stats, which is defined as the time when the first task in DAG starts minus the expected DAG run datetime. This method will be used in the update_state method when the state of the DagRun is updated to a completed status (either success or failure). The method will find the first started task within the DAG and calculate the expected DagRun start time (based on dag.execution_date & dag.schedule_interval), and minus these two values to get the delay. The emitted data may contains outlier (e.g. when the first task was cleared, so the second task's start_date will be used), but we can get rid of the outliers on the stats side through the dashboards tooling built. Note, the stat will only be emitted if the DagRun is a scheduler triggered one (i.e. external_trigger is False).

_emit_duration_stats_for_finished_state(self)[source]¶

verify_integrity(self, session: Session = None)[source]¶

Verifies the DagRun by checking for removed tasks or tasks that are not in the database yet. It will set state to removed or add the task if required.

Parameters: session (Session) -- Sqlalchemy ORM Session

static get_run(session: Session, dag_id: str, execution_date: datetime)[source]¶

Get a single DAG Run

Parameters

session (Session) -- Sqlalchemy ORM Session
dag_id (unicode) -- DAG ID
execution_date (datetime) -- execution date

Returns

DagRun corresponding to the given dag_id and execution date if one exists. None otherwise.

Return type

airflow.models.DagRun

classmethod get_latest_runs(cls, session=None)[source]¶: Returns the latest DagRun for each DAG

schedule_tis(self, schedulable_tis: Iterable[TI], session: Session = None)[source]¶

Set the given task instances in to the scheduled state.

Each element of schedulable_tis should have it's task attribute already set.

Any DummyOperator without callbacks is instead set straight to the success state.

All the TIs should belong to this DagRun, but this code is in the hot-path, this is not checked -- it is the caller's responsibility to call this function only with TIs from a single dag run.

airflow.models.dagrun¶

Module Contents¶

`airflow.models.dagrun`¶