priority_weight defines priorities in the executor queue. The default
1, and can be
bumped to any integer. Moreover, each task has a true
priority_weight that is calculated based on its
weight_rule which defines weighting method used for the effective total priority weight of the task.
By default, Airflow’s weighting method is
downstream. You can find other weighting methods in
There are three weighting methods.
The effective weight of the task is the aggregate sum of all downstream descendants. As a result, upstream tasks will have higher weight and will be scheduled more aggressively when using positive weight values. This is useful when you have multiple DAG run instances and desire to have all upstream tasks to complete for all runs before each DAG can continue processing downstream tasks.
The effective weight is the aggregate sum of all upstream ancestors. This is the opposite where downstream tasks have higher weight and will be scheduled more aggressively when using positive weight values. This is useful when you have multiple DAG run instances and prefer to have each DAG complete before starting upstream tasks of other DAG runs.
The effective weight is the exact
priority_weightspecified without additional weighting. You may want to do this when you know exactly what priority weight each task should have. Additionally, when set to
absolute, there is bonus effect of significantly speeding up the task creation process as for very large DAGs
priority_weight parameter can be used in conjunction with Pools.