• Home
• Core Concepts

Core Concepts

Here you can find detailed documentation about each one of the core concepts of Apache Airflow™ and how to use them, as well as a high-level architectural overview.

Architecture

• Architecture Overview
  • Airflow components
  • Deploying Airflow components
  • Architecture Diagrams
  • Workloads
  • Control Flow
  • User interface

Workloads

• DAGs
  • Declaring a DAG
  • Loading DAGs
  • Running DAGs
  • DAG Assignment
  • Default Arguments
  • The DAG decorator
  • Control Flow
  • Setup and teardown
  • Dynamic DAGs
  • DAG Visualization
  • DAG & Task Documentation
  • SubDAGs
  • TaskGroups vs SubDAGs
  • Packaging DAGs
  • .airflowignore
  • DAG Dependencies
  • DAG pausing, deactivation and deletion
  • DAG Auto-pausing (Experimental)
• DAG Runs
  • DAG Run Status
  • Re-run DAG
  • External Triggers
  • To Keep in Mind
• Tasks
  • Relationships
  • Task Instances
  • Timeouts
  • SLAs
  • Special Exceptions
  • Zombie/Undead Tasks
  • Executor Configuration
• Operators
  • Jinja Templating
  • Reserved params keyword
• Sensors
• TaskFlow
  • Context
  • Logging
  • Passing Arbitrary Objects As Arguments
  • Sensors and the TaskFlow API
  • History
• Executor
  • Executor Types
  • Writing Your Own Executor
• Auth manager
  • Why pluggable auth managers?
  • Writing your own auth manager
• Object Storage
  • Cloud Object Stores are not real file systems
  • Basic Use
  • Configuration
  • Path API
  • Extensions
  • Copying and Moving
  • External Integrations

Communication

• XComs
  • Object Storage XCom Backend
  • Custom XCom Backends
  • Working with Custom XCom Backends in Containers
  • Working with Custom Backends in K8s via Helm
• Variables
• Params
  • DAG-level Params
  • Task-level Params
  • Referencing Params in a Task
  • JSON Schema Validation
  • Use Params to Provide a Trigger UI Form
  • Disabling Runtime Param Modification

Debugging

• Debugging Airflow DAGs

Previous Next

Was this entry helpful?