Docker Image for Apache Airflow¶

For the ease of deployment in production, the community releases a production-ready reference container image.

The Apache Airflow community, releases Docker Images which are reference images for Apache Airflow. Every time a new version of Airflow is released, the images are prepared in the apache/airflow DockerHub for all the supported Python versions.

You can find the following images there (Assuming Airflow version 2.9.0):

apache/airflow:latest - the latest released Airflow image with default Python version (3.8 currently)
apache/airflow:latest-pythonX.Y - the latest released Airflow image with specific Python version
apache/airflow:2.9.0 - the versioned Airflow image with default Python version (3.8 currently)
apache/airflow:2.9.0-pythonX.Y - the versioned Airflow image with specific Python version

Those are “reference” regular images. They contain the most common set of extras, dependencies and providers that are often used by the users and they are good to “try-things-out” when you want to just take Airflow for a spin,

You can also use “slim” images that contain only core airflow and are about half the size of the “regular” images but you need to add all the Reference for package extras and providers that you need separately via Building the image.

apache/airflow:slim-latest - the latest released Airflow image with default Python version (3.8 currently)
apache/airflow:slim-latest-pythonX.Y - the latest released Airflow image with specific Python version
apache/airflow:slim-2.9.0 - the versioned Airflow image with default Python version (3.8 currently)
apache/airflow:slim-2.9.0-pythonX.Y - the versioned Airflow image with specific Python version

The Apache Airflow image provided as convenience package is optimized for size, and it provides just a bare minimal set of the extras and dependencies installed and in most cases you want to either extend or customize the image. You can see all possible extras in Reference for package extras. The set of extras used in Airflow Production image are available in the Dockerfile.

However, Airflow has more than 60 community-managed providers (installable via extras) and some of the default extras/providers installed are not used by everyone, sometimes others extras/providers are needed, sometimes (very often actually) you need to add your own custom dependencies, packages or even custom providers. You can learn how to do it in Building the image.

The production images are build in DockerHub from released version and release candidates. There are also images published from branches but they are used mainly for development and testing purpose. See Airflow Git Branching for details.

Fixing images at release time¶

The released “versioned” reference images are mostly fixed when we release Airflow version and we only update them in exceptional circumstances. For example when we find out that there are dependency errors that might prevent important Airflow or embedded provider’s functionalities working. In normal circumstances, the images are not going to change after release, even if new version of Airflow dependencies are released - not even when those versions contain critical security fixes. The process of Airflow releases is designed around upgrading dependencies automatically where applicable but only when we release a new version of Airflow, not for already released versions.

What should I do if my security scan shows critical and high vulnerabilities in the image?¶

We often hear questions that our users use various security scanners on the image and find out that there are some critical and high vulnerabilities in the image - not coming from Airflow but for some other components. In general, this is normal and expected that such vulnerabilities are found in the image after it’s been released and fixed - precisely because we are NOT updating the images after they are released as explained above. Also sometimes even the latest releases contain vulnerabilities that are not yet fixed in the base image we use or in the dependencies we use and cannot upgrade, because some of our providers have limits and did not manage to upgrade yet and we have no control over that. So it is possible that even the most recent release of our image there are some High and Critical vulnerabilities that are not yet fixed.

What can you do in such case?

First of all - you should know what you should NOT do.

Do NOT send private email to the Airflow Security Team with scan results and asking what to do. The Security team at Airflow takes care exclusively about undisclosed vulnerabilities in Airflow itself, not in the dependencies or in the base image. The security email should only be used to report privately any security issues that can be exploited via Airflow. This is nicely explained in our Security Policy where you can find more details including the need to provide reproducible scenarios and submitting ONE issue per email. NEVER submit multiple vulnerabilities in one email - those are rejected immediately, as they make the process of handling the issue way harder for everyone, including the reporters.

Also DO NOT open a GitHub Issue with the scan results and asking what to do. The GitHub Issues are for reporting bugs and feature requests to Airflow itself, not for asking for help with the security scans on 3rd party components.

So what are your options?

You have four options:

Build your own custom image following the examples we share there - using the latest base image and possibly manually bumping dependencies you want to bump. There are quite a few examples in Building the image which you can follow. You can use “slim” image as a base for your images and rather than basing your image on the “reference” image that has a number of extras and providers installed, you can only install what you actually need and upgrade some dependencies that otherwise would not be possible to upgrade - because some of the provider libraries have limits and did not manage to upgrade yet and we have no control over that. This is the most flexible way to build your image and you can build your process to combine it with quickly upgrading to latest Airflow versions (see point 2. below).
Wait for a new version of Airflow and upgrade to it. Airflow images are updated to latest “non-conflicting” dependencies and use latest “base” image at release time, so what you have in the reference images at the moment we publish the image / release the version is what is “latest and greatest” available at the moment with the base platform we use (Debian Bookworm is the reference image we use). This is one of good strategies you can take - build a process to upgrade your Airflow version regularly - quickly after it has been released by the community, this will help you to keep up with the latest security fixes in the dependencies.
If the base platform we use (currently Debian Bookworm) does not contain the latest versions you want and you want to use other base images, you can take a look at what system dependencies are installed and scripts in the latest Dockerfile of airflow and take inspiration from it and build your own image or copy it and modify it to your needs. See the Dockerfile for the latest version.
Research if the vulnerability affects you or not. Even if there is a dependency with high or critical vulnerability, it does not mean that it can be exploited in Airflow (or specifically in the way you are using Airflow). If you do have a reproducible scenario how a vulnerability can be exploited in Airflow, you should - of course - privately report it to the security team. But if you do not have reproducible scenario, please do some research and try to understand the impact of the vulnerability on Airflow. That research might result in a public GitHub Discussion where you can discuss the impact of the vulnerability if you research will indicate Airflow might not be impacted or private security email if you find a reproducible scenario on how to exploit it.

How do I discuss publicly about public CVEs in the image?

The security scans report public vulnerabilities in 3rd-party components of Airflow. Since those are already public vulnerabilities, this is something you can talk about but others also are talking about. So you can do research on your own first. Try to find discussions about the issues, how others were handling it and possibly even try to explore, whether the vulnerability can be exploited in Airflow or not. This is a very valuable contribution to the community you can do in order to help others to understand the impact of the vulnerability on Airflow. We highly appreciate our commercial users do it, because Airflow is maintained by volunteers, so if you or your company can spend some time and skills of security researchers to help the community to understand the impact of the vulnerability on Airflow, it could be a fantastic contribution to the community and way to give back to the project that your company uses for free.

You are free to discuss it publicly, open a GitHub Discussion mentioning your findings and research you’ve done so far. Ideally (as a way to contribute to Airflow) you should explain the findings of your own security team in your company to help to research and understand the impact of the vulnerability on Airflow (and your way of using it). Again - strong suggestion is to open ONE discussion per vulnerability. You should NOT post scan results in bulk - this is not helpful for a discussion, and you will not get meaningful answers if you will attempt to discuss all the issues in one discussion thread.

Yes - we know it’s the easy way to copy & paste your result and ask others what to do, but doing it is going to likely result in silence because such actions in the community as seen as pretty selfish way of getting your problems solved by tapping into time of other volunteers, without spending your time on making it easier for them to help. If you really want to get help from the community, focus your discussion on particular CVE, provide your findings - including analyzing your report in detail and finding which binaries and base images exactly are causing the scanner to report the vulnerability. Remember that only you have access to your scanner and you should bring as much helpful information so that others can comment on it. Show that you have done your homework and that you bring valuable information to the community.

Opening a GitHub Discussion for this kind of issues is also a great way to communicate with the maintainers and security team in an open and transparent way - without reverting to the private security mailing list (which serves different purpose as explained above). If after such a discussion there will be a way to remove such a vulnerability from the scanned image - great, you can even contribute a PR to the Dockerfile to remove the vulnerability from the image. Maybe such a discussion will lead to a PR to allow Airflow to upgrade to newer dependency that fixes the vulnerability or remove it altogether, or maybe there is already a way to mitigate it or maybe there is already a PR that someone works on to fix it. All this can (and should) be discussed publicly and transparently in a GitHub Discussion, not via private security email, nor GitHub Issues which are exclusively about Airflow Issues not 3rd-party components public security issues.

Support¶

The reference Docker Image supports the following platforms and database:

Intel platform (x86_64)¶

Postgres Client
MySQL Client
MSSQL Client

ARM platform (aarch64)¶

ARM support is experimental, might change any time.

Postgres Client
MySQL Client (MySQL 8)
MSSQL Client

Note that MySQL on arm has experimental support through MariaDB client library.

Usage¶

The AIRFLOW_HOME is set by default to /opt/airflow/ - this means that DAGs are by default in the /opt/airflow/dags folder and logs are in the /opt/airflow/logs

The working directory is /opt/airflow by default.

If no AIRFLOW__DATABASE__SQL_ALCHEMY_CONN variable is set then SQLite database is created in ${AIRFLOW_HOME}/airflow.db.

For example commands that start Airflow see: Executing commands.

Airflow requires many components to function as it is a distributed application. You may therefore also be interested in launching Airflow in the Docker Compose environment, see: Running Airflow in Docker.

You can use this image in Helm Chart as well.