Secret backends¶
This is a summary of all Apache Airflow Community provided implementations of secret backends exposed via community-managed providers.
Airflow has the capability of reading connections, variables and configuration from Secret Backends rather than from its own Database. While storing such information in Airflow’s database is possible, many of the enterprise customers already have some secret managers storing secrets, and Airflow can tap into those via providers that implement secrets backends for services Airflow integrates with.
Note
Secret Backend integration do not allow writes to the secret backend.
This is a design choice as normally secret stores require elevated permissions to write as it is a protected resource.
That means Variable.set(...) will write to the Airflow metastore even if you use secret backend.
If you need to update a value of a secret stored in the secret backend you must do it explicitly. That can be done
by using operator that writes to the secret backend of your choice.
Warning
If you have key foo in secret backend and you will do Variable.set(key='foo',...) it will create
Airflow Variable with key foo in the Airflow metastore. It means you will have 2 secrets with key foo.
While this is possible, Airflow detects that this situation is likely wrong and output to the task log a warning that
explains while the write request is honored it will be ignored with the next read. The reason for this is when
executing Variable.get('foo'), it will read the value from the secret backend. The value stored in Airflow
metastore will be ignored due to priority given to the secret backend.
You can also take a look at Secret backends available in the core Airflow in Secrets Backend and here you can see the ones provided by the community-managed providers: