Skip to content

Common airflow.cfg files across all containers in default docker-compose.yaml #49681

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 25, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 3 additions & 6 deletions airflow-core/docs/howto/docker-compose/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -71,8 +71,7 @@ x-airflow-common:
# for other purpose (development, test and especially production usage) build/extend Airflow image.
_PIP_ADDITIONAL_REQUIREMENTS: ${_PIP_ADDITIONAL_REQUIREMENTS:-}
# The following line can be used to set a custom config file, stored in the local config folder
# If you want to use it, outcomment it and replace airflow.cfg with the name of your config file
# AIRFLOW_CONFIG: '/opt/airflow/config/airflow.cfg'
AIRFLOW_CONFIG: '/opt/airflow/config/airflow.cfg'
volumes:
- ${AIRFLOW_PROJ_DIR:-.}/dags:/opt/airflow/dags
- ${AIRFLOW_PROJ_DIR:-.}/logs:/opt/airflow/logs
Expand Down Expand Up @@ -252,8 +251,8 @@ services:
echo " https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html#before-you-begin"
echo
fi
mkdir -p /sources/logs /sources/dags /sources/plugins
chown -R "${AIRFLOW_UID}:0" /sources/{logs,dags,plugins}
mkdir -p /opts/airflow/{logs,dags,plugins,config}
chown -R "${AIRFLOW_UID}:0" /opts/airflow/{logs,dags,plugins,config}
exec /entrypoint airflow version
# yamllint enable rule:line-length
environment:
Expand All @@ -264,8 +263,6 @@ services:
_AIRFLOW_WWW_USER_PASSWORD: ${_AIRFLOW_WWW_USER_PASSWORD:-airflow}
_PIP_ADDITIONAL_REQUIREMENTS: ''
user: "0:0"
volumes:
- ${AIRFLOW_PROJ_DIR:-.}:/sources

airflow-cli:
<<: *airflow-common
Expand Down
22 changes: 15 additions & 7 deletions airflow-core/docs/howto/docker-compose/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -125,8 +125,8 @@ Setting the right Airflow user
------------------------------

On **Linux**, the quick-start needs to know your host user id and needs to have group id set to ``0``.
Otherwise the files created in ``dags``, ``logs`` and ``plugins`` will be created with ``root`` user ownership.
You have to make sure to configure them for the docker-compose:
Otherwise the files created in ``dags``, ``logs``, ``config`` and ``plugins`` will be created with
``root`` user ownership. You have to make sure to configure them for the docker-compose:

.. code-block:: bash

Expand All @@ -143,6 +143,17 @@ safely ignore it. You can also manually create an ``.env`` file in the same fold

AIRFLOW_UID=50000

Initialize airflow.cfg (Optional)
---------------------------------

If you want to initialize ``airflow.cfg`` with default values before launching the airflow service, run.

.. code-block:: bash

docker compose run airflow-cli airflow config list

This will seed ``airflow.cfg`` with default values in ``config`` folder.

Initialize the database
-----------------------

Expand Down Expand Up @@ -346,12 +357,9 @@ Special case - Adding a custom config file

If you have a custom config file and wish to use it in your Airflow instance, you need to perform the following steps:

1) Remove comment from the ``AIRFLOW_CONFIG: '/opt/airflow/config/airflow.cfg'`` line
in the ``docker-compose.yaml`` file.

2) Place your custom ``airflow.cfg`` file in the local config folder.
1) Replace the auto-generated ``airflow.cfg`` file in the local config folder with your custom config file.

3) If your config file has a different name than ``airflow.cfg``, adjust the filename in
2) If your config file has a different name than ``airflow.cfg``, adjust the filename in
``AIRFLOW_CONFIG: '/opt/airflow/config/airflow.cfg'``

Networking
Expand Down