Airflow Logging Configs are ignored / not used in Airflow 3.x

[r-richmond]

Apache Airflow version

3.0.3

What happened?

Despite all of the documentation on how to configure logging in Airflow 3, none of these configs are used and instead Airflow uses a hard-coded setting.

Specifically

rdjouder 3 weeks ago
Hello,
Looking at the code of airflow.sdk.log.configure_logging I see that conf.get("logging","logging_config_class", ... ) is not called.
Also logging.config.dictConfig is called with hard coded defaults thus completly erasing what configure_logging() from logging_config.py is doing.
I have a custom logging config that is working fine as of airflow 2.10.X and older.
Is my analysis wrong ? Is it a bug ?

ashb 3 weeks ago
It's intentional I'm afraid, and I am in the process of completely overhauling all of Airflow's logging to use structlog and to be able to produce JSON logs in this PR #52651

ashb 3 weeks ago
The logger is configured but Airflow won't use it anymore. The reason is that logs from the task process is sent as JSON to the supervisor process and written to file, and the UI then takes that JSON and renders it for display.

What you think should happen instead?

Airflow log configs should be used.

Anything else?

If I understand correctly, it will be resolved with #52651.

Creating this issue just for tracking 3.x blockers. Please add affected_version_3.0 label.

Code of Conduct

• I agree to follow this project's Code of Conduct

[ashb]

@r-richmond Are you most interested in the logging format for the daemons/airflow components, or for the display of task logs in the UI?

[r-richmond]

@r-richmond Are you most interested in the logging format for the daemons/airflow components, or for the display of task logs in the UI?

@ashb Honestly I'm interested in several things

1. Adding line number & file name to the format before msg
2. Moving source to the start of line/before msg instead of at the end of the line/after msg
3. Eventually further debugging to see if I can figure out why running DBT via bashOperator logs so much differently in v3 than it does in v2. (in v2 it logs once per sql statement whereas in v3 it logs once per line which floods the logs with a header for every line; I suspect this is due to some underlying changes and may not be avoidable but the user complaints I've been subject to dictate I at least try and look).
4. As for the display of task logs the big feedback item from myself and from my users is that select/copy is much harder in v3 than v2. Also scrolling is much worse as it gets very slow after several hundred lines & page up/down don't work properly anymore. (This is almost certainly outside of what can be controlled with logging configs but is something I've very interested in sharing.

Those are the big items IMO, FWIW I opened a slack thread when I first stumbled on this stuff. Eventually I found the above linked discussion which explained some of my 2 -> 3 logging pain hence me opening this issue. Sorry for the wall of text hopefully that helps.

[ashb]

@r-richmond All of those are about viewing the task logs, correct?

The reason I ask is that in #52651 I'm re-working all the logging and config, but display and fields of Task logs specifically will mostly need to be handled in the UI

[r-richmond]

Yes this is all directed at task logs. No issues with audit logs and others at the moment.

Edit: actually can you clarify? I'm assuming that once you implement the above fix for the configs the logging displayed in the task logs will respect those configs (i.e things like 1&2 above not 3 or 4). Or is there another layer that needs be done for the format of the logs in the task log viewer to be updated?