Ensure that filename and lineno of logger calls are present in the Task Logs#55581
Merged
kaxil merged 1 commit intoapache:mainfrom Sep 15, 2025
Merged
Conversation
ashb
requested review from
amoghrajesh,
bbovenzi,
jscheffl,
pierrejeambrun,
ryanahamilton and
shubhamraj-git
as code owners
boring-cyborg
bot
added
area:ConfigTemplates
area:logging
area:task-sdk
area:UI
ashb
force-pushed
the
update-task-log-caller-info
branch
2 times, most recently
from
7c6d656 to
39be1aa
Compare
eladkal
approved these changes
Sep 14, 2025
This comment was marked as resolved.
This comment was marked as resolved.
ashb
force-pushed
the
update-task-log-caller-info
branch
2 times, most recently
from
8c2a4ea to
024b1a5
Compare
kaxil
approved these changes
Sep 15, 2025
kaxil
reviewed
Sep 15, 2025
…sk Logs This was present in 2.x (and earlier) in the default config which most people left as is. This brings the information back (though displayed in task logs in a different format) Additionally, although the `log_format` config is not respected for the display, we do examine it to see if any of the other "callsite parameters" (as structlog calls them) such as `processName` or `process` (the pid) are present, and if they are, those will be recorded in the JSON task logs too. As part of this, the already-ignored (since apache#52651) colored_log_format has been removed from the config so it doesn't show up in the docs. THe "Toggle Source" option in the front end now also hides the "loc" (short for location. Maybe should be localized? Ditto with "source" though) field. I also added a unit test to the "Toggle Source" in the front end which wasn't covered by any unit tests. Fixes apache#54145
ashb
force-pushed
the
update-task-log-caller-info
branch
from
024b1a5 to
31bde20
Compare
kaxil
pushed a commit
that referenced
this pull request
Sep 15, 2025
…sk Logs (#55581) This was present in 2.x (and earlier) in the default config which most people left as is. This brings the information back (though displayed in task logs in a different format) Additionally, although the `log_format` config is not respected for the display, we do examine it to see if any of the other "callsite parameters" (as structlog calls them) such as `processName` or `process` (the pid) are present, and if they are, those will be recorded in the JSON task logs too. As part of this, the already-ignored (since #52651) colored_log_format has been removed from the config so it doesn't show up in the docs. THe "Toggle Source" option in the front end now also hides the "loc" (short for location. Maybe should be localized? Ditto with "source" though) field. I also added a unit test to the "Toggle Source" in the front end which wasn't covered by any unit tests. Fixes #54145 (cherry picked from commit 6801bca)
yash1thsa
pushed a commit
to yash1thsa/airflow
that referenced
this pull request
Sep 16, 2025
…sk Logs (apache#55581) This was present in 2.x (and earlier) in the default config which most people left as is. This brings the information back (though displayed in task logs in a different format) Additionally, although the `log_format` config is not respected for the display, we do examine it to see if any of the other "callsite parameters" (as structlog calls them) such as `processName` or `process` (the pid) are present, and if they are, those will be recorded in the JSON task logs too. As part of this, the already-ignored (since apache#52651) colored_log_format has been removed from the config so it doesn't show up in the docs. THe "Toggle Source" option in the front end now also hides the "loc" (short for location. Maybe should be localized? Ditto with "source" though) field. I also added a unit test to the "Toggle Source" in the front end which wasn't covered by any unit tests. Fixes apache#54145
suman-himanshu
pushed a commit
to suman-himanshu/airflow
that referenced
this pull request
Sep 17, 2025
…sk Logs (apache#55581) This was present in 2.x (and earlier) in the default config which most people left as is. This brings the information back (though displayed in task logs in a different format) Additionally, although the `log_format` config is not respected for the display, we do examine it to see if any of the other "callsite parameters" (as structlog calls them) such as `processName` or `process` (the pid) are present, and if they are, those will be recorded in the JSON task logs too. As part of this, the already-ignored (since apache#52651) colored_log_format has been removed from the config so it doesn't show up in the docs. THe "Toggle Source" option in the front end now also hides the "loc" (short for location. Maybe should be localized? Ditto with "source" though) field. I also added a unit test to the "Toggle Source" in the front end which wasn't covered by any unit tests. Fixes apache#54145
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This was present in 2.x (and earlier) in the default config which most people
left as is. This brings the information back (though displayed in task logs in
a different format)
Additionally, although the
log_formatconfig is not respected for thedisplay, we do examine it to see if any of the other "callsite parameters" (as
structlog calls them) such as
processNameorprocess(the pid) arepresent, and if they are, those will be recorded in the JSON task logs too.
As part of this, the already-ignored (since #52651) colored_log_format has
been removed from the config so it doesn't show up in the docs.
THe "Toggle Source" option in the front end now also hides the "loc" (short
for location. Maybe should be localized? Ditto with "source" though) field. I
also added a unit test to the "Toggle Source" in the front end which wasn't
covered by any unit tests.
Some screenshots of what it looks like:
Where it appears in task logs (orange line is a highlight in the screenshot, not the real thing)
When "Toggle Source" is turned off, both
sourceandlocare hidden.When a celery worker is run with
AIRFLOW__LOGGING__LOG_FORMAT='[%(asctime)s] {%(filename)s:%(lineno)d %(processName)s} %(levelname)s - %(message)s'we seeprocess_namein the output logsFixes #54145
Closes #54433
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.