Skip to content

[api][plan][python] Use request-scoped metric groups for chat token metrics#861

Open
jlon wants to merge 1 commit into
apache:mainfrom
jlon:fix-action-scoped-token-metrics
Open

[api][plan][python] Use request-scoped metric groups for chat token metrics#861
jlon wants to merge 1 commit into
apache:mainfrom
jlon:fix-action-scoped-token-metrics

Conversation

@jlon

@jlon jlon commented Jun 26, 2026

Copy link
Copy Markdown

What

  • Capture the current action metric group when a chat request is initiated.
  • Record delayed chat token metrics against that captured group instead of the mutable metric group on cached chat model resources.
  • Add Java and Python regression coverage for resource metric-group rebinding.

Why

Cached resources can be rebound when another action retrieves the same resource before the original request records token usage. The delayed token metrics should stay under the action that initiated the request.

Closes #859.

Tests

  • mvn -pl api -DskipITs -Dspotless.check.skip=true -Dcheckstyle.skip=true -Dtest=BaseChatModelSetupTokenMetricsTest test
  • mvn -pl plan -am -DskipITs -Dspotless.check.skip=true -Dcheckstyle.skip=true -Dsurefire.failIfNoSpecifiedTests=false -Dtest=ChatModelActionTest,ChatModelActionRetryTest test
  • bash -lc 'export PATH="$HOME/.local/bin:$PATH"; cd python && uv run --no-sync pytest flink_agents/api/chat_models/tests/test_token_metrics.py -q'
  • bash -lc 'export PATH="$HOME/.local/bin:$PATH"; cd python && uv run --no-sync ruff format --check flink_agents/api/chat_models/chat_model.py flink_agents/api/chat_models/tests/test_token_metrics.py flink_agents/plan/actions/chat_model_action.py && uv run --no-sync ruff check flink_agents/api/chat_models/chat_model.py flink_agents/api/chat_models/tests/test_token_metrics.py flink_agents/plan/actions/chat_model_action.py'
  • bash -lc 'export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64; export PATH="$JAVA_HOME/bin:$PATH"; mvn -pl api,plan -am spotless:check -Dspotless.skip=false -DskipTests -DskipITs -Dcheckstyle.skip=true'

@github-actions github-actions Bot added doc-label-missing The Bot applies this label either because none or multiple labels were provided. fixVersion/0.4.0 priority/major Default priority of the PR or issue. labels Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-label-missing The Bot applies this label either because none or multiple labels were provided. fixVersion/0.4.0 priority/major Default priority of the PR or issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Action-scoped metrics can be recorded under the wrong action for cached resources

1 participant