Emit gen_ai.client.token.usage and operation.duration metrics from the OTel middleware - #666
Conversation
026998e to
5773147
Compare
This comment has been minimized.
This comment has been minimized.
5773147 to
23dfb1a
Compare
github-actions
Bot
added
the
parity-approved
This comment has been minimized.
This comment has been minimized.
…el middleware
The OTel middleware recorded token counts only as span attributes, which are
not aggregatable across runs, so a spend/latency dashboard saw nothing from the
Go port. Add a Meter alongside the tracer and record the two histograms the
GenAI semantic conventions define and the Python/.NET SDKs emit:
gen_ai.client.token.usage ({token}) split by gen_ai.token.type input/output, and
gen_ai.client.operation.duration (s). Both are recorded on the normal and
early-return exit paths, tagged with operation/provider/agent name plus
error.type when the run faults. Histogram-construction failures leave a working
tracer-only middleware.
23dfb1a to
0076565
Compare
This comment has been minimized.
This comment has been minimized.
# Conflicts: # go.mod # provider/otelprovider/otel.go
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Pull request overview
This PR extends the provider/otelprovider OpenTelemetry middleware to emit GenAI semantic-convention metrics in addition to existing span attributes, enabling cross-run aggregation for token usage and operation duration (dashboards/alerts) while preserving per-run span detail.
Changes:
- Add OTel Metrics instruments (
gen_ai.client.token.usage,gen_ai.client.operation.duration) to the existing tracing middleware and record them per run. - Capture and attach shared identifying attributes (and
error.typeon failures) to emitted metric data points. - Add black-box tests using an in-memory
sdkmetric.ManualReaderto assert emitted histograms and error attribution.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| provider/otelprovider/otel.go | Adds Meter-based histograms and records token-usage + duration metrics from the middleware. |
| provider/otelprovider/otel_test.go | Adds metric-focused tests validating histogram emission and error tagging. |
| go.mod | Promotes OTel metric and sdk/metric dependencies to direct requirements. |
| go.sum | Records new module checksums introduced by metric dependencies. |
| attrs := []attribute.KeyValue{ | ||
| attribute.String(attrKeyOperationName, opInvoke), | ||
| attribute.String(attrKeyProviderName, cmp.Or(a.ProviderName(), "unknown")), | ||
| attribute.String(attrKeyAgentName, a.Name()), | ||
| } | ||
| if errorType != "" { | ||
| attrs = append(attrs, attribute.String(attrKeyErrorType, errorType)) | ||
| } |
|
PratikDhanave (@PratikDhanave) conflicts |
# Conflicts: # go.mod # go.sum
github-actions
Bot
added
area:provider
This comment has been minimized.
This comment has been minimized.
github-actions
Bot
added
failed-auto-risk
github-actions
Bot
added
pending-auto-risk
Cross-SDK Parity Review — ✅ ApprovedThis PR adds Python
.NET
Go contract
Attribute keys used (
|
What
The OTel middleware previously recorded token counts purely as span attributes (
gen_ai.usage.*) viasetUsage, and never measured run duration at all. It imported onlyotel/{attribute,codes,trace}and held a lonetracer— no Meter, no histograms.Span attributes describe a single run and are not aggregatable across runs, so a spend/latency dashboard summing across many invocations saw nothing from the Go port. This change adds a Meter alongside the tracer and emits the two histograms the OTel GenAI semantic conventions define:
gen_ai.client.token.usage(unit{token}) — two data points per run, one taggedgen_ai.token.type=input(InputTokenCount) and oneoutput(OutputTokenCount).gen_ai.client.operation.duration(units) — one sample per run, measured from beforespan.Start.Every data point carries the run's identifying attributes (
gen_ai.operation.name,gen_ai.provider.name,gen_ai.agent.name), pluserror.typewhen the run faulted. Recording is factored into arecordMetricshelper (mirroringsetUsage) shared by both the normal post-loop path and the early-return path. Histogram construction errors are tolerated:NewMiddlewarestill returns a working tracer-only middleware andrecordMetricsno-ops on a nil histogram.Why (cross-SDK parity)
This aligns the Go port with the OTel GenAI semantic conventions and the Python/.NET references, which both emit these two histograms. Cross-language dashboards can now aggregate Go runs alongside the rest. Span-attribute accounting stays exactly as-is; metrics complement it (single-run detail vs. cross-run aggregation) rather than replacing it. Not a duplicate of the span PRs (#655/#646) or usage-accounting PRs (#594/#609/#551-554).
Tests
provider/otelprovider/otel_test.gogains two black-box tests driving an in-memorysdkmetric.ManualReader:TestOtel_Run_RecordsUsageAndDurationMetrics— runs the middleware over a fakeRunFuncyieldingUsageContent(100 input / 50 output), collectsResourceMetrics, and asserts the twogen_ai.client.token.usagedata points (input=100, output=50 bygen_ai.token.type) with the shared attributes, plus onegen_ai.client.operation.durationsample.TestOtel_Run_MetricsIncludeErrorType— assertserror.typeis present on the duration metric when the run faults.Both fail before the change and pass after.
go build ./...,go vet ./provider/otelprovider/..., andgo test ./provider/otelprovider/...are green.Open design questions
MiddlewareConfigis unchanged — the Meter reusesSourceNamefor its instrumentation scope, matching the tracer. Open to a separate meter name if preferred.AdditionalCountsand cache/reasoning sub-counters are not yet emitted as metric data points (the conventions only define input/output token-type dimensions); could be added if there's appetite.