Repository navigation
feat(ai): add per-request context to span processor - #1060
Conversation
![posthog[bot]](https://avatars.githubusercontent.com/in/1203049?s=60&v=4){kind=small}
| ) | ||
| self._processor = BatchSpanProcessor(exporter) | ||
| self._captured: Dict[Tuple[int, int], RequestContext] = {} | ||
| self._captured_lock = threading.Lock() |
There was a problem hiding this comment.
P1 · Reinitialize the capture lock after a fork
When a process forks while another thread holds _captured_lock, the child inherits a locked mutex without the thread that releases it. Every subsequent on_end blocks, including spans without PostHog context, hanging application work despite the underlying batch processor's fork handling. Reinitialize the lock and clear captured state after fork.
| snapshot = RequestContext( | ||
| distinct_id=get_context_distinct_id() or None, | ||
| session_id=get_context_session_id() or None, | ||
| tags=dict(get_tags()), |
There was a problem hiding this comment.
P2 · Freeze mutable tag values when capturing the context
When a dictionary or list passed to contexts.tag changes before the span ends, dict(get_tags()) retains the same mutable value. with_request_context then exports the updated value instead of the context at span start; a dictionary tagged with tier='pro' exports tier='free' after mutation. Copy or normalize each value during capture.
There was a problem hiding this comment.
leaving this one on purpose. values are read when the span ends so spans that turn out not to be AI spans (http, db...) cost almost nothing. snapshotting at start would bring that cost back to guard against code mutating a tagged list while an llm call is in flight, which we don't expect. noted in the docstring that tag values are read at span end
| if captured: | ||
| with self._captured_lock: | ||
| self._captured[_span_key(span)] = captured |
There was a problem hiding this comment.
P2 · Release captured contexts when abandoned spans are collected
When a manually started span is abandoned without end(), _captured retains its context even after the span is garbage collected. This applies to non-AI spans too, and retains arbitrary tag objects until processor shutdown, causing memory growth in long-lived services. Tie entries to span lifetime with weak-reference cleanup or enforce bounded retention.
| if isinstance(value, _PRIMITIVE_TYPES): | ||
| return value | ||
| if isinstance(value, (list, tuple)) and _is_homogeneous(value): | ||
| return list(value) |
There was a problem hiding this comment.
P2 · Validate integer tags against OTLP's signed 64-bit range
A context tag containing an integer outside the signed 64-bit range, such as 2**63, passes through to_attribute_value unchanged, including inside homogeneous lists. OTLP's AnyValue.int_value cannot represent it, so the exporter logs an encoding error and drops the entire property. Validate integer bounds and stringify oversized values before export.
posthog-python Compliance ReportDate: 2026-10-14T11:19:35.903198+00:00 ✅ All Tests Passed!121/121 tests passed Capture_V1 Tests✅ 95/95 tests passed View Details
Capture_Ai Tests✅ 5/5 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
Feature_Flags_Local_Evaluation Tests✅ 4/4 tests passed View Details
|
|
[Medium impact] Adds per-request context capture to the AI span processor. Fix the inherited lock before merging; forked workers can hang when spans end. Reviews (1) · Last reviewed commit: "feat: span processor request context" · Reviewed by Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| ) | ||
| self._processor = BatchSpanProcessor(exporter) | ||
| self._captured: Dict[Tuple[int, int], RequestContext] = {} | ||
| self._captured_lock = threading.Lock() |
There was a problem hiding this comment.
If another parent thread holds _captured_lock when a worker forks, the child inherits the locked object but not the thread that can release it. Every subsequent on_end() waits forever, even for non-AI spans or spans without PostHog context, so requests can hang.
Reset the lock and clear _captured in a child-process fork hook.
Knowledge Base Used: Framework integrations
Prompt To Fix With AI
This is a comment left during a code review.
Path: posthog/ai/otel/processor.py
Line: 82
Comment:
**Forked workers can hang**
If another parent thread holds `_captured_lock` when a worker forks, the child inherits the locked object but not the thread that can release it. Every subsequent `on_end()` waits forever, even for non-AI spans or spans without PostHog context, so requests can hang.
Reset the lock and clear `_captured` in a child-process fork hook.
**Knowledge Base Used:** [Framework integrations](https://app.greptile.com/posthog-org-19734/-/custom-context/knowledge-base/posthog/posthog-python/-/docs/framework-integrations.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| snapshot = RequestContext( | ||
| distinct_id=get_context_distinct_id() or None, | ||
| session_id=get_context_session_id() or None, | ||
| tags=dict(get_tags()), |
There was a problem hiding this comment.
dict(get_tags()) copies the keys but keeps references to mutable tag values. If a caller tags a list, starts a span, then appends to that list before ending the span, the exported tag contains the appended value rather than the value captured at span start.
Convert each tag to an independent attribute value during on_start(), preserving the per-value error guards, and add a mutation test.
Prompt To Fix With AI
This is a comment left during a code review.
Path: posthog/ai/otel/_request_context.py
Line: 36
Comment:
**Saved tags change later**
`dict(get_tags())` copies the keys but keeps references to mutable tag values. If a caller tags a list, starts a span, then appends to that list before ending the span, the exported tag contains the appended value rather than the value captured at span start.
Convert each tag to an independent attribute value during `on_start()`, preserving the per-value error guards, and add a mutation test.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
leaving this one on purpose. values are read when the span ends so spans that turn out not to be AI spans (http, db...) cost almost nothing. snapshotting at start would bring that cost back to guard against code mutating a tagged list while an llm call is in flight, which we don't expect. noted in the docstring that tag values are read at span end
There was a problem hiding this comment.
That makes sense. Reading tags at span end keeps the common non-AI span path cheap, and the documented end-time semantics make the mutable-value behavior an intentional tradeoff rather than an unnoticed snapshot bug. No change needed.
| from ...contexts import get_context_distinct_id, get_context_session_id, get_tags | ||
|
|
||
| DISTINCT_ID_ATTRIBUTE = "posthog.distinct_id" | ||
| SESSION_ID_ATTRIBUTE = "$ai_session_id" |
There was a problem hiding this comment.
Should this be $session_id? The rest of the SDK treats the context session as the web/replay session. add_context_tags writes it as $session_id (client.py:269), and the Django and ASGI middleware fill it from the posthog-js X-POSTHOG-SESSION-ID header. Stamping it as $ai_session_id has three effects:
- AI events get no
$session_id, so the trace view's replay link never shows (getSessionIDreads only$session_id). - Ingestion fills
$ai_session_idfrom the instrumentation's own session ID (e.g.gen_ai.conversation.id,anthropic.session.id) only while it's unset, so this replaces the real conversation grouping with the browser session. - The native wrappers in
posthog.aicapture under the same context and send$session_id, so OTel and wrapper events disagree.
Would mapping it to $session_id, and leaving $ai_session_id to span attributes or an explicit tag, work? posthog-js#5299 and posthog-go#371 use the same mapping, so all three would change together.
| def request(user): | ||
| with new_request_context(): | ||
| contexts.identify_context(user) | ||
| both_inside.wait() |
There was a problem hiding this comment.
The barrier releases before either span starts, and each span starts and ends straight away, so the two spans can run one after the other. A single shared "last captured" slot would still pass, and so would moving the capture from on_start to on_end, since every test ends its span inside the context. Could this start both spans inside their contexts, wait on the barrier, and then end them after the contexts exit? That covers concurrent spans and the streaming case the on_start snapshot exists for.
| pypi/posthog: minor | ||
| --- | ||
|
|
||
| `PostHogSpanProcessor` now adds the active PostHog context to the AI spans it exports: the distinct ID from `identify_context`, the session ID from `set_context_session`, and context tags as properties. Spans started inside `posthog.new_context()` reach PostHog under the right person, so one process can trace many users. Attributes already on a span are kept, and other processors on the provider never see the context. Apps already using PostHog contexts (for example the Django, Flask or ASGI request middleware) will see their AI events attributed to the context's person after upgrading. |
There was a problem hiding this comment.
Could this also say that the middleware tags (email, $ip, $user_agent, $current_url, ...) now land on every AI event? The Go changeset mentions it, and it's the part of the upgrade a privacy-minded user would want to know about.
| warn_if_posthog_ai_gateway_otel_attributes(span.attributes) | ||
| if captured: | ||
| try: | ||
| span = with_request_context(span, captured) |
There was a problem hiding this comment.
P1 · Provide a privacy filter before automatically exporting context tags
With Django middleware and a Client.before_send scrubber removing email and $current_url, this enrichment newly uploads those raw properties through OTLP. with_request_context copies middleware tags directly, and the batch processor bypasses the client's scrubber, exposing previously filtered request data. Provide a filtering hook or explicit opt-in before exporting tags.
| extra = { | ||
| key: value | ||
| for key, value in request_context_attributes(context).items() | ||
| if key not in own and not (has_identity and key == DISTINCT_ID_ATTRIBUTE) |
There was a problem hiding this comment.
P2 · Preserve span identity across all context identity aliases
With posthog.distinct_id='bob' on a span and an ai.telemetry.metadata.posthog_distinct_id='alice' context tag, this guard adds the tag because it only excludes the canonical key. Ingestion prioritizes the metadata alias and attributes Bob's event to Alice, violating span precedence. Suppress all context identity aliases when the span already has an identity.
| def with_request_context(span: ReadableSpan, context: RequestContext) -> ReadableSpan: | ||
| """Return a copy of ``span`` with the context's values added, keeping attributes it already has.""" | ||
| own = span.attributes or {} | ||
| has_identity = any(key in own for key in IDENTITY_ATTRIBUTES) |
There was a problem hiding this comment.
P2 · Check that existing identity attributes contain usable IDs
When a span contains user.id='' or user.id=42, has_identity becomes true and blocks a valid ID from identify_context('alice'). Ingestion accepts only nonempty strings for identity, so it ignores the span value and assigns a resource or anonymous identity instead of Alice. Only treat identity attributes as set when ingestion can use their values.
| context=span.get_span_context(), | ||
| parent=span.parent, | ||
| resource=span.resource, | ||
| attributes={**own, **extra}, |
There was a problem hiding this comment.
P2 · Apply configured span limits to context attributes
When a provider configures SpanLimits or OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT, context tags bypass those safeguards because with_request_context supplies a plain dictionary to ReadableSpan. Large strings and arbitrary tag counts reach the batch queue and OTLP encoder unbounded, increasing memory and export costs. Apply the originating span's limits when merging context attributes.
2 participants
💡 Motivation and Context
PostHogSpanProcessorsends every AI event under one person, or none.PostHogSpanProcessornow adds the active PostHog context to the AI spans it exports:identify_context→posthog.distinct_idset_context_session→$session_idposthog.new_context(), as for any other capture.How it works
on_startstores the active context for the span, keyed by trace and span ID. It does nothing when no context is active.on_endreleases the entry for every span. For AI spans it forwards a copy of the span with the context attributes merged in.posthog.distinct_id,user.idor theai.telemetry/ai.settingsdistinct ID keys), noposthog.distinct_idis added from the context or from a tag.$session_id, the same ascapture, so AI events link to replays.$ai_session_idstays free for the instrumentation's own conversation ID, or an explicit$ai_session_idtag.os.fork. A span that is never ended releases its entry when it is garbage-collected.datetime,UUIDandDecimalbecome strings, with ISO format for datesNone, empty strings and values that can't be encoded are skippedshutdownclears the store.PostHogTraceExportercan't do this, because an exporter only sees finished spans. Its docstring now says so.Note
Apps that already use PostHog contexts, for example through the Django, Flask or ASGI request middleware, will see their AI events attributed to the context's person after upgrading. The changeset says so.
Same change in posthog-js (
@posthog/ai) and posthog-go (otel).💚 How did you test it?
posthog/test/ai/otel/test_processor.pyrun a realTracerProvider. They cover:$ai_generationevent with the right distinct ID,$session_idand tag. The same run passed for Node and Go.ReadableSpanconstructor used for the copy is identical on opentelemetry-sdk 1.20, 1.27 and 1.40.📝 Checklist
If releasing new changes
sampo addto generate a changeset file🤖 Agent context
Autonomy: Human-driven (agent-assisted)
claude-opus-5-5) and subagents, directed by @carlos-marchal-ph.(trace_id, span_id)rather than a weak reference to the span, because the SDK passeson_enda freshReadableSpanobject.🤖 Generated with Claude Code