Skip to content

fix: add agent environment details to telemetry - #667

Open
JoshMock wants to merge 6 commits into
mainfrom
telemetry-agent
Open

JoshMock wants to merge 6 commits into
mainfrom
telemetry-agent

Conversation

@JoshMock

Copy link
Copy Markdown
Member
  • adds @elastic/agent-env standalone package
  • incorporates agent short code into x-elastic-client-meta
  • adds LLM model + vendor (if found) to user-agent

Fixes #397

@github-actions

Copy link
Copy Markdown
Contributor

👋 This PR can use the first-failure repair loop on GitHub Actions and Buildkite.

Labels

  • auto-loop: on a CI or Buildkite failure, the bot can push a fix
  • skip-ai-review: silences the review bot
  • skip-auto-loop: added when you comment /stop

Comments

  • /stop: halt the repair loop
  • /bad on a finding: the bot writes a short do not re-flag line from the finding and your note into .github/skills/ai-review-memory.md. A reason is optional.
    • Example: /bad false positive. This path is already encoded.

github-actions[bot]

This comment was marked as outdated.

@github-actions

github-actions Bot commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

MegaLinter analysis: Success

Descriptor Linter Files Fixed Errors Warnings Elapsed time
✅ COPYPASTE jscpd yes no no 0.64s
✅ REPOSITORY gitleaks yes no no 55.86s
✅ REPOSITORY git_diff yes no no 0.12s
✅ REPOSITORY secretlint yes no no 10.94s
✅ REPOSITORY trivy yes no no 25.21s
✅ TYPESCRIPT eslint 4 0 0 3.16s

Notices

📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining SECURITY_SUGGESTIONS: false)

See detailed reports in MegaLinter artifacts
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff

MegaLinter is graciously provided by OX Security
Show us your support by starring ⭐ the repository

@github-actions

Copy link
Copy Markdown
Contributor

First failed job: Check SPDX license header

Run: https://github.com/elastic/cli/actions/runs/35374182817
Job: https://github.com/elastic/cli/actions/runs/35374182817/job/105694875864

2026-09-18T17:25:11.5825962Z Correct: test/config/loader.test.ts
2026-09-18T17:25:11.5848361Z Correct: test/config/schema.test.ts
2026-09-18T17:25:11.5870518Z Correct: test/config/secret-store.test.ts
2026-09-18T17:25:11.5894545Z Correct: test/config/store.test.ts
2026-09-18T17:25:11.5920523Z Correct: test/config/text-output.test.ts
2026-09-18T17:25:11.5943881Z Correct: test/config/writer.test.ts
2026-09-18T17:25:11.5965731Z Correct: test/confirmation.test.ts
2026-09-18T17:25:11.5990059Z Correct: test/docs/ask.test.ts
2026-09-18T17:25:11.6011947Z Correct: test/docs/chat.test.ts
2026-09-18T17:25:11.6034013Z Correct: test/docs/client.test.ts
2026-09-18T17:25:11.6056038Z Correct: test/docs/read.test.ts
2026-09-18T17:25:11.6078576Z Correct: test/docs/renderer.test.ts
2026-09-18T17:25:11.6100580Z Correct: test/docs/search.test.ts
2026-09-18T17:25:11.6126551Z Correct: test/docs/stream.test.ts
2026-09-18T17:25:11.6150464Z Correct: test/es/apis.test.ts
2026-09-18T17:25:11.6182046Z Correct: test/es/errors.test.ts
2026-09-18T17:25:11.6207212Z Correct: test/es/handler.test.ts
2026-09-18T17:25:11.6231525Z Correct: test/es/helpers/bulk-ingest.test.ts
2026-09-18T17:25:11.6266511Z Correct: test/es/helpers/msearch.test.ts
2026-09-18T17:25:11.6288957Z Correct: test/es/helpers/register.test.ts
2026-09-18T17:25:11.6311500Z Correct: test/es/helpers/scroll-search.test.ts
2026-09-18T17:25:11.6342862Z Correct: test/es/helpers/shared.test.ts
2026-09-18T17:25:11.6364693Z Correct: test/es/helpers/watch.test.ts
2026-09-18T17:25:11.6386630Z Correct: test/es/register.test.ts
2026-09-18T17:25:11.6411277Z Correct: test/es/request-builder.test.ts
2026-09-18T17:25:11.6436063Z Correct: test/es/search-schema.test.ts
2026-09-18T17:25:11.6458243Z Correct: test/es/types.test.ts
2026-09-18T17:25:11.6480209Z Correct: test/extension/env.test.ts
2026-09-18T17:25:11.6514816Z Correct: test/extension/installer.test.ts
2026-09-18T17:25:11.6536602Z Correct: test/extension/register.test.ts
2026-09-18T17:25:11.6559540Z Correct: test/extension/runner.test.ts
2026-09-18T17:25:11.6584912Z Correct: test/extension/search.test.ts
2026-09-18T17:25:11.6608571Z Correct: test/extension/store.test.ts
2026-09-18T17:25:11.6630524Z Correct: test/factory-core.test.ts
2026-09-18T17:25:11.6662890Z Correct: test/factory.test.ts
2026-09-18T17:25:11.6685106Z Correct: test/kb/apis.test.ts
2026-09-18T17:25:11.6707366Z Correct: test/kb/handler.test.ts
2026-09-18T17:25:11.6733275Z Correct: test/kb/kibana-client.test.ts
2026-09-18T17:25:11.6758670Z Correct: test/kb/register.test.ts
2026-09-18T17:25:11.6780876Z Correct: test/kb/request-builder.test.ts
2026-09-18T17:25:11.6805994Z Correct: test/lib/ajv-validate-real-schemas.test.ts
2026-09-18T17:25:11.6831447Z Correct: test/lib/ajv-validate.test.ts
2026-09-18T17:25:11.6854147Z Correct: test/lib/cloud-client.test.ts
2026-09-18T17:25:11.6879266Z Correct: test/lib/es-client.test.ts
2026-09-18T17:25:11.6908083Z Correct: test/lib/is-loopback-host.test.ts
2026-09-18T17:25:11.6932125Z Correct: test/lib/json-schema-args-collisions.test.ts
2026-09-18T17:25:11.6954394Z Correct: test/lib/json-schema-args.test.ts
2026-09-18T17:25:11.6976363Z Correct: test/lib/json-schema-refs.test.ts
2026-09-18T17:25:11.6998495Z Correct: test/lib/kibana-client.test.ts
2026-09-18T17:25:11.7020296Z Correct: test/lib/manifest-drift.test.ts
2026-09-18T17:25:11.7042306Z Correct: test/lib/meta.test.ts
2026-09-18T17:25:11.7064258Z Correct: test/lib/output-transform.test.ts
2026-09-18T17:25:11.7086214Z Correct: test/lib/sanitize.test.ts
2026-09-18T17:25:11.7109222Z Correct: test/lib/sse.test.ts
2026-09-18T17:25:11.7131253Z Correct: test/output.test.ts
2026-09-18T17:25:11.7152976Z Correct: test/packaging.test.ts
2026-09-18T17:25:11.7175294Z Correct: test/sanitize/register.test.ts
2026-09-18T17:25:11.7198824Z Correct: test/scripts/generate-schema-loaders.test.ts
2026-09-18T17:25:11.7221032Z Correct: test/scripts/repair-loop.test.ts
2026-09-18T17:25:11.7244408Z Correct: test/setup-non-interactive.ts
2026-09-18T17:25:11.7266305Z Correct: test/status/checks.test.ts
2026-09-18T17:25:11.7288897Z Correct: test/status/format.test.ts
2026-09-18T17:25:11.7311985Z Correct: test/status/register.test.ts
2026-09-18T17:25:11.7414082Z ##[error]Process completed with exit code 1.
2026-09-18T17:25:11.7530424Z Post job cleanup.
2026-09-18T17:25:11.8203304Z [command]/usr/bin/git version
2026-09-18T17:25:11.8240808Z git version 2.55.0
2026-09-18T17:25:11.8266035Z Temporarily overriding HOME='/home/runner/work/_temp/4826455f-e164-4ed8-b01c-c81bf8c9c084' before making global git config changes
2026-09-18T17:25:11.8266744Z Adding repository directory to the temporary git global config as a safe directory
2026-09-18T17:25:11.8269777Z [command]/usr/bin/git config --global --add safe.directory /home/runner/work/cli/cli
2026-09-18T17:25:11.8295190Z Removing SSH command configuration
2026-09-18T17:25:11.8300182Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2026-09-18T17:25:11.8327247Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2026-09-18T17:25:11.8509189Z Removing HTTP extra header
2026-09-18T17:25:11.8512806Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2026-09-18T17:25:11.8539157Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2026-09-18T17:25:11.8716248Z Removing includeIf entries pointing to credentials config files
2026-09-18T17:25:11.8721000Z [command]/usr/bin/git config --local --name-only --get-regexp ^includeIf\.gitdir:
2026-09-18T17:25:11.8745273Z [command]/usr/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
2026-09-18T17:25:11.9041707Z Cleaning up orphan processes

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Clean binary builds cannot resolve the unbuilt package, and some detected harnesses are still omitted from telemetry.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds agent-harness and LLM attribution to CLI telemetry through a new reusable environment-detection package.

Changes:

  • Detects agent harnesses and models from environment variables.
  • Adds attribution to User-Agent and client-meta headers.
  • Updates packaging, tests, licensing, and documentation.
File summaries
File Description
src/lib/meta.ts Integrates detected attribution into telemetry headers.
test/lib/meta.test.ts Tests telemetry attribution and opt-out behavior.
packages/agent-env/src/index.ts Implements agent and model detection.
packages/agent-env/test/index.test.ts Tests detection, confidence, and normalization.
packages/agent-env/README.md Documents the detection package.
packages/agent-env/package.json Defines the new workspace package.
packages/agent-env/tsconfig.json Configures TypeScript compilation.
packages/agent-env/LICENSE Adds package licensing.
packages/config-resolver/package.json Makes the existing package publishable.
packages/config-resolver/LICENSE Adds package licensing.
package.json Adds and bundles the new dependency.
package-lock.json Locks workspace and dependency changes.
tsconfig.json Adds the new project reference.
NOTICE.txt Updates bundled dependency notices.
scripts/openrouter-models.mjs Adds an ancillary model-listing utility.
Review details

Suppressed comments (1)

test/lib/meta.test.ts:228

  • Despite the comment, this snapshots only five of the detector's marker variables. An inherited marker such as CURSOR_TRACE_ID makes the “no agent” case return Cursor metadata and can make the Claude case low-confidence. Use the complete detector input set so these tests are harness-independent.
describe('agentMetaSegment', () => {
  // Snapshot every env var agentMetaSegment can read so branch tests are
  // deterministic regardless of the harness running the suite.
  const AGENT_VARS = ['AI_AGENT', 'AGENT', 'CLAUDECODE', 'CLAUDE_CODE', 'PI_CODING_AGENT']
  const snapshot = new Map(AGENT_VARS.map((k) => [k, process.env[k]]))
  • Files reviewed: 13/15 changed files
  • Comments generated: 13
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread packages/agent-env/package.json Outdated
Comment thread scripts/openrouter-models.mjs Outdated
Comment thread scripts/openrouter-models.mjs Outdated
Comment thread packages/agent-env/package.json
Comment thread packages/agent-env/package.json
Comment thread src/lib/meta.ts Outdated
Comment thread test/lib/meta.test.ts
Comment thread test/lib/meta.test.ts Outdated
Comment thread packages/agent-env/README.md
Comment thread packages/config-resolver/package.json
@JoshMock JoshMock added the auto-loop Opt this PR into the CI and review repair loop label Sep 18, 2026
Assisted-by: Pi Coding Agent (Claude Sonnet 4.5)
Include the LLM model name (with vendor prefix when provided by agent-env)
in the user-agent string as a semicolon-separated token inside the
existing parenthetical, e.g.:
  elastic-cli/0.5.0 (linux x64; Node.js v22.x.x; anthropic/claude-sonnet-4-6)

Assisted-by: Pi Coding Agent (Claude Sonnet 4.6)
github-actions[bot]

This comment was marked as outdated.

isKnownAgent used `v in AGENT_SHORT_CODES` which walks the prototype chain,
so AGENT=constructor or AGENT=toString would pass the guard and be treated as
a known agent. agentMetaOf used a bracket read without an own-property check,
so AI_AGENT=constructor would resolve to the Object constructor function and
stringify it into the ag= client-meta header.

- isKnownAgent: Object.hasOwn(AGENT_SHORT_CODES, v)
- agentMetaOf: guard with Object.hasOwn before reading the short code

Assisted-by: Pi Coding Agent (Claude Sonnet 4.5)
The README documents these vars as taking priority over the marker table, but
collectVotes was appending them as extra votes. AI_AGENT=devin + CLAUDECODE=1
produced two disagreeing votes (confidence 0.5), failing the default 0.95
threshold and silently discarding the explicit opt-in.

collectVotes now returns early when AI_AGENT or AGENT is set, so the marker
table is skipped entirely. The winning agent is always the opt-in value with
confidence 1 -- matching the documented override semantics.

Assisted-by: Pi Coding Agent (Claude Sonnet 4.5)
@github-actions

Copy link
Copy Markdown
Contributor

PR type mismatch: title uses fix: add agent environment details to telemetry, diff looks like feat instead.

The diff adds a new agent-env package and new telemetry headers for the spawning harness rather than correcting a bug.

This affects the version bump and changelog entry release-please generates on merge. Update the PR title (and squash commit message) to match, or push a follow-up commit that justifies the current prefix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-loop Opt this PR into the CI and review repair loop

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add attribution of CLI harness/source when spawning the Elastic CLI

2 participants