Vendor macOS Unified Logging receiver with reliable cursor tracking

[kroepke]

Vendors macosunifiedloggingreceiver from opentelemetry-collector-contrib v0.153.0 (commit 42f9491) into this repo, replacing the upstream contrib module in the build — same pattern as the existing windowseventlogreceiver.

The upstream live mode re-emits boundary events on every poll and has no persistence; this fork makes live mode production-reliable.

Key Changes:

• Added a cursor in local storage based on (machTimestamp, threadID) to be able to dedup on second boundaries, which is the granularity we can get data from the /usr/bin/log command
• Cursor persistence takes into account reboots via bootUUID and the predicate hash - changes invalidate the cursor
• Polling with exponential backoff via min_poll_interval (1s default to avoid log show amplify its own output) up to max_poll_interval
• We always use ndjson output, message body is eventMessage, anything else is prefixed with macos.
• Added extra checks that we execute only an Apple signed, untampered-with, /usr/bin/log

The collector builds cross-platform, but the important tests are essentially no-ops.

Non-obvious detail: We only advance the cursor after ConsumeLogs succeeds to make the backpressure in otel-collector work.

[Copilot]

Pull request overview

This PR vendors the upstream macOS Unified Logging receiver into this repository and wires it into the local build/test tooling, with additional live-mode correctness improvements (cursor persistence + boundary-second dedup) and platform integrity checks for /usr/bin/log.

Changes:

• Added a new receiver/macosunifiedloggingreceiver Go module implementing the macOS unified logging receiver (live + archive modes), including cursor persistence and extensive unit/integration tests.
• Integrated the new receiver into the repo Taskfiles and the collector builder configuration/components list.
• Updated top-level/module dependencies (go.mod/go.sum) to support the new receiver and align OTel component versions.

Reviewed changes

Copilot reviewed 41 out of 47 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
Taskfile.yml	Adds the macOS unified logging receiver module to formatting/test/vulncheck orchestration.
receiver/macosunifiedloggingreceiver/testdata/test_config.yaml	Adds YAML configs used by receiver config-loading tests.
receiver/macosunifiedloggingreceiver/Taskfile.yml	Adds module-local go test task.
receiver/macosunifiedloggingreceiver/storage.go	Storage client resolution helper for persisting the live-mode cursor.
receiver/macosunifiedloggingreceiver/storage_test.go	Tests for storage client resolution and test storage/host fakes.
receiver/macosunifiedloggingreceiver/severity.go	Maps macOS messageType to OTel severity numbers.
receiver/macosunifiedloggingreceiver/severity_test.go	Unit tests for severity mapping.
receiver/macosunifiedloggingreceiver/receiver.go	Core receiver implementation: polling, parsing, batching, cursoring, persistence.
receiver/macosunifiedloggingreceiver/receiver_test.go	Unit tests covering dedup, cursor behavior, error handling, oversized lines, etc.
receiver/macosunifiedloggingreceiver/receiver_integration_darwin_test.go	Darwin-only integration test against real unified logging (logger + cursor dedup).
receiver/macosunifiedloggingreceiver/README.md	Receiver documentation, config options, security model, and behavior notes.
receiver/macosunifiedloggingreceiver/parser.go	NDJSON parsing and OTel log record mapping.
receiver/macosunifiedloggingreceiver/parser_test.go	Parser/mapping unit tests.
receiver/macosunifiedloggingreceiver/metadata.yaml	mdatagen metadata for receiver type/status/tests config.
receiver/macosunifiedloggingreceiver/logrunner.go	Runner interface abstraction for invoking log.
receiver/macosunifiedloggingreceiver/logrunner_darwin.go	Darwin implementation of log runner using integrity-verified /usr/bin/log.
receiver/macosunifiedloggingreceiver/internal/metadata/generated_status.go	Generated component status metadata.
receiver/macosunifiedloggingreceiver/internal/metadata/generated_logs.go	Generated logs builder for emitted logs.
receiver/macosunifiedloggingreceiver/internal/metadata/generated_logs_test.go	Generated tests for logs builder behavior.
receiver/macosunifiedloggingreceiver/integrity_darwin.go	/usr/bin/log (and optionally codesign) integrity verification.
receiver/macosunifiedloggingreceiver/integrity_darwin_test.go	Darwin-only tests for integrity checks.
receiver/macosunifiedloggingreceiver/go.sum	Dependency lockfile for the receiver module.
receiver/macosunifiedloggingreceiver/go.mod	New receiver module definition and dependencies.
receiver/macosunifiedloggingreceiver/generated_package_test.go	Generated goleak TestMain for the receiver package.
receiver/macosunifiedloggingreceiver/generated_component_test.go	Generated factory/config/lifecycle tests from mdatagen.
receiver/macosunifiedloggingreceiver/factory.go	Factory entrypoint and default config construction.
receiver/macosunifiedloggingreceiver/factory_others.go	Non-darwin factory returning unsupported-platform error.
receiver/macosunifiedloggingreceiver/factory_others_test.go	Tests for non-darwin factory behavior.
receiver/macosunifiedloggingreceiver/factory_darwin.go	Darwin factory creating the real receiver with exec runner + config validation.
receiver/macosunifiedloggingreceiver/factory_darwin_test.go	Tests for default config values on darwin.
receiver/macosunifiedloggingreceiver/doc.go	Package docs + mdatagen directive.
receiver/macosunifiedloggingreceiver/cursor.go	Cursor implementation (boundary-second dedup + predicate hash + persistence).
receiver/macosunifiedloggingreceiver/cursor_test.go	Cursor unit tests (dedup, reboot reset, idle poll, round-trip).
receiver/macosunifiedloggingreceiver/config.schema.yaml	Schema for receiver configuration (used by tooling/docs).
receiver/macosunifiedloggingreceiver/config.go	Darwin-only config validation, predicate validation, archive glob resolution.
receiver/macosunifiedloggingreceiver/config_test.go	Darwin-only tests for config validation and archive glob resolution.
receiver/macosunifiedloggingreceiver/config_defaults_test.go	Tests for default config values.
receiver/macosunifiedloggingreceiver/config_common.go	Platform-neutral config struct shared by factories and docs.
receiver/macosunifiedloggingreceiver/cadence.go	Live polling backoff cadence implementation.
receiver/macosunifiedloggingreceiver/cadence_test.go	Unit tests for cadence backoff/reset behavior.
go.sum	Updates root dependency lockfile.
go.mod	Updates root module dependencies (incl. added replaces/indirects).
changelog/unreleased/pr-TBD-macos-unified-logging-live-mode.toml	Adds an unreleased changelog entry describing the new receiver.
builder/go.mod	Adds local receiver module + adjusts OTel contrib dependency versions.
builder/components.go	Switches builder wiring to use the local macOS receiver module.
builder/builder-config.yaml	Updates builder config to reference the local receiver module + replace entry.

Files not reviewed (3)

• builder/components.go: Generated file
• receiver/macosunifiedloggingreceiver/generated_component_test.go: Generated file
• receiver/macosunifiedloggingreceiver/generated_package_test.go: Generated file

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[kroepke]

@kroepke I noticed that the --start parameter is set using the local timezone.

Should we use the --timezone parameter to make it explicit? It seems to work as-is; just wondering about edge cases.

/usr/bin/log show --style ndjson --start 2026-07-21 14:47:00 --predicate subsystem IN {'com.apple.opendirectoryd','co...

I had initially thought that this is fine, since we are removing start_date for now and only use the relative max_log_age thus making the timezone irrelevant, but then it dawned on me: if the timezone changes, e.g. due to travel, this might become a big issue regardless, also for the stored cursor. So yes, it should all be UTC.

[bernd]

As discussed with @kroepke, here is the raw Claude Code and Codex review output.

We discussed that we are going to remove the archive mode to simplify the code base.

Claude Code Review Comments

  Issues found

  1. Archive mode silently loses data on every error path — receiver.go:321-338. ConsumeLogs errors are discarded (_ =), the wait() error and stderr are discarded, and
  scanner.Err() is never checked. A line over the 10 MB scanner cap or any read error makes Scan() return false and the rest of that archive is silently skipped,
  indistinguishable from success. Live mode handles all three of these carefully; archive mode got none of that. Also, on shutdown the outer loop keeps iterating and logs a
  spurious "failed to start log for archive" for each remaining archive instead of returning when ctx is done.

  2. The final cursor persist runs on the already-canceled shutdown context — receiver.go:253. During Shutdown, cancel() fires while a poll is in flight; pollOnce still
  commits and then calls storage.Set(ctx, …) with the canceled ctx. Whether that write survives depends on the backend (bbolt-based file_storage ignores ctx today; a
  redis-backed store would fail), so by contract the last poll's progress can be lost and its events re-emitted as duplicates on the next start. Use
  context.WithoutCancel(ctx) (or persist once more in Shutdown) for the cursor write.

  3. A storage read error at startup silently discards the cursor — receiver.go:69. The err == nil && len(data) > 0 pattern swallows Get failures, so a transient storage
  error means a silent fresh start and up to max_log_age (24 h) of duplicates. The two other "can't use the cursor" cases right below it log a Warn/Info; this one should
  too — or arguably fail Start, since the whole design treats persistence as required.

  4. min_poll_interval can be configured below the safe floor the code relies on — config.go:159. The comment on the field says it's "kept above 100ms because log show logs
  its own invocation, which can otherwise sustain a tight self-feeding poll loop" — but validation only rejects negative values. min_poll_interval: 1ms passes validation
  and produces exactly that self-feeding hot loop. Validate should enforce the documented floor. Related design note: because log show logs itself, a predicate-less config
  sees ≥1 "new" event every poll, so the cadence stays pinned at the floor and the README's exponential-backoff behavior effectively never engages.

  5. macos.userID: 0 conflates "field absent" with "root" — parser.go:97. All integer attributes are unconditionally set, so an ndjson record without userID (or processID,
  etc.) emits 0 — which for userID specifically means root, a misleading signal for exactly the security-monitoring use cases this receiver targets. Consider pointer fields
  (*int64) or presence tracking for at least userID, so absent ≠ 0.

  6. Live mode's line cap regressed from 10 MB to 1 MB, and oversized events are lost permanently — receiver.go:26. Upstream delivered lines up to 10 MB; live mode now
  skips anything over 1 MB, and once later events advance the cursor past that second the event is unrecoverable. The skip-don't-stall logic itself is sound, but the 1 MB
  choice is both a silent regression and inconsistent with archive mode's 10 MB buffer.

  7. The parser's stated invariant isn't what the caller implements — parser.go:50 says "the cursor must never advance over an event it cannot time," but pollOnce
  (receiver.go:205) just logs a Warn on parse error and keeps going; subsequently delivered events advance batchSecond right past the unparseable event's second. Skipping
  is probably the only practical behavior, but the comment promises something the code doesn't do — one of them should change.

  8. config.schema.yaml is stale — it documents neither storage nor min_poll_interval, and its format description still describes the upstream behavior that no longer
  exists. Nothing in this repo consumes the file, but if it feeds Graylog-side config tooling the new (and now-required) storage option won't surface there.

  9. Missing storage: is only caught at collector start, not config validation — storage.go:20. Config.Validate could check ArchivePath == "" && StorageID == nil and fail
  at otelcol validate time with the same clear message, instead of at Start.

  Smaller notes: the cursor is re-persisted every poll even when unchanged (harmless write churn, easy to skip by comparing); the seen set is unbounded within the boundary
  second (fine in practice — one second of logs); and the integrity check runs once at startup while log is exec'd every poll, a TOCTOU window that SIP itself renders
  mostly theoretical.

  Overall: the core cursor/backpressure design is solid and unusually well-tested — the real gaps are concentrated in archive mode's error handling (#1) and the persistence
  edge cases (#2, #3).

Codex Review Comments

  Issues found

  1. Archive mode silently loses data on every error path — receiver.go:321-338. ConsumeLogs errors are discarded (_ =), the wait() error and stderr are discarded, and
  scanner.Err() is never checked. A line over the 10 MB scanner cap or any read error makes Scan() return false and the rest of that archive is silently skipped,
  indistinguishable from success. Live mode handles all three of these carefully; archive mode got none of that. Also, on shutdown the outer loop keeps iterating and logs a
  spurious "failed to start log for archive" for each remaining archive instead of returning when ctx is done.

  2. The final cursor persist runs on the already-canceled shutdown context — receiver.go:253. During Shutdown, cancel() fires while a poll is in flight; pollOnce still
  commits and then calls storage.Set(ctx, …) with the canceled ctx. Whether that write survives depends on the backend (bbolt-based file_storage ignores ctx today; a
  redis-backed store would fail), so by contract the last poll's progress can be lost and its events re-emitted as duplicates on the next start. Use
  context.WithoutCancel(ctx) (or persist once more in Shutdown) for the cursor write.

  3. A storage read error at startup silently discards the cursor — receiver.go:69. The err == nil && len(data) > 0 pattern swallows Get failures, so a transient storage
  error means a silent fresh start and up to max_log_age (24 h) of duplicates. The two other "can't use the cursor" cases right below it log a Warn/Info; this one should
  too — or arguably fail Start, since the whole design treats persistence as required.

  4. min_poll_interval can be configured below the safe floor the code relies on — config.go:159. The comment on the field says it's "kept above 100ms because log show logs
  its own invocation, which can otherwise sustain a tight self-feeding poll loop" — but validation only rejects negative values. min_poll_interval: 1ms passes validation
  and produces exactly that self-feeding hot loop. Validate should enforce the documented floor. Related design note: because log show logs itself, a predicate-less config
  sees ≥1 "new" event every poll, so the cadence stays pinned at the floor and the README's exponential-backoff behavior effectively never engages.

  5. macos.userID: 0 conflates "field absent" with "root" — parser.go:97. All integer attributes are unconditionally set, so an ndjson record without userID (or processID,
  etc.) emits 0 — which for userID specifically means root, a misleading signal for exactly the security-monitoring use cases this receiver targets. Consider pointer fields
  (*int64) or presence tracking for at least userID, so absent ≠ 0.

  6. Live mode's line cap regressed from 10 MB to 1 MB, and oversized events are lost permanently — receiver.go:26. Upstream delivered lines up to 10 MB; live mode now
  skips anything over 1 MB, and once later events advance the cursor past that second the event is unrecoverable. The skip-don't-stall logic itself is sound, but the 1 MB
  choice is both a silent regression and inconsistent with archive mode's 10 MB buffer.

  7. The parser's stated invariant isn't what the caller implements — parser.go:50 says "the cursor must never advance over an event it cannot time," but pollOnce
  (receiver.go:205) just logs a Warn on parse error and keeps going; subsequently delivered events advance batchSecond right past the unparseable event's second. Skipping
  is probably the only practical behavior, but the comment promises something the code doesn't do — one of them should change.

  8. config.schema.yaml is stale — it documents neither storage nor min_poll_interval, and its format description still describes the upstream behavior that no longer
  exists. Nothing in this repo consumes the file, but if it feeds Graylog-side config tooling the new (and now-required) storage option won't surface there.

  9. Missing storage: is only caught at collector start, not config validation — storage.go:20. Config.Validate could check ArchivePath == "" && StorageID == nil and fail
  at otelcol validate time with the same clear message, instead of at Start.

  Smaller notes: the cursor is re-persisted every poll even when unchanged (harmless write churn, easy to skip by comparing); the seen set is unbounded within the boundary
  second (fine in practice — one second of logs); and the integrity check runs once at startup while log is exec'd every poll, a TOCTOU window that SIP itself renders
  mostly theoretical.

  Overall: the core cursor/backpressure design is solid and unusually well-tested — the real gaps are concentrated in archive mode's error handling (#1) and the persistence
  edge cases (#2, #3).

[kroepke]

@kroepke I noticed that the --start parameter is set using the local timezone.

Should we use the --timezone parameter to make it explicit? It seems to work as-is; just wondering about edge cases.

/usr/bin/log show --style ndjson --start 2026-07-21 14:47:00 --predicate subsystem IN {'com.apple.opendirectoryd','co...

this is fixed in 90a0da2 using explict timezone info in the timestamp itself, which is more reliable

[kroepke]

Code review

Found 1 issue:

1. Config.Validate() defaults the Format field with if cfg.Format == "" { cfg.Format = "default" } (AGENTS.md says "Use cmp.Or(val, fallback) instead of if val == zero { val = fallback } for defaulting zero values"). The PR already uses cmp.Or correctly for the poll intervals in receiver.go, so this is the one remaining spot.

collector/receiver/macosunifiedloggingreceiver/config.go

Lines 33 to 38 in 73fadd8

	// Set default format if not specified
	if cfg.Format == "" {
	cfg.Format = "default"
	}

For context: all nine items from the earlier CHANGES_REQUESTED review, both Copilot findings, and the timezone thread were verified as addressed at 73fadd8 (archive-mode removal, context.WithoutCancel cursor persist, startup cursor-read error propagation, 100ms min_poll_interval floor, *int64 optional fields, 10MB line cap, parser comment accuracy, config.schema.yaml removal, Validate()-time storage check, %q dereference, README predicate-handling section). Command construction passes the predicate as a single argv element via exec.CommandContext with no shell, so no injection path was found; the README matches the current config surface and defaults.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[bernd]

A few findings.