Skip to content

Harden .NET client security, reconnects, and Native AOT - #411

Merged
Connor Peet (connor4312) merged 12 commits into
mainfrom
davidfowl-harden-dotnet-client
Aug 24, 2026
Merged

Harden .NET client security, reconnects, and Native AOT#411
Connor Peet (connor4312) merged 12 commits into
mainfrom
davidfowl-harden-dotnet-client

Conversation

@davidfowl

@davidfowl David Fowler (davidfowl) commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • create Unix client-ID temporary files with owner-only permissions before writing any bytes
  • preserve atomic replacement and Windows behavior across net8.0 and netstandard2.0
  • tighten caller-owned configuration, serializer, value-object, and invalid-state API boundaries before first publication
  • validate negotiated protocol versions and snapshot mutable WebSocket options before connection callbacks
  • make multi-host reconnects preserve live subscriptions and the highest applied server sequence, publish replay/snapshot state before Connected, and serialize connection replacement with subscription bookkeeping
  • prevent cancelled or non-cooperative connection attempts from leaking or overwriting the committed client
  • generate System.Text.Json metadata for the complete protocol model and remove reflection from union and reducer dispatch
  • publish and execute a reflection-disabled Native AOT smoke application that calls the real PingAsync API
  • add ClientConfig.TimeProvider so request timeouts, keep-alive, reconnect scheduling, and host timestamps are deterministic in tests
  • prevent timed-out queued requests from being transmitted without canceling active WebSocket writes and aborting the shared connection

Security

FileClientIdStore no longer creates a default-permission temporary file and chmods it after content is written. On Unix:

  • net8.0 creates the file with FileStreamOptions.UnixCreateMode = 0600
  • netstandard2.0 uses native open(..., O_CREAT | O_EXCL | O_CLOEXEC, 0600) followed by fchmod(0600) before exposing the stream
  • unknown Unix platforms fail closed rather than guessing an O_CLOEXEC value
  • failures occur before client-ID bytes are written and temporary files are cleaned up

Regression coverage exercises both paths on Linux and verifies the temporary file is empty and exactly owner-readable/writable at the pre-write boundary.

API and runtime review

The complete public surfaces of Microsoft.AgentHostProtocol and Microsoft.AgentHostProtocol.Abstractions were inspected for both target frameworks and checked against examples and tests. The two TFMs expose matching APIs. High-confidence corrections include configuration ownership, frozen serializer options, valid StringOrMarkdown construction, HostId value semantics, accidental visibility, protocol negotiation validation, connection cleanup, duplicate host registration, reconnect correctness, and deterministic time control.

The runtime review also exposed reconnect gaps now covered by regression tests: dynamic subscription replay, live sequence advancement only after successful application, snapshot reconnect without re-initialization, snapshot delivery through SubscriptionEventSnapshot, replay missing pruning while retaining stateless subscriptions, handshake event buffering, root-action mirroring, old-pump draining, replacement readiness, post-ack subscription races, commit/abandon arbitration, and cleanup when a transport factory ignores cancellation.

Native AOT

The default serializer now uses generated metadata for the full canonical wire model. JSON-RPC shape converters, discriminated unions, snapshots, tool inputs, reducer action metadata, and the closed dictionary used by PingAsync all use generated metadata rather than runtime serialization/reflection. The generated context remains internal so its generated members do not become compatibility surface; consumers can compose the single AhpJsonMetadata.Default resolver API instead.

Normal reflection-enabled .NET applications retain compatibility for application-defined values passed through generic serializer/client APIs. Reflection-disabled or Native AOT applications provide their own source-generated resolver for custom values. CI publishes and runs a reflection-disabled native executable that exercises initialization, framing, unions, wire enums, snapshots, scalar unions, and a real client ping round trip.

Intentionally deferred

  • Generated union wrappers still expose parameterless construction and mutable AhpUnion.Value; correcting this safely requires coordinated generator/converter changes across the generated model surface.
  • Server-request handler delegates do not accept cancellation tokens; adding them is a cross-cutting public API design change.
  • Unknown/malformed notifications are skipped rather than surfaced through a raw-notification or diagnostic API.
  • The canonical generated package intentionally exposes the complete wire model; splitting that model would trade compatibility surface for package/versioning complexity and needs a separate design decision.
  • Generated literal-discriminant defaults and the legacy session-action/ActionType gap are tracked separately in fix(dotnet): initialize generated literal discriminants #414 rather than duplicated here.

Validation

  • dotnet format whitespace --verify-no-changes --no-restore
  • Release builds for netstandard2.0 and net8.0
  • 524 .NET tests on Windows
  • reflection-disabled Native AOT publish and execution on Windows
  • NuGet packing for both packages and both TFMs
  • npm test (typecheck, lint, release/changelog/generated checks, TypeScript tests, and 100% reducer coverage)
  • generated-source and changelog-fragment freshness checks
  • iterative final code review with no remaining findings

Secure Unix client ID writes before content reaches disk and tighten pre-release API ownership and invariants.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve live subscriptions and server sequence across reconnects, buffer handshake events, validate negotiated versions, and close connection lifecycle races.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 05359f98-1e07-4a16-8d29-7d22c32aa23a
Generate System.Text.Json metadata for the complete protocol model, remove reflection from runtime union and reducer dispatch, and add a reflection-disabled Native AOT smoke test to CI.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 05359f98-1e07-4a16-8d29-7d22c32aa23a
@davidfowl David Fowler (davidfowl) changed the title Harden .NET client ID persistence and pre-release APIs Harden .NET client security, reconnects, and Native AOT Aug 23, 2026
Fail stalled test runs after five minutes and report tests running longer than 30 seconds so hosted-runner hangs identify the owning test.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 05359f98-1e07-4a16-8d29-7d22c32aa23a
Invoke the xUnit executable directly so its global timeout and long-running-test diagnostics apply, with an outer process timeout as a fail-safe.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 05359f98-1e07-4a16-8d29-7d22c32aa23a
Begin the asynchronous HttpListener accept directly instead of scheduling it through Task.Run, preventing the client handshake from waiting forever on constrained CI runners.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 05359f98-1e07-4a16-8d29-7d22c32aa23a
Avoid a runtime-dependent close-during-connect race by waiting for the client handshake before aborting the loopback server connection.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 05359f98-1e07-4a16-8d29-7d22c32aa23a
Keep the established loopback harness behavior and release the oversized server payload only after the client handshake completes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 05359f98-1e07-4a16-8d29-7d22c32aa23a
Inject a faulting WebSocket through an internal constructor so exception mapping is tested without relying on platform-specific loopback abort timing.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 05359f98-1e07-4a16-8d29-7d22c32aa23a
Use an internal connect delegate to verify option snapshot ordering without depending on a loopback WebSocket receive race.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 05359f98-1e07-4a16-8d29-7d22c32aa23a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 05359f98-1e07-4a16-8d29-7d22c32aa23a

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR significantly hardens the .NET client implementation before publication, focusing on security (client-id storage), correctness under reconnect/multi-host scenarios, and compatibility with trimming/Native AOT by moving JSON handling to source-generated metadata and deterministic time control.

Changes:

  • Secure Unix client-id temp file creation in FileClientIdStore (owner-only permissions before any bytes are written), with regression coverage.
  • Add deterministic time control (ClientConfig.TimeProvider) and apply it across request timeouts, keep-alive, reconnect backoff, and host timestamps, updating tests accordingly.
  • Enable trimming/Native AOT for net8.0 by generating STJ metadata for the full wire model, removing reflection-based dispatch, and adding a CI-published Native AOT smoke app.
Show a summary per file
File Description
scripts/generate-csharp.ts Emits STJ source-gen context + generated reducer metadata to eliminate reflection usage.
docs/.changes/20260823-dotnet-time-provider.json Changelog fragment for ClientConfig.TimeProvider.
docs/.changes/20260821-dotnet-client-id-permissions.json Security changelog fragment for secure Unix temp file creation.
docs/.changes/20260821-dotnet-api-quality.json Changelog fragment for .NET API/Native AOT/trim hardening work.
clients/dotnet/tests/AgentHostProtocol.Tests/WebSocketTransportTests.cs Adds deterministic option snapshotting tests; replaces flaky abnormal-close test with a test double.
clients/dotnet/tests/AgentHostProtocol.Tests/TransportTests.cs Updates config-mutation expectations; validates new defaults (incl. TimeProvider).
clients/dotnet/tests/AgentHostProtocol.Tests/MultiHostClientTests.cs Adds extensive multi-host/reconnect regression coverage; introduces fake time provider usage.
clients/dotnet/tests/AgentHostProtocol.Tests/FixRegressionTests.cs Adds timeout/send-queue regressions and time-provider-driven tests.
clients/dotnet/tests/AgentHostProtocol.Tests/FileClientIdStoreTests.cs Adds regression verifying Unix temp file is empty + 0600 before writing content.
clients/dotnet/tests/AgentHostProtocol.Tests/ClientTests.cs Adds protocol negotiation validation test; makes keep-alive tests deterministic via FakeTimeProvider.
clients/dotnet/tests/AgentHostProtocol.Tests/ApiQualityTests.cs New tests pinning public API boundaries and snapshot/caller-ownership behavior.
clients/dotnet/tests/AgentHostProtocol.Tests/AgentHostProtocol.Tests.csproj Adds Microsoft.Extensions.TimeProvider.Testing for deterministic time tests.
clients/dotnet/tests/AgentHostProtocol.AotSmoke/Program.cs New reflection-disabled Native AOT smoke test program.
clients/dotnet/tests/AgentHostProtocol.AotSmoke/AgentHostProtocol.AotSmoke.csproj Native AOT publish configuration for the smoke executable.
clients/dotnet/src/AgentHostProtocol/WebSocketTransport.cs Makes transport wrap WebSocket, adds ConnectCoreAsync, snapshots options early.
clients/dotnet/src/AgentHostProtocol/TimeProviderCompatibility.cs Polyfill helpers to use TimeProvider APIs across TFMs.
clients/dotnet/src/AgentHostProtocol/SystemTextJsonAhpSerializer.cs Freezes options, plugs in generated metadata resolver, reflection fallback only when enabled.
clients/dotnet/src/AgentHostProtocol/Reducers.cs Replaces reflection-based action metadata with generated metadata lookup.
clients/dotnet/src/AgentHostProtocol/Hosts/MultiHostStateMirror.cs Moves host identity to value-semantic HostId keys and improves host-drop logic.
clients/dotnet/src/AgentHostProtocol/Hosts/MultiHostClient.cs Major reconnect correctness + caller-owned config snapshotting + serialized connection replacement.
clients/dotnet/src/AgentHostProtocol/Hosts/HostId.cs Adds value-equality operators for HostId.
clients/dotnet/src/AgentHostProtocol/Hosts/HostedResourceKey.cs Tightens API surface (percent-escape helper no longer public).
clients/dotnet/src/AgentHostProtocol/Hosts/HostConfig.cs Makes TransportFactory required; adds snapshotting of nested mutable config.
clients/dotnet/src/AgentHostProtocol/Hosts/FileClientIdStore.cs Secure Unix temp-file creation (0600 before write) + native netstandard path + fail-closed behavior.
clients/dotnet/src/AgentHostProtocol/Generated/ActionMetadata.generated.cs New generated action-type and wire-name mapping used by reducers.
clients/dotnet/src/AgentHostProtocol/Errors.cs Updates transport error kind docs to reflect new handshake protocol validation behavior.
clients/dotnet/src/AgentHostProtocol/AhpClient.cs Adds ClientConfig.TimeProvider, time-provider-driven delays/timeouts, protocol negotiation validation, send-queue timeout behavior.
clients/dotnet/src/AgentHostProtocol/AgentHostProtocol.csproj Marks net8.0 assets trimmable/AOT-compatible; adds TimeProvider package for netstandard.
clients/dotnet/src/AgentHostProtocol.Abstractions/Json/UnionConverter.cs Switches union dispatch to metadata-based GetTypeInfo (AOT/trim friendly).
clients/dotnet/src/AgentHostProtocol.Abstractions/Json/StringOrMarkdown.cs Tightens value semantics + null rejection in factories and converter.
clients/dotnet/src/AgentHostProtocol.Abstractions/Json/IAhpSerializer.cs Removes reflection-unsafety annotations now that default path is metadata-based.
clients/dotnet/src/AgentHostProtocol.Abstractions/Json/AhpJsonTypeInfo.cs Helper for typed JsonTypeInfo<T> retrieval from options metadata.
clients/dotnet/src/AgentHostProtocol.Abstractions/Json/AhpJsonMetadata.cs Public surface exposing the generated resolver without exposing the full context API.
clients/dotnet/src/AgentHostProtocol.Abstractions/Generated/State.generated.cs Uses AhpJsonTypeInfo overloads for converters (AOT/trim safe).
clients/dotnet/src/AgentHostProtocol.Abstractions/Generated/Messages.generated.cs Uses metadata-based deserialize/serialize in message shape converter.
clients/dotnet/src/AgentHostProtocol.Abstractions/Generated/JsonSerializerContext.generated.cs New source-gen context listing all protocol wire model types.
clients/dotnet/src/AgentHostProtocol.Abstractions/AgentHostProtocol.Abstractions.csproj Marks net8.0 assets trimmable/AOT-compatible under analyzers.
clients/dotnet/README.md Updates DI usage and documents TimeProvider + trimming/Native AOT guidance.
clients/dotnet/docs/decisions/serialization.md Updates decision record to reflect the now-implemented source-gen metadata approach.
clients/dotnet/AgentHostProtocol.slnx Adds Native AOT smoke project to solution.
.github/workflows/ci.yml Adds Native AOT publish/run step and switches test execution to bounded-time runner invocation.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

  • Files reviewed: 37/41 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread clients/dotnet/src/AgentHostProtocol/AhpClient.cs
@joshmouch

Copy link
Copy Markdown
Contributor

One thing I noticed reading through the generator changes here — there's a discriminant-default bug in generate-csharp.ts that predates this PR and that this PR doesn't touch. Fix is up as #414; flagging it here since you're in that file.

literalValue is computed at :338/:343 and stored at :368, but it's never read — grep -n literalValue scripts/generate-csharp.ts returns only those declaration and write sites. So the generated types emit their discriminant with no initializer:

public ActionType Type { get; init; }

and since RootAgentsChanged is the first member of ActionType, default(ActionType) is root/agentsChanged:

var action = new StateAction(new ChatDraftChangedAction { /* ... */ });
// serializes as {"type":"root/agentsChanged", ...}

UnionConverter.Write serializes by the runtime type precisely so that "every property (including the variant's own discriminator field) is written", so the record's own Type is what reaches the wire. 94 records across Actions, State and Commands.

What makes it .NET-specific isn't the missing initializer, it's that C#'s zero value is a valid member. Rust can't express it — StateAction is #[serde(tag = "type")] over an enum, so the discriminant rides on the variant. Swift and Kotlin are the same shape. Go can, but type ActionType string zero-values to "", which isn't a valid discriminant, so it fails loudly. C# is the only one where the default is well-formed and wrong, so it's the only one that mis-tags silently.

#414 keeps clear of your serializer/AOT work — I checked it applies on top of this PR's head, where the defect is also present.

One thing I left out of #414 because it's your call rather than a patch: the 16 hand-written session actions in SESSION_ACTION_TYPES_CS have wire values that aren't members of ActionType at all (session/turnStarted, session/delta, session/error, …). They deserialize fine — the variant map keys on the wire string — but there's no enum member to serialize back out, so reading one and writing it returns root/agentsChanged. That's a round-trip break rather than a default bug, and fixing it means either adding those members to ActionType or moving those records to a plain string discriminant. Happy to implement whichever you'd prefer.

Worth noting neither is reachable by CI today: zero of the 42 fixtures in types/test-cases/round-trips/ cover any of those 16 wire values.

Comment thread clients/dotnet/src/AgentHostProtocol/Hosts/MultiHostClient.cs Outdated
Comment thread scripts/generate-csharp.ts Outdated
Comment thread clients/dotnet/src/AgentHostProtocol/Hosts/MultiHostClient.cs Outdated
Comment thread clients/dotnet/src/AgentHostProtocol/Hosts/MultiHostClient.cs
Comment thread clients/dotnet/src/AgentHostProtocol/AhpClient.cs
Comment thread clients/dotnet/src/AgentHostProtocol/Hosts/MultiHostClient.cs Outdated
Comment thread clients/dotnet/src/AgentHostProtocol/Hosts/MultiHostClient.cs
Josh (joshmouch) added a commit to joshmouch/agent-host-protocol that referenced this pull request Aug 24, 2026
…mask

The first version of this change shipped without a regression test, on the
reasoning that a two-syscall race is not observable in-process. That was wrong:
the earlier attempt failed because the poller walked 64 named paths per
iteration behind a Task.yield, not because the window is too narrow. Polling
contentsOfDirectory in a tight loop observes the pre-fix leak in 19 of 20 runs.

testClientIdIsNeverObservableAtLoosePermissionsDuringStore watches the
destination directory while a store runs and fails if any file in it is ever
seen carrying group or other bits. Against the previous implementation it fails
naming the leaked temp -- 'h.clientid.sb-... was 0644'. The assertion is
'nothing was ever loose', which the current implementation satisfies by
construction, so it cannot fail spuriously; a 1 MB payload widens the write
enough to make missing a real regression vanishingly unlikely.

Writing it surfaced a defect in the fix itself. open(2)'s mode argument is
masked by the process umask, so under a umask carrying 0o200 the file was
created 0400 -- read-only, where the chmod-after-write shape it replaced had
always produced exactly 0600. Adding fchmod, which is not masked, keeps the
closed window and restores the exact mode. This is what the netstandard2.0 leg
of the .NET fix in microsoft#411 does, which I had cited without matching.

testPersistedModeIsExactlyOwnerOnlyRegardlessOfUmask covers that directly: it
fails with 0400 if the fchmod is removed.
Josh (joshmouch) added a commit to joshmouch/agent-host-protocol that referenced this pull request Aug 24, 2026
AGENTS.md: 'Do not edit CHANGELOG.md files for normal feature/fix PRs' --
changelogs are assembled from docs/.changes/ fragments by the release
maintainer per RELEASING.md. microsoft#411 does this correctly with three fragments;
this PR was editing clients/swift/CHANGELOG.md directly. Replaced with a
fragment, typed 'security' rather than 'fixed'.
Josh (joshmouch) added a commit to joshmouch/agent-host-protocol that referenced this pull request Aug 24, 2026
AGENTS.md: 'Do not edit CHANGELOG.md files for normal feature/fix PRs' --
changelogs are assembled from docs/.changes/ fragments by the release
maintainer per RELEASING.md. microsoft#411 does this correctly; this PR was editing
clients/dotnet/CHANGELOG.md directly.
Harden reconnect commit ordering and teardown, publish reconnect snapshots, make post-ack subscription state reliable, and exercise PingAsync under Native AOT.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 05359f98-1e07-4a16-8d29-7d22c32aa23a
@connor4312
Connor Peet (connor4312) merged commit 50340f0 into main Aug 24, 2026
9 checks passed
@connor4312
Connor Peet (connor4312) deleted the davidfowl-harden-dotnet-client branch August 24, 2026 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants