feat: run Fleet Server discovery without Nmap

[ankitgoswami]

Reviewable diff: +456/-752 across 19 files (excludes generated, test, and story files).

Summary

Fleet Server performs network discovery through the shared TCP scanner, removing the remaining Nmap runtime and Go dependency. Explicit IP lists and ranges continue to invoke plugins directly, including virtual miners without TCP listeners. The server keeps streaming identified devices and retains its broader subnet and address support.

Stack: #1035 (this PR) → #1036 → #1037

This diff targets main, which includes #1034 shared target/scanner semantics and Fleet Node adoption. This layer completes runtime replacement; the existing wire mode name remains until #1037.

Nonterminal operator warnings land in #1036; the final API name and node/list limits land in #1037.

How it works

Server and Fleet Node network scans use the same shared iterator to interleave subnets and choose a random starting address in each, wrapping around once. A broad subnet therefore cannot consume every probe before later subnets get a turn. Retrying can reach different addresses, though overlap remains possible; the ten-minute budget and shared TCP limits are unchanged. The UI keeps previously found miners when rescanning, and the timeout message suggests retrying or narrowing the range. Fleet Nodes preserve overlap deduplication for detected local subnets and include retry guidance in deadline acknowledgments. No scan progress is stored.

Unresolvable IP-list entries are skipped in this layer. PR #1036 introduces their nonterminal warnings together with client support.

Hostname lookups run concurrently within the existing discovery limit, while literal addresses proceed independently. Duplicate inputs are resolved once, and aliases resolving to the same target are probed once. Cancellation stops and waits for the lookup workers.

All three modes share lazy target resolution and a bounded host worker pipeline sized to the request. Network mode feeds only open TCP ports into those workers; explicit lists and ranges feed validated endpoints directly to plugins. Each host serializes discovered-device persistence through the existing collision fallback. The ten-minute scan budget bounds the stream; late plugin calls retain their own buffered result channel and cannot write to a closed client stream.

Diagrams

flowchart LR
  R["Network, list or range request"] --> V["Shared target resolution and port validation"]
  V --> N["Network scan"]
  N --> I["Interleaved subnets with random starting addresses"]
  I --> S["Shared TCP scanner"]
  S --> H["Bounded host identification"]
  V --> E["Explicit IP list or range"]
  E --> H
  H --> D["Existing serialized persistence and collision fallback"]
  D --> O["Deduplicated device stream"]

Loading

Areas of the code involved

Area / package / file	What changed	Why it matters for review
server/internal/domain/pairing	Replace Nmap execution with shared scanning and bounded plugin work	Review stream lifetime, persistence ordering and collision fallback
server/cmd/fleetnode	Use shared scan ordering for explicit network targets and detected local subnets	Preserve target limits, overlap deduplication and partial reports
server/internal/domain/netscan	Add lazy randomized, interleaved address iteration; keep parser predicates private	Review wraparound and bounded memory; shared scanner limits stay unchanged
client/src/protoFleet/features/onboarding/components/Miners	Retain displayed results on rescan	Retries add miners without clearing earlier results
server/internal/domain/netutil	Remove unused range helpers after callers move to netscan	Avoid a second implementation of address iteration
server/go.mod and server/go.sum	Remove github.com/Ullaakut/nmap/v3	The workspace no longer depends on Nmap
Server Dockerfiles and deployment compose profiles	Remove Nmap packages and scanning capabilities	Fleet API drops NET_RAW; simulator and HA networking remain intact

Key technical decisions & trade-offs

• Preserve the private-address restriction from main for network scans, including DNS answers, using shared target validation. Explicit list/range behavior is unchanged.

• Keep direct plugin discovery for explicit endpoints so plugins that do not use TCP remain supported; share the host workers and persistence path across modes.

• Keep known-subnet aggregate discovery uncapped and lazily enumerate broad IPv4 ranges.

• Preserve one serialized persistence path per host while probing candidate ports concurrently.

• Bound noncooperative Discover calls without releasing their global plugin permit before they actually return.

• Keep supported production capability RPCs context-bound through the existing gRPC driver implementation.

Testing & validation

• Focused node, server pairing and shared scanner race tests pass after restacking, including node target limits, interleaving, overlapping subnet deduplication and partial-report timeout guidance; lint passes for all three areas.

• Ordering and pairing race tests pass: subnet fairness, wraparound, complete target coverage, lazy broad-range iteration, cancellation and the timeout message. Across the updated stack, server lint, TypeScript checking and 35 focused client tests pass, including retention of prior results on both network and manual rescan.

• Focused discovery unit/race tests and pairing lint pass, including blocked DNS, saturated lookup workers, cancellation and resolved-target deduplication.

• A regression test uses a closed loopback port and an in-memory discoverer in both explicit modes, with the real scanner left in place. The former all-open test adapter is removed.

• Targeted pairing race tests pass for network and explicit modes, cancellation, partial scan failures, collision fallback and noncooperative plugin calls; shared target tests cover DNS policy.

• Production server build and full server lint pass; HA deployment-profile checks pass.

• The Go workspace module graph contains no Nmap dependency. Database integration tests and real-network qualification remain for CI or a test environment.

Post-deploy monitoring & validation

Release the complete stack with server, nodes and client updated together. On a controlled network, compare expected miners against both server and node discovery, exercise an incomplete scan, and confirm that retained results and its warning reach the client. Pause rollout if expected devices disappear, scans cannot stop within their budgets, or completed reports are lost. No deployment was performed for these draft PRs.

---

Restack validation: affected unit/race suites, server lint, protobuf lint, client type check and 33 focused client tests pass. Database integration tests remain for CI.

[github-actions]

🔐 Codex Security Review

Note: This is an automated security-focused code review generated by Codex.
It should be used as a supplementary check alongside human review.
False positives are possible - use your judgment.

Scope summary

• Reviewed pull request diff only (87bed2f37552c53a6a558c4723c4ed18b641d97f...dfd4440046be643fa021ff9f8b617be7e6b3ce66, exact PR three-dot diff)
• Model: gpt-5.6-sol

💡 Click "edited" above to see previous reviews for this PR.

---

Review Summary

Overall Risk: HIGH

Findings

[HIGH] Automated review incomplete

• Category: Other
• Description: The automated review produced no usable result for 87bed2f37552c53a6a558c4723c4ed18b641d97f...dfd4440046be643fa021ff9f8b617be7e6b3ce66 (workflow run 34875446412; reason: codex-job-timeout, elapsed: unknown, budget: 9 minutes).
• Impact: The pull request has not received complete automated security, correctness, and reliability analysis.
• Recommendation: Require human review before merging. Do not treat this result as approval-free or low risk.

Notes

Human review is required because the bounded automated review was incomplete.

---

_{Generated by Codex Security Review |
Triggered by: @ankitgoswami |
Review workflow run}

[Copilot]

🟡 Changes recommended

An IP-list DNS failure can terminate client consumption and discard slower Fleet Node results.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR replaces Fleet Server Nmap discovery with the shared TCP scanner while preserving direct plugin probing for explicit targets.

Changes:

• Adds bounded, cancellable discovery workers.
• Removes Nmap code, dependencies, packages, and capabilities.
• Adds scanner, cancellation, and explicit-target tests.

File summaries

File	Description
server/internal/domain/pairing/service.go	Integrates shared scanning and updates cancellation.
server/internal/domain/pairing/service_test.go	Uses shared discovery limits.
server/internal/domain/pairing/service_internal_test.go	Removes obsolete Nmap tests.
server/internal/domain/pairing/scan.go	Implements server discovery through netscan.
server/internal/domain/pairing/scan_test.go	Tests scanning, fallback, and cancellation.
server/internal/domain/netutil/iputil.go	Removes superseded IP utilities.
server/internal/domain/netutil/iputil_test.go	Removes superseded utility tests.
server/internal/domain/netscan/targets.go	Makes parser predicates private.
server/go.mod	Removes the Nmap module.
server/go.sum	Removes Nmap checksums.
server/Dockerfile.dev	Removes the development Nmap package.
server/Dockerfile	Removes the runtime Nmap package.
server/docker-compose.base.yaml	Drops NET_RAW.
deployment-files/server/Dockerfile	Removes deployed Nmap installation.
deployment-files/ha/fleet-compose.yaml	Removes HA raw-socket capability.
deployment-files/ha/tests/test-profile.sh	Verifies reduced capabilities.

Review details

• Files reviewed: 15/16 changed files
• Comments generated: 1
• Review effort level: Balanced

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[ankitgoswami]

🤖

An IP-list DNS failure can terminate client consumption and discard slower Fleet Node results.

Addressed: this layer skips unresolvable IP-list entries; #1036 adds nonterminal warnings with client support. Regression coverage checks valid results remain available in both layers.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 63ce08ed72

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a2f61cdcc

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f8aca31c64

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".