Skip to content

fix: chunk large Docker container lists to prevent WebSocket message loss#304

Open
jaydeep-pipaliya wants to merge 549 commits into
fosrl:mainfrom
jaydeep-pipaliya:fix/chunk-docker-container-messages
Open

fix: chunk large Docker container lists to prevent WebSocket message loss#304
jaydeep-pipaliya wants to merge 549 commits into
fosrl:mainfrom
jaydeep-pipaliya:fix/chunk-docker-container-messages

Conversation

@jaydeep-pipaliya

@jaydeep-pipaliya jaydeep-pipaliya commented Apr 9, 2026

Copy link
Copy Markdown

What does this PR do?

Companion PR for fosrl/pangolin#2117 — Docker Container View not displaying when >20 containers are running.

Problem

gorilla/websocket.WriteJSON serializes the full container list into a single WebSocket text frame. With 55+ containers (each 1-5KB of JSON metadata), the resulting frame can be 55-275KB. Intermediary proxies (Traefik, nginx, Cloudflare tunnels) can silently drop or truncate frames this large, causing the pangolin server to never receive the container data.

Solution

Added sendContainerList() that automatically chunks large container lists:

// ≤15 containers: single message (backward compatible, zero behavior change)
{"containers": [...]}

// >15 containers: chunked with batch metadata
{"containers": [...], "chunkIndex": 0, "totalChunks": 4, "batchId": "a1b2c3d4"}
{"containers": [...], "chunkIndex": 1, "totalChunks": 4, "batchId": "a1b2c3d4"}
{"containers": [...], "chunkIndex": 2, "totalChunks": 4, "batchId": "a1b2c3d4"}
{"containers": [...], "chunkIndex": 3, "totalChunks": 4, "batchId": "a1b2c3d4"}

Why batchId?

Two concurrent container sends can happen when a manual fetch request and a Docker event fire at the same time. Without batchId, their chunks would interleave and corrupt the accumulated data on the server. Each batch gets a unique ID (reusing the existing generateChainId helper) so the server can track and supersede batches correctly.

Why chunk size of 15?

Each container with full metadata (labels, ports, networks) serializes to 1-5KB. 15 containers ≈ 15-75KB per WebSocket frame — safely under common proxy defaults (Traefik default buffer: 64KB, nginx: 64KB). The threshold was chosen to balance message count vs. frame size.

Changes

main.go:

  • Added sendContainerList() — chunks large lists, passes small lists through unchanged
  • Updated both call sites: initial fetch handler + Docker event monitor callback
  • Reuses existing generateChainId() for batch IDs — no new dependencies

Companion PR

Pangolin server: fosrl/pangolin#2817

  • Chunk reassembly with batchId tracking, input validation, typed accumulator, 120s TTL on partial chunks

Testing

  • go build passes
  • ≤15 containers: identical to current behavior (single message, no metadata)
  • >15 containers: splits into chunks of 15 with chunkIndex/totalChunks/batchId
  • Concurrent sends: each gets unique batchId, server handles superseding

oschwartz10612 and others added 30 commits December 6, 2025 11:59
…ker/metadata-action-5.9.0

Bump docker/metadata-action from 5.8.0 to 5.9.0
…ker/setup-qemu-action-3.7.0

Bump docker/setup-qemu-action from 3.6.0 to 3.7.0
…hub/codeql-action-4.31.5

Bump github/codeql-action from 4.31.0 to 4.31.5
…ions/setup-go-6.1.0

Bump actions/setup-go from 6.0.0 to 6.1.0
…es-60be0b6e22

Bump alpine from 3.22 to 3.23 in the minor-updates group
Add robust client connectivity support
Fix Isssue 194 failed go test before 1.7.0-rc1
fix(nix): use new version number, update deps hash
dependabot Bot and others added 26 commits April 3, 2026 12:33
Bumps [actions/stale](https://github.com/actions/stale) from 10.1.1 to 10.2.0.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](actions/stale@9971854...b5d41d4)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-version: 10.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3.12.0 to 4.0.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](docker/setup-buildx-action@8d2750c...4d04d5d)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: 4.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
…asecurity/trivy-action-0.35.0

chore(deps): bump aquasecurity/trivy-action from 0.34.2 to 0.35.0
…ker/setup-buildx-action-4.0.0

chore(deps): bump docker/setup-buildx-action from 3.12.0 to 4.0.0
…ions/stale-10.2.0

chore(deps): bump actions/stale from 10.1.1 to 10.2.0
…tch-updates-a06038febc

chore(deps): bump github.com/gaissmai/bart from 0.26.0 to 0.26.1 in the prod-patch-updates group across 1 directory
Bumps [actions/attest-build-provenance](https://github.com/actions/attest-build-provenance) from 3.2.0 to 4.1.0.
- [Release notes](https://github.com/actions/attest-build-provenance/releases)
- [Changelog](https://github.com/actions/attest-build-provenance/blob/main/RELEASE.md)
- [Commits](actions/attest-build-provenance@96278af...a2bbfa2)

---
updated-dependencies:
- dependency-name: actions/attest-build-provenance
  dependency-version: 4.1.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [sigstore/cosign-installer](https://github.com/sigstore/cosign-installer) from 4.0.0 to 4.1.1.
- [Release notes](https://github.com/sigstore/cosign-installer/releases)
- [Commits](sigstore/cosign-installer@faadad0...cad07c2)

---
updated-dependencies:
- dependency-name: sigstore/cosign-installer
  dependency-version: 4.1.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [docker/login-action](https://github.com/docker/login-action) from 4.0.0 to 4.1.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](docker/login-action@b45d80f...4907a6d)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: 4.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 2.4.2 to 2.6.1.
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](softprops/action-gh-release@5be0e66...153bb8e)

---
updated-dependencies:
- dependency-name: softprops/action-gh-release
  dependency-version: 2.6.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 3.7.0 to 4.0.0.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](docker/setup-qemu-action@c7c5346...ce36039)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-version: 4.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps the prod-minor-updates group with 13 updates:

| Package | From | To |
| --- | --- | --- |
| [go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp](https://github.com/open-telemetry/opentelemetry-go-contrib) | `0.66.0` | `0.67.0` |
| [go.opentelemetry.io/contrib/instrumentation/runtime](https://github.com/open-telemetry/opentelemetry-go-contrib) | `0.66.0` | `0.67.0` |
| [go.opentelemetry.io/otel](https://github.com/open-telemetry/opentelemetry-go) | `1.41.0` | `1.42.0` |
| [go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc](https://github.com/open-telemetry/opentelemetry-go) | `1.41.0` | `1.43.0` |
| [go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc](https://github.com/open-telemetry/opentelemetry-go) | `1.41.0` | `1.43.0` |
| [go.opentelemetry.io/otel/exporters/prometheus](https://github.com/open-telemetry/opentelemetry-go) | `0.63.0` | `0.65.0` |
| [go.opentelemetry.io/otel/metric](https://github.com/open-telemetry/opentelemetry-go) | `1.41.0` | `1.43.0` |
| [go.opentelemetry.io/otel/sdk](https://github.com/open-telemetry/opentelemetry-go) | `1.41.0` | `1.43.0` |
| [go.opentelemetry.io/otel/sdk/metric](https://github.com/open-telemetry/opentelemetry-go) | `1.41.0` | `1.43.0` |
| [golang.org/x/crypto](https://github.com/golang/crypto) | `0.48.0` | `0.49.0` |
| [golang.org/x/net](https://github.com/golang/net) | `0.51.0` | `0.52.0` |
| [golang.org/x/sys](https://github.com/golang/sys) | `0.41.0` | `0.42.0` |
| [google.golang.org/grpc](https://github.com/grpc/grpc-go) | `1.79.3` | `1.80.0` |


Updates `go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp` from 0.66.0 to 0.67.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md)
- [Commits](open-telemetry/opentelemetry-go-contrib@zpages/v0.66.0...zpages/v0.67.0)

Updates `go.opentelemetry.io/contrib/instrumentation/runtime` from 0.66.0 to 0.67.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md)
- [Commits](open-telemetry/opentelemetry-go-contrib@zpages/v0.66.0...zpages/v0.67.0)

Updates `go.opentelemetry.io/otel` from 1.41.0 to 1.42.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](open-telemetry/opentelemetry-go@v1.41.0...v1.42.0)

Updates `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc` from 1.41.0 to 1.43.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](open-telemetry/opentelemetry-go@v1.41.0...v1.43.0)

Updates `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc` from 1.41.0 to 1.43.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](open-telemetry/opentelemetry-go@v1.41.0...v1.43.0)

Updates `go.opentelemetry.io/otel/exporters/prometheus` from 0.63.0 to 0.65.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](open-telemetry/opentelemetry-go@exporters/prometheus/v0.63.0...exporters/prometheus/v0.65.0)

Updates `go.opentelemetry.io/otel/metric` from 1.41.0 to 1.43.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](open-telemetry/opentelemetry-go@v1.41.0...v1.43.0)

Updates `go.opentelemetry.io/otel/sdk` from 1.41.0 to 1.43.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](open-telemetry/opentelemetry-go@v1.41.0...v1.43.0)

Updates `go.opentelemetry.io/otel/sdk/metric` from 1.41.0 to 1.43.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](open-telemetry/opentelemetry-go@v1.41.0...v1.43.0)

Updates `golang.org/x/crypto` from 0.48.0 to 0.49.0
- [Commits](golang/crypto@v0.48.0...v0.49.0)

Updates `golang.org/x/net` from 0.51.0 to 0.52.0
- [Commits](golang/net@v0.51.0...v0.52.0)

Updates `golang.org/x/sys` from 0.41.0 to 0.42.0
- [Commits](golang/sys@v0.41.0...v0.42.0)

Updates `google.golang.org/grpc` from 1.79.3 to 1.80.0
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](grpc/grpc-go@v1.79.3...v1.80.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
  dependency-version: 0.67.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
- dependency-name: go.opentelemetry.io/contrib/instrumentation/runtime
  dependency-version: 0.67.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
- dependency-name: go.opentelemetry.io/otel
  dependency-version: 1.42.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
- dependency-name: go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc
  dependency-version: 1.43.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
- dependency-name: go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
  dependency-version: 1.43.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
- dependency-name: go.opentelemetry.io/otel/exporters/prometheus
  dependency-version: 0.65.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
- dependency-name: go.opentelemetry.io/otel/metric
  dependency-version: 1.43.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
- dependency-name: go.opentelemetry.io/otel/sdk
  dependency-version: 1.43.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
- dependency-name: go.opentelemetry.io/otel/sdk/metric
  dependency-version: 1.43.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
- dependency-name: golang.org/x/crypto
  dependency-version: 0.49.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
- dependency-name: golang.org/x/net
  dependency-version: 0.52.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
- dependency-name: golang.org/x/sys
  dependency-version: 0.42.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
- dependency-name: google.golang.org/grpc
  dependency-version: 1.80.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
…nor-updates-497a73c3c2

chore(deps): bump the prod-minor-updates group with 13 updates
…ker/setup-qemu-action-4.0.0

chore(deps): bump docker/setup-qemu-action from 3.7.0 to 4.0.0
…tprops/action-gh-release-2.6.1

chore(deps): bump softprops/action-gh-release from 2.4.2 to 2.6.1
…ker/login-action-4.1.0

chore(deps): bump docker/login-action from 4.0.0 to 4.1.0
…store/cosign-installer-4.1.1

chore(deps): bump sigstore/cosign-installer from 4.0.0 to 4.1.1
…ions/attest-build-provenance-4.1.0

chore(deps): bump actions/attest-build-provenance from 3.2.0 to 4.1.0
Bumps the prod-minor-updates group with 2 updates: [go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp](https://github.com/open-telemetry/opentelemetry-go-contrib) and [go.opentelemetry.io/contrib/instrumentation/runtime](https://github.com/open-telemetry/opentelemetry-go-contrib).


Updates `go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp` from 0.67.0 to 0.68.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md)
- [Commits](open-telemetry/opentelemetry-go-contrib@zpages/v0.67.0...zpages/v0.68.0)

Updates `go.opentelemetry.io/contrib/instrumentation/runtime` from 0.67.0 to 0.68.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md)
- [Commits](open-telemetry/opentelemetry-go-contrib@zpages/v0.67.0...zpages/v0.68.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
  dependency-version: 0.68.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
- dependency-name: go.opentelemetry.io/contrib/instrumentation/runtime
  dependency-version: 0.68.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
…nor-updates-7fd0df0afe

chore(deps): bump the prod-minor-updates group with 2 updates
…loss

When more than ~20 Docker containers are running, the WebSocket message
containing the full container list can be too large and get dropped by
intermediary proxies (e.g., Traefik), causing the Docker Container View
in Pangolin to show nothing.

Split container lists larger than 15 items into multiple chunked
messages with chunkIndex/totalChunks metadata. The Pangolin server
reassembles chunks before processing.

Small lists (<=15) are sent in a single message for backward
compatibility with older Pangolin versions.

Fixes fosrl/pangolin#2117
Each chunked batch now includes a unique batchId (reusing generateChainId)
so the server can distinguish interleaved sends — e.g., a manual fetch
and a Docker event firing at the same time.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants