Production environment serves two deployments simultaneously from the default hostname; ratio oscillates and never converges

[JakeHarveyy]

Summary

A production environment serves two different deployments simultaneously from the default *.azurestaticapps.net hostname. Which one a request receives appears to be per-request. The share served the current build oscillates rather than converging — 27% → 78% → 40% within minutes, with no deployment occurring in between — and requests are answered from artifact sets over two hours and two deployments old.

Every deployment reports Deployment Complete. ARM reports the environment Ready with a current lastUpdatedOn throughout.

This is on the default hostname, with no custom domain, no CDN, and no enterprise-grade edge in the path.

Standard SKU, East Asia. The hostname is redacted below; I'm happy to provide it privately to anyone at Microsoft who wants to reproduce this directly.

Reproduction

Each deployment stamps /build-id.txt with its GitHub Actions run id, and a route rule makes that one path anonymous with Cache-Control: no-store. Every request below uses a unique query string, so no cache anywhere can return a stored copy.

U=https://<app>.azurestaticapps.net/build-id.txt
for i in $(seq 1 40); do
  rm -f /tmp/b
  curl -s -o /tmp/b --max-time 10 -w '%{http_code} ' "$U?cb=$i-$RANDOM"
  tr -d '\r\n' < /tmp/b; echo
done | sort | uniq -c | sort -rn

Expected: 40 identical lines. Actual, six minutes after a successful deployment:

21  200 30511573524-1     <- current build
15  200 30510492511-1     <- two deployments and 30 minutes old
 3  000                   <- no response
 1  503

Two builds, named, in a single 20-request sample

Taken by the deployment workflow itself immediately after reporting success:

11  200 30503495726-1     <- the deployment from 18 minutes earlier
 9  200 30504402496-1     <- the deployment that had just completed

One URL, one moment, two different bodies, both 200.

It oscillates rather than converging

Sampling the same URL after a deployment at 03:11 UTC on 2026-07-30. No deployment occurred between these samples.

UTC	current build	previous build	transport errors
03:13:44	17/20	0	3× 500
03:20:36	11/40 (27%)	29/40	—
03:22:10	31/40 (78%)	9/40	—
03:52:51	16/40 (40%)	24/40	—

The share served the new build fell to zero and recovered, twice, with nothing deployed.

staticwebapp.config.json is affected independently of content

After a different deployment, the stale responses were not old content but old routing rules. A route granting anonymous access to /build-id.txt was present in the deployed config; a share of replicas did not have it and returned 302 -> /login via the 401 responseOverride instead:

Minutes after deployment	200	302 (previous config)
7	29/40	11/40 (27%)
13	24/40	16/40 (40%)
25	18/20	2/20 (10%)

So the artifact set and the configuration can land on different replicas independently.

Platform-generated errors while ARM reports Ready

• 500 on 3 of 20 requests, served from the Azure Static Web Apps branded error page
• 502 — "Web server received an invalid response while acting as a gateway or proxy server"
• Connections dropped with no response at all, up to 4 in 40

User-visible impact

Mixing is per-request, not per-session, so a single page load fetches its HTML from one replica and its hashed JS chunks from another. When those come from different builds the chunk filenames don't match and the page 404s.

For an app with rolesSource and route-level allowedRoles, this presents as fresh sign-ins failing intermittently: authentication completes (Entra logs errorCode 0), /.auth/complete completes, and then the post-login landing page 404s. Existing session cookies keep working, so it goes unnoticed for hours and looks random. Two users hit this before we understood what we were looking at.

Ruled out by direct measurement

Application code	Rolled production back to the last known-healthy tree — stayed broken. The suspect commit signs in correctly on three separate staging environments.
Any cache — browser, CDN, or platform edge	Unique query string per request; 15 of 40 still returned a superseded build. Response carries cache-control: no-store.
Custom domain / third-party CDN	Not in the path. All measurements are against the default hostname.
Deployment not landing, or wrong app_location / output_location	The new build is served, by 21 of those 40 requests. A path fault would fail uniformly rather than split.
Managed functions	/build-id.txt is a static file; no function participates in serving it. The stale-config case is a route rule, also not function-served.
Deploy cadence	Two weeks in April sustained ~2 production deploys/day with no incidents — a higher rate than the incident window.
App settings / app registration	Byte-identical to the staging environments that work. No resource changes in the activity log for 7 days.
Leftover preview environments	Deleted all of them (az staticwebapp environment list → default only) and re-measured: still split 25/14.
The resource itself	The same fault occurred on a different Static Web App, in a different region and resource group, days earlier. We migrated specifically to escape it.

Mitigations attempted, none effective

• Redeploy — the product group's mitigation on SWA - consistent response payload #1281. Two consecutive deployments both failed verification; still split 21 minutes later.
• Deploy the identical artifact twice per run — on the theory that if current and previous are byte-identical, a lagging replica serves the same bytes. Replicas turn out to be stuck on artifact sets far further back than one deployment, so this bought nothing. Reverted.
• Cache-Control: no-store — already set and returned. The reporter on SWA - consistent response payload #1281 had globalHeaders no-store and still received a previous deployment's index.html.
• Deleting preview environments — suggested on a related Q&A thread. No effect.
• Not attempted, on the strength of Custom domains intermittently return platform 404 while default hostnames are 100% reliable (West Europe) #1766: delete and recreate. That reporter saw the fault reproduce on brand-new resources and concluded the stale state is keyed to the domain rather than the resource. MS Q&A 5910175 additionally documents resource-name reuse causing traffic to split between a populated and an empty environment.

Possible correlation with the deployment client

StaticSitesClient self-updates and its version is printed in every run log. Ours:

Deployment	Client	Outcome
2026-07-21	20260714.240.1	last healthy deployment
2026-07-28	20260720.247.2	first broken deployment — never recovered
2026-07-29 → 30	20260720.247.2	broken
2026-07-30	20260726.254.1	broken

Nothing was deployed between 21 and 28 July and the site served normally throughout. Rolling the code back on 28 July did not help — the deployment client is the one variable a rollback cannot undo.

#1778 reports a different symptom (Next.js function provisioning failing outright) but names the same version pair and the same last-good date of 2026-07-21: "The only delta between the last success and the failures is the deployment client version printed in the action logs."

This is correlation across two organisations, not a claim of causation. Flagging it because it is checkable on your side and we cannot pin the client version to test it ourselves.

Prior art

#1281 (September 2023) is the same fault, and Microsoft confirmed it privately at the time:

"if I spam the reload button of a browser on our site 10 times within 10 seconds it could return 5 404 errors and 5 200 successes … it's also clear that the SWA could randomly return a different index.html file which most likely belonged to a previous deployment of the SWA"

"I got confirmation from Azure that the fault is indeed on their end."

Eight organisations in that thread. Resolved server-side within about five days. No status-page notification was issued.

#1563 (open since 2024-10-21) describes the same serving behaviour — "as if it's being load balanced, and you don't always get the same result" — and has had no maintainer response in 21 months.

Documented behaviour (MS Q&A 625770, Microsoft moderator, 2021-11-22):

"Static Web App deployments are atomic and we won't shift to serving the new content until it is readily available for all edge nodes that can serve it. There should never be a time where a user is getting files from different commits when requesting the files directly."

That statement does not appear in the product documentation, and the observed behaviour contradicts it.

Questions

1. Why does one environment serve two artifact sets, and two configurations, after Deployment Complete?
2. What bounds replica lag? We observe artifact sets more than two hours and two deployments old.
3. Why does the share of requests served the new build fall over time rather than rise to 100%?
4. What accounts for the 500, 502 and dropped connections while ARM reports Ready?
5. Did anything in the content-distribution path change between StaticSitesClient 20260714.240.1 and 20260720.247.2? Is there a supported way to pin the deployment client version?
6. Was there an unpublished Static Web Apps incident between 2026-07-20 and 2026-07-30? Eleven issues were filed in this repository in that window from multiple organisations, and the Azure status history shows nothing for the service.
7. What forces convergence, given that redeployment does not, and that Custom domains intermittently return platform 404 while default hostnames are 100% reliable (West Europe) #1766 reports recreation reproducing the fault?

Environment

• Azure Static Web Apps, Standard SKU, East Asia
• Default *.azurestaticapps.net hostname; no custom domain, no CDN, no enterprise-grade edge, no private endpoint in the measurement path
• Azure/static-web-apps-deploy@v1, GitHub Actions, deployment token auth, skip_app_build: true
• VitePress static output plus one managed function used as rolesSource
• staticwebapp.config.json with allowedRoles route rules and responseOverrides
• Reproducible on demand as of 2026-07-31