Skip to content

Fix header union member invalidation - #1478

Merged
jafingerhut merged 2 commits into
p4lang:mainfrom
fruffy:header-union-invalidation
Nov 9, 2026
Merged

jafingerhut merged 2 commits into
p4lang:mainfrom
fruffy:header-union-invalidation

Conversation

@fruffy

@fruffy fruffy commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Invalidate the entire header union when setInvalid() is called on any member, including an already-invalid member, as required by the P4 specification.

  • Propagate invalidation to the active member and its hidden validity field.
  • Update remove_header in simple_switch, psa_switch, and pna_nic to remove the active member and account for its packet length.
  • Copy only the active member in assign_union, preserving valid copies and self-assignment under the corrected invalidation semantics.
  • Enable field arithmetic for union arguments so copies retain extracted field values.
  • Add regression coverage for member invalidation, union copies, and packet-length accounting.

Companion to p4lang/p4c#4853; addresses the runtime side of p4lang/p4c#4448.

Invalidate the active union member even when setInvalid is called on an already-invalid sibling. Update target remove_header primitives to account for the active member's packet length, and copy only the active member in assign_union to preserve union assignment semantics.

Add coverage for validity bits, repeated invalidation, union assignment and self-assignment, and simple_switch packet-length accounting.

Companion to p4lang/p4c#4853; addresses the runtime side of p4lang/p4c#4448.

Signed-off-by: fruffy <fruffy@nyu.edu>
Union assignments use assign_header, which reads the arithmetic value of each member field. Enable these values for union action and expression arguments, including members of union stacks, just as for header arguments.

Add a JSON-loading regression that checks nonzero extracted fields referenced only through union arguments.

Signed-off-by: fruffy <fruffy@nyu.edu>
fruffy added a commit to p4lang/p4c that referenced this pull request Oct 31, 2026
Preserve whole-union assignments in BMv2 midends and encode local union references correctly. Keep CopyStructures constructor compatibility and tuple-copy behavior while fixing the stale configuration member.

Pin the runtime fixes from p4lang/behavioral-model#1478 and remove the union-invalidation expected failures. Add packet coverage for local union copies, both active members, and an invalid source.

Validated with BMv2 runtime suites, compiler and Testgen checks, eight Testgen STF cases, eight serial PTF cases, and the deterministic copy regression.

Signed-off-by: fruffy <fruffy@nyu.edu>
@fruffy
fruffy requested a review from jafingerhut November 2, 2026 02:32

@jafingerhut jafingerhut left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I checked the changes carefully. I did not also carefully check for whether there are other places in the code that ought to be changed, but I expect if any remain, we would find them eventually in testing.

@jafingerhut
jafingerhut merged commit 5c72250 into p4lang:main Nov 9, 2026
11 checks passed
@kfcripps

Copy link
Copy Markdown

Claude says that this PR broke the container-image job on P4C's main branch. I haven't verified its claims to be true, but overall the response seems reasonable.

From https://github.com/p4lang/p4c/actions/runs/42835177789/job/129895362272?pr=5876:

CI p4c p4testgen failures

User: @kfcripps

@kfcripps

Explain why this job failed.

Copilot

The "Run p4c tests (Docker image)" step failed. CTest exited with code 8 because 4 P4Testgen tests failed:

4118 - testgen-p4c-bmv2-ptf/issue1897-bmv2.p4 (Failed)
4344 - testgen-p4c-bmv2-ptf/union4-bmv2.p4 (Failed)
4650 - testgen-p4c-bmv2-stf/issue1897-bmv2.p4 (Failed)
4876 - testgen-p4c-bmv2-stf/union4-bmv2.p4 (Failed)

Both issue1897-bmv2.p4 and union4-bmv2.p4 failed under both the PTF and STF backends. Since the same two programs fail consistently across two independent backends, this looks like a code/test problem (P4Testgen generating invalid packets/traces for these programs), not a flaky or infrastructure issue.

The tail doesn't show the actual assertion/diff. Want me to pull the detailed failure output for one of these tests to pinpoint the root cause and work out a fix?

@kfcripps

This job is broken on main branch, so this PR did not cause it to fail. Please find out which PR broke the job and find the root cause.

Copilot

I'll investigate. Let me first pull the detailed failure output for one test, and look at the recent history on main for the failing tests.

CTest output doesn't show per-test detail. Both failures are in STF and PTF backends (which actually run the generated packets through bmv2), while metadata/protobuf-ir backends pass. Let me look at recent main commits touching P4Testgen's STF/PTF code.

Both failing programs involve header unions, and only STF/PTF (which run packets through bmv2) fail. Let me find when the container-image job first started failing on main.

Key finding: the same commit 2745eece7 passed at 16:17 UTC on 11-09 but failed at 20:15 UTC and every run since. The p4c source didn't change between those runs, so no p4c PR caused this. The container-image job rebuilds a Docker image fresh each run, pulling in external dependencies. Let me confirm what the test image installs (likely bmv2/behavioral-model).

The base image is p4lang/behavioral-model:latest, pulled fresh every build. A new bmv2 image published between 16:17 and 20:15 on 11-09 is the likely trigger. Let me check recent behavioral-model commits.

Found the root cause. Let me confirm what bmv2 PR #1478 changed.

Confirmed. Here's the diagnosis.

Root cause: a behavioral-model (bmv2) change, not a p4c PR

No p4c PR broke this. The container-image job's own history proves it: the identical p4c commit 2745eece7 passed at 16:17 UTC on 2026-11-09 and then failed at 20:15 UTC and on every run since — with no change to the p4c source in between.

The job rebuilds a Docker image from BASE_IMAGE=p4lang/behavioral-model:latest (Dockerfile#L1) on every run, so it silently picked up a new bmv2 build.

The breaking change is bmv2 PR #1478 "Fix header union member invalidation", merged 2026-11-09T17:12:20Z — right in the window between the last pass and first failure. It changes simple_switch/psa/pna runtime semantics so that setInvalid() on a union member now invalidates the whole header union (per the P4 spec).

That exactly explains the 4 failures, all header-union programs, and only in the STF/PTF backends (which actually execute packets through simple_switch), while the metadata/protobuf-ir backends — which don't run bmv2 — still pass:

testgen-p4c-bmv2-ptf/issue1897-bmv2.p4
testgen-p4c-bmv2-ptf/union4-bmv2.p4
testgen-p4c-bmv2-stf/issue1897-bmv2.p4
testgen-p4c-bmv2-stf/union4-bmv2.p4

P4Testgen generates expected outputs modeling the old bmv2 union behavior, which no longer match the new runtime output → CTest exits with code 8.

Likely fix

This is the runtime companion to p4c PR #4853. p4c's symbolic model of header-union invalidation must be updated to match bmv2's new spec-compliant semantics (and the two STF/PTF expected traces regenerated). Essentially p4c needs the matching #4853-side change merged/landed.

Want me to pull the detailed STF diff for one test and draft the p4c-side fix (update the union invalidation model + regenerate expectations)?

@kfcripps

Copy link
Copy Markdown

To unblock merging of P4C PRs, perhaps we should revert this PR, and then re-merge this PR after p4lang/p4c#4853 is approved and ready to be merged, so that p4lang/p4c#4853 can be merged immediately after, therefore minimizing the amount of time that the container-image job will be broken in P4C's main branch?

kfcripps added a commit to kfcripps/behavioral-model that referenced this pull request Nov 10, 2026
This reverts commit 5c72250.

p4lang#1478 changed bmv2's header union invalidation semantics, which broke
four P4Testgen STF/PTF tests in p4c's container-image CI job (built on
p4lang/behavioral-model:latest). Revert it until the companion p4c
change, p4lang/p4c#4853, is ready to land alongside it.

Signed-off-by: Kyle Cripps <kyle@pensando.io>
@kfcripps

kfcripps commented Nov 10, 2026 •

Copy link
Copy Markdown

To unblock merging of P4C PRs, perhaps we should revert this PR, and then re-merge this PR after p4lang/p4c#4853 is approved and ready to be merged, so that p4lang/p4c#4853 can be merged immediately after, therefore minimizing the amount of time that the container-image job will be broken in P4C's main branch?

@fruffy @jafingerhut If this is how you wish to proceed, then feel free to merge #1485 that I've just opened. Otherwise, let me know and I'll close it and let you resolve this in a different way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants