Skip to content

GC/2.5a from mweppc (kss alias rule) for PAL/USA; pppRand vector randomizers match in all regions - #17827

Draft
frioux wants to merge 2 commits into
zcanann:stagingfrom
frioux:pal-compiler-kss
Draft

frioux wants to merge 2 commits into
zcanann:stagingfrom
frioux:pal-compiler-kss

Conversation

@frioux

@frioux frioux commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Update: the compiler change moved from doldecomp/mweppc#3 (closed) to PrimeDecomp/mwcc-tools#3, the same change in mwcc-tools' gc2/. This PR still installs the compiler from an mweppc release (tools/download_mweppc.py); that part needs reworking once mwcc-tools decides how the compiler is distributed. References to mweppc#3 below are historical.

Draft: depends on doldecomp/mweppc#3; switch the pinned URL/SHA when that is merged and released. Until then the build stops at DOWNLOAD build/compilers/GC/2.5a/mwcceppc.exe with a message naming the pending value, so this PR cannot build or merge by accident.

What changed

  • GC/2.5a now comes from mweppc instead of a byte patch: it is mweppc's GC_2_5a, GC 2.5 built from the decompiled compiler source with one extra rule (proposed upstream in Add GC 2.5a: 2.5 with FFCC's store alias rule doldecomp/mweppc#3, modelled on mweppc's BFBB variant GC_2_0p2). tools/download_mweppc.py (new) installs mwcceppc.exe and lmgr326b.dll from the 2.5a/ directory of mweppc's release archive into <compilers>/GC/2.5a/, checking the archive's pinned SHA-256 first. It runs as a ninja pre-compile step (DOWNLOAD), also into a --compilers directory; an unchanged compiler is not rewritten (restat). tools/patch_compiler_pal.py and its rule are gone.
  • The rule, kss: in Alias.c operand_alias, a store whose base register has several reaching definitions, some with an alias and some without, keeps the alias of the ones that have it when that alias is a single static object (whole or part, not a local). Stock 2.5 makes every such access worst_case. Loads and stores into locals are unchanged.
  • configure.py builds the PAL and USA Game library with GC/2.5a (Japan keeps GC/2.0p1k).
  • Source: the 12 pppRand{,Up,Down}{CV,HCV,IV,FV} functions read *valuePtr per channel instead of a cached scale/x local.
  • docs/compiler_baseline.md: GC/2.5a section (now describes the mweppc variant, links mweppc#3), evidence and controls.
  • tools/project.py: GC/2.5a → mwcc_247_105 for decomp.me scratches; a compiler produced by a pre-compile step no longer has to exist at configure time with --compilers.

Evidence

report.json, exact functions, vs staging (unchanged from this PR's first version):

region before → after changed functions
PAL +9 / -0 pppRand, Up, Down × CV/HCV/IV: 99.47–99.60 → 100
USA +9 / -0 same nine
JP +12 / -0 the nine + pppRandFV/UpFV/DownFV 99.61 → 100

No partial score is lower in any region; the 12 units link (MatchingFor all three regions).

Measured over all 269 PAL / 268 USA game units with mweppc's GC_2_5a (scored like report.json):

  • Stock 2.5/2.6/2.7 are byte-identical on every PAL/USA game unit; 2.0 only loses. No stock compiler gives the retail pppRand shape.
  • Staging source: GC/2.5a objects byte-identical to stock GC/2.5 (269/269 PAL, 268/268 USA). Per-channel source: stock 2.5 PAL 3063 / USA 3058, GC/2.5a 3075 / 3070.
  • Controls: the rule on every access (Japan's ksa): PAL -18 on staging source, -15 and no gain on the per-channel source; on every store: -1 (CMenuPcs::loadData, a store into a local array); stores to statics only (kss): +0/-0.

How validated

mweppc has no release with GC_2_5a yet, so validation used a locally built one (mweppc branch gc-2.5a, ninja build/GC_2_5a/clang/mwcceppc.exe, LLVM 23 on macOS), packed into an archive with the release layout (2.5a/mwcceppc.exe, 2.5a/lmgr326b.dll) and pinned in a local, uncommitted RELEASE = ("file://…", "<sha256>"), so the real download, SHA-256 check and install path ran:

  • ninja all_source progress build/<V>/report.json build/<V>/ok for GCCP01, GCCE01 and GCCJGC: all pass, three main.dol: OK.
  • report.json identical to this PR's previous (byte-patched GC/2.5a) results in all three regions: +0/-0 exact, no partial moves.
  • Rebuilt PAL and USA with the byte-patched GC/2.5a in place of mweppc's: all 490 objects per region byte-identical.
  • With RELEASE = None (as committed) the step fails with the explanation; a wrong SHA-256 stops before anything is written.
  • The clang-built mwcceppc.exe is a Windows PE and runs through the pinned wibo 1.0.3 like the bundled compilers.

What to update when mweppc releases GC 2.5a

One line, tools/download_mweppc.py:

RELEASE = None

becomes the release archive's URL and its SHA-256 (GitHub shows it as the asset's digest), e.g.

RELEASE = ("https://github.com/doldecomp/mweppc/releases/download/<tag>/mweppc-release.zip", "<sha256>")

If that release names the directory something other than 2.5a/ (or the version is renamed in review), MEMBER_DIR beside it changes too. Then rebuild all three regions and mark this ready.

Why plausible

The retail PAL code loads *valuePtr once and keeps it in a register across the per-channel stores while the target loads stay ordered after each store; stock 2.5 can only produce the single load from a cached local, which numbers the FPR before the int-to-float constant (the long-standing f2/f3 swap, ~60 source variants and 16 compilers tried in #16539). The per-channel source is the same source the Japanese retail binary was built from (it matches JP with the existing GC/2.0p1k), so all three regions now share one natural source. The rule is a narrower sibling of the Japanese ksa correction, and now lives as readable source in mweppc instead of an x86 patch.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QmcpERTQ1WaRZ73F12eUis

…h in all regions

Add tools/patch_compiler_pal.py, a SHA-guarded ninja pre-compile step that
derives GC/2.5a from the bundled stock GC/2.5, and build the PAL and USA
Game library with it. It adds one alias-analysis correction, kss: in
operand_alias, a store whose base register has several reaching
definitions, some with an alias and some without, keeps the alias of the
ones that have it when that alias is a single static object. Stock GC 2.5
makes every such access worst_case. The patch is a new .ffpatch section
with three position-independent stubs and three REL32 redirects inside
operand_alias; no existing section moves and no relocated byte is touched.

The clause was found by rebuilding every PAL/USA game unit with
source-built GC 2.5 variants (mweppc) and ported to the stock binary; the
port and the source build give byte-identical objects on all 269 PAL units.
On the unchanged staging source GC/2.5a is byte-identical to stock GC/2.5
on every PAL unit and changes no USA function.

With it, the natural per-channel source of the pppRand vector randomizers
(`target[i] += rand*(in->delta[i], *valuePtr)`, no cached `scale` local)
matches. That is also the Japanese retail source: with the unchanged
GC/2.0p1k it matches the nine CV/HCV/IV functions and the three FV
functions whose PAL-tuned locals had left Japan at 99.6.

report.json, exact functions (staging bfe75c9 baselines, plus the two
functions staging gained since):
  PAL  +9 / -0  (pppRand{,Up,Down}{CV,HCV,IV} 99.47-99.60 -> 100)
  USA  +9 / -0  (same nine)
  JP  +12 / -0  (the nine plus pppRand{,Up,Down}FV 99.61 -> 100)
No partial score is lower in any region. The twelve units now link in all
three regions; all three DOL checksums pass. Controls and per-clause
evidence are in docs/compiler_baseline.md.

tools/project.py: map GC/2.5a to decomp.me's mwcc_247_105 for scratches,
and do not require a compiler produced by a pre-compile step to exist at
configure time (ninja derives it before any compile).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QmcpERTQ1WaRZ73F12eUis
GC/2.5a is now doldecomp/mweppc's GC_2_5a (GC 2.5 built from the
decompiled source with the kss rule, doldecomp/mweppc#3) rather than a
byte patch of the bundled GC/2.5. tools/download_mweppc.py installs it
from mweppc's release archive, pinned by URL and SHA-256; until that
release exists RELEASE is None and the build stops with a message
naming the pending value. tools/patch_compiler_pal.py and its ninja rule
are removed.

Validated with a locally built GC_2_5a: objects byte-identical to the
byte-patched GC/2.5a; report.json unchanged in all three regions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QmcpERTQ1WaRZ73F12eUis
@frioux
frioux marked this pull request as draft November 10, 2026 01:08
@frioux frioux changed the title Derive GC/2.5a (kss alias correction) for PAL/USA; pppRand vector randomizers match in all regions GC/2.5a from mweppc (kss alias rule) for PAL/USA; pppRand vector randomizers match in all regions Nov 10, 2026
@frioux

frioux commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

Approach changed at the author's request: instead of shipping a byte patch of the stock GC 2.5 binary (tools/patch_compiler_pal.py), the kss rule is proposed as a new version, GC 2.5a, of the compiler decompilation itself (doldecomp/mweppc#3), where it is readable, reviewable source next to mweppc's BFBB variant 2.0p2. This PR now downloads that build from a pinned mweppc release instead (tools/download_mweppc.py). It is a draft until the mweppc PR is merged and released; then only the pinned URL/SHA-256 in tools/download_mweppc.py needs setting. Results are unchanged: the mweppc build gives byte-identical objects to the old patched binary, and report.json is identical in all three regions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant