Skip to content

ELF: Apply the header fields a p-code opinion constrains - #740

Open
zardus wants to merge 1 commit into
masterfrom
feature/fix-cle-pcode-opinions
Open

ELF: Apply the header fields a p-code opinion constrains#740
zardus wants to merge 1 commit into
masterfrom
feature/fix-cle-pcode-opinions

Conversation

@zardus

@zardus zardus commented Aug 13, 2026

Copy link
Copy Markdown
Member

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Ghidra's ELF opinions state their secondary attribute as e_flags, but the matcher
compared it against e_type, and a hexadecimal or bit-pattern secondary raised
ValueError and kept its opinion regardless. The opinions also often omit endian,
and nothing else held the chosen language to the byte order the file declares.

So EM_PARISC did not load at all, its one opinion being keyed to e_flags 528; a
double-float LoongArch object resolved to lp64f rather than lp64d; and a
little-endian NDS32 object resolved to NDS32:BE:32:default. Both constraints now
come from the ELF header.

Where several languages stay genuinely compatible, as for RISC-V or 68000, they
all still do. The regression loads the hppa fixture, which does not load today,
and a new NDS32 one.

Validation: #740 (comment)

sync: angr/binaries#177

Ghidra's ELF opinions state their secondary attribute as e_flags: the RISC-V
opinion documents "e_flags: used as secondary" and the AVR8 one says "Elf
e_flags are used for the secondary attribute". The matcher compared it against
e_type, which pyelftools reports as a name such as ET_EXEC, so a decimal
secondary never matched, and a hexadecimal or bit-pattern secondary raised
ValueError and kept its opinion unconditionally.

The opinions also frequently omit endian, and nothing else held the selected
language to the byte order the file declares, so a language of the opposite
byte order stayed a candidate and could be selected on the order
pypcode.Arch.enumerate() happened to return.

EM_PARISC therefore did not load at all, since its one ELF opinion is keyed to
e_flags 528; a double-float LoongArch object resolved to lp64f rather than
lp64d; and a little-endian NDS32 object resolved to NDS32:BE:32:default.
@zardus

zardus commented Aug 13, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 0966e0817cb506b90746bf5c90dd67452542e29b against baseline 45c6509c753d07f740099035cd41f7f473dc6f31.

  • Regression: pytest tests/test_arch_detect.py — on the baseline test_elf_hppa raises ArchNotFound: Can't find architecture info for architecture em_parisc with 32 bits and Iend_BE endness, test_elf_nds32 fails assert 'NDS32:BE:32:default' == 'NDS32:LE:32:default', and test_opinion_secondary_forms reports no _pcode_secondary_matches; all four pass on head
  • Reverting only the byte-order line fails test_elf_nds32; reverting only the e_flags change fails the other two
  • Focused: pytest tests/test_arch_detect.py — 4 passed
  • Full suite: pytest tests -n 4 — 228 passed, 9 skipped
  • Lint/type: every configured pre-commit hook over all files, clean; against the merge base, pylint 10.00 -> 10.00 and pyright badness 0.2736 -> 0.2695 on cle/backends/elf/elf.py
  • Workspace gate: cle only; the machine was carrying another long job, so the cross-repository gate was not run
  • Fixture: Add a little-endian NDS32 object binaries#177

Measured over a private corpus of 234056 ELF objects, 91402 of which reach ELF._extract_pcode_arch. Each selection was scored only against the file's own header: the language's byte order must equal EI_DATA, its size must fit elfclass, and some opinion admitting it must carry an e_flags constraint the file satisfies.

Selection Objects
Corrected, was a load failure 5740
Corrected, unsound selection replaced 5329
Unchanged selection, impossible candidate dropped 9227
Made worse 0
More than one candidate, before / after 27613 / 13057
  • EM_PARISC, e_flags 0x210: ArchNotFound -> pa-risc:BE:32:default, for instance the object with sha256 6cdf3252a73fc432c250024fe36d6e92cbe916cc79496b82ae6528781b7ab390
  • EM_LOONGARCH, e_flags 0x43: Loongarch:LE:64:lp64f -> Loongarch:LE:64:lp64d, for instance sha256 0007ca00769917ac91d9bc978fe87ca6ecc748a8dd27ac705a962d6b1f2e8695
  • Still more than one candidate: 68000 over four variants (7327 objects), avr8 over four (3857), RISC-V default against AndeStar_v5 (1873)
  • Ten of these objects were loaded through cle.Loader on both revisions and matched the scored result exactly

Every e_machine any ELF opinion names was then enumerated against both byte orders, both ELF classes and every e_flags value any opinion distinguishes, 11832 combinations. The only ones that lose their last candidate are byte orders no language exists for at all (big-endian V850, Hexagon, CR16 and LoongArch) and LoongArch with an ABI modifier outside 1 to 3, which the psABI does not define. No corpus object is either.

Blast radius, the same 300 randomly chosen corpus ELFs scored through angr.Project and CFGFast on each revision, with no language pinned: 274 unchanged, 21 hppa objects going from ArchNotFound to pa-risc:BE:32:default with block recovery (13 to 32563 blocks each), 5 LoongArch objects moving from lp64f to lp64d with their block counts within two. Nothing went from a successful score to an error, and none of the changed objects recovered a block outside an executable range or inside data.

Caveats: pypcode.Arch.enumerate() still returns languages in filesystem order, so the sets that remain ambiguous are still resolved arbitrarily.

@angr-bot

Copy link
Copy Markdown
Member

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_740

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants