Skip to content

Load Mach-O bundles and kernel extensions - #728

Open
zardus wants to merge 1 commit into
masterfrom
feature/fix-cle-macho-filetypes
Open

Load Mach-O bundles and kernel extensions#728
zardus wants to merge 1 commit into
masterfrom
feature/fix-cle-macho-filetypes

Conversation

@zardus

@zardus zardus commented Aug 10, 2026

Copy link
Copy Markdown
Member

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

The mach-o backend picks its base address with a chain of filetype tests whose else arm refuses everything it has no rule for, so kernel extensions, bundles, object files and dSYMs are all rejected as unsupported.

Bundles and kexts are linked relative to 0 exactly as dylibs are, so they share the dylib branch now. MH_OBJECT stays refused, since it needs the per-section mapping ELF does for ET_REL, and dSYMs stay refused but say they carry only debug information. The same block also let an unknown filetype escape as a bare ValueError and derived pic by masking a field that holds an ordinal. Separately, the library ordinal a bundle uses for symbols bound to whatever loaded it is not an index into imported_libraries, and raised IndexError while resolving relocations.

Regressions load tests/aarch64/IPwnKit.macho.kext, unused in angr/binaries so far because it could not be loaded, and rewrite one header field of existing fixtures in temporary copies for the rest.

Validation: #728 (comment)

@zardus

zardus commented Aug 10, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head eebcd256ad9ffb47981f516c66daf5489d11136c against baseline 45c6509c753d07f740099035cd41f7f473dc6f31, with angr/binaries at 12d015e510e2a5fe1217ef061166b7ed7fda7a04. Python 3.12.13, pytest 9.1.1, pylint 4.0.7, pyright 1.1.411.

Rebased from b58ea02 onto current master. The one conflict was in tests/test_macho.py against #729, and it was textual only: both branches appended imports, module constants and new tests to the same three places. Both sides are kept whole. #729 changed _load_segment, this branch changes header parsing in __init__ and the library ordinal in symbol.py, so the two production changes do not touch the same code and neither makes the other redundant; the only Mach-O backend change between the old and the new baseline is #729's.

  • Regression: python -m pytest tests/test_macho.py — 7 of 17 fail on baseline, all pass on head
  • Focused: python -m pytest tests/test_macho.py — 17 passed
  • Mach-O and rebasing suites: test_macho.py, test_macho_bindinghelper.py, test_macho_dyld.py, test_macho_dyld_structs.py, test_macho_libs.py, test_macho_reloc.py, test_universal2.py, test_rebase.py, test_arch_detect.py — 57 passed, 9 skipped
  • Full suite: python -m pytest tests/ — 232 passed, 9 skipped (the 9 are the pre-existing TODO skips in tests/test_macho_bindinghelper.py)
  • Lint/type: pylint score and pyright badness per changed file, against the merge base — cle/backends/macho/macho.py 10.00 -> 10.00, badness 0.1713 -> 0.1682; cle/backends/macho/symbol.py 9.94 -> 9.94, badness 0.0 -> 0.0; tests/test_macho.py 9.93 -> 9.96, badness 0.0552 -> 0.0358
  • Pre-commit: complete configured hook set over all files — 22 passed, 2 skipped for having no matching files, tree unchanged

Reproducer on the baseline, with a fixture that has been in angr/binaries since 2705d97 (sha256 eeac765a855674a309ec741f090162c38364fcdd6ace04a604a5fa9ca7a27966):

import cle
cle.Loader("binaries/tests/aarch64/IPwnKit.macho.kext", auto_load_libs=False)
# cle.errors.CLECompatibilityError: Unsupported Mach-O file type: 11.
# Please open an issue if you need support for this

Baseline failures, one per defect:

Test Failure on baseline
test_kext CLECompatibilityError: Unsupported Mach-O file type: 11
test_bundle same, for filetype 8
test_dsym_is_rejected refusal text does not say the file holds debug information
test_unsupported_filetype_is_named refusal names the filetype as 1, not MH_OBJECT
test_unknown_filetype ValueError: 99 is not a valid MachoFiletype, which Loader does not treat as a compatibility signal
test_executable_library_ordinal IndexError: list index out of range at cle/backends/macho/symbol.py, raised out of Loader.__init__ through MachOSymbolRelocation.resolve_symbol
test_non_pie_executable_is_not_pic pic is True on an executable with MH_PIE cleared

Each production hunk was reverted on its own, with the rest of the change in place:

  • cle/backends/macho/macho.py reverted entirely — 7 failed, 10 passed
  • only the LIBRARY_ORDINAL_EXECUTABLE branch deleted from cle/backends/macho/symbol.pytest_executable_library_ordinal fails with IndexError
  • only the old pic expression restored — test_bundle and test_non_pie_executable_is_not_pic fail
  • only self.pic or dropped from the new pic expression — test_non_pie_executable_is_not_pic fails on its force_rebase case

Scale, from a corpus sweep of 60,387 Mach-O and universal2 units loaded through angr.Project(..., auto_load_libs=False). That sweep predates this branch: it ran on cle 638301bbf02865e53394e6643e1eb6e66083ee94 with angr d4ebf5654b76aab69be69598220c15c80e73bb73, and the refusal is unchanged at the baseline above, as the reproducer shows. 17,273 were refused by this one message, which is 28.6% of the Mach-O units in the sweep and 99.3% of every Mach-O load failure it saw. 7,901 were MH_BUNDLE and 297 MH_KEXT_BUNDLE; the 9,018 MH_OBJECT and 57 MH_DSYM units stay refused. All 25 kexts and 32 of 40 bundles sampled from that sweep load through the dylib path unchanged when only the filetype field is rewritten; seven of the eight remaining bundle failures are the library-ordinal-255 IndexError this change fixes.

Caveats:

  • MH_OBJECT and MH_DSYM remain unsupported deliberately, and MH_FILESET and the other filetypes are still refused, now by name.
  • Only MH_KEXT_BUNDLE is covered by a real unmodified file. The bundle, dSYM, object-file and damaged-header cases rewrite one 32-bit header field of a fixture already in angr/binaries, and the library ordinal case re-encodes the kext's dynamic-lookup imports, so this needs no new binary.
  • A universal binary whose slices are bundles now takes the path fat dylibs already take, with whatever limitations that path has; nothing here changes it.
  • Only cle is checked out for this branch. archinfo, claripy, pyvex, pypcode, angr and angr-management are installed from wheels and untouched, so their suites do not run.

@angr-bot

Copy link
Copy Markdown
Member

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_728

The mach-o backend chose a base address with a chain of filetype tests whose
else arm refused everything it had no rule for, so MH_BUNDLE, MH_KEXT_BUNDLE,
MH_OBJECT and MH_DSYM were all rejected as unsupported even though nothing
after that block reads the filetype at all. Bundles and kexts want the base
address dylibs already get, because their segment vaddrs are relative to 0 as
well, so give the three of them one branch. MH_OBJECT stays refused, since it
needs the per-section mapping and relocation processing ELF does for ET_REL,
and MH_DSYM stays refused too but now says it carries debug information rather
than inviting a support request that should not be granted.

The refusal named the filetype by number because IntEnum.__str__ has been
int.__str__ since Python 3.11, and a filetype outside the enum escaped as a
bare ValueError raised while formatting that same message. Parse the field into
the enum once, where an unknown value becomes a CLECompatibilityError like
every other malformed header this backend reports.

A bundle binds its undefined symbols against the executable that dlopen-ed it,
which both the symbol table and the chained fixup imports encode as library
ordinal 255. That ordinal does not index imported_libraries, so resolving such
a symbol raised IndexError out of Loader.__init__. Handle it beside the
dynamic-lookup ordinal, which has the same shape.

pic was computed with a bitwise and against MH_DYLIB on a field that holds an
ordinal rather than a bitmask, so it was true for every filetype but
MH_OBJECT, MH_BUNDLE and MH_DYLIB_STUB, including non-PIE executables that the
loader then treated as rebasable. Test membership of the same zero-based
filetypes instead, and let the executable branch stand on the filetype alone,
which is what kept the wrong pic harmless there.

tests/aarch64/IPwnKit.macho.kext has sat in the binaries repository unused
since commit 2705d97 because it could not be loaded; it covers MH_KEXT_BUNDLE
now. The other cases rewrite one header field of an existing fixture in a
temporary copy.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants