8000 Mach-O: Load relocatable objects that map nothing instead of failing an assertion by zardus · Pull Request #878 · angr/cle · GitHub
[go: up one dir, main page]

Skip to content

Mach-O: Load relocatable objects that map nothing instead of failing an assertion - #878

Open
zardus wants to merge 1 commit into
masterfrom
feature/fix-c-366
Open

zardus wants to merge 1 commit into
masterfrom
feature/fix-c-366

Conversation

@zardus
@zardus zardus commented Oct 22, 2026 •
Copy link
Copy Markdown
Member

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

cle.Loader cannot load a Mach-O relocatable object that maps no memory, such as the new fixture tests/aarch64/relocatable_object_empty.macho, compiled from a translation unit with no code. It fails with an assertion that carries no message:

  File ".../site-packages/cle/loader.py", line 1062, in _map_object
    assert obj.min_addr <= obj.max_addr
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

The whole load is lost, so angr cannot open the object at all. A megabench object failed the same way in two corpus sweeps; the validation record identifies it.

Root cause

MachO inherits Backend.max_addr, the highest max_addr among the object's segments, and a segment's max_addr is vaddr + memsize - 1. The empty object's only segment has vmaddr 0 and vmsize 0, so the object's max_addr is -1, one below its min_addr of 0. MachO._load_segment already treats a segment with vmsize 0 as mapping nothing and backs none of it, but max_addr still counted it.

Fix

MachO.max_addr counts only segments with a nonzero vmsize. When there are none it falls back to the base address, which is what Backend.max_addr returns for an object with no segments or sections. The empty object now loads with min_addr and max_addr both 0 and nothing backed.

The rule is in the Mach-O backend rather than in Backend.max_addr: the same change there would move max_addr for four ELF files in angr/binaries whose highest PT_LOAD is empty, and angr places the initial program break from max_addr.

Testing

test_relocatable_object_empty loads the new fixture and checks its extent and that nothing is backed; it fails on master with the AssertionError above. Over 3709 other megabench Mach-O objects that corpus sweeps have recorded, the load result and every loaded object's extent are unchanged.

The fixture is in angr/binaries#264, which needs to merge first.

Validation: #878 (comment)

session: sharpen

A relocatable object can map no memory at all: compiled from a translation
unit with no code and without debug info, it has one segment with vmsize 0
holding an empty __text. MachO inherited Backend.max_addr, which counts every
segment, so the object's max_addr was that segment's vmaddr minus one, below
its min_addr, and the assertion in Loader._map_object lost the whole load.

A segment with vmsize 0 maps nothing, which _load_segment already assumes when
it backs none of it. MachO.max_addr now counts only segments with a nonzero
vmsize, and falls back to the base address when there are none, as
Backend.max_addr does for an object with no segments or sections.
@zardus
zardus commented Oct 22, 2026
Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 8508ba3eac1f389123d41784f5d4e7f14359889d against baseline 8d902867faeb69bffc7d98970dc7bb048d084226, with the fixture from angr/binaries#264 at 3737e91f17a90e71b22af8bc8148db34cffb2184.

A megabench object that failed to load in two corpus sweeps now loads; on 3709 other megabench Mach-O objects, nothing changed.

  • Regression: pytest --import-mode=append -p no:cacheprovider -q tests/test_macho.py::test_relocatable_object_empty fails on the baseline (AssertionError at cle/loader.py:1062, in Loader._map_object) and passes on the head.
  • Focused: pytest --import-mode=append -p no:cacheprovider -q tests/test_macho.py: baseline 1 failed (the new test), 16 passed; head 17 passed.
  • Workspace gate: the complete gate (run-all-tests.sh) ran at 486af4e3, which differs from this head only in one source comment and one test docstring. 13 of its 14 suites pass, among them cle (339 passed, 9 skipped), angr (5064 passed, 50 skipped, 42 xfailed), pyvex, archinfo, pypcode, angr-management, test-inputs and pre-commit. The workspace suite, which tests the workspace's own tooling, fails with 6 failures and 6 errors; the same 6 and 6 fail on a clean checkout of that tooling without this change. At this head the cle suite was run again: 339 passed, 9 skipped.
  • Lint/type at this head, merge-base relative (run-ci-diff-checks.py --repository cle): pylint cle/backends/macho/macho.py 10.00 to 10.00 and tests/test_macho.py 9.96 to 9.96; pyright errors 21 to 21 and 2 to 2.

Dataset evidence

One environment per side, built from the same trees apart from cle: angr afd6f92d, archinfo db4531e5, pyvex 0a915d9e (vex fd41c675), pypcode ce8f00da, pysoot 16ab55db; cle 8d902867 before and 8508ba3e after. measure.py loads each object in its own process through the corpus sweep's loader, with the load options the corpus declares for it, and records the load result, every loaded object's type, min_addr and max_addr, and for each Mach-O object its segments and the SHA-256 of its backed bytes. Its cfg and deep modes run the sweep's CFG and decompilation lanes instead. compare.py compares two such runs object by object.

Object Recorded by the sweeps Before After
08f1b4ee91819b70a47d02216368a8c6a9c50fc9761ede84a8d255b5115dffbf load-error, AssertionError in Loader._map_object, in a deep-lane and a cfg-lane epoch AssertionError at cle/loader.py:1062 loads: min_addr and max_addr 0, its only segment of vmsize 0, nothing backed; the CFG and decompilation lanes finish with 0 functions

That object is unavailable: its bytes are not published here. Two more megabench objects, which no sweep has reached yet, fail and load the same way: 82be3c686832032a0e80c757aa87c90eee24a5b1c6a16fdfbe67475d95f7b0cc and a9731237b20e40cb8d2d47bb01fb39988da57acfa092d5d220c479a1d27c76ca, also unavailable.

Regression set: 3709 other megabench Mach-O objects that the sweeps have recorded. They are all 3293 other relocatable objects (1780 arm64, 1513 x86_64); the 16 others with a segment of vmsize 0 (executables: 13 with an empty __DWARF, 3 with another empty segment); and 400 others in a seeded hash order (179 executables, 67 dylibs, 153 bundles and 1 dSYM file; 50 of them fat). On both sides 3555 load, and the 153 bundles and the dSYM file are refused with CLECompatibilityError, because cle does not support those Mach-O file types (8 and 10). Every one of the 3709 gives identical results on both sides. The CFG lane on the 16 with an empty segment gives identical function and block counts on both sides.

Alternative not taken: on the baseline env, over the 1285 files under tests/ in angr/binaries 3737e91f that load (86 others raise, and 8 Java archives were not measured), computing Backend.max_addr without zero-size regions changes 4 objects, all ELF with an empty PT_LOAD above the rest of the image: tests/i386/packed_elf32 (0x80f0da3 to 0xc51b82), tests/x86_64/packed_elf64 (0x6d1407 to 0x45bf2b), tests/x86_64/patchrex/replace_function_patch_with_function_reference (0xb00127 to 0xa001c4) and tests/armel/patcherex/replace_function_patch_with_function_reference (0xb00077 to 0xa000bd). The only file whose load raises AssertionError there is the new fixture.

Commands
python -P measure.py base measure-base-load.jsonl load
python -P measure.py cand measure-cand2-load.jsonl load
python -P compare.py measure-base-load.jsonl measure-cand2-load.jsonl
python -P measure.py base measure-base-cfg.jsonl cfg OBJECT...
python -P measure.py cand measure-cand2-cfg.jsonl cfg OBJECT...
python -P measure.py base measure-base-deep.jsonl deep EMPTY...
python -P measure.py cand measure-cand2-deep.jsonl deep EMPTY...
python - tests/aarch64/relocatable_object_empty.macho < load_empty.py
pytest --import-mode=append -p no:cacheprovider -q tests/test_macho.py

OBJECT... is the 3 empty objects above and the 16 with an empty segment; EMPTY... is the 3 empty objects. load_empty.py is in the before/after comment.

@zardus
8000 zardus commented Oct 22, 2026
Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Loading tests/aarch64/relocatable_object_empty.macho (from angr/binaries#264) with cle before and after this change, from the root of an angr/binaries checkout:

python - tests/aarch64/relocatable_object_empty.macho < load_empty.py
import sys

import cle

ld = cle.Loader(sys.argv[1], auto_load_libs=False)
obj = ld.main_object
print(obj)
print("min_addr", hex(obj.min_addr), "max_addr", hex(obj.max_addr))
print("segments", [(seg.segname, hex(seg.vaddr), seg.memsize) for seg in obj.segments])
print("backers", list(obj.memory.backers()))

The nix store prefix of each traceback path is shortened to .../site-packages/; nothing else is edited.

Before — the load fails the min_addr <= max_addr assertion in Loader._map_object:

cle master 8d90286
Binary is not using MH_TWOLEVEL namespacing.This isn't properly implemented yet and will degrade results in unpredictable ways.Please open an issue if you encounter this with a binary you can share
No entry point found
Traceback (most recent call last):
  File "<stdin>", line 5, in <module>
  File ".../site-packages/cle/loader.py", line 187, in __init__
    self.initial_load_objects = self._internal_load(
                                ^^^^^^^^^^^^^^^^^^^^
  File ".../site-packages/cle/loader.py", line 944, in _internal_load
    self._map_object(obj)
  File ".../site-packages/cle/loader.py", line 1062, in _map_object
    assert obj.min_addr <= obj.max_addr
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

After — the object loads and maps nothing:

with this change, 8508ba3
Binary is not using MH_TWOLEVEL namespacing.This isn't properly implemented yet and will degrade results in unpredictable ways.Please open an issue if you encounter this with a binary you can share
No entry point found
<MachO Object relocatable_object_empty.macho, maps [0x0:0x0]>
min_addr 0x0 max_addr 0x0
segments [('', '0x0', 0)]
backers []

@angr-bot
Copy link
Copy Markdown
Member

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_878

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0