Skip to content

proofs: expose native EVMYulLean transition target#1743

Open
Th0rgal wants to merge 171 commits intomainfrom
codex/native-evmyullean-transition
Open

proofs: expose native EVMYulLean transition target#1743
Th0rgal wants to merge 171 commits intomainfrom
codex/native-evmyullean-transition

Conversation

@Th0rgal
Copy link
Copy Markdown
Member

@Th0rgal Th0rgal commented Apr 22, 2026

Summary

  • expose Native.interpretIRRuntimeNative as the executable native EVMYulLean target for emitted IR runtime Yul
  • preserve pre-existing event history when projecting native EVMYulLean results back to YulResult
  • close the executable/source-surface migration blockers reported against the transition: environment accessors, mapping-struct storage agreement, overloaded function identity, direct helper-call lowering, packed full-word storage writes, and dynamic arrays of static tuple elements
  • document the clean transition plan from the backend-parameterized Verity interpreter to native EvmYul.Yul.callDispatcher

Fixed issues

Closes #1738.
Closes #1739.
Closes #1740.
Closes #1741.
Closes #1742.
Closes #1744.
Closes #1745.

Refs #1737.
Refs #1722.

Why

PR #1735 was only a docs/sync cleanup and did not close the native-runtime work from #1722. The current public theorem target still runs through interpretYulRuntimeWithBackend .evmYulLean, which uses Verity's custom Yul statement interpreter with EVMYulLean-backed builtins. That bridge is useful, but it is not the simpler final architecture requested in #1737.

This PR is now the native transition foundation plus the reported frontend/executable parity fixes needed before the remaining work can focus on the native public theorem target.

What changed

  • Added the native IR-runtime harness entry point Native.interpretIRRuntimeNative over EvmYul.Yul.callDispatcher.
  • Added result/state projection coverage for storage, events, environment fields, halt/revert/error behavior, and explicit observable storage slots.
  • Fixed verity_contract environment reads such as blockTimestamp, blockNumber, chainid, blobbasefee, contractAddress, msgSender, and msgValue, including executable .run state reads.
  • Aligned mapping-struct executable helpers with the compiler/native abstract mapping-slot formula, including packed member masking.
  • Introduced signature-based function identity for overloads while preserving Solidity-facing names for selectors and ABI dispatch.
  • Confirmed direct function-name calls are the user-facing internal-helper surface; explicit internalCall/internalCallAssign remain only lower-level compilation-model IR.
  • Added first-class packed full-word storage writes through setPackedStorage root offset word, lowering to Stmt.setStorageWord / sstore(root.slot + offset, word).
  • Added dynamic array support for static tuple elements on arrayLength/arrayElement; dynamic element arrays such as Array String and Array Bytes still fail closed.
  • Updated docs and CI guards so the repo does not claim native EVMYulLean is authoritative before the public theorem target flips.

Remaining work for #1737

  • prove native lowering invariants for dispatcher/function partitioning and builtin primop lowering
  • prove native state bridge lemmas for calldata, selector, caller/source, callvalue, address, block fields, storage, transient storage, memory, and returndata
  • prove result projection lemmas for normal values, stop, return, revert rollback, logs, and hard native errors
  • widen executable native coverage for dispatcher selection, memory-heavy return/revert/log paths, returndata, external calls, static-call permissions, and mapping helper lowering
  • introduce a public preservation theorem targeting Native.interpretIRRuntimeNative or a total wrapper around it
  • only then move execYulFuel / execYulFuelWithBackend to reference-oracle status and update trust-boundary docs accordingly

Validation

  • lake build Contracts.Smoke
  • lake build Contracts.MacroTranslateInvariantTest
  • lake build Contracts.MacroTranslateRoundTripFuzz
  • lake build Compiler.Proofs.YulGeneration.Backends.EvmYulLeanNativeHarness Compiler.Proofs.YulGeneration.Backends.EvmYulLeanNativeSmokeTest
  • python3 scripts/check_selectors.py
  • python3 -m unittest scripts/test_native_transition_api.py scripts/test_check_native_transition_doc.py scripts/test_check_selectors.py -v
  • python3 -m unittest scripts/test_generate_macro_property_tests.py -v
  • python3 scripts/check_macro_property_test_generation.py
  • python3 scripts/check_lean_hygiene.py
  • python3 scripts/check_verification_status_doc.py
  • python3 scripts/check_layer2_boundary_sync.py
  • python3 scripts/check_layer2_boundary_catalog_sync.py
  • python3 scripts/check_docs_workflow_sync.py
  • git diff --check
  • make check

Note

Medium Risk
Expands the compiler IR surface (new array/Storage ops and overload handling) and adds a large native-execution proof seam; mistakes could change generated Yul or validation behavior for contracts relying on these features.

Overview
Adds a native EVMYulLean execution seam by introducing result-comparison predicates and end-to-end theorems in Compiler/Proofs/EndToEnd.lean that relate Native.interpretIRRuntimeNative (via callDispatcher) to the existing EVMYulLean-backed interpreter, parameterized by an explicit observable storage-slot projection.

Extends the compilation model and lowering to support packed full-word storage writes (Stmt.setStorageWord) and dynamic array element word access (Expr.arrayElementWord), including new checked helper functions, param-loading bounds/stride fixes for arrays of multi-word static tuples, and usage analysis so only the required helpers become reserved/emitted.

Tightens validation and determinism: external overloads are now de-duplicated by function signature (while internal helpers remain keyed by name), internal/external name collisions are rejected, and new smoke/tests cover the added IR forms and updated macro behaviors. CI/docs are updated so the weekly EVMYulLean fork conformance probe also builds the native harness and related oracle/smoke tests.

Reviewed by Cursor Bugbot for commit 4f58dad. Bugbot is set up for automated code reviews on this repo. Configure here.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fc6b2b9c8a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Compiler/Proofs/YulGeneration/Backends/EvmYulLeanNativeHarness.lean Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 19edb78f49

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/check_selectors.py Outdated
Comment thread Compiler/ModuleInput.lean Outdated
Comment thread Compiler/CompilationModel/UsageAnalysis.lean
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c61a0a8e5c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Compiler/TypedIRCompiler.lean Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 070a5f855f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Verity/Macro/Translate.lean Outdated
Comment on lines +2102 to +2103
let matchedFns := candidates.filter (fun fn =>
fn.params.map (fun param => param.ty) == argTypes)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Relax overload matching for numeric literal arguments

This overload resolver requires exact type equality (fn.params ... == argTypes), but inferPureExprType classifies numeric literals as uint256. As a result, helper calls like foo(1) stop resolving when the only matching overload expects int256 (or other non-uint256 numeric types), even though the prior name/arity path accepted them and arithmetic typing already has signed-literal compatibility logic. This introduces a compile-time regression for contracts that pass literals to typed internal helper calls; the resolver should apply literal-compatible matching rather than strict equality.

Useful? React with 👍 / 👎.

@Th0rgal
Copy link
Copy Markdown
Member Author

Th0rgal commented Apr 24, 2026

Pushed 54c5e8df (proofs: stage native dispatcher bridge work).

What this commit actually adds:

  • extends the native harness with the missing error-path/block-append/switch-hit lemmas needed to reason about selected native dispatcher cases that halt before the default path runs
  • adds the simpleStorage lowering packaging and bridge wrapper in Compiler/Proofs/EndToEnd.lean, so the remaining public-theorem seam is reduced to one concrete witness instead of an open lowering existential
  • adds docs/NATIVE_EVMYULLEAN_DONE_GRAPH.md to record the dependency graph and current bottleneck for the native transition

What is still left to do:

  • the public theorem is still blocked at the concrete witness in Compiler/Proofs/EndToEnd.lean:
    nativeCallDispatcherAgreesWithInterpreter ... simpleStorageNativeContract
  • the current harness can now consume a concrete selected-body execution fact, but the repo still does not prove that the emitted simpleStorage selected bodies execute to the required native YulHalt results
  • concretely, the missing proof layer is:
    • store body native execution: calldataload(4), sstore(0, value), stop()
    • retrieve body native execution: mstore(0, sload(0)), return(0, 32)
    • miss/default revert body packaging
    • the native calldataload(4) decode fact needed for the store path
  • without those body-execution theorems, the switch lemmas added here still cannot discharge the final hBody obligations that produce the dispatcher-agreement witness

Advice to complete this PR cleanly:

Plan A:

  1. Prove three concrete native body-execution lemmas in Compiler/Proofs/YulGeneration/Backends/EvmYulLeanNativeHarness.lean for the lowered store, retrieve, and miss/revert bodies.
  2. Add the native calldataload(4) word/decode lemma for the store case.
  3. Derive a concrete simpleStorage_nativeCallDispatcherAgreesWithInterpreter theorem.
  4. Replace the remaining witness argument in simpleStorage_endToEnd_native_evmYulLean_of_callDispatcher_bridge with that derived theorem and expose the final public theorem.

Plan B:

  1. Add one slightly more general harness theorem for compiler-emitted guarded straight-line bodies ending in STOP, RETURN, or REVERT.
  2. Instantiate it for the simpleStorage selected bodies.
  3. Lift through the existing switch-hit lemmas and close the same witness in EndToEnd.
  4. Use that theorem family for future contracts so this PR does not stop at a one-off simpleStorage proof.

Plan C:

  1. Keep the public seam explicit for now.
  2. Land only the missing concrete selected-body native simulation family first.
  3. Flip the public theorem in a final follow-up commit once the dispatcher witness is an actual theorem instead of a parameter.

My recommendation is Plan A first. The current tree is already structured around a concrete simpleStorage seam, and the missing gap is now narrow enough that proving the exact selected-body native execution theorems is the shortest honest route to deriving the native theorem. Plan B is better only if we want immediate reuse for more emitted contracts in this same PR.

Validation on this branch remains green:

  • lake build Compiler.Proofs.YulGeneration.Backends.EvmYulLeanNativeHarness Compiler.Proofs.EndToEnd

Comment thread Compiler/Proofs/IRGeneration/FunctionBody.lean Outdated
@Th0rgal Th0rgal force-pushed the codex/native-evmyullean-transition branch from 0672929 to 39ae307 Compare April 24, 2026 16:47
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 39ae307375

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Compiler/CompilationModel/Compile.lean Outdated
Comment on lines +132 to +136
let valueExpr ← compileExpr fields dynamicSource value
let slotExpr :=
if wordOffset == 0 then YulExpr.lit slot
else YulExpr.call "add" [YulExpr.lit slot, YulExpr.lit wordOffset]
pure [YulStmt.expr (YulExpr.call "sstore" [slotExpr, valueExpr])]
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Mirror setStorageWord writes to alias slots

Stmt.setStorageWord currently emits exactly one sstore at the resolved base slot, but fields can carry aliasSlots (from slot-alias compatibility) that are intentionally kept in sync by other write paths like setStorage. With the current lowering, setPackedStorage/setStorageWord on an aliased field updates only the primary slot and leaves aliases stale, so contracts relying on alias mirroring for migration/back-compat can observe inconsistent state depending on which slot is read.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4f58dad. Configure here.

let slots := slot :: f.aliasSlots
match slots with
| [] =>
throw s!"Compilation error: internal invariant failure: no write slots for field '{field}' in setStorageWord"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreachable match arm on always-nonempty list

Low Severity

slots is defined as slot :: f.aliasSlots, which is always a cons cell, making the | [] => match arm at line 138 unreachable dead code. The throw will never execute because a list constructed with :: can never match []. The remaining two arms ([singleSlot] and _) already cover all possible shapes of a nonempty list, so the dead branch adds misleading noise about an "internal invariant failure" that structurally cannot occur.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 4f58dad. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment