chore: self-contained revert tests, contract reorg, and failure analysis by mojtaba-esk · Pull Request #3033 · sei-protocol/sei-chain

mojtaba-esk · 2026-03-06T12:01:24Z

Describe your changes and provide context

This PR proposes a number of improvements on EVM RPC tests. The failing tests' fixtures are manually analyzed and updated to have Ethereum-expected behavior with adding the required steps and tags.
Updated the README and analysis doc describe the suite, removed tests, and failure breakdown.

Replaces 4 execution-apis revert/estimateGas fixtures that relied on fixed addresses with Sei fixtures that use a deployable reverter contract (REVERTER); script sets SEI_EVM_IO_REVERTER_ADDRESS.

Related: PLT-158

Testing performed to validate your change

Latest run: 162 tests, 133 passed, 29 failed (82.1% pass) which previously was 61.6%
Failed cases are documented in FAILED_TEST_ANALYSIS.md with suggested RPC fixes.

…t-reorg,-and-failure-analysis

github-actions · 2026-03-06T12:02:23Z

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
`✅ passed`	`✅ passed`	`✅ passed`	`✅ passed`	Mar 13, 2026, 7:03 AM

github-actions · 2026-03-06T12:02:23Z

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
`✅ passed`	`✅ passed`	`✅ passed`	`✅ passed`	Mar 6, 2026, 12:02 PM

codecov · 2026-03-06T12:04:05Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 58.32%. Comparing base (9029c60) to head (cde1eb1).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3033      +/-   ##
==========================================
- Coverage   58.32%   58.32%   -0.01%     
==========================================
  Files        2076     2078       +2     
  Lines      171478   170782     -696     
==========================================
- Hits       100020    99608     -412     
+ Misses      62549    62263     -286     
- Partials     8909     8911       +2

Flag	Coverage Δ
sei-chain-pr	`79.34% <100.00%> (?)`
sei-db	`70.41% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
integration_test/evm_module/rpc_io_test/io.go	`79.34% <100.00%> (+0.45%)`	⬆️

... and 261 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

masih

Left a couple of comments about the assertion logic and missing coverage on panic/other failure modes.

Great to see irrelevant tests getting removed 🙌

integration_test/evm_module/scripts/evm_rpc_tests.sh

masih · 2026-03-06T12:24:38Z

integration_test/evm_module/rpc_io_test/testdata/eth_call/call-revert-abi-error-sei.iox

->> {"jsonrpc":"2.0","id":1,"method":"eth_call","params":[{"from":"0x0000000000000000000000000000000000000000","gas":"0x186a0","input":"0x01","to":"0x0ee3ab1371c93e7c0c281cc0c2107cdebc8b1930"},"latest"]}
+// Self-contained: script deploys reverter contract; runner substitutes __REVERTER__ with its address.
+// Expects eth_call to return JSON-RPC error code 3 with ABI-encoded Error("user error").
+>> {"jsonrpc":"2.0","id":1,"method":"eth_call","params":[{"from":"0x0000000000000000000000000000000000000000","gas":"0x186a0","input":"0x01","to":"__REVERTER__"},"latest"]}


This now only covers the error path; we are missing coverage for panic now that *-panic.io is removed?

Good point!
we were only covering the error path. I've restored panic coverage.

masih · 2026-03-06T12:26:02Z

...gration_test/evm_module/rpc_io_test/testdata/eth_estimateGas/estimate-call-abi-error-sei.iox

-// speconly: client response is only checked for schema validity.
->> {"jsonrpc":"2.0","id":1,"method":"eth_estimateGas","params":[{"from":"0x0102030000000000000000000000000000000000","input":"0x01","to":"0x0ee3ab1371c93e7c0c281cc0c2107cdebc8b1930"}]}
+// Self-contained: script deploys reverter; runner substitutes __REVERTER__. Expects eth_estimateGas to return error (revert).
+>> {"jsonrpc":"2.0","id":1,"method":"eth_estimateGas","params":[{"from":"0x0000000000000000000000000000000000000000","input":"0x01","to":"__REVERTER__"}]}


Similar to eth_call comment, i think now that estimate-failed-call.io is removed we are missing tests for panic and other revert/failure modes.

Agreed. I've added the panic fixture.

masih · 2026-03-06T12:27:16Z

integration_test/evm_module/rpc_io_test/RPC_IO_README.md

-| eth_simulateV1                     | ethSimulate-transfer-over-BlockStateCalls.iox                                     | FAIL   | Sei          | Spec-only check failed | response kind mismatch: expected result=true error=false, actual result=false error=true |
-| eth_simulateV1                     | ethSimulate-two-blocks-with-complete-eth-sends.iox                                | FAIL   | Sei          | Spec-only check failed | response kind mismatch: expected result=true error=false, actual result=false error=true |
-| eth_simulateV1                     | ethSimulate-use-as-many-features-as-possible.iox                                  | FAIL   | Sei          | Spec-only check failed | response kind mismatch: expected result=true error=false, actual result=false error=true |
-| eth_syncing                        | check-syncing.iox                                                                 | FAIL   | Sei          | Spec-only check failed | response kind mismatch: expected result=true error=false, actual result=false error=true |


Oh I see we are removing simualatev1 tests.

…t-reorg,-and-failure-analysis

masih

Apart from || true LGTM 🙌

integration_test/evm_module/scripts/evm_rpc_tests.sh

…t-reorg,-and-failure-analysis

bdchatham

A few minor concerns with files moved and slight inconsistencies between FAILED_TEST_ANALYSIS and RPC_IO_README but non-blockers

bdchatham · 2026-03-10T23:11:45Z

eth_signTransaction/signTransaction-unsupported.io

Was this move intention? I'm guessing yes so that the test runner doesn't find it

Nice catch! Thank you.
Actually the move was unintentional. The file ended up at repo root probably by a mouse drag :D I've restored it.

monty-sei

No noticeable issues stand out. All good on my end ✅

…t-reorg,-and-failure-analysis

chore: self-contained revert tests, contract reorg, and failure analysis

eb4c706

mojtaba-esk requested review from bdchatham, masih and monty-sei March 6, 2026 12:01

mojtaba-esk self-assigned this Mar 6, 2026

mojtaba-esk added the non-app-hash-breaking label Mar 6, 2026

Merge branch 'main' into mojtaba/self-contained-revert-tests,-contrac…

0d46c7a

…t-reorg,-and-failure-analysis

Mojtaba added 2 commits March 6, 2026 12:13

chore: add the previous run summary as well

2ec9674

chore: clean up summary

d96564b

masih reviewed Mar 6, 2026

View reviewed changes

Mojtaba and others added 2 commits March 9, 2026 11:52

chore: applied feedback

040bd7c

Merge branch 'main' into mojtaba/self-contained-revert-tests,-contrac…

0890820

…t-reorg,-and-failure-analysis

mojtaba-esk requested a review from masih March 9, 2026 11:53

mojtaba-esk and others added 2 commits March 9, 2026 15:26

Merge branch 'main' into mojtaba/self-contained-revert-tests,-contrac…

fb287a7

…t-reorg,-and-failure-analysis

Merge branch 'main' into mojtaba/self-contained-revert-tests,-contrac…

3ea12de

…t-reorg,-and-failure-analysis

masih approved these changes Mar 9, 2026

View reviewed changes

integration_test/evm_module/scripts/evm_rpc_tests.sh Show resolved Hide resolved

mojtaba-esk added 2 commits March 10, 2026 09:58

Merge branch 'main' into mojtaba/self-contained-revert-tests,-contrac…

6ec5aad

…t-reorg,-and-failure-analysis

Merge branch 'main' into mojtaba/self-contained-revert-tests,-contrac…

f30b9f2

…t-reorg,-and-failure-analysis

bdchatham approved these changes Mar 10, 2026

View reviewed changes

monty-sei approved these changes Mar 11, 2026

View reviewed changes

masih and others added 4 commits March 11, 2026 06:27

Merge branch 'main' into mojtaba/self-contained-revert-tests,-contrac…

62a16e4

…t-reorg,-and-failure-analysis

Merge branch 'main' into mojtaba/self-contained-revert-tests,-contrac…

e7eda85

…t-reorg,-and-failure-analysis

Merge branch 'main' into mojtaba/self-contained-revert-tests,-contrac…

69fe030

…t-reorg,-and-failure-analysis

apply feedback

cde1eb1

mojtaba-esk enabled auto-merge March 13, 2026 07:02

mojtaba-esk added this pull request to the merge queue Mar 13, 2026

Merged via the queue into main with commit 410a6ec Mar 13, 2026
38 checks passed

mojtaba-esk deleted the mojtaba/self-contained-revert-tests,-contract-reorg,-and-failure-analysis branch March 13, 2026 07:33

Conversation

mojtaba-esk commented Mar 6, 2026

Describe your changes and provide context

Testing performed to validate your change

Uh oh!

github-actions bot commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 6, 2026

Uh oh!

codecov bot commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

masih left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

masih Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

mojtaba-esk Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

masih Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

mojtaba-esk Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

masih Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

masih left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bdchatham left a comment

Choose a reason for hiding this comment

Uh oh!

bdchatham Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

mojtaba-esk Mar 13, 2026

Choose a reason for hiding this comment

Uh oh!

monty-sei left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions bot commented Mar 6, 2026 •

edited

Loading

codecov bot commented Mar 6, 2026 •

edited

Loading