Skip to content

feat(ai): gist report with call flow and etherscan hyperlinks - #338

Merged
spalen0 merged 3 commits into
mainfrom
feat/ai-report-call-flow
Aug 11, 2026
Merged

feat(ai): gist report with call flow and etherscan hyperlinks#338
spalen0 merged 3 commits into
mainfrom
feat/ai-report-call-flow

Conversation

@spalen0

@spalen0 spalen0 commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

What

The AI explanation gist was just the raw LLM detail. It now publishes a structured report — metadata header, the Telegram summary, a deterministic call flow, and the analysis.

Example output

Before https://gist.wavey.info/ZtoYmJ5CG6TvziTDhuiSuhQz
After https://gist.wavey.info/BJvmyq789oM6imwWOxvRFJDx

Both are the same Infinifi addFarms(uint256,address[]) timelock operation, generated by the real pipeline (live Etherscan labels + LLM).

Changes

  • New utils/llm/report.py — builds the gist page. The call flow comes straight from the decoded calldata (per-call target, signature, ABI parameter names, ETH value, nested bytes payloads unwrapped) rather than being asked of the LLM, so it can't be hallucinated, re-ordered, or summarized away. Every address is rendered full-length (per the repo's "no truncated addresses" rule) as a link to the chain's explorer, annotated with its contract label / token symbol.
  • Addresses are always hyperlinked in the analysis_build_prompt now emits an --- Address Links --- section containing the exact markdown for each address in the transaction, and the system prompt requires the detail to use it. The model copies a line instead of assembling an explorer URL or guessing which explorer a chain uses. The TLDR stays plain text, since it goes to Telegram unchanged.
  • Contract header links to Etherscan via ReportContext.label_address, defaulting to the executing timelock/Safe. Safe multisend batches label the utility contract instead, so _explain_safe_tx() passes the outer target in that path.
  • Gist titles go from a constant AI Transaction Analysis to <contract> - <DD/MM/YYYY HH:MM> - <RISK> (UTC), e.g. Infinifi Shorttimelock - 11/08/2026 10:04 - LOW, so a list of reports is scannable. Falls back to protocol name, then the old constant.

Bugs found while validating live

  • _marker_pattern matched "Detail" inside a ## Detailed Analysis heading, slicing the published report down to ed Analysis — visible at the top of the "before" gist above. Fixed with a word-boundary lookahead.
  • A leading "Detailed Analysis" heading in the LLM output now gets stripped so it doesn't double up with the report's own ## Analysis.

Refactor

looks_like_calldata / try_decode_inner_calldata / MAX_BYTES_RECURSION_DEPTH moved from ai_explainer.py to utils/calldata/decoder.py (their natural home) so report.py can share them without a circular import. collect_unique_addresses extracted so the label lookup and the links block cover exactly the same address set.

Testing

  • New tests/test_llm_report.py (25 cases) plus explainer-side coverage for the links section, address collection, title, and the marker-regex fix.
  • uv run ruff format . / uv run ruff check . clean, uv run mypy clean for the touched files.
  • uv run pytest tests/752 passed, 0 failed. A second commit fixes test_monitoring_tasks_exist_in_jobs_yaml, which was already failing on main: the hourly maple job was disabled in jobs.yaml (71c02bb) but monitoring.yaml still listed protocols/maple/main.py as scheduled. Marked the entry disabled: true, following the euler/pendle/silo convention (shared Safe/timelock monitors stay listed).
  • Validated end-to-end against the live Infinifi transaction, including the published gists linked above.

🤖 Generated with Claude Code

spalen0 and others added 3 commits August 11, 2026 12:04
The AI gist was just the raw LLM detail. It now publishes a structured
report: metadata header, the Telegram summary, a deterministic call flow,
and the analysis.

The call flow is built in Python from the decoded calldata (target,
signature, ABI param names, ETH value, nested bytes payloads) rather than
asked of the LLM, so it can't be hallucinated or summarized away. Every
address is rendered full-length as a block-explorer link, labeled with its
contract name / token symbol.

For the LLM's own prose, the prompt now carries an Address Links section
with the exact markdown per address and the system prompt requires every
address in the detail to use it — the model copies a line instead of
assembling an explorer URL (or picking the wrong chain's explorer). The
TLDR stays plain text since it goes to Telegram unchanged.

Gist titles go from a constant "AI Transaction Analysis" to
"<contract> - <DD/MM/YYYY HH:MM> - <RISK>" so a list of reports is
scannable.

Two fixes found while validating live:
- the DETAIL marker regex matched inside "## Detailed Analysis" and sliced
  the report down to "ed Analysis"
- a leading "Detailed Analysis" heading now gets stripped so it doesn't
  double up with the report's own "## Analysis"

looks_like_calldata / try_decode_inner_calldata / MAX_BYTES_RECURSION_DEPTH
move to utils/calldata/decoder.py (their natural home) so report.py can
share them without a circular import.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The hourly maple job was disabled in jobs.yaml (71c02bb) but monitoring.yaml
still listed protocols/maple/main.py as a scheduled task, so
test_monitoring_tasks_exist_in_jobs_yaml has been failing on main.

Follows the euler/pendle/silo convention: the protocol entry is marked
disabled while keeping its shared Safe/timelock monitors listed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A tuple parameter such as configure((address,uint256)) fell through the
scalar branch, so the report stringified the whole tuple — the address came
out as raw lowercase text instead of a checksummed explorer link. The same
gap in _extract_address_args meant those addresses never reached the label
lookup or the prompt's Address Links section, leaving the LLM without a link
to copy for them either.

Arrays and tuples are now decomposed recursively (array_element_type /
tuple_component_types) on both paths; iter_address_values walks the same
type structure for collection. Plain address[] rendering is unchanged.

decoder._split_top_level becomes public split_top_level_types so the report
can decompose a tuple type without re-implementing the parser.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@spalen0

spalen0 commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator Author

[P2] Handle addresses inside tuple parameters — utils/llm/report.py:152

Confirmed and fixed in 0df1c72.

Reproduced first — _parse_param_types does support tuples, so (address,uint256) arrives as a real param and hit the scalar fallback:

- `(address,uint256)`: `('0x79e1b8e45932a7c802ea3dab3844e5dea68d971f', 5)`
addresses collected: ['0xF5f2718708f471e43968271956CC01aaA8c46119']   ← struct address missing

The second half was the more damaging one: _extract_address_args didn't walk tuples either, so the address never reached the label lookup or the Address Links block, and the LLM had no link to copy for it in the prose.

Now:

- `(address,address,uint256)`:
  - `address`: [`0x79e1B8e45932A7C802eA3dAb3844e5DEa68d971f`](https://etherscan.io/address/0x79e1B8e45932A7C802eA3dAb3844e5DEa68d971f)
  - `address`: [`0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48`](https://etherscan.io/address/0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48)
  - `uint256`: `860,000,000,000,000,000`
  • Arrays and tuples are decomposed recursively via array_element_type / tuple_component_types, covering arrays of structs ([0], [1] index bullets) and nested tuples.
  • iter_address_values() walks the same type structure and is shared with _extract_address_args, so rendering and collection can't drift apart.
  • Plain address[] keeps its compact index-free bullets — no output churn for the common case.
  • Arity mismatch between a tuple type and its value falls back to the scalar render rather than raising.
  • decoder._split_top_level is now public split_top_level_types so the report decomposes tuple types with the same parser instead of a second implementation.

8 new tests in tests/test_llm_report.py::TestCompositeParams plus a collection-side case in test_ai_explainer.py. Full suite: 760 passed.

One related gap left deliberately out of scope: _format_decoded_calls (the prompt's Decoded Calldata section, pre-existing on main) still prints tuples raw. That section is accurate, just unannotated, and the LLM now gets the nested addresses via Address Links — happy to do it as a follow-up if you'd rather it match.

@spalen0
spalen0 merged commit b19b830 into main Aug 11, 2026
3 checks passed
@spalen0
spalen0 deleted the feat/ai-report-call-flow branch August 11, 2026 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant