aeo: snapshot staleness gate + Slack alert implementation#254
Conversation
…udit skill - Hard-block audit if Peec snapshot is 14+ days old (previously just noted it) - Add staleness exit path: write stale-snapshot report, post Slack alert, exit - Add concrete curl command for Slack posting (all three message types) - Update run log format to include 'Snapshot stale' as a status option - Add stale Slack message format (⚠️ ) alongside existing PR and no-change formats Co-Authored-By: Oz <oz-agent@warp.dev>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
I'm starting a first review of this pull request. You can view the conversation on Warp. I completed the review and no human review was requested for this pull request. Comment Powered by Oz |
![oz-for-oss[bot]](https://avatars.githubusercontent.com/in/3450139?s=60&v=4){kind=small}
There was a problem hiding this comment.
Overview
This PR updates the AEO cross-link audit skill to hard-block stale Peec snapshots, add a stale-snapshot report path, and make Slack notifications more concrete.
Concerns
- The freshness gate exits after the stale report and Slack alert but does not mention writing the run log, while the later stale-report section requires a run log entry. This can cause skipped run history.
- The Slack
curlexample places the bot token in command-line arguments and does not safely JSON-encode multiline message text, so it risks token exposure and failed notifications. - The stale report template requires an age and generated date even for the newly documented missing-file or missing-
generated_atcases.
Security
- The Slack bot token should not be expanded into a
curlheader argument where it can appear in process listings; use a restricted temporary config/header mechanism or API helper that keeps the secret out of argv.
No approved spec context was available, so no spec-drift findings were identified.
Verdict
Found: 0 critical, 4 important, 0 suggestions
Request changes
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
| - **Peec snapshot** - Read the Peec snapshot files at `/workspace/buzz/aeo-snapshots/docs/agents-orchestration/latest.json` and `/workspace/buzz/aeo-snapshots/docs/agents-orchestration/latest.md`. These contain pre-exported Peec data (prompts, recommendations, source URLs, query vocabulary, and visibility scores) for agents, cloud agents, and orchestration. The snapshots are generated locally (where Peec OAuth works) and committed to the buzz repo so cloud agents can use them. If the snapshot files do not exist or are empty, note that Peec data was unavailable and proceed with other sources. | ||
| - **Peec snapshot** - Check freshness before using any data: | ||
| 1. Read `generated_at` from `/workspace/buzz/aeo-snapshots/docs/agents-orchestration/latest.json`. | ||
| 2. If the file is missing, `generated_at` is absent, or the snapshot is **14 days old or older**: write the stale-snapshot report (see "Stale snapshot report" below), post the stale Slack alert (step 8), and exit. Do not continue the audit or open a PR. |
There was a problem hiding this comment.
|
|
||
| ```bash | ||
| curl -s -X POST https://slack.com/api/chat.postMessage \ | ||
| -H "Authorization: Bearer $SLACK_BOT_TOKEN" \ |
There was a problem hiding this comment.
$SLACK_BOT_TOKEN into a curl header argument exposes the token in process argv; use a restricted temporary curl config/header mechanism or Slack helper that keeps the secret out of command-line arguments.
| -H "Content-Type: application/json" \ | ||
| -d "{ | ||
| \"channel\": \"$SLACK_CHANNEL_ID\", | ||
| \"text\": \"<message text here>\", |
There was a problem hiding this comment.
jq or Python so newlines and backslashes cannot make the request invalid.
| ## AEO cross-link audit — snapshot stale | ||
|
|
||
| **Date:** YYYY-MM-DD | ||
| **Snapshot age:** N days (generated YYYY-MM-DD) |
There was a problem hiding this comment.
generated_at, but those cases have no age or generated date; add explicit missing/unknown wording so agents do not invent values.
- Add run log entry step to all early-exit paths (was only in stale-snapshot report section, not source data or workflow step 1) - Replace curl-based Slack posting with Python to keep SLACK_BOT_TOKEN out of process argv (security) and handle JSON encoding of multiline text correctly - Add explicit fallback wording to stale-snapshot report and Slack message for missing file and absent generated_at cases, so agents do not invent values Co-Authored-By: Oz <oz-agent@warp.dev>
2 participants
What
Hardens the recurring AEO crosslink audit skill against stale Peec snapshot data.
Changes to
.agents/skills/aeo_crosslink_audit/SKILL.mdStaleness gate (hard block)
The skill now checks
generated_atinlatest.jsonbefore doing anything else. If the snapshot is missing or 14+ days old, the agent aborts immediately — no audit, no PR. Previously the agent noted the staleness in the PR body but still ran.Stale-snapshot report
Defines the exact report text the agent writes to the Oz run output when aborting due to a stale snapshot, including the action required (run
refresh-peec-aeo-snapshotin a local Warp session).Concrete Slack curl implementation
Step 8 now includes the actual curl command using
$SLACK_BOT_TOKENand$SLACK_CHANNEL_ID. Previously it only said "post a message" without specifying how.New stale Slack message format⚠️ message format alongside the existing ✅ (PR opened) and ℹ️ (no change) formats. Points directly to the refresh skill path so whoever sees the alert knows exactly what to run.
Adds a
Run log format update
Log header now supports
Snapshot staleas a status, and theNo-change reasonfield includessnapshot stale — N days oldas an option.Context
PR #245 (the first live crosslink audit run) used an 18-day-old Peec snapshot. The skill flagged it in the PR body, but still opened the PR. This PR closes that gap. A companion change in the
buzzrepo (PR #102) creates therefresh-peec-aeo-snapshotlocal skill and updates the snapshot to today.Conversation: https://staging.warp.dev/conversation/7dc909c3-5f3a-45e2-a0f2-154cca3e42b0
Co-Authored-By: Oz oz-agent@warp.dev