AgentReady

AgentReady is an open-source, local-first scanner for AI coding-agent readiness. It answers a different question than lint, CI, Scorecard, or secret scanners: not “is this code good?”, but “can a coding agent operate in this repository safely and leave reviewable evidence?”

The core question is:

Can an AI coding agent understand this repository, choose the right context, make a bounded change, verify it, and leave humans with a reviewable result?

AgentReady is a command-line tool and library. It scans a repository on disk, observes facts with deterministic detectors, evaluates them against built-in checks, and emits a readiness report and an experimental score. No external service is contacted, and the repository's own scripts are never executed.

It is designed around the major coding agents — Codex, Claude Code, GitHub Copilot, Cursor, Windsurf, Cline, Roo, and Gemini.

See example output before installing: high-readiness report and improvement-plan report.

What It Scans

AgentReady helps teams understand whether a repository exposes the information and capabilities an autonomous coding agent needs:

• agent instruction surfaces such as AGENTS.md, CLAUDE.md, .github/copilot-instructions.md, .cursor/rules/*.mdc, and tool-specific rule files
• repository shape: source, tests, docs, generated files, and binary/minified assets
• verification command surfaces across ecosystems (Node scripts, Makefiles, CMake, Bazel, Go, Rust, Python, Gradle, Maven, .NET)
• agent capability surfaces such as MCP server configs, Claude Code skills, hooks/settings, plugin manifests, and code-intelligence/LSP config
• safety signals in package scripts such as install-time lifecycle hooks, destructive commands, network-piped shells, and deploy/publish paths
• CI workflows and how they map to local checks
• context-efficiency risks such as oversized always-on instruction files or large checked-in files
• documentation entrypoints (README, CONTRIBUTING, architecture/development notes, environment templates)

Why It Exists

Most repositories were not designed for autonomous agents. Even when CI passes, agents can still struggle because they cannot find local conventions, choose the right commands, avoid generated files, or understand which change is risky.

AgentReady treats repository readiness as agent operability, not generic code quality. It is descriptive before prescriptive: early output shows what is present, what is missing, what overlaps, and what may create friction.

AgentReady complements, rather than replaces, existing checks:

• CI/lint/test prove the current change still works; AgentReady checks whether an agent can discover the right checks before editing.
• Scorecard/security scanners look for supply-chain and vulnerability signals; AgentReady looks for agent-facing context, command, capability, and review surfaces.
• Repo docs help humans; AgentReady verifies whether that guidance is findable, scoped, and machine-reportable.

The intended output is a prioritized improvement plan for agent onboarding, plus stable evidence that CI and enterprise tools can consume.

Architecture

Evidence collection is separated from policy:

• Detectors observe facts about the repository (lib/repo-readiness/detectors/).
• Checks evaluate those facts against rules and emit findings (lib/repo-readiness/checks/).
• Scoring converts findings into an experimental readiness score, plus a per-category (docs/commands/ci/instructions/files/safety) dimension-score rollup so, e.g., unsafe scripts don't get averaged away by strong CI (lib/repo-readiness/core/scoring.ts, lib/repo-readiness/checks/catalog.ts).
• Reporters render console, JSON, and markdown output (lib/repo-readiness/reporters/).
• The scan engine wires these together (lib/repo-readiness/core/scan-engine.ts).

See docs/product/architecture.md for the full model, docs/product/positioning.md for the product boundary, and docs/roadmap/v0.3-issue-drafts.md for the next milestone issue drafts.

Design guarantees

Four properties back the trust model this scanner asks you to rely on:

• Local-first and non-networked. No external service is contacted and the scanned repository's own scripts are never executed (see "What It Scans" above). The optional LLM analyze layer is the one exception, and it is off unless explicitly configured. Every default scan report lists the platform-level controls this guarantee means it cannot see (branch protection, required status checks, environment approval rules, ...) under "Not verified from repository contents," so their absence never reads as "confirmed fine."
• The optional LLM layer can run without AgentReady ever holding a credential. If you run AgentReady from inside an agent host (Claude Code, Cursor, …), the bundled MCP server lets that host's own model do the reasoning over stdio — see "Use your agent's own model" under Analyze below.
• diff never mutates your working tree. It scans each ref through a temporary, isolated git worktree and works even with uncommitted changes — see "Diff (PR readiness)" below.
• Config and report shapes are versioned, published contracts, not incidental JSON: JSON Schema is generated from the same Zod schemas the scanner validates against, exported via the package's ./schemas/* subpath, and drift-checked in CI — see "Configuration" below.

Install And Run

Prerequisites: Node.js 24+ for the current development branch.

git clone https://github.com/napetrov/agentready.git
cd agentready
npm ci
npm run agentready -- scan .

The package name is reserved in this repository as the scoped @napetrov/agentready, and the installed command name remains agentready. The package is not published yet; until the first npm release, use the local development command above or the first-party GitHub Action from this repository.

Scan

npm run agentready -- scan .                              # human summary
npm run agentready -- scan . --format json                # machine-readable report
npm run agentready -- scan . --format markdown            # markdown report
npm run agentready -- scan . --format sarif --output a.sarif # SARIF for code scanning
npm run agentready -- scan . --fail-on warning --min-score 80 # gate the exit code

The legacy --json / --markdown / --sarif flags are still accepted. Both scan and diff support --fail-on <off|info|warning|error> (default error) and --min-score <0-100>; the process exits non-zero when a gate trips.

Policy packs

--policy <name> applies a team-specific severity policy to gating without changing the raw findings or score. Four packs ship today: default (a no-op), enterprise (escalates missing/non-portable agent instructions, install/deploy/high-risk-capability/automatic-hook-execution safety signals, and protected-path CODEOWNERS gaps for organization-wide rollout governance), oss (escalates stale command references and contribution-onboarding gaps for repos that rely on external contributors), and ml-scientific (relaxes large-fixture and unified-lint-command gates for research/scientific-computing repos, where both are routine rather than neglect):

npm run agentready -- scan . --policy enterprise --fail-on error
npm run agentready -- scan . --policy oss --fail-on error
npm run agentready -- scan . --policy ml-scientific --fail-on error

A non-default policy also prints its severity adjustments (with reasons) to human-readable output. See docs/product/policy-packs.md for the design and what's still open (a config-file policyOptions shape for tuning thresholds without a CLI flag on every invocation).

Diff (PR readiness)

diff compares two git refs and fails on new regressions. It uses a temporary git worktree, so it never mutates your working tree and works even with uncommitted changes:

npm run agentready -- diff --base origin/main --head HEAD . --fail-on-regression

Batch (portfolio) scans

batch scans multiple repositories in one invocation and emits an aggregated summary — no hosted service required. Pass explicit paths, --root <dir> to scan every immediate subdirectory of dir, or both:

npm run agentready -- batch ~/repos/service-a ~/repos/service-b
npm run agentready -- batch --root ~/repos --format markdown --output portfolio.md
npm run agentready -- batch --root ~/repos --min-score 70   # gate on any repo below 70

One repo failing to scan never aborts the batch; it's reported per-repo instead. --fail-on-scan-error (default on; pass --no-fail-on-scan-error to disable) and --min-score gate the exit code.

batch is deliberately local-only: it scans repositories already present on disk rather than a GitHub organization directly. Auto-discovering and cloning every repo in an org would mean AgentReady itself makes network calls and holds a GitHub credential, breaking the no-external-service guarantee every other command relies on. Clone (or git clone --depth 1) the org's repos yourself — a CI job, gh repo list <org> --limit 1000 --json name -q '.[].name' | xargs -I{} gh repo clone <org>/{} (--limit matters: gh repo list defaults to 30 results), or an existing script all work — into one directory and point --root at it.

Explain a finding

explain prints the rationale, remediation, and references for a readiness rule. Pass a finding id from a report or a bare rule id:

npm run agentready -- explain commands.test.missing
npm run agentready -- explain files.large:assets/blob.bin
npm run agentready -- explain --list            # all documented rule ids

Init

init scaffolds a starter .agentready.json (and, with --agents, a starter AGENTS.md). Existing files are left untouched unless you pass --force:

npm run agentready -- init .            # write .agentready.json
npm run agentready -- init . --agents   # also scaffold AGENTS.md

Analyze (optional LLM augmentation)

analyze runs a deterministic scan and then an optional, opt-in LLM layer that judges things deterministic checks cannot — e.g. whether an AGENTS.md is actually actionable, not merely present. It produces an augmented report: the deterministic score is never changed; a separate, clearly-labeled augmented score and an itemized list of adjustments are reported alongside it.

The layer is off unless a provider is configured via the environment (otherwise analyze runs deterministic-only). One adapter covers hosted OpenAI and local servers (Ollama, vLLM, LM Studio):

# Local model (no data leaves your machine):
OLLAMA_HOST=http://localhost:11434 npm run agentready -- analyze .

# Any OpenAI-compatible endpoint:
AGENTREADY_LLM_BASE_URL=https://api.openai.com/v1 \
AGENTREADY_LLM_MODEL=gpt-4o-mini OPENAI_API_KEY=sk-... \
  npm run agentready -- analyze . --format markdown

The deterministic scan/diff commands never call a model and are unaffected. See docs/product/llm-analytics-design.md.

Use your agent's own model (MCP / library)

If AgentReady runs inside an agent (Claude Code, Cursor, …) you can reuse the host's model instead of configuring a provider — AgentReady holds no credentials. The bundled MCP server exposes the host-delegated flow over stdio:

npm run agentready:mcp   # JSON-RPC 2.0 over stdio

It offers three tools: agentready_scan (deterministic, no model), agentready_analyze_prepare (returns prompts + sliced evidence for the host model to answer), and agentready_analyze_finalize (folds the host's answers into an augmented report). Library consumers can instead inject their own client via analyzeWithProvider(...) from the agentready/analyze export.

GitHub Action

Gate pull requests on readiness with the bundled action. It writes a job summary, can post a sticky pull-request comment, sets outputs, and can emit SARIF for code scanning:

permissions:
  contents: read
  security-events: write

steps:
  - uses: actions/checkout@v4
    with:
      fetch-depth: 0
  - uses: actions/setup-node@v4
    with:
      node-version: 24
  - id: agentready
    uses: napetrov/agentready@main
    with:
      mode: diff
      base-ref: origin/${{ github.base_ref || 'main' }}
      head-ref: HEAD
      fail-on-regression: true
      min-score: 80
      upload-sarif: true
  - if: always() && steps.agentready.outputs.sarif-report-path
    uses: github/codeql-action/upload-sarif@v3
    with:
      sarif_file: ${{ steps.agentready.outputs.sarif-report-path }}

Inputs include path, mode, base-ref, head-ref, config, fail-on-severity, fail-on-regression, min-score, policy, job-summary, pr-comment, pr-comment-condition, github-token, upload-sarif, output-dir, tool-version, analyze, and analyze-min-score; outputs include score, findings-count, regressions-count, the report paths, policy-adjustments-count, (when policy is non-default) policy-effective-score, and (when analyze is on) augmented-score/augmented-report-path. See action.yml for the authoritative contract.

Pull-request comment

Set pr-comment: true to post the markdown report as a pull-request comment. The action keeps a single sticky comment and updates it in place on each run rather than stacking new ones. It needs pull-requests: write and uses the workflow github-token by default:

permissions:
  contents: read
  pull-requests: write

steps:
  - uses: actions/checkout@v4
    with:
      fetch-depth: 0
  - uses: napetrov/agentready@main
    with:
      mode: diff
      base-ref: origin/${{ github.base_ref || 'main' }}
      head-ref: HEAD
      pr-comment: true

By default (pr-comment-condition: on-findings) the comment is posted only when a run has new findings or regressions, so clean runs leave the PR thread quiet — the verdict still lands in the job summary. Once findings are resolved, an existing comment is updated to its cleared state rather than left showing stale findings. Set pr-comment-condition: always to comment on every run, including clean ones.

It is fail-open: on a missing permission or a non-pull_request run it logs a notice and continues without failing the job. Omit pr-comment (and the pull-requests: write permission) if the job summary is enough.

Fork pull requests: for PRs from forks, GitHub gives the default github.token read-only access, so the comment can't be posted — the step logs a warning and the job still passes. The job summary still works in that case; use it (or a pull_request_target workflow, with the usual care around untrusted code) if you need comments on fork PRs.

Optional LLM augmentation in CI (GitHub Models)

Set analyze: true to also run the LLM layer. The CI-native token source is GitHub Models — the workflow's built-in GITHUB_TOKEN with models: read, no secret to manage. It is opt-in: AgentReady only uses it when AGENTREADY_USE_GITHUB_MODELS=1 is set, so the ambient token never silently enables model calls.

permissions:
  contents: read
  models: read            # required for GitHub Models
steps:
  - uses: actions/checkout@v4
  - uses: napetrov/agentready@main
    with:
      analyze: true
      analyze-min-score: 70   # optional: gate on the augmented score
    env:
      AGENTREADY_USE_GITHUB_MODELS: '1'
      GITHUB_TOKEN: ${{ github.token }}

The deterministic gates run first and are unaffected; augmented-score gating is opt-in via analyze-min-score. Without a provider, analyze runs deterministic-only.

Evaluation / benchmarks

AgentReady should earn trust by comparing readiness findings against real agent friction, not just by existing. Two separate evaluation efforts back that up:

• The optional LLM analyze layer has its own offline eval harness. npm run agentready:eval (bin/agentready-eval.ts) runs the real analyzer pipeline over a labeled gold corpus of canned model responses and reports precision/recall/F1, a confusion matrix, and confidence calibration — no live model call required. analyze-corpus.test.ts enforces a floor in CI, so a regression in hallucination guards, score folding, or id drift fails the build before it ships. The same harness can score a live model in a one-off recording run.
• The deterministic core is dogfooded against real repositories. Every release is rescanned against a small corpus of real OSS/scientific repositories (see dev/REAL-REPO-EVAL.md and the "Post-dogfood hardening plan" in dev/BACKLOG.md) to catch false positives before they reach users — this drove concrete detector fixes (e.g. a Python Copyright comment no longer implies pyright type-check coverage). The public benchmark plan comparing readiness findings against real agent task friction is in docs/product/evaluation.md; npm run agentready:benchmark automates the scan half of that corpus today. The human-judgment half — classifying a reviewed repository's findings as true_positive/false_positive/false_negative/severity_mismatch/ policy_mismatch/not_observable_locally — is captured as structured data under reports/evaluation/calibration/.

Configuration

Optional scanner config is discovered (via cosmiconfig, restricted to data-only formats) from any of: package.json#agentready, .agentready.json, agentready.config.json, .agentreadyrc[.json|.yaml|.yml], or agentready.config.yaml/.yml. Discovery is rooted at the scanned directory and never walks up into parent directories. Executable config (.js/.ts/...) is deliberately not loaded — AgentReady never executes repository code — so JS/TS config files are refused rather than run.

{
  "ignorePaths": ["fixtures/**", "public/vendor/**"],
  "largeFileWarningBytes": 1000000,
  "largeFileErrorBytes": 5000000,
  "allowMinifiedFiles": false,
  "errorOnWarnings": false
}

ignorePaths adds AgentReady-specific exclusions on top of the repository's own .gitignore files, which the scanner already honours (root and nested, with git's hierarchy semantics). Common output directories (node_modules, dist, build, out, coverage, .git, .next, .turbo, .vercel) are always skipped.

Use --config <path> to load a config file from another location (JSON or YAML). Validate a config and print the normalized effective settings with:

npm run agentready -- validate-config .

The config and report shapes are published as JSON Schema under schemas/ (also available via the package's ./schemas/* export), generated from the same schemas the scanner validates against.

Documentation

• Architecture
• Feature Roadmap
• Use Cases
• Language And Tooling
• Development Guide

Development

npm ci
npm run type-check
npm run lint
npm test
npm run build

npm run build compiles the CLI and library to dist/ via tsconfig.build.json.

Status

This repository is pre-1.0. The score is experimental and should be treated as a structured signal, not a compliance certification.

License

MIT. See LICENSE.