diff --git a/.github/workflows/external-reference-check.yml b/.github/workflows/external-reference-check.yml index 18a0713..d8bb498 100644 --- a/.github/workflows/external-reference-check.yml +++ b/.github/workflows/external-reference-check.yml @@ -21,6 +21,25 @@ on: # Mondays at 12:00 UTC. Catches link rot in unchanged content. - cron: "0 12 * * 1" workflow_dispatch: + # Lets OTHER repos reuse this exact check instead of copying it, so the + # logic never drifts. A consumer adds a ~10-line caller workflow: + # + # jobs: + # external-references: + # uses: amd/skills/.github/workflows/external-reference-check.yml@main + # permissions: + # contents: read + # issues: write + # + # The lychee config is NOT configurable: callers always use the canonical + # .github/lychee.toml from amd/skills (pinned to the same commit as this + # workflow), so neither the logic nor the config can drift. + workflow_call: + inputs: + markdown_glob: + description: "Glob of markdown files to check, relative to the caller repo root." + type: string + default: "./**/*.md" permissions: contents: read @@ -34,11 +53,44 @@ jobs: - name: Check out repository uses: actions/checkout@v4 + # The lychee config always comes from amd/skills so it can never drift. + # When this repo runs the check on itself, the working-tree copy is used + # (so PRs that edit the config are tested against their own changes). + # When another repo calls this workflow, the canonical config is fetched + # from amd/skills at the SAME commit as this workflow file + # (github.job_workflow_sha), keeping logic and config in lockstep. + # Sparse-checkout grabs only the config file so its markdown can't + # pollute the scan. + - name: Fetch canonical lychee config + if: ${{ github.repository != 'amd/skills' }} + uses: actions/checkout@v4 + with: + repository: amd/skills + ref: ${{ github.job_workflow_sha || 'main' }} + path: .external-reference-check-config + sparse-checkout: | + .github/lychee.toml + sparse-checkout-cone-mode: false + persist-credentials: false + + - name: Resolve lychee config path + id: cfg + shell: bash + env: + IS_CANONICAL: ${{ github.repository == 'amd/skills' }} + run: | + set -euo pipefail + if [ "${IS_CANONICAL}" = "true" ]; then + echo "path=.github/lychee.toml" >> "$GITHUB_OUTPUT" + else + echo "path=.external-reference-check-config/.github/lychee.toml" >> "$GITHUB_OUTPUT" + fi + - name: Check external references id: lychee uses: lycheeverse/lychee-action@v2 with: - args: --config .github/lychee.toml --no-progress "./**/*.md" + args: --config ${{ steps.cfg.outputs.path }} --no-progress "${{ inputs.markdown_glob || './**/*.md' }}" # Fail the workflow on PR and manual runs so broken external # references show up as a red check (mark this workflow as # not-required in branch protection if you don't want it to diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 891c13a..81aaa4f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -29,13 +29,17 @@ Best for cross-cutting skills that do not have a natural product home. ``` 8. Open a pull request. The `validate` GitHub Actions workflow runs `./.github/scripts/check.sh` and must pass before merge. See [Validating locally](#validating-locally) for the full set of enforced rules. -### Path B: Skills authored in a product repository +### Path B: Skills authored in a product repository (federation) -Best for skills that should ship and version with a product (HIP, MIGraphX, Ryzen AI, Lemonade, etc.). +Best for skills that should ship and version with a product (HIP, MIGraphX, Ryzen AI, Lemonade, etc.). Your repo stays the source of truth and the catalog vendors a pinned copy. This is called **federation**, and the full walkthrough lives in [docs/federating-your-repo.md](docs/federating-your-repo.md). -1. Add the skill folder to your product repository; a common location is `.agents/skills//`. -2. Open a pull request here that adds (or extends) an entry in [`.github/scripts/sources.yml`](.github/scripts/sources.yml) — the master list — naming your repo, a pinned ref, the sub-path that holds skill folders, and your skill's folder name. -3. Once the catalog change merges, dispatch the **Import external skills** workflow from the Actions tab. It shallow-clones your repo at the pinned ref, vendors the skill into `skills//`, updates `.claude-plugin/marketplace.json`, and opens a follow-up pull request. Validation then runs against the same rules as in-repo skills before merge. +The short version: + +1. Keep each skill as a folder with a valid `SKILL.md` and `skill-card.md` in your AMD-owned repo (a common location is `skills/` or `.agents/skills//`). +2. Add (or extend) an entry in [`.github/scripts/sources.yml`](.github/scripts/sources.yml) — the master list — naming your repo, a pinned ref, the sub-path that holds skill folders, and each skill's folder name. +3. Vendor the skills and refresh the manifests locally, then open a pull request here for review. Validation runs against the same rules as in-repo skills before merge. + +See [docs/federating-your-repo.md](docs/federating-your-repo.md) for the exact `sources.yml` schema, the import commands, and how to run the catalog's checks in your own repo. ## Is this task a good fit for a skill? diff --git a/docs/federating-your-repo.md b/docs/federating-your-repo.md new file mode 100644 index 0000000..696d6fb --- /dev/null +++ b/docs/federating-your-repo.md @@ -0,0 +1,83 @@ +# Federate Your Repo Into the Catalog + +How to list skills that live in **your own AMD repo** in this catalog. Your repo +stays the source of truth; the catalog vendors a pinned copy. + +This is the detailed version of **Path B** in +[CONTRIBUTING.md](../CONTRIBUTING.md#path-b-skills-authored-in-a-product-repository-federation). +Start there for an overview of how it compares to authoring skills directly in +this repo. + +> **Eligibility: AMD-owned repositories only.** The source `repo` must be under +> an AMD GitHub org (e.g. `AMD-AGI/...`). Non-AMD repos are not accepted. + +## Prerequisites + +- Each skill is a folder with a valid `SKILL.md` and `skill-card.md`. + See [CONTRIBUTING.md](../CONTRIBUTING.md) and [skill-cards.md](skill-cards.md). +- Skills live in a known directory in your repo (e.g. `skills/`). +- Pick a branch to track (e.g. `main` or a release branch). + +## Add your source + +Edit [`.github/scripts/sources.yml`](../.github/scripts/sources.yml) and append an entry: + +```yaml +sources: + - name: amd-myproject # kebab-case source id + repo: AMD-Org/MyProject # must be AMD-owned + ref: main # branch to track (e.g. main or a release branch) + path: skills # dir in your repo holding the skill folders + license: MIT # SPDX id, carried into the marker file + skills: + - name: my-skill # folder name in your repo + as: myproject-my-skill # local catalog name: - +``` + +Use `as:` to namespace skills as `-` so catalog names stay unique. + +## Import + +Run the import scripts locally (they read `sources.yml` from your working tree), +then open a PR for review. + +1. Vendor the skills and refresh the manifests: + + ```bash + uv run .github/scripts/import_external_skills.py # vendor into skills// + uv run .github/scripts/generate_cursor_marketplace.py + ./.github/scripts/check.sh # validate + ``` + +2. Commit `skills/**`, `.github/scripts/sources.yml`, and the manifests. +3. Open a PR; a maintainer reviews and merges once CI passes. + +## Catch failures before nightly + +The catalog runs checks against your skills. Run the **same** checks in your own +repo by calling them as reusable workflows, so you catch breakage during normal +development instead of in the catalog's nightly run. The logic and config live in +`amd/skills`, so green in your repo means green in the catalog — and you never copy +or maintain the check yourself. + +Add a caller workflow to your repo (e.g. `.github/workflows/skills-checks.yml`): + +```yaml +name: skills-checks +on: + pull_request: + workflow_dispatch: +jobs: + external-references: + uses: amd/skills/.github/workflows/external-reference-check.yml@main + permissions: + contents: read + issues: write +``` + +## Update or remove + +Automatic refresh and pruning will soon be enabled through nightly workflows. + +Never hand-edit vendored skills under `skills/`; changes must come from your repo +via re-import, or they'll be overwritten. diff --git a/eval/behavioral/harness.py b/eval/behavioral/harness.py index 95ae0da..feb9a44 100644 --- a/eval/behavioral/harness.py +++ b/eval/behavioral/harness.py @@ -101,8 +101,6 @@ def check_api_reachable(model: str | None = DEFAULT_MODEL, timeout: int = 60) -> if model: cmd += ["--model", model] - # Prompt goes over stdin (see `_run_agent` for why) -- consistent here even - # though this one is single-line. try: proc = subprocess.run( cmd, capture_output=True, text=True, encoding="utf-8", @@ -147,11 +145,6 @@ def _run_agent(prompt_text: str, workspace: Path, model: str | None, effort: str if effort: cmd += ["--effort", effort] - # Pass the prompt over stdin rather than as an argv string. On Windows, when - # `claude` resolves to a .cmd/.ps1 shim, a multi-line command-line argument - # is re-parsed by cmd.exe/PowerShell and truncated at the first newline. - # stdin is a raw byte stream and is immune to that on all platforms, so - # multi-line test prompts stay intact. proc = subprocess.run( cmd, cwd=str(workspace), capture_output=True, text=True, encoding="utf-8", input=prompt_text, env=_claude_env(),