Skip to content

MUL-6188: refresh the skill for CLI v0.4.26 and lint against drift - #2

Merged
Bohan-J merged 4 commits into
mainfrom
refresh-skill-for-cli-0.4.26
Aug 14, 2026
Merged

MUL-6188: refresh the skill for CLI v0.4.26 and lint against drift#2
Bohan-J merged 4 commits into
mainfrom
refresh-skill-for-cli-0.4.26

Conversation

@Bohan-J

@Bohan-J Bohan-J commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Refreshes the skill against CLI v0.4.26 (it was last touched in June) and adds a lint so it cannot silently rot again.

Prompted by multica-ai/multica#6968, which asked for an official Multica operator skill — this repo already is one, but it had drifted far enough that parts of it were misleading.

The drift that had teeth

Every command already documented is still valid — the damage was in what was missing.

Missing Why it mattered
--no-start on status / assign / update The skill had a whole section on "status changes enqueue work" but no way to avoid it. An agent recording progress on work already underway would start a duplicate run every time.
--compact The CLI calls it the recommended agent read. The skill taught the most token-expensive one instead.
Resolved-thread folding + --full Folding is on by default. An agent that doesn't know that reads it as data loss.
Working-directory restriction on file args (MUL-4252) --content-file rejects paths outside the cwd. An agent following habit and writing to /tmp just fails, with nothing in the skill to explain why.
--parent is mandatory for comment-triggered tasks The server rejects a top-level comment from one.
issue create --allow-duplicate Creation now refuses active duplicates; the skill didn't say so, or that the right response is to read the existing issue rather than retry with the flag.

Surface that was never documented

issue search, runs, run-messages, rerun, cancel-task, label, property, subscriber, usage — plus the top-level label, property, and chat namespaces. Also issue list --sort/--direction/--offset, --start-date, --attachment-id, and mention://project/.

New sections cover reading comments cheaply, finding things with issue search, suppressing runs with --no-start, and inspecting runs. There's also a short "When the CLI Cannot Do It" section — say what's unsupported and point at Web, rather than letting a partial run read as a complete one. That's the gap behind #6968's degradation request.

Keeping it from happening again

Documenting the surface once doesn't keep it documented. scripts/lint-skill-commands.py checks the docs against a real multica binary, in two directions:

  • validity — every documented command path and flag must exist. Notably, cobra prints the parent's help and exits 0 for an unknown subcommand, so issue frobnicate looks valid on exit status alone; the lint compares against the USAGE line instead.
  • coverage — every top-level namespace and every issue subcommand must be mentioned in SKILL.md. Deliberate omissions live in UNDOCUMENTED_OK with a reason (currently daemon, update, plugin, issue reorder).

CI runs it on push/PR and daily. The daily run is the real mechanism: the CLI moves under a repository that doesn't change, which is exactly how this drift happened.

Run it locally with:

scripts/lint-skill-commands.py --verbose

Review fixes (2nd commit)

All four blocking findings were real and are fixed:

  1. Version floor. Bisected the releases: 0.4.23/0.4.24/0.4.25 lack --no-start, 0.4.26 has it. SKILL.md and both READMEs now state v0.4.26 and tell the agent to stop and ask for an upgrade. The version lives in one place (MIN_CLI_VERSION) and the lint asserts the docs state it. CI now runs a [min, latest] matrix.
  2. Lint hole. resolve_path() returning None was treated as a root-level invocation, so multica frobnicate passed silently. Stray bare words are now separated from flag values and reported. Added scripts/test-lint.py covering unknown top-level command (bare and with a valid global flag), unknown subcommand, unknown flag, undocumented command, stale allowlist entry, and a no-false-positives case. Writing it immediately found a second bug: --profile=dev was compared with its value attached, so inline flag values were falsely reported.
  3. --parent. Restored to [--parent <comment-id>] in the reference, with the "required only for comment-triggered tasks" condition stated identically in SKILL.md and the Cursor rule.
  4. mention://member/. Corrected — it notifies a person. @all is now listed as a workspace-wide notification. Only issue and project mentions are inert.

Non-blocking items applied: permissions: contents: read, sha256 verification against checksums.txt before executing the downloaded binary (anchored to exactly one entry), actions/checkout pinned to a commit SHA, and the confirm-list narrowed so read-only agent/skill/autopilot queries no longer read as requiring confirmation.

Review fixes (3rd commit)

Both remaining blockers reproduced before fixing.

  1. Boolean global flags hid bad commands. stray_words() assumed any flag without = consumes the next token, so multica --debug frobnicate treated frobnicate as --debug's value. Real CLI: unknown command "frobnicate", exit 1. Lint: clean. The earlier regression case used multica frobnicate --debug — flag after the command — and so never exercised the flag-before-command order cobra also accepts.

    Flag arity is now read from the help text: cobra prints the value type between a flag and its description, so a flag with no type is boolean and consumes nothing. Added cases for --debug, -h, --help, -v, --version in that position, plus one asserting a value-taking flag does still consume its argument. 13 cases total.

  2. The Cursor rule had no version guard. It teaches --no-start, and both README and CURSOR.md offer it as a standalone copy into another project — so that distribution path could drift from the floor indefinitely. It now carries the same stop-and-upgrade boundary, CURSOR.md states the requirement, and both files joined VERSION_DECLARED_IN, so the guard covers every surface a user can install on its own.

Verification

  • Both CI jobs pass; checksum verification, the regression suite, and the version guard run in each.
  • Lint against v0.4.26: 109 documented invocations, 42 commands covered, all 5 declaring files verified.
  • Regression suite: 13 cases, all passing.
  • multica --debug frobnicate is now reported; confirmed the real CLI rejects it too.
  • Stripping the version string from the Cursor rule fails the lint (negative test).
  • Lint against v0.4.25 still fails with 7 --no-start errors — the floor remains enforced, not just asserted.
  • python3 -m py_compile, YAML parse, and git diff --check all clean.
  • Note: min and latest both resolve to v0.4.26 today, so the matrix does not yet exercise two different versions — the v0.4.25 run is what demonstrates the floor.
  • Manifests bumped to 1.1.0. Tagged releases are still not set up — that part of #6968 remains open.

The skill was last updated in June and the CLI has moved since. The
documented commands were all still valid, but the omissions had teeth:

- `--no-start` did not exist here at all, while the skill devoted a whole
  section to "status changes enqueue work". An agent recording progress on
  work already underway had no way to avoid starting a duplicate run.
- `--compact` (which the CLI calls out as the recommended agent read) and
  resolved-thread folding were both missing, so the skill taught the most
  expensive read and left an agent to misread folding as data loss.
- File arguments are now rejected outside the working directory (MUL-4252).
  An agent following habit and writing to /tmp just fails.
- `--parent` is mandatory for comment-triggered tasks, and `issue create`
  refuses active duplicates. Neither was mentioned.

Half the `issue` namespace was also undocumented: search, runs,
run-messages, rerun, cancel-task, label, property, subscriber, usage. The
top-level `label`, `property`, and `chat` namespaces were missing too.

Documenting the surface once does not keep it documented, so
scripts/lint-skill-commands.py checks the docs against a real binary:
every documented command and flag must exist, and every command the CLI
exposes must be mentioned in SKILL.md. Deliberate omissions go in
UNDOCUMENTED_OK with a reason. CI runs it on push and daily — the daily
run is the point, since the CLI moves under a repository that does not.

Also adds guidance for reporting what the CLI cannot do rather than
papering over it, which is the failure mode behind #6968.
…laims

All four blocking findings were real. Verified each before fixing.

1. No minimum CLI version was declared, while `--no-start` requires it.
   Confirmed by bisecting releases: 0.4.23/0.4.24/0.4.25 lack the flag,
   0.4.26 has it. SKILL.md and both READMEs now state v0.4.26 and tell the
   agent to stop and ask for an upgrade rather than improvise. The version
   lives in one place (MIN_CLI_VERSION) and the lint asserts the docs state
   it, so the floor cannot drift from the prose. CI now lints against both
   the declared minimum and the latest release: dropping the former lets the
   docs quietly raise the floor, dropping the latter lets the CLI move out
   from under them.

2. The lint accepted unknown top-level commands. `resolve_path()` returning
   None was treated as a root-level invocation like `multica --version`, so
   `multica frobnicate` passed silently. Now stray bare words are separated
   from flag values and reported. The earlier check only ever covered unknown
   *sub*commands, which is why this was missed.

   scripts/test-lint.py covers all of it: unknown top-level command (bare and
   with a valid global flag), unknown subcommand, unknown flag, undocumented
   command, stale allowlist entry, and a no-false-positives case. Writing it
   immediately found a second bug — `--profile=dev` was compared with its
   value attached, so inline flag values were reported as unknown flags.

3. `--parent` was written as globally required in the command reference,
   though the CLI requires it only for comment-triggered tasks. An agent
   following the reference would attach a new top-level comment to an
   arbitrary thread. Restored to `[--parent <comment-id>]`, with the
   condition stated the same way in SKILL.md and the Cursor rule.

4. `mention://member/` was described as an inert link; it notifies a person.
   Corrected in SKILL.md and the Cursor rule, and `@all` is now listed as a
   workspace-wide notification. Only issue and project mentions are inert.
   Pre-existing, but this PR rewrote that block.

Non-blocking items also applied: `permissions: contents: read`, sha256
verification of the release archive against checksums.txt before executing it
(anchored to one exact entry), actions/checkout pinned to a commit SHA, and
the confirm-list narrowed so read-only agent/skill/autopilot queries no longer
read as requiring confirmation.
…on guard

Both findings reproduced before fixing.

1. `multica --debug frobnicate` slipped through. stray_words() assumed any
   flag without `=` consumes the next token, so a boolean global flag ate the
   bad command name. Real CLI: `unknown command "frobnicate"`, exit 1. Lint:
   no errors. The previous regression case used `multica frobnicate --debug`,
   which puts the flag *after* the command and so never exercised the
   flag-before-command order cobra also accepts.

   Flag arity now comes from the help text: cobra prints the value type
   between the flag and its description, so a flag with no type is boolean and
   consumes nothing. Added cases for --debug, -h, --help, -v and --version in
   that position, plus a case asserting a value-taking flag still does consume
   its argument. 13 cases total.

2. The Cursor rule teaches `--no-start` but declared no minimum version, and
   CURSOR.md/README explicitly offer it as a standalone copy — so that
   distribution path could drift from the floor indefinitely. It now carries
   the same stop-and-upgrade boundary, CURSOR.md states the requirement, and
   both files are in VERSION_DECLARED_IN so the guard covers every surface a
   user can install on its own.

Verified: stripping the version from the rule now fails the lint; all five
declaring files pass; v0.4.25 still fails with the --no-start errors.
Elon's nit: the value-taking-flag case had a resolvable command after it
(`... issue list`), so resolve_path() succeeded and stray_words() — the
branch it meant to cover — never ran. Dropped the command so the fixture
reaches that branch, and added direct stray_words() assertions so the
coverage survives a future reshuffle of the fixture.

Mutation-checked: forcing takes_value=True (the original bug) now fails 6
cases; it previously failed none of the value-taking ones.
@Bohan-J
Bohan-J merged commit f391633 into main Aug 14, 2026
2 checks passed
@Bohan-J
Bohan-J deleted the refresh-skill-for-cli-0.4.26 branch August 14, 2026 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant