MUL-6188: refresh the skill for CLI v0.4.26 and lint against drift - #2
Merged
Conversation
The skill was last updated in June and the CLI has moved since. The documented commands were all still valid, but the omissions had teeth: - `--no-start` did not exist here at all, while the skill devoted a whole section to "status changes enqueue work". An agent recording progress on work already underway had no way to avoid starting a duplicate run. - `--compact` (which the CLI calls out as the recommended agent read) and resolved-thread folding were both missing, so the skill taught the most expensive read and left an agent to misread folding as data loss. - File arguments are now rejected outside the working directory (MUL-4252). An agent following habit and writing to /tmp just fails. - `--parent` is mandatory for comment-triggered tasks, and `issue create` refuses active duplicates. Neither was mentioned. Half the `issue` namespace was also undocumented: search, runs, run-messages, rerun, cancel-task, label, property, subscriber, usage. The top-level `label`, `property`, and `chat` namespaces were missing too. Documenting the surface once does not keep it documented, so scripts/lint-skill-commands.py checks the docs against a real binary: every documented command and flag must exist, and every command the CLI exposes must be mentioned in SKILL.md. Deliberate omissions go in UNDOCUMENTED_OK with a reason. CI runs it on push and daily — the daily run is the point, since the CLI moves under a repository that does not. Also adds guidance for reporting what the CLI cannot do rather than papering over it, which is the failure mode behind #6968.
…laims All four blocking findings were real. Verified each before fixing. 1. No minimum CLI version was declared, while `--no-start` requires it. Confirmed by bisecting releases: 0.4.23/0.4.24/0.4.25 lack the flag, 0.4.26 has it. SKILL.md and both READMEs now state v0.4.26 and tell the agent to stop and ask for an upgrade rather than improvise. The version lives in one place (MIN_CLI_VERSION) and the lint asserts the docs state it, so the floor cannot drift from the prose. CI now lints against both the declared minimum and the latest release: dropping the former lets the docs quietly raise the floor, dropping the latter lets the CLI move out from under them. 2. The lint accepted unknown top-level commands. `resolve_path()` returning None was treated as a root-level invocation like `multica --version`, so `multica frobnicate` passed silently. Now stray bare words are separated from flag values and reported. The earlier check only ever covered unknown *sub*commands, which is why this was missed. scripts/test-lint.py covers all of it: unknown top-level command (bare and with a valid global flag), unknown subcommand, unknown flag, undocumented command, stale allowlist entry, and a no-false-positives case. Writing it immediately found a second bug — `--profile=dev` was compared with its value attached, so inline flag values were reported as unknown flags. 3. `--parent` was written as globally required in the command reference, though the CLI requires it only for comment-triggered tasks. An agent following the reference would attach a new top-level comment to an arbitrary thread. Restored to `[--parent <comment-id>]`, with the condition stated the same way in SKILL.md and the Cursor rule. 4. `mention://member/` was described as an inert link; it notifies a person. Corrected in SKILL.md and the Cursor rule, and `@all` is now listed as a workspace-wide notification. Only issue and project mentions are inert. Pre-existing, but this PR rewrote that block. Non-blocking items also applied: `permissions: contents: read`, sha256 verification of the release archive against checksums.txt before executing it (anchored to one exact entry), actions/checkout pinned to a commit SHA, and the confirm-list narrowed so read-only agent/skill/autopilot queries no longer read as requiring confirmation.
…on guard Both findings reproduced before fixing. 1. `multica --debug frobnicate` slipped through. stray_words() assumed any flag without `=` consumes the next token, so a boolean global flag ate the bad command name. Real CLI: `unknown command "frobnicate"`, exit 1. Lint: no errors. The previous regression case used `multica frobnicate --debug`, which puts the flag *after* the command and so never exercised the flag-before-command order cobra also accepts. Flag arity now comes from the help text: cobra prints the value type between the flag and its description, so a flag with no type is boolean and consumes nothing. Added cases for --debug, -h, --help, -v and --version in that position, plus a case asserting a value-taking flag still does consume its argument. 13 cases total. 2. The Cursor rule teaches `--no-start` but declared no minimum version, and CURSOR.md/README explicitly offer it as a standalone copy — so that distribution path could drift from the floor indefinitely. It now carries the same stop-and-upgrade boundary, CURSOR.md states the requirement, and both files are in VERSION_DECLARED_IN so the guard covers every surface a user can install on its own. Verified: stripping the version from the rule now fails the lint; all five declaring files pass; v0.4.25 still fails with the --no-start errors.
Elon's nit: the value-taking-flag case had a resolvable command after it (`... issue list`), so resolve_path() succeeded and stray_words() — the branch it meant to cover — never ran. Dropped the command so the fixture reaches that branch, and added direct stray_words() assertions so the coverage survives a future reshuffle of the fixture. Mutation-checked: forcing takes_value=True (the original bug) now fails 6 cases; it previously failed none of the value-taking ones.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refreshes the skill against CLI v0.4.26 (it was last touched in June) and adds a lint so it cannot silently rot again.
Prompted by multica-ai/multica#6968, which asked for an official Multica operator skill — this repo already is one, but it had drifted far enough that parts of it were misleading.
The drift that had teeth
Every command already documented is still valid — the damage was in what was missing.
--no-startonstatus/assign/update--compact--full--content-filerejects paths outside the cwd. An agent following habit and writing to/tmpjust fails, with nothing in the skill to explain why.--parentis mandatory for comment-triggered tasksissue create --allow-duplicateSurface that was never documented
issue search,runs,run-messages,rerun,cancel-task,label,property,subscriber,usage— plus the top-levellabel,property, andchatnamespaces. Alsoissue list --sort/--direction/--offset,--start-date,--attachment-id, andmention://project/.New sections cover reading comments cheaply, finding things with
issue search, suppressing runs with--no-start, and inspecting runs. There's also a short "When the CLI Cannot Do It" section — say what's unsupported and point at Web, rather than letting a partial run read as a complete one. That's the gap behind #6968's degradation request.Keeping it from happening again
Documenting the surface once doesn't keep it documented.
scripts/lint-skill-commands.pychecks the docs against a realmulticabinary, in two directions:issue frobnicatelooks valid on exit status alone; the lint compares against theUSAGEline instead.issuesubcommand must be mentioned inSKILL.md. Deliberate omissions live inUNDOCUMENTED_OKwith a reason (currentlydaemon,update,plugin,issue reorder).CI runs it on push/PR and daily. The daily run is the real mechanism: the CLI moves under a repository that doesn't change, which is exactly how this drift happened.
Run it locally with:
Review fixes (2nd commit)
All four blocking findings were real and are fixed:
--no-start, 0.4.26 has it.SKILL.mdand both READMEs now state v0.4.26 and tell the agent to stop and ask for an upgrade. The version lives in one place (MIN_CLI_VERSION) and the lint asserts the docs state it. CI now runs a[min, latest]matrix.resolve_path()returningNonewas treated as a root-level invocation, somultica frobnicatepassed silently. Stray bare words are now separated from flag values and reported. Addedscripts/test-lint.pycovering unknown top-level command (bare and with a valid global flag), unknown subcommand, unknown flag, undocumented command, stale allowlist entry, and a no-false-positives case. Writing it immediately found a second bug:--profile=devwas compared with its value attached, so inline flag values were falsely reported.--parent. Restored to[--parent <comment-id>]in the reference, with the "required only for comment-triggered tasks" condition stated identically inSKILL.mdand the Cursor rule.mention://member/. Corrected — it notifies a person.@allis now listed as a workspace-wide notification. Only issue and project mentions are inert.Non-blocking items applied:
permissions: contents: read, sha256 verification againstchecksums.txtbefore executing the downloaded binary (anchored to exactly one entry),actions/checkoutpinned to a commit SHA, and the confirm-list narrowed so read-onlyagent/skill/autopilotqueries no longer read as requiring confirmation.Review fixes (3rd commit)
Both remaining blockers reproduced before fixing.
Boolean global flags hid bad commands.
stray_words()assumed any flag without=consumes the next token, somultica --debug frobnicatetreatedfrobnicateas--debug's value. Real CLI:unknown command "frobnicate", exit 1. Lint: clean. The earlier regression case usedmultica frobnicate --debug— flag after the command — and so never exercised the flag-before-command order cobra also accepts.Flag arity is now read from the help text: cobra prints the value type between a flag and its description, so a flag with no type is boolean and consumes nothing. Added cases for
--debug,-h,--help,-v,--versionin that position, plus one asserting a value-taking flag does still consume its argument. 13 cases total.The Cursor rule had no version guard. It teaches
--no-start, and both README and CURSOR.md offer it as a standalone copy into another project — so that distribution path could drift from the floor indefinitely. It now carries the same stop-and-upgrade boundary, CURSOR.md states the requirement, and both files joinedVERSION_DECLARED_IN, so the guard covers every surface a user can install on its own.Verification
multica --debug frobnicateis now reported; confirmed the real CLI rejects it too.--no-starterrors — the floor remains enforced, not just asserted.python3 -m py_compile, YAML parse, andgit diff --checkall clean.minandlatestboth resolve to v0.4.26 today, so the matrix does not yet exercise two different versions — the v0.4.25 run is what demonstrates the floor.