Skip to content

Configure pstack with only the selected model providers #72

Closed

Description

@arjitj2

Setup currently probes all four baseline model families and requires every family in the final role map. A user with Codex or a subset of external subscriptions cannot configure pstack when an unused CLI such as Grok is unavailable.

Derive required probes from the operator's selected role assignments. Preserve complete role coverage and existing assignments, report failures for selected providers without silently substituting another model, and leave configuration unchanged until selected probes pass and the operator confirms.

Activity

  1. mast-factory commented on Sep 23, 2026

    @mast-factory
    Type feature request — Support a role-assignment-derived provider subset instead of requiring the baseline four-family setup profile.
    Route Plan fix
    Severity 🟡 medium — the supported setup path is blocked for users who intentionally lack any unused baseline provider CLI or subscription.
    Confidence high — the mandatory four-family gates are explicit in setup, tests, and the PR that introduced per-family effort selection.
    Effort medium — setup sequencing, role validation, alias handling, smoke behavior, structural tests, and user-facing docs must remain aligned.
    Impact medium — this unblocks a meaningful subset-provider workflow, while existing four-provider installations continue to work and manual configuration remains a risky workaround.
    Next step Plan around PR #73 as the focused implementation candidate, reconcile its alias-only/assignment-derived contract with #33 and PR #55, then require exact-head installed behavioral evidence before merge.

    Understanding

    This is a valid feature request rather than a regression. Merged PR #16 intentionally made effort reconfiguration deterministic by collecting and probing exactly Fable, Sol, Grok, and Opus, requiring every family in the final role map, and gating writes and smoke on all four. Those assumptions remain at plugins/pstack/skills/setup-pstack/SKILL.md:46, :50, :72, :112, and :118; plugins/pstack/skills/poteto-mode/scripts/runner/model-matrix.test.ts:24 and :55 structurally lock the same quartet. That conflicts with README.md:35, which says core workflows can run with fewer models.

    The requested direction should preserve the existing safety contract while changing its scope: normalize and retain a complete role map first, derive distinct concrete provider/model/effort probes from the resulting assignments, require a real inherited native-route probe for assigned aliases, write nothing until every required probe passes and the operator confirms, and smoke only configured descriptors without introducing an unassigned provider. Selected failures must require repair or explicit reassignment; silent substitution remains forbidden.

    Open PR #73 directly implements that contract across setup instructions, matrix tests, docs, changelog/notice, and a behavioral-test plan. Its head 7feabe1518ac680caa514e304d342d22733f12ca is directly based on current main, is GitHub-mergeable, and passed both automated reviews. Independent local verification at that exact head passed 159 Bun tests, strict TypeScript typechecks, manifest parsing, static invariants, and git diff --check. The remaining verification gap is live installed-candidate execution with actual provider CLIs; tests/setup-selected-providers.md is a plan, not evidence that those scenarios ran.

    Issue #33 and PR #55 overlap but are not exact duplicates. They model a non-empty concrete active-family set and treat alias-only configuration as invalid, while #72/PR #73 derive requirements directly from final role assignments and support alias-only sheets after native delegation validation. Planning should choose and document one coherent contract before either implementation lands.

    Assumptions

    Open questions

    Reproduction

    At current main commit de67e6b40511814171e5e4c8ad7af3b79f07c9ee:

    grep -nE 'Ask exactly four|Probe only the four|at least one descriptor from each matrix family|all four probes|all four chosen descriptors' \
      plugins/pstack/skills/setup-pstack/SKILL.md

    The command returns mandatory gates at lines 46, 50, 72, 112, and 118. Therefore a role map that omits an unused baseline family cannot complete supported setup even if every selected route works. The declarative setup contract reproduces the limitation without external credentials.

    At PR #73 head, bun run test passed 159 tests, bun run typecheck passed all strict TypeScript projects, and the repository static/manifest checks passed. Live Claude/Codex/Grok setup scenarios were not executed in this sandbox because provider credentials and installed harness state are not available.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions