fix(api): advertise all enabled llm slots in discovery, not just loaded ones#1153
Merged
Conversation
…ed ones Both hal0_slot_alias_models (/v1/models) and _collect_chat_slots (Hermes provision) filtered to only warm slots — but dispatch already cold-loads on demand when a request addresses a cold slot by alias. This mismatch hid code, nano, and other cold slots from both Hermes and pi-coder even though addressing them by name would auto-load them. Removes the loaded-model-id check from hal0_slot_alias_models (and the now-unused _loaded_model_ids helper) and the _is_ready filter from _collect_chat_slots. Both surfaces now advertise every enabled llm slot with a configured model — same criteria as hal0_chat_slot_alias_map. Tests: 11 slot-alias/hermes-collect tests pass; 88 hermes_provision pass.
The canonical hal0/agent, hal0/utility, hal0/npu live-resolve virtual names were injected into /v1/models alongside the per-slot alias entries, creating duplicate model entries for the same slots. Since the slot aliases (agent, utility, npu, etc.) are now discoverable via hal0_slot_alias_models, the separate live-resolve advertisement is redundant. Dispatch still resolves the canonical names — this only removes them from the discovery catalog. Hermes uses model.default: "hal0/agent" in its config, which routes through the dispatch LiveSlotResolver, not through model discovery.
thinmintdev
force-pushed
the
fix/discovery-cold-slot-visibility
branch
from
28b982c to
4e63c51
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Both
hal0_slot_alias_models(/v1/modelscatalog) and_collect_chat_slots(Hermes provision) filtered to only warm slots — but dispatch already cold-loads on demand when a request addresses a cold slot by alias. This mismatch hidcode,nano, and other cold chat slots from both Hermes and pi-coder, even though addressing them by name would auto-load them.Root cause: the "only advertise what's loaded" rule was written before the alias dispatch path gained cold-start loading. The alias map (
hal0_chat_slot_alias_map) already included every enabled llm slot with a model — including cold ones. Only discovery lagged behind.Fix
Two surfaces, same rule — advertise every enabled
type=="llm"slot with a configured model, same criteria ashal0_chat_slot_alias_map:hal0_slot_alias_models: removed the_loaded_model_idshelper and themodel_id not in loadedfilter_collect_chat_slots: removed the_is_ready()gateTests