A Hermes port of HeyCubit/effortless (MIT): a fast judge reads each prompt once per turn and picks the reasoning effort for it — easy questions run cheap, hard jobs get high — by rewriting the LLM request before it ships.
- An
llm_requestmiddleware fires once per user turn (keyed onturn_id), reads the next user message plus recent context, asks a judge, and rewrites every reasoning-effort carrier already present in the provider kwargs (reasoning_effort,extra_body.reasoning.effort,reasoning.effort). It never invents a carrier a provider doesn't use, never touchesmessages, and never changes the model — so the prompt-cache prefix stays intact. - On a retried request within the same turn the cached verdict is reapplied.
- A named level in the message ("max effort", "effort: high") wins without a judge; a short follow-up ("go", "ok") keeps the current effort; a judge that is unsure (<0.5) or down falls through to the fallback judge, and if that fails too the current effort is kept.
- Verdicts are logged (last 50) in profile-scoped plugin state and shown by
/effortlessin interactive sessions.
| Backend | Config | Notes |
|---|---|---|
jev (default) |
typesafe_key setting (stored as EFFORTLESS_HERMES_TYPESAFE_KEY in .env), or the shared TYPESAFE_API_KEY env var |
TypeSafe systemone, ~0.3 s, returns confidence |
local (default fallback) |
local_base_url (default http://127.0.0.1:8080/v1), local_model |
any OpenAI-compatible server; must be running |
custom |
custom_base_url, custom_model, custom_api_key (stored as EFFORTLESS_HERMES_CUSTOM_API_KEY in .env) |
any OpenAI-compatible API |
Secrets are read only through Hermes' profile-scoped secret store (agent.secret_scope),
never from .env files or os.environ directly. When no key is set, the jev judge is
skipped and the fallback chain runs; when nothing answers, the current effort is kept —
the plugin never blocks a prompt.
enabled (bool, true) · judge_backend (jev|local|custom) · fallback_backend (local|custom|jev|none)
· judge_timeout_seconds (3.0) · effort_floor (low) · effort_ceiling (max) ·
effort_bias (-2..+2) · model_map (dict, off) · model_map_always (bool, false) ·
judge connection keys per the table above.
A post_api_request observer hook records the first request of every turn — input
tokens, context length, duration, model — into the plugin's profile-scoped state.
/effortless shows the latest context fill; /effortless stats aggregates the recent
window: verdicts per effort, per judge, average judge latency, average input tokens and
context fill. There is deliberately no cost-savings estimate — what a prompt would
have cost at another effort is not knowable (same honesty rule as upstream effortless).
Cache warmth is not reported: the hook payload carries usage and timing, but no explicit
cache-hit field, and we won't guess one.
model_map lets a verdict steer the model, not just the effort, for multi-model
gateways:
plugins:
entries:
effortless-hermes:
settings:
model_map: { low: <cheap model id>, medium: <balanced id>, high: <strong id> }- Applies to the whole turn (reapplied on retries), never mid-turn.
- By default it only fires on a fresh chat (first user turn). In an established
chat, switching models re-reads the entire conversation at full price on the new
model — the provider prompt cache is keyed by model.
model_map_always: trueopts into that cost explicitly. - Model ids must be servable by the chat's own provider — the middleware swaps the model id on the request; it does not switch providers, keys, or endpoints.
- This is generic on purpose: any OpenAI-compatible router that serves several models behind one endpoint (a self-hosted router, an aggregator) can be driven by effort verdicts. No specific gateway is named or required.
/effortless— show the last verdicts/effortless test— send a sample prompt through both judges
- Network calls: with the default config the plugin sends your next message (up to
4000 chars) and a short excerpt of recent conversation (up to 1600 chars) to
https://api.typesafe.ai/v1/systemone(TypeSafe Jev) on the first request of each turn, solely to pick the effort. Thelocalbackend sends the same text to the base URL you configure (default127.0.0.1:8080); thecustombackend to your configured API. No other data leaves the machine; there is no telemetry, no usage reporting, no file access beyond the plugin's own state. - Read-only telemetry: a
post_api_requestobserver hook reads token usage and context length from Hermes' own hook payloads and stores them locally in plugin state for the/effortless statscommand. Nothing is sent anywhere. - Latency budget: each judge is bounded by
judge_timeout_seconds(default 3 s; local and custom judges allow up to 8 s). A judge that hangs is abandoned, not awaited. - Credentials: keys come from the settings hub (stored in Hermes'
.env) or theTYPESAFE_API_KEYenv var, resolved throughagent.secret_scope.get_secretso profile-scoped secrets stay scoped. The plugin never reads.envfiles itself and never writes credentials anywhere. - Code: pure-stdlib Python (urllib), no pip dependencies, no shell-outs, no background processes, no self-updating code.
Providers accept different reasoning_effort vocabularies, and some accept
different ones per route. The plugin clamps every effort it writes — and now
also every effort already on the request — to the vocabulary the current
provider/model pair accepts:
| Route | Accepted | medium maps to |
|---|---|---|
Surplus Intelligence (api.surplusintelligence.ai), agentic requests (any model) |
low, high, max |
high |
| glm-5.3 (direct), glm-4.5, deepseek | low–high/max incl. medium |
medium (legal) |
| kimi-k3 | low, high, max |
high |
| unknown | full ladder | medium (legal) |
Two behaviors worth knowing:
- Sanitize-before-ship: if the request already carries an effort the
provider would reject (a config-level
agent.reasoning_effort: mediumon a low/high/max route, for example), the middleware rewrites it to the nearest accepted level on every request — including follow-ups, judge-outage turns and retries — so a provider 400 ('reasoning_effort' must be one of: 'low', 'high', 'max') can't ship. - medium → high, not low: on restricted ladders a judged
mediummaps up tohigh. The judge called the prompt "normal work: a few tool calls";lowmeans "no tools, no planning", which would under-power it.
- 2026-10-14, Hermes v0.21.7, glm-5.3 via surplusintelligence:
"What day comes after Tuesday" →
reasoning_effort: lowin the live request dump (jev, 99% sure, 331 ms); "Refactor the entire auth system…" →high(82% sure); retry reapply confirmed via double dumps;hermes plugins validate --install-depsclean.
Per-prompt model routing (effortless keeps it off by default too), the desktop status bar
UI, the cache-countdown timer, handoff/compact triggers, and Anthropic thinking budget
carriers (providers without a text effort carrier are skipped, never guessed).
MIT. Judge prompts and heuristics adapted from HeyCubit/effortless (MIT).
-
0.3.1 (2026-10-17): Surp-hosted glm-5.3 clamps to low/high/max. Live-verified: api.surplusintelligence.ai accepts medium/xhigh on plain chat but 400s on agentic requests (tools attached or long agent-style history) with "'reasoning_effort' must be one of: 'low', 'high', 'max'". Wire vocabularies now take
base_url/tools, and cached per-turn efforts are re-clamped on retry re-apply instead of being pasted verbatim. -
0.4.0 (2026-10-18): outcome feedback loop. A quality-class failure (reason
incomplete_response/empty_response, orfinish_reason=error) escalates that session's effort one step for the next turn AND for retries of the failed turn (retries now ship more thinking, not the verdict that just failed). Bias caps at +2; two clean first-calls decay it one step. Rate limit, auth, billing and request-shape 400s never escalate — those are not thinking problems. Newapi_request_errorobserver registration, newfeedback_enabledsetting (default true),failure biasline in/effortless stats,finish_reasonrecorded per turn, and the per-session maps are now bounded (200 sessions) so long-lived gateways don't leak.