fix(config): cap auto-derived context to fit VRAM

[localai-bot]

Symptom

When a model is imported without an explicit context_size, LocalAI defaults the model's context to the GGUF's full trained window (n_ctx_train). For long-context models (128k / 256k / 1M) that KV cache cannot fit the GPU and the backend aborts on load (exitCode=-1), even though the model file is fine.

Reproduced live on a 20 GB card:

• gemma-4-26b-a4b-it-qat-q4_0 defaulted to context=262144 → abort
• qwythos-9b-claude-mythos-5-1m defaulted to context=1048576 → abort

Root cause

core/config/gguf.go's guessGGUFFromFile set cfg.ContextSize to f.EstimateLLaMACppRun().ContextSize — the model's max trained context, unbounded — whenever the user left context_size unset.

The fix

Auto-derive a conservative default instead of chasing the trained max:

1. Base cap (always applied): min(trainedMax, DefaultAutoContextSize) where DefaultAutoContextSize = 8192. A small model (trained < 8k) keeps its trained window; a long-context model caps at 8k. Users opt into more via an explicit context_size. This applies on every host, including CPU / unknown-VRAM, so it never regresses those paths.
2. VRAM as a downward-only safety: when a per-device VRAM ceiling is detected (xsysinfo.MinPerGPUVRAM, the smallest card — matching hardware_defaults.go's localGPU) and even the 8k cap would not fit it with ~20% headroom, step down through candidate contexts to the largest that fits, floored at DefaultContextSize (4096). The per-context footprint is estimated with gpustack/gguf-parser-go's EstimateLLaMACppRun (all layers offloaded, per-device NonUMA figure).

VRAM-unknown fallback rationale

If per-device VRAM is unknown (0) or no GPU is detected, we do not clamp — the bug is GPU OOM-on-load, and the 8k base cap is already safe. Clamping on a missing/zero reading would risk shrinking the window on CPU / unified-memory / detection-gap hosts, so those paths are deliberately left at the base cap. This bounds the blast radius of the change.

Explicit context_size always wins (guessGGUFFromFile only acts when it is nil).

Files changed

• core/config/defaults.go — new DefaultAutoContextSize = 8192 constant (with WHY comment).
• core/config/gguf.go — guessGGUFFromFile now routes the auto-derived context through autoContextSize.
• core/config/context_fit.go — new: autoContextSize, the headroom fit check, and the injectable perDeviceVRAM / estimateContextVRAM seams.
• core/config/context_fit_internal_test.go — new Ginkgo v2 specs.

The estimate and VRAM detection are package vars so tests inject deterministic values (mirrors the localGPU seam in hardware_defaults_internal_test.go).

Test coverage (Ginkgo v2 + Gomega)

• long-context model + ample VRAM → capped at DefaultAutoContextSize (8192), not the trained max;
• small model (trained 4096) → kept at 4096, not inflated;
• big model + tiny VRAM where 8k does not fit → below 8192, ≥ DefaultContextSize, and the chosen context's footprint actually fits VRAM with headroom;
• nothing fits → floors at DefaultContextSize;
• VRAM unknown (0) → unchanged (8192 base cap);
• explicit context_size set → guessGGUFFromFile leaves it untouched.

go test ./core/config/... → 276 specs pass. golangci-lint run ./core/config/ → 0 issues.

Notes

• Scoped to core/config (gguf.go + new context_fit.go + defaults.go); does not touch EffectiveBatchSize, PhysicalBatchForContext, largeContextForDevice, or ApplyHardwareDefaults, to avoid conflicts with the concurrent embedding-batch VRAM-cap PR.
• The local pre-commit coverage gate was skipped at the maintainer's request (core.hooksPath=/dev/null); CI runs the full suite.
• DCO: only the human maintainer certifies DCO, so this commit carries no Signed-off-by / Co-Authored-By — please sign off on merge.