feat: add self-hosted LLM load test suite (ENG-1107)

[pandeymangg]

Summary

Load test scaffolding for the self-hosted Qwen/vLLM runtime that powers Formbricks AI (translation, chart generation, survey generation). Companion to scripts/embeddings/; mirrors its layout and operator workflow.

Implements ENG-1107. Blocked on ENG-1103 (model sizing) and ENG-1105 (EU EKS vLLM deployment) for the actual staging run — the suite itself is deployment-agnostic and runs the day staging is up.

What's exercised

Script	Workflow	Path
k6/llm-direct-translation.js	AI survey translation	direct → vLLM /v1/chat/completions
k6/llm-direct-surveygen.js	AI survey generation	direct → vLLM
k6/llm-direct-chartgen.js	AI chart generation	direct → vLLM
k6/web-survey-gen.js	Survey gen end-to-end	Formbricks web → vLLM
run-full-suite.sh mixed	Concurrent mixed traffic	three scenarios in parallel
run-cold-start.sh	Cold start / rollout restart	vLLM deployment timing

Three of the four direct-LLM scripts each test a different token shape — prefill-heavy (chart-gen), decode-heavy (survey-gen), and balanced (translation) — so vLLM is stressed across the workload profiles it'll see in production, not just one.

What's in the PR

• k6/ — four scenario scripts, profile-driven (smoke|baseline|mid|burst|stress), with per-request prompt_tokens / completion_tokens / tokens_per_sec / schema_valid / malformed_json / missing_keys metrics on top of k6's built-in latency + throughput.
• data/ — 37 hand-crafted fixtures. System prompts copied verbatim from apps/web/modules/ee/ai-translation/lib/translate-fields.ts and apps/web/app/api/v3/surveys/generate/prompt.ts so the LLM sees exactly what production sends.
• collectors/ — vLLM Prometheus /metrics scraper (queue depth, KV cache %, preemptions, token counters), in-pod nvidia-smi poller (GPU util/memory/temp/power), kubectl-based k8s metrics poller.
• run wrappers — single-scenario, full-suite, and cold-start. Prefer native k6, fall back to Docker.
• report/template.md — fill-in-the-blank report mirroring ENG-1107's "Required Outputs" and "Acceptance Criteria" 1:1, with a section→artifact mapping so the operator never has to figure out where a number lives.

How an operator will use this

# Env (against staging EKS)
export LLM_URL=http://localhost:8000/v1 LLM_API_KEY=... MODEL=Qwen/Qwen2.5-7B-Instruct
export RESPONSE_FORMAT=json_schema VLLM_METRICS_URL=http://localhost:8000/metrics
export FORMBRICKS_URL=https://app.staging.formbricks.com FORMBRICKS_API_KEY=fbk_... FORMBRICKS_WORKSPACE_ID=...
export NAMESPACE=formbricks-stage POD_SELECTOR='app.kubernetes.io/name in (vllm,formbricks-web)'

cd scripts/llm
PROFILE=baseline ./run-full-suite.sh translation
PROFILE=baseline ./run-full-suite.sh surveygen
PROFILE=baseline ./run-full-suite.sh chartgen
PROFILE=baseline ./run-full-suite.sh web
PROFILE=mid      ./run-full-suite.sh mixed
PROFILE=stress   ./run-full-suite.sh surveygen
./run-cold-start.sh cache-warm

cp report/template.md report/runs/<ts>/report.md   # fill from artifacts

Full README covers env vars, profiles, metrics, and per-run artifact layout.

Note on chart-gen and translation

Both are Next.js Server Actions, not public HTTP routes. We test the LLM behavior under their workload shapes (identical prompts, identical structured-output schemas) by hitting vLLM directly. Web-pod overhead for those two is captured indirectly via the always-on k8s-metrics.sh poller.

The web-mediated path is only exercised for survey-gen, which is the one workflow with a real HTTP route (POST /api/v3/surveys/generate). Worth filing a follow-up to expose chart-gen / translation as HTTP routes too — both for load-test coverage and for third-party integrations.

Test plan

• All k6 + lib JS files pass node --check
• All shell scripts pass bash -n
• All three fixture JSON files parse
• No leftover local-only references in README, comments, or scripts
• Once EKS staging vLLM exists: smoke run, then baseline on each scenario, then mixed, then fill report/template.md

[pandeymangg]

@coderabbitai pls review 🙏