PromptDeck is a local, open-source desktop studio for prompting and benchmarking LLMs side-by-side — fully offline, with no data ever leaving your machine.
Run the same prompt against multiple local models — via Ollama, LM Studio, or a llama.cpp server — watch them stream in real time, and compare Tokens/sec (TPS), Time-To-First-Token (TTFT), and total duration, with results saved locally in SQLite for later review.
-
🧭 App Shell workspace — icon-rail sidebar (Bench, Tests, Cloud, History, Board, Config) plus a slim 48px TopBar with a provider connection pill (status dot, server URL, model count). Result actions live in a contextual toolbar that only appears when there are results, so model outputs always get maximum space
-
☁️ Cloud providers — manage OpenAI-compatible endpoints 9Router-style: curated catalog (OpenRouter, DeepSeek, Groq, Together AI, OpenAI, xAI Grok, Kimi, NVIDIA NIM, Ollama Cloud, Gemini) split into Free Tier / API Key sections, custom endpoints, named API keys per provider, and a detail page with connection testing plus model picking (search, per-model 1-token probe, enable toggles)
-
⚡ Compare 2 to 4 models simultaneously — run the same prompt across local and cloud models at once and watch them stream side-by-side
-
🧪 Test Suite (batch execution) — run a list of 1–10 test prompts across all selected models sequentially and compare them in a scoreboard matrix: per-cell TPS, 👑 crown for the fastest model on each test, average-TPS + wins summary, click-to-read full outputs, Markdown copy, and CSV download
-
🔌 Multi-provider support — connect to Ollama, LM Studio, or any
llama.cppserver (OpenAI-compatible API) -
📊 Live metrics & visual charts — TPS, TTFT, and total duration tracked per run, with bar charts comparing every model at a glance (
~marks estimated TPS when the server doesn't report token usage)
-
🏆 Winner votes & leaderboard — crown the best answer per run and track win rate plus average speed per model over time
-
🔍 Side-by-side diff view — compare any two outputs line-by-line with similarity percentage and added/removed counts
-
🕘 Local history — every benchmark run is saved to SQLite and browsable from a sidebar, with winner crowns,
~markers for estimated TPS, and full theme support
-
📝 Markdown rendering — model outputs render with full GFM support (tables, code blocks, etc.)
-
💾 Savable prompt presets — save your own frequently-used prompts alongside the built-in quick presets
-
📄 Rich PDF export — export any benchmark run as a fully-formatted PDF report, preserving Markdown rendering (tables, code blocks, bold text)
-
⌨️ Keyboard shortcuts —
Ctrl+Enter(orCmd+Enter) to run,Escto stop generation or close panels -
⚙️ Settings panel — provider & connection, appearance, language, and advanced model controls (system prompt, temperature, top-p, context length) in a slide-over panel, all persisted across restarts
-
🌑 Three themes — Light, Dark, and AMOLED Dark (pure-black
#000000canvas with neon glows). Cycle from the header or pick explicitly in settings
-
🌍 Language picker — English, Deutsch, and فارسی (full RTL support) in a grid-style popup, opened from the header or settings
-
📋 One-click export — copy a Markdown report or full JSON history to clipboard, or download history as CSV
-
🔒 100% local & private — built with Tauri, no telemetry, no cloud calls
| Layer | Technology |
|---|---|
| Shell | Tauri 2 (Rust) |
| Frontend | React + TypeScript + Vite |
| Styling | Tailwind CSS v4 |
| State | Zustand (persisted) |
| Database | SQLite (via tauri-plugin-sql) |
| Charts | Recharts |
| PDF Export | jsPDF + html2canvas |
| i18n | react-i18next |
| Markdown | react-markdown + remark-gfm |
Before running PromptDeck, make sure you have at least one local LLM backend running:
- Ollama — running locally (
ollama serve), with at least one model pulled:ollama pull llama3
- or LM Studio — with a model loaded and the local server started (Developer tab → Start Server)
- or a
llama.cppserver — running with its OpenAI-compatible endpoint enabled
You'll also need:
# 1. Clone the repo
git clone https://github.com/Cadman021/prompt-deck.git
cd prompt-deck
# 2. Install dependencies
npm install
# 3. Run in development mode
npm run tauri devTo build a production binary for your OS:
npm run tauri build- Launch the app and pick your provider (Ollama, LM Studio, or llama.cpp) from the Config panel — PromptDeck auto-detects available models
- For cloud models, open the Cloud page, add an API key on a provider, refresh its models, and enable the ones you want to use
- On the Bench page, add up to 4 model columns (local and cloud mixable), write a prompt (or pick/save a preset)
- Hit Run (or
Ctrl+Enter) — all selected models stream their responses live - Compare TPS / TTFT instantly via the chart, crown a winner, open the Diff view for any two outputs, or check the Leaderboard for long-term stats
- Need a deeper comparison? Switch to the Tests page, stack up to 10 prompts, hit Run suite, and read the scoreboard to see which model wins across the whole batch
- Revisit past runs from the history sidebar, or export the whole report as PDF, Markdown, JSON, or CSV
- Cloud providers — catalog, API keys, model picking, and benchmarking in Bench + Test Suite
- Cost/pricing estimation for cloud runs (per-run spend, usage totals)
- OS-keychain storage for API keys (currently local persisted store)
- Test Suite — batch execution with scoreboard matrix
- AMOLED Dark theme (+ Light / Dark)
- Side-by-side diff view between two model outputs
- Export benchmark history as CSV
Contributions and ideas are very welcome — see Contributing below.
Issues and PRs are welcome! If you'd like to add a feature or fix a bug:
- Fork the repo
- Create a branch (
git checkout -b feature/my-feature) - Commit your changes
- Open a Pull Request
Please open an issue first for larger changes so we can discuss the approach. See CONTRIBUTING.md for more details.
This project is licensed under the MIT License.
If you find PromptDeck useful, consider giving it a star — it helps others discover the project and motivates continued development!