PromptMan: Native Desktop LLM Workbench
One prompt. Many models. Real numbers. Native Rust desktop LLM workbench with side-by-side streaming & token cost auditing.
Creator & Lead Rust Systems Architect
2026 – Present • Production v1.0.0
Open Source Systems Engineering
Choosing a model, tuning a prompt, and keeping spend under control are usually three disjointed tools and a messy spreadsheet. PromptMan unifies the workflow into a native desktop application written in Rust 1.95+ with egui. Runs OpenAI, Anthropic, Gemini, Groq, Mistral, and local engines (Ollama, LM Studio, vLLM, llama.cpp) side-by-side with live parallel token streaming, word-level diffing, judge model scoring, and hardware metrics (CPU, RAM, NVIDIA GPU sampling via nvidia-smi). Keys are securely stored in the OS credential store (Windows Credential Manager, macOS Keychain, Linux Secret Service) and runs are persisted in local SQLite. Includes promptman-cli for CI/CD test gates.
A high-performance native desktop workbench built in Rust & egui for LLM prompt engineering, live multi-model streaming comparison, automated CSV regression evaluation, and exact local/cloud token cost auditing. 0% Electron, 0% telemetry.
CHALLENGE & ARCHITECTURAL SOLUTION
Prompt engineers and software teams juggle multiple disparate web interfaces and fragile spreadsheets to compare model quality, calculate real token expenditure, and prevent regression bugs. Web-based playgrounds leak sensitive prompts to third parties, and Electron wrappers consume 500MB+ of system memory.
Engineered a native immediate-mode desktop application in Rust with sub-40MB memory footprint, zero web views, and zero background analytics. All telemetry, credentials, and conversation trees remain on the user's physical machine, accompanied by a headless CLI tool for CI test suites.
ARCHITECTURAL BLUEPRINTS
Immediate-Mode GPU Rendering with egui
Zero DOM overhead and zero Chromium instances. Immediate-mode rendering provides buttery 60fps responsiveness, instant state restoration, and lightweight multi-window management across Windows, macOS, and Linux.
Asynchronous Tokio Multi-Stream Engine
Concurrent HTTP Server-Sent Events (SSE) pipelines stream up to four model outputs in parallel, measuring time-to-first-token and throughput while applying word-level diff algorithms in real time.
OS-Level Keychain & Local SQLite Persistence
API keys are securely held in platform keychains (Windows Credential Manager, macOS Keychain, Linux Secret Service). All runs, batch audits, and prompt versions are cataloged locally in a single SQLite ledger (promptman.db).
Automated Evaluation Engine & CI/CD Runner
Enables regression testing across CSV datasets with regex, JSON validation, and substring checks. The standalone promptman-cli binary emits structured JSONL logs and standard exit codes for GitHub Actions pipelines.
THE SIX WORKBENCH MODULES
A unified desktop surface built on egui. Zero Electron, zero web views, zero Node runtime, zero accounts, and zero telemetry.
Iterative Prompt Drafting
Focused single-model conversations, editable message history, live token counter, concise reply toggles, memory sliding window, and instant Markdown export.
4-Way Parallel Streaming
Send one prompt to up to four models simultaneously. Real-time token streaming, word-level diffing between answers, and judge model scoring on a 0-10 scale.
Automated CSV Evaluation
Test prompt robustness over CSV rows. Built-in validation rules: contains, regex match, valid JSON syntax, and expected-column assertions with pass/fail rate grids.
Named Prompt Vault
Named, tagged, and version-controlled system prompts and templates. Instant one-click loading into the Composer with variable injection ({{variables}}).
Real Spend & Win Rates
Time-series expenditure tracking, token cost breakdown per model, latency curves, and empirical model win-rates calculated across head-to-head comparison runs.
Keychain & Ollama Bridge
OS keychain credential storage (never saved in plaintext files), local Ollama pull/preload/delete manager, and monthly budget warning caps with hard-stop controls.
The exact same Rust core engine powers a standalone headless CLI binary (promptman-cli) for automating prompt regression testing in GitHub Actions CI pipelines:
# Run automated prompt check in CI with JSONL output
$ promptman-cli --models openai/gpt-4o,ollama/llama3 --prompt "Capital of France?" --contains paris
$ promptman-cli --models mistral/mistral-small-latest --csv cases.csv --format jsonl > results.jsonl
CORE SYSTEM CAPABILITIES
Side-by-side parallel streaming across up to 4 cloud and local models simultaneously
Live time-to-first-token (TTFT), total latency, tokens-per-second, and exact cost readouts
Batch evaluation engine running CSV test cases with contains, regex, JSON schema, and expected column checks
Native compiled Rust 1.95+ & immediate-mode egui architecture with 0% Electron and 0% telemetry
Hardware telemetry sampling: real-time monitoring of CPU, RAM, and NVIDIA GPU usage during local inference
Cryptographic OS keychain integration: API keys stored in Windows Credential Manager, macOS Keychain, and Linux Secret Service
Integrated promptman-cli binary with exit codes and JSONL output for automated CI/CD pipeline gating
NEED SCALABLE ARCHITECTURE CRAFTED WITH RESTRAINT?
Whether you need to architect a high-concurrency web platform, build a native desktop application in Rust, or deploy production Android apps — let's build together.