Prompt Management
Version, manage, and collaborate on your AI instructions. Prompts in EvalsHub are versioned so you can run experiments and A/B tests against specific versions and track quality over time.
What is a prompt?
A prompt is a named template (e.g. "Customer support reply", "Search summarization") that has one or more versions. Each version has a system prompt, model, and optional settings. When you run an experiment or an A/B test, you select a prompt and (for experiments) a specific version. Traces from your app can be attributed to a version via the SDK (promptVersionId), so you can compare scores and latency per version in the dashboard.
Versioning
When you edit a prompt (e.g. change the system message or model), EvalsHub can create a new version instead of overwriting. That way you keep a history of what was deployed and can run experiments comparing old vs new. Use versions to:
- Run experiments on a specific version before shipping.
- A/B test two versions in production (see A/B Testing).
- Attribute production traces to a version so the dashboard shows quality and latency per version.
Variables and templates
Prompt text can include variables (e.g. {{input}}, {{context}}) that are filled at runtime from your dataset or application. When you run an experiment, dataset rows supply the values: the column mapped to "Input" (and any other mapped columns) are available as variables. In the app, you configure how dataset columns map into the prompt template so each row gets a fully resolved prompt.
Where to manage prompts
Create and edit prompts and versions from your project in the dashboard. You choose the model and provider, write the system prompt (and any user-prompt template), and save. That prompt is then available when creating experiments or configuring A/B tests. For production traffic to be attributed to a version, pass promptVersionId in the SDK options (see SDK Reference).