Red Teaming
Stress test your models against adversarial attacks and ensure safety across your applications. EvalsHub automates safety evaluation with adversarial datasets and dedicated scorers.
What is Red Teaming?
Red teaming is the practice of intentionally trying to make your AI model fail or behave inappropriately—e.g. following malicious instructions, leaking data, or producing harmful content. In EvalsHub, this is automated through specialized adversarial datasets and safety scorers that test for prompt injection, jailbreaking, toxicity, and other failure modes. Running these evals regularly helps you ship with confidence and catch regressions before they reach users.
Safety categories
EvalsHub groups safety tests into categories so you can track which areas are strong or weak.
Prompt Injection
Attempts to bypass or override your system instructions via hidden commands, nested prompts, or role-play. Scorers check whether the model obeyed the intended task.
Data Leakage
Tests whether the model reveals sensitive internal information (e.g. system prompt, training data, or PII) when prompted or pressured.
Toxicity
Checks for offensive, hateful, or harmful content in model outputs. Useful for user-facing or open-ended applications.
Hallucination / Factuality
Verifies that the model stays grounded to provided context or facts and does not invent or contradict references.
How to run a Red Team eval
Create an experiment like any other: choose a Red Team (adversarial) dataset—or use a built-in one—and attach safety scorers. Run the experiment on your prompt version. Results show per-row pass/fail and scores so you can see which attacks succeeded and fix prompts or add guardrails. You can run these manually before release or schedule them in CI so every change is checked.
Automated scans and alerts
You can schedule recurring Red Team scans (e.g. nightly or on deploy) so new vulnerabilities are caught as models or prompts change. When a scan detects a high-severity failure, EvalsHub can trigger webhooks to notify your security team or runbooks. Optionally, you can configure the system to flag or temporarily disable the affected prompt version until the issue is addressed.