Feature  ·  Simulations

Test your AI app against real data before deploying.

Run up to 100 scenarios per batch against named datasets. Turn production failures into regression tests in one click.

Prompt, HTTP, and conversation scenarios. Batch runs. Trace-to-dataset conversion.

Start free →View docs
Zespan Simulations
Works withPrompt scenariosHTTP scenariosConversation scenariosLLM evaluatorBatch runsTrace-to-dataset

100

scenarios per batch

500

items per dataset

3

scenario types

1.0  Simulations

3 Scenario Types

What you get

Prompt: run a template against each dataset itemHTTP: call an endpoint and assert on the responseConversation: multi-turn simulation with assertions per turn

Prompt scenarios run a template against input items. HTTP scenarios call an external endpoint and evaluate the response. Conversation scenarios simulate multi-turn exchanges end to end. Attach a default dataset and custom LLM evaluator per scenario.

Zespan simulations view showing scenario list with types, statuses, and evaluator config

2.0  Simulations

Datasets

What you get

Named datasets: per-project, browsable, deletableItems: input, expectedOutput (optional), metadata (optional)500 items per call — bulk import supported

Named datasets hold your test inputs with optional expected outputs and metadata. Add up to 500 items per call. Datasets persist per project and can be reused across multiple simulation scenarios.

Zespan dataset view showing named datasets with item counts

3.0  Simulations

Trace-to-Dataset

What you get

From Trace Explorer: select traces → 'Add to dataset'Mark expected output: annotate what the correct answer should have beenInstant regression suite: production failures become test cases automatically

Select any production trace and add it to a dataset in one click. Turn real failures, edge cases, and high-cost outliers into regression tests without copy-pasting. Build test coverage from incidents as they happen.

Zespan trace-to-dataset conversion with production trace selected

4.0  Simulations

Batch Runs & Progress

What you get

Up to 100 scenarios per batch, dataset-fedReal-time progress: refreshBatchRunProgress for live statusExperiment labels: tag batches for grouping and head-to-head comparison

Run up to 100 scenarios in a single batch with a full dataset as input. Runs execute asynchronously with real-time progress tracking. Tag batches with experiment labels for comparison across changes.

Setup

Under 5 minutes,
two lines of code.

No forking and no architecture changes. Traces appear within seconds of the first agent run, with cost attribution, eval scores, and anomaly alerts on by default.

typescript
// Create a dataset via API
import { Zespan } from '@zespan/sdk';
const lt = new Zespan({ apiKey: process.env.ZESPAN_API_KEY });

await lt.datasets.addItems('my-regression-suite', [
  { input: 'How do I reset my password?', expectedOutput: '...' },
  { input: 'Cancel my subscription', expectedOutput: '...' },
]);

// Or create from production traces in one click in the UI:
// Trace Explorer → select traces → "Add to dataset"

Common questions

What assertions can I configure per scenario?

Each scenario supports: contains (output must include a string), not_contains (output must not include a string), regex (output matches a pattern), and max_latency_ms (response must arrive within N milliseconds). You can also attach a custom LLM evaluator for scoring beyond simple assertions.

How do I run a simulation before deploying a prompt change?

Create a dataset from production traces (or manually). Set up a prompt scenario with the new prompt version and your regression dataset. Run the batch — if any assertions fail or eval scores drop, you see it before any code ships. This is your LLM CI/CD pipeline.

What's the difference between a simulation run and an evaluation run?

Simulation runs test a specific scenario against your own application endpoint or prompt template — they're end-to-end tests you control. Evaluation runs score existing traces using an LLM judge — they measure quality after the fact. They're complementary: simulations for pre-deploy testing, evaluations for ongoing production monitoring.

Can I run simulations against a live HTTP endpoint?

Yes. HTTP scenarios call any URL you configure, send the dataset item as input, receive the response, and evaluate it with your assertion config or custom LLM evaluator. Useful for testing a staging environment before promoting to production.

Explore more features

All features →
TracingSee exactly what your agents are doing.Every LLM call, agent step, and tool invocation captured as a structured trace — with cost, latency, and tokens per span.Agent MonitoringKnow which agents are healthy, and which aren't.Composite health scores, delegation graphs, and per-agent cost attribution — built for systems with many cooperating AI agents.EvaluationsMeasure output quality on every trace, automatically.12 built-in LLM-as-judge templates run on every new trace with no setup. Track quality trends, catch regressions, and run manual eval campaigns.GuardrailsStop bad outputs before they reach users.7 guardrail types run inline on every LLM request — block, warn, redact, or log. PII, toxicity, topic drift, format, cost ceiling, and custom rules.

Your agents are running.
Do you know what they’re doing?

Observe, evaluate, guard, and control every agent, live in minutes. No credit card required.

Start free →Talk to us →
Free tier availableUnder 5 min setupNo credit card