Braintrust
Platform for tracing agents in production, running evals and turning real traffic into regression tests
Braintrust is an observability and evaluation platform for AI agents. It captures agent traces in real time, including prompts, responses and tool calls, finds recurring patterns in production and turns them into evals, so teams can raise quality with each release and catch regressions.
Key features:
- Inspect logs and individual traces in real time.
- Score outputs with LLM judges, autoevals, custom code or human review.
- Turn production traces into datasets for regression tests.
- Iterate on prompts in playgrounds and run experiments.
- Run evaluations and generate test cases with Loop, a built-in AI agent.
- Track latency, cost and quality on Brainstore, a database built for AI traces.
Braintrust has native SDKs for Python, TypeScript, Go and other languages, with on-prem or hosted deployment on Enterprise. The free Starter plan includes $10 of model credits, 1 GB of processed data and 10k scores a month with unlimited users; Pro is $249/month.