How is Confident AI different from DeepEval?
DeepEval is Confident AI's open-source evaluation framework for running LLM tests locally or in CI. Confident AI is the cloud platform that builds on DeepEval, adding collaboration features, dataset management, tracing, real-time monitoring, and dashboards for team-wide use.
Does Confident AI offer LLM observability?
Yes, Confident AI captures every LLM call as a trace, providing full context including inputs, outputs, tool calls, latency, token cost, and metadata. Users can drill into production requests, set up alerts for quality degradation, and monitor trends over time.
Can Confident AI be self-hosted?
Yes, Confident AI offers a fully self-hosted deployment option in addition to its managed cloud service. This allows users to run the entire platform within their own VPC or on-prem infrastructure, ensuring all data remains within their network. Self-hosting is available with the Enterprise plan.
Is there a free trial for paid plans?
Confident AI offers a Free tier with generous limits that is available indefinitely. For Starter and Premium plans, users can begin with the Free tier and upgrade when ready, without requiring a credit card to get started.
Can I use Confident AI in CI/CD pipelines?
Yes, DeepEval integrates directly into CI pipelines, allowing teams to run regression tests on every pull request. If quality metrics fall below defined thresholds, the build can fail, preventing low-quality prompts from reaching production environments.