Evals & Testing
Measure and monitor LLM/agent output quality.
google 5.7k · 2 days agoCLI + agent-skills layer that turns your coding assistant into a Google Cloud agent-lifecycle expert: scaffold ADK projects, run and evaluate them, then deploy and publish to Gemini Enterprise.
microsoft 60.5k · 4 months agoMulti-agent conversation framework for building LLM applications with cooperating agents.
confident-ai 17.7k · todayPytest for LLM apps: 40+ research-backed metrics — G-Eval, RAG suite, agent task completion, hallucination — as unit tests you run in CI, judged by any LLM including local ones.
confident-ai 2.6k · 8 days agoOpen-source red-teaming framework for LLM systems: 50+ vulnerabilities, jailbreak/injection/multi-turn attacks against agents, RAG pipelines and chatbots — plus guardrails. Runs locally.
future-agi 1.7k · todaySelf-hostable platform for the whole agent-quality loop: tracing, evals, simulations, datasets, guardrails and an LLM gateway — one feedback loop from prototype to production. Apache 2.0.
Giskard-AI 5.8k · yesterdayGiskard v3: modular Python evals and red-teaming for agentic systems — scenario-based checks with LLM-as-judge, plus an automatic vulnerability scanner across OWASP LLM Top-10 categories.
cmu-l3 276 · 3 days agoCMU framework that turns real software — browsers, IDEs, EMRs, CAD — into standardized agent environments: start the app, hand the agent a task, score it with automatic verifiers.
Trace, test and monitor LLM apps in production.
Tracer-Cloud 10.7k · todayOpen-source framework for AI SRE agents plus the RL training and evaluation environment they need — connect 60+ tools you already run and investigate incidents on your own infra.
explodinggradients 15.4k · 5 months agoEvaluation toolkit for your RAG and agent pipelines — faithfulness, relevance, and more.