README.md
Ragas provides metrics and tooling to evaluate retrieval-augmented generation and agent pipelines, so you can measure faithfulness, answer relevance and context quality.
pip install ragas
Explore / Ragas
explodinggradients
Ragas
Evaluation toolkit for your RAG and agent pipelines — faithfulness, relevance, and more.
15,390 1,637 Python Apache-2.0updated 5 months ago
Continue your stack
What teams reach for next — and why each earns a place beside Ragas. Ranked by curator confidence.
Pairs well with
LlamaIndex
Data framework for connecting custom data sources to LLMs — ingestion, indexing, retrieval.
Why it fitsTurns 'looks right' into a number — score LlamaIndex pipelines before you ship.
AutoGen
Multi-agent conversation framework for building LLM applications with cooperating agents.
Why it fitsScore the outputs of your multi-agent conversations for faithfulness and relevance.
PageIndex
Vectorless, reasoning-based RAG — builds a hierarchical tree index from long documents so an LLM retrieves by relevance instead of similarity. No chunking, no embeddings, no vector DB.
Why it fitsPageIndex is a retrieval pipeline; ragas measures its faithfulness and relevance so you can compare it against vector RAG on your own corpus.
Alternative to See all + compare →
deepeval
Pytest for LLM apps: 40+ research-backed metrics — G-Eval, RAG suite, agent task completion, hallucination — as unit tests you run in CI, judged by any LLM including local ones.
Why it fitsThe two default pip installs for LLM evaluation: Ragas is RAG-centric (faithfulness, relevance); DeepEval covers the same RAG suite plus agents, chatbots, safety metrics and pytest-style CI integration. Broader tool vs sharper tool.
giskard-oss
Giskard v3: modular Python evals and red-teaming for agentic systems — scenario-based checks with LLM-as-judge, plus an automatic vulnerability scanner across OWASP LLM Top-10 categories.
Why it fitsOverlapping LLM-evaluation job: ragas is the specialist for RAG pipeline metrics; Giskard v3 covers multi-turn agent scenarios with built-in checks and LLM-as-judge, with RAG evaluation still pending its v3 port.
agents-cli
CLI + agent-skills layer that turns your coding assistant into a Google Cloud agent-lifecycle expert: scaffold ADK projects, run and evaluate them, then deploy and publish to Gemini Enterprise.
Why it fitsagents-cli ships a full agent-eval suite — trace generation, metric grading, LLM-as-judge, failure-mode clustering, prompt optimization — which overlaps Ragas's job. Difference: Ragas is eval-only and framework-agnostic, while agents-cli's eval is one stage of a GCP-bound scaffold/deploy pipeline.