giskard-oss vs Ragas
Giskard v3: modular Python evals and red-teaming for agentic systems — scenario-based checks with LLM-as-judge, plus an automatic vulnerability scanner across OWASP LLM Top-10 categories. — versus — Evaluation toolkit for your RAG and agent pipelines — faithfulness, relevance, and more.
Overlapping LLM-evaluation job: ragas is the specialist for RAG pipeline metrics; Giskard v3 covers multi-turn agent scenarios with built-in checks and LLM-as-judge, with RAG evaluation still pending its v3 port.
| giskard-oss | Ragas | |
|---|---|---|
| Stars | 5.6k | 15k |
| Forks | 494 | 1.6k |
| Language | Python | Python |
| License | Apache-2.0 | Apache-2.0 |
| Last activity | today | 4 months ago |
| Topics | evals, security | evals, rag |
| Curated connections | 3 | 5 |
giskard-oss — the curator's take
One of the OG names in LLM testing, freshly rewritten: v3 drops the heavyweight monolith for focused packages — giskard-checks (scenario API for multi-turn agent evals: groundedness, conformity, LLM-judge) and giskard-scan (generates adversarial suites from a plain-language description of your agent, prompt-injection probes included). Async-first, wraps anything callable. Mind the transition, though: v3 is beta, RAG evaluation (RAGET) hasn't been ported yet, and v2 — where scan and RAG eval are mature — is no longer maintained. Python 3.12+ only, and libraries on giskard-core send aggregated telemetry (opt-out documented). Pick ragas for pure RAG metrics; Giskard earns its place on multi-turn agent scenarios plus red-teaming in one stack.
Ragas — the curator's take
Evaluation toolkit for your RAG and agent pipelines — faithfulness, relevance, and more.