RAG & Retrieval

nduckmink 1.2k · 2 months ago

Self-hosted enterprise knowledge hub + MCP server: an LLM pipeline compiles SOPs and docs into a traceable, human-reviewed wiki, then serves it to AI clients scoped by department and role.

chroma-core 29.1k · yesterday

Open-source embedding database for building AI apps with retrieval.

lumina-ai-inc 4.1k · 4 months ago

Document intelligence API in Rust: layout analysis, OCR with bounding boxes, and semantic chunking that turn PDFs, PPTs and Word docs into RAG-ready structured chunks.

cocoindex-io 11.4k · 2 days ago

Rust-core incremental indexing engine: declare Target = F(Source) in Python and it keeps vector/graph/relational targets fresh forever, reprocessing only the delta — with per-row lineage.

vitali87 4.7k · today

Parses a polyglot monorepo with Tree-sitter into a Memgraph knowledge graph: query it in plain English (NL→Cypher), trace data flow, find dead code, edit via AST-surgical patches.

cognis-digital 7 · 1 months ago

No-train, on-prem code knowledge graph served to AI agents over MCP — symbols, call edges, cross-language links and blast-radius queries, with a hash-chained audit log of every read.

unclecode 78.7k · today

The 74k-star LLM-native crawler: turns any site into clean, RAG-ready Markdown — adaptive crawling, JS rendering, extraction strategies, Docker deploy. Python, Apache-2.0.

stanfordnlp 37.4k · yesterday

Program — don't prompt — your language models. Compile declarative pipelines into optimized prompts.

NanoNets 3.8k · today

Context layer for large codebases: a graph of plain-English markdown nodes — no embeddings, no index — agents read like any repo file. Claude Code hooks + MCP; 42% fewer tokens in its bench.

yifanfeng97 3.3k · 8 days ago

Knowledge-extraction CLI: LLMs turn documents into structured graphs, hypergraphs and spatio-temporal knowledge — with an MCP server for agents and Obsidian vault export.

Kaelio 1.5k · 1 months ago

Self-improving context layer for data agents — ingests dbt/Looker/wikis, maps your warehouse, builds a semantic layer with approved metrics, and serves Claude Code/Codex via CLI and MCP.

run-llama 51.8k · yesterday

Data framework for connecting custom data sources to LLMs — ingestion, indexing, retrieval.

1Panel-dev 22.5k · today

Open-source enterprise agent platform: RAG pipelines (upload or crawl docs), a visual workflow engine with MCP tool-use, and zero-code embedding into existing business systems.

opendatalab 78k · yesterday

Heavyweight document-to-markdown/JSON parser — PDFs plus Office (docx/pptx/xlsx) through layout analysis and OCR into LLM-ready output for RAG and agentic pipelines. 73k stars, self-hostable.

morphik-org 3.7k · 28 days ago

Multimodal retrieval engine for visually rich documents: ingestion, visual-first search over charts, tables and diagrams, knowledge graphs and cache-augmented generation — one engine, not a pipeline.

allenai 19.4k · 4 months ago

Open toolkit that linearizes messy PDFs — scans, tables, equations, handwriting — into clean ordered Markdown with a self-hosted vision-language model. Built for LLM training data and RAG ingestion.

lfnovo 37.1k · 4 days ago

Self-hosted NotebookLM alternative: multi-modal sources, vector plus full-text search, context-aware chat and multi-speaker podcast generation — 18+ model providers incl. Ollama, full REST API.

opendataloader-project 28.6k · 2 days ago

Deterministic PDF parser for AI pipelines: #1 extraction accuracy (0.907) on its public bench, bounding boxes on every element, 0.015s/page — plus the first open PDF auto-tagging for accessibility.

VectifyAI 35.3k · today

Vectorless, reasoning-based RAG — builds a hierarchical tree index from long documents so an LLM retrieves by relevance instead of similarity. No chunking, no embeddings, no vector DB.

firecrawl 16.3k · yesterday

Firecrawl's Rust PDF triage: classifies text-based vs scanned in ~10-50ms, extracts positioned text and clean Markdown without OCR — routing the ~54% of PDFs that never needed a model.

Evokoa 1k · 4 days ago

PostgreSQL extension adding graph search, traversal and shortest-path over your existing tables — a derived graph index queried from plain SQL, no separate graph DB or query language. Rust.

StarTrail-org 9.6k · 20 days ago

Berkeley's visual RAG: render pages and PDFs to screenshot tiles and retrieve with a VLM embedder — tables, charts and layout survive. pixelshot CLI plus a hosted 8.28M-page Wikipedia index.

explodinggradients 15.4k · 5 months ago

Evaluation toolkit for your RAG and agent pipelines — faithfulness, relevance, and more.

hanxiao 52 · 1 months ago

Airgapped closed-corpus QA testbed: a local Qwen agent in a Pi harness explores a .zip dataroom with grep/embeddings/rerankers under a token budget — a bed to study search as test-time compute.

superlinked 2.8k · 10 days ago

Self-hosted inference cluster for everything agents call besides the big LLM: embeddings, rerankers, OCR, NER, guardrails and small LLMs — 100+ models, one OpenAI-compatible API, K8s stack included.

supavec 1.2k · 7 months ago

Open-source RAG-as-a-service (the Carbon.ai alternative): upload any data source, get vector search and a chat API in minutes — Supabase-based, multi-tenant with RLS, streaming responses.

MODSetter 16k · yesterday

Open-source competitive-intelligence platform for agents: live Reddit/YouTube/TikTok/Maps/search connectors; scheduled agents produce briefs and alerts into a cited knowledge base. REST + MCP.

bosun-ai 774 · yesterday

Rust framework for LLM apps: an agent harness, compile-time-typed task graphs, and streaming RAG pipelines — MCP toolboxes, human-in-the-loop approval, tracing with Langfuse support.

neuml 12.9k · yesterday

All-in-one AI framework around an embeddings database — dense, sparse, graph and relational fused — with pipelines, workflows, agents and MCP/web APIs. Python, bindings for JS/Java/Rust/Go.

baidu 24.2k · 22 days ago

Baidu's open OCR VLM that parses entire multi-page documents in one shot — 'unlimited' long-horizon parsing pushing DeepSeek-OCR further. MIT weights on HF; serve via transformers, vLLM or SGLang.

Zipstack 7.1k · today

LLM-driven platform turning unstructured documents into structured data: a no-code Prompt Studio to define extractions, then deploy as APIs or ETL pipelines. Self-hosted, AGPL + enterprise.

xberg-io 9.2k · yesterday

Rust-core document-intelligence engine with 15 language bindings: turns 96 formats — PDF, Office, images, audio, code — into clean text, tables and RAG-ready chunks. Library, CLI, REST or MCP.

alibaba 15.5k · today

Alibaba's open-source in-process vector database: billion-scale similarity search embedded in your app, with DiskANN on-disk indexing, native full-text search and hybrid retrieval.