RAG & Retrieval
Connect LLMs to your data — ingestion, indexing, retrieval.
nduckmink 1.5k · 4 months agoSelf-hosted enterprise knowledge hub + MCP server: an LLM pipeline compiles SOPs and docs into a traceable, human-reviewed wiki, then serves it to AI clients scoped by department and role.
chroma-core 29.4k · 5 days agoOpen-source embedding database for building AI apps with retrieval.
lumina-ai-inc 4.2k · 22 days agoDocument intelligence API in Rust: layout analysis, OCR with bounding boxes, and semantic chunking that turn PDFs, PPTs and Word docs into RAG-ready structured chunks.
cocoindex-io 11.6k · 3 days agoRust-core incremental indexing engine: declare Target = F(Source) in Python and it keeps vector/graph/relational targets fresh forever, reprocessing only the delta — with per-row lineage.
vitali87 5.2k · 3 days agoParses a polyglot monorepo with Tree-sitter into a Memgraph knowledge graph: query it in plain English (NL→Cypher), trace data flow, find dead code, edit via AST-surgical patches.
cognis-digital 7 · 2 months agoNo-train, on-prem code knowledge graph served to AI agents over MCP — symbols, call edges, cross-language links and blast-radius queries, with a hash-chained audit log of every read.
unclecode 84.4k · 6 days agoThe 74k-star LLM-native crawler: turns any site into clean, RAG-ready Markdown — adaptive crawling, JS rendering, extraction strategies, Docker deploy. Python, Apache-2.0.
stanfordnlp 38.4k · 3 days agoProgram — don't prompt — your language models. Compile declarative pipelines into optimized prompts.
trailhq 9.3k · 3 days agoContext layer for large codebases: a graph of plain-English markdown nodes — no embeddings, no index — agents read like any repo file. Claude Code hooks + MCP; 42% fewer tokens in its bench.
HelixDB 6.1k · 3 days agoRust OLTP graph database on object storage with native vector and BM25 search: property graph, traversal-prefiltered ANN and full-text in one transactional engine; Rust, TS, Go, Python SDKs.
yifanfeng97 4k · 3 days agoKnowledge-extraction CLI: LLMs turn documents into structured graphs, hypergraphs and spatio-temporal knowledge — with an MCP server for agents and Obsidian vault export.
jordan-gibbs 3.7k · 6 days agoDeep-research harness for Claude Code: a 16-step, tier-adaptive pipeline with adversarial critics, cite-checking and 250+ sources per run, every source kept in a persistent markdown+SQLite vault.
rahulnyk 4.1k · 1 months agoNotebook recipe that turns any text corpus into a concept graph with a local Mistral 7B via Ollama — chunk, extract concepts and relations, add proximity edges — for Graph RAG and KG QA.
Kaelio 1.6k · 20 days agoSelf-improving context layer for data agents — ingests dbt/Looker/wikis, maps your warehouse, builds a semantic layer with approved metrics, and serves Claude Code/Codex via CLI and MCP.
jeffhajewski 694 · 8 days agoEmbedded single-file graph database in Zig: graph traversal, HNSW vector search and BM25 full-text in one query language, plus a durable event log — built for Graph RAG and local agent memory.
run-llama 12.8k · todayLlamaIndex's local document parser in Rust: PDFium text with bounding boxes at ~2-5 ms/page, selective Tesseract or HTTP OCR, Markdown/JSON output, screenshots; Python, Node, WASM.
run-llama 52.3k · 4 days agoData framework for connecting custom data sources to LLMs — ingestion, indexing, retrieval.
nashsu 20k · 3 days agoDesktop app implementing Karpathy's LLM Wiki pattern: an LLM ingests your documents into a persistent, interlinked wiki with a knowledge graph, hybrid search, deep research and an MCP server.
datalab-to 40k · 18 days agoDatalab's 39k-star PDF-to-Markdown/JSON converter: a layout pipeline plus an optional LLM pass for tables, forms and equations, with chunk output and form-value extraction built in.
1Panel-dev 22.9k · 3 days agoOpen-source enterprise agent platform: RAG pipelines (upload or crawl docs), a visual workflow engine with MCP tool-use, and zero-code embedding into existing business systems.
mtrnix 103 · 4 days agoSelf-hosted agent memory stack in Docker: Postgres + Qdrant + Neo4j hybrid retrieval, a temporal knowledge graph, ontology layer and freshness checks behind one MCP-native API.
opendatalab 80.8k · 3 days agoHeavyweight document-to-markdown/JSON parser — PDFs plus Office (docx/pptx/xlsx) through layout analysis and OCR into LLM-ready output for RAG and agentic pipelines. 73k stars, self-hostable.
morphik-org 3.7k · 6 days agoMultimodal retrieval engine for visually rich documents: ingestion, visual-first search over charts, tables and diagrams, knowledge graphs and cache-augmented generation — one engine, not a pipeline.
allenai 19.7k · 6 months agoOpen toolkit that linearizes messy PDFs — scans, tables, equations, handwriting — into clean ordered Markdown with a self-hosted vision-language model. Built for LLM training data and RAG ingestion.
lfnovo 39.6k · 4 days agoSelf-hosted NotebookLM alternative: multi-modal sources, vector plus full-text search, context-aware chat and multi-speaker podcast generation — 18+ model providers incl. Ollama, full REST API.
opendataloader-project 29.4k · 3 days agoDeterministic PDF parser for AI pipelines: #1 extraction accuracy (0.907) on its public bench, bounding boxes on every element, 0.015s/page — plus the first open PDF auto-tagging for accessibility.
volcengine 38.9k · 3 days agoVolcengine's context database: memories, resources and skills as one `viking://` filesystem agents ls, tree and grep — L0/L1/L2 tiers, traceable retrieval, sessions distilled into memory.
VectifyAI 35.9k · 7 days agoVectorless, reasoning-based RAG — builds a hierarchical tree index from long documents so an LLM retrieves by relevance instead of similarity. No chunking, no embeddings, no vector DB.
firecrawl 19.4k · 3 days agoFirecrawl's Rust PDF triage: classifies text-based vs scanned in ~10-50ms, extracts positioned text and clean Markdown without OCR — routing the ~54% of PDFs that never needed a model.
Evokoa 1.1k · 8 days agoPostgreSQL extension adding graph search, traversal and shortest-path over your existing tables — a derived graph index queried from plain SQL, no separate graph DB or query language. Rust.
StarTrail-org 10.1k · 4 days agoBerkeley's visual RAG: render pages and PDFs to screenshot tiles and retrieve with a VLM embedder — tables, charts and layout survive. pixelshot CLI plus a hosted 8.28M-page Wikipedia index.
vibrantlabsai 15.9k · 7 months agoEvaluation toolkit for your RAG and agent pipelines — faithfulness, relevance, and more.
hanxiao 53 · 3 months agoAirgapped closed-corpus QA testbed: a local Qwen agent in a Pi harness explores a .zip dataroom with grep/embeddings/rerankers under a token budget — a bed to study search as test-time compute.
superlinked 3.3k · 3 days agoSelf-hosted inference cluster for everything agents call besides the big LLM: embeddings, rerankers, OCR, NER, guardrails and small LLMs — 100+ models, one OpenAI-compatible API, K8s stack included.
supavec 1.2k · 9 months agoOpen-source RAG-as-a-service (the Carbon.ai alternative): upload any data source, get vector search and a chat API in minutes — Supabase-based, multi-tenant with RLS, streaming responses.
supermemoryai 31k · 3 days agoMemory and context engine for AI: fact extraction, user profiles, contradiction handling and forgetting, hybrid RAG + memory search, connectors, agent plugins and a one-binary local mode.
MODSetter 16.3k · 5 days agoOpen-source competitive-intelligence platform for agents: live Reddit/YouTube/TikTok/Maps/search connectors; scheduled agents produce briefs and alerts into a cited knowledge base. REST + MCP.
bosun-ai 787 · 3 days agoRust framework for LLM apps: an agent harness, compile-time-typed task graphs, and streaming RAG pipelines — MCP toolboxes, human-in-the-loop approval, tracing with Langfuse support.
RyanCodrai 17.3k · 18 days agoRust vector index with Python bindings built on Google's TurboQuant: no training step, online ingest, hand-written SIMD kernels — 10M x 1536 vectors in ~4 GB, with allowlist-filtered search.
neuml 13k · 4 days agoAll-in-one AI framework around an embeddings database — dense, sparse, graph and relational fused — with pipelines, workflows, agents and MCP/web APIs. Python, bindings for JS/Java/Rust/Go.
baidu 26.5k · 2 months agoBaidu's open OCR VLM that parses entire multi-page documents in one shot — 'unlimited' long-horizon parsing pushing DeepSeek-OCR further. MIT weights on HF; serve via transformers, vLLM or SGLang.
Zipstack 7.3k · 3 days agoLLM-driven platform turning unstructured documents into structured data: a no-code Prompt Studio to define extractions, then deploy as APIs or ETL pipelines. Self-hosted, AGPL + enterprise.
deeplethe 8k · 3 days agoDeepLethe's open 'enterprise world model': one Rust binary plus Postgres running a bitemporal knowledge graph with ontology packs, cited hybrid search, an agent harness and MCP. Air-gap ready.
Tencent 1.7k · 11 days agoTencent WeChat's universal multimodal embedding family (2B/4B/9B): one vector space for text, images, video, visual documents and interleaved inputs, with Matryoshka dimensions from 64 to 4096.
xberg-io 9.3k · 3 days agoRust-core extraction orchestrator with 15 language bindings: 96 formats — PDF, Office, images, audio, code, web — to clean text, tables and RAG-ready chunks. OCR and structured extraction built in.
zvec-ai 3.8k · 3 days agozg: ripgrep, BM25 and vector search behind one local-first CLI for humans and agents — index a workspace once, search code, docs and data by meaning, then verify with exact text or regex.
alibaba 16k · 3 days agoAlibaba's open-source in-process vector database: billion-scale similarity search embedded in your app, with DiskANN on-disk indexing, native full-text search and hybrid retrieval.