PixelRAG

Berkeley's visual RAG: render pages and PDFs to screenshot tiles and retrieve with a VLM embedder — tables, charts and layout survive. pixelshot CLI plus a hosted 8.28M-page Wikipedia index.

6,913 580 Python Apache-2.0updated 4 days ago

rag vision

View on GitHubDispute this mapping →

Curator's take

The paper's claim — screenshots beat parsed text for RAG — matters when your answers live in visual structure: tables, charts, infographics, layout-heavy PDFs that text chunkers flatten into noise. Zero-setup on-ramp is real: a hosted 8.28M-page Wikipedia index with no API key, and the pixelbrowse Claude Code plugin gives the agent screenshot-based page reading (pixelshot via Playwright/CDP, no MCP server). NOT a drop-in for text RAG stacks: your reader must be a VLM, tile indexes cost more storage/compute than text embeddings, and retrieval quality rides on their LoRA-tuned Qwen3-VL-Embedding model. Research codebase — expect pipeline assembly, not a product.

Mapped by ShipWithAI editors · links verified

README.md

Official codebase for PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation

Yichuan Wang*, Zhifei Li*, Zirui Wang, Paul Teiletche, Lesheng Jin
Matei Zaharia†, Joseph E. Gonzalez†, Sewon Min†

_{* Equal contribution † Equal advising}
_{Work done at Berkeley SkyLab & BAIR & Berkeley NLP}

Search any document by how it looks, not just the text it contains.

What it is · Give Claude eyes · How it works · Pipelines

pip install pixelrag

The two core operations — render a page to screenshots, search a visual index:

# Render any page or document to screenshot tiles
pixelshot https://en.wikipedia.org/wiki/Python --output ./tiles

# Search a hosted index of 8.28M Wikipedia pages — no setup, runs against the live API
curl -X POST https://api.pixelrag.ai/search \
  -H "Content-Type: application/json" \
  -d '{"queries": [{"text": "What is the capital of France?"}], "n_docs": 5}'

Live, hosted endpoint — https://api.pixelrag.ai serves a pre-built index of 8.28M Wikipedia pages. No setup, no API key. It even takes an image as the query (visual search) — see the API reference →.

Or try it in the browser at pixelrag.ai, or run the demo notebook in Colab — it renders a page and searches the hosted index, with the images inline.

What it is

PixelRAG renders documents — web pages, PDFs, images — as screenshots and retrieves over the images directly. Visual structure that HTML parsing throws away — tables, charts, layout, infographics — stays intact, so the reader model can actually answer questions about it. Wikipedia's 8.28M articles ship as a pre-built index; the pipeline itself is general-purpose.

Give Claude eyes

The renderer also ships as a Claude Code plugin — the pixelbrowse skill. Instead of fetching raw HTML, Claude screenshots a page with pixelshot and reads the image, so it sees charts, diagrams, tables, and layout the way a person d

Continue your stack

What teams reach for next — and why each earns a place beside PixelRAG. Ranked by curator confidence.

Alternative to See all + compare →

MinerU

Heavyweight document-to-markdown/JSON parser — PDFs plus Office (docx/pptx/xlsx) through layout analysis and OCR into LLM-ready output for RAG and agentic pipelines. 73k stars, self-hostable.

Why it fitsTwo answers to the same RAG-ingestion problem: MinerU parses PDFs/Office through layout analysis and OCR into LLM-ready markdown; PixelRAG skips parsing entirely and retrieves over rendered screenshot tiles. Parse-to-text vs stay-in-pixels.

olmocr

Open toolkit that linearizes messy PDFs — scans, tables, equations, handwriting — into clean ordered Markdown with a self-hosted vision-language model. Built for LLM training data and RAG ingestion.

Why it fitsolmocr linearizes messy PDFs into clean ordered Markdown so a text pipeline can index them; PixelRAG argues the linearization step is the loss — embed the page image and let a VLM read the tile directly.