[{"data":1,"prerenderedAt":4},["ShallowReactive",2],{"readme:deepdive":3},"\u003Ch1>DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL\u003C\u002Fh1>\n\u003Cdiv align=\"center\">\u003Cp>\u003Ca href=\"https:\u002F\u002Fgithub.com\u002FTHUDM\u002FDeepDive\" rel=\"nofollow ugc noopener\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fstars\u002FTHUDM\u002FDeepDive?style=social\" alt=\"GitHub\" \u002F>\u003C\u002Fa>\n\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2509.10446\" rel=\"nofollow ugc noopener\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FarXiv-2509.10446-b31b1b.svg\" alt=\"arXiv\" \u002F>\u003C\u002Fa>\n\u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fzai-org\u002FDeepDive\" rel=\"nofollow ugc noopener\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F%F0%9F%A4%97%20Dataset-DeepDive-blueviolet\" alt=\"Dataset\" \u002F>\u003C\u002Fa>\n\u003Ca href=\"#\" rel=\"nofollow ugc noopener\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F%F0%9F%A4%97%20Model-Coming%20soon-ffcc00\" alt=\"Model\" \u002F>\u003C\u002Fa>\u003C\u002Fp>\n\u003C\u002Fdiv>\u003Cdiv align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FTHUDM\u002Fdeepdive\u002FHEAD\u002Fassets\u002Fcombine_head_figure.svg\" alt=\"Multi-Turn RL Training\" width=\"100%\" \u002F>\n  \u003Cp>\u003Cem> \u003C\u002Fem>\u003C\u002Fp>\n\u003C\u002Fdiv>\u003Ch2>🔥 News\u003C\u002Fh2>\n\u003Cul>\n\u003Cli>\u003Cstrong>[2026\u002F06\u002F17]\u003C\u002Fstrong> Released the \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fdeepdive\u002Fblob\u002FHEAD\u002Ftraining\" rel=\"nofollow ugc noopener\">training code\u003C\u002Fa> with the DeepDive slime rollout setup.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>[2025\u002F10\u002F02]\u003C\u002Fstrong> Released the complete \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fdeepdive\u002Fblob\u002FHEAD\u002Fqa_synthetic\" rel=\"nofollow ugc noopener\">data construction pipeline\u003C\u002Fa> — now fully available in the repository.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>[2025\u002F09\u002F17]\u003C\u002Fstrong> QA pairs and SFT trajectories have been fully open-sourced, totaling 4,108 entries. Check them out on \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fzai-org\u002FDeepDive\" rel=\"nofollow ugc noopener\">Hugging Face Dataset DeepDive\u003C\u002Fa>.\u003C\u002Fli>\n\u003Cli>Model checkpoints are currently under development – coming soon!\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Ch2>Overview\u003C\u002Fh2>\n\u003Cp>\u003Cstrong>DeepDive\u003C\u002Fstrong> presents an automated approach for training deep search agents that can navigate complex, multi-step information-seeking tasks. Our method combines automated data synthesis from knowledge graphs with end-to-end multi-turn reinforcement learning to create agents capable of sophisticated long-horizon reasoning and web browsing.\u003C\u002Fp>\n\u003Ch3>\u003Cstrong>Key Features\u003C\u002Fstrong>\u003C\u002Fh3>\n\u003Cul>\n\u003Cli>\u003Cstrong>Automated Deep Search Data Synthesis\u003C\u002Fstrong>: Generate challenging QA pairs from knowledge graphs through controlled random walks\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Multi-Turn RL Training for Browsing\u003C\u002Fstrong>: End-to-end reinforcement learning for deep search capabilities\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Test-Time Scaling\u003C\u002Fstrong>: Supports scaling via tool calls and parallel sampling\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Ch2>\u003Cstrong>Method Overview\u003C\u002Fstrong>\u003C\u002Fh2>\n\u003Ch3>\u003Cstrong>Stage 1: Automated Data Synthesis from Knowledge Graphs\u003C\u002Fstrong>\u003C\u002Fh3>\n\u003Cdiv align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FTHUDM\u002Fdeepdive\u002FHEAD\u002Fassets\u002Fkg_data_pipeline.svg\" alt=\"Data Synthesis Pipeline\" width=\"100%\" \u002F>\n\u003Cp>\u003Cem>\u003C\u002Fem>\u003C\u002Fp>\n\u003C\u002Fdiv>\u003Cp>We propose an automated method to synthesize complex, difficult, and hard-to-find questions from open knowledge graphs. The process involves three key steps:\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Knowledge Graph Random Walks\u003C\u002Fstrong>: Starting from an initial node $v_0$, we navigate through the graph for $k$ steps to form a path $P=[v_0, v_1, \\ldots, v_k]$, where each step $(v_i, v_{i+1})$ is a valid edge in the graph. We choose longer path lengths ($k > 5$) to increase reasoning complexity.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Entity Obfuscation\u003C\u002Fstrong>: We combine each node $v_i$ in the path with its corresponding attributes to form an attribute-rich path:\u003C\u002Fp>\n\u003Cp>$$P_A = [(v_0, [a_0^0, a_0^1, \\ldots]), (v_1, [a_1^0, a_1^1, \\ldots]), \\ldots, (v_k, [a_k^0, a_k^1, \\ldots])]$$\u003C\u002Fp>\n\u003Cp>An LLM then obfuscates information along the entire path, generalizing specific details and creating \"blurry entities\" that require deep search to resolve.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Difficulty Filtering\u003C\u002Fstrong>: We use a frontier model (GPT-4o) with basic search to attempt each question four times. Only questions that the frontier model fails in all attempts are retained, ensuring high difficulty.\u003C\u002Fp>\n\u003Ch3>\u003Cstrong>Stage 2: End-to-End Multi-Turn Reinforcement Learning\u003C\u002Fstrong>\u003C\u002Fh3>\n\u003Cdiv align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FTHUDM\u002Fdeepdive\u002FHEAD\u002Fassets\u002Fdeepdive_rl.svg\" alt=\"Multi-Turn RL Training\" width=\"100%\" \u002F>\n\u003Cp>\u003Cem>\u003C\u002Fem>\u003C\u002Fp>\n\u003C\u002Fdiv>\u003Cp>We apply end-to-end multi-turn RL to enhance the agent's long-horizon reasoning and browsing capabilities. The training process follows an iterative cycle where at step $t$, the agent generates chain-of-thought $c_t$, executes browsing action $a_t$, and observes web content $o_t$.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Multi-Turn GRPO Training\u003C\u002Fstrong>: We employ Group Relative Policy Optimization with normalized advantages:\u003C\u002Fp>\n\u003Cp>$$\nA_i = \\frac{r_i - \\text{mean}({r_k}\u003Cem>{k=1}^G)}{\\text{std}({r_k}\u003C\u002Fem>{k=1}^G)}\n$$\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Strict Binary Rewards\u003C\u002Fstrong>: A trajectory receives reward +1 if and only if both format correctness and answer accuracy are satisfied:\u003C\u002Fp>\n\u003Cp>$$\nr(\\mathcal{T}) = \\begin{cases} 1, & (\\foral\u003C\u002Fp>\n",1784240407127]