GEO
LLMS

PageIndex：基于推理的RAG新范式，让大语言模型智能检索专业文档

2026/1/31
PageIndex：基于推理的RAG新范式，让大语言模型智能检索专业文档
AI Summary (BLUF)

PageIndex is a document indexing system that transforms lengthy PDFs into semantic tree structures optimized for LLMs, enabling reasoning-based retrieval that outperforms traditional vector similarity approaches. It's particularly effective for financial reports, regulatory documents, and technical manuals where domain expertise and multi-step reasoning are required.

PageIndex是一个文档索引系统，可将冗长PDF转换为语义树结构，专为大语言模型优化，实现基于推理的检索，超越传统向量相似度方法。特别适用于需要领域专业知识和多步推理的财务报告、监管文件和技术手册。

引言

你是否对长篇专业文档的向量数据库检索准确性感到失望？传统的基于向量的RAG系统依赖于语义相似性而非真正的相关性。但在检索中，我们真正需要的是相关性——这需要推理能力。当处理需要领域专业知识和多步推理的专业文档时，相似度搜索常常不尽人意。基于推理的RAG提供了更好的选择：让大语言模型能够思考和推理，找到最相关的文档部分。受AlphaGo启发，Vectify AI提出使用树搜索来执行结构化文档检索。PageIndex 是一个文档索引系统，它从长文档构建搜索树结构，为基于推理的RAG做好准备。

Are you disappointed with the accuracy of vector database retrieval for long, professional documents? Traditional vector-based RAG systems rely on semantic similarity rather than true relevance. However, in retrieval, what we truly need is relevance—which requires reasoning capability. When dealing with specialized documents that demand domain expertise and multi-step reasoning, similarity search often falls short. Reasoning-based RAG offers a better alternative: enabling large language models to think and reason to find the most relevant document sections. Inspired by AlphaGo, Vectify AI proposes using tree search for structured document retrieval. PageIndex is a document indexing system that builds a search tree structure from long documents, preparing them for reasoning-based RAG.

PageIndex 是什么？

PageIndex能将冗长的PDF文档转换为语义树结构，类似于*“目录”*但专为大语言模型(LLMs)优化。它特别适用于：财务报告、监管文件、学术教科书、法律或技术手册，以及任何超出LLM上下文限制的文档。

PageIndex can transform lengthy PDF documents into a semantic tree structure, akin to a "table of contents" but specifically optimized for Large Language Models (LLMs). It is particularly suitable for: financial reports, regulatory documents, academic textbooks, legal or technical manuals, and any document exceeding the context window of an LLM.

主要特点

  • 层次树结构
    让大语言模型能够逻辑性地遍历文档——就像一个智能的、为LLM优化的目录。

    Hierarchical Tree Structure
    Enables LLMs to traverse documents logically—like an intelligent, LLM-optimized table of contents.

  • 精确页面引用
    每个节点都包含其摘要和开始/结束页面的物理索引，实现精准检索。

    Precise Page References
    Each node contains its summary and the physical index of its start/end pages, enabling precise retrieval.

  • 无需人为分块
    不使用任意分块。节点遵循文档的自然结构。

    No Arbitrary Chunking
    Does not use arbitrary chunking. Nodes follow the document's natural structure.

  • 适用于大规模文档
    设计用于轻松处理数百甚至上千页的文档。

    Scalable for Large Documents
    Designed to easily handle documents spanning hundreds or even thousands of pages.

PageIndex 格式

以下是输出示例。查看更多示例文档和生成的树结构。

{
  "title": "金融稳定性",
  "node_id": "0006",
  "start_index": 21,
  "end_index": 22,
  "summary": "美联储...",
  "nodes": [
    {
      "title": "监测金融脆弱性",
      "node_id": "0007",
      "start_index": 22,
      "end_index": 28,
      "summary": "美联储的监测..."
    },
    {
      "title": "国内和国际合作与协调",
      "node_id": "0008",
      "start_index": 28,
      "end_index": 31,
      "summary": "2023年，美联储与..."
    }
  ]
}

其实看到这里，我们会发现RAG之前很多框架或者算法都有类似的思想：例如LlamaIndex的Node实现，比如Raptor的层级聚类，还有Mineru的PDF转换生成Markdown，然后我们可以解析成类似具有章节信息的json数据。那PageIndex的亮点在哪里呢？其实在最后一部分“使用PageIndex进行基于推理的RAG”的实现。相比之前的Advanced和Modular RAG，Agentic RAG更加智能。接着我们往下看怎么实现的？

At this point, we might notice that many previous RAG frameworks or algorithms have had similar ideas: for example, LlamaIndex's Node implementation, Raptor's hierarchical clustering, and Mineru's PDF-to-Markdown conversion which can be parsed into JSON data with chapter information. So, what is the highlight of PageIndex? It lies in the implementation of the final section, "Using PageIndex for Reasoning-Based RAG." Compared to previous Advanced and Modular RAG approaches, Agentic RAG is more intelligent. Let's continue to see how it is implemented.

使用方法

按照以下步骤从PDF文档生成PageIndex树结构。

Follow these steps to generate a PageIndex tree structure from a PDF document.

  1. 安装依赖项

    pip3 install -r requirements.txt

    Install Dependencies

    pip3 install -r requirements.txt

  2. 设置OpenAI API密钥
    在根目录创建一个.env文件并添加你的API密钥：

    CHATGPT_API_KEY=你的openai密钥

    Set OpenAI API Key
    Create a .env file in the root directory and add your API key:

    CHATGPT_API_KEY=your_openai_key

  3. 对PDF运行PageIndex

    python3 run_pageindex.py --pdf_path /path/to/your/document.pdf

    你可以通过额外的可选参数自定义处理过程：

    --model                 使用的OpenAI模型 (默认: gpt-4o-2024-11-20)
--toc-check-pages       检查目录的页数 (默认: 20)
--max-pages-per-node    每个节点的最大页数 (默认: 10)
--max-tokens-per-node   每个节点的最大token数 (默认: 20000)
--if-add-node-id        添加节点ID (yes/no, 默认: yes)
--if-add-node-summary   添加节点摘要 (yes/no, 默认: no)
--if-add-doc-description 添加文档描述 (yes/no, 默认: yes)

    Run PageIndex on PDF

    python3 run_pageindex.py --pdf_path /path/to/your/document.pdf

    You can customize the processing with additional optional parameters:

    --model                 OpenAI model to use (default: gpt-4o-2024-11-20)
--toc-check-pages       Number of pages to check for TOC (default: 20)
--max-pages-per-node    Max pages per node (default: 10)
--max-tokens-per-node   Max tokens per node (default: 20000)
--if-add-node-id        Add node ID (yes/no, default: yes)
--if-add-node-summary   Add node summary (yes/no, default: no)
--if-add-doc-description Add document description (yes/no, default: yes)

云API (测试版)

不想自己部署？试试Vectify AI的PageIndex托管API。托管版本使用Vectify AI自定义的OCR模型更准确地识别PDF，为复杂文档提供更好的树结构。在这个表单留下你的邮箱，免费获得1,000页处理额度。

Don't want to deploy it yourself? Try Vectify AI's PageIndex Hosted API (Beta). The hosted version uses Vectify AI's custom OCR model for more accurate PDF recognition, providing better tree structures for complex documents. Leave your email in this form to receive 1,000 free processing pages.

案例研究：Mafin 2.5

Mafin 2.5是一个专为财务文档分析设计的最先进基于推理的RAG模型。它基于PageIndex构建，在FinanceBench基准测试中达到了惊人的98.7%准确率——显著优于传统的基于向量的RAG系统。PageIndex的分层索引使得能够精确导航和提取复杂财务报告（如SEC文件和财报披露）中的相关内容。👉 查看完整基准测试结果，了解详细比较和性能指标。

Case Study: Mafin 2.5
Mafin 2.5 is a state-of-the-art reasoning-based RAG model designed for financial document analysis. Built on PageIndex, it achieved an impressive 98.7% accuracy on the FinanceBench benchmark—significantly outperforming traditional vector-based RAG systems. PageIndex's hierarchical indexing enables precise navigation and extraction of relevant content from complex financial reports (such as SEC filings and earnings disclosures). 👉 View the full benchmark results for detailed comparisons and performance metrics.

使用PageIndex进行基于推理的RAG

使用PageIndex构建基于推理的检索系统，无需依赖语义相似度。非常适合需要细微区分的领域特定任务。

Using PageIndex for Reasoning-Based RAG
Build a reasoning-based retrieval system using PageIndex, eliminating reliance on semantic similarity. Ideal for domain-specific tasks requiring nuanced discrimination.

预处理工作流示例

  1. 使用PageIndex处理文档，生成树结构。
  2. 将树结构及其对应的文档ID存储在数据库表中。
  3. 将每个节点的内容存储在单独的表中，通过节点ID和树ID进行索引。

Example Preprocessing Workflow

  1. Process the document using PageIndex to generate the tree structure.
  2. Store the tree structure and its corresponding document ID in a database table.
  3. Store the content of each node in a separate table, indexed by node ID and tree ID.

基于推理的RAG框架示例

  1. 查询预处理：分析查询以确定所需知识。
  2. 文档选择：搜索相关文档及其ID。
  3. 从数据库获取相应的树结构。
  4. 节点选择：搜索树结构以识别相关节点。
  5. LLM生成：
    • 从数据库获取所选节点的相应内容。
    • 格式化并提取相关信息。
    • 将组装的上下文与原始查询一起发送给LLM生成有依据的回答。

Example Reasoning-Based RAG Framework

  1. Query Preprocessing: Analyze the query to determine required knowledge.
  2. Document Selection: Search for relevant documents and their IDs.
  3. Retrieve Corresponding Tree Structure from the database.
  4. Node Selection: Search the tree structure to identify relevant nodes.
  5. LLM Generation:
    • Retrieve the corresponding content of the selected nodes from the database.
    • Format and extract relevant information.
    • Send the assembled context along with the original query to the LLM to generate a grounded answer.

节点选择的示例提示

prompt = f"""
给你一个问题和一个文档的树结构。
你需要找出所有可能包含答案的节点。

问题: {question}

文档树结构: {structure}

请用以下JSON格式回复:
{{
    "thinking": <关于在哪里寻找的推理过程>,
    "node_list": [node_id1, node_id2, ...]
}}
"""

看到这里我们自然明白了，PageIndex不需要切块向量是因为通过将文档转换为节点，然后用大模型进行选择，之前RAG是检索+排序=现在的LLM Judge。同时这个问题就是，当多文档或者文档篇幅比较多的时候，LLM去做选择成本比较高的。

Seeing this, we naturally understand that PageIndex does not require chunking and vectorization because it converts documents into nodes and then uses the LLM for selection. Previously, RAG was about retrieval + ranking, whereas now it's about LLM as a Judge. However, this approach raises the issue of high cost when the LLM performs selection across multiple documents or very lengthy documents.

本文参与 腾讯云自媒体同步曝光计划，分享自作者个人站点/博客。 原始发表：2025-04-22，如有侵权请联系 cloudcommunity@tencent.com 删除

This article is part of the Tencent Cloud Media Synchronized Exposure Plan, shared from the author's personal site/blog. Originally published: 2025-04-22. If there is any infringement, please contact cloudcommunity@tencent.com for removal.

标签
AI应用开发生成式AI大语言模型AI大模型LLMS
← 返回文章列表
分享到：微博
下一篇
生成即流量：GEO智启全域增长新范式

版权与免责声明：本文仅用于信息分享与交流，不构成任何形式的法律、投资、医疗或其他专业建议，也不构成对任何结果的承诺或保证。

文中提及的商标、品牌、Logo、产品名称及相关图片/素材，其权利归各自合法权利人所有。本站内容可能基于公开资料整理，亦可能使用 AI 辅助生成或润色；我们尽力确保准确与合规，但不保证完整性、时效性与适用性，请读者自行甄别并以官方信息为准。

若本文内容或素材涉嫌侵权、隐私不当或存在错误，请相关权利人/当事人联系本站，我们将及时核实并采取删除、修正或下架等处理措施。 也请勿在评论或联系信息中提交身份证号、手机号、住址等个人敏感信息。