什么是Semantic Router？2024高效语义决策层指南 | Geoz.com.cn

项目概述

Semantic Router 是一个专为大型语言模型（LLM）和智能体（Agent）设计的高效决策层。其核心创新在于，它能够基于对用户查询的语义化理解直接进行路由决策，而无需等待LLM生成完整响应。这种方法不仅能显著提升系统的整体响应速度，还能有效降低对LLM API的调用频率和成本。

Semantic Router is a high-performance decision layer specifically designed for Large Language Models (LLMs) and Agents. Its core innovation lies in its ability to make routing decisions based on a semantic understanding of user queries, without waiting for an LLM to generate a full response. This approach not only significantly improves the overall system response speed but also effectively reduces the frequency and cost of LLM API calls.

Semantic Router 的核心价值体现在以下几个方面：

• 高效处理用户查询：能够快速理解用户意图并将查询路由到合适的处理函数。
• 减少LLM调用：通过语义路由基于语义理解而非关键词匹配的路由决策机制，通过向量相似度计算将用户查询分配到合适的处理函数。减少不必要的LLM API调用，降低成本。
• 灵活的路由配置：支持多种配置方式，包括基于文本相似度和混合路由策略。
• 广泛的模型兼容性：支持多种主流嵌入模型将文本转换为向量表示的模型，用于语义相似度计算。Semantic Router支持多种嵌入模型，如OpenAI、Cohere、HuggingFace等。，如OpenAI、Cohere、HuggingFace等。

The core value of Semantic Router is reflected in the following aspects:

• Efficient User Query Processing: Quickly understands user intent and routes queries to appropriate handler functions.
• Reduced LLM Invocations: Minimizes unnecessary LLM API calls through semantic routing, lowering costs.
• Flexible Routing Configuration: Supports various configuration methods, including text similarity-based and hybrid routing strategies.
• Extensive Model Compatibility: Compatible with multiple mainstream embedding models, such as OpenAI, Cohere, HuggingFace, etc.

作为一个开源项目，Semantic Router 提供了简单易用的API，并支持多种集成方案，是构建智能Agent或复杂对话系统的理想基础组件。

As an open-source project, Semantic Router provides a simple and easy-to-use API while supporting multiple integration schemes, making it an ideal foundational component for building intelligent Agents or complex dialogue systems.

核心架构与代码结构

代码库目录结构

Semantic Router 采用清晰的模块化设计，其代码库结构如下：

semantic_router/
├── __init__.py
├── encoders/            # 嵌入模型实现
│   ├── __init__.py
│   ├── bedrock.py
│   ├── cohere.py
│   ├── fastembed.py
│   ├── google.py
│   ├── huggingface.py
│   ├── litellm.py
│   ├── openai.py
│   └── voyageai.py
├── index/               # 索引实现
│   ├── __init__.py
│   ├── cohere.py
│   ├── hybrid.py
│   ├── semantic.py
│   └── vector.py
├── llms/                # 支持的语言模型
│   ├── __init__.py
│   ├── anthropic.py
│   ├── base.py
│   ├── cohere.py
│   ├── gemini.py
│   ├── litellm.py
│   ├── mistral.py
│   ├── mock.py
│   └── openai.py
├── route.py             # 路由核心定义
├── routers/             # 路由器实现
│   ├── __init__.py
│   ├── base.py
│   ├── hybrid.py
│   └── semantic.py
├── schema.py            # 数据模型定义
└── utils/               # 工具函数
    ├── __init__.py
    ├── logger.py
    ├── models.py
    └── similarity.py

Semantic Router employs a clear modular design. The structure of its codebase is as follows:
(Directory structure translation omitted for brevity, as it's primarily code paths. The key point is the modular organization into encoders/, index/, llms/, routers/, etc.)

功能流程

Semantic Router 的核心工作流程可以分为以下几个步骤：

1. 用户查询输入：系统接收用户的自然语言查询。
2. 语义匹配：
   • 查询通过嵌入模型将文本转换为向量表示的模型，用于语义相似度计算。Semantic Router支持多种嵌入模型，如OpenAI、Cohere、HuggingFace等。转换为向量表示。
   • 与所有预定义路由的向量进行相似度比较。
3. 路由决策：
   • 如果相似度超过设定的阈值，将查询路由到匹配的处理函数。
   • 如果没有匹配或相似度低于阈值，则转向默认处理逻辑（如交由通用LLM处理）。
4. 执行响应：调用对应的处理函数生成并返回最终响应。

The core workflow of Semantic Router can be divided into the following steps:

1. User Query Input: The system receives a user's natural language query.
2. Semantic Matching:
   • The query is converted into a vector representation via an embedding model.
   • Its similarity is compared with the vectors of all predefined routes.
3. Routing Decision:
   • If the similarity exceeds a set threshold, the query is routed to the matching handler function.
   • If no match is found or similarity is below the threshold, it falls back to a default logic (e.g., passed to a general-purpose LLM).
4. Response Execution: The corresponding handler function is invoked to generate and return the final response.

关键组件深度分析

1. Route 类

Route 类是 Semantic Router 的核心组件之一，定义在 semantic_router/route.py 文件中。它用于创建具体的路由配置，指定如何处理特定语义范围的查询。

The Route class is one of the core components of Semantic Router, defined in the semantic_router/route.py file. It is used to create specific routing configurations, specifying how to handle queries within a particular semantic scope.

主要特性：

• 语义范围定义：通过示例语句（utterances）来定义该路由所覆盖的语义范围。
• 匹配精度控制：可设置相似度阈值（score_threshold），精确控制匹配的严格程度。
• 同步/异步支持：支持同步和异步两种方式调用关联的处理函数。
• 序列化能力：可被序列化为字典格式，便于配置的持久化存储和恢复。
• 函数模式：支持以OpenAI Function Calling等格式定义函数模式，增强与LLM的集成。

Key Features:

• Semantic Scope Definition: Defines the semantic scope covered by the route through example statements (utterances).
• Matching Precision Control: Allows setting a similarity threshold (score_threshold) to precisely control the strictness of matching.
• Sync/Async Support: Supports both synchronous and asynchronous invocation of associated handler functions.
• Serialization Capability: Can be serialized into a dictionary format, facilitating persistent storage and restoration of configurations.
• Function Schema: Supports defining function schemas in formats like OpenAI Function Calling, enhancing integration with LLMs.

关键方法：

def __call__(self, *args, **kwargs):
    """同步调用路由对应的处理函数"""
    # Synchronously call the handler function corresponding to the route

async def acall(self, *args, **kwargs):
    """异步调用路由对应的处理函数"""
    # Asynchronously call the handler function corresponding to the route

def to_dict(self):
    """将路由序列化为字典格式"""
    # Serialize the route into a dictionary format

2. SemanticRouter 类

SemanticRouter 类是主要的路由器实现，定义在 semantic_router/routers/semantic.py 中，继承自 BaseRouter。它负责管理路由集合并执行用户查询的匹配逻辑。

The SemanticRouter class is the primary router implementation, defined in semantic_router/routers/semantic.py, inheriting from BaseRouter. It is responsible for managing the collection of routes and executing the matching logic for user queries.

主要特性：

• 模型集成：集成多种嵌入模型将文本转换为向量表示的模型，用于语义相似度计算。Semantic Router支持多种嵌入模型，如OpenAI、Cohere、HuggingFace等。，负责将文本查询转换为向量表示。
• 路由管理：管理多个Route规则，支持动态添加和移除。
• 匹配引擎：提供核心的相似度计算和最佳匹配选择逻辑。
• 操作模式：全面支持同步（route）和异步（aroute）操作模式。

Key Features:

• Model Integration: Integrates various embedding models, responsible for converting text queries into vector representations.
• Route Management: Manages multiple Route rules, supporting dynamic addition and removal.
• Matching Engine: Provides core similarity calculation and best-match selection logic.
• Operation Modes: Fully supports both synchronous (route) and asynchronous (aroute) operation modes.

关键方法：

def route(self, text: str, **kwargs) -> Union[RouteChoice, None]:
    """执行路由匹配，返回最佳匹配结果"""
    # Perform route matching and return the best match result

async def aroute(self, text: str, **kwargs) -> Union[RouteChoice, None]:
    """异步执行路由匹配"""
    # Asynchronously perform route matching

def add_route(self, route: Route) -> None:
    """添加新的路由规则"""
    # Add a new route rule

3. 嵌入模型将文本转换为向量表示的模型，用于语义相似度计算。Semantic Router支持多种嵌入模型，如OpenAI、Cohere、HuggingFace等。支持

Semantic Router 通过 encoders 模块提供了对多种嵌入模型将文本转换为向量表示的模型，用于语义相似度计算。Semantic Router支持多种嵌入模型，如OpenAI、Cohere、HuggingFace等。的广泛支持，确保了灵活性和性能。

Semantic Router provides extensive support for multiple embedding models through the encoders module, ensuring flexibility and performance.

主要支持的模型：

• OpenAI Embeddings：利用OpenAI的文本嵌入模型将文本转换为向量表示的模型，用于语义相似度计算。Semantic Router支持多种嵌入模型，如OpenAI、Cohere、HuggingFace等。（如text-embedding-3-small）。
• Cohere Embed：使用Cohere公司的嵌入技术。
• HuggingFace：支持本地部署和云端托管的HuggingFace模型。
• Google Embeddings：集成Google的生成式AI嵌入API。
• FastEmbed：一个高性能的本地嵌入选项，轻量且快速。
• VoyageAI：支持VoyageAI的专用嵌入方案。

Main Supported Models:

• OpenAI Embeddings: Utilizes OpenAI's text embedding models (e.g., text-embedding-3-small).
• Cohere Embed: Uses Cohere's embedding technology.
• HuggingFace: Supports both locally deployed and cloud-hosted HuggingFace models.
• Google Embeddings: Integrates Google's Generative AI embedding API.
• FastEmbed: A high-performance local embedding option, lightweight and fast.
• VoyageAI: Supports VoyageAI's specialized embedding solutions.

所有嵌入器都实现了一个统一的接口，核心方法包括：

def encode(self, texts: List[str], **kwargs) -> List[List[float]]:
    """将文本列表转换为向量表示"""
    # Convert a list of texts into vector representations

async def aencode(self, texts: List[str], **kwargs) -> List[List[float]]:
    """异步方式将文本转换为向量"""
    # Asynchronously convert texts into vectors

4. 索引实现

index 模块提供了不同的索引策略，用于高效存储和检索路由向量，这是处理大量路由规则时保持低延迟的关键。

The index module provides different indexing strategies for efficient storage and retrieval of route vectors, which is key to maintaining low latency when handling a large number of route rules.

索引类型：

• 向量索引：基础的向量相似度匹配，通常基于余弦相似度。
• 语义索引：可能包含更复杂的语义相关性判断逻辑。
• 混合索引结合关键词和语义的混合匹配策略，通过index模块实现，用于高效存储和检索路由向量，支持向量索引、语义索引和混合索引。：结合关键词匹配和语义相似度的混合匹配策略，提供更高的准确性和鲁棒性。

Index Types:

• Vector Index: Basic vector similarity matching, typically based on cosine similarity.
• Semantic Index: May contain more complex semantic relevance judgment logic.
• Hybrid Index: A hybrid matching strategy combining keyword matching and semantic similarity, offering higher accuracy and robustness.

实践示例与分析

基础使用示例

以下代码展示了 Semantic Router 的一个基本使用流程：

from semantic_router import Route, RouteLayer
from semantic_router.encoders import OpenAIEncoder

# 1. 创建编码器
encoder = OpenAIEncoder()

# 2. 定义路由处理函数
def weather_handler(query):
    return f"Weather query processed: {query}"

def news_handler(query):
    return f"News query processed: {query}"

# 3. 定义路由配置
weather_route = Route(
    name="weather",
    utterances=[
        "What's the weather like today?",
        "Will it rain tomorrow?",
        "Temperature forecast for this weekend"
    ],
    handler=weather_handler
)

news_route = Route(
    name="news",
    utterances=[
        "What's happening in the world?",
        "Latest tech news",
        "Breaking news updates"
    ],
    handler=news_handler
)

# 4. 创建路由层
router = RouteLayer(encoder=encoder, routes=[weather_route, news_route])

# 5. 使用路由处理查询
query = "How hot will it be on Saturday?"
result = router.route(query)
if result:
    response = result.route.handler(query)
    print(response)  # 输出：Weather query processed: How hot will it be on Saturday?
else:
    print("No matching route found")

The following code demonstrates a basic usage workflow of Semantic Router:
(Code translation omitted. The steps are: 1. Create encoder, 2. Define handlers, 3. Define routes with example utterances, 4. Create router layer, 5. Route a query and handle it.)

这个例子清晰地展示了 Semantic Router 的核心工作流程。值得注意的是，系统成功地将查询 “How hot will it be on Saturday?” 识别为天气相关查询，并路由到 weather_handler，尽管这个确切的句子并不在 weather_route 的示例语句（utterances）列表中。这正体现了其基于语义理解而非简单关键词匹配的核心能力。

This example clearly demonstrates the core workflow of Semantic Router. It is noteworthy that the system successfully identified the query "How hot will it be on Saturday?" as a weather-related query and routed it to the weather_handler, even though this exact sentence was not in the weather_route's example utterances list. This precisely demonstrates its core capability based on semantic understanding rather than simple keyword matching.

高级功能

混合路由

Semantic Router 支持混合路由策略，结合了关键词匹配的准确性和语义匹配的灵活性。

Semantic Router supports a hybrid routing strategy, combining the precision of keyword matching with the flexibility of semantic matching.

from semantic_router.routers import HybridRouter

router = HybridRouter(
    encoder=encoder,
    routes=[weather_route, news_route],
    keyword_weight=0.3,  # 关键词匹配的权重
    semantic_weight=0.7   # 语义匹配的权重
)

函数调用支持

它可以定义与OpenAI Function Calling兼容的函数模式，便于在LLM Agent框架中集成。

It can define function schemas compatible with OpenAI Function Calling, facilitating integration within LLM Agent frameworks.

weather_route = Route(
    name="get_weather",
    utterances=["What's the weather like?"],
    function_schema={
        "name": "get_weather",
        "description": "Get weather information",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"},
                "date": {"type": "string"}
            },
            "required": ["location"]
        }
    }
)

配置持久化

路由配置可以轻松保存到文件并从文件加载，简化了部署和版本管理。

Routing configurations can be easily saved to and loaded from files, simplifying deployment and version management.

# 保存路由配置
router.save("router_config.json")

# 从配置文件加载
new_router = RouteLayer.from_file("router_config.json", encoder)

总结

Semantic Router 作为一个专为LLM和Agent时代设计的智能决策层，具备以下核心优势：

Semantic Router, as an intelligent decision layer designed for the era of LLMs and Agents, possesses the following core advantages:

• 高效语义路由基于语义理解而非关键词匹配的路由决策机制，通过向量相似度计算将用户查询分配到合适的处理函数。：基于深度语义理解进行路由，比传统关键词匹配更准确、更相关。
• 灵活的配置生态：支持多种路由策略、索引方式和嵌入模型将文本转换为向量表示的模型，用于语义相似度计算。Semantic Router支持多种嵌入模型，如OpenAI、Cohere、HuggingFace等。，能适应从简单到复杂的各种应用场景。
• 广泛的兼容性：集成了几乎所有主流嵌入模型将文本转换为向量表示的模型，用于语义相似度计算。Semantic Router支持多种嵌入模型，如OpenAI、Cohere、HuggingFace等。和LLM提供商，用户可根据成本、性能和隐私需求自由选择。
• 现代化的API设计：全面支持同步和异步操作，原生适配高并发场景。
• 良好的可扩展性：模块化架构使其易于集成新的模型、索引策略或自定义逻辑。

• Efficient Semantic Routing: Routes based on deep semantic understanding, making it more accurate and relevant than traditional keyword matching.
• Flexible Configuration Ecosystem: Supports various routing strategies, indexing methods, and embedding models, adaptable to a wide range of applications from simple to complex.
• Extensive Compatibility: Integrates with almost all mainstream embedding models and LLM providers, allowing users to freely choose based on cost, performance, and privacy requirements.
• Modern API Design: Fully supports both synchronous and asynchronous operations, natively suited for high-concurrency scenarios.
• Excellent Extensibility: Modular architecture makes it easy to integrate new models, indexing strategies, or custom logic.

适用场景：

• 构建复杂Agent系统：作为大脑的“调度中心”，高效分发查询给专用工具或知识库。
• 优化成本与延迟：在LLM调用前进行过滤，避免将简单或可预处理的查询发送给昂贵的LLM。
• 实现混合处理系统：结合规则引擎、专用函数和通用LLM能力，构建稳健且高效的AI应用。

Applicable Scenarios:

• Building Complex Agent Systems: Acts as the "dispatch center" for the brain, efficiently distributing queries to specialized tools or knowledge bases.
• Optimizing Cost and Latency: Filters queries before LLM invocation, avoiding sending simple or pre-processable queries to expensive LLMs.
• Implementing Hybrid Processing Systems: Combines rule engines, specialized functions, and general-purpose LLM capabilities to build robust and efficient AI applications