知识图谱与深度学习融合的航空安全问答系统，相比GPT-3和BERT QA哪个更准确？

摘要

在商业航空领域，存在大量文档，例如事故报告（NTSB, ASRS）和监管指令（ADs）。需要一个系统来高效访问这些多样化的知识库，以满足航空业的维护、合规和安全等需求。本文提出了一种基于知识图谱A structured knowledge base that represents entities and their relationships in a graph format.引导的深度学习A branch of machine learning using deep neural networks to process complex tasks like image and speech recognition.问答系统能够理解自然语言问题并从知识源中检索或生成答案的计算机系统。，用于航空安全领域。我们从飞机事故报告中构建了一个知识图谱A structured knowledge base that represents entities and their relationships in a graph format.，并将此资源贡献给研究社区。该资源的有效性通过上述问答系统能够理解自然语言问题并从知识源中检索或生成答案的计算机系统。进行了测试和验证。从上述文档构建的自然语言查询被转换为SPARQL一种用于查询RDF图数据库的标准查询语言和协议，支持从知识图谱中检索和操作数据。（RDF图数据库的接口语言）查询并得到回答。在深度学习A branch of machine learning using deep neural networks to process complex tasks like image and speech recognition.方面，我们采用了两种不同的问答模型：（i）BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding. QA，它是一个由基于Sentence-BERTBERT的变体，专门优化句子级语义相似度计算和嵌入生成，常用于信息检索和聚类。的段落检索和基于BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding.的问答组成的流水线；（ii）最新发布的GPT-3A large language model developed by OpenAI with 175 billion parameters, known for its advanced text generation capabilities.。我们在从事故报告创建的一组查询上评估了我们的系统。我们的融合问答系统能够理解自然语言问题并从知识源中检索或生成答案的计算机系统。在准确率上比GPT-3A large language model developed by OpenAI with 175 billion parameters, known for its advanced text generation capabilities.提高了9.3%，比BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding. QA提高了40.3%。因此，我们推断KG-DL融合方法比任何单一方法表现更优。

In the commercial aviation domain, there are a large number of documents, such as accident reports (NTSB, ASRS) and regulatory directives (ADs). There is a need for a system to efficiently access these diverse repositories to meet the demands of the aviation industry, including maintenance, compliance, and safety. This paper proposes a Knowledge Graph-guided Deep Learning-based Question Answering system for aviation safety. We construct a Knowledge Graph from aircraft accident reports and contribute this resource to the research community. The efficacy of this resource is tested and validated through the aforementioned QA system. Natural language queries constructed from the mentioned documents are converted into SPARQL一种用于查询RDF图数据库的标准查询语言和协议，支持从知识图谱中检索和操作数据。 (the interface language for RDF graph databases) queries and answered. On the deep learning side, we employ two different QA models: (i) BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding. QA, a pipeline consisting of Sentence-BERTBERT的变体，专门优化句子级语义相似度计算和嵌入生成，常用于信息检索和聚类。-based passage retrieval and BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding.-based question answering, and (ii) the recently released GPT-3A large language model developed by OpenAI with 175 billion parameters, known for its advanced text generation capabilities.. We evaluate our system on a set of queries created from accident reports. Our combined QA system achieves a 9.3% increase in accuracy over GPT-3A large language model developed by OpenAI with 175 billion parameters, known for its advanced text generation capabilities. and a 40.3% increase over BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding. QA. Thus, we conclude that the KG-DL hybrid approach performs better than either method alone.

引言：航空领域知识管理的挑战

航空安全依赖于对海量、多源、非结构化文档（如详细的事故调查报告、安全通告和法规文件）的深刻理解与即时检索。传统的关键词搜索或简单的文档管理系统难以满足复杂、精准的查询需求，例如“找出所有与特定型号发动机在高原机场发生的非包容性故障相关的事故报告及其建议措施”。这催生了对更智能、更结构化知识访问方式的需求。

Aviation safety relies on a deep understanding and instant retrieval of vast, multi-source, unstructured documents, such as detailed accident investigation reports, safety bulletins, and regulatory documents. Traditional keyword searches or simple document management systems struggle to meet complex and precise query demands, such as "Find all accident reports and their recommended actions related to non-contained failures of a specific engine model at high-altitude airports." This has spurred the need for more intelligent and structured knowledge access methods.

核心方法论：KG-DL融合框架

本文的核心创新在于提出并验证了一个知识图谱A structured knowledge base that represents entities and their relationships in a graph format.与深度学习A branch of machine learning using deep neural networks to process complex tasks like image and speech recognition.相融合的问答框架。该框架并非简单地将两种技术并列，而是让它们优势互补，形成一个协同系统。

The core innovation of this paper lies in proposing and validating a Knowledge Graph and Deep Learning hybrid question-answering framework. This framework does not merely juxtapose the two technologies but allows them to complement each other's strengths, forming a synergistic system.

知识图谱A structured knowledge base that represents entities and their relationships in a graph format.的构建与应用

首先，系统从非结构化的航空事故报告文本中提取实体（如飞机型号、部件、故障模式、地点）和关系（如“导致”、“发生于”、“涉及”），构建一个结构化的航空安全知识图谱A structured knowledge base that represents entities and their relationships in a graph format.。这个图谱将离散的事实连接成网络，为复杂推理奠定了基础。

First, the system extracts entities (e.g., aircraft model, component, failure mode, location) and relationships (e.g., "caused by," "occurred at," "involved") from unstructured aviation accident report texts to construct a structured Aviation Safety Knowledge Graph. This graph connects discrete facts into a network, laying the groundwork for complex reasoning.

当用户提出自然语言问题时，系统会尝试将其解析并转换为可以在知识图谱A structured knowledge base that represents entities and their relationships in a graph format.上执行的SPARQL一种用于查询RDF图数据库的标准查询语言和协议，支持从知识图谱中检索和操作数据。查询。这种方式特别擅长回答涉及多跳关系、实体属性聚合或明确事实查找的问题。

When a user poses a natural language question, the system attempts to parse and convert it into a SPARQL一种用于查询RDF图数据库的标准查询语言和协议，支持从知识图谱中检索和操作数据。 query that can be executed on the knowledge graph. This approach is particularly adept at answering questions involving multi-hop relationships, entity attribute aggregation, or explicit fact-finding.

深度学习A branch of machine learning using deep neural networks to process complex tasks like image and speech recognition.模型的角色

然而，并非所有问题都适合或能够被完整地转换为图谱查询，尤其是那些需要理解上下文、语义消歧或从大段文本中综合答案的问题。这时，深度学习A branch of machine learning using deep neural networks to process complex tasks like image and speech recognition.模型发挥作用。

However, not all questions are suitable or can be fully converted into graph queries, especially those requiring contextual understanding, semantic disambiguation, or synthesizing answers from large text segments. This is where deep learning models come into play.

本文评估了两种代表性的深度学习A branch of machine learning using deep neural networks to process complex tasks like image and speech recognition.QA模型：

1. BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding. QA流水线：首先使用Sentence-BERTBERT的变体，专门优化句子级语义相似度计算和嵌入生成，常用于信息检索和聚类。进行相关段落检索，然后使用BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding.模型在检索到的段落中进行精确答案抽取。这是一个经典的“检索-阅读”两阶段模型。
2. GPT-3A large language model developed by OpenAI with 175 billion parameters, known for its advanced text generation capabilities.：利用大规模生成式预训练模型的强大能力，以端到端的方式直接根据问题和提供的上下文生成答案。

This paper evaluates two representative deep learning QA models:

1. BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding. QA Pipeline: First uses Sentence-BERTBERT的变体，专门优化句子级语义相似度计算和嵌入生成，常用于信息检索和聚类。 for relevant passage retrieval, then employs a BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding. model for precise answer extraction within the retrieved passages. This is a classic "retrieve-and-read" two-stage model.
2. GPT-3A large language model developed by OpenAI with 175 billion parameters, known for its advanced text generation capabilities.: Leverages the powerful capabilities of a large-scale generative pre-trained model to generate answers directly in an end-to-end manner based on the question and provided context.

系统的融合决策流程

KG-DL框架的关键在于一个决策机制。系统需要判断对于一个给定问题，是更适合用知识图谱A structured knowledge base that represents entities and their relationships in a graph format.查询来回答，还是更适合用深度学习A branch of machine learning using deep neural networks to process complex tasks like image and speech recognition.模型来处理，或者是否需要结合两者的结果。论文中虽未详述具体融合算法（如基于置信度分数、问题分类），但其整体架构体现了这种混合智能的思想。

The key to the KG-DL framework lies in a decision mechanism. The system needs to determine whether a given question is better suited for a knowledge graph query, a deep learning model, or requires a combination of results from both. Although the paper does not detail the specific fusion algorithm (e.g., based on confidence scores, question classification), its overall architecture embodies this concept of hybrid intelligence.

性能评估与关键发现

作者在从真实航空事故报告衍生出的查询集上对系统进行了全面评估。结果清晰地证明了融合方法的优越性。

The authors conducted a comprehensive evaluation of the system on a query set derived from real aviation accident reports. The results clearly demonstrate the superiority of the hybrid approach.

模型性能对比分析

下表量化了不同方法在测试集上的准确率表现：

Model / System Performance Comparison

Model / System	Core Approach	Accuracy	Key Advantage	Main Limitation
BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding. QA (Baseline)	Retrieval + Extractive Reading Comprehension	Baseline Value	Precise answers, traceable to source text	Depends on retrieval quality, struggles with complex reasoning
GPT-3A large language model developed by OpenAI with 175 billion parameters, known for its advanced text generation capabilities. (Baseline)	Generative End-to-End Model	+31.0% over BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding. QA	Strong language generation, handles complex questions	May produce "hallucinations," factual accuracy can be unstable
KG-DL Hybrid System (This Paper)	Knowledge Graph Query + DL Model Fusion	+9.3% over GPT-3A large language model developed by OpenAI with 175 billion parameters, known for its advanced text generation capabilities. +40.3% over BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding. QA	Highest Accuracy, combines precise fact retrieval with semantic understanding	High system complexity, depends on KG construction quality

核心洞见

从评估结果可以得出几个重要结论：

1. 互补效应：知识图谱A structured knowledge base that represents entities and their relationships in a graph format.提供了精确、结构化的关系查询能力，而深度学习A branch of machine learning using deep neural networks to process complex tasks like image and speech recognition.模型（尤其是大语言模型如GPT-3A large language model developed by OpenAI with 175 billion parameters, known for its advanced text generation capabilities.）提供了强大的自然语言理解和生成能力。两者结合有效弥补了各自的短板。
2. 准确性优先：在航空安全这类对事实准确性要求极高的领域，纯生成式模型（GPT-3A large language model developed by OpenAI with 175 billion parameters, known for its advanced text generation capabilities.）的“幻觉”问题是不可接受的。通过引入知识图谱A structured knowledge base that represents entities and their relationships in a graph format.作为可靠的事实基准，融合系统显著提升了答案的可信度。
3. 1+1>2：简单的模型并列无法实现40.3%的巨大提升。这证明了论文中设计的引导与融合机制的有效性，使得整体系统性能超越了任一组成部分。

Several important conclusions can be drawn from the evaluation results:

1. Complementary Effect: Knowledge Graphs provide precise, structured relational query capabilities, while deep learning models (especially large language models like GPT-3A large language model developed by OpenAI with 175 billion parameters, known for its advanced text generation capabilities.) offer powerful natural language understanding and generation. Their combination effectively compensates for each other's weaknesses.
2. Accuracy First: In high-stakes domains like aviation safety where factual accuracy is paramount, the "hallucination" problem of pure generative models (GPT-3A large language model developed by OpenAI with 175 billion parameters, known for its advanced text generation capabilities.) is unacceptable. By introducing the knowledge graph as a reliable factual benchmark, the hybrid system significantly enhances answer reliability.
3. 1+1>2: A simple juxtaposition of models could not achieve the substantial 40.3% improvement. This proves the effectiveness of the guidance and fusion mechanism designed in the paper, enabling the overall system performance to surpass any individual component.

总结与展望

本文的工作具有双重贡献：一是为航空安全研究社区提供了一个宝贵的结构化知识图谱A structured knowledge base that represents entities and their relationships in a graph format.资源；二是实证了一种高效的KG-DL混合问答架构。该研究为垂直领域（如医疗、金融、法律）构建高可靠性智能问答系统能够理解自然语言问题并从知识源中检索或生成答案的计算机系统。提供了可借鉴的范式，即利用领域知识图谱A structured knowledge base that represents entities and their relationships in a graph format.约束和增强大语言模型的能力，从而实现既精准又智能的信息服务。

未来的工作可以沿着以下几个方向深入：探索更精细化的融合策略（如神经符号推理）、扩展知识图谱A structured knowledge base that represents entities and their relationships in a graph format.的覆盖范围（纳入手册、法规）、以及优化系统对复杂、多轮对话式查询的处理能力。

This paper makes a dual contribution: firstly, providing a valuable structured knowledge graph resource for the aviation safety research community; secondly, empirically validating an efficient KG-DL hybrid QA architecture. This research offers a replicable paradigm for building high-reliability intelligent QA systems in vertical domains (e.g., healthcare, finance, law), which involves using domain knowledge graphs to constrain and enhance the capabilities of large language models, thereby achieving information services that are both precise and intelligent.

Future work could delve into the following directions: exploring more refined fusion strategies (e.g., neuro-symbolic reasoning), expanding the coverage of the knowledge graph (to include manuals, regulations), and optimizing the system's ability to handle complex, multi-turn conversational queries.

常见问题（FAQ）

这个航空安全问答系统能够理解自然语言问题并从知识源中检索或生成答案的计算机系统。相比GPT-3A large language model developed by OpenAI with 175 billion parameters, known for its advanced text generation capabilities.和BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding. QA准确率提升了多少？

该系统相比GPT-3A large language model developed by OpenAI with 175 billion parameters, known for its advanced text generation capabilities.准确率提升9.3%，相比BERTBidirectional Encoder Representations from Transformers, a pre-trained natural language processing model for deep bidirectional language understanding. QA提升40.3%。这是通过知识图谱A structured knowledge base that represents entities and their relationships in a graph format.与深度学习A branch of machine learning using deep neural networks to process complex tasks like image and speech recognition.融合框架实现的协同优势。

知识图谱A structured knowledge base that represents entities and their relationships in a graph format.在这个系统中具体起什么作用？

知识图谱A structured knowledge base that represents entities and their relationships in a graph format.从事故报告中提取实体和关系，构建结构化网络。它能将自然语言问题转换为SPARQL一种用于查询RDF图数据库的标准查询语言和协议，支持从知识图谱中检索和操作数据。查询，擅长处理多跳关系和明确事实查找。

为什么需要融合知识图谱A structured knowledge base that represents entities and their relationships in a graph format.和深度学习A branch of machine learning using deep neural networks to process complex tasks like image and speech recognition.两种技术？

知识图谱A structured knowledge base that represents entities and their relationships in a graph format.擅长结构化推理和明确事实查询，而深度学习A branch of machine learning using deep neural networks to process complex tasks like image and speech recognition.擅长语义理解和上下文分析。两者融合互补，形成比单一方法更优的协同系统。