Causes and Solutions for RAG Search Failures in Official and Legal Documents – Recognizing Korean Document Hierarchical Structures
Building a Retrieval-Augmented Generation (RAG) process for official and legal documents is more complex than for general documents. This is because if the Article, Paragraph, Subparagraph, and Item hierarchy and table structures are damaged, the AI may answer based on clauses different from the query.
Electronic documents from administrative agencies are shifting towards HWPX, an open document format that is easy for AI to read. Starting May 2026, central and local Onnara document systems will be mandated to attach only open document formats. While the trend of converting official documents into AI-utilizable forms is growing, in actual RAG implementation, preserving the Article, Paragraph, Subparagraph, and Item hierarchy and official document table structures is critical for search accuracy.
This article summarizes the causes of RAG search failures in official and legal documents and methods to improve search accuracy by preserving Korean document hierarchical structures.
Reasons for Inaccurate RAG Search in Official and Legal Documents
The difficulty in searching official and legal documents within RAG stems from their inherent hierarchical structure of Article, Paragraph, Subparagraph, and Item, as well as multi-column and table structures. If these structures are not properly reflected during the document parsing and chunking process, irrelevant clauses or incomplete evidence may be retrieved.
Korean official documents have three overlapping characteristics: HWP/HWPX formats optimized for the domestic work environment, the legal hierarchy of Part, Chapter, Section, Article, and Article, Paragraph, Subparagraph, and Item as organized by the Korea Legislation Research Institute, and the multi-column and merged table structures unique to official documents. In particular, the Article, Paragraph, Subparagraph, and Item hierarchy serves as a criterion for distinguishing the meaning of clauses, so precisely identifying locations like “Article 3, Paragraph 2” clarifies the basis for an answer. If such structures are damaged during storage, it may be difficult to find relevant evidence for a query, even if the search model or embedding is changed.
The demand for AI adoption in public institutions is also increasing. The Digital Platform Government Committee revised its guidelines for hyper-scale AI in the public sector, adding RAG as a data learning method and including considerations such as building Vector Databases. While systems for converting documents into AI-readable open formats are being established, the practical challenge remains how to reliably recognize HWP/HWPX formats and legal hierarchies.
Three Main Causes of Search Difficulty in Official and Legal Document RAG
There are three primary reasons for search difficulties in RAG for official and legal documents: the need for a dedicated parser to accurately read HWP/HWPX, the potential for the Article, Paragraph, Subparagraph, and Item hierarchy to be broken during chunking, and the difficulty in maintaining the multi-column and merged table structures of official documents. All three issues arise when the structure of Korean documents is not sufficiently reflected in the parsing and chunking process.
HWP/HWPX: Document Formats in the Domestic Work Environment Requiring Dedicated Parsers
HWP/HWPX are widely used formats for public and corporate documents in Korea, but major global LLMs do not inherently integrate dedicated parsers for them. A Document Parser interprets file structures to extract text, table, and layout information. Therefore, accurately reading HWP/HWPX requires a dedicated parser capable of interpreting their specific structures.
According to the article, the common belief that “AI cannot read Hangeul documents” is only half true. The key point is not that AI is inherently unable to read HWP/HWPX, but rather that the degree to which text, tables, and layouts are understood varies depending on the parser integrated. If the document structure is not properly extracted during the parsing phase, it can be difficult to find accurate evidence in subsequent chunking and search phases.
Article, Paragraph, Subparagraph, and Item Hierarchy – Loss of Hierarchy Due to Chunking
Laws and official documents maintain meaning through units of Article, Paragraph, Subparagraph, and Item. However, if chunking is based solely on character or token count, a single clause may be split into multiple fragments, or the connection between a superior Article and its subordinate Paragraphs may be severed.
For example, if a title like “Article 3” and its subordinate paragraphs are divided into different chunks, the context of the clause may not be sufficiently reflected during the search phase. In such cases, even if a user queries a specific clause, an incomplete clause or a clause different from the query might be retrieved as evidence.
In actual legal AI projects, it has been confirmed that if clauses are cut at chunk boundaries, provisos like “in the preceding paragraph” or “provided that” can be severed, altering the meaning. Therefore, chunking methods that preserve the Article and Paragraph units are crucial.
Multi-Column and Merged Tables in Official Documents – Issues with Reading Order and Row/Column Relationships Being Damaged
Official documents often include multi-column layouts, merged cells, and nested tables. If these structures are not properly recognized, the reading order may change, or the relationship between table headers and values may be broken.
For a table’s meaning to be understood, the relationship between headers and values must be maintained. For example, headers like “Item” and “Amount” must be linked to the numbers below them to understand what those values represent. The RAG preparation guide from Microsoft Learn also explains that multi-column content should preferably not be parsed in the same way as single-column content, and complex tables like nested tables require separate preprocessing. If these structures are not preserved, contextually insufficient numbers or table fragments may be used as evidence during the search phase.
The core of these three issues is the same: the format, hierarchy, and table structures of Korean documents are not sufficiently reflected in the parsing and chunking process. Therefore, before chunking, it is necessary to accurately parse the original document, recognize the Article and Paragraph hierarchy, and restore the row and column relationships of tables.
How to Recognize Korean Document Hierarchical Structures
Recognizing Korean document hierarchical structures requires three processes: directly parsing HWP/HWPX as original, preserving the document’s title and Article/Paragraph hierarchy as metadata, and restoring the row/column relationships of multi-column and merged tables.
These methods correspond to the three causes examined earlier.
Preserving Text and Structure by Directly Parsing Original HWP/HWPX
By parsing HWP/HWPX directly from the original without converting to PDF or other formats, potential losses during conversion can be reduced, and text, paragraph, and table structures can be extracted together.
HWP documents are internally composed of multiple layers. As shown in the parsing structure organized by Hancom Tech, a SectionList exists under the top-level Document, followed by a ParaList containing paragraphs. Direct parsing of the original allows data extraction while maintaining this hierarchical information. Conversely, converting to PDF and then re-reading can lead to the loss of paragraph hierarchy or table structures.
Recognizing Title and Article/Paragraph Hierarchy with DLA and Preserving as Metadata
Document Layout Analysis (DLA) is a technology that recognizes title, body, table areas, and hierarchical information within a document. By identifying the Article, Paragraph, Subparagraph, and Item hierarchy and storing “which Article/Paragraph this content belongs to” as metadata for each chunk, the context of the clause can be utilized even after chunking.
For example, if a title path like “Chapter 2 > Article 3 > Paragraph 2” is stored with each chunk, the search phase can utilize not only the body text but also the context of the clause to which that chunk belongs. HiChunk research also reported a case where the performance of retrieving evidence sentences necessary for answers improved from 74.1% to 81.0% when chunking reflected the document’s hierarchical structure.
This point is crucial in RAG for official and legal documents. If only the body text remains in a chunk and the Article/Paragraph location is lost, even if the search result appears to be a correct sentence, it might actually be evidence from a different clause. Therefore, it is necessary to recognize the document hierarchy before chunking and preserve that information as metadata.
Restoring Multi-Column and Merged Table Rows/Columns with TSR
Table Structure Recognition (TSR) is a technology that restores table structures by recognizing row/column structures and merged cells within table images. Preserving the table structure helps maintain the connection between headers and values even in tables within official documents.
For official documents scanned as images, OCR (Optical Character Recognition) reads the text within cells, and TSR identifies which row/column that text belongs to. This restoration of row/column relationships is necessary to utilize both headers and values as evidence during the search phase.
The results of these three processes lead to structured data. Only with data that preserves the hierarchy and table relationships of Korean documents does it become easier to maintain clause context in the subsequent chunking phase.
The Principle of Restoring RAG Search with Structured Data that Preserves Hierarchical Structures
Converting documents into structured data that maintains the Article, Paragraph, and Subparagraph hierarchy and table row/column relationships makes it easier to divide documents based on clause context in the subsequent chunking phase. Structured data with preserved structures ensures that relevant clauses and table information can be used as evidence during the search phase.
Conversely, if the hierarchy and table structures are lost during the parsing phase, search accuracy may not significantly improve even by changing embedding models or vector databases. This is because if the input data itself does not retain clause locations and table relationships, RAG will provide answers based on incomplete evidence.
💡 If you want to see how official and legal documents are actually structured, you can check the document parsing results in the Hancom Data Loader live demo.
👉 Try the Hancom Data Loader Live Demo
Why Hancom Data Loader is Necessary for Official and Legal Document RAG Preprocessing
🖥️Hancom Data Loader
The decline in search accuracy for RAG in official and legal documents is not solely due to the chunking or embedding stages. Rather, if the original HWP/HWPX structure, the Article, Paragraph, Subparagraph, and Item hierarchy, and the table’s row/column relationships are not properly extracted beforehand, the evidence available for search becomes inherently incomplete.
Hancom Data Loader is a Document Parsing solution that converts HWP/HWPX/PDF/OOXML documents into structured data that AI can utilize. It does not directly perform chunking but rather extracts and parses document hierarchy and table structures, preserving them for use in the chunking and embedding stages.
Direct Parsing of Original HWP/HWPX and Hierarchical/Table Structuring
Hancom Data Loader directly parses HWP/HWPX without converting them to PDF. This allows for more stable extraction of structural information crucial for official and legal document RAG, such as paragraph hierarchy, clause structure, and table information.
PDFs and images are processed based on OCR, DLA, and TSR, while OOXML is primarily processed for text extraction. Since the appropriate method is applied according to the format and document characteristics, the same preprocessing method is not forcibly applied to all documents.
The extraction results are provided as structured data, such as JSON, allowing for the utilization of document hierarchy information, coordinates, and complex table structures like merged cells or nested tables. In essence, Hancom Data Loader provides a preprocessing foundation that can mitigate the issues of “lost clause locations” and “separated table headers and values” problematic in official and legal document RAG.
Public/Legal Document Processing and Hancom Pedia Integration
Hancom Data Loader can process both structured documents with consistent formats and unstructured documents containing mixed tables and images, according to their document type. Documents with repetitive forms, such as regulations and official notices, are processed based on rules, while documents containing tables and images utilize AI-based analysis to extract structural information.
There are also public sector application cases. According to Digital Daily, in the Gyeonggi Provincial Office of Education’s AI Digital Platform construction project, Hancom Data Loader was responsible for converting over 2,800 school websites, 40,000 guidance materials, and 7,000 policy/legal data into a format that AI could learn from. This is an example of structuring HWP/HWPX/PDF documents to include text, level information, coordinates, and complex tables.
This extracted and parsed structured data can then be utilized in subsequent chunking, embedding, search, and Q stages. If integrated with Hancom Pedia as needed, the structured data created by Hancom Data Loader can be used to build a document-based RAG pipeline.
Frequently Asked Questions (FAQ) Regarding Official and Legal Document RAG
Restoring Multi-Column and Merged Table Rows/Columns with TSR
Table Structure Recognition (TSR) is a technology that restores table structures by recognizing row/column structures and merged cells within table images. Preserving the table structure helps maintain the connection between headers and values even in tables within official documents.
For official documents scanned as images, OCR (Optical Character Recognition) reads the text within cells, and TSR identifies which row/column that text belongs to. This restoration of row/column relationships is necessary to utilize both headers and values as evidence during the search phase.
The results of these three processes lead to structured data. Only with data that preserves the hierarchy and table relationships of Korean documents does it become easier to maintain clause context in the subsequent chunking phase.
The Principle of Restoring RAG Search with Structured Data that Preserves Hierarchical Structures
Converting documents into structured data that maintains the Article, Paragraph, and Subparagraph hierarchy and table row/column relationships makes it easier to divide documents based on clause context in the subsequent chunking phase. Structured data with preserved structures ensures that relevant clauses and table information can be used as evidence during the search phase.
Conversely, if the hierarchy and table structures are lost during the parsing phase, search accuracy may not significantly improve even by changing embedding models or vector databases. This is because if the input data itself does not retain clause locations and table relationships, RAG will provide answers based on incomplete evidence.
💡 If you want to see how official and legal documents are actually structured, you can check the document parsing results in the Hancom Data Loader live demo.
👉 Try the Hancom Data Loader Live Demo
Why Hancom Data Loader is Necessary for Official and Legal Document RAG Preprocessing
🖥️Hancom Data Loader
The decline in search accuracy for RAG in official and legal documents is not solely due to the chunking or embedding stages. Rather, if the original HWP/HWPX structure, the Article, Paragraph, Subparagraph, and Item hierarchy, and the table’s row/column relationships are not properly extracted beforehand, the evidence available for search becomes inherently incomplete.
Hancom Data Loader is a Document Parsing solution that converts HWP/HWPX/PDF/OOXML documents into structured data that AI can utilize. It does not directly perform chunking but rather extracts and parses document hierarchy and table structures, preserving them for use in the chunking and embedding stages.
Direct Parsing of Original HWP/HWPX and Hierarchical/Table Structuring
Hancom Data Loader directly parses HWP/HWPX without converting them to PDF. This allows for more stable extraction of structural information crucial for official and legal document RAG, such as paragraph hierarchy, clause structure, and table information.
PDFs and images are processed based on OCR, DLA, and TSR, while OOXML is primarily processed for text extraction. Since the appropriate method is applied according to the format and document characteristics, the same preprocessing method is not forcibly applied to all documents.
The extraction results are provided as structured data, such as JSON, allowing for the utilization of document hierarchy information, coordinates, and complex table structures like merged cells or nested tables. In essence, Hancom Data Loader provides a preprocessing foundation that can mitigate the issues of “lost clause locations” and “separated table headers and values” problematic in official and legal document RAG.
Public/Legal Document Processing and Hancom Pedia Integration
Hancom Data Loader can process both structured documents with consistent formats and unstructured documents containing mixed tables and images, according to their document type. Documents with repetitive forms, such as regulations and official notices, are processed based on rules, while documents containing tables and images utilize AI-based analysis to extract structural information.
There are also public sector application cases. According to Digital Daily, in the Gyeonggi Provincial Office of Education’s AI Digital Platform construction project, Hancom Data Loader was responsible for converting over 2,800 school websites, 40,000 guidance materials, and 7,000 policy/legal data into a format that AI could learn from. This is an example of structuring HWP/HWPX/PDF documents to include text, level information, coordinates, and complex tables.
This extracted and parsed structured data can then be utilized in subsequent chunking, embedding, search, and Q stages. If integrated with Hancom Pedia as needed, the structured data created by Hancom Data Loader can be used to build a document-based RAG pipeline.
Frequently Asked Questions (FAQ) Regarding Official and Legal Document RAG
Restoring Multi-Column and Merged Table Rows/Columns with TSR
Table Structure Recognition (TSR) is a technology that restores table structures by recognizing row/column structures and merged cells within table images. Preserving the table structure helps maintain the connection between headers and values even in tables within official documents.
For official documents scanned as images, OCR (Optical Character Recognition) reads the text within cells, and TSR identifies which row/column that text belongs to. This restoration of row/column relationships is necessary to utilize both headers and values as evidence during the search phase.
The results of these three processes lead to structured data. Only with data that preserves the hierarchy and table relationships of Korean documents does it become easier to maintain clause context in the subsequent chunking phase.
The Principle of Restoring RAG Search with Structured Data that Preserves Hierarchical Structures
Converting documents into structured data that maintains the Article, Paragraph, and Subparagraph hierarchy and table row/column relationships makes it easier to divide documents based on clause context in the subsequent chunking phase. Structured data with preserved structures ensures that relevant clauses and table information can be used as evidence during the search phase.
Conversely, if the hierarchy and table structures are lost during the parsing phase, search accuracy may not significantly improve even by changing embedding models or vector databases. This is because if the input data itself does not retain clause locations and table relationships, RAG will provide answers based on incomplete evidence.
💡 If you want to see how official and legal documents are actually structured, you can check the document parsing results in the Hancom Data Loader live demo.
👉 Try the Hancom Data Loader Live Demo
Why Hancom Data Loader is Necessary for Official and Legal Document RAG Preprocessing
🖥️Hancom Data Loader
The decline in search accuracy for RAG in official and legal documents is not solely due to the chunking or embedding stages. Rather, if the original HWP/HWPX structure, the Article, Paragraph, Subparagraph, and Item hierarchy, and the table’s row/column relationships are not properly extracted beforehand, the evidence available for search becomes inherently incomplete.
Hancom Data Loader is a Document Parsing solution that converts HWP/HWPX/PDF/OOXML documents into structured data that AI can utilize. It does not directly perform chunking but rather extracts and parses document hierarchy and table structures, preserving them for use in the chunking and embedding stages.
Direct Parsing of Original HWP/HWPX and Hierarchical/Table Structuring
Hancom Data Loader directly parses HWP/HWPX without converting them to PDF. This allows for more stable extraction of structural information crucial for official and legal document RAG, such as paragraph hierarchy, clause structure, and table information.
PDFs and images are processed based on OCR, DLA, and TSR, while OOXML is primarily processed for text extraction. Since the appropriate method is applied according to the format and document characteristics, the same preprocessing method is not forcibly applied to all documents.
The extraction results are provided as structured data, such as JSON, allowing for the utilization of document hierarchy information, coordinates, and complex table structures like merged cells or nested tables. In essence, Hancom Data Loader provides a preprocessing foundation that can mitigate the issues of “lost clause locations” and “separated table headers and values” problematic in official and legal document RAG.
Public/Legal Document Processing and Hancom Pedia Integration
Hancom Data Loader can process both structured documents with consistent formats and unstructured documents containing mixed tables and images, according to their document type. Documents with repetitive forms, such as regulations and official notices, are processed based on rules, while documents containing tables and images utilize AI-based analysis to extract structural information.
There are also public sector application cases. According to Digital Daily, in the Gyeonggi Provincial Office of Education’s AI Digital Platform construction project, Hancom Data Loader was responsible for converting over 2,800 school websites, 40,000 guidance materials, and 7,000 policy/legal data into a format that AI could learn from. This is an example of structuring HWP/HWPX/PDF documents to include text, level information, coordinates, and complex tables.
This extracted and parsed structured data can then be utilized in subsequent chunking, embedding, search, and Q stages. If integrated with Hancom Pedia as needed, the structured data created by Hancom Data Loader can be used to build a document-based RAG pipeline.
Frequently Asked Questions (FAQ) Regarding Official and Legal Document RAG
Why does search often fail when official and legal documents are used in RAG?
For official and legal documents, the meaning can only be accurately understood if the Article, Paragraph, Subparagraph, and Item hierarchy and table structures are maintained together. If these structures are broken during the parsing and chunking process, irrelevant clauses or incomplete evidence may be selected during the search phase. Therefore, before changing embedding models or vector databases, it is essential to first verify that the original document structure has been properly extracted.
Why can’t AI directly read HWP?
HWP requires a dedicated parser that can interpret its internal document structure. Since major global LLMs do not inherently integrate HWP-specific parsers, processing without a separate parser may not sufficiently reflect text, table, and layout structures. In other words, it’s not that AI cannot read HWP at all, but rather that an appropriate Document Parser must be integrated for accurate utilization.
How can the Article, Paragraph, Subparagraph, and Item hierarchy of laws be preserved?
The hierarchy of titles, articles, paragraphs, subparagraphs, and items is initially recognized by analyzing the structure of the original file itself, and DLA is used to supplement hierarchical recognition if necessary. By storing metadata for each chunk indicating which clause the content belongs to, for example, a title path like “Chapter 2 Article 3 Paragraph 2” is saved together, allowing the search phase to utilize not only the body text but also the clause context.
Does Hancom Data Loader also perform chunking?
Hancom Data Loader does not directly perform chunking. It extracts and parses HWP/HWPX/PDF/OOXML documents, converting them into structured data where hierarchy and table structures are preserved. This structured data can then be utilized in subsequent chunking, embedding, and search stages, and can be integrated with Hancom Pedia as needed for use in a RAG pipeline.
Document Parser for Improving RAG Search Accuracy in Official and Legal Documents
The search accuracy of RAG for official and legal documents begins with how well the document structure is extracted before chunking. Only when clause hierarchy and table relationships remain in the structured data can relevant evidence be reliably utilized during the subsequent search phase.
Hancom Data Loader is a Document Parsing solution that converts HWP/HWPX/PDF/OOXML documents into structured data that AI can utilize. In the Hancom Data Loader live demo, you can directly upload documents to check parsing results and consider implementation methods suitable for your RAG preprocessing environment.
👉 Upload a document and check parsing results without installation (Live Demo)
👉 Consult for RAG preprocessing tailored to your company
References
- arXiv, Lu et al., “HiChunk: Evaluating and Enhancing Retrieval-Augmented Generation with Hierarchical Chunking”, 2025
- Microsoft Learn, “Develop an Azure RAG solution – Preparation phase”, 2026
- Korea Legislation Research Institute, “Legal Clause System (Part, Chapter, Section, Article / Article, Paragraph, Subparagraph, Item)”
- Danbi News, Ahn So-hyun, Kim Ye-eun, “Can AI not read Hangeul documents?”, 2026
- Hancom Tech, “Hanword document file format: Parsing the HWP format with Python (2),” 2025
