What is RAG Data Preprocessing? A Guide to Document Preprocessing for AI Answer Quality

In RAG (Retrieval-Augmented Generation) systems, the reason answers remain inaccurate even after adopting AI often lies not in the AI (LLM) itself, but in how documents are handled before they are fed into the AI—namely, RAG data preprocessing. Let’s look at why preprocessing is key in a RAG system, step by step.

What is RAG? The Role of Retrieval-Augmented Generation and Preprocessing

What are the Meaning and Basic Concepts of RAG?

Unlike general LLMs that rely solely on training data, RAG is an AI architecture that searches external documents—such as internal corporate documents, in-house manuals, regulations, and reports—and generates answers based on highly relevant content. The quality of document preprocessing, the very first stage of the pipeline, has a significant impact on subsequent retrieval accuracy and answer quality.

RAG System Architecture and the Role of Preprocessing

RAG works in the following order: Document Upload → Parsing → Chunking → Embedding → Vector DB (embedding storage and retrieval database) Storage → Retrieval → Answer Generation. If preprocessing is inadequate, the AI may pull irrelevant information and generate answers that differ from the facts—i.e., hallucinations, where the AI outputs incorrect content as if it were true without any basis.

If the preprocessing stage is insufficient, errors can carry over into the subsequent chunking, embedding, and retrieval stages. In particular, for formats widely used in Korea such as HWP and HWPX, most major global Document AI solutions do not support direct parsing of the original files, making it difficult to fully preserve their structure with general-purpose tools.

Worried about the burden of document preprocessing and data loss when building a RAG pipeline?

👉 Explore Hancom Data Loader

Why is Structured Conversion Necessary for Unstructured Documents?

Unstructured documents such as HWP, HWPX, PDF, and OOXML cannot be directly understood by AI. Therefore, structured data conversion must precede for RAG to function correctly.

Why Search Errors Occur in Documents with Lost Structure

When document structure is lost, the AI generates irrelevant answers based on it.

Formats that only support text extraction, such as OOXML, or simple OCR (Optical Character Recognition) methods may not preserve table structure, paragraph hierarchy, or reading order information.

For example, if numbers in a table are mixed with body text, AI cannot distinguish which numbers are table values and which content is descriptive text. As a result, it may fail to find relevant information or provide factually incorrect answers. If structural information like table layouts or paragraph hierarchies is lost, the context that AI relies on is diminished.

Structural loss is not just a formatting issue. It becomes a cause of RAG quality degradation, lowering chunking, embedding, and retrieval accuracy as well.

How are OCR and DLA different?

OCR is a character recognition technology, while DLA (Document Layout Analysis) is a technology that understands the overall structure of a document. To improve RAG quality, both technologies are needed together.

Why Building RAG with OCR Alone is Difficult

OCR is a technology that converts characters from images or scanned documents into text. Since it is specialized for character recognition, it is difficult to preserve document structure information such as table structures, paragraph hierarchies, and reading order. In PDFs, the visual order and the actual extraction order can differ, leading to issues where semantic units get mixed up when using OCR alone.

Meanwhile, DLA identifies the structural relationship of extracted text, determining whether it’s a table, heading, or body content. This understanding of structural relationships allows the AI to interpret the same number completely differently depending on whether it’s a value within a table or part of a body explanation.

In summary, OCR is the technology that reads characters, and DLA is the technology that identifies the context—whether those characters belong to a table or a heading. Both technologies must work together within a single pipeline to improve RAG quality.

The Impact of Preserving HWP/HWPX Structure on Search Accuracy

HWP and HWPX are formats with proprietary file structures. Using only general OCR or PDF conversion results in the loss of information regarding tables, footnotes, and indentation, which decreases RAG search accuracy.

Although these are the most commonly used document formats in Korean public institutions, legal, and financial sectors, most major global Document AI solutions only support general-purpose formats like PDF, JPEG, PNG, and TIFF, and do not support direct parsing of HWP/HWPX originals. Direct parsing of the originals is essential to maintain RAG search accuracy.

Directly parsing HWP/HWPX originals requires core technology that can access the proprietary format structure. If processed by converting to PDF like general tools, key formatting such as footnotes, merged cells, and indentation may be lost.

💡 How to directly parse HWP/HWPX originals without conversion, based on Hancom’s 30 years of technology?

👉Explore Hancom Data Loader

RAG Chunking and Embedding: The Impact of Preprocessing Quality

Chunking is the process of dividing a document into semantic units, and embedding is the process of converting text into numerical vectors to make it searchable. Both stages depend on the quality of preprocessing.

Why Chunking Quality Determines Search Accuracy

The choice of chunking strategy directly determines the retrieval accuracy of a RAG system.

The key to chunking is not just cutting text into small pieces, but dividing it while ensuring the meaning of paragraphs is fully maintained. Both Semantic Chunking and Hierarchical Chunking work effectively only when the document structure has been preserved beforehand. If DLA is not performed first, any chunking method becomes less effective.

Whether it is semantic-based or hierarchy-based chunking, the document’s headings, paragraphs, and table hierarchies must first be accurately extracted to work effectively. This is why document parsing quality must be secured before choosing any chunking strategy.

Why Poor Chunking and Embedding Degrade AI Answer Quality

When chunking quality is low, context is separated or semantic units are lost, and the quality of embedding—which converts text into numeric vectors—also declines. Ultimately, vector DB search accuracy and AI answer quality both decline together.

Ultimately, vector DB search accuracy drops, increasing the likelihood of RAG hallucinations. This is why, in most cases, the cause of degraded RAG embedding quality lies in the preprocessing stage rather than the LLM.

Frequently Asked Questions (FAQ) on RAG Preprocessing

How to Solve RAG Document Preprocessing with Hancom Data Loader

As we have seen, RAG performance is determined at the preprocessing stage before the LLM. The document structure must be fully preserved for chunking to maintain semantic units, for embedding quality to improve, and for vector DB search accuracy to be secured. If preprocessing quality declines, all subsequent steps degrade with it.

The problem is that most documents handled in actual corporate environments, such as HWP, HWPX, PDF, and OOXML, are in formats that are difficult to parse while preserving structure. In particular, since global Document AI solutions do not support direct parsing of HWP/HWPX originals, it is difficult to connect them to a RAG pipeline without structural loss using conventional methods.

Hancom Data Loader supports the entire process as a single pipeline, from document parsing—the very first step of RAG preprocessing—to structured data conversion.

✅ Processes DLA, OCR, and TSR composite AI in a single pipeline to accurately extract structured text and object information from documents

✅ Supports all formats actually used in corporate environments, including HWP, HWPX, PDF, and OOXML, and directly parses HWP/HWPX original files without conversion

✅ Automatically determines the hierarchy of headings, paragraphs, and tables through Level Inference-based hierarchical structure extraction and outputs it as structured data, contributing to quality improvement in the subsequent chunking stage.

✅ Prevents external leakage of documents even in network-separated or closed-network environments by fully internalizing on-premise with container-based REST API

✅ A proven solution in public, financial, and legal environments with GS certification

If your current RAG answers are not meeting expectations, check your preprocessing environment with Hancom Data Loader before looking at the LLM.

👉 Go use the Hancom Data Loader live demo

👉 Inquire About Hancom Data Loader

---

References

1. arXiv, “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” Lewis et al., NeurIPS 2020
2. AWS, “What is Retrieval-Augmented Generation (RAG)?”
3. AWS, “What is OCR?”
4. Microsoft, “Chunk large documents for RAG and vector search in Azure AI Search”
5. arXiv, “Revolutionizing RAG with Enhanced PDF Structure Recognition,” Lin, arXiv:2401.12599
6. arXiv, “Document Parsing Unveiled: Techniques, Challenges, and Prospects,” Lin, arXiv:2410.21169
7. arXiv, “SCAN: Semantic Document Layout Analysis for Textual and Visual RAG,” Ueda et al., arXiv:2505.14381, EACL 2026
8. AWS Textract, “Supported Document Formats”
9. Microsoft Azure, “Azure Document Intelligence Overview”
10. Google, “Document AI Supported File Types”