What is RAG Data Preprocessing? A Guide to Document Preprocessing for AI Answer Quality

In RAG (Retrieval-Augmented Generation) systems, the reason behind the question “I implemented AI, but why am I getting irrelevant answers?” often lies in the way documents are processed before being fed into the AI—namely, RAG data preprocessing—rather than the AI (LLM) itself. Let’s look at why preprocessing is key in a RAG system, step by step.

What is RAG? The Role of Retrieval-Augmented Generation and Preprocessing

What are the Meaning and Basic Concepts of RAG?

RAG (Retrieval-Augmented Generation) is an AI architecture that retrieves external documents and generates answers based on their content. The quality of document preprocessing, the very first stage of the pipeline, significantly impacts subsequent search accuracy and answer quality.

Unlike standard LLMs that only refer to training data, RAG searches internal corporate documents, company manuals, laws, and reports to provide answers based on highly relevant evidence.

RAG System Architecture and the Role of Preprocessing

RAG operates in the following order: Document Upload → Parsing → Chunking → Embedding → Vector DB Storage → Retrieval → Answer Generation. If preprocessing is inadequate, errors occur where the AI retrieves incorrect information and generates answers that deviate from the facts—a phenomenon known as hallucination, where the AI outputs incorrect content as if it were true without any basis.

If the preprocessing stage is poor, all subsequent stages—chunking, embedding, and retrieval—are affected in a chain reaction. In particular, formats widely used in Korea, such as HWP and HWPX, are difficult to preserve fully using general tools because major global Document AI solutions, including AWS Textract, Azure Document Intelligence, and Google Document AI, do not support direct parsing of the original files.

Worried about the burden of document preprocessing and data loss when building a RAG pipeline?

👉 Explore Hancom Data Loader

Why is Structured Conversion Necessary for Unstructured Documents?

Unstructured documents such as PDF, HWP/HWPX, and PPTX cannot be directly understood by AI. Therefore, structured data conversion must come first for RAG to function correctly.

Why Search Errors Occur in Documents with Broken Structure

If the document structure is lost, the AI generates incorrect answers based on the broken text.

Formats that only support text extraction, such as OOXML, or simple OCR methods may fail to preserve table structures, paragraph hierarchies, and reading order information.

For example, if numbers in a table get mixed up with the body text, the AI cannot distinguish which numbers are values from the table and which content belongs to explanatory sentences. As a result, it may fail to find relevant information properly or provide incorrect or irrelevant answers. Ultimately, when document structure is lost, the meaning of the information is also lost, which becomes a major cause of RAG hallucinations.

Structural loss is not just a formatting issue. It is the starting point of RAG quality degradation that causes chunking, embedding, and search accuracy to collapse in a chain reaction. Parsing that preserves the structure must be performed first for all subsequent steps to work properly.

What is the Difference Between OCR and Document Layout Analysis (DLA)?

OCR (Optical Character Recognition) is a character recognition technology, while Document Layout Analysis (DLA) is a technology for understanding the overall structure of a document. Both technologies are needed to improve RAG quality.

Why Building RAG with OCR Alone is Difficult

OCR is a technology that converts characters from images or scanned documents into text. Since it is specialized for character recognition, it is difficult to preserve document structure information such as table structures, paragraph hierarchies, and reading order. In PDFs, the visual order and the actual extraction order can differ, leading to issues where semantic units get mixed up when using OCR alone.

On the other hand, Document Layout Analysis (DLA) identifies structural relationships, such as whether the extracted text is a table, a heading, or the body. It is thanks to this structural relationship identification that AI can understand the same number completely differently depending on whether it is a numerical value in a table or an explanation in the body text.

In summary, OCR is a technology for reading characters, while DLA is a technology for understanding the context of whether those characters represent a table or a heading. To improve RAG quality, both technologies must work together within a single pipeline.

The Impact of Preserving HWP/HWPX Structure on Search Accuracy

HWP and HWPX are formats with proprietary file structures. Using only general OCR or PDF conversion results in the loss of information regarding tables, footnotes, and indentation, which decreases RAG search accuracy.

Although these are the most widely used document formats in Korean public institutions, legal, and financial sectors, major global Document AI solutions such as AWS Textract, Azure Document Intelligence, and Google Document AI only support general-purpose formats like PDF, JPEG, PNG, and TIFF, and do not support direct parsing of HWP/HWPX originals. Direct parsing of HWP/HWPX originals is necessary to maintain RAG search accuracy.

Directly parsing HWP/HWPX originals requires core technology that can access the proprietary format structure. If processed by converting to PDF like general tools, key formatting such as footnotes, merged cells, and indentation may be lost.

💡 How to directly parse HWP/HWPX originals without conversion, based on Hancom’s 30 years of technology?

👉Explore Hancom Data Loader

RAG Chunking and Embedding: The Impact of Preprocessing Quality

Chunking is the process of dividing a document into semantic units, and embedding is the process of converting text into numerical vectors to make it searchable. Both stages depend on the quality of preprocessing.

Why Chunking Quality Determines Search Accuracy

Chunking is the process of dividing a document into semantic units, and the selection of a chunking strategy directly determines the Retrieval Accuracy of the RAG system.

The key to chunking is not just cutting text into small pieces, but dividing it while ensuring the meaning of paragraphs is fully maintained. Both Semantic Chunking and Hierarchical Chunking work effectively only when the document structure has been preserved beforehand. If Document Layout Analysis (DLA) is not performed first, any chunking method will hit a limit.

Whether it is semantic-based or hierarchy-based chunking, the document’s headings, paragraphs, and table hierarchies must first be accurately extracted to work effectively. This is why document parsing quality must be secured before choosing any chunking strategy.

Why Poor Chunking and Embedding Degrade AI Answer Quality

If chunking quality is low, the meaning becomes blurred during the embedding stage, which ultimately leads to a decrease in both vector DB search accuracy and AI answer quality.

Low chunking quality leads to separated contexts or broken semantic units, which also lowers the quality of embedding—the process of converting text into numerical vectors. Consequently, the search accuracy of the vector DB (a database for storing and retrieving embeddings) decreases, increasing the possibility of RAG hallucinations. This is why the cause of RAG embedding failure is most often found in the preprocessing stage rather than the LLM.

Frequently Asked Questions (FAQ) on RAG Preprocessing

How to Solve RAG Document Preprocessing with Hancom Data Loader

As we have seen, RAG performance is determined at the preprocessing stage before the LLM. The document structure must be fully preserved for chunking to maintain semantic units, for embedding quality to improve, and for vector DB search accuracy to be secured. If preprocessing fails, all subsequent steps fail with it.

The problem is that most documents handled in actual corporate environments, such as PDF, HWP, HWPX, and Microsoft Office documents, are in formats that are difficult to parse while preserving structure. In particular, since global Document AI solutions do not support direct parsing of HWP/HWPX originals, it is difficult to connect them to a RAG pipeline without structural loss using conventional methods.

Hancom Data Loader supports the entire process as a single pipeline, from document parsing—the very first step of RAG preprocessing—to structured data conversion.

✅ Processes complex AI involving Document Layout Analysis (DLA), OCR, and Table Structure Recognition (TSR) in a single pipeline to accurately extract structured text and object information within documents.

✅ Supports all formats actually used in corporate environments, such as HWP, HWPX, PDF, and OOXML (DOCX, XLSX, PPTX), and directly parses HWP/HWPX original files without conversion.

✅ Automatically determines the hierarchy of headings, paragraphs, and tables through Level Inference-based hierarchical structure extraction and outputs it as structured data, contributing to quality improvement in the subsequent chunking stage.

✅ Prevents external leakage of documents even in network-separated or closed-network environments through complete on-premise internalization based on Docker REST API.

✅ A proven solution in public, financial, and legal environments with GS certification.

If your current RAG answers are not meeting expectations, check your preprocessing environment with Hancom Data Loader before looking at the LLM.

👉 Go to Hancom Data Loader Live Demo

👉 Inquire about Hancom Data Loader implementation

---

References

1. arXiv, “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” Lewis et al., NeurIPS 2020
2. AWS, “What is Retrieval-Augmented Generation (RAG)?”
3. AWS, “What is OCR?”
4. Microsoft, “Chunk large documents for RAG and vector search in Azure AI Search”
5. arXiv, “Revolutionizing RAG with Enhanced PDF Structure Recognition,” Lin, arXiv:2401.12599
6. arXiv, “Document Parsing Unveiled: Techniques, Challenges, and Prospects,” Lin, arXiv:2410.21169
7. arXiv, “SCAN: Semantic Document Layout Analysis for Textual and Visual RAG,” Ueda et al., arXiv:2505.14381, EACL 2026
8. AWS Textract, “Supported Document Formats”
9. Microsoft Azure, “Azure Document Intelligence Overview”
10. Google, “Document AI Supported File Types”