| Mitigating Retrieval Errors in RAG Pipelines Through Ambiguity Detection and Interactive Feedback |
| Paper ID : 1080-ICEEM2025 (R1) |
| Authors |
|
Mahmoud Ibrahim Mahmoud *1, Walaa Medhat Asal2 1Fuculity of computer science Nile University 2Faculty of Computer Science, Nile University |
| Abstract |
| Retrieval-Augmented Generation (RAG) pipelines often struggle when multiple retrieved documents are semantically similar, resulting in ambiguous top-ranked outputs that degrade the quality of the generated responses. This ambiguity increases the risk of hallucinations, especially when the correct passage is not clearly distinguishable from other candidates. While existing solutions—such as linguistic preprocessing, score-based filtering, and retriever fine-tuning—have improved overall retrieval accuracy, they remain insufficient in addressing ambiguity caused by semantically close passages. In this work, we propose a novel interactive retrieval framework that detects ambiguous retrieval scenarios by analyzing the score distribution of top-ranked documents. When ambiguity is detected, the system selectively engages the user by presenting a list of candidate document titles, allowing them to choose the most relevant one. A dedicated classifier is trained to distinguish between confident and ambiguous cases, ensuring minimal disruption to the user experience. Experiments on an Arabic QA dataset demonstrate significant improvements in top-1 retrieval accuracy, validating the effectiveness of incorporating ambiguity detection and selective feedback into RAG pipelines. |
| Keywords |
| Retrieval Augmented Generation (RAG); Ambiguous Results; Interactive Question Answering; Multilingual Retrieval; User-in-the-Loop; Ambiguity Classifier; Knowledge Graph. |
| Status: Accepted |