Retrieval-Augmented Generation for Large Language Models: Architectures, Applications, and Challenges

Abstract

Large Language Models (LLMs) exhibit impressive generative capabilities but remain limited by static parametric knowledge, hallucination, and the inability to access domain-specific or up-to-date information. Retrieval-Augmented Generation (RAG) addresses these limitations by coupling LLMs with external retrieval mechanisms that ground generation in retrieved evidence. This paper presents a systematic study of RAG architectures, tracing their evolution from naive retrieve-then-read pipelines to advanced modular and agentic designs incorporating iterative retrieval, reranking, and self-reflection. We categorize architectural variants along retrieval, indexing, and generation dimensions; survey applications across question answering, healthcare, legal, and enterprise domains; and report an experimental comparison of sparse (BM25), dense (bi-encoder), and hybrid retrieval configurations on a domain-specific question answering benchmark. Our results show that hybrid retrieval with cross-encoder reranking improves answer accuracy by up to 11.4% over naive dense retrieval while reducing hallucination rates. We conclude by analyzing open challenges—retrieval noise, context window limitations, evaluation methodology, latency, and privacy—and outline directions for future research.