EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Enhancing RAG Performance Through Chunking and Text Splitting Techniques

Aadit Kshirsagar

International Journal of Scientific Research in Computer Science, Engineering and Information Technology · 01.11.2024 · 10 (5) · 151-158

DOI: 10.32628/cseit2410593 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Scientific Research in Computer Science, Engineering and Information Technology, online erschienen am 01.11.2024. Verfasst von Aadit Kshirsagar. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In the world of Generative Artificial Intelligence (GenAI) and Large Language Models (LLM), Retrieval-Augmented Generation (RAG) has transformed the way we interact with data. Using RAG, these models can leverage new data contexts to respond to user queries and gain valuable insights. Behind the outstanding capabilities of RAG, a fundamental pre-processing step is present known as chunking. This step plays a crucial role in the effectiveness of these RAG-enhanced models. Chunking involves the breaking down of large text or documents into smaller segments of a fixed size. This allows the retriever to focus on smaller units at a time, making it easier to process and analyse the text. Finding the ideal chunking strategy can be a challenging task. Experimenting and analysis play a decisive role here, as different chunking strategies cater to different use cases. This paper, mainly targeted for an audience that is exploring RAG tuning techniques for higher accuracy, explores the various chunking techniques and their practical implementation using code snippets. After analysing the results for various use cases, the paper also suggests the best use cases for the different chunking strategies. Finally, it concludes by discussing the future potential and extending scope of RAG-enhanced applications.

Zitieren

APA

Kshirsagar, A. (2024). Enhancing RAG Performance Through Chunking and Text Splitting Techniques. International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 10(5), 151-158. https://doi.org/10.32628/cseit2410593

Vancouver

Kshirsagar A. Enhancing RAG Performance Through Chunking and Text Splitting Techniques. International Journal of Scientific Research in Computer Science, Engineering and Information Technology. 2024;10(5):151-158. doi:10.32628/cseit2410593

BibTeX
@article{Kshirsagar2024,
  author = {Kshirsagar, Aadit},
  title = {Enhancing RAG Performance Through Chunking and Text Splitting Techniques},
  journal = {International Journal of Scientific Research in Computer Science, Engineering and Information Technology},
  year = {2024},
  volume = {10},
  number = {5},
  pages = {151--158},
  doi = {10.32628/cseit2410593}
}
Kostenlos starten →