A Perspective on Fine-Tuning and RAG Techniques for Indic Language Applications

This paper describes how Fine-Tuning (FT) and Retrieval-Augmented Generation (RAG) methods can be compared in improving the performance of large language models (LLMs) and are specifically targeted toward the application of the LLMs to Indic languages, especially Marathi. Our new approach for fine-tuning included the use of low-rank adaptation (LoRA) with 4-bit quantization and we created a RAG system that uses indic-sentencesimilarity- sbert to create embeddings. We used the Marathi translated Stanford Alpaca dataset and the Wikipedia corpus as the training datasets, and vLLM was used for the inference engine. Our experience with our previous work on fine-tuned Indic LLMs and with over 7,500 downloads informed our methodology choices in this paper. The results show that RAG was better than traditional fine-tuning on every metric measured, with RAG having higher precision, recall and F1 Score than traditional fine-tuning. In addition, BERTScore was used for evaluation and complete hyperparameter optimization to verify that RAG produced responses that were more contextually aware and factually grounded than traditional approaches, making RAG particularly useful for handling domain knowledge in populated but less-represented indigenous languages. Our research provides additional guidance to enhance the use of language models for processing indic languages while also improving linguistic accuracy and optimizing computational efficiency.