Web Application for Retrieval-Augmented Generation: Implementation and Testing

Source abstract

The purpose of this paper is to explore the implementation of Retrieval-Augmented Gen- 14 eration (RAG) technology with open-source Large Language Models (LLMs). For this purpose, a 15 web-based application named PaSSER that integrates RAG with three models - Mistral:7b, 16 Llama2:7b: Orca2:7b, is developed. In its development various technologies are used, including 17 Blockchain for managing and storing assessment results of built-in testing modules. PaSSER em- 18 ploys a set of evaluation metrics for LLMs’ performance - METEOR, ROUGE, BLEU, Perplexity, 19 Cosine similarity, Pearson correlation, and F1 score. The specialized knowledge base is in the smart 20 agriculture domain. As an illustration, the paper presents results and analysis of two tests. First one 21 is assessing the performance of LLMs across different hardware configurations, the other is deter- 22 mining the model that delivers most accurate and contextually relevant responses within the RAG. 23 The paper also discusses the integration of blockchain technology with LLMs to manage and store 24 assessment results within a blockchain environment. The tests revealed that Mistral:7b demon- 25 strates the best results across most of the evaluated metrics. The discussion section comments on 26 technical and hardware considerations that affect the performance of LLMs. Future development 27 will focus in leveraging new larger pre-trained open-source LLMs, exploring different finetuning 28 approaches, and further integration with Blockchain and IPFS.