Retrieval-Augmented Generation for Taxpayer Advisory in an LMIC Setting: A Proof-of-Concept Evaluation of Statutory Legal Retrieval

Source abstract

Abstract Objective Tax administrations in low- and middle-income country settings are expanding digital services, yet reliable, legally grounded taxpayer guidance remains difficult to scale. This study developed and evaluated a proof-of-concept retrieval-augmented generation framework for taxpayer advisory using national income-tax legislation as an authoritative knowledge source. The objective was to test whether semantic retrieval could identify statutory provisions accurately enough to support grounded downstream language-model prompting. The legislation was transformed into 198 knowledge units, embedded with all-MiniLM-L6-v2, indexed in ChromaDB, and evaluated using 900 automatically generated questions linked to expected provisions. Results The retriever achieved 72.00% Top-1 accuracy, 86.78% Top-3 accuracy, 28.93% Precision@3, 86.78% Recall@3, and mean reciprocal rank 0.7880. Many Top-1 errors were recovered within the first three results, indicating ranking ambiguity among semantically related provisions rather than complete retrieval failure. The prototype integrated legal document processing, vector search, provision retrieval and structured prompt construction in an end-to-end workflow. The results support technical feasibility for grounded taxpayer information, but generation quality, legal correctness, usability and compliance effects were not evaluated. Validation should include additional tax instruments, real taxpayer queries, professional legal review, multilingual support and cross-jurisdiction testing before deployment in resource-constrained public-service settings.