Pengembangan Chatbot Informasi Hukum Layanan Publik Berbasis Retrieval-Augmented Generation Menggunakan LangChain dan OpenAI di Ombudsman DIY
DOI:
https://doi.org/10.52436/1.jpti.995Kata Kunci:
Chatbot Hukum, Kecerdasan Buatan, LangChain, Layanan Publik, Ombudsman, Retrieval-Augmented GenerationAbstrak
Hubungan antara masyarakat dan lembaga layanan publik kerap menghadapi berbagai tantangan, khususnya dalam praktik maladministrasi, penyalahgunaan wewenang, dan kurangnya transparansi. Untuk menjawab permasalahan tersebut, penelitian ini mengusulkan pengembangan chatbot berbasis kecerdasan buatan generatif dengan pendekatan Retrieval-Augmented Generation (RAG) menggunakan LangChain dan OpenAI. Sistem ini dirancang untuk menyajikan informasi hukum yang akurat, kontekstual, dan mudah dipahami oleh masyarakat. Metode yang digunakan dalam penelitian ini meliputi perancangan sistem chatbot dengan arsitektur RAG menggunakan LangChain dan OpenAI. Dokumen hukum diolah menjadi embeddings, disimpan dalam basis data vektor Chroma, dan digunakan dalam proses prompt engineering untuk menghasilkan jawaban yang kontekstual. Evaluasi sistem dilakukan melalui penyebaran kuesioner kepada lima ahli dari Lembaga Ombudsman DIY, dengan analisis data menggunakan pendekatan deskriptif. Evaluasi sistem dilakukan dengan memberikan kuesioner kepada lima ahli dari LO DIY dan menggunakan analisis deskrpitif. Hasil evaluasi menunjukkan bahwa sistem memperoleh skor tinggi dalam indikator Perceived Usefulness (rata – rata = 12) dan Relevansi (rata – rata = 8), serta skor sangat tinggi dalam indikator Akurasi (rata – rata = 18,6) dan indikator Clarity (rata – rata = 8,4). Dengan demikian, penerapan teknologi RAG dalam pengembangan chatbot berpotensi meningkatkan pemahaman masyarakat terhadap hukum layanan publik serta memperkuat transparansi dan akuntabilitas dalam penyelenggaraan pelayanan publik. Hal ini menunjukkan potensi strategis pemanfaatan AI dalam mendorong tata kelola pelayanan publik yang lebih responsif, akuntabel, dan inklusif.
Unduhan
Referensi
Y. E. N. Ida, Memahami Diskresi Lembaga Ombudsman DIY: Perspektif Pelayanan Publik. Daerah Istimewa Yogyakarta: Deepublish, 2025.
Lembaga Ombudsman DIY, Laporan Kinerja Archives - Lembaga Ombudsman DIY. 2025. [Online]. Available: https://ombudsman.jogjaprov.go.id/category/publikasi/laporan/
Z. D. Prasetio, N. Setiawan, and M. Ariffin, “MEMBANGUN LITERASI HUKUM DAN PENDIDIKAN DALAM MASYARAKAT,” Dianmas Bhakti: Jurnal Pengabdian Pada Masyarakat, vol. 1, no. 1, pp. 13–17, Oct. 2024, doi: 10.54035/dianmas.v1i1.480.
Y. Febrianty, A. Ariyanto, H. Fitri, and N. R. Ryendra, “The effect of legal education on public legal awareness,” Journal of Public Representative and Society Provision, vol. 5, no. 1, pp. 204–221, Mar. 2025, doi: 10.55885/jprsp.v5i1.532.
H. Abburi, M. Suesserman, N. Pudota, B. Veeramani, E. Bowen, and S. Bhattacharya, “Generative AI Text Classification using Ensemble LLM Approaches,” arXiv (Cornell University), Jan. 2023, doi: 10.48550/arxiv.2309.07755.
P. Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP tasks,” arXiv (Cornell University), Jan. 2020, doi: 10.48550/arxiv.2005.11401.
S. Prasad, H. Gupta, and A. Ghosh, “Leveraging the potential of large language models,” Informatica, vol. 48, no. 8, May 2024, doi: 10.31449/inf.v48i8.5635.
J. G. Meyer et al., “ChatGPT and large language models in academia: opportunities and challenges,” BioData Mining, vol. 16, no. 1, Jul. 2023, doi: 10.1186/s13040-023-00339-9.
S. Wu et al., “Retrieval-Augmented Generation for Natural Language Processing: A survey,” arXiv (Cornell University), Jul. 2024, doi: 10.48550/arxiv.2407.13193.
H. Cui et al., “CURIE: Evaluating LLMs on Multitask Scientific Long Context Understanding and Reasoning,” arXiv (Cornell University), Mar. 2025, doi: 10.48550/arXiv.2503.13517.
Y. Gao et al., “Retrieval-Augmented Generation for Large Language Models: A survey,” arXiv (Cornell University), Jan. 2023, doi: 10.48550/arxiv.2312.10997.
A. T. U. Br. Lubis, N. S. Harahap, S. Agustian, M. Irsyad, and I. Afrianty, “Question Answering System pada Chatbot Telegram Menggunakan Large Language Models (LLM) dan Langchain (Studi Kasus UU Kesehatan),” MALCOM Indonesian Journal of Machine Learning and Computer Science, vol. 4, no. 3, pp. 955–964, May 2024, doi: 10.57152/malcom.v4i3.1378.
N. A. B. Permadi, N. N. S. H, N. L. Handayani, and N. Yusra, “IMPLEMENTASI QUESTION ANSWERING SYSTEM TAFSIR AL-AZHAR MENGGUNAKAN LANGCHAIN DAN LARGE LANGUAGE MODEL BERBASIS CHATBOT TELEGRAM,” Jurnal Teknoif Teknik Informatika Institut Teknologi Padang, vol. 12, no. 1, pp. 62–69, Apr. 2024, doi: 10.21063/jtif.2024.v12.1.62-69.
E. R. Aquino, P. De Saqui-Sannes, and R. A. Vingerhoeds, “A methodological assistant for UML and SYSML use case diagrams,” in Communications in computer and information science, 2021, pp. 298–322. doi: 10.1007/978-3-030-67445-8_13.
P. Jha, M. Sahu, and T. Isobe, “A UML Activity Flow Graph-Based Regression Testing approach,” Applied Sciences, vol. 13, no. 9, p. 5379, Apr. 2023, doi: 10.3390/app13095379.
Davis, F. D. (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, 13(3), 319-340.
M. Zou and L. Huang, “To use or not to use? Understanding doctoral students’ acceptance of ChatGPT in writing through technology acceptance model,” Frontiers in Psychology, vol. 14, Oct. 2023, doi: 10.3389/fpsyg.2023.1259531.
I. Iancu and B. Iancu, “Interacting with chatbots later in life: A technology acceptance perspective in COVID-19 pandemic situation,” Frontiers in Psychology, vol. 13, Jan. 2023, doi: 10.3389/fpsyg.2022.1111003.
S. Alagarsamy and S. Mehrolia, “Exploring chatbot trust: Antecedents and behavioural outcomes,” Heliyon, vol. 9, no. 5, p. e16074, May 2023, doi: 10.1016/j.heliyon.2023.e16074.
R. S. Goodman et al., “Accuracy and reliability of Chatbot responses to physician questions,” JAMA Network Open, vol. 6, no. 10, p. e2336483, Oct. 2023, doi: 10.1001/jamanetworkopen.2023.36483.
V. Ponzo et al., “Comparison of the accuracy, completeness, reproducibility, and consistency of different AI chatbots in providing nutritional advice: an exploratory study,” Journal of Clinical Medicine, vol. 13, no. 24, p. 7810, Dec. 2024, doi: 10.3390/jcm13247810.
M. W. Wagner and B. B. Ertl-Wagner, “Accuracy of information and references using ChatGPT-3 for retrieval of clinical radiological information,” Canadian Association of Radiologists Journal, vol. 75, no. 1, pp. 69–73, Apr. 2023, doi: 10.1177/08465371231171125.
S. Holmes, R. Bond, A. Moorhead, J. Zheng, V. Coates, and M. McTear, “Towards validating a chatbot usability scale,” in Lecture notes in computer science, 2023, pp. 321–339. doi: 10.1007/978-3-031-35708-4_24.
P. Gupta et al., “Answerability: A custom metric for evaluating chatbot performance,” ACL Anthology, pp. 316–325, Jan. 2022, doi: 10.18653/v1/2022.gem-1.27.
R. Olszewski, K. Watros, M. Ma?czak, J. Owoc, K. Jeziorski, and J. Brzezi?ski, “Assessing the response quality and readability of chatbots in cardiovascular health, oncology, and psoriasis: A comparative study,” International Journal of Medical Informatics, vol. 190, p. 105562, Oct. 2024, doi: 10.1016/j.ijmedinf.2024.105562.
E. Abouzeid, R. Wassef, A. Jawwad, and P. Harris, “Chatbots’ role in generating single best answer Questions for Undergraduate medical Student Assessment: Comparative analysis,” JMIR Medical Education, vol. 11, p. e69521, May 2025, doi: 10.2196/69521.
J. Y.-S. Yau et al., “Comparison and Accuracy of Prospective Assessments of Four Large Language Model Chatbot Responses to Patient Questions about Emergency Care (Preprint),” Journal of Medical Internet Research, vol. 26, p. e60291, Sep. 2024, doi: 10.2196/60291.
O. Chalyi, “An Evaluation of General-Purpose AI Chatbots: A comprehensive Comparative analysis,” InfoScience Trends, vol. 1, no. 1, pp. 52–66, Jun. 2024, doi: 10.61186/ist.202401.01.07.
G. Liu and C. Ma, “Measuring EFL learners’ use of ChatGPT in informal digital learning of English based on the technology acceptance model,” Innovation in Language Learning and Teaching, vol. 18, no. 2, pp. 125–138, Jul. 2023, doi: 10.1080/17501229.2023.2240316.
D. Johnson et al., “Assessing the accuracy and reliability of AI-Generated Medical Responses: An evaluation of the Chat-GPT model,” Research Square (Research Square), Feb. 2023, doi: 10.21203/rs.3.rs-2566942/v1.
Z. Ye et al., “An assessment of ChatGPT’s responses to frequently asked questions about cervical and breast cancer,” BMC Women S Health, vol. 24, no. 1, Sep. 2024, doi: 10.1186/s12905-024-03320-8.
R. Maroncelli et al., “Probing clarity: AI-generated simplified breast imaging reports for enhanced patient comprehension powered by ChatGPT-4o,” European Radiology Experimental, vol. 8, no. 1, Oct. 2024, doi: 10.1186/s41747-024-00526-1.
Setiyawami and Sugiyono, Metode Penelitian Sumber Daya Manusia, 1st ed. ALFABETA, 2022.
