Klasifikasi Keluhan Pengguna Berbasis Large Language Model untuk Layanan Digital

Abstract

The rapid growth of digital services such as e-commerce applications, digital banking, and app-based public services has significantly increased the volume of user complaints, making manual complaint classification inefficient. This study aims to develop and evaluate a Large Language Model (LLM)-based approach for classifying user complaints on digital services using a zero-shot prompting scheme. A small sample of 60 user complaints was collected and grouped into five categories: customer service, application or technical issues, payment and transaction, delivery, and data security and privacy. The research method consists of data collection, text preprocessing, labelling, LLM-based classification, and model performance evaluation. Testing on the small sample shows that the LLM was able to classify complaints with an accuracy of 86.7%, a weighted precision of 87.2%, a weighted recall of 86.7%, and a weighted F1-score of 86.8%. Most misclassifications occurred between contextually overlapping categories, namely payment or transaction and application or technical issues. The findings indicate that LLMs have strong potential as an efficient and accurate solution to support automated user complaint handling in digital services.