A Comparative Study on the Combination of Classification Algorithm and Language Model Implementation for Smart Accounting System

Micro, Small, and Medium Enterprises (MSMEs) normally dealing with financial documentation and reporting problems due to in sufficient budget for hiring professional accounting services. Although some of them might have utilized off the shelf accounting softwares, they still face many obstacles in compiling a proper financial documentation because the employed software do not have an automatic transaction classification capability to assist users in recording any transactions. This study was aimed to investigate the opportunity of implementing automatic transaction classification for accounting system by using a Natural Language Processing (NLP) approach to automatically interpret the suitable account for any financial transactions based on the text written on the transaction forms. An experiment was conducted to compare the performance of eight combinations comprising of four classification algorithms (i.e. SVM, KNN3, KNN5, and NB) with two language models (i.e. TF-IDF and BoW). The result showed that KNN5 and TF-IDF pair gave highest performance with accuracy 82,5%, precision 82,54%, recall/sensitivity 83,7%, specificity 92,06%, and F1 Score 81,5%.