DIAJENG - Evaluation ChatGPT User Sentiment using Naive Bayes: A Review of Confusion Matrix and Classification Report

Evaluation ChatGPT User Sentiment using Naive Bayes: A Review of Confusion Matrix and Classification Report

Jurnal Riset Sistem dan Teknologi Informasi

Universitas 'Aisyiyah Surakarta

📄 Abstract

The development of artificial intelligence (AI) technology, particularly in natural language processing (NLP), has led to various innovations, including ChatGPT. Its growing popularity highlights the need for user sentiment analysis. This study evaluates user sentiment toward ChatGPT using the Naive Bayes algorithm. The dataset, obtained from Kaggle, consists of 500 labeled English tweets categorized as positive, neutral, or negative. The process involved text preprocessing, TF-IDF feature extraction, data splitting (80% training, 20% testing), and model training. The results show an accuracy of 56%, with the highest f1-score in the negative class (0.67) and the lowest in the neutral class (0.38). The model exhibits classification imbalance, with high precision but low recall in the neutral class, and high recall but low precision in the positive class. The confusion matrix further confirms frequent misclassifications between classes. These findings reflect the limitations of Naive Bayes in handling contextual relationships in text data. Improvements can be achieved through data balancing, enhanced NLP-based feature representation, and the application of more complex classification algorithms.

🔖 Keywords

#AI; ChatGPT; Kaggle; NLP; Naive Bayes; AI; ChatGPT; Kaggle; NLP; Naive Bayes

ℹ️ Informasi Publikasi

Tanggal Publikasi

31 July 2025

Volume / Nomor / Tahun

Volume 3, Nomor 2, Tahun 2025

📝 HOW TO CITE

Dianda Rifaldi; Tri Stiyo Famuji; Bella Okta Sari Miranda; Fauzan Purma Ramadhan; Iriene Putri Mulyadi; Vanji Saputra6; Fanani, Galih Pramuja Inngam, "Evaluation ChatGPT User Sentiment using Naive Bayes: A Review of Confusion Matrix and Classification Report," Jurnal Riset Sistem dan Teknologi Informasi, vol. 3, no. 2, Jul. 2025.

ACM

ACS

APA

ABNT

Chicago

Harvard

IEEE

MLA

Turabian

Vancouver

Evaluation ChatGPT User Sentiment using Naive Bayes: A Review of Confusion Matrix and Classification Report

📄 Abstract

🔖 Keywords

ℹ️ Informasi Publikasi

📝 HOW TO CITE

Download Citation

📚 References & Citations

🔗 Artikel Terkait dari Jurnal yang Sama

Evaluation of Multimedia Laboratory Infrastructure of 'Asiyiyah University Surakarta using COBIT 5.0 Method

Design Agricultural Drone System Based on IT Governance using TOGAF Approach

Artificial Neural Network-Based Diabetes Prediction Analysis Using CDC Diabetes Health Indicators Data: Analisis dan Prediksi Diabetes Menggunakan Artificial Neural Network dengan Dataset CDC Diabetes Health Indicators

Depression Prediction Among University Students Using a Random Forest Algorithm Based on Psychosocial Data: Prediksi Depresi Mahasiswa: Pendekatan Berbasis Data Psikososial Menggunakan Algoritma Random Forest

Decision Support System For Diabetes Diagnosis Using Simple Additive Weighting Method

Classification of Imbalanced Data Using Random Forest Algorithm with SMOTE and SMOTE-ENN (Case Study on Stunting Data

📊 Statistik Sitasi Jurnal

Tren Sitasi per Tahun

Artikel Tersitasi Lainnya

PENERAPAN METODE SUPPORT VECTOR MACHINE (SVM) PADA KLASIFIKA...

Optimisasi Algoritma Genetika dengan Particle Swarm Optimiza...

Analisis dan Pengujian Kerentanan Website Menggunakan OWASP...

Optimalisasi Desain UI/UX untuk Meningkatkan Aksesibilitas T...

Implementation of the Bootstrap Framework in Creating an Inf...