Improving the Accuracy of Halal Tourism Sentiment Classification Using a Stacking Ensemble Method Based on Machine Learning Algorithms
DOI:
https://doi.org/10.29103/sisfo.v10i2.27962Keywords:
Halal Tourism, Sentiment Classification, Stacking Ensemble, Machine Learning, SMOTEAbstract
Halal tourism has become an increasingly important sector due to the growing diversity of tourist preferences and the rapid development of digital platforms that generate large amounts of opinion data. However, sentiment classification in halal tourism remains challenging because tourist opinions often contain complex patterns and class imbalance. This study aims to improve the accuracy of halal tourism sentiment classification by applying a stacking ensemble approach based on machine learning algorithms. The proposed model combines Support Vector Machine (SVM), Multinomial Naïve Bayes (MNB), and AdaBoost as base learners, while Logistic Regression is used as the meta-learner to integrate the prediction results of the base models. The research process includes data preprocessing, class balancing using Synthetic Minority Over-sampling Technique (SMOTE), and model validation using 5-Fold Cross-Validation. The evaluation was conducted using accuracy, precision, recall, F1-score, confusion matrix, and ROC curve. The experimental results show that the stacking ensemble model achieved an average accuracy of 80.55%, outperforming the individual base models, namely SVM with 79.77%, MNB with 79.27%, and AdaBoost with 70.77%. In addition, the stacking model obtained an AUC value of 0.8795, indicating strong discriminative ability in distinguishing sentiment classes. These results demonstrate that the stacking ensemble approach can improve classification performance and provide a more robust and reliable model for halal tourism sentiment analysis. Therefore, the proposed method can support data-driven decision-making in halal tourism management and promotion.
References
[1] H. Ferdiansyah, “Pengembangan Pariwisata Halal di Indonesia Melalui Konsep Smart Tourism,” Journal of Sustainable Tourism Research, vol. 4, no. 1, pp. 30–34, 2020, doi: 10.24198/tornare.v2i1.25831.
[2] J. Ipmawati, S. Saifulloh, and K. Kusnawi, “Analisis Sentimen Tempat Wisata Berdasarkan Ulasan pada Google Maps Menggunakan Algoritma Support Vector Machine,” MALCOM: Indonesian Journal of Machine Learning and Computer Science, vol. 4, no. 1, pp. 247–256, Jan. 2024, doi: 10.57152/malcom.v4i1.1066.
[3] N. A. M. Roslan, N. M. Diah, Z. Ibrahim, Y. Munarko, and A. E. Minarno, “Automatic plant recognition using convolutional neural network on Malaysian medicinal herbs: the value of data augmentation,” International Journal of Advances in Intelligent Informatics, vol. 9, no. 1, pp. 136–147, Mar. 2023, doi: 10.26555/ijain.v9i1.1076.
[4] P. Lahagun, B. Devkota, S. Giri, and P. Budha, “Machine Learning-Based Social Media Review Analysis for Recommending Tourist Spots,” Journal of Engineering and Sciences, vol. 3, no. 1, pp. 45–52, Jun. 2024, doi: 10.3126/jes2.v3i1.66234.
[5] Q. Sui and S. K. Ghosh, “Active Learning for Stacking and AdaBoost-Related Models,” Stats (Basel)., vol. 7, no. 1, pp. 110–137, Mar. 2024, doi: 10.3390/stats7010008.
[6] H. Zhu and A. Zhu, “Application Research of the XGBoost-SVM Combination Model in Quantitative Investment Strategy,” in International Conference on Systems and Informatics, Kunming: Institute of Electrical and Electronics Engineers Inc., Jan. 2022, pp. 1–7. doi: 10.1109/ICSAI57119.2022.10005355.
[7] J. M. Ahn, J. Kim, and K. Kim, “Ensemble Machine Learning of Gradient Boosting (XGBoost, LightGBM, CatBoost) and Attention-Based CNN-LSTM for Harmful Algal Blooms Forecasting,” Toxins (Basel)., vol. 15, no. 10, pp. 1–15, Oct. 2023, doi: 10.3390/toxins15100608.
[8] Agusviyanda, Hamdani, M. K. Anam, Agustin, and M. I. Wibowo, “Stacking Ensemble Machine Learning Model for Early Detection of Chronic Kidney Disease in Indonesia,” Rabit : Jurnal Teknologi dan Sistem Informasi Univrab, vol. 10, no. 2, pp. 1244–1252, Jul. 2025, doi: 10.36341/rabit.v10i2.6501.
[9] M. M. R. Mubarak, Y. H. Chrisnanto, and P. N. Sabrina, “Implementation of Random Forest Using Smote and Smoteenn in Customer Churn Classification in E-Commerce,” Enrichment: Journal of Multidisciplinary Research and Development, vol. 1, no. 8, pp. 463–477, Nov. 2023, doi: 10.55324/enrichment.v1i8.69.
[10] H. Rhomadhona and J. Permadi, “Klasifikasi Berita Kriminal Menggunakan Naïve Bayes Classifier (NBC) dengan Pengujian K-Fold Cross Validation,” Jurnal Sains dan Informatika, vol. 5, no. 2, pp. 108–117, 2019, doi: 10.34128/jsi.v5i2.177.
[11] R. J. Dhanal and V. R. Ghorpade, “Aspect-based sentiment-analysis using topic modelling and machine-learning,” International Journal of Electrical and Computer Engineering, vol. 14, no. 6, pp. 6689–6698, Dec. 2024, doi: 10.11591/ijece.v14i6.pp6689-6698.
[12] E. Ikhwan Saputra, M. Khairul Anam, H. Yenni, Hamdani, and A. Zamsuri, “Optimalisasi Algoritma Support Vector Machine pada Aspect-Based Sentiment Analysis Menggunakan GridSearchCV,” Zonasi (Jurnal Sistem Informasi), vol. 7, no. 1, pp. 271–279, 2025, doi: 10.31849/zn.v7i1.17800.
[13] M. K. Anam, M. I. Mahendra, W. Agustin, Rahmaddeni, and Nurjayadi, “Framework for Analyzing Netizen Opinions on BPJS Using Sentiment Analysis and Social Network Analysis (SNA),” Intensif, vol. 6, no. 1, pp. 2549–6824, 2022, doi: 10.29407/intensif.v6i1.15870.
[14] B. N. Pikir, M. K. Anam, H. Asnal, Rahmaddeni, and T. A. Fitri, “Sentiment Analysis of Technology Utilization by Pekanbaru City Government Based on Community Interaction in Social Media,” JAIA – Journal Of Artificial Intelligence And Applications, vol. 2, no. 1, pp. 32–40, 2021.
[15] R. S. Putra, W. Agustin, M. K. Anam, L. Lusiana, and S. Yaakub, “The Application of Naïve Bayes Classifier Based Feature Selection on Analysis of Online Learning Sentiment in Online Media,” Jurnal Transformatika, vol. 20, no. 1, pp. 44–56, Jul. 2022, doi: 10.26623/transformatika.v20i1.5144.
[16] Junadhi, Agustin, M. Rifqi, and M. K. Anam, “Sentiment Analysis of Online Lectures using K-Nearest Neighbors based on Feature Selection,” Jurnal Nasional Pendidikan Teknik Informatika (JANAPATI), vol. 11, no. 3, pp. 216–225, Dec. 2022, doi: 10.23887/janapati.v11i3.51531.
[17] A. N. Ulfah, M. K. Anam, N. Y. S. Munti, S. Yaakub, and M. B. Firdaus, “Sentiment Analysis of the Convict Assimilation Program on Handling Covid-19,” JUITA: Jurnal Informatika, vol. 10, no. 2, pp. 209–216, 2022, doi: 10.30595/juita.v10i2.12308.
[18] I. Najiyah and I. Hariyanti, “Sentimen Analisis Covid-19 Dengan Metode Probabilistic Neural Network Dan Tf-Idf,” Jurnal Responsif : Riset Sains dan Informatika, vol. 3, no. 1, pp. 100–111, 2021, doi: 10.51977/jti.v3i1.488.
[19] G. Popoola, K. K. Abdullah, G. S. Fuhnwi, and J. Agbaje, “Sentiment Analysis of Financial News Data using TF-IDF and Machine Learning Algorithms,” in 2024 IEEE 3rd International Conference on AI in Cybersecurity, ICAIC 2024, Institute of Electrical and Electronics Engineers Inc., 2024, pp. 1–6. doi: 10.1109/ICAIC60265.2024.10433843.
[20] O. Chamorro-Atalaya et al., “K-Fold Cross-Validation through Identification of the Opinion Classification Algorithm for the Satisfaction of University Students,” International journal of online and biomedical engineering, vol. 19, no. 11, pp. 140–158, 2023, doi: 10.3991/ijoe.v19i11.39887.
[21] S. Rabbani, Agustin, Rahmiati, Susandri, and M. K. Anam, “Optimalisasi Kinerja Klasifikasi Teks Berdasarkan Analisis Berbasis Aspek dan Model Hybrid Deep Learing,” Indonesian Journal of Computer Science, vol. 13, no. 3, pp. 4838–4847, 2024, doi: 10.33022/ijcs.v13i3.4034.
[22] K. Rahayu, M. K. Anam, L. Efrizoni, Nurjayadi, and T. A. Fitri, “Optimasi Teknik Voting Pada Sentimen Analisis Pemilihan Presiden 2024 Menggunakan Machine Learning,” The Indonesian Journal of Computer Science, vol. 13, no. 4, pp. 6289–6302, Jul. 2024, doi: 10.33022/ijcs.v13i4.4119.
[23] M. K. Anam, S. Sumijan, K. Karfindo, and M. B. Firdaus, “Comparison Analysis of HSV Method, CNN Algorithm, and SVM Algorithm in Detecting the Ripeness of Mangosteen Fruit Images,” Indonesian Journal of Artificial Intelligence and Data Mining, vol. 7, no. 2, pp. 348–356, May 2024, doi: 10.24014/ijaidm.v7i2.29739.
[24] S. Susanti, A. A. Sari, M. K. Anam, M. Jamaris, and H. Hamdani, “Sistem Prediksi Keuntungan Influencer Pengguna E-Commerce Shopee Affiliates menggunakan Metode Naïve Bayes,” Edumatic: Jurnal Pendidikan Informatika, vol. 6, no. 2, pp. 394–403, Dec. 2022, doi: 10.29408/edumatic.v6i2.6787.
[25] C. Piao, N. Wang, and C. Yuan, “Rebalance Weights AdaBoost‐SVM Model for Imbalanced Data,” Comput. Intell. Neurosci., vol. 2023, no. 1, pp. 1–26, Jan. 2023, doi: 10.1155/2023/4860536.
[26] L. L. Van FC, M. K. Anam, S. Bukhori, A. K. Mahamad, S. Saon, and R. L. V. Nyoto, “The Development of Stacking Techniques in Machine Learning for Breast Cancer Detection,” Journal of Applied Data Sciences, vol. 6, no. 1, pp. 71–85, Jan. 2025, doi: 10.47738/jads.v6i1.416.
[27] F. A. Rafrastara, C. Supriyanto, C. Paramita, and Y. P. Astuti, “Deteksi Malware menggunakan Metode Stacking berbasis Ensemble,” Jurnal Informatika: Jurnal pengembangan IT, vol. 8, no. 1, pp. 11–16, 2023, doi: 10.30591/jpit.v8i1.4606.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Nadya Rahmi, Ilmawati Ilmawati
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors retain copyright and grant the journal right of first publication and this work is licensed under a Creative Commons Attribution-ShareAlike 4.0 that allows others to share the work with an acknowledgement of the works authorship and initial publication in this journal.
All articles in this journal may be disseminated by listing valid sources and the title of the article should not be omitted. The content of the article is liable to the author.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
In the dissemination of articles by the author must declare the Sisfo: Jurnal Ilmiah Sistem Informasi as the first party to publish the article.
