Klasifikasi dan Pengenalan Emosi dari Ekspresi Wajah Menggunakan CNN-BiLSTM dengan Teknik Data Augmentation

Sutarti; Fariza Syaqialloh

doi:10.51454/decode.v5i1.1038

Authors

Sutarti Sistem Komputer Universitas Serang Raya
Fariza Syaqialloh Sistem Komputer Universitas Serang Raya

DOI:

https://doi.org/10.51454/decode.v5i1.1038

Keywords:

CNN-BiLSTM, Data Augmentation, Facial Expression Recognition, Performa

Abstract

Teknologi computer vision terus berkembang dan diterapkan dalam berbagai bidang, termasuk interaksi manusia dan komputer (human-computer interaction), khususnya dalam pengenalan emosi dari ekspresi wajah atau facial expression recognition (FER). Penelitian ini bertujuan untuk mengembangkan model CNN-BiLSTM yang dilengkapi dengan data augmentation untuk mendeteksi emosi seperti marah, bahagia, takut, sedih, terkejut, jijik, dan netral. Dataset yang digunakan dalam penelitian ini adalah dataset publik FER2013, yang terdiri dari 35.887 gambar wajah grayscale berukuran 48x48 piksel. Teknik augmentasi data meliputi rotasi, flipping, dan pergeseran gambar untuk meningkatkan keragaman data dan mencegah overfitting. Model dilatih menggunakan optimizer Adam, dengan hyperparameter berupa batch size sebesar 32, jumlah epoch sebanyak 100, dan learning rate 0.001. Hasil pengujian menunjukkan bahwa model CNN-BiLSTM mencapai akurasi validasi sebesar 95%, lebih tinggi dibandingkan CNN yang hanya mencapai akurasi 70%. Selain itu, validation loss pada CNN-BiLSTM lebih stabil, menunjukkan kemampuan generalisasi yang lebih baik dibandingkan CNN, yang mengalami fluktuasi signifikan akibat kemungkinan overfitting dan kurang optimalnya parameter seperti learning rate. Analisis menggunakan confusion matrix menunjukkan bahwa CNN-BiLSTM menghasilkan klasifikasi yang lebih akurat untuk setiap emosi. Model CNN-BiLSTM memberikan kinerja lebih unggul dalam mendeteksi emosi pada FER, terutama dalam menghadapi tantangan seperti pencahayaan, posisi wajah, dan kompleksitas ekspresi.

References

Akhand, M.A.H., Roy, S., Siddique, N., Kamal, M.A.S., and Shimamura, T. (2021). Facial Emotion Recognition Using Transfer Learning in the Deep CNN. Electronics 2021, 10(9), 1036. https://doi.org/10.3390/electronics10091036.

Adiatma, B. C. L., Utami, E., & Hartanto, A. D. (2021). Pengenalan Ekspresi Wajah Menggunakan Deep Convolutional Neural Network. Explore, 11(2), 75. https://doi.org/10.35200/explore.v11i2.478

Algarni, M., Saeed, F., Al-Hadhrami, T., Ghabban, F., & Al-Sarem, M. (2022). Deep Learning-Based Approach for Emotion Recognition Using Electroencephalography (EEG) Signals Using Bi-Directional Long Short-Term Memory (Bi-LSTM). Sensors, 22(8). https://doi.org/10.3390/s22082976

Almeida, J., Vilaça, L., Teixeira, I. N., & Viana, P. (2021). applied sciences Emotion Identification in Movies through Facial Expression Recognition. Appl. Sci, 11(15), 6827. https://doi.org/10.3390/app11156827

Avtar, R., & Dhingra, S. (2023). Measurement : Sensors Empirical analysis of multiple modalities for emotion recognition using convolutional neural network. Measurement: Sensors, 26(January), 100716. https://doi.org/10.1016/j.measen.2023.100716

Białek, C., Matiolański, A., and Grega, M. (2023). An Efficient Approach to Face Emotion Recognition with Convolutional Neural Networks, Electronics 2023, 12(12), 1–21. https://doi.org/10.3390/electronics12122707.

Budiman, A., Fabian, Yaputera, R. A., Achmad, S., & Kurniawan, A. (2022). Student attendance with face recognition (LBPH or CNN): Systematic literature review. Procedia Computer Science, 216, 31–38. https://doi.org/10.1016/j.procs.2022.12.108

Bursic, S., Boccignone, G., Ferrara, A., & Amelio, A. D. (2020). Improving the Accuracy of Automatic Facial Expression Recognition in Speaking Subjects with Deep Learning, applied sciences 1–15. https://doi.org/10.3390/app10114002

Cahya, F. N., Hardi, N., Riana, D., & Hadiyanti, S. (2021). Klasifikasi Penyakit Mata Menggunakan Convolutional Neural Network (CNN). Sistemasi, 10(3), 618. https://doi.org/10.32520/stmsi.v10i3.1248

Canedo, D., & Neves, A. J. R. (2019). Facial expression recognition using computer vision: A systematic review. Applied Sciences (Switzerland), 9(21), 1–31. https://doi.org/10.3390/app9214678

Gautam, C., & Seeja, K. R. (2023). Emotion recognition using Handcrafted features and CNN Conference on Chahak using Facial emotion recognition Handcrafted features and CNN. Procedia Computer Science, 218(2022), 1295–1303. https://doi.org/10.1016/j.procs.2023.01.108

Krishnaveni, K. (2023). A novel framework using binary attention mechanism based deep convolution neural network for face emotion recognition. Measurement: Sensors, 30, 100881. https://doi.org/10.1016/j.measen.2023.100881

Meena Jindal, & Khushwant Kaur. (2024). Enhancing Emotion Recognition through Multimodal Systems and Advanced Deep Learning Techniques. International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 10(3), 656–661. https://doi.org/10.32628/cseit24103216

Mellouk, W., & Handouzi, W. (2020). Facial emotion recognition using deep learning : review and insights Facial emotion recognition using deep learning : review and insights the Future. Procedia Computer Science, 175, 689–694. https://doi.org/10.1016/j.procs.2020.07.101

Meyer-Lindenberg, H., Moessnang, C., Oakley, B., Ahmad, J., Mason, L., Jones, E. J. H., … Loth, E. (2022). Facial expression recognition is linked to clinical and neurofunctional differences in autism. Molecular Autism, 13(1), 1–18. https://doi.org/10.1186/s13229-022-00520-7

Ming, Y., Qian, H., & Guangyuan, L. (2022). CNN-LSTM Facial Expression Recognition Method Fused with Two-Layer Attention Mechanism,. Computational Intelligence and Neuroscience Volume 2022, 1-9. https://doi.org/10.1155/2022/7450637

Paraskevopoulou, G., Spyrou, E., & Perantonis, S. (2022). A Data Augmentation Approach for Improving the Performance of Speech Emotion Recognition, Sigmap, 61–69. https://doi.org/10.5220/0011148000003289

Qi, X., Wu, C., Shi, Y., Qi, H., Duan, K., & Wang, X. (2023). A Convolutional Neural Network Face Recognition Method Based on BiLSTM and Attention Mechanism. Computational Intelligence and Neuroscience, 2023(1). https://doi.org/10.1155/2023/2501022

Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on Image Data Augmentation for Deep Learning. Journal of Big Data, 6(1). https://doi.org/10.1186/s40537-019-0197-0

Suartika E. P, I Wayan, Wijaya Arya Yudhi, S. R. (2016). Klasifikasi Citra Menggunakan Convolutional Neural Network (CNN) Pada Caltech 101. Jurnal Teknik ITS, 5(1), 76. Retrieved from http://repository.its.ac.id/48842/

Teja Chavali, S., Tej Kandavalli, C., Sugash, T. M., & Subramani, R. (2022). Smart Facial Emotion Recognition with Gender and Age Factor Estimation. Procedia Computer Science, 218(2022), 113–123. https://doi.org/10.1016/j.procs.2022.12.407

Widodo, S., Setiawan, D., Ridwan, T., & Ambari, R. (2022). Perancangan Deteksi Emosi Manusia berdasarkan Ekspresi Wajah Menggunakan Algoritma VGG16. Syntax : Jurnal Informatika, 11(01), 01–12. https://doi.org/10.35706/syji.v11i01.6594