Evaluasi Kinerja Model Transformer: Mengukur Kesesuaian Capaian Pembelajaran Mata Kuliah
DOI:
https://doi.org/10.52436/1.jpti.1213Kata Kunci:
BERT, CPL, CPMK, Evaluasi Kurikulum, SBERT, TransformerAbstrak
Kurikulum berorientasi pada hasil (Outcome-Based Education/OBE) menjadi pendekatan penting dalam pendidikan tinggi untuk memastikan kesesuaian antara Capaian Pembelajaran Lulusan (CPL) dan kebutuhan dunia kerja. Namun, tantangan dalam implementasinya masih muncul, khususnya dalam memastikan konsistensi antara CPL dan Capaian Pembelajaran Mata Kuliah (CPMK). Penelitian ini mengevaluasi efektivitas dua model transformer berbasis deep learning, BERT dan SBERT, dalam mengotomatisasi proses evaluasi kesesuaian CPL–CPMK. Eksperimen dilakukan pada data kurikulum dari Universitas 'Aisyiyah Surakarta. Hasil menunjukkan bahwa BERT unggul dalam tugas klasifikasi teks dengan nilai akurasi 0,61, precision 0,72, dan F1 score 0,73. Sebaliknya, SBERT menunjukkan recall lebih tinggi (0,75), yang berguna dalam konteks minimalisasi kesalahan tipe dua. Temuan ini menjadikan BERT sebagai model yang lebih cocok digunakan untuk mengukur keselarasan antara CPL dengan CPMK
Unduhan
Referensi
I. arum tri Rahayu, L. Hidayati, I. Nahari, E. E. Saputra, and N. A. P. Reztanty, “Analisa Ketercapaian Program Learning Outcomes Berbasis Obe Pada Program Studi S-1 Pendidikan Tata Busana Fakultas Teknik Universitas Negeri Surabaya,” J. Vocat. Tech. Educ., vol. 4, no. 2, pp. 33–43, 2022, doi: 10.26740/jvte.v4n2.p33-43.
M. I. Muzakir and Susanto, “Implementasi Kurikulum Outcome Based Education (Obe) Dalam Sistem Pendidikan Tinggi Di Era Revolusi Industri 4.0 ((Implementation of the Outcome Based Education (OBE) Curriculum in the Higher Education System in the Era of Industrial Revolution 4.0),” Edukasiana J. Islam. Educ., vol. 2, no. 1, pp. 118–139, 2023.
M. V. Panichkina, I. A. Sinyavskaya, and E. V. Shestova, “Challenges of Professional Adaptation of University Graduates in Response to the Economics’ Digital Transformation,” in 2018 XVII Russian Scientific and Practical Conference on Planning and Teaching Engineering Staff for the Industrial and Economic Complex of the Region (PTES), IEEE, Nov. 2018, pp. 44–46. doi: 10.1109/PTES.2018.8604207.
S. Setiono, S. Windyariani, and A. Juhanda, “Implementasi Sistem Penilaian Berbasis Oucome Based Education di Perguruan Tinggi,” J. Pendidik., vol. 11, no. 1, pp. 1–9, 2023, doi: 10.36232/pendidikan.v11i1.2617.
A. H. A. Rasyid, B. Yunitasari, I. W. Susila, D. Dewanto, Y. Yunus, and D. I. Santoso, “Pengembangan Model Evaluasi Pembelajaran Berbasis Obe,” J. Pendidik. (Teori dan Prakt., vol. 7, no. 1, pp. 8–17, 2022, doi: 10.26740/jp.v7n1.p8-17.
U. M. Ishaq, M. F. Wicaksono, and S. Nurhayati, “Aplikasi Probe Untuk Penilaian Capaian Pembelajaran Mahasiswa Pada Kurikulum OBE (Outcame-Based Education),” Komputika J. Sist. Komput., vol. 12, no. 2, pp. 67–74, 2023, doi: 10.34010/komputika.v12i2.9763.
S. Masruroh and S. Shofia Hilabi, “Implementasi Pendidikan Agama Islam Berbasis Outcame Based Education (OBE) di Perguruan Tinggi Umum,” J. Dirosah Islam., vol. 4, no. 3, pp. 374–381, 2022, doi: 10.47467/jdi.v4i3.1263.
M. Dzulkifli, A. Samsudin, and A. S. Budi, “Analisis Tingkat Kompetensi Mahasiswa Terhadap Capaian Pembelajaran Mata Kuliah Pada Dacum Chart Program Studi Teknologi Industri Pangan,” J. Jendela Pendidik., vol. 2, no. 03, pp. 347–357, 2022, doi: 10.57008/jjp.v2i03.253.
A. L. Ramdani, D. H. Widyantoro, and R. Munir, “Optimalisasi Rekomendasi Rute Pada Perencanaan Perjalanan Wisata: Studi Pustaka,” MALCOM Indones. J. Mach. Learn. Comput. Sci., vol. 4, no. 2, pp. 515–525, 2024, doi: 10.57152/malcom.v4i2.1213.
M. F. Ramadhan, B. Siswoyo, and S. I. Cirebon, “Mengenal Model BERT dan Implementasinya untuk Analisis Sentimen Ulasan Game,” pp. 395–398.
D. L. Luhurkinanti, P. D. Purnamasari, T. Tsunakawa, and A. A. P. Ratna, “Japanese Short Answer Grading for Japanese Language Learners Using the Contextual Representation of BERT,” IEEE Access, vol. 13, no. January, pp. 17195–17207, 2025, doi: 10.1109/ACCESS.2025.3532659.
G. Wiratmoko, H. Thamrin, and W. E. Pamungkas, “Performance of Machine Learning Algorithms on Automatic Summarization of Indonesian Language Texts,” J. Online Inform., vol. 10, no. 1, pp. 196–204, 2025, doi: 10.15575/join.v10i1.1506.
H. Thamrin and A. Sabardilla, “Efektivitas Algoritma Semantik dengan Keterkaitan Kata dalam Mengukur Kemiripan Teks Bahasa Indonesia,” Khazanah Inform., vol. 1, no. 1, pp. 7–11, 2021.
H. Thamrin, D. Oktafiani, I. I. Rasyid, and I. M. Fauzi, “Classification of SWOT Statements Employing BERT Pre-Trained Model Embedding,” J. Sist. Inf. Bisnis, vol. 14, no. 2, pp. 143–152, 2024, doi: 10.21456/vol14iss2pp143-152.
M. Mokoatle, V. Marivate, D. Mapiye, R. Bornman, and V. M. Hayes, “A review and comparative study of cancer detection using machine learning: SBERT and SimCSE application,” BMC Bioinformatics, vol. 24, no. 1, pp. 1–25, 2023, doi: 10.1186/s12859-023-05235-x.
P. Lombaers, J. de Bruin, and R. van de Schoot, “Reproducibility and Data Storage for Active Learning-Aided Systematic Reviews,” Appl. Sci., vol. 14, no. 9, 2024, doi: 10.3390/app14093842.
J. J. Teijema et al., “Active learning-based systematic reviewing using switching classification models: the case of the onset, maintenance, and relapse of depressive disorders,” Front. Res. Metrics Anal., vol. 8, 2023, doi: 10.3389/frma.2023.1178181.
H. H. Mohammed, E. Dogdu, A. K. Gorur, and R. Choupani, “Multi-Label Classification of Text Documents Using Deep Learning,” in 2020 IEEE International Conference on Big Data (Big Data), 2020, pp. 4681–4689. doi: 10.1109/BigData50022.2020.9378266.
A. S. Harahap, M. Irfan, and K. Usman, “Outcome-Based Education pada Mata Kuliah Kewirausahaan Olahraga Program Studi Pendidikan Jasmani Kesehatan dan Rekreasi,” J. Phys. Heal. Recreat., vol. 4, no. 1, pp. 15–24, 2023, [Online]. Available: http://www.jurnal.stokbinaguna.ac.id/index.php/JPHR/article/view/1652
H. Halabieh et al., “The Future of Higher Education: Identifying Current Educational Problems and Proposed Solutions,” Educ. Sci., vol. 12, no. 12, 2022, doi: 10.3390/educsci12120888.
Ulfah and O. Arifudin, “Analisis Teori Taksonomi Bloom pada Pendidikan di Indonesia,” J. Al-Amar, vol. 4, no. 1, pp. 13–22, 2023.
N. Khalishah and N. Iklilah, “Taksonomi Bloom (Revisi): Tujuan Pendidikan dan Implementasinya dalam Pembelajaran Matematika,” Pros. Semin. Nas. Tadris Mat., pp. 248–266, 2021.
A. Sujatha Ravindran and J. Contreras-Vidal, “An empirical comparison of deep learning explainability approaches for EEG using simulated ground truth,” Sci. Rep., vol. 13, no. 1, pp. 1–20, 2023, doi: 10.1038/s41598-023-43871-8.
A. A. Salih and A. M. Abdulazeez, “Evaluation of Classification Algorithms for Intrusion Detection System: A Review,” J. Soft Comput. Data Min., vol. 2, no. 1, pp. 31–40, 2021, doi: 10.30880/jscdm.2021.02.01.004.
M. Owusu-Adjei, J. Ben Hayfron-Acquah, T. Frimpong, and G. Abdul-Salaam, “Imbalanced class distribution and performance evaluation metrics: A systematic review of prediction accuracy for determining model performance in healthcare systems,” PLOS Digit. Heal., vol. 2, no. 11 November, pp. 1–19, 2023, doi: 10.1371/journal.pdig.0000290.
