Real-Time Classroom Behavior Detection and Visualization System Based on an Improved YOLOv11

Jiajun Li; Nannan Wang; Junhao Zhang; Xiaozhou Yao; Wei Wei

doi:10.61414/mwad4j38

Abstract

Automatic analysis of student behavior in classrooms has gained importance with the rise of smart education and vision technologies. However, the limited real-time accuracy of existing methods severely constrains their practical classroom deployment. To address this issue of low accuracy, we propose an improved YOLOv11-based detector that integrates CARAFE upsampling, DySnakeConv, DyHead, and SMFA fusion modules. This new model for real-time classroom behavior detection captures fine-grained student behaviors with low latency. Additionally, we have developed a visualization system that presents data through intuitive dashboards. This system enables teachers to dynamically grasp classroom engagement by tracking student participation and involvement. The enhanced YOLOv11 model achieves an mAP@0.5 of 87.2% on the evaluated datasets, surpassing baseline models. This significance lies in two aspects. First, it provides a practical technical route for deployable live classroom behavior monitoring and engagement feedback systems. Second, by integrating this proposed system, educators could make data-informed and fine-grained teaching decisions, ultimately improving instructional quality and learning outcomes.

References

1.
Bustos-López, M., Cruz-Ramírez, N., Guerra-Hernández, A., Sánchez-Morales, L. N., Cruz-Ramos, N. A., & Alor-Hernández, G. (2022). Wearables for engagement detection in learning environments: A review. Biosensors, 12(7), 509. https://doi.org/10.3390/bios12070509.
2.
Chen, G., Ji, J., & Huang, C. (2022, April 15–17). Student classroom behavior recognition based on OpenPose and deep learning [Conference session]. 7th International Conference on Intelligent Computing and Signal Processing (ICSP) (pp. 576–579), Xi’an, China. https://doi.org/10.1109/ICSP54964.2022.9778501.
3.
Falcon, S., Alonso, J. B., & Leon, J. (2023). Teachers’ engaging messages, students’ motivation to learn and academic performance: The moderating role of emotional intensity in speech. Teaching and Teacher Education, 136, 104375.
4.
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109. https://doi.org/10.3102/00346543074001059.
5.
Hou, P., & Huang, S. (2025). BCSM-YOLO: An improved product package recognition algorithm for automated retail stores based on YOLOv11. IEEE Access, 13, 139665–139679.
6.
Jia, L., Sun, H., Jiang, J., & Yang, X. (2025). High-quality classroom dialogue automatic analysis system. Applied Sciences, 15(3), 1613. https://doi.org/10.3390/app15031613.
7.
Khanam, R., & Hussain, M. (2024). Yolov11: An overview of the key architectural enhancements. arXiv. https://arxiv.org/abs/2410.17725.
8.
Li, C., Bao, W., Chen, X., Jing, Y., & Qu, X. (2022, May 20–23). SlowFast with DropBlock and smooth samples loss for student action recognition [Conference session]. 14th International Conference on Digital Image Processing (ICDIP) (Vol. 12342, pp. 205–212), Wuhan, China.
9.
Li, C., & Peng, C. (2025, January 17–19). Student classroom behavior recognition based on improved YOLOv7 [Conference session]. 2nd International Conference on Informatics Education and Computer Technology (pp. 64–69), Kuala Lumpur, Malaysia.
10.
Li, J., Wang, N., & Wei, W. (2024). POCO: Pedagogical objects in college classroom dataset [Data set]. GitHub. https://github.com/jwmianzu/POCO-Dataset.
11.
Lin, H., Wei, W., & Lu, H. (2025). Facilitator or barrier? A systematic review on the relationship between artificial intelligence technologies and the development of critical thinking skills. Journal of Educational Technology and Innovation, 7(2), 11–24.
12.
Miao, F., Holmes, W., Huang, R., & Zhang, H. (2021). AI and education: Guidance for policy-makers. UNESCO Publishing.
13.
Qi, Y., He, Y., Qi, X., Zhang, Y., & Yang, G. (2023, October 1–6). Dynamic snake convolution based on topological geometric constraints for tubular structure segmentation [Conference session]. IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 6070–6079), Paris, France.
14.
Renawi, A., Alnajjar, F., Parambil, M., Trabelsi, Z., Gochoo, M., Khalid, S., & Mubin, O. (2022). A simplified real-time camera-based attention assessment system for classrooms: Pilot study. Education and Information Technologies, 27(4), 4753–4770.
15.
Song, F., & Gong, X. (2024). Substitution or empowerment: The impact and response of artificial intelligence teaching on teachers’ teaching rights. China Distance Education, 44(4), 15–27. [In Chinese]
16.
Smith, M. K., Jones, F. H., Gilbert, S. L., & Wieman, C. E. (2013). The classroom observation protocol for undergraduate STEM (COPUS): A new instrument to characterize university STEM classroom practices. CBE—Life Sciences Education, 12(4), 618–627. https://doi.org/10.1187/cbe.13-08-0154.
17.
Sümer, Ö., Goldberg, P., D’Mello, S., Gerjets, P., Trautwein, U., & Kasneci, E. (2021). Multimodal engagement analysis from facial videos in the classroom. IEEE Transactions on Affective Computing, 14(2), 1012–1027.
18.
Wang, H., Gao, C., Fu, H., Ma, C. Z.-H., Wang, Q., He, Z., & Li, M. (2023). Automated student classroom behaviors’ perception and identification using motion sensors. Bioengineering, 10(2), 127. https://doi.org/10.3390/bioengineering10020127.
19.
Wang, J., Chen, K., Xu, R., Liu, Z., Loy, C. C., & Lin, D. (2019, October 27–28). CARAFE: Content-aware reassembly of features [Conference session]. IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 3007–3016), Seoul, Republic of Korea.
20.
Wu, Z. Y., Tang, Y., & Liu, H. (2022). Survey of personalized learning recommendation. Journal of Frontiers of Computer Science and Technology, 16(1), 21–40. [In Chinese]
21.
Wu, Z., Zhen, H., Zhang, X., Bai, X., & Li, X. (2025). SEMA-YOLO: Lightweight small object detection in remote sensing image via shallow-layer enhancement and multi-scale adaptation. Remote Sensing, 17(11), 1917. https://doi.org/10.3390/rs17111917.
22.
Xu, X., Dugdale, D. M., Wei, X., & Mi, W. (2023). Leveraging artificial intelligence to predict young learner online learning engagement. American Journal of Distance Education, 37(3), 185–198.
23.
Yang, F. (2023). SCB-dataset: A dataset for detecting student classroom behavior. arXiv. https://arxiv.org/abs/2304.02488.
24.
Yang, F., & Wang, T. (2023). SCB-dataset3: A benchmark for detecting student classroom behavior. arXiv. https://arxiv.org/abs/2310.02522.
25.
Yang, K. B., Borchers, C., Falhs, A. C., Echeverria, V., Karumbaiah, S., Rummel, N., & Aleven, V. (2024, September 16–20). Leveraging multimodal classroom data for teacher reflection: Teachers’ preferences, practices, and privacy considerations [Conference session]. 19th European Conference on Technology Enhanced Learning (pp. 498–511), Krems, Austria.
26.
Yin, C. C., Sun, Y., Li, G., Peng, J., Ran, F., Wang, Z., & Zhou, J. (2022). Identifying and monitoring students’ classroom learning behavior based on multisource information. Mobile Information Systems, 2022, 9903342.
27.
Zaletelj, J., & Košir, A. (2017). Predicting students’ attention in the classroom from Kinect facial and body features. EURASIP Journal on Image and Video Processing, 2017, 80.
28.
Zhao, C. (2024). AI-assisted assessment in higher education: A systematic review. Journal of Educational Technology and Innovation, 6(4), 39–58.
29.
Zheng, M., Sun, L., Dong, J., & Pan, J. (2024, September 29–October 4). SMFANet: A lightweight self-modulation feature aggregation network for efficient image super-resolution [Conference session]. 18th European Conference on Computer Vision (ECCV) (pp. 359–375), Milan, Italy.
30.
Zheng, Y. F., Zhao, Y. N., Bai, X., & Fu, Q. (2021). Survey of big data visualization in education. Journal of Frontiers of Computer Science and Technology, 15(3), 403–422. [In Chinese]
31.
Zhong, H., Zhang, Y., Shi, Z., Zhang, Y., & Zhao, L. (2025). PS-YOLO: A lighter and faster network for UAV object detection. Remote Sensing, 17(9), 1641. https://doi.org/10.3390/rs17091641.
32.
Zhu, H., Sun, Y., & Yang, J. (2025). Towards responsible artificial intelligence in education: A systematic review on identifying and mitigating ethical risks. Humanities and Social Sciences Communications, 12(1), 1–14.

Scilight Press

Author Information

Abstract

Keywords

References

About Scilight

Journals

Publishing Policies

Contact Us