Study on Facial Composite Feature Analysis for Determining Subject Anxiety Levels on Low-Power Computing Modules
저연산 컴퓨팅 모듈에서 구동 가능한 안면 복합 특성 분석 기반 대상자 긴장 상태 판단에 관한 연구
A contact-free system that reads eye blinks, pupil movement and facial expressions from an ordinary RGB camera and combines them into a single anxiety score, light enough to run on low-end hardware.
Reading anxiety from the face, without contact
Recent studies have utilized external biosignals, particularly those observable on the human face such as expressions and eye blinks, for the analysis of a subject’s level of anxiety. This paper presents the first comprehensive system for contact-free prediction of various facial features from RGB images, surpassing traditional methods reliant on prior knowledge or significant computing power. Our approach introduces modules for analyzing variations in facial features, such as eye blinks, pupil movement, and expressions, developed to operate effectively in low-end hardware environments with limited computing power. Each module has been validated on its respective benchmark dataset, demonstrating comparability with or even exceeding the State-Of-The-Art. A method for deriving a composite anxiety score based on the combination of biological signal changes is further proposed, with additional validation on a privately-conducted user study.
Three facial signals, one score
Each module runs on its own lightweight model so the whole pipeline fits on a low-power computing module next to the camera.
Blink rate from landmarks
Facial landmarks from a lightweight Pixel-in-Pixel network give six points per eye; the eye aspect ratio turns them into a per-frame openness signal and a blink count.
Gaze and pupil changes
Movement of the pupils between frames is analysed as a second facial signal.
Facial expression recognition
A compact expression classifier reads the facial expression in every frame.
Changes in the three signals are combined into a composite anxiety score. Each module was validated on its own benchmark, and the composite score was further validated in a private user study.
Real-time blink detection on a webcam
The released eye-blink module running live: face detection, landmark localisation and a running blink count, all in real time.
The module keeps per-subject statistics of the eye aspect ratio, so the blink threshold adapts to each person instead of using one fixed value.
Because it needs only landmarks from a single RGB frame, it runs on embedded boards and plugs directly into the anxiety-estimation pipeline above.
Featured in national media
The research was covered by a national newspaper and two national broadcasters, including reports on detection technology at customs.
BibTeX
@inproceedings{lee2024facial,
title = {Study on Facial Composite Feature Analysis for Determining Subject Anxiety Levels on Low-Power Computing Modules},
author = {Lee, JunGyu and Lee, Sanghyun Alex and Nam, Gi Pyo},
booktitle = {Proceedings of the IEIE Summer Annual Conference},
pages = {1816--1820},
year = {2024}
}