AT-CNN-LSTM Facial Micro-expression Recognition and its Application in Educational Scene
Zhou, Chi (2020-01-07)
AT-CNN-LSTM Facial Micro-expression Recognition and its Application in Educational Scene
(07.01.2020)
Julkaisu on tekijänoikeussäännösten alainen. Teosta voi lukea ja tulostaa henkilökohtaista käyttöä varten. Käyttö kaupallisiin tarkoituksiin on kielletty.
suljettu
Julkaisun pysyvä osoite on:
https://urn.fi/URN:NBN:fi-fe202001314157
https://urn.fi/URN:NBN:fi-fe202001314157
Tiivistelmä
Humans can convey a lot of important information through facial expressions, and the differentiation of facial expressions can promote the understanding of people’s psychological state. Compared with ordinary facial expressions, facial micro-expressions, as a spontaneous expression, have the characteristics of being unable to suppress or hide, so they are very suitable for judging people’s true emotions. This thesis innovatively applies the proposed AT-CNN-LSTM micro-expression recognition system to the field of education and learning scene. It can measure the learners’ emotional changes in the learning process.
In the field of computer vision, the deep learning method has already demonstrated its powerful feature extraction and classification ability, and it has excellent performance on face recognition, object detection and other tasks. Therefore, some scholars have begun to apply the deep learning method to the field of micro-expression recognition. However, the recognition performance still has great potential to improve. The main work of this thesis is as follows:
(1) Compare and select suitable micro-expression data sets and perform data preprocessing to achieve the needs of deep neural network design. In addition, we do the augmentation for the processed data reasonably to prevent the occurrence of
over-fitting.
(2) A deep neural network structure based on CNN and bidirectional LSTM is proposed, which integrates feature extraction and classification process to realize end-to-end network training mode. In addition, in view of the characteristics of microexpressions which mainly appear in some local areas of face, attention mechanism is added at the back of the network to highlight the impact of local features on the classification performance, and to modify the network structure and control network parameters to prevent the occurrence of over-fitting phenomenon.
(3) Design experiments and collect the sample data of the subjects, fine-tune the model with the model parameters obtained above, reflecting the feedback of microexpression recognition to students’ learning status in educational scenes.
In the field of computer vision, the deep learning method has already demonstrated its powerful feature extraction and classification ability, and it has excellent performance on face recognition, object detection and other tasks. Therefore, some scholars have begun to apply the deep learning method to the field of micro-expression recognition. However, the recognition performance still has great potential to improve. The main work of this thesis is as follows:
(1) Compare and select suitable micro-expression data sets and perform data preprocessing to achieve the needs of deep neural network design. In addition, we do the augmentation for the processed data reasonably to prevent the occurrence of
over-fitting.
(2) A deep neural network structure based on CNN and bidirectional LSTM is proposed, which integrates feature extraction and classification process to realize end-to-end network training mode. In addition, in view of the characteristics of microexpressions which mainly appear in some local areas of face, attention mechanism is added at the back of the network to highlight the impact of local features on the classification performance, and to modify the network structure and control network parameters to prevent the occurrence of over-fitting phenomenon.
(3) Design experiments and collect the sample data of the subjects, fine-tune the model with the model parameters obtained above, reflecting the feedback of microexpression recognition to students’ learning status in educational scenes.