One of our papers has been accepted for publication in Pattern Recognition Letters.
Bibliographic Information
Bo Yang, Jianming Wu, Kazushi Ikeda, Gen Hattori, Masaru Sugano, Yusuke Iwasawa, Yutaka Matsuo. “Face-mask-aware Facial Expression Recognition based on Face Parsing and Vision Transformer”, Pattern Recognition Letters, In Press, Pre-proof (2022).
https://doi.org/10.1016/j.patrec.2022.11.004
Overview
As wearing face masks is becoming an embedded practice due to the COVID-19 pandemic, facial expression recognition (FER) that takes face masks into In this paper, we propose a face parsing and vision transformer-based method to improve the accuracy of First, in order to improve the precision of distinguishing the unobstructed facial region as well as those parts of the face covered by a mask, we re-train First, in order to improve the precision of distinguishing the unobstructed facial region as well as those parts of the face covered by a mask, we re-train a face-mask-aware face parsing model, based on the existing face parsing dataset automatically relabeled with a face mask and pixel Second, we propose a vision transformer with a cross attention mechanism-based FER classifier, capable of taking both occluded and non-occluded Second, we propose a vision Transformer with a cross attention mechanism-based FER classifier, capable of taking both occluded and non-occluded facial regions into account and reweigh these two parts automatically to get the best facial expression recognition performance. The proposed method outperforms existing state-of-the-art face-mask-aware FER methods, as well as other occlusion-aware FER methods, on two datasets that contain three The proposed method outperforms existing state-of-the-art face-mask-aware FER methods, as well as other occlusion-aware FER methods, on two datasets that contain three kinds of emotions (M-LFW-FER and M-KDDI-FER datasets) and two datasets that contain seven kinds of emotions (M-FER-2013 and M-CK+ datasets).
