image: The 18 facial actions are illustrated (reported by row, from left-to-right according to the numbering used in the released dataset: (2) raise eyebrows as if by surprise; (3) bring the eyebrows together as if in trouble; (4)–(6) close right / left eye forcefully, forming wrinkles; (5)–(7) close right / left eye smoothly, without wrinkles; (8) extrude lips to show teeth and gums; (9) lift the lateral portion of the nostrils by squeezing the eyebrows at the base of the nose; (10) lift the chin bringing out the lower lip; (11)–(12) move the right / left corner of the mouth to the right / left, respectively; (13) bring the lips forward as if to give a kiss; (14) bring the ends of the lips back as if for smiling; (15)–(16) smile from the right / left side; (17) move the corners of the mouth down; (18) compress cheeks and lips on the teeth; (19) smile without showing your teeth. The neutral scan [acquisition (1) is not reported]. view more
Credit: Beijing Zhongke Journal Publising Co. Ltd.
This study is led by Dr. Claudio FERRARI (Department of Engineering and Architecture, University of Parma, Italy), and Prof. Stefano BERRETTI, Prof. Pietro PALA, and Prof. Alberto Del BIMBO (Department of Information Engineering, University of Florence, Italy). This paves the way for the use of the proposed tool in remote medical rehabilitation monitoring.
The work described in the manuscript introduces the following innovations:
• An automatic method to estimate local and asymmetric face deformations from a set of 2D landmarks extracted from a video sequence.
• A two-step fitting process based on a sparse 3D Morphable Face Model (3DMM) is proposed to disentangle muscular movements from shape changes due to different identities.
• The 3DMM employed allows to accurately identify and measure asymmetries when performing facial actions.
• Application to face rehabilitation: patients recovering from facial paralysis can monitor the progress of their rehabilitation without the need of specialized equipment. The user can measure how accurately he/she can reproduce some facial actions, evaluating possible intensity deficiency.
• Immediate feedback to the subject.
Reference:
Claudio FERRARI, Stefano BERRETTI, Pietro PALA, Alberto Del BIMBO. Measuring 3D face deformations from RGB images of expression rehabilitation exercises. Virtual Reality & Intelligent Hardware, 2022, 4(4): 306–323 doi: 10.1016/j.vrih.2022.05.004
Journal
Virtual Reality
Article Title
Measuring 3D face deformations from RGB images of expression rehabilitation exercises
Article Publication Date
9-Sep-2022