PIGan: Pose Invariant face reenactment for game character using a GAN-based face rotation technique
game character face, pose invariant, face reenactment, GAN, attention map, rotate module
Junyoung OH, Kyungha MIN, Heekyung YANG
The recent emergence of generative adversarial networks (GANs) accelerate the research on face reenactment. Face reenactment is a technique for synthesizing a face image of desired expressions from a source face using a target image of desired expressions or a vector with facial expression information. In this paper, we propose a GAN architecture-based method for reenacting the expressions of game characters with various angles through a rotation module and action unit (AU) vectors. In the first step, we devise a rotate module that synthesizes the frontalized face images from the face of a game character with arbitrary poses. This prevents the side effects of the face poses in reenacting the facial expressions. We feed the frontalized image as well as a vector with facial expression information to the generator in order to synthesize a face with the desired expression. The generator generates an attention mask indicating a region of interest of facial expression and a color mask indicating color information for the expression. This allows us to synthesize a reenacted image that generates facial expressions while preserving the identity and other attachments of a game character. The attention mask and color mask incorporate to synthesize a reenacted image with target expression. Finally, we recover the original pose of a character from the frontalized reenacted image.
KCI Link : https://www.kci.go.kr/kciportal/ci/sereArticleSearch/ciSereArtiView.kci?sereArticleSearchBean.artiId=ART002697956