AugTheFace

AugTheFace?

AugTheFace is computer vision-based script for image augmentation from images. It uses the true bbox to crop the face images from images. Then, dlib based detector is used for identifying the face features. Those features are applied for preventing inappropriate face images. Using an image reenactment model created from "Latent Image Animator: Learning to Animate Images via Latent Space Navigation", the available face images can imitate the driving images. Finally, AugTheFace uses MTCNN to define the new bounding boxes of face images. The augmented face images are put back to original images following coordinate of new boxes. AugTheFace identifies all face images which are available, however, it saves image with single augmented face.

How to use?

• Clone this repo

  git clone https://github.com/Rayhchs/AugTheFace.git

• Download pre-trained weights

  Pre-trained checkpoints can be found in LIA. AugTheFace only uses vox.pt so put the model in ./checkpoints. AugTheFace also uses dlib detector which can be downloaded here.

• Setup Environment

  • python3.8.10

  pip install -r requirements.txt

• Datasets

  AugTheFace uses the ground truth bbox followed by format of widerface for cropping the face images. Input image and driving image can be any types of image, however, face in driving image should occupy at least half of image.

• Results

  Augmented images would be saved in ./res defaultly. Please notice that not all of images can be used for augmentation.

Path Arguments

Argument	Explanation
source_path	Path of image where you want to augment
driving_path	Path of image where you want your source image to imitate
save_folder	Where to save the augmented results
bbox_dir	Truth data of bounding box of source images

Demo

This repository uses widerface images for demo.

Acknowledgement

Code and pretrain weights heavily borrows from LIA and MTCNN. Thanks for the excellent work!

Latent Image Animator: Learning to Animate Images via Latent Space Navigation:

@inproceedings{
wang2022latent,
title={Latent Image Animator: Learning to Animate Images via Latent Space Navigation},
author={Yaohui Wang and Di Yang and Francois Bremond and Antitza Dantcheva},
booktitle={International Conference on Learning Representations},
year={2022}
}

MTCNN:

@article{7553523,
    author={K. Zhang and Z. Zhang and Z. Li and Y. Qiao}, 
    journal={IEEE Signal Processing Letters}, 
    title={Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks}, 
    year={2016}, 
    volume={23}, 
    number={10}, 
    pages={1499-1503}, 
    keywords={Benchmark testing;Computer architecture;Convolution;Detectors;Face;Face detection;Training;Cascaded convolutional neural network (CNN);face alignment;face detection}, 
    doi={10.1109/LSP.2016.2603342}, 
    ISSN={1070-9908}, 
    month={Oct}
}

Widerface:

@inproceedings{yang2016wider,
	Author = {Yang, Shuo and Luo, Ping and Loy, Chen Change and Tang, Xiaoou},
	Booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
	Title = {WIDER FACE: A Face Detection Benchmark},
	Year = {2016}}