opencv - 如何对齐人脸进行预处理？

Question

我试图编写一个面部对准器作为预处理步骤，然后再将面部输入神经网络。我已经通过dlib的Vahid Kazemi and Josephine Sullivan's回归树整体实现实现(使用Python)找到了人脸的地标，以预测人脸的地标估计。我尝试了许多不同类型的转换都没有用。这是我的基本管道:

在图像和裁剪图像中查找人脸，使其仅具有人脸:

image = <some_image_with_one_face>
face_detector = dlib.get_frontal_face_detector()
detected_face = face_detector(image)
w,h = 160 # so that facenet can use it 
for face_rect in detected_face:

    # First crop the face
    left = face_rect.left()
    top = face_rect.top()
    right = face_rect.right()
    bottom = face_rect.bottom()

    new_face_rect = dlib.rectangle(0, 0, right-left, bottom-top)
    cropped_image = image[top:bottom, left:right, :].copy()

    # Get the the face's pose
    pose_landmarks = face_pose_predictor(cropped_image, new_face_rect)`

查找地标(get_landmark_points返回68x2数组

landmarks = get_landmark_points(pose_landmarks, N_LANDMARKS, dlib_point=False)

定义一些src和dst点:

top_of_nose = landmarks[27]
left_eye = landmarks[36]
right_eye = landmarks[45]
bottom_lip = landmarks[57]

src = [top_of_nose, left_eye, right_eye, bottom_lip]
dst = [[np.int(0.5 * w), np.int(h/3)],\
       [np.int(0.3 * w), np.int(h / 3)],\
       [np.int(0.7 * w), np.int(h / 3)],\
       [np.int(0.5 * w), np.int(h * (2.0/3))]]

估计转换并转换其余地标

transformed_crop = cv2.warpAffine(cropped_image, transform, (w, h))

# Get the transformed landmarks
transformed_landmarks = np.reshape(cv2.transform(np.expand_dims(landmarks, 1),\
                        transform), (N_LANDMARKS, 2)).astype(np.float32)

# Add the boundary points
transformed_landmarks = np.append(transformed_landmarks, boundary_points, axis = 0)

这是2张图像的输入和输出:


尽管这是可行的，但这并不是完美的，因为两个图像中的眼睛和嘴唇的高度相同，但是 x的位置不同。我想知道是否有某种方法可以改善它，还是我使用了错误的技术？如果有人指出我正确的方向，这将很有帮助，谢谢!

Answer 1

deepface封装了dlib和许多其他人脸检测器，并提供了即用型功能。只需调用检测人脸功能即可。

#!pip install deepface
from deepface import DeepFace
import matplotlib.pyplot as plt
backend = 'dlib' opencv, ssd, dlib or mtcnn
img = DeepFace.detectFace("img.jpg", detector_backend = backend)
plt.imshow(img); plt.show()