Swift - 为包含人脸的照片添加狂欢节面具

Question

我有一张有脸的照片。

我有狂欢节面具:



使用此功能，我可以检测人脸:

   let ciImage = CIImage(cgImage: photo)
   let options = [CIDetectorAccuracy: CIDetectorAccuracyHigh]
   let faceDetector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: options)!
   let faces = faceDetector.features(in: ciImage)
   if let face = faces.first as? CIFaceFeature {

   }

如何检测口罩上的孔？

检测到口罩的孔后如何将口罩贴在脸上？

Answer 1

我可能会尝试这种方法:

获取 leftEyePosition、rightEyePosition 和 faceAngle 值。 (CIFaceFeature 的所有部分)

计算左眼和右眼之间的距离。

这是有关如何计算距离的链接:https://www.hackingwithswift.com/example-code/core-graphics/how-to-calculate-the-distance-between-two-cgpoints

使用蒙版的原始尺寸以及到其中一只眼睛中心的 x 和 y 距离创建常量。

根据眼睛的距离，您可以按比例计算 mask 的新宽度。

这应该会给你一个尺寸合适的 mask 。还以相同的方式计算到面具的一只眼睛的中心的新 x 和 y 距离。

再次按比例调整所有值以适合屏幕上的最终预期尺寸。

使用眼睛的坐标将蒙版放在照片上，由蒙版眼睛到角落的距离抵消。

使用 faceAngle 值旋转蒙版。

在将蒙版导入项目之前，将其转换为具有透明背景的png，去除白色背景。您可以在代码中做到这一点，但这需要大量的工作，而且根据掩码源文件的不同，结果可能也不会如此。

更新，我已经尝试过我的解决方案。这是一个简单的 iOS 单屏应用程序，只需将代码复制到 ViewController.swift 文件中，将您的蒙版作为 png 和一张人脸照片作为 photo.jpg 添加到项目中，它应该可以工作。

如果您想尝试，这里有一个指向您的 png 照片的链接:

QPTF1.png

   import UIKit

class ViewController: UIViewController {

    override func viewDidLoad() {
        super.viewDidLoad()
        imageMethod()
    }

    func imageMethod() {

    let uiMaskImage = UIImage(named: "QPTF1.png") //converted to png with transperancy before adding to the project
    let maskOriginalWidth = CGFloat(exactly: 655.0)!
    let maskOriginalHeight = CGFloat(exactly: 364.0)!
    let maskOriginalEyeDistance = CGFloat(exactly: 230.0)! //increase or decrease value to change the final size of the mask
    let maskOriginalLeftEyePossitionX = CGFloat(exactly: 203.0)! //increase or decrease to fine tune mask possition on x axis
    let maskOriginalLeftEyePossitionY = CGFloat(exactly: 200.0)! //increase or decrease to fine tune mask possition on y axis


    //This code assumes the image AND face orientation is always matching the same orientation!
    //The code needs to be adjusted for other cases using UIImage.Orientation to get the orientation and adjusts the coordinates accordingly.
    //CIDetector might also not detect faces which don't have the same orientation as the photo. Try to use CIDetectorImageOrientation to look for other orientations of no face has been detected.
    //Also you might want to use other orientation points and scale values (right eye, nose etc.) in case the left eye, and left to right eye distance is not available.
    //Also this code is very wordy, pretty sure it can be cut down to half the size and made simpler on many places.

    let uiImageFace = UIImage(named: "photo.jpg")
    let ciImageFace = CIImage(image: uiImageFace!)
    let options = [CIDetectorAccuracy: CIDetectorAccuracyHigh]
    let faceDetector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: options)!
    let faces = faceDetector.features(in: ciImageFace!)
    if let face = faces.first as? CIFaceFeature {


        /*
        Getting the distances and angle based on the original photo
        */
        let faceAngle = face.faceAngle
        let rotationAngle = CGFloat(faceAngle * .pi / 180.0)

        //The distance in between the eyes of the original photo.
        let originalFaceEyeDistance = CGPointDistance(from: face.leftEyePosition, to: face.rightEyePosition)


        /*
        Adjusting the mask and its eye coordinates to fit the original photo.
        */

        //Setting the scale mask : original.
        let eyeDistanceScale = maskOriginalEyeDistance / originalFaceEyeDistance

        //The new dimensions of the mask.
        let newMaskWidth = maskOriginalWidth/eyeDistanceScale
        let newMaskHeight = maskOriginalHeight/eyeDistanceScale

        //The new mask coordinates of the left eye in relation to the original photo.
        let newMaskLeftEyePossitionX = maskOriginalLeftEyePossitionX / eyeDistanceScale
        let newMaskLeftEyePossitionY = maskOriginalLeftEyePossitionY / eyeDistanceScale

        /*
        Adjusting the size values to fit the desired final size on the screen.
        */

        //Using the width of the screen to calculate the new scale.
        let screenScale = uiImageFace!.size.width / view.frame.width

        //The new final dimensions of the mask
        let scaledToScreenMaskWidth = newMaskWidth / screenScale
        let scaledToScreenMaskHeight = newMaskHeight / screenScale

        //The new final dimensions of the photo.
        let scaledToScreenPhotoHeight = uiImageFace!.size.height / screenScale
        let scaledToScreenPhotoWidth = uiImageFace!.size.width / screenScale

        //The new eye coordinates of the photo.
        let scaledToScreenLeftEyeFacePositionX = face.leftEyePosition.x / screenScale
        let scaledToScreenLeftEyeFacePositionY = (uiImageFace!.size.height - face.leftEyePosition.y) / screenScale //reversing the y direction

        //The new eye to corner distance of the mask
        let scaledToScreenMaskLeftEyeX = newMaskLeftEyePossitionX / screenScale
        let scaledToScreenMaskLeftEyeY = newMaskLeftEyePossitionY / screenScale

        //The final coordinates for the mask
        let adjustedMaskLeftEyeX = scaledToScreenLeftEyeFacePositionX - scaledToScreenMaskLeftEyeX
        let adjustedMaskLeftEyeY = scaledToScreenLeftEyeFacePositionY - scaledToScreenMaskLeftEyeY

        /*
        Showing the image on the screen.
        */

        let baseImageView = UIImageView(image: uiImageFace!)
        //If x and y is not 0, the mask x and y need to be adjusted too.
        baseImageView.frame = CGRect(x: CGFloat(exactly: 0.0)!, y: CGFloat(exactly: 0.0)!, width: scaledToScreenPhotoWidth, height: scaledToScreenPhotoHeight)
        view.addSubview(baseImageView)

        let maskImageView = UIImageView(image: uiMaskImage!)
        maskImageView.frame = CGRect(x: adjustedMaskLeftEyeX, y: adjustedMaskLeftEyeY, width: scaledToScreenMaskWidth, height: scaledToScreenMaskHeight)
            maskImageView.transform = CGAffineTransform(rotationAngle: rotationAngle)
            view.addSubview(maskImageView)
        }

    }

    func CGPointDistanceSquared(from: CGPoint, to: CGPoint) -> CGFloat {
        return (from.x - to.x) * (from.x - to.x) + (from.y - to.y) * (from.y - to.y)
    }

    func CGPointDistance(from: CGPoint, to: CGPoint) -> CGFloat {
        return sqrt(CGPointDistanceSquared(from: from, to: to))
    }

}

结果:



这是我未注释的扫描眼睛的方法。它仍然有一些怪癖，但应该是一个起点。

 import UIKit

class ViewController: UIViewController {

    override func viewDidLoad() {
        super.viewDidLoad()
        imageMethod()
    }

    func imageMethod() {

        struct coords {
            let coord: (x: Int, y: Int)
            let size: Int
        }

    let uiMaskImage = UIImage(named: "QPTF1.png") //converted to png with transperancy before adding to the project

    let uiMaskImage2 = UIImage(named: "QPTF1.png")
    let ciMaskImage2 = CIImage(image: uiMaskImage2!)
    let context = CIContext(options: nil)
    let cgMaskImage = context.createCGImage(ciMaskImage2!, from: ciMaskImage2!.extent)

    let pixelData = cgMaskImage!.dataProvider!.data
    let data: UnsafePointer<UInt8> = CFDataGetBytePtr(pixelData)

    let alphaLevel: CGFloat = 0.0 //0.0 - 1.0 set higher to allow images with partially transparent eyes, like sunglasses.

    var possibleEyes: [coords] = []

    let frame = 10
    var detailLevel = 6

    let sizeX = Int((uiMaskImage?.size.width)!)
    let sizeY = Int((uiMaskImage?.size.height)!)

    var points: [(x: Int, y: Int)] = []

    var pointA_X = sizeX / 4
    var pointA_Y = sizeY / 4
    var pointB_X = sizeX / 4
    var pointB_Y = sizeY * 3 / 4
    var pointC_X = sizeX * 3 / 4
    var pointC_Y = sizeY / 4
    var pointD_X = sizeX * 3 / 4
    var pointD_Y = sizeY * 3 / 4

    var nextXsmaller = pointA_X / 2
    var nextYsmaller = pointA_Y / 2

    points.append((x: pointA_X, y: pointA_Y))
    points.append((x: pointB_X, y: pointB_Y))
    points.append((x: pointC_X, y: pointC_Y))
    points.append((x: pointD_X, y: pointD_Y))

    func transparentArea(_ x: Int, _ y: Int) -> Bool {
        let pos = CGPoint(x: x, y: y)
        let pixelInfo: Int = ((Int(uiMaskImage2!.size.width) * Int(pos.y)) + Int(pos.x)) * 4
        let a = CGFloat(data[pixelInfo+3]) / CGFloat(255.0)
        if a <= alphaLevel {
            return true
        } else {
            return false
        }
    }

    func createPoints(point: (x: Int, y: Int)) {

        pointA_X = point.x - nextXsmaller
        pointA_Y = point.y - nextYsmaller

        pointB_X = point.x - nextXsmaller
        pointB_Y = point.y + nextYsmaller

        pointC_X = point.x + nextXsmaller
        pointC_Y = point.y - nextYsmaller

        pointD_X = point.x + nextXsmaller
        pointD_Y = point.y + nextYsmaller

        points.append((x: pointA_X, y: pointA_Y))
        points.append((x: pointB_X, y: pointB_Y))
        points.append((x: pointC_X, y: pointC_Y))
        points.append((x: pointD_X, y: pointD_Y))

    }

    func checkSides(point: (x: Int, y: Int)) {

        var xNeg = (val: 0, end: false)
        var xPos = (val: 0, end: false)
        var yNeg = (val: 0, end: false)
        var yPos = (val: 0, end: false)

        if transparentArea(point.x, point.y) {

            xNeg.val = point.x
            xPos.val = point.x
            yNeg.val = point.y
            yPos.val = point.y

            while true {

                if transparentArea(xNeg.val, point.y) {
                    xNeg.val -= 1
                    if xNeg.val <= frame {
                        break
                    }
                } else {
                    xNeg.end = true
                }
                if transparentArea(xPos.val, point.y) {
                    xPos.val += 1
                    if xPos.val >= sizeX-frame {
                        break
                    }
                } else {
                    xPos.end = true
                }

                if transparentArea(point.x, yNeg.val) {
                    yNeg.val -= 1
                    if yNeg.val <= frame {
                        break
                    }
                } else {
                    yNeg.end = true
                }

                if transparentArea(point.x, yPos.val) {
                    yPos.val += 1
                    if yPos.val >= sizeY-frame {
                        break
                    }
                } else {
                    yPos.end = true
                }

                if xNeg.end && xPos.end && yNeg.end && yPos.end {

                    let newEyes = coords(coord: (point.x, point.y), size: (xPos.val - xNeg.val) * (yPos.val - yNeg.val) )

                    possibleEyes.append(newEyes)

                    break
                }
            }
        }
    }

    while detailLevel > 0 {

        print("Run: \(detailLevel)")


        for (index, point) in points.enumerated().reversed() {

            //checking if the point is inside of an transparent area
            checkSides(point: point)

            points.remove(at: index)

            if detailLevel > 1 {
                    createPoints(point: point)
            }
        }
        detailLevel -= 1
        nextXsmaller = nextXsmaller / 2
        nextYsmaller = nextYsmaller / 2

    }

    possibleEyes.sort { $0.coord.x > $1.coord.x }

    var rightEyes = possibleEyes[0...possibleEyes.count/2]
    var leftEyes = possibleEyes[possibleEyes.count/2..<possibleEyes.count]

    leftEyes.sort { $0.size > $1.size }
    rightEyes.sort { $0.size > $1.size }

    leftEyes = leftEyes.dropLast(Int(Double(leftEyes.count) * 0.01))
    rightEyes = rightEyes.dropLast(Int(Double(leftEyes.count) * 0.01))

    let sumXleft = ( leftEyes.reduce(0) { $0 + $1.coord.x} ) / leftEyes.count
    let sumYleft = ( leftEyes.reduce(0) { $0 + $1.coord.y} ) / leftEyes.count

    let sumXright = ( rightEyes.reduce(0) { $0 + $1.coord.x} ) / rightEyes.count
    let sumYright = ( rightEyes.reduce(0) { $0 + $1.coord.y} ) / rightEyes.count


    let maskOriginalWidth = CGFloat(exactly: sizeX)!
    let maskOriginalHeight = CGFloat(exactly: sizeY)!
    let maskOriginalLeftEyePossitionX = CGFloat(exactly: sumXleft)!
    let maskOriginalLeftEyePossitionY = CGFloat(exactly: sumYleft)!
    let maskOriginalEyeDistance = CGPointDistance(from: CGPoint(x: sumXright, y: sumYright), to: CGPoint(x: sumXleft, y: sumYleft))

    //This code assumes the image AND face orientation is always matching the same orientation!
    //The code needs to be adjusted for other cases using UIImage.Orientation to get the orientation and adjusts the coordinates accordingly.
    //CIDetector might also not detect faces which don't have the same orientation as the photo. Try to use CIDetectorImageOrientation to look for other orientations of no face has been detected.
    //Also you might want to use other orientation points and scale values (right eye, nose etc.) in case the left eye, and left to right eye distance is not available.
    //Also this code is very wordy, pretty sure it can be cut down to half the size and made simpler on many places.

    let uiImageFace = UIImage(named: "photo3.jpg")
    let ciImageFace = CIImage(image: uiImageFace!)
    let options = [CIDetectorAccuracy: CIDetectorAccuracyHigh]
    let faceDetector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: options)!
    let faces = faceDetector.features(in: ciImageFace!)
    if let face = faces.first as? CIFaceFeature {


        /*
        Getting the distances and angle based on the original photo
        */
        let faceAngle = face.faceAngle
        let rotationAngle = CGFloat(faceAngle * .pi / 180.0)

        //The distance in between the eyes of the original photo.
        let originalFaceEyeDistance = CGPointDistance(from: face.leftEyePosition, to: face.rightEyePosition)


        /*
        Adjusting the mask and its eye coordinates to fit the original photo.
        */

        //Setting the scale mask : original.
        let eyeDistanceScale = maskOriginalEyeDistance / originalFaceEyeDistance

        //The new dimensions of the mask.
        let newMaskWidth = maskOriginalWidth/eyeDistanceScale
        let newMaskHeight = maskOriginalHeight/eyeDistanceScale

        //The new mask coordinates of the left eye in relation to the original photo.
        let newMaskLeftEyePossitionX = maskOriginalLeftEyePossitionX / eyeDistanceScale
        let newMaskLeftEyePossitionY = maskOriginalLeftEyePossitionY / eyeDistanceScale

        /*
        Adjusting the size values to fit the desired final size on the screen.
        */

        //Using the width of the screen to calculate the new scale.
        let screenScale = uiImageFace!.size.width / view.frame.width

        //The new final dimensions of the mask
        let scaledToScreenMaskWidth = newMaskWidth / screenScale
        let scaledToScreenMaskHeight = newMaskHeight / screenScale

        //The new final dimensions of the photo.
        let scaledToScreenPhotoHeight = uiImageFace!.size.height / screenScale
        let scaledToScreenPhotoWidth = uiImageFace!.size.width / screenScale

        //The new eye coordinates of the photo.
        let scaledToScreenLeftEyeFacePositionX = face.leftEyePosition.x / screenScale
        let scaledToScreenLeftEyeFacePositionY = (uiImageFace!.size.height - face.leftEyePosition.y) / screenScale //reversing the y direction

        //The new eye to corner distance of the mask
        let scaledToScreenMaskLeftEyeX = newMaskLeftEyePossitionX / screenScale
        let scaledToScreenMaskLeftEyeY = newMaskLeftEyePossitionY / screenScale

        //The final coordinates for the mask
        let adjustedMaskLeftEyeX = scaledToScreenLeftEyeFacePositionX - scaledToScreenMaskLeftEyeX
        let adjustedMaskLeftEyeY = scaledToScreenLeftEyeFacePositionY - scaledToScreenMaskLeftEyeY

        /*
        Showing the image on the screen.
        */

        let baseImageView = UIImageView(image: uiImageFace!)
        //If x and y is not 0, the mask x and y need to be adjusted too.
        baseImageView.frame = CGRect(x: CGFloat(exactly: 0.0)!, y: CGFloat(exactly: 0.0)!, width: scaledToScreenPhotoWidth, height: scaledToScreenPhotoHeight)
        view.addSubview(baseImageView)

        let maskImageView = UIImageView(image: uiMaskImage!)
        maskImageView.frame = CGRect(x: adjustedMaskLeftEyeX, y: adjustedMaskLeftEyeY, width: scaledToScreenMaskWidth, height: scaledToScreenMaskHeight)
        maskImageView.transform = CGAffineTransform(rotationAngle: rotationAngle)
        view.addSubview(maskImageView)
        }

    }

    func CGPointDistanceSquared(from: CGPoint, to: CGPoint) -> CGFloat {
        return (from.x - to.x) * (from.x - to.x) + (from.y - to.y) * (from.y - to.y)
    }

    func CGPointDistance(from: CGPoint, to: CGPoint) -> CGFloat {
        return sqrt(CGPointDistanceSquared(from: from, to: to))
    }

}