ios11 - “项目”ARFaceAnchor 指向屏幕

Question

我正在尝试在屏幕上叠加 ARFaceAnchor 顶点来完成两个场景1)让虚拟面保持中心(屏幕上)位置，但反射(reflect)几何形状的变化。顶点2) 让虚拟脸部与实际脸部重叠(来自预览层)。

我关注了Rickster's advice在这里，但仅成功地将面部从某些角度投影到屏幕上(仅出现在左下角并旋转)。我不太熟悉每个矩阵的不同用途，但这就是我到目前为止所了解的。有什么建议吗？

let modelMatrix = faceAnchor.transform
var points: [CGPoint] = []

faceAnchor.geometry.vertices.forEach {
    // Convert the vertex position from model space to camera space (use the anchor’s transform)
    let vertex4 = vector_float4($0.x, $0.y, $0.z, 1)
    let vertexCamera = simd_mul(modelMatrix, vertex4)

    // Multiply with the camera projection with that vector to get to normalized image coordinates 
    let normalizedImageCoordinates = simd_mul(projectionMatrix, vertexCamera)

    let point = CGPoint(x: CGFloat(normalizedImageCoordinates.x), y: CGFloat(normalizedImageCoordinates.y))
    points.append(point)
} 

Answer 1

对于那些感兴趣的人，这里是(2)的解决方案 - 您可以标准化这些点以保持面部居中(1)

let faceAnchors = anchors.flatMap { $0 as? ARFaceAnchor }

guard !faceAnchors.isEmpty,
    let camera = session.currentFrame?.camera,
    let targetView = SomeUIView() else { return }

// Calculate face points to project to screen

let projectionMatrix = camera.projectionMatrix(for: .portrait, viewportSize: targetView.bounds.size, zNear: 0.001, zFar: 1000)  // A transform matrix appropriate for rendering 3D content to match the image captured by the camera
let viewMatrix = camera.viewMatrix(for: .portrait)        // Returns a transform matrix for converting from world space to camera space.

let projectionViewMatrix = simd_mul(projectionMatrix, viewMatrix)

for faceAnchor in faceAnchors  {

    let modelMatrix = faceAnchor.transform                  //  Describes the face’s current position and orientation in world coordinates; that is, in a coordinate space relative to that specified by the worldAlignment property of the session configuration. Use this transform matrix to position virtual content you want to “attach” to the face in your AR scene.
    let mvpMatrix = simd_mul(projectionViewMatrix, modelMatrix)

    // Calculate points

    let points: [CGPoint] = faceAnchor.geometry.vertices.flatMap({ (vertex) -> CGPoint? in

        let vertex4 = vector_float4(vertex.x, vertex.y, vertex.z, 1)

        let normalizedImageCoordinates = simd_mul(mvpMatrix, vertex4)

        return CGPoint(x: CGFloat(normalizedImageCoordinates.x ),
                       y: CGFloat(normalizedImageCoordinates.y ))
    })

}