python - OpenCV中Lucas-Kanade跟踪器中的错误和状态输出

Question

阅读了OpenCV文档之后，我仍然无法确切知道st, err中的p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)以及何时需要使用它们。

Answer 1

如果您对自己不了解的内容更加具体，那会更好。 documentation for calcOpticalFlowPyrLK() 说:

status – output status vector (of unsigned chars); each element of the vector is set to 1 if the flow for the corresponding features has been found, otherwise, it is set to 0.
err – output vector of errors; each element of the vector is set to an error for the corresponding feature, type of the error measure can be set in flags parameter; if the flow wasn’t found then the error is not defined (use the status parameter to find such cases).

因此，假设您在图像或框架的边框上有一个功能，而在下一个图像或框架中，该功能将消失。在下一帧中找不到该功能，因此 status中该功能的相应索引将为0。否则，该索引处的 status将为1。
err的值为0时，未定义 status值。否则，错误基本上是“特征周围的相邻像素在两个图像/帧之间的对应程度如何”。在 flags参数下，这更加清楚了:

OPTFLOW_LK_GET_MIN_EIGENVALS use minimum eigen values as an error measure (see minEigThreshold description); if the flag is not set, then L1 distance between patches around the original and a moved point, divided by number of pixels in a window, is used as a error measure.

L1距离只是窗口内的绝对差之和。因此，只需在两个图像/帧中的特征周围的窗口中查看，并计算每个窗口中相应像素之间的绝对差，然后对其求和。因此，当未设置标志时会发生这种情况。如果是，则文档指示我们查看 minEigThreshold参数的描述:

minEigThreshold – the algorithm calculates the minimum eigen value of a 2x2 normal matrix of optical flow equations (this matrix is called a spatial gradient matrix in [Bouguet00]), divided by number of pixels in a window; if this value is less than minEigThreshold, then a corresponding feature is filtered out and its flow is not processed, so it allows to remove bad points and get a performance boost.

因此，使用该标志，您可以看到每个特征的最小特征值-如果您想确定要使用的阈值，则很有用。在OpenCV文档(Bouguet00)中引用的用于LK功能跟踪器金字塔式实现的 original paper中，它详细说明了第3节(功能选择)中的含义:

So far, we have described the tracking procedure that takes care of following a point u on an image I to another location v on another image J. However, we have not described means to select the point u on I in the first place. This step is called feature selection. It is very intuitive to approach the problem of feature selection once the mathematical ground for tracking is led out. Indeed, the central step of tracking is the computation of the optical flow vector η^k (see pseudo-code of algorithm in section 2.4). At that step, the G matrix is required to be invertible, or in other words, the minimum eigenvalue of G must be large enough (larger than a threshold). This characterizes pixels that are "easy to track".