java - 在 Android 和 OpenCV 中使用 BlockingQueue 进行多处理

Question

我正在尝试使用 Opencv for Android 提高我的帧速率。当我让进程运行时，我没有注意到我的帧速率有任何变化。我想知道我的逻辑是否有问题。同样在运行大约 30 秒后，它因缓冲区错误而崩溃。

我试图做的是将主线程用于视频的 I/O 显示，将第二个线程用于对象检测。

我的主要 Activity 是使用“实现 CvCameraViewListener2”的制作人，这会将每个帧放入阻塞队列 IN 并将它们从阻塞队列 OUT 中取出。

然后我有一个带有处理逻辑的 Consumer runnable。这使它们脱离 IN 阻塞队列处理它们，然后将它们放在 OUT 队列中。

public final class CameraActivity extends FragmentActivity
    implements CvCameraViewListener2 {
  Consumer consumer1 ;
    private BlockingQueue<Mat> inFrames = new LinkedBlockingQueue<Mat>(11);
private BlockingQueue<Mat> outFrames = new LinkedBlockingQueue<Mat>(11);

  @Override
  public void onCameraViewStarted(final int width,
        final int height) {
    consumer1 = new Consumer(inFrames,outFrames);
    Thread thread1 = new Thread(consumer1);
    thread1.start();

  }

  @Override
  public void onCameraViewStopped() {
    consumer1.stopRunning();
    consumer2.stopRunning();
  }

  @Override
  public Mat onCameraFrame(final CvCameraViewFrame inputFrame) {
    final Mat rgba = inputFrame.rgba();
    // This is the producer of the blocking queue
    try {
        inFrames.put(inputFrame.rgba());
    } catch (InterruptedException e) {
    }

    try {
        return outFrames.take();
    } catch (InterruptedException e) {
        return rgba;
    }

消费类

public class Consumer implements Runnable {
private final BlockingQueue<Mat> queueIn;
private final BlockingQueue<Mat> queueOut;
private boolean isRunning;

public Consumer(BlockingQueue qIn, BlockingQueue qOut) {
    queueIn = qIn;
    queueOut = qOut;
    isRunning = true;
}
@Override
public void run() {
    try {
        while (isRunning) { consume(queueIn.take()); }
    } catch (InterruptedException ex) { }
}
void consume(Mat src) {
    Mat mIntermediateMat = new Mat(src.rows(), src.cols(), CvType.CV_8UC1);
    Mat dst = new Mat(src.size(), CvType.CV_8UC3);
    Mat mHsv = new Mat(src.size(), CvType.CV_8UC3);
    Mat mHsv2 = new Mat(src.size(), CvType.CV_8UC3);

    src.copyTo(dst);
    // Convert to HSV
    Imgproc.cvtColor(src, mHsv, Imgproc.COLOR_RGB2HSV, 3);

    // Remove all colors except the reds
    Core.inRange(mHsv, new Scalar(0, 86, 72), new Scalar(39, 255, 255), mHsv);
    Core.inRange(mHsv, new Scalar(150, 125, 100), new Scalar(180,255,255), mHsv2); 
    Core.bitwise_or(mHsv, mHsv2, mHsv);

    /// Reduce the noise so we avoid false circle detection
    Imgproc.GaussianBlur(mHsv, mHsv, new Size(7, 7), 2);
    // Find Circles
    Imgproc.HoughCircles(mHsv, mIntermediateMat, Imgproc.CV_HOUGH_GRADIENT, 2.0, 100);

    // Find the largest circle
    int maxRadious = 0;
    Point pt = new Point(0,0);
    if (mIntermediateMat.cols() > 0) {
        for (int x = 0; x < mIntermediateMat.cols(); x++)
        {
            double vCircle[] = mIntermediateMat.get(0,x);

            if (vCircle == null)
                break;

            int radius = (int)Math.round(vCircle[2]);
            if (radius > maxRadious) {
                maxRadious = radius;
                pt = new Point(Math.round(vCircle[0]), Math.round(vCircle[1]));
            }

        }

      // Draw the larest circle in Red
        int iLineThickness = 5;
        Scalar red = new Scalar(255, 0, 0);
        // draw the found circle
        Core.circle(dst, pt, maxRadious, red, iLineThickness);
        try{
            queueOut.put(dst);
        } catch (InterruptedException e) {
        }
    }
}

public void stopRunning() {
    this.isRunning = false;
}
}

这是我在大约 30 秒后得到的错误

01-16 14:20:23.358      510-586/? E/InputDispatcher﹕ channel '428188e8 my.com/my.com.CameraActivity (server)' ~ Channel is unrecoverably broken and will be disposed!
01-16 14:20:23.508  10788-24667/? E/Surface﹕ queueBuffer: error queuing buffer to SurfaceTexture, -32
01-16 14:20:23.508  10788-24667/? E/NvOmxCamera﹕ Queue Buffer Failed. Skipping buffer.
01-16 14:20:23.508  10788-24667/? E/NvOmxCamera﹕ Dequeue Buffer Failed
01-16 14:20:23.578  10788-17442/? E/NvOmxCamera﹕ Already called release()

---

我想补充一点，我在 Nexus 7 (2012) 上运行，它是 quod-core

Answer 1

只是一个想法，但您可以尝试在这一行复制输入 rgba 吗:

  inFrames.put(inputFrame.rgba().clone());

当您在队列中添加并保存对最后捕获图像的引用时，我认为如果 CvCamera 无法释放和重新分配空间，它可能会感到不安；这可能解释了崩溃。

就加速而言，大概在单个 CPU 设备上，只有当您开始跳过对要从队列中取出的帧的处理时，您才会加快速度。