java - OpenCV VideoWriter for Java 在 MacOS 上无法按预期工作

Question

试图解决 http://wiki.bitplan.com/index.php/OpenCV#OpenCV_VideoWriter 上的问题我找到了以下相关链接:

• Writing x264 from OpenCV 3 with FFmpeg on Linux
• https://answers.opencv.org/question/120081/videowriter-recording-mp4-x264-opencv-31/
• opencv VideoWriter under OSX producing no output
• http://www.fourcc.org/codecs.php
• opencv write webcam output to avi file

然后想出了 the code below尝试扩展名和 FOURCC 代码的组合。

环境是使用 Java 8 和 Macports 2.6.2 的 Mac Os 10.13.6。使用 OpenCV 3.4.8我还在 Ubuntu 18.04 LTS 上使用 OpenJDK 1.8.0_232 和 libopencv_java343.so 进行了测试

消息和文件显示我尝试的 21 种组合均无效。

视频录制工作可能出现的问题和解决方案是什么？

生成的消息是:

recording: 640x480 mov video with MJPG
OpenCV: FFMPEG: tag 0x47504a4d/'MJPG' is not supported with codec id 7 and format 'mov / QuickTime / MOV'
OpenCV: FFMPEG: fallback to use tag 0x6765706a/'jpeg'
recording: 640x480 mov video with X264
OpenCV: FFMPEG: tag 0x34363258/'X264' is not supported with codec id 27 and format 'mov / QuickTime / MOV'
OpenCV: FFMPEG: fallback to use tag 0x31637661/'avc1'
recording: 640x480 mov video with H264
OpenCV: FFMPEG: tag 0x34363248/'H264' is not supported with codec id 27 and format 'mov / QuickTime / MOV'
OpenCV: FFMPEG: fallback to use tag 0x31637661/'avc1'
recording: 640x480 mov video with MP4V
OpenCV: FFMPEG: tag 0x5634504d/'MP4V' is not supported with codec id 12 and format 'mov / QuickTime / MOV'
OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v'
recording: 640x480 mov video with AVC1
OpenCV: FFMPEG: tag 0x31435641/'AVC1' is not supported with codec id 27 and format 'mov / QuickTime / MOV'
OpenCV: FFMPEG: fallback to use tag 0x31637661/'avc1'
recording: 640x480 mov video with FMP4
OpenCV: FFMPEG: tag 0x34504d46/'FMP4' is not supported with codec id 12 and format 'mov / QuickTime / MOV'
OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v'
recording: 640x480 mov video with JPEG
OpenCV: FFMPEG: tag 0x4745504a/'JPEG' is not supported with codec id 7 and format 'mov / QuickTime / MOV'
OpenCV: FFMPEG: fallback to use tag 0x6765706a/'jpeg'
recording: 640x480 avi video with MJPG
recording: 640x480 avi video with X264
recording: 640x480 avi video with H264
recording: 640x480 avi video with MP4V
OpenCV: FFMPEG: tag 0x5634504d/'MP4V' is not supported with codec id 12 and format 'avi / AVI (Audio Video Interleaved)'
OpenCV: FFMPEG: fallback to use tag 0x34504d46/'FMP4'
recording: 640x480 avi video with AVC1
OpenCV: FFMPEG: tag 0x31435641/'AVC1' is not supported with codec id 27 and format 'avi / AVI (Audio Video Interleaved)'
OpenCV: FFMPEG: fallback to use tag 0x34363248/'H264'
recording: 640x480 avi video with FMP4
recording: 640x480 avi video with JPEG
OpenCV: FFMPEG: tag 0x4745504a/'JPEG' is not supported with codec id 7 and format 'avi / AVI (Audio Video Interleaved)'
OpenCV: FFMPEG: fallback to use tag 0x47504a4d/'MJPG'
recording: 640x480 mpg video with MJPG
OpenCV: FFMPEG: tag 0x47504a4d/'MJPG' is not supported with codec id 7 and format 'mpeg / MPEG-1 Systems / MPEG program stream'
recording: 640x480 mpg video with X264
OpenCV: FFMPEG: tag 0x34363258/'X264' is not supported with codec id 27 and format 'mpeg / MPEG-1 Systems / MPEG program stream'
recording: 640x480 mpg video with H264
OpenCV: FFMPEG: tag 0x34363248/'H264' is not supported with codec id 27 and format 'mpeg / MPEG-1 Systems / MPEG program stream'
recording: 640x480 mpg video with MP4V
OpenCV: FFMPEG: tag 0x5634504d/'MP4V' is not supported with codec id 12 and format 'mpeg / MPEG-1 Systems / MPEG program stream'
recording: 640x480 mpg video with AVC1
OpenCV: FFMPEG: tag 0x31435641/'AVC1' is not supported with codec id 27 and format 'mpeg / MPEG-1 Systems / MPEG program stream'
recording: 640x480 mpg video with FMP4
OpenCV: FFMPEG: tag 0x34504d46/'FMP4' is not supported with codec id 12 and format 'mpeg / MPEG-1 Systems / MPEG program stream'
recording: 640x480 mpg video with JPEG
OpenCV: FFMPEG: tag 0x4745504a/'JPEG' is not supported with codec id 7 and format 'mpeg / MPEG-1 Systems / MPEG program stream'

生成的文件是:

-rw-r--r-- 1 wf    5686  9. Jan 10:19 test_AVC1_2020-01-09101907.avi
-rw-r--r-- 1 wf     185  9. Jan 10:19 test_AVC1_2020-01-09101907.mov
-rw-r--r-- 1 wf       0  9. Jan 10:19 test_AVC1_2020-01-09101907.mpg
-rw-r--r-- 1 wf    5686  9. Jan 10:19 test_FMP4_2020-01-09101907.avi
-rw-r--r-- 1 wf     185  9. Jan 10:19 test_FMP4_2020-01-09101907.mov
-rw-r--r-- 1 wf       0  9. Jan 10:19 test_FMP4_2020-01-09101907.mpg
-rw-r--r-- 1 wf    5686  9. Jan 10:19 test_H264_2020-01-09101907.avi
-rw-r--r-- 1 wf     185  9. Jan 10:19 test_H264_2020-01-09101907.mov
-rw-r--r-- 1 wf       0  9. Jan 10:19 test_H264_2020-01-09101907.mpg
-rw-r--r-- 1 wf    5686  9. Jan 10:19 test_JPEG_2020-01-09101907.avi
-rw-r--r-- 1 wf     185  9. Jan 10:19 test_JPEG_2020-01-09101907.mov
-rw-r--r-- 1 wf       0  9. Jan 10:19 test_JPEG_2020-01-09101907.mpg
-rw-r--r-- 1 wf    5686  9. Jan 10:19 test_MJPG_2020-01-09101907.avi
-rw-r--r-- 1 wf     185  9. Jan 10:19 test_MJPG_2020-01-09101907.mov
-rw-r--r-- 1 wf       0  9. Jan 10:19 test_MJPG_2020-01-09101907.mpg
-rw-r--r-- 1 wf    5686  9. Jan 10:19 test_MP4V_2020-01-09101907.avi
-rw-r--r-- 1 wf     185  9. Jan 10:19 test_MP4V_2020-01-09101907.mov
-rw-r--r-- 1 wf       0  9. Jan 10:19 test_MP4V_2020-01-09101907.mpg
-rw-r--r-- 1 wf    5686  9. Jan 10:19 test_X264_2020-01-09101907.avi
-rw-r--r-- 1 wf     185  9. Jan 10:19 test_X264_2020-01-09101907.mov
-rw-r--r-- 1 wf       0  9. Jan 10:19 test_X264_2020-01-09101907.mpg

Junit 测试用例

package nl.vaneijndhoven.dukes.imageview;

import static org.junit.Assert.assertTrue;

import java.io.File;

import org.junit.BeforeClass;
import org.junit.Test;
import org.opencv.core.Mat;
import org.opencv.core.Size;

import com.bitplan.opencv.NativeLibrary;

import nl.vaneijndhoven.opencv.video.ImageUtils;

/**
 * test the video recording functionality
 * 
 * @author wf
 *
 */
public class TestVideoRecorder {
  @BeforeClass
  public static void setup() throws Exception {
    NativeLibrary.load();
  }

  @Test
  public void testVideoRecorder() {
    byte[] testImage = DebugImageServer.testImage();
    Mat testMat = ImageUtils.imageBytes2Mat(testImage);
    boolean isColor = true;
    for (String ext : VideoRecorder.exts) {
      for (String FOURCC : VideoRecorder.FOURCCs) {
        Size frameSize = new Size(testMat.width(), testMat.height());
        VideoRecorder recorder = new VideoRecorder("test", isColor);
        recorder.ext=ext;
        recorder.FOURCC=FOURCC;
        String msg = String.format("recording: %dx%d %s video with %s",
            testMat.width(), testMat.height(), ext, FOURCC);
        System.out.println(msg);

        recorder.start(25.0, frameSize);
        for (int i = 1; i <= 50; i++) {
          recorder.recordMat(testMat);
        }
        recorder.stop();
        File videoFile = new File(recorder.path);
        assertTrue(videoFile.exists());
      }
    }
  }
}

录像机

package nl.vaneijndhoven.dukes.imageview;

import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;

import org.opencv.core.Mat;
import org.opencv.core.Size;
import org.opencv.videoio.VideoWriter;

/**
 * record videos
 * @author wf
 *
 */
public class VideoRecorder {
  public static transient final String DATE_FORMAT="yyyy-MM-ddHHmmss";
  public static transient final DateFormat dateFormat=new SimpleDateFormat(DATE_FORMAT);

  String name;
  private VideoWriter save;
  private Size frameSize;
  private double fps;
  String path;
  boolean started;
  boolean isColor;
  public static String exts[]= {"mov","avi","mpg"};
  String ext="mov";
  public static String FOURCCs[]= { "MJPG", "X264", "H264", "MP4V", "AVC1", "FMP4", "JPEG"};
  String FOURCC="mp4v";

  /**
   * construct me
   * @param name - the name of the video - a timestamp will be added
   * @param isColor - true if the video is a color video
   */
  public VideoRecorder(String name, boolean isColor) {
    this.name=name;
    this.isColor=isColor;
    started=false;
  }

  // https://stackoverflow.com/questions/53158765/record-and-save-video-stream-use-opencv-in-java
  public void start(double fps, Size frameSize) {
    this.fps=fps;
    this.frameSize=frameSize;
    int fourcc = VideoWriter.fourcc(FOURCC.charAt(0), FOURCC.charAt(1), FOURCC.charAt(2), FOURCC.charAt(3)); 
    Date now = new Date();
    String timestamp=dateFormat.format(now);
    path=String.format("/tmp/%s_%s_%s.%s", name,FOURCC,timestamp,ext);
    save = new VideoWriter(path,fourcc, this.fps, this.frameSize, isColor);
    started=true;
  }

  /**
   * stop the recording
   */
  public void stop() {
    save.release();
    started=false;
  }

  /**
   * record a single frame
   * @param mat - open cv frame to be recorded
   */
  public void recordMat(Mat mat) {
    save.write(mat);
  }

}

Answer 1

问题在于测试图像是具有 4 个 channel 的 PNG 图像。

following changes修复了这种情况:

确保我们使用 jpg:

DebugImageServer.ext=".jpg";

显示使用的 channel 数:

 String msg = String.format("recording: %dx%d %d channel %s video with %s",
            testMat.width(), testMat.height(),testMat.channels(), ext, FOURCC);
        System.out.println(msg);

现在有很多组合可以工作:

-rw-r--r-- 1 wf     81845  9. Jan 11:05 test_avc1_2020-01-09110505.mov
-rw-r--r-- 1 wf     87402  9. Jan 11:05 test_avc1_2020-01-09110506.avi
-rw-r--r-- 1 wf     81920  9. Jan 11:05 test_avc1_2020-01-09110507.mpg
-rw-r--r-- 1 wf    389053  9. Jan 11:05 test_fmp4_2020-01-09110505.mov
-rw-r--r-- 1 wf    395162  9. Jan 11:05 test_fmp4_2020-01-09110506.avi
-rw-r--r-- 1 wf    391168  9. Jan 11:05 test_fmp4_2020-01-09110507.mpg
-rw-r--r-- 1 wf     81845  9. Jan 11:05 test_h264_2020-01-09110506.mov
-rw-r--r-- 1 wf     87402  9. Jan 11:05 test_h264_2020-01-09110507.avi
-rw-r--r-- 1 wf     81920  9. Jan 11:05 test_h264_2020-01-09110508.mpg
-rw-r--r-- 1 wf   3618564  9. Jan 11:05 test_jpeg_2020-01-09110506.mov
-rw-r--r-- 1 wf   3624510  9. Jan 11:05 test_jpeg_2020-01-09110507.avi
-rw-r--r-- 1 wf   3631104  9. Jan 11:05 test_jpeg_2020-01-09110508.mpg
-rw-r--r-- 1 wf   3618564  9. Jan 11:05 test_mjpg_2020-01-09110506.mov
-rw-r--r-- 1 wf   3624510  9. Jan 11:05 test_mjpg_2020-01-09110507.avi
-rw-r--r-- 1 wf   3631104  9. Jan 11:05 test_mjpg_2020-01-09110508.mpg
-rw-r--r-- 1 wf    389053  9. Jan 11:05 test_mp4v_2020-01-09110506.mov
-rw-r--r-- 1 wf    395162  9. Jan 11:05 test_mp4v_2020-01-09110507.avi
-rw-r--r-- 1 wf    391168  9. Jan 11:05 test_mp4v_2020-01-09110508.mpg
-rw-r--r-- 1 wf     81845  9. Jan 11:05 test_x264_2020-01-09110506.mov
-rw-r--r-- 1 wf     87402  9. Jan 11:05 test_x264_2020-01-09110507.avi
-rw-r--r-- 1 wf     81920  9. Jan 11:05 test_x264_2020-01-09110508.mpg
-rw-r--r-- 1 wf    389053  9. Jan 11:05 test_xvid_2020-01-09110506.mov
-rw-r--r-- 1 wf    395162  9. Jan 11:05 test_xvid_2020-01-09110507.avi
-rw-r--r-- 1 wf    391168  9. Jan 11:05 test_xvid_2020-01-09110508.mpg