I started playing with WebCodecs API. I try to create mp4 video from set of jpg images.
Starting with something simple, I'm trying to create an mp4 from one jpg image. When the output callback is invoked it is saving video chunk into the mp4 file.

我开始使用WebCodecs API。我尝试从一组jpg图像创建MP4视频。从一些简单的东西开始，我正在尝试从一个jpg图像创建一个MP4。当调用输出回调时，它将视频块保存到MP4文件中。

 const downloadVideo = (data) => {

       console.log("downlaodVideo data:" + data);
       const link = document.createElement("a");
 
       const file = new Blob([ Uint8Array.from( data ) ] , {type: "application/octet-stream"});
       link.href = URL.createObjectURL(file);
       link.download = "video.h264";
       link.click();
       URL.revokeObjectURL(link.href);
  };

$(document).ready(async function () {


     const videoEncoder = new VideoEncoder(
                {
                async output(chunk, metadata) {
                    console.log("encoder queue:" + videoEncoder.encodeQueueSize);
                    console.log("timestamp:" + chunk.timestamp + " counter:" + counter);
                    console.log("length:" + chunk.byteLength);
                    console.log("video chunk type:" + chunk.type);
                    console.log("duration:" + chunk.duration)
                    console.log(JSON.stringify(metadata));
                    console.log("decoder config:" + metadata.decoderConfig );
                    
                    var myMetadata;
                    var videoBlob;

                    if (metadata.decoderConfig) {
                        //# save the decoder's description/config ...
                        //# is SPS and PPS metadata needed by players to display H.264)
                        console.log("decoder config description:" + metadata.decoderConfig.description);
                        myMetadata = new Uint8Array( metadata.decoderconfig.description );
                    }

                    videoBlob = new ArrayBuffer(chunk.byteLength);
                    chunk.copyTo(videoBlob);

                    //# combine the two arrays (in the shown order of appearance)
                    var outputBytes = [...myMetadata, ...videoBlob];
  

                    downloadVideo(outputBytes);
                },
                error(error) {
                    console.log(output_Bytes);
                },
            }
            );
    

            const encoderConfig = {
                codec: "avc1.42001E",
                avc: { format: 'annexb' },
                height: 480,
                width: 640,
                framerate: 1,
                latencyMode: "realtime",
                bitrate: 2_000_000, // 2 Mbps
            };



            const support = await VideoEncoder.isConfigSupported(encoderConfig);
                
            if (support.supported) {
                console.log("Video Encoder configured!");
                videoEncoder.configure(encoderConfig );
            }
            else {
                console.error("Video codec not supported!");
            }




            $('#create_movie').click( function(ev) {

                const myImage = new Image();
                myImage.src = "samples/pic0.jpg";

                var imageBitmap = null;
                var imageBitmapPromise = null;

                myImage.onload = () => {

                    imgContext.drawImage(myImage,0,0);
                    imageBitmapPromise = createImageBitmap(myImage);

                    imageBitmap = imageBitmapPromise.then( result => {
                        const ms = 1_000_000; // 1µs
                        const fps = 10;
                        return new VideoFrame(result, {
                            timestamp: (ms * 1) / fps,
                            duration: ms / fps
                        },false);
                    }).then(vidFrame => {
                        console.log("imageBitmap:" + vidFrame);
                        console.log("video frame timestamp:" + vidFrame.timestamp);
                        console.log("format:" + vidFrame.format);
                        videoEncoder.encode(vidFrame, { keyFrame: true });

                        vidFrame.close();
                    });

                }


            });

}

Unfortunately, the video opens with an error message: The file you trying to play is empty file
I generated mp4 from the same jpg file with ffmpeg tool:

不幸的是，视频打开时会出现一条错误消息：您试图播放的文件是我用ffmpeg工具从相同的jpg文件生成的MP4空文件：

ffmpeg  -i pic%d.jpg -vcodec mpeg4 test.mp4

The structure of both files is quite different:

两个文件的结构完全不同：

On the left is video generated by ffmpeg. It contains some metadata that are missing in vido file generated by my code. So my question is: how to properly generate video file using WebCodecs API?

左边是由ffmpeg生成的视频。它包含一些由我的代码生成的VIDO文件中缺失的元数据。所以我的问题是：如何使用WebCodecs API正确生成视频文件？

"I try to create mp4 video from set of jpg images"

WebCodecs doesn't do any muxing into containers like MP4 or AVI etc. It only gives the raw encoded frames (video or audio) and then the code-writer themselves will decide on an output container format (if needed).

WebCodecs不对MP4或AVI等容器进行任何多路转换。它只给出原始的编码帧(视频或音频)，然后代码编写者自己决定输出容器格式(如果需要)。

Are you assuming you need MP4 to display it?

If yes: H.264 is playable as it is, since it's a video format.

你认为你需要MP4来显示它吗？如果是：H.264是可以播放的，因为它是一种视频格式。

Or you know for sure that you need MP4 output (eg: for HTML5 playback)?
If yes: Good luck. Not hard but very tedious to double-check everything.

或者你确定你需要MP4输出(例如：用于HTML5播放)？如果是：祝你好运。复查每一件事并不难，但非常繁琐。

Is it for preview purposes?

Use a player like VLC to play your saved H.264 file If you simply want to check that your code worked fine. VLC runs on desktop (so just drag your file into its window to check the visuals).

如果你只是想检查你的代码是否工作正常，使用VLC这样的播放器来播放你保存的H.264文件。VLC在桌面上运行（因此只需将您的文件拖到其窗口中以检查视觉效果）。

For online playback you can either find a JS-ready muxer like: JMuxer as shown in the code below, or else enjoy learning about MP4 structure then write the code. For writing own MP4 bytes I tried to explain a starting point on another Question, but they never responded.

对于在线播放，您可以找到一个支持JS的复用器，如下面的代码所示：JMuxer，或者享受学习MP4结构然后编写代码。对于编写自己的MP4字节，我试图解释另一个问题的起点，但他们从未回应。

Outside of webcodecs, you can just draw images into canvas then encode into MP4 using H.264-MP4-Encoder.

在网络编解码器之外，你只需将图像绘制到画布中，然后使用H.264-MP4-编码器编码成MP4即可。

You first need a correctly encoded H.264 file or no player will accept it.
(see below for fixes to your code)...

您首先需要一个正确编码的H.264文件，否则没有播放器会接受它。(有关代码的修复，请参阅以下内容)...

To encode a playable raw H.264:

You need to set Annex-B as the output format

您需要将附录B设置为输出格式

videoEncoder.configure( {
                codec: "avc1.42001E",
                avc: { format: 'annexb' },
                height: 480,
                width: 640,
                framerate: 1,
                bitrate: 2_000_000, // 2 Mbps
            });

To encode your H.264 contained inside MP4 (using JMuxer):

<!DOCTYPE html>
<html>
<body>

<button id="create_movie" >  create video </button>

<br><br>

<!-- for preview of input Image's pixels -->
<canvas id="myCanvas" width="320" height="240" > </canvas

<!-- for preview of JMuxer API result -->
<video id="player" width="640" height="480" > </video>

<!-- for using JMuxer API -->
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/jmuxer.min.js"> 
</script>

<script>

const player = document.getElementById("player");
const canvas = document.getElementById("myCanvas");
var ctx = canvas.getContext("2d");
            
//# run function create_movie() on click ...
document.getElementById("create_movie").addEventListener( "click", create_movie );

//var myImage = new Image();
var myImage = document.createElement( "img");
var imageBitmap; var imageBitmapPromise;
var videoBlob;


const encoder_init =    {
                            output: handle_chunk,
                            error: (e) => { console.log("::: Webcodecs Encoder Error ::: " + "/n" + e.message) },
                        };

const encoder_config =  {
                            //codec: "avc1.42001E",
                            codec: "avc1.42C016",
                            avc: {  format: "annexb" },
                            //avc: {  format: "avc" },
                            width: 640,
                            height: 480,
                            framerate: 1,
                            latencyMode: "realtime",
                            bitrateMode: "constant",
                            bitrate: 2_000_000, // 2 Mbps
                        };

//# will update entry values as needed per frame encoding
const chunk_init =  {
                        type: "null",
                        timestamp: 0,
                        duration: 5000,
                    };
                    


const videoEncoder = new VideoEncoder( encoder_init );
videoEncoder.configure( encoder_config );

console.log("video encoder created! " + videoEncoder);

//# if using JMuxer to create MP4 container 
var jmuxer = new JMuxer(
                            {
                                node: "player", //# ID of <video> tag for showing output
                                mode: "video",
                                flushingTime: 100,
                                fps: 1,
                                maxDelay: 1,
                                clearBuffer: true,
                                debug: true
                            }
                        );

function create_movie( )
{
    //alert("create_movie");
    
    //# start loading an Image as the input frame
    load_image_frame("test_640x480.jpg");
    
}

async function load_image_frame( input_path )
{
    myImage.addEventListener("load", () => { handle_load_image(); } );
    
    myImage.src = input_path;
    myImage.decode();
}

async function handle_load_image( )
{
    let img_BMP = await createImageBitmap(  myImage );
    //alert( img_BMP );

    //# (optional) preview pixels on canvas
    ctx.drawImage(img_BMP, 0, 0, 640, 480, 0, 0, 320, 240);
    
    //# func encode_frame params --> ( in_Bitmap , in_type , in_timestamp , in_duration )
    encode_frame( img_BMP, "key" , 0 , 5000 );
}

function encode_frame( in_Bitmap , in_type , in_timestamp , in_duration )
{
    //alert("encode_frame");
    
    //# per frame settings...
    chunk_init.type = in_type;
    chunk_init.timestamp = in_timestamp;
    chunk_init.duration = in_duration;
    
    curr_Frame = new VideoFrame( in_Bitmap, chunk_init );
    
    if (videoEncoder.encodeQueueSize > 2) 
    {
        //# drop this frame (since 2 is too many frames in 1 queue)
        curr_Frame.close();
    } 
    else 
    {
        //# encode as "keyframe" (or false == a "delta" frame)
        videoEncoder.encode( curr_Frame , { keyFrame: true } );
    }   
}

async function handle_chunk( chunk, metadata ) 
{
    console.log("timestamp: " + chunk.timestamp);
    console.log("length: " + chunk.byteLength);
    console.log("video chunk type: " + chunk.type);
    console.log("duration: " + chunk.duration)
    
    //console.log(JSON.stringify(metadata));
    
    //# metadata shows only if the format is "avc" (eg: is not needed for "annexb") ....
    if( encoder_config.avc.format == "avc" )
    {
        if (metadata.decoderConfig) 
        {
            //# save the decoder description ( is SPS and PPS for AVC/MP4 )
            myConfigBytes = new Uint8Array(  metadata.decoderConfig.description );
            
            //# preview the description... (is bytes Integers, not String chars)
            alert( 
                    "Config bytes for MP4 \"stsd\" atom : " + "\n" +
                    print_hex_from_arr( myConfigBytes ) 
                 );
        }
        
    }

    //# get actual bytes of H.264 encoded data ...
    videoBlob = new Uint8Array( chunk.byteLength );
    chunk.copyTo( videoBlob );
    
    //### choose output option: raw H264 file or MP4 created with JMuxer
    
    //### option 1 : H.264 --> save playable raw H.264 file
    //downloadVideo(videoBlob);
    
    //### option 2 : MP4 --> encode MP4 using JMuxer
    encode_MP4( videoBlob );
    
    //# clear encoder buffers
    videoEncoder.flush();
    
}

function encode_MP4( videoBlob )
{
    //# show play controls (if wanted)
    player.controls = "controls";
    
    //# add an encoded video frame for streaming
    jmuxer.feed(
                    {
                        video: videoBlob,
                        duration: 5000
                    }
                );  
}

function print_hex_from_arr( input_array )
{
    let tmp_hex_str = "";
    let tmp_byte_val = "";
    
    //# write each value into an array slot
    for ( let i=0; i < input_array.length; i++ ) 
    { 
        tmp_byte_val = input_array[i].toString(16);
        
        if( (tmp_byte_val.length % 2) !== 0 ) 
        { tmp_byte_val = ( "0" + tmp_byte_val ); }
        
        tmp_hex_str += tmp_byte_val;
        tmp_hex_str += " ";
    }
    
    return ( tmp_hex_str.toUpperCase() );
}


</script>

</body>
</html>