image-processing - 将YUV4:4:4转换为YUV4:2:2图像

Question

互联网上有很多关于YUV4：4：4到YUV4：2：2格式之间差异的信息，但是，我找不到任何内容可以告诉您如何将YUV4：4：4转换为YUV4：2：2 。由于此类转换是使用软件执行的，因此我希望应该有一些开发人员能够完成此转换，并且可以将我定向到描述转换算法的资源中。当然，拥有软件代码会很不错，但是可以使用该理论就足以编写我自己的软件。具体来说，我想了解像素结构以及转换期间如何管理字节。

我发现了几个类似的问题，例如this和this，但是无法回答我的问题。另外，我将此问题发布在Photography forum上，他们将其视为软件问题。

Answer 1

之所以找不到特定的描述，是因为有很多方法可以做到。
让我们从维基百科开始：https://en.wikipedia.org/wiki/Chroma_subsampling#4:2:2


4：4：4：
三个Y'CbCr分量中的每个分量均具有相同的采样率，因此没有色度二次采样。该方案有时用于高端胶片扫描仪和电影后期制作中。


和


4：2：2：
两个色度分量以亮度采样率的一半进行采样：水平色度分辨率减半。这将未压缩视频信号的带宽减少了三分之一，几乎没有视觉差异。


注意：术语YCbCr和YUV可互换使用。
https://en.wikipedia.org/wiki/YCbCr


Y'CbCr通常与YUV色彩空间相混淆，通常YCbCr和YUV术语可以互换使用，从而引起一些混淆；当提及视频或数字形式的信号时，术语“ YUV”主要表示“ Y'CbCr”。


数据存储器排序：
同样，不止一种格式。
英特尔IPP文档定义了两个主要类别：“像素顺序图像格式”和“平面图像格式”。
这里有一个很好的文档：https://software.intel.com/en-us/node/503876
有关YUV像素排列格式，请参见此处：http://www.fourcc.org/yuv.php#NV12。
请参考：http://scc.ustc.edu.cn/zlsc/sugon/intel/ipp/ipp_manual/IPPI/ippi_ch6/ch6_image_downsampling.htm#ch6_image_downsampling以获取下采样说明。

让我们假设“像素顺序”格式：

YUV 4:4:4 data order: Y0 U0 V0  Y1 U1 V1  Y2 U2 V2  Y3 U3 V3  
YUV 4:2:2 data order: Y0  U0    Y1  V0    Y2  U1    Y3  V1  

每个元素都是一个字节，Y0是内存中的低字节。
上面描述的4：2：2数据顺序称为UYVY或 YUY2像素格式。

转换算法：


“朴素的子采样”：
每秒“扔” U / V组件：
拿 U0然后扔 U1，拿 V0然后扔 V1 ...
来源： Y0 U0 V0 Y1 U1 V1 Y2 U2 V2
目的地： Y0 U0 Y1 V0 Y2 U2 Y3 V2
我不推荐它，因为它会导致 aliasing工件。
平均每个 U / V对：
将目标 U0等于源 (U0+U1)/2，与 V0相同...
来源： Y0 U0 V0 Y1 U1 V1 Y2 U2 V2
目的地： Y0 (U0+U1)/2 Y1 (V0+V1)/2 Y2 (U2+U3)/2 Y3 (V2+V3)/2
使用其他插值方法对U和V进行下采样（例如三次插值）。
通常，与简单平均值相比，您将看不到任何差异。




C实现：

该问题未标记为C，但是我认为以下C实现可能会有所帮助。
以下代码通过平均每个U / V对将像素排序的YUV 4：4：4转换为像素排序的YUV 4：2：2：

//Convert single row I0 from pixel-ordered YUV 4:4:4 to pixel-ordered YUV 4:2:2.
//Save the result in J0.
//I0 size in bytes is image_width*3
//J0 size in bytes is image_width*2
static void ConvertRowYUV444ToYUV422(const unsigned char I0[],
                                     const int image_width,
                                     unsigned char J0[])
{
    int x;

    //Process two Y,U,V triples per iteration:
    for (x = 0; x < image_width; x += 2)
    {
        //Load source elements
        unsigned char y0    = I0[x*3];                  //Load source Y element
        unsigned int u0     = (unsigned int)I0[x*3+1];  //Load source U element (and convert from uint8 to uint32).
        unsigned int v0     = (unsigned int)I0[x*3+2];  //Load source V element (and convert from uint8 to uint32).

        //Load next source elements
        unsigned char y1    = I0[x*3+3];                //Load source Y element
        unsigned int u1     = (unsigned int)I0[x*3+4];  //Load source U element (and convert from uint8 to uint32).
        unsigned int v1     = (unsigned int)I0[x*3+5];  //Load source V element (and convert from uint8 to uint32).

        //Calculate destination U, and V elements.
        //Use shift right by 1 for dividing by 2.
        //Use plus 1 before shifting - round operation instead of floor operation.
        unsigned int u01    = (u0 + u1 + 1) >> 1;       //Destination U element equals average of two source U elements.
        unsigned int v01    = (v0 + v1 + 1) >> 1;       //Destination U element equals average of two source U elements.

        J0[x*2]     = y0;   //Store Y element (unmodified).
        J0[x*2+1]   = (unsigned char)u01;   //Store destination U element (and cast uint32 to uint8).
        J0[x*2+2]   = y1;   //Store Y element (unmodified).
        J0[x*2+3]   = (unsigned char)v01;   //Store destination V element (and cast uint32 to uint8).
    }
}


//Convert image I from pixel-ordered YUV 4:4:4 to pixel-ordered YUV 4:2:2.
//I - Input image in pixel-order data YUV 4:4:4 format.
//image_width - Number of columns of image I.
//image_height - Number of rows of image I.
//J - Destination "image" in pixel-order data YUV 4:2:2 format.
//Note: The term "YUV" referees to "Y'CbCr".

//I is pixel ordered YUV 4:4:4 format (size in bytes is image_width*image_height*3):
//YUVYUVYUVYUV
//YUVYUVYUVYUV
//YUVYUVYUVYUV
//YUVYUVYUVYUV
//
//J is pixel ordered YUV 4:2:2 format (size in bytes is image_width*image_height*2):
//YUYVYUYV
//YUYVYUYV
//YUYVYUYV
//YUYVYUYV
//
//Conversion algorithm:
//Each element of destination U is average of 2 original U horizontal elements
//Each element of destination V is average of 2 original V horizontal elements
//
//Limitations:
//1. image_width must be a multiple of 2.
//2. I and J must be two separate arrays (in place computation is not supported). 
static void ConvertYUV444ToYUV422(const unsigned char I[],
                                  const int image_width,
                                  const int image_height,
                                  unsigned char J[])
{
    //I0 points source row.
    const unsigned char *I0;    //I0 -> YUYVYUYV...

    //J0 and points destination row.
    unsigned char *J0;          //J0 -> YUYVYUYV

    int y;  //Row index

    //In each iteration process single row.
    for (y = 0; y < image_height; y++)
    {
        I0 = &I[y*image_width*3];   //Input row width is image_width*3 bytes (each pixel is Y,U,V).

        J0 = &J[y*image_width*2];   //Output row width is image_width*2 bytes (each two pixels are Y,U,Y,V).

        //Process single source row into single destination row
        ConvertRowYUV444ToYUV422(I0, image_width, J0);
    }
}

YUV 4：2：2的平面表示

平面表示可能比“像素顺序”格式更直观。
在平面表示中，每个颜色通道表示为一个单独的矩阵，可以将其显示为图像。

例：


RGB格式的原始图像（转换为YUV之前）：

YUV 4：4：4格式的图像通道：

（左YUV三元组用灰度表示，右YUV三元组用假色表示）。
YUV 4：2：2格式的图像通道（在水平 Chroma subsampling之后）：

（左YUV三元组用灰度表示，右YUV三元组用“假色”表示）。


如您所见，在4：2：2格式中，U和V通道在水平轴上被下采样（缩小）。

备注：
U和V通道的“假色”表示用于强调Y是 Luma通道，U和V是 Chrominance通道。



高阶插值和抗混叠滤波器：
以下MATLAB代码示例显示了如何使用高阶插值和抗锯齿滤波器执行下采样。
该示例还显示了 FFMPEG使用的下采样方法。
注意：您无需了解MATLAB编程即可了解示例。
您确实需要通过 Kernel和图像之间的卷积来进行图像过滤的一些知识。

%Prepare the input:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
load('mandrill.mat', 'X', 'map'); %Load input image
RGB = im2uint8(ind2rgb(X, map));  %Convert to RGB (the mandrill sample image is an indexed image)
YUV = rgb2ycbcr(RGB);             %Convert from RGB to YUV (MATLAB function rgb2ycbcr uses BT.601 conversion formula)

%Separate YUV to 3 planes (Y plane, U plane and V plane)
Y = YUV(:, :, 1);
U = YUV(:, :, 2);
V = YUV(:, :, 3);

U = double(U); %Work in double precision instead of uint8.

[M, N] = size(Y); %Image size is N columns by M rows.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


%Linear interpolation without Anti-Aliasing filter:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Horizontal down-sampling U plane using Linear interpolation (without Anti-Aliasing filter).
%Simple averaging is equivalent to linear interpolation.
U2 = (U(:, 1:2:end) + U(:, 2:2:end))/2;
refU2 = imresize(U, [M, N/2], 'bilinear', 'Antialiasing', false); %Use MATLAB imresize function as reference
disp(['Linear interpolation max diff = ' num2str(max(abs(double(U2(:)) - double(refU2(:)))))]); %Print maximum difference.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


%Cubic interpolation without Anti-Aliasing filter:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Horizontal down-sampling U plane using Cubic interpolation (without Anti-Aliasing filter).
%Following operations are equivalent to cubic interpolation:
%1. Convolution with filter kernel [-0.125, 1.25, -0.125]
%2. Averaging pair elements
fU = imfilter(U, [-0.125, 1.25, -0.125], 'symmetric');
U2 = (fU(:, 1:2:end) + fU(:, 2:2:end))/2;
U2 = max(min(U2, 240), 16); %Limit to valid range of U elements (valid range of U elements in uint8 format is [16, 240])
refU2 = imresize(U, [M, N/2], 'cubic', 'Antialiasing', false); %Use MATLAB imresize function as reference
refU2 = max(min(refU2, 240), 16); %Limit to valid range of U elements
disp(['Cubic interpolation max diff = ' num2str(max(abs(double(U2(:)) - double(refU2(:)))))]); %Print maximum difference.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


%Linear interpolation with Anti-Aliasing filter:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Horizontal down-sampling U plane using Linear interpolation with Anti-Aliasing filter.
%Remark: The Anti-Aliasing filter is the filter used by MATLAB specific implementation of 'bilinear' imresize.
%Following operations are equivalent to Linear interpolation with Anti-Aliasing filter:
%1. Convolution with filter kernel [0.25, 0.5, 0.25]
%2. Averaging pair elements
fU = imfilter(U, [0.25, 0.5, 0.25], 'symmetric');
U2 = (fU(:, 1:2:end) + fU(:, 2:2:end))/2;
refU2 = imresize(U, [M, N/2], 'bilinear', 'Antialiasing', true); %Use MATLAB imresize function as reference
disp(['Linear interpolation with Anti-Aliasing max diff = ' num2str(max(abs(double(U2(:)) - double(refU2(:)))))]); %Print maximum difference.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


%Cubic interpolation with Anti-Aliasing filter:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Horizontal down-sampling U plane using Cubic interpolation with Anti-Aliasing filter.
%Remark: The Anti-Aliasing filter is the filter used by MATLAB specific implementation of 'cubic' imresize.
%Following operations are equivalent to Linear interpolation with Anti-Aliasing filter:
%1. Convolution with filter kernel [-0.0234375, -0.046875, 0.2734375, 0.59375, 0.2734375, -0.046875, -0.0234375]
%2. Averaging pair elements
h = [-0.0234375, -0.046875, 0.2734375, 0.59375, 0.2734375, -0.046875, -0.0234375];
fU = imfilter(U, h, 'symmetric');
U2 = (fU(:, 1:2:end) + fU(:, 2:2:end))/2;
U2 = max(min(U2, 240), 16); %Limit to valid range of U elements
refU2 = imresize(U, [M, N/2], 'cubic', 'Antialiasing', true); %Use MATLAB imresize function as reference
refU2 = max(min(refU2, 240), 16); %Limit to valid range of U elements
disp(['Cubic interpolation with Anti-Aliasing max diff = ' num2str(max(abs(double(U2(:)) - double(refU2(:)))))]); %Print maximum difference.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


%FFMPEG implementation of horizontal down-sampling U plane.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%FFMPEG uses cubic interpolation with Anti-Aliasing filter (different filter kernel):
%Remark: I didn't check the source code of FFMPEG to verify the values of the filter kernel.
%I can't tell how FFMPEG actually implements the conversion.
%Following operations are equivalent to FFMPEG implementation (with minor differences):
%1. Convolution with filter kernel [-115, -231, 1217, 2354, 1217, -231, -115]/4096
%2. Averaging pair elements
h = [-115, -231, 1217, 2354, 1217, -231, -115]/4096;
fU = imfilter(U, h, 'symmetric');
U2 = (fU(:, 1:2:end) + fU(:, 2:2:end))/2;
U2 = max(min(U2, 240), 16); %Limit to valid range of U elements (FFMPEG actually doesn't limit the result)

%Save Y,U,V planes to file in format supported by FFMPEG
f = fopen('yuv444.yuv', 'w');
fwrite(f, Y', 'uint8');
fwrite(f, U', 'uint8');
fwrite(f, V', 'uint8');
fclose(f);

%For executing FFMPEG within MATLAB, download FFMPEG and place the executable in working directory (ffmpeg.exe for Windows)
%FFMPEG converts source file in YUV444 format to destination file in YUV422 format.
if isunix
    [status, cmdout] = system(['./ffmpeg -y -s ', num2str(N), 'x', num2str(M), ' -pix_fmt yuv444p -i yuv444.yuv -pix_fmt yuv422p yuv422.yuv']);
else
    [status, cmdout] = system(['ffmpeg.exe -y -s ', num2str(N), 'x', num2str(M), ' -pix_fmt yuv444p -i yuv444.yuv -pix_fmt yuv422p yuv422.yuv']);
end
f = fopen('yuv422.yuv', 'r');
refY = (fread(f, [N, M], '*uint8'))';
refU2 = (fread(f, [N/2, M], '*uint8'))'; %Read down-sampled U plane (FFMPEG result from file).
refV2 = (fread(f, [N/2, M], '*uint8'))';
fclose(f);

%Limit to valid range of U elements.
%In FFMPEG down-sampled U and V may exceed valid range (there is probably a way to tell FFMPEG to limit the result).
refU2 = max(min(refU2, 240), 16);

%Difference exclude first column and last column (FFMPEG treats the margins different than MATLAB)
%Remark: There are minor differences due to rounding (I guess).
disp(['FFMPEG Cubic interpolation with Anti-Aliasing max diff = ' num2str(max(max(abs(double(U2(:, 2:end-1)) - double(refU2(:, 2:end-1))))))]);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

不同种类的下采样方法的示例。
使用抗锯齿滤波器的线性插值与三次插值：
在第一个示例（山d）中，没有可见的差异。
在第二个示例（圆形和矩形）中，可见的差异很小。
第三个示例（行）演示了混叠工件。
备注：显示的图像是使用三次插值从YUV422向上采样到YUV444并从YUV444转换为RGB的图像。


线性插值与三次抗锯齿（mandrill）：

线性插值与三次抗锯齿（圆形和矩形）：

线性插值与三次使用抗锯齿（演示锯齿伪像）：