Return to Digital Photography Articles

JPEG Chroma Subsampling

What is Chroma Subsampling?

The JPEG (JFIF) compressed file format can produce significant reductions in file size through lossy compression. The techniques used to achieve these levels of JPEG compression take advantage of the limitations of the human eye. The compression algorithm saves space by discarding additional image information / detail that may not be as noticeable to the human observer.

It is widely known that we are much more sensitive to changes in luminance (brightness) than we are to chrominance (color) differences. Because of this, the JPEG format can discard a lot more color information than luminance in the compression process. To facilitate the different compression requirements of the two "channels" of image information, the JPEG file format translates 8-bit RGB data (Red, Green, Blue) into 8-bit YCbCr data (Luminance, Chroma Blue, Chroma Red). Now, with the brightness seperated into a separate data channel, it is much easier to change the compression algorithm used for one channel versus the others.

Chroma subsampling is the process whereby the color information in the image is sampled at a lower resolution than the original. The images below show various types of chroma subsampling (from 1x1 to 2x2) and the effect it has on reducing the color information. Pay special attention to the horizontal and vertical stripes.

1x1 (No Subsampling)   2x1 (Subsampling)
 
 
1x2 (Subsampling)   2x2 (Subsampling)
 

 

NOTE: The above images depict the results of enlargement with nearest neighbour algorithm after decompressing the chroma-subsampled JPEG images. Therefore, what you observe is the actual impact of the subsampling process.

In the above series, a series of red and blue stripes were drawn with exactly the same Y value (after conversion from RGB to YCbCr) or intensity. When the JPEG compression algorithm performs the chroma subsampling step, it averages out the color information across pairs of pixel, but still preserves the luminance. By choosing RGB values for Red (RGB=97,0,0) and Blue (RGB=0,0,255) that have the same converted Y (luminance component), the resulting stripes will disappear (after "averaging"). Note that the red phosphors are significantly "brighter" than the blue phosphors given the same input stimulation (RGB value).

You can also see the more unpleasant artifacts due to chroma subsampling in regions of significant color changes in adjacent pixels. This is particularly noticeable on the green circle.

Digital Cameras and Chroma Subsampling

Most digital SLR cameras use 2x1 chroma sub-sampling, while some point and shoot digicams use 2x2. The Sigma / Foveon dSLR cameras are one of the few that don't do any chroma subsampling. The color channels from a typical 6 megapixel image (3072x2048 pixels) is first converted into a 3 megapixel image (1536x2048 pixels). In the case of 2x1 (4:2:2) chroma subsampling -- only one for every 2 pixels is used in the horizontal direction, but every pixel in the vertical direction is retained. Some graphics programs (such as Photoshop) often use 2x2 (4:2:0) chroma subsampling, which would further reduce the color information to 1536x1024 pixels.

With the reduction in color "detail", one has less information to compress, resulting in a smaller JPEG file. When it comes time to display the compressed JPEG, the compressed color channel information is uncompressed and then duplicated (doubled) across the adjacent pixels before being merged with the luminance channel for display.

The existence of chroma subsampling in JPEG compression also explains why one can achieve better compression ratios (ratio between original file size and compressed file size) with color photos than with grayscale photos. With chroma subsampling, one gets better compression out of the chrominance channels than the luminance channel. The overall compression ratio for a color photo is therefore somewhere between the two ratios. As a grayscale image has no color information to discard, the overall compression ratio is dictated purely by the lower luminance compression ratio.

Various levels of YCbCr subsampling:

  • 4:4:4 - The resolution of chrominance information (Cb & Cr) is preserved at the same rate as the luminance (Y) information. Also known as 1x1 (or subsampling disabled).
  • 4:2:2 - Half of the horizontal resolution in the chrominance is dropped (Cb & Cr), while the full resolution is retained in the vertical direction, with respect to the luminance. This is also known as 2x1 chroma subsampling, and is quite common for digital cameras.
  • 4:1:1 - Only a quarter of the chrominance information is preserved in the horizontal direction with respect to the luminance information. I don't think this format is nearly as common as the other variations.
  • 4:2:0 - With respect to the information in the luminance channel (Y), the chrominance resolution in both the horizontal and vertical directions is cut in half. This form is also known as 2x2 chroma subsampling.

Steps in JPEG Compression Chroma Subsampling

In order for a JPEG Encoder to take advantage of the file size savings through chroma subsampling, it performs the following basic steps:

  • Convert color space from RGB to YCbCr
  • For each component Cb and Cr: If horizontal chroma subsampling is specified, output Cb and Cr channels are generated from the average of each adjacent pairs of pixels in the horizontal direction. Similar process for vertical subsampling. In each case, the output channel resolution is halved.
  • For the luminance component, Y, the output channel is identical to the input, irrespective of the chroma subsampling configuration.
  • Remaining JPEG compression steps. But, the SOF0 (Baseline DCT) JPEG marker will now show that components 2 and 3 have a sampling factor other than 1x1.

Please see the relevant section in the JPEG Decoder Design page for more details on how the different subsampled channels are encoded in a JPEG file.

When is Chroma Subsampling is bad

The fact that JPEG chroma subsampling relies on the limitations of the human eye means that it is not a particularly good mechanism for compressing images used by other "sensors" (such as in the medical field, etc.), where the chrominance may be equally as important as the luminance.

Photoshop Quality and subsampling:

As an aside, note that Photoshop CS2 uses different chroma subsampling levels depending on the Save JPEG Quality settings:

  • Photoshop Save As Quality 0-6 - 2x2 Chroma Subsampling
  • Photoshop Save As Quality 7-12 - 1x1 No Chroma Subsampling
  • Photoshop Save For Web Quality 0-50 - 2x2 Chroma Subsampling
  • Photoshop Save For Web Quality 51-100 - 1x1 No Chroma Subsampling

 


Reader's Comments:

Please leave your comments or suggestions below!
2014-08-10J7N
 Excellent article. I have observed that chroma subsampling is often unconfigurable in software and forcibly activated. This might be one reason why JPEG files are generally expected to be of inherently low quality (even quality 100 with subsampling indroduces noticeable blurring on a full detail source).

To the examples where subsampling is bad, I would like to add the following cases.

A) Any low or "web" resolution photos which, heavily undersampled with regard to the resolution they were acquired in, and which are to be viewed at close distance from the screen, at which any accompanying 8pt text is also readable.

B) Synthetic images which use the entire RGB gamut, even when they are intended to be viewed by a human. (screenshots with continuous tone sections - hence why the choice to use JPEG, and video game footage)

C) JPEG files that are expected to be edited and transcoded later, in which case repeated subsampling adds to the generation losses.

Possibly related to C. Up until recent versions, Adobe Photoshop would upscale the chromas upon opening a JPEG file using nearest neighbor method, which result in the characteristic blocky reds. There are at least a couple of threads on their support forum, where senior members assert that they have correctly interpreted the vague JPEG spec, and that this behavior is correct, even after they had been presented screnshots demonstrating the inferior quality.

I wonder if the intent of the simplified chroma processing was to skip interpolation and directly re-use any of the identical chroma samples of the macropixel upon saving (avoiding quality loss due to upscaling and downscaling).

Still, it is a disaster if such a coarse image is edited (possibly while zoomed out), without realizing the error.
 Thanks for sharing some great thoughts on other issues involving chroma subsampling!
2013-12-13Markus Siebeneicher
 Thanks for your time to write this up!
 Glad it was useful!
2013-09-08chen c
 Hi
in your blog, you say the sigma cammera's chroma subsampling is 4:4:4.
It is wrong.
Because the pictures in your database have been compressed by the computer softwere of sigma. That is to say, one picture "raw" is compressed by the computer softwere of sigma.
it is 4:2:0 . you can check it by yourself
2013-08-22chen chao
 Hello, In your blog, you say "The Sigma / Foveon dSLR cameras are one of the few that don't do any chroma subsampling." .
But , I don't think so. Because I use sigma DP2Mto take some pictures and I find it uses 2*1 in chroma subsampling.
And,
Could you lists other cameras' names which don't do any chroma subsampling ?
Thank you !
2013-08-18chen c
 Hi,
Could we change the JPEG Compression Chroma Subsampling ? For example , change 4:2:2 to 4:4:4 in one cammera.
Thanks
 Hi -- yes, one can change the chroma subsampling through recompressing the images. The easiest way to do this would be with jpegtran
2012-06-15Rong
 I am doing experiments in Adobe PS and Matlab to compress the images with different quality. I am aiming at making certain that the different impact of different compression tools on images. How to do if I want to do subsampling in a specific mode(say 4:2:2, or 4:1:1) ? Can it be done in Matlab?
2012-02-29or
 Thank you for a most informative and needed page!
2010-11-23Salma
 Thanks for your great effort and help.

I wanted to ask about the subsampling/upsampling approaches used in JEPG compressed images. What are they exactly? And how are they implemented?
Thanks.
2010-03-13Jem
 The JPEG standard allows for Hx and Vx to be from 1 to 4. I've seen Sony uses 3:1:1 in some cameras. Have you seen, in JPEGSnoop's DB, weird subsamplings like:
Component 0: 3x1
Component 1: 2x1
MCU size must then be 24x8 pixels
In theory this is legal as per JFIF and JPEG standards, but it breaks the MCU completeness. Should the decoder need to average neighbouring component 1 samples in order to produce 24 pixels from the 16 pixels in the 2 blocks of component 1?
 Interesting question. In my separate database (currently ~ 11,000 signatures), I have only observed one entry with "2x3" (HxV), the rest were 1x1, 1x2, 2x1 and 2x2. Unfortunately, I don't have access to the image itself to dig deeper.
2008-10-23Mayank
 Hi
I really appreciate your quick reply.

Thanks a lot.
2008-10-23Mayank
 Hi
I concepts related to JPEG subsampling are really crystal clear but i did not understand what is H1V1, H2V2, H2V1 and H1V2, these are some common terms that are used for subsampling and i am also reading a document that are using these terms.

Also I understood the Y Cb Cr concept of image but what is AC component and DC component related to Y Cb Cr.

Kindly put more light on these two things.

Thanks in advance
 The H1V1 etc. were simplifications I made in an effort to simplify this for others. HxVy means there is x subsampling in the Horizontal direction and y subsampling in the vertical direction. So, H2V1 would be the typical 2x1 chroma subsampling we see from most digital cameras (ie. 4:2:2).

With regard to the AC and DC components, this is related to the DCT (Discrete Cosine Transform) that is used to improve the compressibility of natural image data by discarding "information" in the higher frequency image components. Have a look at my JPEG compression page for more details on this. There are a lot of great resources on the web that explain this concept in further detail.
2008-09-30Adrian
 Do you know the preference of 2x1 color subsampling over 1x2 subsampling? Is this a human visual system (HVS) thing?
And if people don't mind subsampling in one direction, why don't use the digital cameras 2x2 subsampling? As far as I know is the camera's CDD sensor's color resolution lower than the luminance resolution anyway.
 Great question -- this is something that I have not seen answered in any literature I've read. My assumption is that this originated from video, where chroma subsampling was typically done in horizontal direction (decreasing the chroma frequency). I haven't come across any details regarding the HVS that would suggest any increased ability to discern color detail in the vertical direction versus the horizontal. Note that some cheaper digicams do actually use 4:2:2 (2x2), but this is less common.
2008-07-12Musa
 Calvin,
Your knowledge on JPEG, compression and overall imaging is exceptional!!
I searched last 5 days to find solution to my problem. But I am lost.

I have online photo album site. Where users uload photos.
The resampledresized images (500X375 or 375X500) from original jpg files are losing quality.

I uploaded the same file to Flickr, the quality is far better.
I can't get rid of this problem even with HighQuality options.

g.InterpolationMode = InterpolationMode.HighQualityBicubic;
g.SmoothingMode = SmoothingMode.HighQuality;
g.PixelOffsetMode = PixelOffsetMode.HighQuality;
g.CompositingQuality = CompositingQuality.HighQuality;

Also using quality 100 when saving.

I played with irfanview and found that if I check Disable color subsampling option during save, I get same quality and file size like Flickr.

My question: How can I Disable color subsampling programatically in my code when I create the 500X375 images??

I will wait for your response. Have a great day!!
Thanks
Musa
 
2008-05-18Anton
 Hello.

Why you do not note the 1x2 subsampling in the "Various levels of YCbCr subsampling" section?

The most real-life photos have many vertical lines (walls, trees, grass, ...) where the color changes. So I measured for 10 of my photos that quadratic-averaged difference between the original image and image with subsampled chroma channels is smaller with 1x2 subsampling than with 2x1 subsampling. Than means that 1x2 subsampling is often better than 2x1.

Why digital cameras with Bayer matrices use subsampling other than 2x2 ? The image interpolation algotithm built in camera do not produce such "fast" color changes, so the 2x2 subsampling should be enough.
2008-04-29giritharan
 sir,
1) how to transfer 4.2.0 chroma sample component
and what is the value of H x V
is it H = 2; & V = 2;
2) is it possible to transfer 4Y,1Cb,1Cr of having 4 horizontal & 1 vertical data units ( continuous 4 Ycomponent )
 Hi Giritharan --

4:2:0 is what I refer to as 2x2 (subsampled by 2 in both horizontal and vertical dimensions). You can definitely have 4Y,1Cb,1Cr per MCU (4x1) -- but this is very uncommon in the case of digital photos. This type of chroma subsampling is known as 4:1:1, which is used by many DV camcorders. Hope that helps.
2008-01-15Antonio Barbra
 Ok,
If i see where there is error [reported by] jpeg snoop (in a photo) and then i calculated the offset for that error, once i know the location do i simply convert the hex values back to binary and then break that code down based on the symbols from the hufmann tree to get the actual values of the color components?
 Yes, you can do that, but that is the hard way :) I have added a feature to JPEGsnoop that will automatically report all of this decoding for you! Simply go to Options->Scan Segment->Detailed Decode... and then enter in the starting MCU coordinates. When you next Reprocess the file, all of the huffman decoding will be reported for you!
2007-10-19Sameer
 Hi Calvin,
Thanks a ton for putting together such an exhaustive amount of info on JPEG. Just a beginner question from me...

How many CbCr blocks are there for Y blocks for each of the sub-sampling formats?
So lets say that the sub-sampling format is H2V1 then there is 1 Cb and 1 Cr block for 4 Y blocks...Am i correct?
For H2V2 it should 1 Cb and 1 Cr for 2 Y blocks
I hope my question is clear and makes sense.

Thanks a lot in advance,
Sameer
 Have a look at my summary of the different chroma subsampling sequences. You'll find that for a single MCU in H2V1 mode (16x8 pixels), there are two Y components, 1 Cb and 1 Cr component. Each Y component corresponds to an 8x8 pixel region while each Cb/Cr component corresponds to the entire 16x8 pixel region.
2007-07-06Mike Lee
 After reading this article I might ask if there is a "simple" means to convert an image from one subsampling level to another? Why might I want to do this?? Well, if I take my original camera JPEG (2:1), edit it and save it in Photoshop (either 2:2 or 1:1), I can not get back a 2:1 subsampled image that I may want to work with, e.g. using jpegtran's crop and drop feature, where subsampling ratios must match. Any ideas?

Thank you
 Converting a JPEG image with one chroma subsampling to another will likely require decompression followed by recompression. This is because in one direction (2:1 -> 1:1), the color channel pixels will be duplicated, while in the other direction (1:1 -> 2:1), the color channel pixels will be averaged / filtered.

So, you could accomplish this with:
djpeg -bmp input.jpg tmp.bmp
cjpeg -sample 2x1 tmp.bmp ouput_2x1.jpg
2007-06-04vinodh
 can i know what is the scan segment
 The scan segment is the actual JPEG encoded bitstream (variable length codes from huffman coding the image) inside a JPEG JFIF file. Use a tool such as JPEGsnoop to see the different segments / markers in a JPEG file. What I was referring to as the "scan segment" is the section of the JPEG file that begins at the "Start of Scan" (SOS) marker.
2007-05-31vinodh
 I have one doubt how to find the compression ratio . i will tell what i did in the project. i converted rgb to yuv and i have take y format and applied dct and quantization and then run length coding and huffman coding . huffman coding generated bits after that what i have to do to find the compression ration thanks in advance.
 The compression ratio is easy to calculate. Take the total size of the raw bitmap and divide it by the total size of the scan data segment.

The total size (in bytes) of the raw bitmap for 24-bit images (8 bits per channel per pixel) is:
3 * DimensionX * DimensionY.

For example:
Original bitmap 3072x2048 pixels, 24-bit
Scan segment is 2372 KB

Compression Ratio = (3 * 3072 * 2048) / (2428748) : 1
Compression Ratio = (18774368) / (2428748) : 1
Compression Ratio = 7.77 : 1
2007-04-27pro
 Hey Calvin-

You are right on mark about the do_fancy_upsampling mode in IJG. I read more about the feature in the advanced features section in the IJG manual.

Apparently the feature is set to true by default. Setting it to false right before the call to jpeg_start_decompress avoids the average effect I was encountering.

So your suggestion totally fixed the problem I was encountering.

Thanks,
-pro
 Great, glad to hear you've got it solved!
2007-04-27pro
 Hey Calvin-

Thx for the reply.

There was a mistake in my previous post. The link to the compressed image pointed to a image compressed using 4:4:4 compression as you noted. I have updated the link with the image that uses 4:2:0 compression. This is the image that shows the averaging effect.

< Images: original JPEG and Cb channel extraction >

To clarify:
  • The image was decoded using IJG 6b.
  • The rgb to YCbCr conversion was done using the formula provided in the jpeg documentation: http://www.jpeg.org/public/jfif.pdf
  • Each channel in the YCbCr image was written out as a 8 bit bmp image
  • The Cb channel image was opened in irfanview and then 'snoop image magnifier' was used to look at the pixel values. This snooping tool does not do any pixel averaging during magnification. The tool can be found here:
    http://grail.cs.washington.edu/software-data/snoop/
  • I have also confirmed the averaging effect using Photoshop's eyedroper tool
Am I doing something wrong? Is it possible that IJB 6b is averaging adjacent block boundaries during 4:2:0 decompressiion as a simple deblocking mechanism?

Thanks,
-pro
 Thanks for the corrections -- makes a lot more sense now!

I think you are probably using the equivalent of the "do_fancy_upsampling" mode of the IJG decoder. In this mode, the H2V2 (4:2:0) subsampled chrominance channels get upsampled through a simple triangle filter, which will reduce some of the blocking that may appear in some images. The filtering will likely cause the MCU edge effects that you are observing.

The library call you are invoking to perform the decompression is probably operating in this mode, rather than the straightforward point replication (nearest neighbour). In the original IJG source code you should find the filtering in the routine: h2v2_fancy_upsample() in jdsamp.c.
2007-04-23pro
 Hey Calvin-
From reading your articles I was expecting jpeg decoders (I'm using IJG 6b) to simply double the chroma channels in the respective axis when decompressing a chroma subsampled jpeg image. However, when I was observing the decompressed chroma channels I found the channel values to be inconsistent with the doubling hypothesis near the block boundaries. It looks like some kind of averaging is being done at the block boundaries for the chroma channels. Is this expected? Ideally i would like to obtain the chroma channels without this kind of boundary averaging. To see what I mean here are some images for your review:

< Links to original sample image sources >

The original jpeg image is using 4:2:0 chroma sampling.

If you look at Cb or Cr channel images closely (e.g. using snoop) you should see the averaging effecting near block boundaries.

Here is a screen capture of snoop:

I was wondering if you knew what was the source of this effect.
Thanks,
-pro
 Hi there -- The example JPEG image that you linked to does not actually have any chroma subsampling (i.e. the JPEG segments indicate it is 4:4:4 / 1x1, not 4:2:0 / 2x2). You can confirm this by running JPEGsnoop or other decoders on the file.

That said, it appears that whatever image decoding program you are using (Snoop? I'm not familiar with this program, and it shouldn't be confused with my JPEGsnoop!) is not performing an accurate view of the color conversion into the color channels. In fact, your program appears to be smoothing near the boundaries of the 8x8 MCUs as you have noted.

One way that you can confirm that this is a problem with your decoder application is to do the following:
  • Open the original JPEG image in Photoshop
  • Select an isolated 8x8 pixel block of even color (i.e. no high frequency content)
  • With a method such as Photoshop's eyedropper tool, note the RGB values that exist on and near the boundary of the MCU.
You will likely find that there is no change in value throughout the MCU, even up to the boundary / edge. With color conversion, each pixel is treated independently, so this would imply that the Cb or Cr channels will also be continuous right up to the boundaries as well.

Conclusion: In fact, I am almost certain that the problem is with the way that the program is trying to display enlarged (i.e. >100%) images in its view window. It appears to be using a resampling (interpolation) process with a smoothing routine built-in, rather than using a simple nearest neighbor algorithm. For natural images, this is often more aesthetically-pleasing and is a common method -- but it does not give you an accurate view of the content of each pixel!
2007-04-17vinodh
 Thank you any have i have corrected the problem. the reason of the problem is i have declared array_u[i][j] as unsigned char it should be float then you will get the correct result .
 Great. Glad you found it!
2007-04-17vinodh
 okay i will send the code to you. you just take a look at it. it is just a simple code. It wont take much time.

Encode
for(i=0;i<240;i+=2)
{
  for(j=0;j<320;j+=2)
  {
    array_u[i][j]=(imgU1[i][j] +
                   imgU1[i][j+1] +
                   imgU1[i+1][j] +
                   imgU1[i+1][j+1]) /4.0;

    array_v[i][j]=(imgV1[i][j] +
                   imgV1[i][j+1]+
                   imgV1[i+1][j] +
                   imgV1[i+1][j+1]) /4.0;
  }
}

Decode
for(i=0;i<240;i+=2)
{
  for(j=0;j<320;j+=2)
  {
    imgU1[i][j]     = array_u[i][j];
    imgU1[i][j+1]   = array_u[i][j];
    imgU1[i+1][j]   = array_u[i][j];
    imgU1[i+1][j+1] = array_u[i][j];
  }
}


for(i=0;i<240;i+=2)
{
  for(j=0;j<320;j+=2)
  {
    imgV1[i][j]     = array_v[i][j];
    imgV1[i][j+1]   = array_v[i][j];
    imgV1[i+1][j]   = array_v[i][j];
    imgV1[i+1][j+1] = array_v[i][j];
  }
}

j1=0;
for(i=0;i<240;i++)
{
  for(j=0;j<320;j++)
  {
    imgU[j1] = imgU1[i][j];
    imgV[j1] = imgV1[i][j];
    j1++;
  }
}
 I don't see any problem with your basic algorithm. It would be difficult for me to deduce what might be wrong as it doesn't look like the problem is in the way that you are subsampling (and later expanding) the chrominance channels.

I think the best way for you to debug this is to generate some extremely simple small YUV images (that only use solid colors that fill an entire 8x8 MCU). This way you can eliminate any of the AC components and focus on the DC YUV values that you are reading from the image. Then, it should be possible to see if the chroma subsampling before & after calculations appear correct.
2007-04-16vinodh
 Ya initially i converted rgb to yuv and i converted yuv back to rgb, then only i used chroma formats. once you convert into 4:2:0 . size of the matrix reduces by half and four values form a single value. Now tell me how can we convert this single value again to four values. this is my doubt
 The single chrominance value should be copied out four times to neighboring pixels, so it is surprising that it isn't working for you. To keep things simple, are the DC components working? For example, if you run your 4:2:0 subsampled YUV file (assuming it is a JPEG) in JPEGsnoop, can you see the image correctly?
2007-04-16vinodh
 i have converted rgb image to yuv and i used the chroma format 4:2:0 , by taking the average of the adjacent pixels. i have to convert back again to rgb image using these average values. once i tried to copy the average value to the four adjacent location. i am not gettting required rgb image. Please help me in these aspects. Thanks in adavance
 What you are doing sounds correct (so long as you're only doing this for the U & V channels, not Y). So, I am not sure why your output may not be correct. What does it look like / how is it wrong? Have you tested your RGB -> YUV -> RGB conversions first (with no chroma subsampling)?

Double-check that the sequence you are using to encode and decode the subsampled MCUs is as is shown on my JPEG decoder page under Chroma Subsampling.
2006-11-30Doug Pardee
 This sentence sounds backwards to me: "By choosing RGB values for Red (RGB=255,0,0) and Blue (RGB=0,0,97) that have the same converted Y (luminance component), the resulting stripes will disappear."

Should that have been "Red (RGB=97,0,0) and Blue (RGB=0,0,255)"?
 Yes, that was a typo. Thanks for picking up on it!

 


Leave a comment or suggestion for this page:

(Never Shown - Optional)