JPEG Compression, Quality and File Size

How does JPEG compression work?

When a JPEG file is opened in an image editor, a large number of steps must be performed before the raw image (one RGB triplet per pixel) can be displayed or edited. It is easier to understand the process by looking at the reverse process — ie. what happens when one generates a JPEG file (ie. save) from the raw image data. In summary, the JPEG algorithm involves the following stages:

• Color Space Conversion - The image first undergoes a color space conversion where it is remapped from RGB (Red Green Blue) triplets into YCbCr (Luminance, Chrominance Blue, Chrominance Red) triplets. This color space conversion assists the use of different quantization tables (one for luminance, the other for chrominance).
• Segmentation into Blocks - The raw image data is chopped into 8x8 pixel blocks (these blocks are the Minimum Coded Unit). This means that the JPEG compression algorithm depends heavily on the position and alignment of these boundaries.
• Discrete Cosine Transformation (DCT) - The image is transformed from a spatial domain representation to a frequency domain representation. This is perhaps the most confusing of all steps in the process and hardest to explain. Basically, the contents of the image are converted into a mathematical representation that is essentially a sum of wave (sinusoidal) patterns. For example, the binary sequence 101010 can be represented by a wave that repeats every two pixels. The sequence 1100110011 can be represented by a wave that repeats every four pixels. Similarly, the sequence 1001101011 can be represented by the sum of several simpler waves. Now imagine that this mapping to wave equations (known as the DCT basis functions) is done in both the X and Y directions.
• Quantization - Given the resulting wave equations from the DCT step, they are sorted in order of low-frequency components (changes that occur over a longer distance across the image block) to high-frequency components (changes that might occur every pixel). Is it widely known that humans are more critical of errors in the low-frequency information than the high-frequency information. The JPEG algorithm discards many of these high-frequency (noise-like) details and preserves the slowly-changing image information. This is done by dividing all equation coefficients by a corresponding value in a quantization table and then rounding the result to the nearest integer. Components that either had a small coefficient or a large divisor in the quantiziation table will likely round to zero. The lower the quality setting, the greater the divisor, causing a greater chance of a zero result. On the converse, the highest quality setting would have quantization table values of all 1's, meaning the all of the original DCT data is preserved.
  
  An important point to realize here is that the quantization table used for this step differs between nearly all digital cameras and software packages. Since this is the most significant contributor of compression or recompression "error", one is almost always going to suffer image degradation in resaving from different compressors / sources. Camera manufacturers independently choose an arbitrary "image quality" name (or level) to assign to the 64-value quantization matrix that they devise, and so the names cannot be compared between makes or even models by the same manufacturer (i.e. Canon's "Fine" vs Nikon's "Fine").
  
  Please see my article on JPEG Quantization Tables for the actual tables used in the Canon, Nikon, Sigma, Photoshop CS2 and IrfanView digital photos.
• Zigzag Scan - The resulting matrix after quantization will contain many zeros. The lower the quality setting, the more zeros will exist in the matrix. By re-ordering the matrix from the top-left corner into a 64-element vector in a zig-zag pattern, the matrix is essentially sorted from low-frequency components to high-frequency components. As the high-frequency components are the most likley to round to zero, one will typically end up with a run of zeros at the end of the 64-entry vector. This is important for the next step.
• DPCM on DC component - On a block-by-block basis, the difference in the average value (across the entire block, the DC component) is encoded as a change from the previous block's value. This is known as Differential Pulse Code Modulation.
• RLE on AC components - On the individual entries in the 64-element vector (the AC components), a Run Length Encoding stores each value along with the number of zeros preceeding it. As the 1x64 vector contains a lot of zeros, it is more efficient to save the non-zero values and then count the number of zeros between these non-zero values. The RLE stores a skip and a value, where skip is the number of zeros before this component, and the value is the next non-zero component.
• Entropy Coding / Huffman Coding - A dictionary is created which represents commonly-used strings of values with a shorter code. More common strings / patterns use shorter codes (encoded in only a few bits), while less frequently used strings use longer codes. So long as the dictionary (Huffman table) is stored in the file, it is an easy matter to lookup the encoded bit string to recover the original values. See my JPEG Huffman Coding tutorial.

Examine your JPEG Files!

I have written a free Windows utility that examines and displays all of the details described above in your JPEG files.

Download JPEGsnoop here.

Where does the error come from?

By far the biggest contributor to the error (ie. file size savings) in the JPEG algorithm is the quantization step. This is also the step that allows tuning by the user. A user may choose to have a slightly smaller file while preserving much of the original (ie. high quality, or low compression ratio), or a much smaller file size with less accuracy in matching the original (ie. low quality, or high compression ratio). The tuning is simply done by selecting the scaling factor to use with the quantization table.

The act of rounding the coefficients to the nearest integer results in a loss of image information (or more specifically, adds to the error). With larger quality scaling factors (ie. low image quality setting or high numbers in the quantization table), the amount of information that is truncated or discarded becomes significant. It is this stage (when combined with the Run Length Encoding that compresses the zeros) that allows for significant compression capabilities.

There are other contributors to the compression error, such as the color space conversions, but the quantization step is the most important.

Please see my results in the JPEG Quantization Table article for a more accurate comparison between software packages and their quality settings.

JPEG Chroma Subsampling

In order to further improve JPEG compression rates, chroma subsampling is used to reduce the amount of image information to compress. Please refer to my article on Chroma Subsampling for more information on the 2x1 and 2x2 subsampling typically used in digital cameras and image editors such as Photoshop.

Breakthrough in JPEG compression?

Up until now, it has been widely assumed that the JPEG image compression is about as good as it gets as far as compression rates are concerned (unless one uses fractal compression, etc.). Compressing the JPEG files again by Zip or other generic compression programs typically offers no further improvement in size (and often does the reverse, increasing the size!).

As documented in a whitepaper written by the authors of StuffIt (Allume Systems, formerly Alladin Systems), they have apparently developed software that will further compress JPEG files by up to an additional 30%! Considering how many years the highly-compressed JPEG algorithm has been around, it is surprising to see any new developments that offer this degree of increased compression. Note that the "Stuffit Image Format" (SIF) uses lossless compression of the lossy-compressed original JPEG image. Therefore, there is no further image quality reduction in this Stuffit additional compression.

On Slashdot.org, there have been many theories as to how this additional compression could be achieved, but it seems that many feel that it must be through either replacement of the Huffman coding portion (and using arithmetic coding instead) or alternatives to the zig-zag reordering scan. It seems that the consensus is that SIF uses an implementation of arithmetic coding.

On first glance, this would seem to potentially revolutionize the photo industry. Imagine how this could affect online image hosts, or personal archiving needs. Saving 30% of the file size is a significant improvement. Unfortunately, a few significant problems are immediately apparent, possibly killing the adoption of this format:

• Proprietary Standard - I cannot see this format taking off simply because a single company owns the format. Would you trust your entire photo collection to a single company's utility? The company can charge what it likes, has no guarantees about the company's future, etc. At least Adobe tried to do things right by releasing their DNG (Digital Negative format) specification to the open community, allowing many other developers to back the format. Allume / Stuffit sees this as a potential financial jackpot.
• Processor Intensive / Slow - Unlike the methods used in the standard JPEG file compression scheme, the SIF method is apparently extremely slow. As tested by ACT (Archive Comparison Test website), a 1.8 GHz Pentium computer took nearly 8 seconds to compress or decompress a 3 megapixel file. While this is less of an issue for those wishing to archive photos to CD, for example, it is obvious that this would prevent the algorithm from ever being supported in most embedded applications (including within a digital camera).

Resaving and workflow

When resaving after making changes, I strive to preserve the quality of the original as much as possible and not lose additional detail to compression round-off error. Therefore, one should keep in mind a few suggestions about resaving:

Original	Save as...	Notes
TIFF	TIFF	If the original was uncompressed, then it makes sense to resave it as uncompressed
BMP	BMP	If the original was uncompressed, then it makes sense to resave it as uncompressed
JPG	TIFF or BMP	Best approach: Allows the best preservation of detail by saving in a lossless format. Unfortunately, this approach complicates things as most catalog programs don't handle the change of file type very well (as it changes the filename).
JPG	JPG	Alternate approach: While not as good as the previous approach that saved it in a lossless format, this can be adequate if the compression algorithm is the same (ie. same quantization tables & quality settings). If this is not possible, then resaving with a quality setting that is high enough (ie. striving for less compression than the original) might be the only choice.

If one intends to edit a JPEG file and resave it as a JPEG, the issue of recompression error should be given consideration. If a little additional error is not going to be of much concern (especially if it is nearly invisible), then resaving to match file size might be an adequate solution. If, however, the goal is to preserve the original image's detail as much as possible, then one has to take a closer look at the way the files are saved.

Option 1 - Resaving with no recompression error

All of the software applications that advertise "lossless" operations (such as lossless rotation) will resave the file with no additional recompression error. The only way that this can work is if the settings used in the compression algorithm match the settings of the original, identically. Any differences in the settings (more specifically, the quantization table and the quality setting / factor) will cause additional compression error.

Unfortunately, it is very difficult (as a user) to determine what these settings were, let alone have any control over them (besides the quality factor). Besides cjpeg, I haven't seen any other programs that actually allow you to configure the quantization tables yourself.

In an attempt to identify whether or not this option is even possible, I have compared the quantization tables of my digital camera to a couple imaging applications.

Fortunately, if one is resaving an image in the application that originally created the image (eg. saved an image in Photoshop, re-opened it, edited it and then resaved it in Photoshop), one can almost achieve this by simply resaving with the same quality settings as were used the previous time. As the quantization table is hardcoded, the user must ensure that the quality setting matches (not higher or lower than) the original. If one forgot what settings were used in the original, it is possible to make an educated guess by performing a couple test saves to compare by file size (across quality settings) to get a very rough idea.

Option 2 - Resaving with minimal recompression error

If one is resaving a photo with a different program than originally created the original (eg. Photoshop CS resaving an edited version of a photo direct from a digital camera), it is not possible to resave without some additional "loss" (recompression error) . The problem here is that the quantization tables and quality settings either are not known or they cannot be set. This is the most likely scenario for users editing their digital photos.

In this scenario, the goal is no longer "lossless resaving" but minimizing the additional recompression error that will be introduced. Making a very rough assumption, one can get an equivalent level of detail by resaving with settings that uses similar quantization tables. There are many reasons why this ends up being a rough assumption, but it should give a close approximation to the same level of detail.

Compression Quality and File Size

The following details the effect of JPEG quality on file size from several popular image editing programs. Unfortunately, each graphics program tends to use its own compression quality scale and quantization tables, and therefore, one can't simply transfer quality settings from one application to another.

As described in the section above, if one cannot guarantee lossless resaving (because of differences in the quantization tables), then it is worth looking at the quantization table comparison for a guideline.

Knowing what quality level is roughly equivalent to the original image helps in determining an appropriate quality level for resaving. Ideally, one doesn't want to resave at a lower quality level (and therefore lose image detail / quality), and on the other side, one shouldn't save at a higher quality setting as it simply wastes space and can in fact introduce extra recompression noise!

Digital Photo Source Characteristics

The source file for comparison purposes is a JPEG image shot with a Canon 10D digital SLR camera, recording a 6 megapixel image (3072x2048 pixels) at ISO 400 and super-fine quality. The source file size is 2280 KB.

Photoshop CS - JPEG Compression

For more detailed information, please see the article: Photoshop Save As vs Save for Web.

Notes:

• Photoshop CS2 allows a range of quality settings in the Save As dialog box from 0..12 in integer increments.
• Photoshop CS2 allows a range of quality settings in the Save For Web dialog box from 0..100 in integer increments.
• A JPEG quality setting of around 11 achieved a similar file size to what was originally produced by the Canon 10D digital SLR camera (in super-fine mode).
• Photoshop CS has three modes of saving JPEGs: Baseline, Baseline Optimized and Progressive. The difference in file size between these three modes (progressive was set to 3-scan) was in the order of about 20 KB for a 1 MB file. Of course this is dependent upon the content of the image, but it demonstrates the rough order of magnitude expected in the different modes.
• An ICC profile was attached to the image, but it is simply the default sRGB, which is relatively insignificant (~ 4 KB).

Photoshop CS and Chroma Sub-sampling

Although it is not advertised, I have determined that Photoshop CS uses chroma subsampling only in certain quality level settings.

Photoshop does not allow the user to select whether or not Chroma Subsampling is used in the JPEG compression. Instead, 2x2 subsampling is used for all Save As at Quality 6 and below, while it is disabled (ie. 1x1 subsampling) for Save As at Quality 7 and higher. Similarly, it is used for all Save For Web operations at Quality 50 and below, while it is not used in Save For Web at Quality 51 and above.

Irfanview - JPEG Compression

Notes:

• Irfanview allows a range of quality settings from 0..100 in integer increments.
• It also allows one to select whether or not Chroma Subsampling is used in the JPEG compression.
• With Chroma Subsampling disabled, it appears that a quality setting of 96 achieves comparable file size to the original.
• With Chroma Subsampling enabled, a quality setting of around 98-99 creates a comparable file size to the original. However, it should be noted that the digital camera itself is using chroma subsampling (2x1), so that this figure is not particularly useful. In other words, if the original source used chroma subsampling, then there is no point in resaving it without chroma subsampling — the additional CbCr color information is already gone. In the example image I have used for the above analysis, chroma subsampling offered approximately 25% savings in file size over the same JPEG compression without.

Miscellaneous Topics

Beware of Photoshop Save For Web

Although Photoshop's Save for the Web dialog seems great as it lets one interactively set the image dimension, palette depth and compression quality, it performs one potentially disastrous effect: the removal of EXIF metadata!

You will find that the file sizes after using "Save for the Web" will be smaller than if you had simply chosen "Save As..." with equal compression settings. The difference is in the lost image metadata (time / date / aperture / shutter speed / etc).

The only time that it is worth using "Save for the Web" is when one is creating web graphics (where you are optimizing for speed) or deliberately wants to eliminate the metadata (eg. for photo galleries, privacy, etc.).

offset $2a9e SOF0 (baseline DCT Huffman) (length 17)
  sample precision 8
  width 2048, height 3072  components 3
    id 1 horizontal sampling 1, vertical sampling 2, quantization table 0
    id 2 horizontal sampling 1, vertical sampling 1, quantization table 1
    id 3 horizontal sampling 1, vertical sampling 1, quantization table 1