Return to Digital Photography Articles
JPEG Lossless Rotation, Lossless Crop
With a photographer's desire to preserve the digital negatives as much as possible, there is a strong need to be able to rotate photos without reducing the quality in the process. The rotation might be done for a variety of reasons, including: correctly orienting a vertical portrait shot when the camera doesn't have a built-in orientation sensor, rotation for stylistic reasons, true rotation to match the EXIF orientation flag, etc. Similarly, one may want to crop a photo without degrading the quality of the uncropped portions of the image.
As the "loss" (more properly: additive compression error) is completely dictated by the details of how JPEG compression works, it is important to understand some of the basic concepts. Read my summary on JPEG compression.
Standard rotate and crop error / loss
Normal Rotate Error
To introduce the idea of incremental error, let's look at an example. Let's say a user opens up IMAGE_A.JPG in an image editor. The user rotates the content clockwise by 90 degrees and then resaves it as IMAGE_B.JPG. Now, the user opens IMAGE_B.JPG, rotates the content counter-clockwise by 90 degrees and then resaves it as IMAGE_C.JPG. If these overall rotate operations were truly "lossless", then we'd expect that IMAGE_A.JPG would be identical to IMAGE_C.JPG. But we will often find this not to be the case! Not only does the image content change, but sometimes even the metadata (information from the digital camera) is lost in the process.
The image on the right shows an example of the difference between IMAGE_A.JPG and IMAGE_C.JPG when using Photoshop CS with the "Rotate Canvas" command and saving each time at "Maximum" quality setting (10). The two files were then overlayed with a "Difference" layer mode, and then a Levels adjustment was made to remap the white-point to a value of 3 or higher (to make it visible). The histogram showed that the maximum RGB difference between the original and the final version is 13 / 255, but the mean difference was only about 1.10 / 255 (standard deviation 0.82). Therefore, it would be very hard to visually observe the differences (additional error) due to the rotate and resave operation (at quality setting 10), but they obviously exist. Obviously, the selection of the quality setting has a significant impact on the degree of cumulative error that we are introducing.
How normal rotate and save operations cause incremental error
Now let's revisit the original rotation scenario. When IMAGE_A.JPG is opened in Photoshop, the software must perform all of the steps listed above (but in reverse order) to decode the JPEG image. No data is "lost" in this decompression step besides a small amount in the color space conversion (as we are converting an 8-bit value with linear equations to another 8-bit value).
A 90 degree rotate is performed, which can be done in the RGB spatial representation without problem. Now, the user decides to save the file as IMAGE_B.JPG. The user will be prompted with a Save As... dialogue box which asks for a quality setting (ie. 1-12, 1 being the poorest and 12 being the absolute maximum). Photoshop will then have to perform the DCT transform again on the image contents and then re-quantize the data. It is this re-quantizing of image data that causes the majority of the incremental error. As all of the 8x8 pixel blocks have changed content (by the rotate), the DCT result will be different, causing the re-quantization to come up with different results, again suffering the rounding loss.
Lossless rotate and lossless crop
It is first important to understand what is meant when people refer to lossless operations (lossless rotate, lossless crop, etc.). Generally, the idea is that an operation is performed on a portion of an image without incurring any additional recompression error. As others (see Doug Kerr's JPEG PDF article) have carefully pointed out, a better name for these operations would be "reversible" and not "lossless".
Programs that offer the ability to perform lossless rotation generally act on the file, rather than on the image content. This is seen in several programs (such as IrfanView) which actually place the "lossless" operations in a seperate menu tree, outside of the main editing flow. Furthermore, these operations will typically resave the file automatically, rather than ask you for any quality settings, etc. As will be described later, the reason for this is that the quantization table must be preserved exactly as it was before, and any quality setting changes would destroy this prerequisite.
For all lossless opertaions, note that the dimensions of the image must be integer multiples of the minimum coded unit size (MCU, see below). In each case, the image editor software must decompress the JPEG data to the point where the DCT coefficients become available. Then, an operation is performed on the resulting coefficients and the compression is performed again. Since no dequantization or requantization is performed, there is no additional rounding error that would contribute to any compression "loss".
Types of reversible / "lossless" operations:
- Lossless rotate - As the DCT basis functions are symmetric across the X & Y axes, it is a simple matter to remap the coefficients in the matrix from one direction into the other. Rotation can be performed in increments of 90 degrees (ie. 90, 180, 270 degrees).
- Lossless crop - For an image to be cropped reversibly, the program simply tosses out the entries that correspond to image blocks that are to be cropped off. As all other blocks remain untouched, they are recoded identically and reversibly.
- Lossless flip across an axis - Just like the lossless rotate, the flip operation (which can be done across eitherr the X or Y axis) is essentially a remapping of matrix coefficients.
- Lossless extension - Just like lossless crop, it is also possible to add additional area to the image (eg. increase "Canvas Size") by a multiple of the MCU. The original blocks will be left untouched.
Obviously, only the rotate and flip operations are "reversible". The crop and extension operations are dubbed "lossless" as the primary region of the image content has not suffered image recompression / degradation.
Test to see if a program does lossless rotation
Read my article on how to test for lossless rotation in your software.
For a starting point list of applications that support lossless rotation operations, please check out the JPEG Club Lossless Rotation Applications. However, please note that this is not a complete list -- it only includes those applications that were known to reuse the IJG code. There are quite a number of other applications that also provide lossless rotation.
For example, there has been widespread confusion about whether or not Canon's ZoomBrowser EX supports lossless rotation. I have tested Canon ZoomBrowser EX 5.7 and confirmed that it does in fact perform lossless rotation (if you set the Rotate Actual Images option, otherwise it's a virtual rotation).
Minimum Coded Unit / Image Block Size
To make the image encode and decode more manageable, the JPEG compression algorithm subdivides the original image into 8x8 pixel blocks. This is known as the Minimum Coded Unit (MCU). The alignment of these 8x8 boundaries in the image is of great importance to the resulting compression. As the original RGB image content is converted into the YCbCr color space, the algorithm is able to perform a different type of compression on each component (the components being luminance and the two chrominance components). As described on the JPEG compression page, the chrominance components can be compressed more heavily (and result in greater error) than the luminance component due to the nature of our eye's sensitivities. This is the basis for the chroma subsampling, which effectively reduces the resolution of the chrominance component.
By subsampling the chrominance component, the minimum coded unit becomes 16x8 or 16x16 pixels instead of the 8x8 pixel units. 16x8 corresponds to 2:1 chroma subsampling, while 16x16 corresponds to 2:2 chroma subsampling.
Lossless Rotation of odd-sized images or Partial MCUs
One cannot truly perform a lossless rotation on a JPEG whose dimensions are not an integer multiple of the MCU size. In other words, a photo that is 501 x 483 pixels cannot be properly rotated or flipped losslessly!
Please read my article that explains partial MCU and lossless rotate / lossless flip operations.
Photoshop CS: No lossless operations possible!
Note that it is currently impossible to perform lossless rotate or lossless crop in Photoshop, if the original was a digital camera image.
EXIF Orientation and Virtual Rotation
To add to the confusion, some products refer to modification of the EXIF Orientation flag as lossless rotation or virtual rotation. The EXIF metadata attached to a digital camera photo includes a field named "Orientation". This field describes the orientation of the camera when the shot was taken, but only if the camera contains an orientation sensor. Most digital SLRs and some advanced point and shoot cameras contain such a sensor.
By including a field in the metadata that describes the intended orientation of the photo, the camera is saved from the work of having to rotate the image immediately after the shot. Essentially, it marks the photo as "requiring a rotate" by anyone wishing to view the file. If the camera were to automatically rotate the image, a large amount of processing would be required in-camera, leading to far slower frame rates.
But, by saving the camera any of the trouble of rotating the image, the responsibility is left up to every image viewer that wants to open up the image later. Every time an image editor or browser decides to open the image, the software must rotate the original before displaying. This leads to a performance hit in any application that opens the image with the orientation flag set. A drawback to this method is that not all programs support this EXIF orientation flag. In such an application, the image will incorrectly appear un-rotated.
Some programs offer the ability to virtually rotate an image. These programs either modify the EXIF Orientation flag, or just rotate a thumbnail (in the case of a photo catalog program), without touching the original image.
For more details, please see my article on EXIF Orientation and JPEG Rotation.