JPEG Rotation and EXIF Information

one of the main reasons to get a new iPhone was to finally get one that has a non-ridiculous camera. the original iPhone camera was laughably bad even when the iPhone was released, and after three years, it had become something you'd expect as a free giveaway in a cereal box, not in an expensive smartphone. so i gave in and got a iPhone 4, and the camera indeed is quite nice. but now that i make decent pictures again, i recognized that they are not properly rotated anymore when you send them by email. it turns out his is a nice example of compatibility issues, and how they materialize.

first the explanation: older iPhones (or maybe it's the fault of iOS 4? i don't know and it does not really matter) actually rotated the JPEG before attaching it to the message, and they did so based on the accelerometer information embedded in the image that specified the orientation of the camera when the picture was taken (the actual image is always stored in the same landscape orientation). the information from the accelerometer is stored in the orientation field of the Exchangeable Image File Format (EXIF) metadata that is embedded in the JPEG file. the orientation field can have one of eight (yes: eight) values. it turns out that what happens now is that the iPhone does not rotate the image anymore, instead it expects the recipient to understand and support the orientation information and rotate the image accordingly. which means that depending on the mail program the recipient is using, the images may be correctly rotated, or not.

in terms of compatibility, this means that the approach has shifted from supporting all JPEG-aware clients (rotating done by the client) to only supporting EXIF-aware clients (if you don't know when to rotate, it's your problem). i have no statistics about EXIF support in mail clients, but i am sure apple's clients supports EXIF orientation. thunderbird doesn't support it, though, whereas GMail supports it, so there is a mix of support in major clients. being friendly to all JPEG clients and not just to EXIF-aware clients would be nice, so is there anything that would be lost by sticking with the previous aproach of client-side rotation?

with EXIF orientation, the image contains the information of the device orientation in relation to the actual JPEG image. this means that based on this information, it is for example possible to create statisticson how many pictures have been taken in which orientation. this information is lost when the client is rotating, because it then has to remove the orientation information to avoid that EXIF-aware clients are rotating the image a second time. my personal opinion is that EXIF is not actually a part of JPEG, and that it would be better to rotate client-side and be more robust. but i am pretty sure that apple has made sure that all its products are EXIF-aware, and thus in apple-land, everything is working well, and apple never cares a whole lot about how well things work outside of apple-land...

the new rotation handling has some other side-effects, too. for example, flickr does not support EXIF in email uploads, so users need to manually rotate after uploading. in this image, i have done that, but flickr in its infinite wisdom has decided to not rotate the original size image when rotating (don't get confused, it has been rotated now, read on), which means that the scaled down images are rotated, but the original upload isn't. this means you end up with differently oriented pictures in the different sizes made available by flickr, which is not very useful. but as it turns out, if you use flickr's picnik editor instead of the rotate command, and then rotate in picnik, it actually rotates all images (including the original upload), so until flickr fixes the rotate command (or even better, supports EXIF rotation when images are uploaded), it's a good idea to always rotate using picnik.

Comments

[yiming]

JPEG rotation isn't lossless unless a specific set of conditions are met ( http://www.faqs.org/faqs/jpeg%2Dfaq/part1/index.html ). It's kind of a hack that JPEGs could be rotated losslessly at all. EXIF seems a much cleaner and less destructive way to do rotations for compressed images.

[dret]

i think virtually all cameras fit the constraints on images sizes to allow lossless JPEG rotation, and EXIF is just not part of JPEG, it's an independent add-on. in an ideal world, cameras would store the sensor dump in raw as retrieved from the sensor, but JPEGs always would be rotated internally. EXIF then could be used to capture the fact that the image had been rotated for JPEG encoding. i am wondering whether there is any camera out there that does it like this, but i guess EXIF does not even have the appropriate fields to support this kind of scenario.

practically speaking, i am simply annoyed that after three years of working just fine, i now have people telling me that i am sending them pictures in the wrong orientation. technically speaking i may have the moral upper hand, but it makes little sense for me to tell them that they should dump their email clients and switch to one that works as apple thinks it should work.

[Chris]

I wrote a free open source JPEG auto rotator windows explorer context menu extension that rotates the image according to the EXIF orientation property. Also a command line version. I was motivated to write this after trying to show slideshows on various devices (TVs, digital picture frames) that didn't respect the orientation property and since other windows explorer context menu extensions that did this had some limitations

https://github.com/chrdavis/JPEGAutoRotator