java - ImageIO reading slightly different RGB values than other methods

Question

I've found that I'm getting different RGB when using Java (& actually paint.NET) than I am using ImageMagick, Gimp, Python, and Octave. The last 4 all agreeing with eachother and so I'm assuming to be correct.

For these examples, I'm using this test image: http://farm3.static.flickr.com/2811/9177301733_9836174725_o.jpg

Testing pixel x=4144 y=2768

               R    G    B
Java        = (125, 107, 69)
Paint.NET   = (125, 107, 69)
ImageMagick = (128, 106, 67)
Python      = (128, 106, 67)
Octave      = (128, 106, 67)
Gimp        = (128, 106, 67)

What gives?

Here's a quick test using imagemagick:

convert image.jpg -crop 1x1+4144+2768 -depth 8 txt:

output:

# ImageMagick pixel enumeration: 1,1,65535,srgb
0,0: (32896,27242,17219)  #806A43  srgb(128,106,67)

Here's some java and python code that also demonstrates the problem:

import org.apache.commons.io.FileUtils;
import org.junit.Test;

import javax.imageio.ImageIO;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.net.URISyntaxException;
import java.net.URL;

public class ImageIOTest {
    @Test
    public void can_read_file() throws IOException, InterruptedException, URISyntaxException {
        File tempFile = File.createTempFile("image", "jpg");
        FileUtils.copyURLToFile(new URL("http://farm3.static.flickr.com/2811/9177301733_9836174725_o.jpg"), tempFile);

        BufferedImage image = ImageIO.read(tempFile);

        int javaRGB = image.getRGB(4144, 2768);
        int javaRed = (javaRGB >> 16) & 0xFF;
        int javaGreen = (javaRGB >> 8) & 0xFF;
        int javaBlue = (javaRGB >> 0) & 0xFF;
        System.out.printf("rgb: (%d, %d, %d)", javaRed, javaGreen, javaBlue);
    }
}

And here is the corresponding python script:

from PIL import Image
import sys, urllib, cStringIO

file = cStringIO.StringIO(urllib.urlopen("http://farm3.static.flickr.com/2811/9177301733_9836174725_o.jpg").read())

im = Image.open(file)
pix = im.load()
print pix[4144, 2768]

I've tried using this 12monkeys library in the hope that that would fix it but no dice. Any other ideas how I can extract correct RGB values using java? Surely I'm not the first person to have this problem!

Update

I tried getRaster().getSample() but got the same invalid result: System.out.println(raster.getSample(4144, 2768, 0)+","+ raster.getSample(4144, 2768, 1)+","+ raster.getSample(4144, 2768, 2)); output: 125,107,69

More info

Here is some output that shows what RGB values are decoded by three different tools for the first 9 (3x3 square of) pixels in the top left of the image. As you can see, Python and ImageMagick are in unison. Java sometimes matches. I've put an X where java disagrees...:

Tool          [x, y] = (R , G , B )
ImageIO     : [0, 0] = (86, 90, 93)
Python      : [0, 0] = (86, 90, 93)
ImageMagick : [0, 0] = (86, 90, 93)

ImageIO     : [1, 0] = (86, 90, 93)
Python      : [1, 0] = (86, 90, 93)
ImageMagick : [1, 0] = (86, 90, 93)

ImageIO     : [2, 0] = (90, 91, 95) X
Python      : [2, 0] = (88, 92, 95)
ImageMagick : [2, 0] = (88, 92, 95)

ImageIO     : [0, 1] = (85, 93, 95)
Python      : [0, 1] = (85, 93, 95)
ImageMagick : [0, 1] = (85, 93, 95)

ImageIO     : [1, 1] = (85, 93, 95) X
Python      : [1, 1] = (87, 92, 95)
ImageMagick : [1, 1] = (87, 92, 95)

ImageIO     : [2, 1] = (87, 92, 95)
Python      : [2, 1] = (87, 92, 95)
ImageMagick : [2, 1] = (87, 92, 95)

ImageIO     : [0, 2] = (83, 93, 94)
Python      : [0, 2] = (83, 93, 94)
ImageMagick : [0, 2] = (83, 93, 94)

ImageIO     : [1, 2] = (83, 93, 94) X
Python      : [1, 2] = (84, 92, 94)
ImageMagick : [1, 2] = (84, 92, 94)

ImageIO     : [2, 2] = (83, 91, 93)
Python      : [2, 2] = (83, 91, 93)
ImageMagick : [2, 2] = (83, 91, 93)

Why is Java giving different values for some pixels? Alternatively, is there another (fast) way to generate correct values using native Java code?

UPDATE 2016-09-26:

I committed my code that demonstrates this problem and pushed it to github (imageio-test) so that I could easily test it out on different machines. It turns out that Java was consistent across both OSX and Ubuntu Linux, but it was Python, ImageMagick and Octave that were inconsistent. In other words, on the Linux box, all tools agree with each other, and therefore, I'm now thinking that java was right all along, and it's the other tools that are giving incorrect results on OSX! I still don't really understand why and I haven't got any concrete proof as to which values are the correct ones but I'm getting somewhere...

Answer 1

Actually, I'd like turn the problem around, and say I'm surprised that so many different platforms and tools actually produce the same values. :-)

JPEG is lossy

First of all, JPEG is a lossy image compression method. This means that reproducing the exact data of the original is not possible. Or, if you like, several different pixel values may all be "correct" in some way.

The technical reasons why not all JPEG software produce the exact same values from the same source file, is typically different rounding/clamping of values, or integer approximations of floating point operations for better performance. Other variations may stem from different interpolation algorithms applied to restore the subsampled chroma values, for example (ie. a smoother image may look more pleasing to the eye, but isn't necessarily more correct).

Another excellent answer to a similar question states that "The JPEG standard does not require that decoder implementations produce bit-for-bit identical output images", and quotes the Wikipedia JPEG entry:

[...] precision requirements for the decoding [...]; the output from the reference algorithm must not exceed:

• a maximum of one bit of difference for each pixel component
• low mean square error over each 8×8-pixel block
• very low mean error over each 8×8-pixel block
• very low mean square error over the whole image
• extremely low mean error over the whole image

(Note that the above talks about the reference implementation only).

However, with some luck, it seems that all of your software/tools actually end up using (some version of) libjpeg. Because they all use libjpeg, the source of the differences you see is most likely not related to the JPEG decoding.

Color Spaces

Even if all your software converts the JPEG file to a representation using RGB values, there could be differences in the color space they use for this representation.

It does seem that all of the software you are using actually displays the RGB values in the sRGB color space. This is probably the most standard and widely used color space used in mainstream computing, so that is no surprise after all. As the color space is always sRGB, the source of the differences you see is most likely not the color space.

ICC profiles and color matching

The next possible source of color differences, is that color matching (as done by a Color Matching Module, CMM or Color Management System, CMS) is not an 100% exact science (see for example this document on black point compensation or read some of the more technical posts from the Little CMS blog).

Most likely the software running on Mac OS X are using Apple's CMM, while Java is using Little CMS always (from OpenJDK 7 or Oracle JDK/JRE 8), and most software on the Linux platform will likely also use the open source Little CMS (according to the Little CMS home page, "You can find Little CMS in most Linux distributions"). Software on Windows will likely deviate slightly as well (I haven't been able to verify if Paint.Net uses Little CMS, Windows' built in CMM or something else). And of course, using Adobe's CMM (ie. Photoshop) will likely deviate as well.

Again, with some luck, a lot of the software you tested uses the same CMM or CMS engine, Little CMS, so again you will have a lot of equal results. But it seems that some of the software you tested uses different CMMs, and is a probable source of the slight color differences.

In summary

The different pixel values you see are all "correct". The differences stem from different implementations or approximations of algorithms in software, but that does not necessarily mean that one value is correct and the others are wrong.

PS: If you need to reproduce the exact same values across multiple platforms, use the same tool stack/same algorithms on all platforms.