Colours in DNGs exported are not the same as CR2

[ilia3101]

I am not able to test, I do not have camera raw (ok I'll try out in rawtherapee actually), but according to Danne the white balance and colours look different in DNGs exported from MLV App to what you get out of CR2 photos. It should not be the case.

Hi guys. I uploaded a cr2 file and a dng(anamorphic). I wanted to compare image quality etc and I notice colors differ quite a lot from the cr2 file. Plan on investigating further. If you want to see try set white balance the same in adobe camera raw and you´ll notice how different they look:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/testfiles.zip

Now it could be the camera calibration tag missing in dng. Still to be tested. But maybe some matrices or something else is affecting this too?

calibration tag is not to be found if not going through adobe dng converter with a cr2 file. I searched high and low if this factory sensor tag could be found somehow but still a mystery. It's quite interesting though. I use it in MLP when developing cr2 dualiso into dng files. I bake the camera calibration tag into the asShotNeutral tag and get the exact white balance that is when shooting cr2 files.

I will be honest, I'm not sure exactly what this all means, I do not know the DNG tag terminology. Maybe it means the matrices are not added, or it is a simple white balance difference that can be completely corrected by moving the sliders, either way it should be fixed !?

@bouncyball-git Do you know what the issue could be?

Edit: I can confirm the issue: in darktable with matched white balance sliders the pics look different, something is going on.

[dannephoto]

Some links. Check from around #2840:
https://www.magiclantern.fm/forum/index.php?topic=7139.msg172921#msg172921

Bottom line. It's a secret.

Here's some more:
https://www.magiclantern.fm/forum/index.php?topic=21370.msg195122#msg195122

One thing to do is to shoot a cr2 with two completely different eos m cameras with the same settings then look for this information and see if it differs.
exiftool -u INPUT.CR2 | grep 'Camera Color Calibration'

Also. DNG spec:
https://www.google.com/url?sa=t&source=web&rct=j&url=https://www.adobe.com/content/dam/acom/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf&ved=2ahUKEwid-P644dHkAhXJlosKHYXfDIIQFjAAegQIBxAC&usg=AOvVaw3GxojIApr3NSMO2GwuOaUB&cshid=1568514008451

[ilia3101]

I will see if my CR2s have different values for this tag. Would be nice if it turns out it always has same value. I don't see why it would need to be different for different scenes, also don't see why camera matrix is not enough for Adobe.

[dannephoto]

Not different for every scene. Different for every sensor. It's a unique calibration tag for each and every cam produced. But only one per camera.
The calibration is always very small and affects how white balance is treated.

[ilia3101]

So it's different for each individual camera? I think cameras are similar enough that we can re use this tag. If there was actual significant difference, there would need to be different matrices for each cam as well. Seems like Adobe intentionally overcomplicating. I will compare the value of this tag on my current 5D2 with my old one.

I might try and find the value for this tag for as many of the default cameras as possible (and a few copies of each one so we can compare and get an average). Could this be a good plan?

Then we can add it to DNGs while exporting.

[dannephoto]

Na, I say skip it or you will probably end up worsen image on some sensors. In practice we are talking about maybe 4+ or 4- tint and maybe 2-3 steps wb slider.
If we can't find the value to get exact match it defeats the whole purpose imo.

[ilia3101]

In practice we are talking about maybe 4+ or 4- tint and maybe 2-3 steps wb slider.

So if we add this tag, it will still leave that tiny error in and not correct it, but it should correct the massive shift we currently have? It wouldn't increase the error on average anyway.

I am still really confused what the purpose of this tag actually is. It's the CameraCalibration1 and CameraCalibraion2 are the tags we are talking about here right?

It looks like they are another matrix or something :/

[ilia3101]

I look at the post you sent, and the camera calibration tags appear to be matrices, but they only do tiny amounts of channel gain. Maybe we should just include this tag and set it to identity matrix. @dannephoto Can you see what happens if you inject cameracalibration in to the DNGs as just 1,0,0,0,1,0,0,0,1 ?

Also how does it relate to asShotNeutral? Is asshotneutral necessary to have?

[dannephoto]

asshot... tag is the white balance information so yes, needed.
I did some more tests this morning. The camera calibration tag doesn´t seem to do any significant difference to the dng file. I think it´s most useful for dual iso files.
But indeed CR2 files and mlv dng files do look a bit different from each other. I tested now to set camera to tungsten. In post setting up the same white balance didn´t help. Adding the camera calibration tag didn´t help either.

Maybe related to white level? 14bit vs 16 bit? Some hue info not included into the dng file etc?

Anyway. If you want to test adding the camera calibration tag you can use exiftool. This tag comes from my camera :
exiftool "-CameraCalibration1=1.0876 0 0 0 1 0 0 0 1.0286" Input.dng -overwrite_original

One thing to test is to set camera to same kelvin, maybe 6500 and shoot a CR2 and a film a MLV output to dng, then compare. They should look the same but probably won´t.

[ilia3101]

Ok I am too confused by all this. As you recommend not to do this I won't.

Also why is this more useful on dual ISO??

[dannephoto]

Here. Take a look at these files. I think you´ll see what´s going on:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/camera_calibrationdualiso.zip

One dng has the added camera calibration tag, the other dng has not. As you´ll see the CR2 dual iso and the dng will have the exact same white level reading in adobe camera raw.
By the way. You won´t see the camera calibration tag in there because it´s baked into the AsShotNeutral tag...

[bouncyball-git]

Hm... and what the exact name of this DNG tag?