Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please check if SDR LUTs are actually used #7

Open
zmarty opened this issue Jan 21, 2017 · 21 comments
Open

Please check if SDR LUTs are actually used #7

zmarty opened this issue Jan 21, 2017 · 21 comments

Comments

@zmarty
Copy link

zmarty commented Jan 21, 2017

Video without LUT: https://youtu.be/V9Ipg2jA9cg?t=25s
Video with LUT: https://youtu.be/CR5v7oHBA74?t=25s

The MKV I uploaded for the second video contains an attachment named sdr.cube.

Can you confirm the sdr.cube LUT file is actually being used?

The problem is I don't see any difference between the videos. The LUT I uploaded should fix the burnt orange skin tones of the first video but does not.

@zmarty
Copy link
Author

zmarty commented Jan 21, 2017

Here are the command lines I used to add the cube:
--attachment-mime-type application/x-cube ^
--attach-file sdr.cube ^

And here is what MediaInfo says:
MediaInfo

@stevenrobertson
Copy link

Thanks for the report. There is a bug here that's emerged since the HDR launch, and we're looking into it.

@zmarty
Copy link
Author

zmarty commented May 12, 2017

Is this solved? Thanks!

@zmarty
Copy link
Author

zmarty commented Jun 2, 2017

I asked around and other people uploading HDR video are experiencing the same issue. Are we doing this wrong or it's still not fixed yet?

@morphinapg
Copy link

morphinapg commented Jun 5, 2020

I am fairly certain LUTs are still not working. I've put a lot of work into making sure my HDR videos look as good as they can be in SDR, but youtube's just ignoring it. Please fix this.

Note, this is not really an issue with this metadata tool. It attaches the LUT correctly, and this can also be accomplished in MKVToolnix (as well as the other metadata stuff) but it's an issue on the youtube server end.

EDIT: renaming my LUTs with lowercase file extensions instead of uppercase fixed my problem. That's pretty disappointing, although it can't be the same issue OP was having 3 years ago. Perhaps they fixed that but still had poor code for checking whether a LUT was attached.

@MuscularPuky
Copy link

I am fairly certain LUTs are still not working. I've put a lot of work into making sure my HDR videos look as good as they can be in SDR, but youtube's just ignoring it. Please fix this.

Note, this is not really an issue with this metadata tool. It attaches the LUT correctly, and this can also be accomplished in MKVToolnix (as well as the other metadata stuff) but it's an issue on the youtube server end.

EDIT: renaming my LUTs with lowercase file extensions instead of uppercase fixed my problem. That's pretty disappointing, although it can't be the same issue OP was having 3 years ago. Perhaps they fixed that but still had poor code for checking whether a LUT was attached.

mine is lowercase but same problem. youtube SERVER ignores lut, while metadata is fine

@morphinapg
Copy link

I am fairly certain LUTs are still not working. I've put a lot of work into making sure my HDR videos look as good as they can be in SDR, but youtube's just ignoring it. Please fix this.
Note, this is not really an issue with this metadata tool. It attaches the LUT correctly, and this can also be accomplished in MKVToolnix (as well as the other metadata stuff) but it's an issue on the youtube server end.
EDIT: renaming my LUTs with lowercase file extensions instead of uppercase fixed my problem. That's pretty disappointing, although it can't be the same issue OP was having 3 years ago. Perhaps they fixed that but still had poor code for checking whether a LUT was attached.

mine is lowercase but same problem. youtube SERVER ignores lut, while metadata is fine

Make sure the LUT is 32pt. I tried using higher quality LUTs but it wasn't working.

That being said, recently the server had been processing my LUTs incorrectly, swapping red and blue channels, but I've been working with youtube support to get that fixed. Not sure if others are having that issue.

@MuscularPuky
Copy link

I am fairly certain LUTs are still not working. I've put a lot of work into making sure my HDR videos look as good as they can be in SDR, but youtube's just ignoring it. Please fix this.
Note, this is not really an issue with this metadata tool. It attaches the LUT correctly, and this can also be accomplished in MKVToolnix (as well as the other metadata stuff) but it's an issue on the youtube server end.
EDIT: renaming my LUTs with lowercase file extensions instead of uppercase fixed my problem. That's pretty disappointing, although it can't be the same issue OP was having 3 years ago. Perhaps they fixed that but still had poor code for checking whether a LUT was attached.

mine is lowercase but same problem. youtube SERVER ignores lut, while metadata is fine

Make sure the LUT is 32pt. I tried using higher quality LUTs but it wasn't working.

That being said, recently the server had been processing my LUTs incorrectly, swapping red and blue channels, but I've been working with youtube support to get that fixed. Not sure if others are having that issue.

Premiere Pro exports 16 only. ah then

...would you tell me "It's 32 and below, not exactly 32"...?

@morphinapg
Copy link

I am fairly certain LUTs are still not working. I've put a lot of work into making sure my HDR videos look as good as they can be in SDR, but youtube's just ignoring it. Please fix this.
Note, this is not really an issue with this metadata tool. It attaches the LUT correctly, and this can also be accomplished in MKVToolnix (as well as the other metadata stuff) but it's an issue on the youtube server end.
EDIT: renaming my LUTs with lowercase file extensions instead of uppercase fixed my problem. That's pretty disappointing, although it can't be the same issue OP was having 3 years ago. Perhaps they fixed that but still had poor code for checking whether a LUT was attached.

mine is lowercase but same problem. youtube SERVER ignores lut, while metadata is fine

Make sure the LUT is 32pt. I tried using higher quality LUTs but it wasn't working.
That being said, recently the server had been processing my LUTs incorrectly, swapping red and blue channels, but I've been working with youtube support to get that fixed. Not sure if others are having that issue.

Premiere Pro exports 16 only. ah then

...would you tell me "It's 32 and below, not exactly 32"...?

That I'm not sure, but open an image in photoshop, add an adjustment layer for Color Lookup, load the LUT in there, and then go to File -> Export -> Color Lookup Tables and you'll be able to convert it to 32 point that way

@MuscularPuky
Copy link

MuscularPuky commented May 23, 2024

I am fairly certain LUTs are still not working. I've put a lot of work into making sure my HDR videos look as good as they can be in SDR, but youtube's just ignoring it. Please fix this.
Note, this is not really an issue with this metadata tool. It attaches the LUT correctly, and this can also be accomplished in MKVToolnix (as well as the other metadata stuff) but it's an issue on the youtube server end.
EDIT: renaming my LUTs with lowercase file extensions instead of uppercase fixed my problem. That's pretty disappointing, although it can't be the same issue OP was having 3 years ago. Perhaps they fixed that but still had poor code for checking whether a LUT was attached.

mine is lowercase but same problem. youtube SERVER ignores lut, while metadata is fine

Make sure the LUT is 32pt. I tried using higher quality LUTs but it wasn't working.
That being said, recently the server had been processing my LUTs incorrectly, swapping red and blue channels, but I've been working with youtube support to get that fixed. Not sure if others are having that issue.

Premiere Pro exports 16 only. ah then
...would you tell me "It's 32 and below, not exactly 32"...?

That I'm not sure, but open an image in photoshop, add an adjustment layer for Color Lookup, load the LUT in there, and then go to File -> Export -> Color Lookup Tables and you'll be able to convert it to 32 point that way

oh that's interesting method... I suspected Hybird Log Gamma before you reply so published 4 case files. (PQ/HLG, 16/32) I hope youtube process mine as soon as possible within 1h

@MuscularPuky
Copy link

interestingly Windows 11 native player also recognizes LUTs. and I just found out youtube does not use LUT in HLG video ANYMORE. so my LUT was ignored

I tested 2 case files—1 is with Premiere Pro to Photoshop converted LUT, another is with adjusted only in Photoshop LUT

Windows 11
PQ with ANY LUT: ignored
HLG with ANY LUT: works well

youtube
PQ with converted LUT: ignored
HLG with converted LUT: HDR disabled. WHAT??
PQ with original LUT: ignored
HLG with original LUT: converted to SDR by youtube. in HDR, the color is not perfect than PQ but still looks great

so youtube does not apply my LUTs in any case :( I'll wait their process more

@morphinapg
Copy link

interestingly Windows 11 native player also recognizes LUTs. and I just found out youtube does not use LUT in HLG video ANYMORE. so my LUT was ignored

I tested 2 case files—1 is with Premiere Pro to Photoshop converted LUT, another is with adjusted only in Photoshop LUT

Windows 11 PQ with ANY LUT: ignored HLG with ANY LUT: works well

youtube PQ with converted LUT: ignored HLG with converted LUT: HDR disabled. WHAT?? PQ with original LUT: ignored HLG with original LUT: converted to SDR by youtube. in HDR, the color is not perfect than PQ but still looks great

so youtube does not apply my LUTs in any case :( I'll wait their process more

Are you sure your PQ metadata is correct? BT2020 primaries and matrix, and ST2084 transfer fuction? Here's an example of what the mediainfo looks like for one of my videos that gets processed correctly:

image

Note I've had videos process successfully without the MaxCLL/MaxFALL metadata but it's still a good idea to include those if you know the appropriate values.

@MuscularPuky
Copy link

interestingly Windows 11 native player also recognizes LUTs. and I just found out youtube does not use LUT in HLG video ANYMORE. so my LUT was ignored
I tested 2 case files—1 is with Premiere Pro to Photoshop converted LUT, another is with adjusted only in Photoshop LUT
Windows 11 PQ with ANY LUT: ignored HLG with ANY LUT: works well
youtube PQ with converted LUT: ignored HLG with converted LUT: HDR disabled. WHAT?? PQ with original LUT: ignored HLG with original LUT: converted to SDR by youtube. in HDR, the color is not perfect than PQ but still looks great
so youtube does not apply my LUTs in any case :( I'll wait their process more

Are you sure your PQ metadata is correct? BT2020 primaries and matrix, and ST2084 transfer fuction? Here's an example of what the mediainfo looks like for one of my videos that gets processed correctly:

image

Note I've had videos process successfully without the MaxCLL/MaxFALL metadata but it's still a good idea to include those if you know the appropriate values.

yes mine PQ video has BT.2020+ST.2084, and HLG video has arib-std-b67
as I know MaxCLL/MaxFALL data is for HDR10 so if the video is just only HDR, not HDR10 then we can ignore those
if HDR video has no MaxCLL/MaxFALL, I think they'll treat as 1000 but not sure
perhaps youtube LUT processing nedds more time... maybe... ah.

one thing I suspect: if youtube considered the LUT as nonsence—would rather spoil the video, invalid data, incompatible, etc, could they ignore the LUT??

@morphinapg
Copy link

interestingly Windows 11 native player also recognizes LUTs. and I just found out youtube does not use LUT in HLG video ANYMORE. so my LUT was ignored
I tested 2 case files—1 is with Premiere Pro to Photoshop converted LUT, another is with adjusted only in Photoshop LUT
Windows 11 PQ with ANY LUT: ignored HLG with ANY LUT: works well
youtube PQ with converted LUT: ignored HLG with converted LUT: HDR disabled. WHAT?? PQ with original LUT: ignored HLG with original LUT: converted to SDR by youtube. in HDR, the color is not perfect than PQ but still looks great
so youtube does not apply my LUTs in any case :( I'll wait their process more

Are you sure your PQ metadata is correct? BT2020 primaries and matrix, and ST2084 transfer fuction? Here's an example of what the mediainfo looks like for one of my videos that gets processed correctly:
image
Note I've had videos process successfully without the MaxCLL/MaxFALL metadata but it's still a good idea to include those if you know the appropriate values.

yes mine PQ video has BT.2020+ST.2084, and HLG video has arib-std-b67 as I know MaxCLL/MaxFALL data is for HDR10 so if the video is just only HDR, not HDR10 then we can ignore those if HDR video has no MaxCLL/MaxFALL, I think they'll treat as 1000 but not sure perhaps youtube LUT processing nedds more time... maybe... ah.

one thing I suspect: if youtube considered the LUT as nonsence—would rather spoil the video, invalid data, incompatible, etc, could they ignore the LUT??

I think in my experience when I had a LUT formatted in ways they didn't like (like the 64 size one I started with, or the .CUBE in caps) then I believe it interpreted my videos as not being HDR at all, and showing the raw unprocessed signal as if it was being interpreted as a SDR-encoded file. So super washed out, wrong contrast and colors, etc. But this was years ago so I'm not sure if they do things differently now.

But currently as long as I upload with correct primaries/transform/matrix values, and a 32 point .cube LUT with all lowercase letters (I also do no spaces just in case) then it seems to work fine. Yes it can take longer to process, but the SDR portion usually processes first and matches the LUT when it does.

@morphinapg
Copy link

morphinapg commented Jun 17, 2024

interestingly Windows 11 native player also recognizes LUTs. and I just found out youtube does not use LUT in HLG video ANYMORE. so my LUT was ignored
I tested 2 case files—1 is with Premiere Pro to Photoshop converted LUT, another is with adjusted only in Photoshop LUT
Windows 11 PQ with ANY LUT: ignored HLG with ANY LUT: works well
youtube PQ with converted LUT: ignored HLG with converted LUT: HDR disabled. WHAT?? PQ with original LUT: ignored HLG with original LUT: converted to SDR by youtube. in HDR, the color is not perfect than PQ but still looks great
so youtube does not apply my LUTs in any case :( I'll wait their process more

Are you sure your PQ metadata is correct? BT2020 primaries and matrix, and ST2084 transfer fuction? Here's an example of what the mediainfo looks like for one of my videos that gets processed correctly:
image
Note I've had videos process successfully without the MaxCLL/MaxFALL metadata but it's still a good idea to include those if you know the appropriate values.

yes mine PQ video has BT.2020+ST.2084, and HLG video has arib-std-b67 as I know MaxCLL/MaxFALL data is for HDR10 so if the video is just only HDR, not HDR10 then we can ignore those if HDR video has no MaxCLL/MaxFALL, I think they'll treat as 1000 but not sure perhaps youtube LUT processing nedds more time... maybe... ah.

one thing I suspect: if youtube considered the LUT as nonsence—would rather spoil the video, invalid data, incompatible, etc, could they ignore the LUT??

I thought of something. Have you made sure that MIME type on the attachment is set to "application/x-cube"? I don't know whether it's strictly necessary, but the original documentation does list to use that, and I always have.

@thatcherfreeman
Copy link

Hey @morphinapg can you share the LUT that worked for you (ie attach it or link to google drive or something)? I'm running into the same issue and despite making 32 point cube LUTs named "sdr.cube" and attaching them with all the recommended arguments to the mkvmerge tool, and verifying that the LUT is in there, youtube still seems to ignore the LUT when I upload.

@morphinapg
Copy link

Hey @morphinapg can you share the LUT that worked for you (ie attach it or link to google drive or something)? I'm running into the same issue and despite making 32 point cube LUTs named "sdr.cube" and attaching them with all the recommended arguments to the mkvmerge tool, and verifying that the LUT is in there, youtube still seems to ignore the LUT when I upload.

Here's one I attached to one of my recent videos that worked fine the first time I tried

sdr.zip

Note that recently, while my LUTs usually work fine, sometimes youtube doesn't end up processing HDR and I either need to reupload multiple times or contact support to have then reprocess my existing upload attempts. But I also upload very long videos so it's possible that their processing simply gets stuck.

@thatcherfreeman
Copy link

Sure enough, your LUT works and my LUTs don't. I'll report back later if I can find the root cause.

@MuscularPuky
Copy link

interestingly Windows 11 native player also recognizes LUTs. and I just found out youtube does not use LUT in HLG video ANYMORE. so my LUT was ignored
I tested 2 case files—1 is with Premiere Pro to Photoshop converted LUT, another is with adjusted only in Photoshop LUT
Windows 11 PQ with ANY LUT: ignored HLG with ANY LUT: works well
youtube PQ with converted LUT: ignored HLG with converted LUT: HDR disabled. WHAT?? PQ with original LUT: ignored HLG with original LUT: converted to SDR by youtube. in HDR, the color is not perfect than PQ but still looks great
so youtube does not apply my LUTs in any case :( I'll wait their process more

Are you sure your PQ metadata is correct? BT2020 primaries and matrix, and ST2084 transfer fuction? Here's an example of what the mediainfo looks like for one of my videos that gets processed correctly:
image
Note I've had videos process successfully without the MaxCLL/MaxFALL metadata but it's still a good idea to include those if you know the appropriate values.

yes mine PQ video has BT.2020+ST.2084, and HLG video has arib-std-b67 as I know MaxCLL/MaxFALL data is for HDR10 so if the video is just only HDR, not HDR10 then we can ignore those if HDR video has no MaxCLL/MaxFALL, I think they'll treat as 1000 but not sure perhaps youtube LUT processing nedds more time... maybe... ah.
one thing I suspect: if youtube considered the LUT as nonsence—would rather spoil the video, invalid data, incompatible, etc, could they ignore the LUT??

I thought of something. Have you made sure that MIME type on the attachment is set to "application/x-cube"? I don't know whether it's strictly necessary, but the original documentation does list to use that, and I always have.

THX I'll check it

@thatcherfreeman
Copy link

thatcherfreeman commented Aug 8, 2024

Thanks, I've done a variety of testing and have some preliminary results that would likely help folks here.

  • I made .cube LUTs with all lowercase names, and I agree that typically 32 and 33 point LUTs seem to work.
  • I don't think 65 point LUTs work.
  • Importantly, youtube sometimes chooses to ignore your LUT depending on the contents of the LUT. As an example, if your LUT samples the operation f(x) = max(x, 0.075) for each channel (lower bound is somewhere between 0.05 to 0.075), then Youtube will use their default HDR to SDR conversion instead of the LUT. If your operation were like an S curve that maps 0 to 0 and 1 to 1, then youtube's fine with it.
  • If your LUT samples the operation f(x) = 1 - x for all channels, youtube detects this and ignores this LUT, so they're not just blindly looking at only the min/max value in the LUT.
  • It doesn't seem that youtube requires or cares about the TITLE or DOMAIN_MIN or DOMAIN_MAX tags
  • Youtube doesn't seem to care about LF or CRLF line endings, it accepts the LUT either way
  • I was able to get Youtube to not ignore my HDR to SDR conversion LUT that had highly saturated colors (my specific HDR video is rec709 limited so these wouldn't be a problem in practice) if I clamped the LUT to 0-1 outputs
  • Youtube didn't seem to care if every number had six digits after the decimal point or were variable

Basically TLDR is that I think everyone who is running into problems with their HDR to SDR LUT, at this point in time, I would recommend doing the following:

  1. all lowercase file name, ideally no numbers
  2. Make sure you have a 32 or 33 point resolution LUT
  3. inspect the contents of your LUT, perhaps clamp or otherwise ensure that black (0,0,0) is mapped close to zero and that there aren't crazy negatives or large numbers >1 anywhere.

I'll probably investigate this further. It's ridiculous that they have a QC system here that is opaque and doesn't tell you why or even if your LUT was ignored.

@MuscularPuky
Copy link

interestingly Windows 11 native player also recognizes LUTs. and I just found out youtube does not use LUT in HLG video ANYMORE. so my LUT was ignored
I tested 2 case files—1 is with Premiere Pro to Photoshop converted LUT, another is with adjusted only in Photoshop LUT
Windows 11 PQ with ANY LUT: ignored HLG with ANY LUT: works well
youtube PQ with converted LUT: ignored HLG with converted LUT: HDR disabled. WHAT?? PQ with original LUT: ignored HLG with original LUT: converted to SDR by youtube. in HDR, the color is not perfect than PQ but still looks great
so youtube does not apply my LUTs in any case :( I'll wait their process more

Are you sure your PQ metadata is correct? BT2020 primaries and matrix, and ST2084 transfer fuction? Here's an example of what the mediainfo looks like for one of my videos that gets processed correctly:
image
Note I've had videos process successfully without the MaxCLL/MaxFALL metadata but it's still a good idea to include those if you know the appropriate values.

yes mine PQ video has BT.2020+ST.2084, and HLG video has arib-std-b67 as I know MaxCLL/MaxFALL data is for HDR10 so if the video is just only HDR, not HDR10 then we can ignore those if HDR video has no MaxCLL/MaxFALL, I think they'll treat as 1000 but not sure perhaps youtube LUT processing nedds more time... maybe... ah.
one thing I suspect: if youtube considered the LUT as nonsence—would rather spoil the video, invalid data, incompatible, etc, could they ignore the LUT??

I thought of something. Have you made sure that MIME type on the attachment is set to "application/x-cube"? I don't know whether it's strictly necessary, but the original documentation does list to use that, and I always have.

THX I'll check it

all is application/x-cube. I'm sure that YouTube's thinking my cube curve is wrong so ignoring it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants