HDR -> SDR conversion

These CUDA filters are packaged into DGDecodeNV, which is part of DGDecNV.
dmcs
Posts: 36
Joined: Sat Oct 21, 2017 9:40 pm

Re: HDR -> SDR tonemapping

Post by dmcs » Tue Jul 10, 2018 7:47 pm

admin wrote:
Tue Jul 10, 2018 2:00 pm
What do y'all think of this result? If you like it I'll make a release.
I think it looks really good. But, please allow me to be a bit nitpicking :)

There seems to be little artifact exists along the contour if you look closely (literally). Well, either that, or the saturation makes me think it's artifact? Anw, I don't think anyone would notice during normal playback.

Thanks a lot for your hard work. :salute: :salute:
Screenshot_2018-07-10 latest jpg (JPEG Image, 960 × 540 pixels) - Scaled (98%).png
Screenshot_2018-07-10 latest jpg (JPEG Image, 960 × 540 pixels) - Scaled (98%).png (467.63 KiB) Viewed 2500 times

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR tonemapping

Post by admin » Tue Jul 10, 2018 9:16 pm

Yes, I noticed it too. I agree it's probably not noticeable but I may know the cause so why not try to fix it. I'm still just clipping R/G/B values that end up negative after 2020->709. I need to tonemap them as I do for values > 1.0. I'll see if I can mitigate it.

Good eyes! :wow:

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR tonemapping

Post by admin » Tue Jul 10, 2018 11:36 pm

OK, I did some more deep looking. The clipping of R/G/B values < 0 is relatively insignificant. I'm thinking the real problem is that RGB space is not constant hue and saturation for brightness changes. So, for example, if we take an RGB value and reduce its brightness by multipying all the channels by (say) 0.5, there is a subtle hue and saturation change. So in a real image where in gamut colors of a given hue and saturation can be right next to out of gamut ones with the same hue and saturation (one is just brighter), if we scale down the out of gamut one then now we can see blocking because the hues and saturations are now different when they should have stayed the same. Bottom line, we can't do lightness compression in RGB space. I hoped to get away with it, but it is now apparent that it was too cheap. :cry:

Next step, try a more hue/saturation-linear space like xyY, CIELAB, or ICtCp.

dmcs
Posts: 36
Joined: Sat Oct 21, 2017 9:40 pm

Re: HDR -> SDR tonemapping

Post by dmcs » Wed Jul 11, 2018 12:54 am

Darn. That's unfortunate. :? :shock:

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR tonemapping

Post by admin » Wed Jul 11, 2018 6:45 am

After sleeping on it I'm not convinced by my own explanation. Those blocks are much bigger than a pixel so something other than what I suggested must be causing it. Investigations continue today after I take care of renols's issue with DGDecNV.

dmcs
Posts: 36
Joined: Sat Oct 21, 2017 9:40 pm

Re: HDR -> SDR tonemapping

Post by dmcs » Wed Jul 11, 2018 6:54 am

No rush, hehe. Btw, I thought you were going to say "... after the World cup match." :D

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR tonemapping

Post by admin » Wed Jul 11, 2018 8:50 am

I'll be cheering for Croatia. You?

dmcs
Posts: 36
Joined: Sat Oct 21, 2017 9:40 pm

Re: HDR -> SDR tonemapping

Post by dmcs » Wed Jul 11, 2018 9:21 am

I'm neutral. :)

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR tonemapping

Post by admin » Wed Jul 11, 2018 1:52 pm

I definitely now understand why the blocking is occurring. It is, after all, the clipping of values below 0 for the R, G, and/or B channels. I traced the detailed processing for two pixels, one each side of a block boundary. The original YUV420P16 values are close enough that we would expect the output pixels also to be close in YV12 values. But one of the pixels obtained a negative R value after 2020->709 (so it's out of gamut) while the other did not (all others were nonzero). Then when clipped and converted the final Y,U,V values were:

pixel 1: 142,102,36
pixel 2: 158,95,60

These differences are easily visible. For another pair in the same vicinity but without a block boundary between them, the resulting values are close and so no boundary becomes visible.

So, without a doubt, I have to do something more intelligent than clipping values that are below 0. One option is to iteratively reduce the saturation of the pixel by a small amount until it is back in range.

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR tonemapping

Post by admin » Fri Jul 13, 2018 11:04 am

I decided to release 1.5 with the improvement of highlight handling and the removal of hable. The artifacting for very bright highlights is not solved as I cannot find a way to salvage the simple matrix conversion of 2020 to 709. Nevertheless, it is not so perceptible (especially at HD resolution) and affects only very bright highlights.

My plan now is to ditch the simple matrix conversion and do full proper gamut mapping. I'm going to generate 2D lightness/chroma gamut maps for 709 for each hue (sampled at probably one degree around the hue cylinder). Then I will use these to compress out-of-gamut colors onto the gamut boundary. These gamut maps will be pre-generated and stored in the executable for high performance. Given the maps, there are of course many ways to use them and I will test different options. This approach will eliminate the artifacting as clipping of RGB values below 0 will not apply.

Narkyy
Posts: 51
Joined: Thu May 25, 2017 11:51 pm

Re: HDR -> SDR tonemapping

Post by Narkyy » Fri Jul 13, 2018 11:46 am

:hat:
There seems to be some issues with the update :(

v1.4
Image

v1.5
Image

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin » Fri Jul 13, 2018 11:50 am

C'mon Narkyy, I asked you before to explain and not just say "some issues". :evil:

And you know I need the script and a source sample to do anything.

Narkyy
Posts: 51
Joined: Thu May 25, 2017 11:51 pm

Re: HDR -> SDR conversion

Post by Narkyy » Fri Jul 13, 2018 11:56 am

My bad, I thought it was obvious, the lighting is all weird on the window side.
The plain white looks like saturation was applied to it and luminance dropped completely.

Image

Sample: https://mega.nz/#!sVtAnAIQ!lOOwvKGOWy-l ... Y9LyzWQmLk
Script
FFVideoSource("sample.mkv", colorspace="YUV420P10")
ConvertBits(16).DGHDRtoSDR(impl="255", light=150, peak=2)
Last edited by Narkyy on Fri Jul 13, 2018 12:04 pm, edited 1 time in total.

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin » Fri Jul 13, 2018 12:03 pm

Thanks. Will look into it.

EDIT: 1.5 temporarily withdrawn.

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin » Mon Jul 16, 2018 2:16 pm

I've redesigned my gamut mapping and tonemapping. Here is what I am getting for the problematic Narkyy scene with dghdrtosdr(light=150). This is using simple Reinhard tonemapping. I may bring back Mobius but it is currently not behaving as expected. The result looks good to me. Highlights are handled well and artifacting is gone. I have to regression test it on all my other HDR streams.

EDIT: The new method does not require the sat=0.8 tweak but I posted this shot before realizing that and default 0.8 was used. So it may look a bit desaturated.

narkyy.jpg
narkyy.jpg (240.73 KiB) Viewed 2211 times

Narkyy
Posts: 51
Joined: Thu May 25, 2017 11:51 pm

Re: HDR -> SDR conversion

Post by Narkyy » Mon Jul 16, 2018 4:15 pm

Neat, bright spots look softer too. :salute:

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin » Mon Jul 16, 2018 4:20 pm

Glad you like it. Actually there is now a tonemapping strength option that allows you to select along a continuum of hard hightlights and soft highlights. This sample is the softest. If you want a little more dazzle from the brights, you can decrease the strength.

Thanks for this sample, Narkyy. It is now one of my go-to HDR torture tests.

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin » Tue Jul 17, 2018 4:47 pm

Here's 1.5 with the patented Rocky the Squirrel gamut mapping:

http://rationalqm.us/misc/DGHDRtoSDR_1.5.rar

Hammer it with your torture scenes! :twisted:

Narkyy
Posts: 51
Joined: Thu May 25, 2017 11:51 pm

Re: HDR -> SDR conversion

Post by Narkyy » Tue Jul 17, 2018 8:55 pm

:hat:
It looks much better at handling highlights, plus the "tm" parameter is great.
Noticed some differences with v1.4 again, at this point I compare against madVR but it's getting harder to tell which is supposed to be "correct"
Screenshots in order: v1.4 light=250, v1.5 light=250 tm=0.90, madVR 250 nits

Image Image Image

The hue shift seems to be fixed in this sample so that's good. Looking at the red lamps at the top, 1.4 is definitely shifted to the right and looks wider.
The middle of the lights isn't red but it's hard to tell if it's supposed to be, at least from the untonemapped source.

There's a spot of light orange in the sky instead of white, which looks like saturation applied there. There's definitely some banding there in the source itself so that might be the cause?

Otherwise some colors are different, like the blue wavy line at the front of the boat is a bit darker. Not sure if it's a lack of saturation or too much saturation though.
The orange and red stripes in the flag are less saturated as well.

Sample URL: https://mega.nz/#!gZF2FS6a!V4e1acBSNx74 ... tfMlZdTkBc

I'll test other samples where I saw issues before and report back.
Thank you :salute:

Edit:
So after some other samples, there's definitely a desaturation of red/orange happening, compared to v1.4 and madVR.
I'm not sure about other colors yet

From another sample, which reveals the same white spot in deep red lights, as well as desaturation and some fine detail smoothing on the left side (our left) of the face that's probably caused by lower gamma/contrast.
Screenshot is from frame 129.
v1.4 has the colors proper, but hue shift all over the place.

In order: v1.4 light=150 peak=2.0, light=150 tm=0.90, madVR 130 nits

Image Image Image

Sample #2: https://mega.nz/#!5A8FHJJI!t9NhrAw64prb ... pCKsq0QESU

If you compare with madVR at the edge of red light on the face (especially on the nose), you can see the clipping of color whereas madVR has a roll off to lower saturation.
That might be the same issue as in the bright spots on the metal.
Last edited by Narkyy on Tue Jul 17, 2018 10:15 pm, edited 11 times in total.

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin » Tue Jul 17, 2018 9:31 pm

Thanks a million, Narkyy, I'll study it tomorrow as it's getting late here.

dmcs
Posts: 36
Joined: Sat Oct 21, 2017 9:40 pm

Re: HDR -> SDR conversion

Post by dmcs » Wed Jul 18, 2018 12:42 am

Tried the latest version with the same sample I uploaded earlier.
I think the screenshot is pretty much self-explanatory. :D
      
MI5_HDR_Sample002467.png
MI5_HDR_Sample002467.png (1.81 MiB) Viewed 2335 times
      
Edit:
I also notice the desaturation in version 1.5 (the first 3 sets of screenshots - sample 1). In the 4th set (sample 2), the sky has some strange artifact (?!).
I used Mobius to take screenshots in version 1.4

v1.4 -- v1.5
1st
Image Image

2nd
Image Image

3rd
Image Image

4th
Image Image

Samples

Code: Select all

https://mega.nz/#F!BwtXQC4A!r8wPHGJDE4ftwkpzudoVow
Last edited by dmcs on Wed Jul 18, 2018 6:55 am, edited 2 times in total.

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin » Wed Jul 18, 2018 6:44 am

I don't do self-explanatory. What is your point?

dmcs
Posts: 36
Joined: Sat Oct 21, 2017 9:40 pm

Re: HDR -> SDR conversion

Post by dmcs » Wed Jul 18, 2018 6:54 am

admin wrote:
Wed Jul 18, 2018 6:44 am
I don't do self-explanatory. What is your point?
Compared to Mobius in 1.4, the new method destroys color & detail in this sample


Image Image Image Image Image

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin » Wed Jul 18, 2018 7:11 am

Ouch. The new method of gamut mapping does constant hue and luminance, but it apparently desaturates too much to pull colors into gamut. I'll continue tweaking to find a reasonable compromise between desaturation and hue shifts.

Thank you for your test results.

User avatar
admin
Site Admin
Posts: 4415
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin » Wed Jul 18, 2018 9:17 am

I just made a momentous discovery! With a very simple change to the current design the following is now produced for dmcs's torture frame (light=300). Note the lack of any artifacting or hue change. I have to regression test it with my other samples; if everything is copacetic I'll give you a release.

Hope springs eternal!

dmcs2.jpg
dmcs2.jpg (247.24 KiB) Viewed 2314 times

Post Reply