HDR -> SDR conversion

These CUDA filters are packaged into DGDecodeNV, which is part of DGDecNV.
User avatar
alexantr
Posts: 11
Joined: Mon Oct 15, 2018 4:24 pm

Re: HDR -> SDR conversion

Post by alexantr »

Why DGHDRtoSDR has so strange hue?

1. SDR version: https://i.imgur.com/8ns30OO.jpg
2. HDR to SDR with DGTonemap 1.2: https://i.imgur.com/B2jbxdf.jpg
3. HDR to SDR with DGHDRtoSDR 1.10: https://i.imgur.com/zGFdNzF.jpg

avs with DGTonemap:

Code: Select all

loadplugin("D:\Programs\DG\x64\DGDecodeNV.dll")
loadplugin("D:\Programs\DG\x64\DGTonemap.dll")
loadplugin("D:\Programs\DG\x64\avsresize.dll")
SetFilterMTMode("z_ConvertFormat", MT_MULTI_INSTANCE)
DGSource("e:\encode\Coco.dgi", fulldepth=true)
ConvertBits(10)
z_ConvertFormat(pixel_type="RGBPS", colorspace_op="2020ncl:st2084:2020:l=>rgb:linear:2020:l", dither_type="none")
DGReinhard()
z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:linear:2020:l=>709:709:709:l", dither_type="ordered")
Crop(0, 276, 0, -276)
Spline36Resize(1920, 804)
prefetch(4)
avs with DGHDRtoSDR:

Code: Select all

loadplugin("D:\Programs\DG\x64\DGDecodeNV.dll")
loadplugin("D:\Programs\DG\x64\DGHDRtoSDR.dll")
DGSource("e:\encode\Coco.dgi", fulldepth=true)
Crop(0, 276, 0, -276)
Spline36Resize(1920, 804)
DGHDRtoSDR()
prefetch(6)
Another video:

1. HDR with madVR: https://i.imgur.com/mTlruX1.jpg
2. HDR to SDR with DGTonemap 1.2: https://i.imgur.com/HArYxUt.jpg
3. HDR to SDR with DGHDRtoSDR 1.10: https://i.imgur.com/GaoQ2Go.jpg
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin »

"strange hue"

Not a really precise formulation but I can look into it if you provide me with an unprocessed source stream sample. I can't do much with JPGs.
User avatar
alexantr
Posts: 11
Joined: Mon Oct 15, 2018 4:24 pm

Re: HDR -> SDR conversion

Post by alexantr »

Please checkout source file: https://mega.nz/#!GsdHmazY!Vo6zfUcpm31E ... HOZbqerSlA

Resulting file after DGHDRtoSDR: https://mega.nz/#!z0MlgaDR!NJlZg76uyf0B ... hDBMeXAcow

Resulting file after DGTonemap: https://mega.nz/#!T8sRhQIZ!4IvY35zNQnwa ... ZK2QLIf7bw

Every my tested video has same problem. I can see almost similar result only after appending Tweak(hue=10).
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin »

Thanks, I'll look into it.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin »

Please try out this version:

http://rationalqm.us/hdr/DGHDRtoSDR_1.11.rar

I added some hue correction for the gamut mapping. We'll never exactly match other solutions. What I am aiming for is reasonable results for most streams and fast processing in CUDA mode.
DAE avatar
Guest 3
Posts: 67
Joined: Mon Mar 26, 2018 6:00 am

Re: HDR -> SDR conversion

Post by Guest 3 »

Image Image Image Image

I see similar problem with the colors. It is probably due to the Hable operator implemented with DGTonemap plugin with its dafaults.
At least, I’ve seen same results with ffmpeg tonemapping filters.
Three of above screens are results of Hable tonemapping. The fourth is from the source UHD .hevc.

Working with 240 frames cut from the source Gladiator 2000 UHD I've encountered two more problems:
  • DGDecNV(), when works, processes 236 frames.

Code: Select all

>avs2yuv64 -depth 10 -raw DGTonemap.avs - | x265-64bit-8bit-2018-10-08 - ...
yuv  [info]: 1920x1080 fps 24000/1001 i420p8 unknown frame count
raw  [info]: output file:
...
x265 [info]: HEVC encoder version 2.8+74-fd517ae68f93
...
x265 [info]: tools:
avisynth 16-bit hack enabled
DGTonemap.avs: 960x1080, YV12, 10-bits, progressive, 24000/1001 fps, 236 frames
encoded 236 frames in 293.53s (0.80 fps), 59210.41 kb/s, Avg QP:16.28
If I use FFMS2() as a source instead I have

Code: Select all

>avs2yuv64 -depth 10 -raw DGTonemapMod.avs - | 
...
DGTonemapMod.avs: 960x1080, YV12, 10-bits, progressive, 24 fps, 240 frames
encoded 240 frames in 314.47s (0.76 fps), 58830.89 kb/s, Avg QP:16.23
  • HDR2SDR plugin doesn't work properly.
It process more than 500 frames and I cancel it. Seems like a program loop.
The result seems as if color channels' frames are overlaid.

Working environment
Window 10 Pro v.1803 Build 17134.345
GeForce GTX 1050 Video driver v.398.82
User avatar
alexantr
Posts: 11
Joined: Mon Oct 15, 2018 4:24 pm

Re: HDR -> SDR conversion

Post by alexantr »

admin wrote:
Wed Oct 17, 2018 11:13 am
Please try out this version:

http://rationalqm.us/hdr/DGHDRtoSDR_1.11.rar

I added some hue correction for the gamut mapping. We'll never exactly match other solutions. What I am aiming for is reasonable results for most streams and fast processing in CUDA mode.
Awesome! Thanks!
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin »

@ alexantr

You're welcome, and thank you for your valuable report.

@Guest 3
Guest 3 wrote:
Wed Oct 17, 2018 11:49 am
Working with 240 frames cut from the source Gladiator 2000 UHD I've encountered two more problems:
DGDecNV(), when works, processes 236 frames.
There may be a good reason for that. If you link me to your cut stream I can look into it.
HDR2SDR plugin doesn't work properly.
I don't know that plugin.
DAE avatar
Guest 3
Posts: 67
Joined: Mon Mar 26, 2018 6:00 am

Re: HDR -> SDR conversion

Post by Guest 3 »

Tonemapping isn't a trivial task and the operators least at their defaults aren't universal.
I've just found a JOHN HABLE's publication at http://filmicworlds.com/blog/filmic-ton ... er-curves/
proposing Filmic Tonemapping with Piecewise Power Curves.
Here is his source code https://github.com/johnhable/fw-public
I think it would be nice if we had this model implemented as aa AviSynth.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin »

@Guest 3

Are you going to ignore my request for your source file, etc.?
DAE avatar
Guest 3
Posts: 67
Joined: Mon Mar 26, 2018 6:00 am

Re: HDR -> SDR conversion

Post by Guest 3 »

@admin
HDR2SDR I don't know that plugin.
Sorry, I meant DGHDRtoSDR.dll v.DGHDRtoSDR_1.10.rar
As to
@Guest 3
Are you going to ignore my request for your source file, etc.?
I'm missing something.
Nevertheless, my source is Gladiator 2000 Extended 2160p UHD BluRay REMUX HDR HEVC DTS-X-EPSiLON.
Do you want a 10/20 sec cut from it or what?
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin »

Please give me the cut that you made, you know, the one you say has 240 frames but for which DGSource() says 236.

You say DGHDRtoSDR is not working properly, so please give me the source file (if different from the one above), and the exact Avisynth script that you used. I expect the issue is related to your other tools and process, as DGHDRtoSDR cannot create non-existing frames. It simply returns frames requested by the application that is processing the Avisynth script.
DAE avatar
Guest 3
Posts: 67
Joined: Mon Mar 26, 2018 6:00 am

Re: HDR -> SDR conversion

Post by Guest 3 »

As to the
DGHDRtoSDR plugin doesn't work properly
I found some ambiguities, at least for me and I had a successful run with this plugin.
Here is a brief summary from the run.

Code: Select all

>avs2yuv64 -depth 10 -raw DGHDRtoSDR.avs - | x265-64bit-8bit-2018-10-08 - --input-depth 8 --input-res 1920x1080 --input-csp "i420" ... color1.hevc
yuv  [info]: 1920x1080 fps 24000/1001 i420p8 unknown frame count
raw  [info]: output file: color1.hevc
x265 [info]: HEVC encoder version 2.8+74-fd517ae68f93
...
x265 [info]: tools: deblock sao
avisynth 16-bit hack enabled
DGHDRtoSDR.avs: 960x1080, YV12, 10-bits, progressive, 24000/1001 fps, 236 frames
x265 [info]:...
...
encoded 236 frames in 287.14s (0.82 fps), 57349.70 kb/s, Avg QP:16.25

Code: Select all

DGHDRtoSDR.avs
loadplugin("C:\Programs\AviSynth+\plugins64+\dgdecodenv.dll")
loadplugin("C:\Programs\AviSynth+\plugins64+\dghdrtosdr.dll")
DGSource("F:\Gladiator.copy\Gladiator10sec.dgi",fulldepth=true)
DGHDRtoSDR()
ConvertBits(8)
Spline36Resize(1920,1080)
prefetch(6)
Here is a link to Google drive where I've uploaded my test source - Gladiator10sec.hevc
https://drive.google.com/drive/folders/ ... p=sharing
Didn't see download option though. Maybe, it is a default.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin »

Thank you for the source file. The index file sees 238 frame starts but playing the script in Vdub2 delivers only 237 (0-236). That is likely due to the way you have cut the file, i.e., not a clean cut at the end. So I don't see any problem. I won't speculate on what your avs2yuv64 and 16-bit hack stuff may be doing, and I do not provide support for them.

You haven't shown any issue with DGHDRtoSDR. Your ConvertBits(8) call is not needed as DGHDRtoSDR delivers 8-bit by default. Also, if you resize in DGSource() it will run much faster. Or resize your way but before the DGHDRtoSDR call.

Bottom line: no problems found.
User avatar
hydra3333
Posts: 394
Joined: Wed Oct 06, 2010 3:34 am
Contact:

Re: HDR -> SDR conversion

Post by hydra3333 »

hydra3333 wrote:
Fri Oct 05, 2018 7:38 pm
Just a note about the .rar file not opening in 7zip commandline,

Per http://forum.???.org/showthread.php?p=1 ... ost1853939 when I try to manually use standalone z7ip to even list the rar file, it fails.

Would it be possible to consider using .zip or something ?
admin wrote:
Fri Oct 05, 2018 8:30 pm
I'll look into it tomorrow. Thanks for pointing it out. I used to always use ZIP but then I saw lots of people using RAR.
Hello again.
Per this issue with vsrepo crashing https://github.com/vapoursynth/vsrepo/i ... -430883357 when looking for DGHDRtoSDR
  • would it be possible to consider using .zip or something ? :)
  • would it be possible to keep older versions of stuff online for a month or so to provide buffer time for vsrepo to catch up (I haven't looked into vsrepo's mechanism for keeping up to date with new versions, will do so shortly).
  • If a product is non-licensed, maybe could github and tagged releases be a possible alternative, since vsrepo seems to use those OK ?
I really do like it here.
DAE avatar
Guest 3
Posts: 67
Joined: Mon Mar 26, 2018 6:00 am

Re: HDR -> SDR conversion

Post by Guest 3 »

admin wrote:
Wed Oct 17, 2018 4:13 pm
Thank you for the source file. The index file sees 238 frame starts but playing the script in Vdub2 delivers only 237 (0-236). That is likely due to the way you have cut the file, i.e., not a clean cut at the end. So I don't see any problem. I won't speculate on what your avs2yuv64 and 16-bit hack stuff may be doing, and I do not provide support for them.

You haven't shown any issue with DGHDRtoSDR. Your ConvertBits(8) call is not needed as DGHDRtoSDR delivers 8-bit by default. Also, if you resize in DGSource() it will run much faster. Or resize your way but before the DGHDRtoSDR call.

Bottom line: no problems found.
Thank you very much for your help.
I would like to point out the John Hable's post at http://filmicworlds.com/blog/filmic-ton ... er-curves/.
His approach is to give more and easy control over the Filmic Tonemapping curves.
He says:
There are several specific issues I’m hoping to address from the Uncharted 2 curve:
(implemented in Hable tonemap operator)
Simple intuitive controls:
Direct control over dynamic range:
Well behaved curves:
Controls in engine:
Fast, closed form: The curve should be simple and fast to evaluate.
Lerpable parameters:Simple inverse:
Convolve with output gamma:
Code on Github: Makes integration simple.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin »

Anyone working on HDR to SDR already is familiar with the Hable stuff. It's all subjective, and personally I have no interest in a "filmic look" or "crisper blacks".
DAE avatar
Dion
Posts: 28
Joined: Sun Dec 04, 2016 12:30 am

Re: HDR -> SDR conversion

Post by Dion »

Question about this new setting..
hue=10.0 Hue adjustment used to correct hue shift due to the gamut mapping.
Values above 0.0 shift toward red. Values below 0.0 shift toward green. The
default value is 10.0.
Shouldn't it default to 0? Just wondering this change is all.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin »

The gamut mapping was creating a small noticable green bias for some streams; the default hue setting corrects for that. You can set it to 0 if you prefer.
DAE avatar
Dion
Posts: 28
Joined: Sun Dec 04, 2016 12:30 am

Re: HDR -> SDR conversion

Post by Dion »

admin wrote:
Sat Oct 20, 2018 7:22 pm
The gamut mapping was creating a small noticable green bias for some streams; the default hue setting corrects for that. You can set it to 0 if you prefer.
Alright. Yes I do prefer zero.

Seems like the versions before 1.11 has "0" as the default even tho there is no hue option.. Just have to remember to add the hue line to scripts now. :D
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin »

Maybe 10 is a bit too strong. I'll reduce it in the next version.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin »

I slipstreamed 1.11 to reduce default hue to 5. Hopefully, you can live with that. I do think this hue correction is a good thing. If you can show otherwise, please do.
DAE avatar
sv503
Posts: 1
Joined: Thu Jan 31, 2019 1:34 am

Re: HDR -> SDR conversion

Post by sv503 »

Hi, Donald!
Please give recommendations, how to select the parameter "Prefetch"? What does this parameter affect? Thank's.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: HDR -> SDR conversion

Post by admin »

It's an Avisynth+ thing that you can google. There are so many variables affecting it that your best bet is to use Avsmeter to time your script and empirically determine the best value. For some scripts, any prefetch reduces performance.
DAE avatar
Keyser78
Posts: 8
Joined: Thu Feb 07, 2019 1:33 am

Re: HDR -> SDR conversion

Post by Keyser78 »

Hello :-) Which version of Dgsource (dgdecnv) has parameter "fulldepth"? I use 2053, checked others :-( There's no in manual either.
Post Reply