Page 13 of 17

Re: DGDenoise

Posted: Sun Oct 25, 2020 8:21 am
by Sharc
Rocky wrote:
Sun Oct 25, 2020 6:51 am
Good point about the zero value. Will do. I'll do it for both luma and chroma strength and then we can ditch the chroma option.
Thank you.
What temporal filter are you using?
temporalsoften(),fluxsmoothST(),CnR2(), mainly for VHS sources, filtering the even/odd grouped fields.

Re: DGDenoise

Posted: Mon Oct 26, 2020 4:16 am
by Sharc
DGDenoise introduces a slight color shift towards lower values in the histogram, for color=true.
Also, some color artefacts appear at the very bottom and right side of the filtered picture, like a bar of few pixels.
Is this to be expected, or is it on my system only?

Script for testing, for example

Code: Select all

source=BlankClip(color=color_magenta, width=720, height=576, pixel_type="YV12")  #or any other color
a=source
b=source.DGDenoise(strength=0.001,cstrength=0.9,searchw=5,chroma=true)
interleave(a.Histogram("levels"),b.Histogram("levels"))

Re: DGDenoise

Posted: Mon Oct 26, 2020 7:55 pm
by Rocky
Thank you, Sharc. That behavior is not good. I'm calling on Sherman to look into this. Multitasking -- it's a good thing!

Re: DGDenoise

Posted: Wed Nov 11, 2020 4:00 am
by Sharc
Thank you for the new version of DGDenoise.
The slight color shift and border effect seem still to be there, at least on my system .... :scratch:

Re: DGDenoise

Posted: Wed Nov 11, 2020 5:39 am
by Rocky
I'll look into that today.

Re: DGDenoise

Posted: Wed Nov 11, 2020 4:36 pm
by Rocky
Yes, I duplicated it. It's not horribly bad as the value is changed only by 1 or 2 but it should be fixed.

Re: DGDenoise

Posted: Wed Nov 11, 2020 8:05 pm
by Rocky
Hehe. I got lucky and found the color shift problem 10 seconds after opening DGDenoise.cu for inspection. Gotta regression test and make sure it's theoretically correct (seems to be but wanna be thorough about it). Thanks for pointing it out. It may point to similar issues in my other CUDA filters, too, so I am really grateful for this report, Sharc. :salute:

The edge effect is another matter. It is insignificant for reasonable strength values. e.g., it is absent at cstrength 0.5 and really prominent at cstrength 5.0. Strength 0.9 is quite strong. Still, I'll look at that tomorrow.

Re: DGDenoise

Posted: Thu Nov 12, 2020 5:27 am
by Rocky
Can you please test this? Both problems should be fixed. Thank you, Sharc. You are my knight in shining armor!

http://rationalqm.us/misc/DGDecodeNV_Sharc.dll

Rename it properly of course. This is the 64-bit DLL. Let me know if you need 32-bit. When you bless things, I'll make a slipstream.

BTW, DGBob() is the only other affected filter. That will be fixed too when I slipstream.

Re: DGDenoise

Posted: Thu Nov 12, 2020 1:07 pm
by Sharc
It's perfect now, thank you :hat:
And yes it would be great if you would include the 32 bit .dll in the official slipstream. I am still using much of 32 bit avisynth, so far.

Re: DGDenoise

Posted: Thu Nov 12, 2020 2:50 pm
by Rocky
Jolly good, I will make a slipstream later this evening after swimming. Thank you!

Re: DGDenoise

Posted: Fri Nov 13, 2020 1:48 pm
by Rocky
Darn, I went to test things in 32-bit and found that the Release mode crashes but the Debug mode succeeds. That's gonna make debugging hard.

Can you please test the released DGDecodeNV.dll DGDenoise in 32-bit. Is it OK for you (other than the chroma shift)? I tried it but it crashed so I'm thinking I may have a 32-bit Avisynth+ issue, because if it was that broken someone would have told me. If it works for you please tell me your Avisynth+ version.

Thank you.

Re: DGDenoise

Posted: Fri Nov 13, 2020 2:36 pm
by Rocky
Looks like a problem in BlankClip (or maybe 32-bit VirtualDub), because BlankClip() crashes but DGSource() does not.

#source=BlankClip(color=color_magenta, width=720, height=576, pixel_type="YV12") # crashes in 32-bit release mode
source=DGSource("Alba.dgi") # OK in 32-bit release mode
b=source.DGDenoise(strength=1.0,cstrength=1.0,searchw=5)
interleave(source.Histogram("levels"),b.Histogram("levels"))

Please advise your test results.

Re: DGDenoise

Posted: Fri Nov 13, 2020 2:44 pm
by Rocky
Getting interesting now! This fails:

loadplugin("D:\Don\Programming\C++\DGDecNV\DGDecodeNV\x32\Release\dgdecodenv.dll")
source=BlankClip(color=color_magenta, width=720, height=576, pixel_type="YV12") #or any other color
b=source.DGDenoise(strength=1.0,cstrength=1.0,searchw=5)
interleave(source.Histogram("levels"),b.Histogram("levels"))

But this succeeds:

loadplugin("D:\Don\Programming\C++\DGDecNV\DGDecodeNV\x32\Release\dgdecodenv.dll")
tmp=DGSource("Alba.dgi") # not used
source=BlankClip(color=color_magenta, width=720, height=576, pixel_type="YV12") #or any other color
b=source.DGDenoise(strength=1.0,cstrength=1.0,searchw=5)
interleave(source.Histogram("levels"),b.Histogram("levels"))

So it looks like DGSource() initialization is doing something that DGDenoise requires. Investigating...

Re: DGDenoise

Posted: Fri Nov 13, 2020 5:06 pm
by Sharc
I noticed another oddity:
It crashes when I load the script an run it the first time -> Access violation.
Then I have to change any of the DGDenoise parameters and from then onwards it works.
I had this before, but got so much used to this oddity that I just used it that way ... assuming it's on my system only :belly-laugh:

Re: DGDenoise

Posted: Fri Nov 13, 2020 8:42 pm
by Rocky
You're a funny guy, Sharc! Love ya. You have the system from hell. Not!

Please immediately report access violations experienced with DG tools. Hey, I know it's a once-in-a-blue-moon thing, but do what ya gotta do. :twisted:

Sherman says we should just drop 32-bit. Oh, the impetuousness of youth.

Re: DGDenoise

Posted: Fri Nov 13, 2020 9:00 pm
by Curly
Tomorrow is another day.

And Sunday is a big occasion at the Moose Lodge. Albert has been invited to give the keynote speech. We're going to grill him about the DG situation during the Q&A. Natasha says it will be like squeezing blood from an orange. Anemic!

DGDenoise

Posted: Fri Nov 13, 2020 9:04 pm
by Britney
Leave Albert alone!


Re: DGDenoise

Posted: Sat Nov 14, 2020 8:07 am
by Rocky
All right, it appears to be a bug in the older CUDA libraries I use (to support older cards). DGSource() always performs cuInit() but if the script doesn't use DGSource() then the other filters have to perform the cuInit(). I had code in them that calls cuDeviceGetCount() as the first thing. It is supposed to return CUDA_ERROR_NOT_INITIALIZED if cuInit() has not been previously called [by DGSource()]. If I get that error the filter calls cuInit(). OK, sounds great, and it works for 64-bit, but for 32-bit that call is crashing instead of returning the error code.

I have a workaround that is working and hope to make a slipstream later today. I have to first test all the filters in both 32- and 64-bit, as well as testing chains of filters with and without DGSource().

Re: DGDenoise

Posted: Sun Nov 15, 2020 5:04 am
by Sharc
I confirm that the 32 bit .dll no longer crashes here. All ok now :D

DGDenoise

Posted: Sun Nov 15, 2020 5:10 am
by Rocky
Great to hear. Thank you for your report and testing.

Green border

Posted: Tue Feb 01, 2022 1:11 pm
by Guest 2
I gave a try to DGDenoise as simple filter for anime:

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 2)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\1_33 Porco Rosso\porco.dgi",ct=24,cb=24,cl=0,cr=0)
ConvertBits(16)
DGDenoise(strength=0.5,cstrength=0.15,blend=0.1,cblend=0.1,searchw=9)
fmtc_bitdepth (bits=10,dmode=8)
Prefetch(6)

and it gave me a nice green border.

Any idea?

Image

DGDenoise

Posted: Wed Feb 02, 2022 10:45 am
by Rocky
Hmm, looks like chroma denoising is broken in DGDenoise() for P16. Get rid of cstrength (either remove it or set it to zero) until I fix this. Also, DGDenoise() accepts YV12 so you don't need the ConvertBits(), unless you need it for the rest of the script.

Is that a progressive bluray? If it's a DVD don't forget to decomb before denoising.

Good to see another Hayao Miyazaki fanboy. ;)

DGDenoise

Posted: Wed Feb 02, 2022 12:00 pm
by Rocky
Please re-download DGDecNV 239 and update DGDecodeNV.dll. Should be fixed. Thank you for pointing this out. Chroma denoising was broken for all YUV420P16 streams.

DGDenoise

Posted: Wed Feb 02, 2022 12:18 pm
by Guest 2
Rocky wrote:
Wed Feb 02, 2022 10:45 am
Also, DGDenoise() accepts YV12 so you don't need the ConvertBits(), unless you need it for the rest of the script.
Doesn't it work more precisely with higher bitplanes? If not, hint hint :)
Rocky wrote:
Wed Feb 02, 2022 10:45 am
Is that a progressive bluray?
Yup.
Rocky wrote:
Wed Feb 02, 2022 10:45 am
Good to see another Hayao Miyazaki fanboy. ;)
:agree:

DGDenoise

Posted: Wed Feb 02, 2022 12:30 pm
by Rocky
Guest 2 wrote:
Wed Feb 02, 2022 12:18 pm
Doesn't it work more precisely with higher bitplanes?
Nope. The low bits are filled with zeros anyway when upconverting the bit depth.