Page 14 of 17

Re: DGDenoise

Posted: Wed Nov 11, 2020 4:36 pm
by Rocky
Yes, I duplicated it. It's not horribly bad as the value is changed only by 1 or 2 but it should be fixed.

Re: DGDenoise

Posted: Wed Nov 11, 2020 8:05 pm
by Rocky
Hehe. I got lucky and found the color shift problem 10 seconds after opening DGDenoise.cu for inspection. Gotta regression test and make sure it's theoretically correct (seems to be but wanna be thorough about it). Thanks for pointing it out. It may point to similar issues in my other CUDA filters, too, so I am really grateful for this report, Sharc. :salute:

The edge effect is another matter. It is insignificant for reasonable strength values. e.g., it is absent at cstrength 0.5 and really prominent at cstrength 5.0. Strength 0.9 is quite strong. Still, I'll look at that tomorrow.

Re: DGDenoise

Posted: Thu Nov 12, 2020 5:27 am
by Rocky
Can you please test this? Both problems should be fixed. Thank you, Sharc. You are my knight in shining armor!

http://rationalqm.us/misc/DGDecodeNV_Sharc.dll

Rename it properly of course. This is the 64-bit DLL. Let me know if you need 32-bit. When you bless things, I'll make a slipstream.

BTW, DGBob() is the only other affected filter. That will be fixed too when I slipstream.

Re: DGDenoise

Posted: Thu Nov 12, 2020 1:07 pm
by Sharc
It's perfect now, thank you :hat:
And yes it would be great if you would include the 32 bit .dll in the official slipstream. I am still using much of 32 bit avisynth, so far.

Re: DGDenoise

Posted: Thu Nov 12, 2020 2:50 pm
by Rocky
Jolly good, I will make a slipstream later this evening after swimming. Thank you!

Re: DGDenoise

Posted: Fri Nov 13, 2020 1:48 pm
by Rocky
Darn, I went to test things in 32-bit and found that the Release mode crashes but the Debug mode succeeds. That's gonna make debugging hard.

Can you please test the released DGDecodeNV.dll DGDenoise in 32-bit. Is it OK for you (other than the chroma shift)? I tried it but it crashed so I'm thinking I may have a 32-bit Avisynth+ issue, because if it was that broken someone would have told me. If it works for you please tell me your Avisynth+ version.

Thank you.

Re: DGDenoise

Posted: Fri Nov 13, 2020 2:36 pm
by Rocky
Looks like a problem in BlankClip (or maybe 32-bit VirtualDub), because BlankClip() crashes but DGSource() does not.

#source=BlankClip(color=color_magenta, width=720, height=576, pixel_type="YV12") # crashes in 32-bit release mode
source=DGSource("Alba.dgi") # OK in 32-bit release mode
b=source.DGDenoise(strength=1.0,cstrength=1.0,searchw=5)
interleave(source.Histogram("levels"),b.Histogram("levels"))

Please advise your test results.

Re: DGDenoise

Posted: Fri Nov 13, 2020 2:44 pm
by Rocky
Getting interesting now! This fails:

loadplugin("D:\Don\Programming\C++\DGDecNV\DGDecodeNV\x32\Release\dgdecodenv.dll")
source=BlankClip(color=color_magenta, width=720, height=576, pixel_type="YV12") #or any other color
b=source.DGDenoise(strength=1.0,cstrength=1.0,searchw=5)
interleave(source.Histogram("levels"),b.Histogram("levels"))

But this succeeds:

loadplugin("D:\Don\Programming\C++\DGDecNV\DGDecodeNV\x32\Release\dgdecodenv.dll")
tmp=DGSource("Alba.dgi") # not used
source=BlankClip(color=color_magenta, width=720, height=576, pixel_type="YV12") #or any other color
b=source.DGDenoise(strength=1.0,cstrength=1.0,searchw=5)
interleave(source.Histogram("levels"),b.Histogram("levels"))

So it looks like DGSource() initialization is doing something that DGDenoise requires. Investigating...

Re: DGDenoise

Posted: Fri Nov 13, 2020 5:06 pm
by Sharc
I noticed another oddity:
It crashes when I load the script an run it the first time -> Access violation.
Then I have to change any of the DGDenoise parameters and from then onwards it works.
I had this before, but got so much used to this oddity that I just used it that way ... assuming it's on my system only :belly-laugh:

Re: DGDenoise

Posted: Fri Nov 13, 2020 8:42 pm
by Rocky
You're a funny guy, Sharc! Love ya. You have the system from hell. Not!

Please immediately report access violations experienced with DG tools. Hey, I know it's a once-in-a-blue-moon thing, but do what ya gotta do. :twisted:

Sherman says we should just drop 32-bit. Oh, the impetuousness of youth.

Re: DGDenoise

Posted: Fri Nov 13, 2020 9:00 pm
by Curly
Tomorrow is another day.

And Sunday is a big occasion at the Moose Lodge. Albert has been invited to give the keynote speech. We're going to grill him about the DG situation during the Q&A. Natasha says it will be like squeezing blood from an orange. Anemic!

DGDenoise

Posted: Fri Nov 13, 2020 9:04 pm
by Britney
Leave Albert alone!


Re: DGDenoise

Posted: Sat Nov 14, 2020 8:07 am
by Rocky
All right, it appears to be a bug in the older CUDA libraries I use (to support older cards). DGSource() always performs cuInit() but if the script doesn't use DGSource() then the other filters have to perform the cuInit(). I had code in them that calls cuDeviceGetCount() as the first thing. It is supposed to return CUDA_ERROR_NOT_INITIALIZED if cuInit() has not been previously called [by DGSource()]. If I get that error the filter calls cuInit(). OK, sounds great, and it works for 64-bit, but for 32-bit that call is crashing instead of returning the error code.

I have a workaround that is working and hope to make a slipstream later today. I have to first test all the filters in both 32- and 64-bit, as well as testing chains of filters with and without DGSource().

Re: DGDenoise

Posted: Sun Nov 15, 2020 5:04 am
by Sharc
I confirm that the 32 bit .dll no longer crashes here. All ok now :D

DGDenoise

Posted: Sun Nov 15, 2020 5:10 am
by Rocky
Great to hear. Thank you for your report and testing.

Green border

Posted: Tue Feb 01, 2022 1:11 pm
by Guest 2
I gave a try to DGDenoise as simple filter for anime:

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 2)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\1_33 Porco Rosso\porco.dgi",ct=24,cb=24,cl=0,cr=0)
ConvertBits(16)
DGDenoise(strength=0.5,cstrength=0.15,blend=0.1,cblend=0.1,searchw=9)
fmtc_bitdepth (bits=10,dmode=8)
Prefetch(6)

and it gave me a nice green border.

Any idea?

Image

DGDenoise

Posted: Wed Feb 02, 2022 10:45 am
by Rocky
Hmm, looks like chroma denoising is broken in DGDenoise() for P16. Get rid of cstrength (either remove it or set it to zero) until I fix this. Also, DGDenoise() accepts YV12 so you don't need the ConvertBits(), unless you need it for the rest of the script.

Is that a progressive bluray? If it's a DVD don't forget to decomb before denoising.

Good to see another Hayao Miyazaki fanboy. ;)

DGDenoise

Posted: Wed Feb 02, 2022 12:00 pm
by Rocky
Please re-download DGDecNV 239 and update DGDecodeNV.dll. Should be fixed. Thank you for pointing this out. Chroma denoising was broken for all YUV420P16 streams.

DGDenoise

Posted: Wed Feb 02, 2022 12:18 pm
by Guest 2
Rocky wrote:
Wed Feb 02, 2022 10:45 am
Also, DGDenoise() accepts YV12 so you don't need the ConvertBits(), unless you need it for the rest of the script.
Doesn't it work more precisely with higher bitplanes? If not, hint hint :)
Rocky wrote:
Wed Feb 02, 2022 10:45 am
Is that a progressive bluray?
Yup.
Rocky wrote:
Wed Feb 02, 2022 10:45 am
Good to see another Hayao Miyazaki fanboy. ;)
:agree:

DGDenoise

Posted: Wed Feb 02, 2022 12:30 pm
by Rocky
Guest 2 wrote:
Wed Feb 02, 2022 12:18 pm
Doesn't it work more precisely with higher bitplanes?
Nope. The low bits are filled with zeros anyway when upconverting the bit depth.

DGDenoise

Posted: Wed Feb 02, 2022 12:33 pm
by Guest 2
Rocky wrote:
Wed Feb 02, 2022 12:30 pm
Nope.
Please check the zip file because it seems you put x86 build in main directory instead of x64.

Edit: well, it seems fixed now :)

DGDenoise

Posted: Wed Feb 02, 2022 12:46 pm
by Rocky
Guest 2 wrote:
Wed Feb 02, 2022 12:33 pm
Please check the zip file because it seems you put x86 build in main directory instead of x64.
Nah, probably PEBKAC on your side. Am I wrong?

Don't you hate it when you drop your last pistachio nut on the floor and can't find it? You have that unfinished feeling all day.

DGDenoise

Posted: Wed Feb 02, 2022 12:50 pm
by Natasha
Blind squirrels can find acorns but full sighted Rocky can't find a pistachio. What's up with that?

DGDenoise

Posted: Wed Feb 02, 2022 12:50 pm
by Guest 2
Rocky wrote:
Wed Feb 02, 2022 12:46 pm
Nah, probably PEBKAC on your side. Am I wrong?
Perhaps :mrgreen:
Rocky wrote:
Wed Feb 02, 2022 12:46 pm
Don't you hate it when you drop your last pistachio nut on the floor and can't find it? You have that unfinished feeling all day.
Gremlins built an entire miniature car with all the stuff I lost under kitchen furniture. :lol:

DGDenoise

Posted: Wed Feb 02, 2022 12:52 pm
by Bullwinkle
I dropped a load on the floor and had no problem finding it.