nVidia NGX investigation

These CUDA filters are packaged into DGDecodeNV, which is part of DGDecNV.
User avatar
Rocky
Moose Approved
Posts: 1644
Joined: Fri Sep 06, 2019 12:57 pm

nVidia NGX investigation

Post by Rocky »

After a request from astrayred, I tested the NGX video superresolution. It is underwhelming in my view. You can try it. Get these:

http://rationalqm.us/misc/vsr.exe
and
http://rationalqm.us/misc/vsr_input.mp4 (or provide your own)

Then open a CMD window and do:

vsr.exe --input vsr_input.mp4 --factor 2 --output vsr_output.mp4
(--factor 3 is also supported)

I didn't upload vsr_output.mp4 because it is several GB.

Please test, compare to other upsizing methods, and report here. You need an RTX-enabled card.
DAE avatar
astrayred
Posts: 11
Joined: Wed Dec 04, 2013 8:55 pm

Re: nVidia NGX investigation

Post by astrayred »

Hi Rocky,

Thanks especially as this was my request! I'm still waiting on my 3080 (as you know the stock situation is pretty bad now), but happy to test it once I get it!
User avatar
Rocky
Moose Approved
Posts: 1644
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky »

I will also try the ISR (image superresolution) because it could be the AVC encoding that is the cause of the underwhelming result.
User avatar
Rocky
Moose Approved
Posts: 1644
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky »

I just tried the image superresolution and it looks much better, pretty darned good actually. But quite slow. I'll make and post some images and the executable for y'all.

astrayred, what are you looking for? A resizing filter based on it? That's what I am thinking. Are you interested in the frame interpolation stuff too? I don't care about inpainting. :P
User avatar
DJATOM
DG Approved/Moose Approved
Posts: 157
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM »

It says I'm missing NvDecoder.dll, NvEncoder.dll, avformat-57.dll, avutil-55.dll and avcodec-57.dll files.
User avatar
Rocky
Moose Approved
Posts: 1644
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky »

DJATOM, so great to see you! As great a man as could be imagined. So honored.

Hmm, let's see if I can redistribute those, or give you links to them. But let me get the isr.exe up for you. It shouldn't need all that stuff. I'm watching Sherman right now, ensuring he keeps the miter saw calibrated and doesn't get too sloppy with the adhesives. At $11 per piece, we don't want any goof-ups. He's a good boy. Gonna do great things in this world.
DAE avatar
astrayred
Posts: 11
Joined: Wed Dec 04, 2013 8:55 pm

Re: nVidia NGX investigation

Post by astrayred »

Hi Rocky!

I'm interesting mainly in the resizing function, with frame interpolation being a second priority. Many thanks for your hard work and I'm happy to buy you and the team a beer or coffee too :) Can't wait to get my 3080 card to try this out!
User avatar
Rocky
Moose Approved
Posts: 1644
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky »

Thank you, astrayred. That's what I was thinking too. It's a deal.

@all

I have uploaded the image resizing app isr.exe. This is closer to what we would use for a filter.

http://rationalqm.us/misc/isr.exe

If there are missing dependencies let me know and I'll take care of that for you.

Here's a typical command line:

isr.exe --input wolf_2K.jpg --factor 2 --output output.png

Factor can be 2/4/8 IIRC.

Looks pretty good to me! Good enough for y'all? Let's see.
User avatar
DJATOM
DG Approved/Moose Approved
Posts: 157
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM »

https://pastebin.com/4KaU8N49 it still depends on extra libraries :oops:
User avatar
Rocky
Moose Approved
Posts: 1644
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky »

Please install the 140 runtimes and report back:

https://www.microsoft.com/en-ca/downloa ... x?id=48145

EDIT: OpenCL stuff should be in your nVidia driver. What version are you running? I run 456.38.
User avatar
DJATOM
DG Approved/Moose Approved
Posts: 157
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM »

I have all redists installed (2015 and 2017 superseded by 2019). As for driver version,
Image
User avatar
Rocky
Moose Approved
Posts: 1644
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky »

As I understand it, you must have 140 installed. It is hard-coded as a dependency.

If after installing 140 you still have a problem please show the exact output from isr.exe.
User avatar
DJATOM
DG Approved/Moose Approved
Posts: 157
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM »

Image
It says I have a newer version of that product installed. Can't continue.
User avatar
Rocky
Moose Approved
Posts: 1644
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky »

Please show your isr.exe command line and exact error output.
User avatar
DJATOM
DG Approved/Moose Approved
Posts: 157
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM »

Well, I managed to fix missing dll stuff (downloaded opencv_word342.dll from internet and copy-renamed it to opencv_core342.dll, opencv_imgcodecs342.dll and opencv_imgproc342.dll), now it outputs
C:\Users\DJATOM\Downloads>isr

NVIDIA NGX ISR Sample


--factor Super resolution factor (2, 4, 8)
--input Input image filename
--output Output image filename
--wd Base directory for image input and output files
But it still doesn't work
isr --factor 2 --input "C:\Users\DJATOM\Desktop\test.vpy - 128870.png" --output "C:\Users\DJATOM\Desktop\test.vpy - 128870.2x.png"

NVIDIA NGX ISR Sample
NVSDK_NGX_Feature_ImageSuperResolution Unavailable on this System
Probably I will have to build proper Opencv dlls to make it work, but I've built it in the past and that's a long process, I have no time for now.
User avatar
Rocky
Moose Approved
Posts: 1644
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky »

You shouldn't have to do anything with OCL. Investigating...
User avatar
Rocky
Moose Approved
Posts: 1644
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky »

DJATOM what are your OS details please?
User avatar
Rocky
Moose Approved
Posts: 1644
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky »

OK, I know what is going on. Standby for some stuff you need...
User avatar
Rocky
Moose Approved
Posts: 1644
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky »

Please download:

http://rationalqm.us/misc/ISR.rar

Unzip everything into a directory. Open a CMD window in that directory and run your command. I am looking into the right way to distribute NGX stuff but this should get you going with ISR.

Thank you for your assistance, DJATOM!
User avatar
DJATOM
DG Approved/Moose Approved
Posts: 157
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM »

User avatar
Rocky
Moose Approved
Posts: 1644
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky »

Sweet. When you've had some time to run tests, we'd love to hear your opinion on whether it is impressive enough to make a filter based on it. Note that most of the slowness is the one-time CUDA and NGX setup/teardown, and the file IO. I'd expect it to be faster when just cranking frames for Avisynth/Vapoursynth.

I don't know if the pre-trained network is optimized for real video or CGI/animation. That will be interesting to look into as well. I suspect the latter given the rapid adoption of DLSS into games.
User avatar
DJATOM
DG Approved/Moose Approved
Posts: 157
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM »

I'm pretty satisfied with results, it's visually as good as eedi3+nnedi3 but probably better and with less computational effort. I'd like to see Vapoursynth implementation for that and use for my anime re-scaling script.
User avatar
DJATOM
DG Approved/Moose Approved
Posts: 157
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM »

https://slow.pics/c/WWEG58rB - new comparison, now added eedi3+nnedi3.
NGX looks slightly sharper and less noisy, but still impressive.
User avatar
Rocky
Moose Approved
Posts: 1644
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky »

Very interesting, thank you. Noise reduction could be seen as a positive. I'll go ahead and whip up a filter. Are you looking for HBD support? 4:2:0 good enough?
User avatar
DJATOM
DG Approved/Moose Approved
Posts: 157
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM »

Grayscale 8-16 bit will be pretty enough for me. But if it possible to retain float from GPU, that's even better.
If it requires 4:2:0 input, it's fine. I'll pad input with dummy color planes when needed.
Post Reply