nVidia NGX investigation

These CUDA filters are packaged into DGDecodeNV, which is part of DGDecNV.
User avatar
Rocky
Moose Approved
Posts: 1052
Joined: Fri Sep 06, 2019 12:57 pm

nVidia NGX investigation

Post by Rocky » Tue Oct 20, 2020 9:29 am

After a request from astrayred, I tested the NGX video superresolution. It is underwhelming in my view. You can try it. Get these:

http://rationalqm.us/misc/vsr.exe
and
http://rationalqm.us/misc/vsr_input.mp4 (or provide your own)

Then open a CMD window and do:

vsr.exe --input vsr_input.mp4 --factor 2 --output vsr_output.mp4
(--factor 3 is also supported)

I didn't upload vsr_output.mp4 because it is several GB.

Please test, compare to other upsizing methods, and report here. You need an RTX-enabled card.

astrayred
Posts: 10
Joined: Wed Dec 04, 2013 9:55 pm

Re: nVidia NGX investigation

Post by astrayred » Wed Oct 21, 2020 1:19 am

Hi Rocky,

Thanks especially as this was my request! I'm still waiting on my 3080 (as you know the stock situation is pretty bad now), but happy to test it once I get it!

User avatar
Rocky
Moose Approved
Posts: 1052
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky » Wed Oct 21, 2020 4:14 am

I will also try the ISR (image superresolution) because it could be the AVC encoding that is the cause of the underwhelming result.

User avatar
Rocky
Moose Approved
Posts: 1052
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky » Wed Oct 21, 2020 1:55 pm

I just tried the image superresolution and it looks much better, pretty darned good actually. But quite slow. I'll make and post some images and the executable for y'all.

astrayred, what are you looking for? A resizing filter based on it? That's what I am thinking. Are you interested in the frame interpolation stuff too? I don't care about inpainting. :P

User avatar
DJATOM
Moose Approved
Posts: 71
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM » Sun Oct 25, 2020 2:19 pm

It says I'm missing NvDecoder.dll, NvEncoder.dll, avformat-57.dll, avutil-55.dll and avcodec-57.dll files.
RTX 2070 | Ryzen R9 3900X (no OC) | 64 GB RAM

User avatar
Rocky
Moose Approved
Posts: 1052
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky » Sun Oct 25, 2020 2:45 pm

DJATOM, so great to see you! As great a man as could be imagined. So honored.

Hmm, let's see if I can redistribute those, or give you links to them. But let me get the isr.exe up for you. It shouldn't need all that stuff. I'm watching Sherman right now, ensuring he keeps the miter saw calibrated and doesn't get too sloppy with the adhesives. At $11 per piece, we don't want any goof-ups. He's a good boy. Gonna do great things in this world.

astrayred
Posts: 10
Joined: Wed Dec 04, 2013 9:55 pm

Re: nVidia NGX investigation

Post by astrayred » Sun Oct 25, 2020 11:38 pm

Hi Rocky!

I'm interesting mainly in the resizing function, with frame interpolation being a second priority. Many thanks for your hard work and I'm happy to buy you and the team a beer or coffee too :) Can't wait to get my 3080 card to try this out!

User avatar
Rocky
Moose Approved
Posts: 1052
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky » Mon Oct 26, 2020 7:52 pm

Thank you, astrayred. That's what I was thinking too. It's a deal.

@all

I have uploaded the image resizing app isr.exe. This is closer to what we would use for a filter.

http://rationalqm.us/misc/isr.exe

If there are missing dependencies let me know and I'll take care of that for you.

Here's a typical command line:

isr.exe --input wolf_2K.jpg --factor 2 --output output.png

Factor can be 2/4/8 IIRC.

Looks pretty good to me! Good enough for y'all? Let's see.

User avatar
DJATOM
Moose Approved
Posts: 71
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM » Tue Oct 27, 2020 5:39 am

https://pastebin.com/4KaU8N49 it still depends on extra libraries :oops:
RTX 2070 | Ryzen R9 3900X (no OC) | 64 GB RAM

User avatar
Rocky
Moose Approved
Posts: 1052
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky » Tue Oct 27, 2020 9:52 am

Please install the 140 runtimes and report back:

https://www.microsoft.com/en-ca/downloa ... x?id=48145

EDIT: OpenCL stuff should be in your nVidia driver. What version are you running? I run 456.38.

User avatar
DJATOM
Moose Approved
Posts: 71
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM » Tue Oct 27, 2020 3:46 pm

I have all redists installed (2015 and 2017 superseded by 2019). As for driver version,
Image
RTX 2070 | Ryzen R9 3900X (no OC) | 64 GB RAM

User avatar
Rocky
Moose Approved
Posts: 1052
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky » Tue Oct 27, 2020 4:03 pm

As I understand it, you must have 140 installed. It is hard-coded as a dependency.

If after installing 140 you still have a problem please show the exact output from isr.exe.

User avatar
DJATOM
Moose Approved
Posts: 71
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM » Wed Oct 28, 2020 6:26 am

Image
It says I have a newer version of that product installed. Can't continue.
RTX 2070 | Ryzen R9 3900X (no OC) | 64 GB RAM

User avatar
Rocky
Moose Approved
Posts: 1052
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky » Wed Oct 28, 2020 6:55 am

Please show your isr.exe command line and exact error output.

User avatar
DJATOM
Moose Approved
Posts: 71
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM » Wed Oct 28, 2020 7:05 am

Well, I managed to fix missing dll stuff (downloaded opencv_word342.dll from internet and copy-renamed it to opencv_core342.dll, opencv_imgcodecs342.dll and opencv_imgproc342.dll), now it outputs
C:\Users\DJATOM\Downloads>isr

NVIDIA NGX ISR Sample


--factor Super resolution factor (2, 4, 8)
--input Input image filename
--output Output image filename
--wd Base directory for image input and output files
But it still doesn't work
isr --factor 2 --input "C:\Users\DJATOM\Desktop\test.vpy - 128870.png" --output "C:\Users\DJATOM\Desktop\test.vpy - 128870.2x.png"

NVIDIA NGX ISR Sample
NVSDK_NGX_Feature_ImageSuperResolution Unavailable on this System
Probably I will have to build proper Opencv dlls to make it work, but I've built it in the past and that's a long process, I have no time for now.
RTX 2070 | Ryzen R9 3900X (no OC) | 64 GB RAM

User avatar
Rocky
Moose Approved
Posts: 1052
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky » Wed Oct 28, 2020 7:08 am

You shouldn't have to do anything with OCL. Investigating...

User avatar
Rocky
Moose Approved
Posts: 1052
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky » Wed Oct 28, 2020 11:54 am

DJATOM what are your OS details please?

User avatar
Rocky
Moose Approved
Posts: 1052
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky » Wed Oct 28, 2020 12:01 pm

OK, I know what is going on. Standby for some stuff you need...

User avatar
Rocky
Moose Approved
Posts: 1052
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky » Wed Oct 28, 2020 12:08 pm

Please download:

http://rationalqm.us/misc/ISR.rar

Unzip everything into a directory. Open a CMD window in that directory and run your command. I am looking into the right way to distribute NGX stuff but this should get you going with ISR.

Thank you for your assistance, DJATOM!

User avatar
DJATOM
Moose Approved
Posts: 71
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM » Thu Oct 29, 2020 5:56 am

RTX 2070 | Ryzen R9 3900X (no OC) | 64 GB RAM

User avatar
Rocky
Moose Approved
Posts: 1052
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky » Thu Oct 29, 2020 8:38 am

Sweet. When you've had some time to run tests, we'd love to hear your opinion on whether it is impressive enough to make a filter based on it. Note that most of the slowness is the one-time CUDA and NGX setup/teardown, and the file IO. I'd expect it to be faster when just cranking frames for Avisynth/Vapoursynth.

I don't know if the pre-trained network is optimized for real video or CGI/animation. That will be interesting to look into as well. I suspect the latter given the rapid adoption of DLSS into games.

User avatar
DJATOM
Moose Approved
Posts: 71
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM » Thu Oct 29, 2020 10:45 am

I'm pretty satisfied with results, it's visually as good as eedi3+nnedi3 but probably better and with less computational effort. I'd like to see Vapoursynth implementation for that and use for my anime re-scaling script.
RTX 2070 | Ryzen R9 3900X (no OC) | 64 GB RAM

User avatar
DJATOM
Moose Approved
Posts: 71
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM » Thu Oct 29, 2020 10:57 am

https://slow.pics/c/WWEG58rB - new comparison, now added eedi3+nnedi3.
NGX looks slightly sharper and less noisy, but still impressive.
RTX 2070 | Ryzen R9 3900X (no OC) | 64 GB RAM

User avatar
Rocky
Moose Approved
Posts: 1052
Joined: Fri Sep 06, 2019 12:57 pm

Re: nVidia NGX investigation

Post by Rocky » Thu Oct 29, 2020 11:08 am

Very interesting, thank you. Noise reduction could be seen as a positive. I'll go ahead and whip up a filter. Are you looking for HBD support? 4:2:0 good enough?

User avatar
DJATOM
Moose Approved
Posts: 71
Joined: Fri Oct 16, 2015 6:14 pm

Re: nVidia NGX investigation

Post by DJATOM » Thu Oct 29, 2020 11:27 am

Grayscale 8-16 bit will be pretty enough for me. But if it possible to retain float from GPU, that's even better.
If it requires 4:2:0 input, it's fine. I'll pad input with dummy color planes when needed.
RTX 2070 | Ryzen R9 3900X (no OC) | 64 GB RAM

Post Reply