Page 1 of 2

nVidia NGX investigation

Posted: Tue Oct 20, 2020 9:29 am
by Rocky
After a request from astrayred, I tested the NGX video superresolution. It is underwhelming in my view. You can try it. Get these:

http://rationalqm.us/misc/vsr.exe
and
http://rationalqm.us/misc/vsr_input.mp4 (or provide your own)

Then open a CMD window and do:

vsr.exe --input vsr_input.mp4 --factor 2 --output vsr_output.mp4
(--factor 3 is also supported)

I didn't upload vsr_output.mp4 because it is several GB.

Please test, compare to other upsizing methods, and report here. You need an RTX-enabled card.

Re: nVidia NGX investigation

Posted: Wed Oct 21, 2020 1:19 am
by astrayred
Hi Rocky,

Thanks especially as this was my request! I'm still waiting on my 3080 (as you know the stock situation is pretty bad now), but happy to test it once I get it!

Re: nVidia NGX investigation

Posted: Wed Oct 21, 2020 4:14 am
by Rocky
I will also try the ISR (image superresolution) because it could be the AVC encoding that is the cause of the underwhelming result.

Re: nVidia NGX investigation

Posted: Wed Oct 21, 2020 1:55 pm
by Rocky
I just tried the image superresolution and it looks much better, pretty darned good actually. But quite slow. I'll make and post some images and the executable for y'all.

astrayred, what are you looking for? A resizing filter based on it? That's what I am thinking. Are you interested in the frame interpolation stuff too? I don't care about inpainting. :P

Re: nVidia NGX investigation

Posted: Sun Oct 25, 2020 2:19 pm
by DJATOM
It says I'm missing NvDecoder.dll, NvEncoder.dll, avformat-57.dll, avutil-55.dll and avcodec-57.dll files.

Re: nVidia NGX investigation

Posted: Sun Oct 25, 2020 2:45 pm
by Rocky
DJATOM, so great to see you! As great a man as could be imagined. So honored.

Hmm, let's see if I can redistribute those, or give you links to them. But let me get the isr.exe up for you. It shouldn't need all that stuff. I'm watching Sherman right now, ensuring he keeps the miter saw calibrated and doesn't get too sloppy with the adhesives. At $11 per piece, we don't want any goof-ups. He's a good boy. Gonna do great things in this world.

Re: nVidia NGX investigation

Posted: Sun Oct 25, 2020 11:38 pm
by astrayred
Hi Rocky!

I'm interesting mainly in the resizing function, with frame interpolation being a second priority. Many thanks for your hard work and I'm happy to buy you and the team a beer or coffee too :) Can't wait to get my 3080 card to try this out!

Re: nVidia NGX investigation

Posted: Mon Oct 26, 2020 7:52 pm
by Rocky
Thank you, astrayred. That's what I was thinking too. It's a deal.

@all

I have uploaded the image resizing app isr.exe. This is closer to what we would use for a filter.

http://rationalqm.us/misc/isr.exe

If there are missing dependencies let me know and I'll take care of that for you.

Here's a typical command line:

isr.exe --input wolf_2K.jpg --factor 2 --output output.png

Factor can be 2/4/8 IIRC.

Looks pretty good to me! Good enough for y'all? Let's see.

Re: nVidia NGX investigation

Posted: Tue Oct 27, 2020 5:39 am
by DJATOM
https://pastebin.com/4KaU8N49 it still depends on extra libraries :oops:

Re: nVidia NGX investigation

Posted: Tue Oct 27, 2020 9:52 am
by Rocky
Please install the 140 runtimes and report back:

https://www.microsoft.com/en-ca/downloa ... x?id=48145

EDIT: OpenCL stuff should be in your nVidia driver. What version are you running? I run 456.38.

Re: nVidia NGX investigation

Posted: Tue Oct 27, 2020 3:46 pm
by DJATOM
I have all redists installed (2015 and 2017 superseded by 2019). As for driver version,
Image

Re: nVidia NGX investigation

Posted: Tue Oct 27, 2020 4:03 pm
by Rocky
As I understand it, you must have 140 installed. It is hard-coded as a dependency.

If after installing 140 you still have a problem please show the exact output from isr.exe.

Re: nVidia NGX investigation

Posted: Wed Oct 28, 2020 6:26 am
by DJATOM
Image
It says I have a newer version of that product installed. Can't continue.

Re: nVidia NGX investigation

Posted: Wed Oct 28, 2020 6:55 am
by Rocky
Please show your isr.exe command line and exact error output.

Re: nVidia NGX investigation

Posted: Wed Oct 28, 2020 7:05 am
by DJATOM
Well, I managed to fix missing dll stuff (downloaded opencv_word342.dll from internet and copy-renamed it to opencv_core342.dll, opencv_imgcodecs342.dll and opencv_imgproc342.dll), now it outputs
C:\Users\DJATOM\Downloads>isr

NVIDIA NGX ISR Sample


--factor Super resolution factor (2, 4, 8)
--input Input image filename
--output Output image filename
--wd Base directory for image input and output files
But it still doesn't work
isr --factor 2 --input "C:\Users\DJATOM\Desktop\test.vpy - 128870.png" --output "C:\Users\DJATOM\Desktop\test.vpy - 128870.2x.png"

NVIDIA NGX ISR Sample
NVSDK_NGX_Feature_ImageSuperResolution Unavailable on this System
Probably I will have to build proper Opencv dlls to make it work, but I've built it in the past and that's a long process, I have no time for now.

Re: nVidia NGX investigation

Posted: Wed Oct 28, 2020 7:08 am
by Rocky
You shouldn't have to do anything with OCL. Investigating...

Re: nVidia NGX investigation

Posted: Wed Oct 28, 2020 11:54 am
by Rocky
DJATOM what are your OS details please?

Re: nVidia NGX investigation

Posted: Wed Oct 28, 2020 12:01 pm
by Rocky
OK, I know what is going on. Standby for some stuff you need...

Re: nVidia NGX investigation

Posted: Wed Oct 28, 2020 12:08 pm
by Rocky
Please download:

http://rationalqm.us/misc/ISR.rar

Unzip everything into a directory. Open a CMD window in that directory and run your command. I am looking into the right way to distribute NGX stuff but this should get you going with ISR.

Thank you for your assistance, DJATOM!

Re: nVidia NGX investigation

Posted: Thu Oct 29, 2020 5:56 am
by DJATOM

Re: nVidia NGX investigation

Posted: Thu Oct 29, 2020 8:38 am
by Rocky
Sweet. When you've had some time to run tests, we'd love to hear your opinion on whether it is impressive enough to make a filter based on it. Note that most of the slowness is the one-time CUDA and NGX setup/teardown, and the file IO. I'd expect it to be faster when just cranking frames for Avisynth/Vapoursynth.

I don't know if the pre-trained network is optimized for real video or CGI/animation. That will be interesting to look into as well. I suspect the latter given the rapid adoption of DLSS into games.

Re: nVidia NGX investigation

Posted: Thu Oct 29, 2020 10:45 am
by DJATOM
I'm pretty satisfied with results, it's visually as good as eedi3+nnedi3 but probably better and with less computational effort. I'd like to see Vapoursynth implementation for that and use for my anime re-scaling script.

Re: nVidia NGX investigation

Posted: Thu Oct 29, 2020 10:57 am
by DJATOM
https://slow.pics/c/WWEG58rB - new comparison, now added eedi3+nnedi3.
NGX looks slightly sharper and less noisy, but still impressive.

Re: nVidia NGX investigation

Posted: Thu Oct 29, 2020 11:08 am
by Rocky
Very interesting, thank you. Noise reduction could be seen as a positive. I'll go ahead and whip up a filter. Are you looking for HBD support? 4:2:0 good enough?

Re: nVidia NGX investigation

Posted: Thu Oct 29, 2020 11:27 am
by DJATOM
Grayscale 8-16 bit will be pretty enough for me. But if it possible to retain float from GPU, that's even better.
If it requires 4:2:0 input, it's fine. I'll pad input with dummy color planes when needed.