[RESOLVED] not able to feed the encoder

Support forum for DGDecNV
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [RESOLVED] not able to feed the encoder

Post by admin »

gonca wrote:
Fri Oct 13, 2017 5:08 pm
DGSourceNV on first card... NVEnc on second card
Ah, so simple.

I think to maximize things you need a mobo with two PCIEx16 slots and lots of PCI lanes.
but you only live once and its only money, gotta have some fun along the way, and this is one of the ways.
Ditto that!
DAE avatar
Guest

Re: [RESOLVED] not able to feed the encoder

Post by Guest »

Ah, so simple
Simple minds, simple solutions.
I think to maximize things you need a mobo with two PCIEx16 slots and lots of PCI lanes
An AMD Ryzen or Threadripper compatible mobo will carry enough PCIEx16 slots and the CPU will support the required lanes

Intel might require an extreme edition CPU with the extra cost

Either way, the key is the NVidia cards and the VPU chip in them
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [RESOLVED] not able to feed the encoder

Post by admin »

i7-8700K + X370 mobo is begging to be my next beast, although it might be worth giving AMD a look, or skipping a generation.

https://www.tweaktown.com/news/59247/in ... index.html

Core i7-8700K: 6C/12T @ 3.7GHz base/4.7GHz boost, 95W TDP.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [RESOLVED] not able to feed the encoder

Post by admin »

The link I gave asserts 40 lanes. Still, I suppose, 64 is better.
DAE avatar
Guest

Re: [RESOLVED] not able to feed the encoder

Post by Guest »

intel website
https://ark.intel.com/products/126684/I ... o-4_70-GHz

They confirm 16 lanes

Interesting read on threadripper re:core count now and possibly later
https://www.extremetech.com/computing/2 ... prise-hood
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [RESOLVED] not able to feed the encoder

Post by admin »

Back to feeding Vapoursynth with DGSource(fulldepth=True)

I've found out why Vapoursynth doesn't like CS_YUV420P16 returned by DGSource(fulldepth=True). Following is the code in avisynth_compat.cpp. It's pretty obvious that Vapoursynth simply does not accept CS_YUV420P16 from Avisynth+. It should be easy to add support.

Code: Select all

static void VS_CC avisynthFilterInit(VSMap *in, VSMap *out, void **instanceData, VSNode *node, VSCore *core, const VSAPI *vsapi) {
    WrappedClip *clip = (WrappedClip *) * instanceData;

    if (!clip->preFetchClips.empty())
        clip->fakeEnv->uglyNode = clip->preFetchClips.front();

    const VideoInfo &viAvs = clip->clip->GetVideoInfo();
    ::VSVideoInfo vi;
    vi.height = viAvs.height;
    vi.width = viAvs.width;
    vi.numFrames = viAvs.num_frames;
    vi.fpsNum = viAvs.fps_numerator;
    vi.fpsDen = viAvs.fps_denominator;
    vs_normalizeRational(&vi.fpsNum, &vi.fpsDen);

    if (viAvs.IsYV12())
        vi.format = vsapi->getFormatPreset(pfYUV420P8, core);
    else if (viAvs.IsYV24())
        vi.format = vsapi->getFormatPreset(pfYUV444P8, core);
    else if (viAvs.IsYV16())
        vi.format = vsapi->getFormatPreset(pfYUV422P8, core);
    else if (viAvs.IsYV411())
        vi.format = vsapi->getFormatPreset(pfYUV411P8, core);
    else if (viAvs.IsColorSpace(VideoInfo::CS_YUV9))
        vi.format = vsapi->getFormatPreset(pfYUV410P8, core);
    else if (viAvs.IsY8())
        vi.format = vsapi->getFormatPreset(pfGray8, core);
    else if (viAvs.IsYUY2())
        vi.format = vsapi->getFormatPreset(pfCompatYUY2, core);
    else if (viAvs.IsRGB32())
        vi.format = vsapi->getFormatPreset(pfCompatBGR32, core);
    else
        vsapi->setError(out, "Avisynth Compat: Only YV12, YUY2 and RGB32 supported");

    vi.flags = 0;
    vsapi->setVideoInfo(&vi, 1, node);
}
It seems we just have to add two lines to the if-else:

Code: Select all

else if (viAvs.IsColorSpace(VideoInfo::CS_YUV420P16))
    vi.format = vsapi->getFormatPreset(pfYUV420P16, core);
That will at least get Vapoursynth to accept the colorspace. Whether it then will correctly take the 16-bit data from Avisynth+ remains to be seen, although it seems like a pretty good bet that it will be OK.

Interestingly, the vs_normalizeRational() call is a bit surprising, considering how not so long ago Myrsloik refused to do that and insisted all filters must do it themselves.
DAE avatar
Guest

Re: [RESOLVED] not able to feed the encoder

Post by Guest »

Not a Vapoursynth user but I tried.
Posted in Myros' thread
Wonder how bad He'll abuse me

Boy, you are patient :bravo:
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [RESOLVED] not able to feed the encoder

Post by admin »

Seems we have to make a Vapoursynth fork. Does anyone have a Windows build environment set up?
DAE avatar
Guest

Re: [RESOLVED] not able to feed the encoder

Post by Guest »

With your help and patience I can try to set one up
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [RESOLVED] not able to feed the encoder

Post by admin »

Thanks, Luis, but I have already started setting it up on my machine.
DAE avatar
Guest

Re: [RESOLVED] not able to feed the encoder

Post by Guest »

If there is anything I can do to help let me know
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [RESOLVED] not able to feed the encoder

Post by admin »

Will do.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [RESOLVED] not able to feed the encoder

Post by admin »

That was easier than I hoped. :lol:

You only have to rebuild the AvsCompat project and then use the AvsCompat.dll to replace the one in the Vapoursynth install directory. With two minor changes (one already given above) I have this script opening and displaying just fine in VirtualDubFilterMod (delivers P016 to VDFM):

import vapoursynth as vs
core = vs.get_core()
core.avs.LoadPlugin(path="D:/Don/Programming/C++/DGDecNV/DGDecodeNV/x64/Release/DGDecodeNV.dll")
video = core.avs.DGSource('d:/tmp/vapoursynth/beach10bit.dgi',fulldepth=True)
video.set_output()

I'll just do a bit more testing and then I will release the patched DLL (with source changes).
DAE avatar
Guest

Re: [RESOLVED] not able to feed the encoder

Post by Guest »

That was quick
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [RESOLVED] not able to feed the encoder

Post by admin »

At my age there's no time to waste. ;)
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [RESOLVED] not able to feed the encoder

Post by admin »

I released the package for high-bit-depth from Avisynth filters via Vapoursynth (see the Binaries Notification thread). If anyone would like to do additional testing that would be helpful.
User avatar
hydra3333
Posts: 394
Joined: Wed Oct 06, 2010 3:34 am
Contact:

Re: [RESOLVED] not able to feed the encoder

Post by hydra3333 »

admin wrote:
Tue Oct 17, 2017 5:34 pm
You only have to rebuild the AvsCompat project and then use the AvsCompat.dll to replace the one in the Vapoursynth install directory. With two minor changes (one already given above) I have this script opening and displaying just fine in VirtualDubFilterMod (delivers P016 to VDFM):

import vapoursynth as vs
core = vs.get_core()
core.avs.LoadPlugin(path="D:/Don/Programming/C++/DGDecNV/DGDecodeNV/x64/Release/DGDecodeNV.dll")
video = core.avs.DGSource('d:/tmp/vapoursynth/beach10bit.dgi',fulldepth=True)
video.set_output()

I'll just do a bit more testing and then I will release the patched DLL (with source changes). Of course we would all prefer that Myrsloik absorbs the changes.
admin wrote:
Wed Oct 18, 2017 6:34 am
I released the package for high-bit-depth from DGDecodeNV via Vapoursynth (see the Binaries Notification thread). If anyone would like to do additional testing that would be helpful.
Thank you. I am not in a position to test high bit depth, but look forward to looking into it when I have a source like that (albeit not in the near future).

I do roll my own patched AvsCompat.dll just to reference some of your new gpu functions ... I see you have included the updated vapoursynth avisynth_compat.cpp and avisynth.h in the latest binaries zip, so I can compare them with vapoursynth's originals and add your patches into my minor ones (providing it doesn't break any licensing). Nice going !
I really do like it here.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [RESOLVED] not able to feed the encoder

Post by admin »

Cool! If you want to tell me the tweaks you made, I can include them in my version too.
DAE avatar
mparade
Posts: 29
Joined: Mon Oct 13, 2014 7:45 am

Re: [RESOLVED] not able to feed the encoder

Post by mparade »

admin wrote:
Fri Oct 13, 2017 3:18 pm

@mparade

Your further testing would be helpful and appreciated. Chow!
I have just started my tests using StaxRip, Avisynth+ and latest DGDecNV with latest NVidia drivers on my 10-bit HDR10 HEVC sources and cards with Pascal architecture. Next week I will buy a HDR TV as well so that I can make more advanced testing.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [RESOLVED] not able to feed the encoder

Post by admin »

Thanks, mparade! In addition to the CUVID fix it would be helpful for you to test the Vapoursynth interface (using the AvsCompat fix I released).

Edit: If you get the latest test build of Vapoursynth R40-test1 you will not need my version of AvsCompat.dll.
User avatar
hydra3333
Posts: 394
Joined: Wed Oct 06, 2010 3:34 am
Contact:

Re: [RESOLVED] Apply project range to audio (for MKV)

Post by hydra3333 »

OK, if you are referring to the last item in this list then I must have been reading it wrongly.
r40:
fixed rgb output sometimes being flipped in avisource
added alpha output settings to avisource, the default is no alpha output
fixed gamma being infinite if not set in levels, bug introduced in r39
removed the hack needed to support avisynth mvtools, the native mvtools has been superior for several years now and removing the hack makes avisynth filter creation much faster
added avisynth+ compatibility
only do prefetching in avfs with vs script when linear access is detected, fixes excessive prefetching that could make opening scripts very slow
is avfs (Avisynth Virtual File System) not something different and unrelated to loading avisynth plugins like yours ?

Ah, sorry, you mean in regard to handling fulldepth rather than in regard to prefetch which I what I was thinking, oops.

Right wrong or indifferent, my patched version of avisynth_compat.cpp ended up with this snippet in regard to prefetch.

Code: Select all

    // MPEG2DEC
    SOURCE(MPEG2Source)
    PREFETCHR0(LumaYV12)
    PREFETCHR0(BlindPP)
    PREFETCHR0(Deblock)

    // Meow
    SOURCE(DGSource)
    PREFETCHR0(DGDenoise)
    PREFETCHR0(DGSharpen)
    // DGBob yadif based deinterlacer http://rationalqm.us/board/viewtopic.php?f=0&t=463&p=6712&hilit=dgbob#p6712  
    // http://rationalqm.us/board/viewtopic.php?f=14&t=559&p=6732&hilit=dgbob#p6732
    temp = int64ToIntS(vsapi->propGetInt(in, "mode", 0, &err));
    //PREFETCH(DGBob, (temp > 0) ? 2 : 1, 1, -2, 2) // close enough?
    // I dont think so.  DG said always previous + current + next ... fudge it a bit for doublerate, what the heck
    switch (temp) {
    case 0:
      PREFETCH(DGBob, 1, 1, -1, 1); break; // single framerate deinterlacing -1,current,+1
    case 1:
      PREFETCH(DGBob, 1, 1, -2, 2); break; // double framerate deinterlacing -2,current,+2 although I reckon hydra3333 got it wrong and its actually -1,current,+1 ... -2,2 can't hurt too badly I guess
    case 2:
      PREFETCH(DGBob, 1, 1, -2, 2); break; // double framerate deinterlacing to single rate (makes slow motion) -2,current,+2 although I reckon hydra3333 got it wrong and its actually -1,current,+1  ... -2,2 can't hurt too badly I guess
    }
    // not sure what to do about PVBob either
    // which is DG's cuda based PureVideo deinterlacer http://rationalqm.us/board/viewtopic.php?f=14&t=559&start=240#p6786 ... act like DGBob for the time being
    temp = int64ToIntS(vsapi->propGetInt(in, "mode", 0, &err));
    switch (temp) {
    case 0:
      PREFETCH(PVBob, 1, 1, -1, 1); break; // single framerate deinterlacing -1,current,+1
    case 1:
      PREFETCH(PVBob, 1, 1, -2, 2); break; // double framerate deinterlacing -2,current,+2 although I reckon hydra3333 got it wrong and its actually -1,current,+1 ... -2,2 can't hurt too badly I guess
    case 2:
      PREFETCH(PVBob, 1, 1, -2, 2); break; // double framerate deinterlacing to single rate (makes slow motion) -2,current,+2 although I reckon hydra3333 got it wrong and its actually -1,current,+1  ... -2,2 can't hurt too badly I guess
    }
    BROKEN(IsCombed)
    PREFETCHR0(FieldDeinterlace)
    PREFETCH(Telecide, 1, 1, -2, 10) // not good
    PREFETCH(DGTelecide, 1, 1, -2, 10) // also not good
    temp = int64ToIntS(vsapi->propGetInt(in, "cycle", 0, &err));
    PREFETCH(DGDecimate, temp - 1, temp, -(temp + 3), temp + 3) // probably suboptimal
    PREFETCH(Decimate, temp - 1, temp, -(temp + 3), temp + 3) // probably suboptimal to
I really do like it here.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [RESOLVED] Apply project range to audio (for MKV)

Post by admin »

Yes, the prefetches were what I was interested in. Thanks!
User avatar
hydra3333
Posts: 394
Joined: Wed Oct 06, 2010 3:34 am
Contact:

Re: [RESOLVED] not able to feed the encoder

Post by hydra3333 »

Hello, a couple of dumb user questions:

I am mis-understanding what's going on with HDR 10bit -> 8bit and "dithering down" to 8bit in DGSource, so a hint or a link would be appreciated if someone could spare the time.

DGSource() manual indicates:
fulldepth: true/false (default: false)
When fulldepth=true and the encoded video is HEVC 10-bit or 12-bit, then DGSource() delivers 16-bit data to Avisynth with the unused lower bits zeroed. The reported pixel format is CS_YUV420P16. If either of the two conditions are not met, then DGSource() delivers 8-bit YV12 or I420 data, as determined by the i420 parameter. When fulldepth=false and the video is HEVC 10-bit or 12-bit, then the video is dithered down to 8-bit for delivery. If you need a reduced color space (less than 16 bits) for your high-bit-depth processing, you can use ConvertBits() as needed after your DGSource() call.
I gather hevc HDR (10/12 bit) to SDR (8 bit) requires "tonemap" type conversion, a la https://forum.doom9.org/showthread.php?p=1831247 ?
Hence I don't understand how to interpret this "When fulldepth=false and the video is HEVC 10-bit or 12-bit, then the video is dithered down to 8-bit for delivery" which seems to "dither down" rather than "tonemap" (poor wording I know, sorry) ?

So I wonder, does 10bit/12bit here not mean HDR10 ? https://en.wikipedia.org/wiki/High-dynamic-range_video

Unless I'm 100% off base, h.264 AVC seems to do 10bit and 12bit HDR too (eg for uploading HDR content to youtube, and maybe there's HDR phones/cameras around now or soon ?) - does that mean DGSource (and say a 1050Ti or 750Ti card; well, a 1050Ti I guess) is capable of decoding mpeg4 AVC 10 bit as well ?
Oh, just spotted this :(
admin wrote:
Fri Jun 02, 2017 6:23 pm
CUVID does not support high-bit-depth AVC.
so I suppose 10/12 bit AVC decoding is off the table.

And ...
admin wrote:
Sun Jun 19, 2016 7:27 am
CUDA and CUVID are not the same thing. I use CUDA for fast color space conversion for display in DGIndexNV. I use CUVID for decoding in both DGIndexNV and DGDecodeNV.
Um, does this imply that gpu accelerated HDR10/HDR10+ -> SDR "tonemap conversion" function is a possibility inside DGSource :) or other DG software ? :)
I really do like it here.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [RESOLVED] not able to feed the encoder

Post by admin »

hydra3333 wrote:
Sat Jan 20, 2018 9:22 pm
I am mis-understanding what's going on with HDR 10bit -> 8bit and "dithering down" to 8bit in DGSource, so a hint or a link would be appreciated if someone could spare the time.
I use the term "dither" in a general way to just mean a bit depth reduction. I don't know the details of the nVidia dithering. It may use "tonemapping". In any case, if it does not serve your needs then you can deliver full-depth and reduce it with other filters.
Hence I don't understand how to interpret this "When fulldepth=false and the video is HEVC 10-bit or 12-bit, then the video is dithered down to 8-bit for delivery" which seems to "dither down" rather than "tonemap" (poor wording I know, sorry) ?
See above.
Um, does this imply that gpu accelerated HDR10/HDR10+ -> SDR "tonemap conversion" function is a possibility inside DGSource :) or other DG software ? :)
See above.

Your remaining questions were answered in a reply to your post at the forum you cited.
Post Reply