DGDecNV (linux)

DG tools for linux
User avatar
Rocky
Moose Approved
Posts: 1877
Joined: Fri Sep 06, 2019 12:57 pm

DGDecNV (linux)

Post by Rocky »

Successfully built and ran the minimal decoder with NALU feeding:

don@debian:~/cuVideoDecode$ g++ -static-libstdc++ -static-libgcc -O3 -o decode cuh264dec.cpp nalu.cpp /usr/lib/x86_64-linux-gnu/libnvcuvid.so /usr/lib/x86_64-linux-gnu/libcuda.so
don@debian:~/cuVideoDecode$ ./decode Alba.264 Alba.yuv
GPU in use: NVIDIA GeForce RTX 2080 Ti
Processed 580 frames
don@debian:~/cuVideoDecode$ ls -l Alba.yuv
-rw-r--r-- 1 don don 1817395200 May 22 12:10 Alba.yuv
don@debian:~/cuVideoDecode$

:belly-laugh:

Next step: Install and test Avisynth+ and Vapoursynth.
Next-next step: Port DGSource().
Next-next-next step: Create linux native CLI indexer (to avoid DGIndexNV under wine).
Next-next-next-next step: Port the other filters in DGDecodeNV.dll.
User avatar
Rocky
Moose Approved
Posts: 1877
Joined: Fri Sep 06, 2019 12:57 pm

DGDecNV (linux)

Post by Rocky »

@DJATOM

Hey DJ, once I get Avisynth+/Vapoursynth installed, what do you recommend for viewing the scripts?
User avatar
gonca
DG Approved/Curly Approved/Moose Approved
Posts: 1102
Joined: Sun Apr 08, 2012 6:12 pm

DGDecNV (linux)

Post by gonca »

For Vapoursynth, I believe Vapoursynth Editor is cross platform
https://bitbucket.org/mystery_keeper/va ... rc/master/

Or Vapoursynth Editor 2
https://bitbucket.org/gundamftw/vapours ... rc/master/
User avatar
DJATOM
DG Approved/Moose Approved
Posts: 167
Joined: Fri Oct 16, 2015 6:14 pm

DGDecNV (linux)

Post by DJATOM »

Yeah, first version is what I'm using to preview fast. For proper script development I'm using stack of VSCode + Vapoursynth preview (VSP): https://github.com/Endilll/vapoursynth-preview
I don't know what to suggest for avisynth tho, probably its possible to make AvsPmod for Linux, it's python2 based and should be crossplatform in mind, yet it might rely on Windows APIs. What I have tested so far is https://github.com/DJATOM/avs2yuv
User avatar
gonca
DG Approved/Curly Approved/Moose Approved
Posts: 1102
Joined: Sun Apr 08, 2012 6:12 pm

DGDecNV (linux)

Post by gonca »

For viewing AVS+ scripts a native text editor might work
User avatar
Rocky
Moose Approved
Posts: 1877
Joined: Fri Sep 06, 2019 12:57 pm

DGDecNV (linux)

Post by Rocky »

Thanks guys. Probably won't bother with Avisynth+, at least initially.
User avatar
gonca
DG Approved/Curly Approved/Moose Approved
Posts: 1102
Joined: Sun Apr 08, 2012 6:12 pm

DGDecNV (linux)

Post by gonca »

@DJATOM
avs2yuv is for avisynth
What do you use for vapoursynth
User avatar
DJATOM
DG Approved/Moose Approved
Posts: 167
Joined: Fri Oct 16, 2015 6:14 pm

DGDecNV (linux)

Post by DJATOM »

vspipe
User avatar
gonca
DG Approved/Curly Approved/Moose Approved
Posts: 1102
Joined: Sun Apr 08, 2012 6:12 pm

DGDecNV (linux)

Post by gonca »

DJATOM wrote:
Mon May 24, 2021 6:32 am
vspipe
Makes sense, it is part of Vapoursynth
Thanks
User avatar
Rocky
Moose Approved
Posts: 1877
Joined: Fri Sep 06, 2019 12:57 pm

DGDecNV (linux)

Post by Rocky »

DJ, can you help with how to do this:

"(if you want to build ffms2 plugin, make sure ffmpeg stuff is installed before next step)"

Thank you!
User avatar
DJATOM
DG Approved/Moose Approved
Posts: 167
Joined: Fri Oct 16, 2015 6:14 pm

DGDecNV (linux)

Post by DJATOM »

You can use that guide to build ffmpeg - https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu
It seems ffms2 is separated plugin, for some reason I thought it was bundled with main vapoursynth repo.
You can get sources here - https://github.com/FFMS/ffms2, which should compile with default options

Code: Select all

./autogen.sh
./configure
make -j$(nproc)
sudo make install
User avatar
Rocky
Moose Approved
Posts: 1877
Joined: Fri Sep 06, 2019 12:57 pm

DGDecNV (linux)

Post by Rocky »

Great, thank you!
User avatar
Rocky
Moose Approved
Posts: 1877
Joined: Fri Sep 06, 2019 12:57 pm

DGDecNV (linux)

Post by Rocky »

Vapoursynth successfully installed:

don@debian:~/vapoursynth$ python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from vapoursynth import core
>>> print(core.version())
VapourSynth Video Processing Library
Copyright (c) 2012-2021 Fredrik Mellbin
Core R53
API R3.6
Options: -

>>>

:belly-laugh:

OK, got it installed and working. But heavens above. Took four hours with many false starts and dead ends, and bits and pieces of information and fixes found in many websites. DJ's instructions are helpful but far from complete. Your average user is likely not going to be able to do this and I cannot give an exact account of what I had to do, due to the aforementioned false starts and dead ends. Perhaps we can console ourselves with the thought that if you use linux you are by definition not an average user.

Immediate next steps: install ffms2, install vapoursynth editor (or previewer), and try to serve Alba.264.

After that we'll be ready to bring up DGSource(). But first, it's backup time!

Grandpa always used to say: "Slow and steady wins the race."
User avatar
Curly
Moose Approved
Posts: 139
Joined: Sun Mar 15, 2020 11:05 am

DGDecNV (linux)

Post by Curly »

Rocky wrote:
Mon May 24, 2021 2:00 pm
don@debian:~/vapoursynth$ python3
Shouldn't that be rocky@debian?
User avatar
Rocky
Moose Approved
Posts: 1877
Joined: Fri Sep 06, 2019 12:57 pm

DGDecNV (linux)

Post by Rocky »

You know we all share all the accounts. Anyone can log in as videoh, even you (behave!). Get over it.
User avatar
Rocky
Moose Approved
Posts: 1877
Joined: Fri Sep 06, 2019 12:57 pm

DGDecNV (linux)

Post by Rocky »

FFMS2 and Vapoursynth Editor successfully installed; video served:

from vapoursynth import core
video = core.ffms2.Source(source='/home/don/cuVideoDecode/Alba.264')
video.set_output()

Image

:belly-laugh:

Everything is now in place to start porting DGSource(). But first, it's backup time!
User avatar
Rocky
Moose Approved
Posts: 1877
Joined: Fri Sep 06, 2019 12:57 pm

DGDecNV (linux)

Post by Rocky »

Rocky wrote:
Tue May 25, 2021 11:21 am
Everything is now in place to start porting DGSource().
Actually, we needed one more thing before proceeding to port DGSource(): how to compile a filter into a *.so file usable by Vapoursynth. Couldn't find much on that with google so I figured it out myself (assisted by https://www.cprogramming.com/tutorial/s ... x-gcc.html). :ugeek:

Starting with the invert_example.c in ~/vapoursynth/sdk, I did this:

* Edit invert_example.c to include the full path to the vapoursynth headers. There's
probably a better way but I am a squirrel.
* gcc -c -Wall -Werror -fPIC invert_example.c
* gcc -shared -o libinvert_example.so invert_example.o
* Now, this script performs inversion:

from vapoursynth import core
video = core.ffms2.Source(source='/home/don/cuVideoDecode/Alba.264')
core.std.LoadPlugin('/home/don/vapoursynth/sdk/libinvert_example.so')
video = core.invert.Filter(video) # source code calls it 'Filter' not 'Invert'
video.set_output()

:belly-laugh:

Now I know how to build DGSource() to make a usable *.so file. You'da thought they would have a Makefile in sdk but don't forget, linux is a priesthood.
User avatar
Rocky
Moose Approved
Posts: 1877
Joined: Fri Sep 06, 2019 12:57 pm

DGDecNV (linux)

Post by Rocky »

I've stripped DGDecodeNV to a minimum for porting. Removed all the accessory filters and Avisynth support. That stuff may come back but for now it greatly simplifies things and eases the porting task. The stripped version still runs correctly with Vapoursynth on windows.
User avatar
gonca
DG Approved/Curly Approved/Moose Approved
Posts: 1102
Joined: Sun Apr 08, 2012 6:12 pm

DGDecNV (linux)

Post by gonca »

Why not make the accessory filters standalone in Windows and Linux
Might make it easier to manage
Just an idea
User avatar
Rocky
Moose Approved
Posts: 1877
Joined: Fri Sep 06, 2019 12:57 pm

DGDecNV (linux)

Post by Rocky »

gonca wrote:
Wed May 26, 2021 4:22 pm
Why not make the accessory filters standalone in Windows and Linux
Might make it easier to manage
Just an idea
Yes, that's what we have in mind. Everything will support native Vapoursynth as well.

To be honest, the dual Avisynth/Vapoursynth DLL is not worth the effort and complication. Applying fixes to separate code bases is actually less painful than (artificially) combining things.
User avatar
Rocky
Moose Approved
Posts: 1877
Joined: Fri Sep 06, 2019 12:57 pm

DGDecNV (linux)

Post by Rocky »

DGDecodeNV has two projects: DGSource and mp4v2. Decided to port mp4v2 first and all the files are now compiling. Gonna generate a shared library and test it cursorily before going for DGSource, because it has less WIN32 dependencies compared to DGSource.
User avatar
Rocky
Moose Approved
Posts: 1877
Joined: Fri Sep 06, 2019 12:57 pm

DGDecNV (linux)

Post by Rocky »

MP4V2 library built and tested:

First, successully built libmp4v2.so. And then:

don@debian:~/Projects/mp4v2$ echo $LD_LIBRARY_PATH
/usr/local/lib
don@debian:~/Projects/mp4v2$ sudo cp libmp4v2.so /usr/local/lib

Now, a simple test app:

Code: Select all

// test.cpp
#include <stdio.h>
#include "mp4.h"

int main(int argc, char **argv)
{
        MP4FileHandle mp4File;
        MP4TrackId trackId;
        const char *name = NULL;

        if (argc < 2)
        {
                printf("Usage: test file.mp4\n");
                exit(1);
        }
        printf("Opening %s\n", argv[1]);
        mp4File = MP4Read(argv[1], 0);
        if (!mp4File)
        {
                printf("Open failed\n");
                exit(1);
         }
        uint32_t numTracks = MP4GetNumberOfTracks(mp4File);
        printf("Number of tracks = %d\n", numTracks);
        for (int i = 0; i <numTracks; i++)
        {
                trackId = MP4FindTrackId(mp4File, i);
                name = MP4GetTrackMediaDataName(mp4File, trackId);
                printf("%d: %s\n", trackId, name);
        }
        return 0;
}

don@debian:~/Projects/mp4v2$ make test
g++ -o test test.cpp libmp4v2.so
don@debian:~/Projects/mp4v2$ ./test dreamtime.mp4
Opening dreamtime.mp4
Number of tracks = 2
1: mp4v
2: mp4a

:belly-laugh:

Not bad for a linux n00b. :o

I'm just using the source code I have in DGDecodeNV and I haven't tried to use any existing mp4v2 linux packages I have seen on line. Pedal to the metal, baby!

OK, let's go for DGSource now. Lots of WIN32 stuff to convert (mostly thread creation and events).
User avatar
DJATOM
DG Approved/Moose Approved
Posts: 167
Joined: Fri Oct 16, 2015 6:14 pm

DGDecNV (linux)

Post by DJATOM »

Great! For Event stuff, you might want to look at https://github.com/rigaya/NVEnc/blob/ma ... _event.cpp. I'm using that solution for my x265's vapoursynth reader :)
User avatar
Rocky
Moose Approved
Posts: 1877
Joined: Fri Sep 06, 2019 12:57 pm

DGDecNV (linux)

Post by Rocky »

That looks interesting. Thank you!

Any CreateThread() equivalent?
User avatar
DJATOM
DG Approved/Moose Approved
Posts: 167
Joined: Fri Oct 16, 2015 6:14 pm

DGDecNV (linux)

Post by DJATOM »

https://github.com/DJATOM/x265-aMod/blo ... g.cpp#L138
Maybe that can be used as refence for your unified threading class, I don't know exact replace for it
Post Reply