[RESOLVED] Demuxing Matroska sources is slower

Support forum for DGDecNV
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: Demuxing Matroska sources is slower

Post by Rocky »

Go for it, DG!
User avatar
DG
Posts: 88
Joined: Thu Dec 31, 2020 9:55 am

Re: Demuxing Matroska sources is slower

Post by DG »

Got rid of the code duplication. Things look a lot cleaner. Back over to you Rocky. Please test it carefully. :salute:
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: Demuxing Matroska sources is slower

Post by Rocky »

Thanks, DG!

Yeah, I looked at the automatic determination and found some code for it online. But you're right, the code is complex with many corner cases. And it made my poor little Sciuridae brain hurt, so I took the easy way out. What with Sherman gone, what's a squirrel to do?
DAE avatar
rack04
Posts: 32
Joined: Tue Sep 21, 2010 6:45 am

Re: Demuxing Matroska sources is slower

Post by rack04 »

Is there a new test version available to test? I also use DGDecNV to demux from MKV and noticed what I thought was "slow" but assumed it was just part of the process. I just indexed and demuxed a 32GB MKV with h264 and ac3 and it took 0:12:21.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: Demuxing Matroska sources is slower

Post by Rocky »

I'll put a test version up after breakfast in a few hours. It will help only if you have HDDs in the flow, rather than all SSD's.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: Demuxing Matroska sources is slower

Post by Rocky »

Here's a test version. If you have an HDD in your flow, then set the option 'Optimize for HDD'. If you have all SSDs, then don't set this option, as it will reduce performance a little.

Note that there has been very little testing by me yet. I plan to test extensively today. Your feedback will be greatly appreciated. It's important that we get this right, as MKV has become quite important with the greater use of MakeMKV for ripping directly to MKV.

http://rationalqm.us/misc/DGIndexNV_mkvfix.exe
DAE avatar
rack04
Posts: 32
Joined: Tue Sep 21, 2010 6:45 am

Re: Demuxing Matroska sources is slower

Post by rack04 »

Here are my test results. I used "Optimize for HDD" since the source and destination are both HDD.

dgdecnv2053.0.0.222 (64 bit): 0:12:20
dgdecnv2053.0.0.222 (64 bit)_mkvfix: 0:09:09
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: Demuxing Matroska sources is slower

Post by Rocky »

To get valid results, you have to do 'flushmem' and power cycle the disks before every run, otherwise the OS and disk caching come into play.

https://chadaustin.me/flushmem/
DAE avatar
rack04
Posts: 32
Joined: Tue Sep 21, 2010 6:45 am

Re: Demuxing Matroska sources is slower

Post by rack04 »

Rocky wrote:
Sat Jan 02, 2021 9:28 am
To get valid results, you have to do 'flushmem' and power cycle the disks before every run, otherwise the OS and disk caching come into play.

https://chadaustin.me/flushmem/
Results didn't change much.

dgdecnv2053.0.0.222 (64 bit)_mkvfix (flushmen & power cycle): 0:09:33
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: Demuxing Matroska sources is slower

Post by Rocky »

Yes, but do it for the case of no optimize for HDD. That's where we expect to see the slowness. Things also depend on how good your HDDs are.
DAE avatar
rack04
Posts: 32
Joined: Tue Sep 21, 2010 6:45 am

Re: Demuxing Matroska sources is slower

Post by rack04 »

Rocky wrote:
Sat Jan 02, 2021 10:41 am
Yes, but do it for the case of no optimize for HDD. That's where we expect to see the slowness. Things also depend on how good your HDDs are.
Understood. My drives are old WD blue drives so I don't expect anything great.
DAE avatar
Boulder
Posts: 113
Joined: Fri Jul 29, 2011 7:22 am

Re: Demuxing Matroska sources is slower

Post by Boulder »

I've compared against one source with TrueHD audio, no differences in the files produced between the official and test version. The optimized version was as fast as in the previous tests.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: Demuxing Matroska sources is slower

Post by Rocky »

Great to hear, Boulder! Thanks for your testing.
DAE avatar
Boulder
Posts: 113
Joined: Fri Jul 29, 2011 7:22 am

Re: Demuxing Matroska sources is slower

Post by Boulder »

I've noticed that the test version uses quite a lot of memory, and doesn't release it when the job is complete even when the clip is closed. I just demuxed one UHD clip and memory usage is about 4600 MB after it's finished and closed. I haven't checked if this happens all the time or just in this one case.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: Demuxing Matroska sources is slower

Post by Rocky »

Ooh, a memory leak. Nice find, thank you Boulder!

Please re-download DGIndexNV_mkvfix. It should be fixed.

http://rationalqm.us/misc/DGIndexNV_mkvfix.exe
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: Demuxing Matroska sources is slower

Post by Rocky »

Released as slipstream 223. Hoping I didn't break anything. Marking RESOLVED.
Post Reply