DGDemux development

[Britney]

I don't see any problem.

Kisses, Brit

[Mr. Peabody]

Propping up straw men is a not-so-subtle form of rhetoric to be eschewed. I refer of course to the quantum measurement straw man.

[Sherman]

eschewed

Does that mean hard to swallow?

@Rocky

Got PGS demuxing working in fast mode. The fast and slow demuxed SUP files are binary identical. Now have to do THD and embedded AC3. Then a little cleanup and we can give it out.

There is enough supported to fully demux La Notte (Criterion). Here are the timings for demuxing everything:

slow: 5:00
fast: 1:19

That's a 3.8 times speedup!

[Natasha]

That's a 3.8 times speedup!

Is that the best you can do? Pathetic.

[Curly]

Give the kid a break. He's doing his best.

[Bullwinkle]

If you can do better, Nattie, bring it. Otherwise, stand down.

You're doing great Sherman. Moose Approved!

[Curly]

Sabine Hossenfelder wrote:

"The two biggest problems with superdeterminism at the moment are (a) the lack of a generally applicable fundamental theory and (b) the lack of experiment." *

From which we easily conclude that it is metaphysical mumbo-jumbo.

* https://arxiv.org/pdf/2010.01324.pdf

[Levi]

Somebody here said you cannot blame a mosquito for being a mosquito. Makes sense to me.

[Sherman]

Thank you Bullwinkle. I really appreciate your support! And Levi too.

I have the first half of THD working, that is, the do-not-split-THD is working. Now working on splitting out the AC3 stream.

[Sherman]

Now I have THD splitting working. All stream types are now working in fast mode. Still gotta do:

* preallocation
* garbage stripping option
* check that all DELAY values are correct

I see the light at the end of the tunnel!

Nattie, how is your implementation coming along?
:belly-laugh:

[Bullwinkle]

...laying floors...

Ya gotta earn your keep, kid.

I see the light at the end of the tunnel!

I can almost taste it.

Nattie, how is your implementation coming along?

That's a pretty fine stomping for a hoomin. Snort!

[Sherman]

Girls and boys! Rocky gave me permission to release an alpha test version of the fast mode implementation, so here it is:

http://rationalqm.us/dgdemux/binaries/D ... t_test.rar

Notes:

0) To run in fast mode, hold down the left SHIFT key when pressing the Show or Demux buttons. If you use DGDemux.exe directly, include the option '-fast'.

1) I haven't found an efficient way yet to do filler NALU stripping for AVC video, so you will see a difference (which is inconsequential) between the demuxed video in slow and fast mode in this case. The fast code simply processes transport packets one-by-one and has no concept of anything that spans multiple packets (such as NALUs); that's what makes it so fast.

2) All other streams should be bit identical in slow and fast mode.

3) All file names should be the same between slow and fast modes, so change the output name if you want to do both ways to compare (without leaving the GUI).

4) Garbage stripping is not yet implemented.

This is the hardest thing I ever did, including laying floors and installing appliances, so I mighta goofed something up. Your testing and feedback will be greatly appreciated. Enjoy!

Mr Peabody and I are off to visit Albert Einstein again, as there are some loose ends regarding Lorenz invariance I want to clear up, and I want to persuade him to join the forum. See ya in a while!

[Bullwinkle]

Tastes great, less filling!

[Guest]

Tested using 4K version of Wonder Woman

external HDD >> internal SSD
normal 16:00
fast 9:31

suspected that HDD was a bottleneck so I copied the movie to an internal SSD

internal SSD >> internal SSD
normal 8:57
fast 2:59

edit
folders containing each set of the demux were identical in size

[Rocky]

Thank you for the testing, gonca. The ideal situation is SSD->SSD with enough free space to ensure all the preallocations succeed. That's where we see 3-4 times speedup.

The directories will not be exactly the same size when there is AVC video containing filler NALUs (not your case). I have been demuxing into two subfolders (fast and slow) by changing the output path. Then get in the folder holding those two and open a DOS prompt. Execute:

fc /b slow\* fast\*

That will compare all the files. If you have the AVC video with filler get that out before running the compare, or you will be inundated with output.

I have tested two disks so far and all is well. Will test more today.

[Bullwinkle]

Nice work. I found some issues doing MONSTERS_UNIVERSITY:

1. The fast video stream is missing a single byte at the end (weird!).

2. The THD embedded AC3 doesn't match.

3. The SUPs don't match.

BTW, I tested slow mode in the test version versus build 43 and everything matched.

[Sherman]

Thanks, Bullwinkle. I'll fix that up ASAP.

[Sherman]

I fixed the SUP and embedded AC3 demuxing in fast mode. The video discrepancy is actually the old architecture outputing an extra spurious byte at the end of the stream. The fast mode is correct and EAC3TO also doesn't output that byte. I'll do some testing and then give an update for the fast mode test build.

I also fixed a problem with DGDemuxGUI. It was not showing the 'Fix THD gaps' message.

[Bullwinkle]

Keep it up Sherman, you're doing great. Apart from the one-byte video difference MONSTERS demuxes identically between slow and fast modes.

Can you check a few more things, please:

* Proper operation of the skip options in fast mode.
* The FixTHDGaps() function seems slow but maybe it is what it is.

Also, don't forget to implement garbage stripping, and AVC filler NALU stripping (if possible).

[Sherman]

Will do, Bullwinkle. Thank you for your testing.

Meanwhile, here is an updated fast mode build with the aforementioned fixes:

http://rationalqm.us/dgdemux/binaries/D ... t_test.rar

* Fixed SUP demuxing (it was broken only for multiple M2TS files in the playlist).
* Fixed THD embedded AC3 demuxing (gaps correction was not being performed).
* Fixed missing 'Fix THD gaps' message in the GUI.

[Guest]

Gonna copy some movies from my archive and will try to test today

[Sherman]

Sweet. Thank you, gonca. Would be great if you can test a 3D disk, demuxing all streams and just the MVC stream.

[Guest]

Copying one of those as well

[Guest]

Gonna report one disc at a time so my head doesn't explode from the data

VENOM 4K
Primary video in slow is one byte bigger than fast
All other tracks, including DV and PGS, are identical

BUMBLEBEE 4K
Had to use no preallocation
Too many streams and preallocation was using up all free space and then some

One byte issue (I realize it might no be an issue but still reporting)
DV matches
Embedded AC3 matches
all other files match

VENOM 1080p
Size mismatch on AVC file
ac3 files match
PGS files match
can't open dts files

BUMBLEBEE 1080p
AVC made file compare go nuts
all other files match

AGE OF EXTINCTION 3D
AVC file made it go nuts again
*.tag files could not be opened
everything else matches

MVC only
match

[Rocky]

That's fine. Maybe edit the post instead of making a new post for every disk?