DGDemux development

DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Movie arrived a few minutes ago
If you need info/comparisons between original, MakeMKV and AnyDVD please let me know
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Rocky wrote:
Fri Jun 12, 2020 5:37 pm
Great to hear.

Just found that the dependent stream can have its own subtitle stream. Complicates things. Where is Sherman?
Would it be its own or the same subtitle with a depth offset for the 3D effect?
I guess that either way it would be its own subtitle :facepalm:
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

gonca wrote:
Fri Jun 12, 2020 5:44 pm
Movie arrived a few minutes ago
If you need info/comparisons between original, MakeMKV and AnyDVD please let me know
Just need to know if AnyDVD creates full SSIF files on windows. Thank you.
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Yes
Full sized ssif files
ssif.png
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Jolly good. Thank you.
User avatar
Emulgator
Posts: 22
Joined: Tue May 12, 2020 9:39 am

Re: DGDemux development

Post by Emulgator »

In my case of TRON_LEGACY_3D the concerning even numbered m2ts contain only 1 stream, the .mvc stream.
Will check Rataouille 3D, Cars 3D, Men in Black 3D, time permitting.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Not sure I follow your point there, Emulgator. Can you please elaborate? Is it about no extra subtitles in the the MVC M2TS? Well, sometimes there are, and sometimes there are not.

I'm moving towards using just the M2TS files rather than the SSIF. The playlist gives us the matching M2TS MVC file(s) so all we have to do is demux the MVC stream from the matched M2TS file(s). It is all one-to-one so no need to maintain separate file lists, etc.

Using just the M2TS files means that all rippers will be compatible.
User avatar
Emulgator
Posts: 22
Joined: Tue May 12, 2020 9:39 am

Re: DGDemux development

Post by Emulgator »

Yes, I mentioned that only for that case of TRON_LEGACY_3D, I didn't mean to challenge your previous statement.
P.P.S.: Just contributing those cases, no doubt that it could be different with other discs:
TRON_LEGACY_3D: One of the 3D pairs primary 00131.m2ts/secondary 00132.m2ts:
mediainfo shows for 00132.m2ts: MVC stream only, no subs. CABAC, 4 RefFrames, 162Gbps VBR (garbage data), 20.4Mbps max.
MIB_3_3D: One of the 3D pairs primary 00342.m2ts/secondary 00343.m2ts:
mediainfo shows for 00343.m2ts: MVC stream only, no subs. CABAC, 3 RefFrames, 8.6Mbps VBR, 20Mbps max.
RATATOUILLE_3D: One of the 3D pairs primary 01214.m2ts/secondary 01255.m2ts:
mediainfo shows for 01255.m2ts: MVC stream only, no subs. CABAC, 3 RefFrames, 8.4Mbps VBR, 18Mbps max.
CARS_3D: One of the 3D pairs primary 01393.m2ts/secondary 01429.m2ts:
mediainfo shows for 01429.m2ts: MVC stream only, no subs. CABAC, 3 RefFrames, 6.5Mbps VBR, 16Mbps max.

Demuxing the secondary.m2ts directly: yes, this is what I hoped to suggest in my post a page earlier.
In these connected .m2ts the second (even).m2ts seem to contain the dependent view, and only that stream.
(these .m2ts were not playable with MPC-HC32 1.9.3 nor MPC-BE32 4969).
Maybe straightforward from there to demux that elementary.
Just in case somebody wants to go on reconstructing .ssif from main/dependant.m2ts pairs and Makemkv's .ssif.smap:
Looks as if ixceuh already wrote a parser script and a ssif rebuilder:
https://www.makemkv.com/forum/viewtopic ... hilit=ssif
P.S. well, found the links to be dead now, but in this thread mhelin, KurianOfBorg, mike_admin tell a bit about their findings..
Makemkv's .ssif.smap structure seems to tell how to concatenate .ssif from the two main/dependant .m2ts source files:
per line calling the appropriate sourcefile, start offset, length of chunk.
But for the reason of demuxing this seems obsolete now, mounting as ssif should only be needed by the playing application.
Maybe useful for counterchecking only, and to go from .ssif files if anybody has them (as tsmuxeR offers IIRC, although I never used that feature).
User avatar
Emulgator
Posts: 22
Joined: Tue May 12, 2020 9:39 am

Re: DGDemux development

Post by Emulgator »

Just for completeness' sake and maybe it helps:
Some conclusions from 4 ssif.smap tables (still soft, 2020 06 19 CN):

Unit length = 2KiB = 2048B (dec)

Base view chunk length seems to vary around 0x2000 (+0x400 / -0x1000)
Last 2..3 base view chunks can be shorter.
Base view chunk length granularity seems to be 0x20 (=32dec = 64KiB),
except for the last chunk, there it seems to be 0x2 (= 4KiB).
P.S.: CARS_3_3D has 0x1 granularity.

Dependent View chunks come first.
3 of 4 3D-BDs have fixed dependent view chunk length.

Last 2..3 dependent view chunks can be shorter.
Dependent view chunk length granularity seems to be 0x20 (=32dec = 64KiB),
except for the last chunk, there it seems to be 0x2 (= 4KiB).
Last dependent chunk length granularity seems to be 0x2 (=4KiB)
P.S.: CARS_3_3D has 0x1 granularity.
---------------------------

Unknown Disc (see above)
Largest Stereoscopic Pair example Base 00005.m2ts + Dependent 00006.m2ts

Base View 00005.m2ts has variable chunk length range 0x1A40..0x2040 (6720..8256dec)
Dependent View 00006.m2ts has fixed chunk length 0xC60 (=3168dec)
-----------------------------

MIB_3_3D
Largest Stereoscopic Pair example Base 00342.m2ts + Dependent 00343.m2ts

Base View 00342.m2ts has variable chunk length range 0x1140..0x1920
Last 2 base view chunks can be shorter, here 0xE40, 0x13B6

Dependent View 00343.m2ts has fixed chunk length 0xC60 (=3168dec)
Last 2 dependent view chunks can be shorter, here 0x780, 0xA4A
------------------------------

RATATOUILLE_3D:
Largest Stereoscopic Pair example Base 01214.m2ts + Dependent 01255.m2ts

Base View 01214.m2ts has variable chunk length range 0x23A0..0x24C0
Last 2 base view chunks can be shorter, here 0x480, 0x9A8

Dependent View 01255.m2ts has fixed chunk length 0xA80
Last 2 dependent view chunks can be shorter, here 0x480, 0x9A8
-----------------------------

CARS_3_3D:
Largest Stereoscopic Pair example Base 01393.m2ts + Dependent 01429.m2ts

Base View 01393.m2ts has variable chunk length range 0x23A0..0x24C0
Last 3 base view chunks can be shorter, here 0x0F00, 0x1080, 0x24F3 -> Granularity 0x1

Dependent View 01429.m2ts has variable chunk length 0x780..0x900
Last 3 dependent view chunks can be shorter, here 0x320, 0x420, 0x7FE -> Granularity 0x1
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Thank you, Emulgator, for your data. It seems then that subs in the dependent stream are rare or maybe even practically nonexistent. So we'll ignore that for now.
User avatar
Sherman
Posts: 576
Joined: Mon Jan 06, 2020 10:19 pm

Re: DGDemux development

Post by Sherman »

Hey Rocky! After I got back from 1935 last night (talking to Einstein of course and also Podolsky to ask why he misrepresented Einstein's argument in the famous EPR paper -- his answer is shocking!) I whipped up a function called DemuxMVC(filename, pid) to demux from a dependent M2TS file. It does not use any existing data structures so it could be launched as a thread and run simultaneously to the existing video demuxing. I tested it succesfully on the first M2TS in TOY_STORY_4_3D. That means I was able to serve the 3D with DGMVCDecode. The output is the same as from EAC3TO with the exception that I do not remove filler NALUs as EAC3TO does. It's not a big deal as the extra data is not that big. I did not want to have to parse at the video ES level, but just at the transport packet level. And hey, we should demux everything, not start arbitrarily deleting stuff that exists in the source. It's a demuxer, OK?

Still to do is to parse the playlist to detect 3D and get the names of the dependent M2TS files to pair with the main file list. It doesn't have to be a separate list but just another field of the existing file list. Then add code to launch the demux for each file while writing to one output file.
Sherman Peabody
Director of Linux Development
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Cool! Sounds like you have a good handle on things. Why don't you go ahead and finish off the rest?
User avatar
Sherman
Posts: 576
Joined: Mon Jan 06, 2020 10:19 pm

Re: DGDemux development

Post by Sherman »

I'm on it! Will try to finish it off before heading back to meet Honoré de Balzac. Maybe a stop to meet Houdini as well.
Sherman Peabody
Director of Linux Development
User avatar
Sherman
Posts: 576
Joined: Mon Jan 06, 2020 10:19 pm

Re: DGDemux development

Post by Sherman »

Hey Rocky! I got it all done and successfully demuxed TOY_STORY_4_3D and tested the resulting streams with DGMVCDecode(). Maybe you should give a test build before slipstreaming it.
Sherman Peabody
Director of Linux Development
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

That's awesome Sherman. I'll release a test build as you suggest but first I want to have a peek at your code. You know what they say: Trust but verify!

What do you think Bullwinkle, has this kid earned his spurs?
User avatar
Bullwinkle
Posts: 338
Joined: Thu Sep 05, 2019 6:37 pm

Re: DGDemux development

Post by Bullwinkle »

Not a bad effort. I noticed one thing. For TOY_STORY_4_3D there are 52 M2TS files in the main playlist. The progress indication seems to work OK. The progress goes up to 99 and then you get Done! not long after. However, for PROMETHEUS_3D there is only one M2TS and the progress reaches 99, then the MVC output file is opened and demuxed, causing a long delay between 99% and Done!

I realize that this is due to the current progress indication design not really being amenable to this 3D support and the whole progress indication function needs to be redesigned. The actual MVC demuxing is working great. Let's get a test version out as it is and think about what we can do about the progress indication. Or maybe we just document it and say who cares?

Progress indicators are notoriously flaky in many applications. That's why Windows 10 indicators now use the funny circling dots. It's just saying "we're working but we're not done yet". :lol:
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

OK y'all, here is a test version of 3D support. Please be aware of the progress issue. Rename this as DGDemux.exe. DGDemuxGUI.exe is not changed. For 3D disks the base video stream line will show (+ MVC). The MVC stream is always demuxed if the base video is demuxed. Don't use any of the skip options or episode demuxing with 3D disks.

http://rationalqm.us/dgdemux/binaries/D ... c_test.exe

:salute: Sherman
User avatar
Bullwinkle
Posts: 338
Joined: Thu Sep 05, 2019 6:37 pm

Re: DGDemux development

Post by Bullwinkle »

Sherman, you need to add preallocation for the MVC file. It's a small thing but we strive for perfection here in DG land. ;)
User avatar
Sherman
Posts: 576
Joined: Mon Jan 06, 2020 10:19 pm

Re: DGDemux development

Post by Sherman »

So sorry. I'll fix progress and preallocation. I really want those spurs!

Daddy and I are heading for the Wayback Machine right now but I'll take care of it when we return. Balzac wants to abandon his great series The Human Comedy. We're going back to convince him to continue it.

https://en.wikipedia.org/wiki/La_Com%C3%A9die_humaine
[The idiot] John Wilson Croker attacked it in the April 1836 issue of the Quarterly Review, excoriating Balzac for immorality, saying "a baser, meaner, filthier scoundrel never polluted society …"
This Croker dork reminds me of modern quantum mysterians (not going to dignify them by calling them physicists). Let's see what we can do about him when we get there. ;)
Sherman Peabody
Director of Linux Development
User avatar
Curly
Posts: 712
Joined: Sun Mar 15, 2020 11:05 am

Re: DGDemux development

Post by Curly »

You should see my spurs. Hang in there Sherman. I'll spur you on.

Image
Curly Howard
Director of EAC3TO Development
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Demuxed Sanctum from disc with AnyDVD running in the background, and Mad Max Fury Road from HDD (MakeMKV) backup
All appears to work well
Might want to display that it is working, during MVC demux, (Working on it, be patient) kind of thing
Selectable MVC stream would be nice, for those times you only need the base view
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

gonca wrote:
Sat Jun 20, 2020 3:26 pm
Demuxed Sanctum from disc with AnyDVD running in the background, and Mad Max Fury Road from HDD (MakeMKV) backup
All appears to work well
Good to hear, thank you.
Might want to display that it is working, during MVC demux, (Working on it, be patient) kind of thing
We can do better than that. If I showed you the current progress reporting code, I would have to hang my head in shame.
Selectable MVC stream would be nice, for those times you only need the base view
I knew someone would ask for that. :facepalm:
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

My case use is that on rare occasions I had to get the 3D version of a movie due to availability.
I only need the 2D view so I can save some time by demuxing only the base view
Since 4K doesn't support 3D I wont have this problem on new discs, but I still have to deal with the old ones
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Sure, I wasn't doubting you. Your wish is my command. :salute:
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Looks like Sherman won't be back for a while, so I added the preallocation for the MVC stream and made the MVC stream individually selectable. I also changed the whole progress reporting system. What you get now is two boxes. One is just a marquee-style progress control that runs when demuxing is running. The other is a text box that shows the current M2TS file being worked on. You can compare that to the M2TS list to get an idea of where you stand.

I'll make a new test version a bit later today.
Post Reply