Manually select M2TS to demux

DAE avatar
emilius256
Posts: 16
Joined: Mon Feb 06, 2012 2:41 pm

Manually select M2TS to demux

Post by emilius256 »

I have a Movie that have multiple playlist.
A different playlist is played depending on the selected language.

One of the playlist is the one iwould like to demux except for few M2TS file that have hard subtitles, while the same M2TS for another language don't have any hard sub.

It would be possible to manually select the M2TS file to demux?

It would be great also only by command line, for example

DGDemux -i E:\BDMV\STREAM\00093.m2ts+E:\BDMV\STREAM\00056.m2ts+E:\BDMV\STREAM\00057.m2ts

or with an input file with the list of M2TS file to demux.

Thank you for your help.
User avatar
Bullwinkle
Posts: 338
Joined: Thu Sep 05, 2019 6:37 pm

Manually select M2TS to demux

Post by Bullwinkle »

Not currently possible with DGDemux and it will be some time before we can do anything there.

But you can do it with DGIndexNV. Open the playlist using Open disk, then go to file open. You can then delete M2TSs and add M2TSs, using the up/down buttons to get the order you want. Then go to demux menu, select what you want and then Save Project.
DAE avatar
emilius256
Posts: 16
Joined: Mon Feb 06, 2012 2:41 pm

Manually select M2TS to demux

Post by emilius256 »

Wow, I never thought about DGIndexNV.

Thank you so much.
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

Good to hear. I'll leave this thread open to remind me that it would be nice to have in DGDemux too.
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Rocky wrote:
Thu Jun 23, 2022 10:35 am
...to remind me that it would be nice to have in DGDemux too.
Yes indeed that would be really nice, and necessary for some jobs.

Like in the first post when a Bluray has multiple playlists which uses some same M2TS files and some are different.
Known as multi Edition Bluray or there are Angles(wich are supported in DGDemux).

To create a Multi-Edition mkv it is necessary to build a "new and individual" segment chain.
eac3to, TsMuxeR and DGIndexNV supports this as input.

I already support DGIndexNV in my chapterEditor to get the keyframes and it is now time to support DGDemux, because eac3to has issues with eac3 streams and TsMuxeR is not able to handle gaps/overlapped audio streams.

When you implement this feature I can support Multi-Edition mkv creation on Linux, and I'm a big step closer to change fully to Linux.
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

Was wondering what to work on next. This could be it. I also want to address a feature request from SomeHumanPerson.
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

hubblec4, can you give me a specific use case that I can use during development? That would include the disk and link to purchase, an MPLS that serves as the base, the M2TS list for that MPLS, and the modified M2TS list that is desired. For extra credit, explain why those changes to the list are required. Thank you.

@emilius256

If you'd like to weigh in with any additional use case(s), please feel free. I understand one case from your original post.
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

Gave this some thought. To implement a general solution that supports arbitrary addition/deletion, re-ordering, and substitution of M2TS files will be difficult. What could be implemented without too much trauma would be to support only re-ordering and substitution. So your use cases are important to determine if what can be done will meet your needs.

BTW, the editing available in DGIndexNV was designed for the case where a list of M2TS files was opened directly (not through Open Disk). If it is used for a disk open, the chapter and subs timings can be wrong for some edits. Correcting those is a nightmare I don't have the stomach for (remember how much trouble we had with the 'skip first' option).

I'm wondering if something like BDEdit is more suited for these things.
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Rocky wrote:
Fri May 05, 2023 8:34 am
hubblec4, can you give me a specific use case that I can use during development? That would include the disk and link to purchase, an MPLS that serves as the base, the M2TS list for that MPLS, and the modified M2TS list that is desired. For extra credit, explain why those changes to the list are required. Thank you.
For develpment I use mostly this bluray
https://www.amazon.de/Terminator-Blu-ra ... B0036GLFPY
A lot of disc features are on board like Seamless Branching and many codecs and so on.

This movie has 3 editions, a theatrical cut, a directors cut and an extended T-1000 cut.
Each edition has it's own mpls with the corresponding M2TS files.
Mpls file are 00017, 00018, 00019

For Matroska is it easy to save all 3 editions in one mkv.
My chapterEditor works as follows:
A user selects all the editions he wants to preserve and can re-order the positions of the editions.
The first edition is like a main edition, because the order of the used M2TS files is untouched(except the first M2TS can be omitted).
Now will be appended all M2TS files from the second edition without the M2TS files which are already in the chain.
And the same for the third edition.

This new segment list is used by MKVToolNix(MTX) to create the mkv file. Video and the Sup streams are no problem, but MTX don't remove the duplicated Audio frames/content.
To get a proper audio stream I use eac3to which also accept the new segment list.

Yes, without an mpls you have no info about the chapters, but this is in this case OK.
(eac3to shows also only the streams, no chapters)
The original chapters can not be used for this multi-edition mkv.
chapterEditor creates new multi-edition-chapters for playback.


I was not entire correct that TsMuxeR accepts M2TS files. (only the GUI)
I can only use a mpls for the CLI tool and therefore I can't use TsMuxeR to generate multi edition mkv.

I had thought about for a workaround and the best could be to create a new mpls file which contains the new segment list. The mpls format is not so complicated, but I never had time to code that.
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

Thank you but can't you give me the M2TS lists that correspond to the three original MPLS files? And also the lists as modified by you, i.e., the exact list used for the demuxing operations that I must do? I can't really continue the discussion without those. Prefer not to have to purchase the disk if possible. I have more questions but need this info first.

I don't know what you mean by "segment list". Is it some special data/file that MKV is designed to accept? I know only about MPLS playlists with corresponding M2TS and CLPI files.
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Rocky wrote:
Fri May 05, 2023 10:54 am
BTW, the editing available in DGIndexNV was designed for the case where a list of M2TS files was opened directly (not through Open Disk).
And could this be done also in DGDemux?

Rocky wrote:
Fri May 05, 2023 10:54 am
I'm wondering if something like BDEdit is more suited for these things.
Mmh.. I have not much experience with this tool and it is "only" a GUI.
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

Please note and reply to my previous post.

Yes, it can be done but with potential distortion of subs timings, depending on how you edit the M2TS list.
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

I have created a chunk of the disc with my DiscShare tool.

The full disc structure are include and for the 3 mpls files a 2mb chunk of each M2TS file.

Write down here the M2TS list is a much work, because many files are used.
But now you have the mpls files and you can see the list quickly.
The new M2TS chain/list is a combination like I described above, but this is only one way.
Matroska has no limitation of the ordering. You could also sort the M2TS list from low to high.


Segment list means a list of the M2TS files which are used by a mpls.
Maybe is segment map the better word.
Attachments
T2chunk.7z
(113.35 MiB) Downloaded 226 times
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Rocky wrote:
Fri May 05, 2023 4:48 pm
Please note and reply to my previous post.

Yes, it can be done but with potential distortion of chapter and subs timings, depending on how you edit the M2TS list.
yes this was a cross post before. :-)

The timings for the chapters are not important, because chapters no longer useful.
But what is with the timings for the subs?
There are no timing issues for audio or video?
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Yes you are doing it right.

I realized that the topic of this thread is not really what I want.
Manually INPUT of M2TS for demux is more precise, sorry I overlooked that.

I want use the CLI tool only, and selecting is a feature for the GUI, after loading a mpls file.
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

Thank you. But PLEASE I need the original MPLS files (or segment lists) and the ones you've created. It's the only way I can know how you EDITED the M2TS lists, which is how I will know the potential gotchas.

BTW, you can cut and paste from the DGDemuxGUI M2TS list.

I have to head to the pool for a few hours. More later...
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Rocky wrote:
Fri May 05, 2023 5:29 pm
Thank you. But PLEASE I need the original MPLS files (or segment lists) and the ones you've created. It's the only way I can know how you EDITED the M2TS lists.
I have uploaded it some posts above. T2chunk.7z
There are the original mpls files, also the structure and m2ts files of the 3 mpls files.
User avatar
Bullwinkle
Posts: 338
Joined: Thu Sep 05, 2019 6:37 pm

Manually select M2TS to demux

Post by Bullwinkle »

Rocky asked for:

"I need the original MPLS files (or segment lists) and the ones you've created."

Your small disk gives us only the original lists and not the ones after your processing. Why won't you give us both? We need your modified lists so we know what you want to demux.
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Bullwinkle wrote:
Fri May 05, 2023 8:22 pm
Rocky asked for:

"I need the original MPLS files (or segment lists) and the ones you've created."

Your small disk gives us only the original lists and not the ones after your processing. Why won't you give us both? We need your modified lists so we know what you want to demux.
I have no new mpls file or new list, this is all automatically done in the background of cE.
cE combines all M2TS files of the mpls files without doubling an M2TS file.

T2 uses a lot of M2TS files for each edition it would be a big amount of work to filter manually the M2TS files.

The order of the M2TS files is also not important, you can use any order you want.

Such a new Segment list of the M2TS files is also different when a user change the order of the editions.
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

You continue to refuse us the information we need to help you. Therefore, we respectfully decline to do any work for you in this area. We simply do not know what to demux and we're not going to play guessing games. You are asking us to accept a list of M2TS to demux but you won't give us an example case where we see the original M2TS lists and the lists you want to pass to us for demuxing. We don't know or care what cE is doing, we just demux what you ask for and you are not willing to tell us what that is. I cannot understand why you don't understand this. If you wish to re-engage to get something done, then give us the information we ask for.

"it would be a big amount of work"

But you expect us to do a lot of work for you? Unreal. You could easily add a line or two of code in cE to output the final list to the console or a file.

"The order of the M2TS files is also not important"

This must be some kind of joke.

Regarding DGDemux updating for linux, if you won't cooperate with us, why should we with you?
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Rocky wrote:
Sat May 06, 2023 8:10 am

"it would be a big amount of work"

But you expect us to do a lot of work for you? Unreal. You could easily add a line or two of code in cE to output the final list to the console or a file.
Here is the segment list of M2TS file which are used when I use all 3 editions with this order (of mpls/editions)
mpls: 00017 - 00018 - 00019
M2TS: 00000+00001+00002+00221+00030+00031+00005+00006
+00007+00008+00032+00010+00011+00012+00013
+00014+00015+00016+00017+00018+00019+00020
+00021+00022+00023+00024+00025+00026+00027
+00206+00207+00208+00209+00210+00211+00212
+00213+00214+00215+00216+00217+00218+00219
+00220

if the order of the mpls/editions is 00019 - 00017 - 00018 the new Segment list of M2TS:
00000+00001+00002+00221+00030+00031+00005+00006
+00219+00008+00032+00010+00011+00012+00013
+00014+00015+00016+00017+00018+00019+00020
+00021+00022+00023+00024+00220+00026+00027
+00007+00025+00206+00207+00208+00209+00210
+00211+00212+00213+00214+00215+00216+00217
+00218

There are many more combinations possible depending on the order of the editions.

Rocky wrote:
Sat May 06, 2023 8:10 am
"The order of the M2TS files is also not important"

This must be some kind of joke.
Not really, for Matroska and ordered chapters is it no problem to play the correct content also when the M2TS content is fully randomly demuxed.

You could also sort the M2TS list from low to high or high to low.

In DGIndexNV I can use a list of M2TS files and save this project, but I guess there is no feature to skip the duplicated audio frames, right?
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

Thank you. I still don't get why the order doesn't matter. Suppose we have a playlist:

0.m2ts+1.m2ts+2.m2ts

If you demux and play this, all the video etc. is in order. If we change it to:

2.m2ts+1.m2ts+0.m2ts

It will play backwards. How does your player or whatever know the correct order to play things in?

With your edited playlist, how does the player know how to re-stitch things to play a given edition?

Please don't just say 'ordered chapters' because I don't know what they are or how they work.

Finally, how can I do correct gaps correction if the order is arbitrary?
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Rocky wrote:
Sat May 06, 2023 10:27 am
Thank you. I still don't get why the order doesn't matter. Suppose we have a playlist:

0.m2ts+1.m2ts+2.m2ts

If you demux and play this, all the video etc. is in order. If we change it to:

2.m2ts+1.m2ts+0.m2ts

It will play backwards. How does your player or whatever know the correct order to play things in?

With your edited playlist, how does the player know how to re-stitch things to play a given edition?

Please don't just say 'ordered chapters' because I don't know what they are or how they work.
OK, and yes I can understand your confusion.
I came to Matroska because this container supports Ordered Chapters (sorry that I have say ordered chapters).
I will try to explain it short.

An ordered chapter has a start and an end time, and a player MUST play the content given by this timestamps.
The order of the chapters enforces the playback.

For your example, let's say each m2ts has a duration of 60 seconds.

2.m2ts+1.m2ts+0.m2ts
to play this chain in the correct order you have to create ordered chapters as follows:

Edition 1
+ Chapter 1 start 120 sec end 180 sec
+ Chapter 2 start 60 sec end 120 sec
+ Chapter 3 start 0 sec end 60 sec

With ordered chapters is it possible to play an entire movie backwards, looping content, slow-motions, skip content and combine content of all files which are in the same folder.
Video, Audio and Subs are never changed, all the playback magic is mastered by the chapters.


Blu-ray uses more or less the same system.
A mpls file is like an edition and the m2ts IN- and OUT- times the same like the start and end time for Matroska chapters.
There are cases where some mpls files uses the same m2ts files but with different IN and OUT times of the m2ts's.

Rocky wrote:
Sat May 06, 2023 10:27 am
Finally, how can I do correct gaps correction if the order is arbitrary?
Mmh I don't know. I have to less knowledge about this gap-thing which occurs on seamless branching discs.
I have used many many years eac3to, and all the time I get proper synchronized audio streams for such multi edition movies where I use a own virtually m2ts list.

I can only guess that eac3to has an algo which can detect this within the m2ts file.
eac3to has also no mpls file as reference as input.
Also MKVToolNix can handle such large new m2ts list's. When I deselect all audio streams than video, subs and the chapters in synch.

I don't know how your algo works to remove duplicated audio frames. Do you need info from the mpls file for this task? If not than I see no problem how much m2ts file in the chain and in which order they are.
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

Thank you for the explanation. It's clear enough to see what you are doing. Let me now explain about audio gaps correction, and then about what we can do for you.

The problem arises when a stream is divided into many M2TS files. Because the granularity of video and audio access units is different, in a given M2TS the audio and video do not end at the same point. It looks like this (again, this is for one M2TS):

++++++++++++++++++ (video)
---------------------------- (audio)

Now if you have the next M2TS appended, there will be a 'gap'. Normally, when the disk is played there are PTS timestamps so that audio sync can be maintained by the player. However, when demuxing, the PTS timestamps are lost. So in this case, during demuxing that excess audio must be removed. It's generally a very small amount, which would be inconsequential if there are not many M2TS files. However, it accumulates and for a large number of M2TS files, the desync becomes obvious.

My form of gaps correction works like this. As the stream plays we track the amount of video time and audio time accumulated. If the difference exceeds a threshold we decide that audio unit(s) must be dropped. They are dropped from the beginning of the following M2TS. It's like Bresenham's algorithm for line drawing.

So it should be clear that the goal is to maintain sync during playback and the tracking of the amount of accumulated audio and video requires the M2TS's to be demuxed in playing order. Trying to correct a gap between two M2TS files that are not in playing order is meaningless and useless.

OK, so where does that leave us? Let's first note that DGIndexNV's CLI can accept a list of input files and it does perform gaps correction. There is currently no option to disable gaps processing so I cannot predict what may happen with your disordered lists. It may have insignificant effects, or not. You would have to do some experiments.

Now, adding list acceptance to DGDemux would merely duplicate what DGIndexNV CLI can already do, except that (currently) DGIndexNV cannot disable gaps processing. However, there are two additional concerns for you: 1) DGIndexNV requires an nVidia card, and 2) there is no Linux version of DGIndexNV.

So, a possible approach is to add the file list input option to DGDemux and warn users that the results may or may not be acceptable for disordered lists. That would be true with or without gaps processing, because with it, it's doing meaningless things, and without it, you may have the original desync problem possibly occurring. At least you would not need nVidia and you'd have the option to use or not use gaps processing. We could probably give you such a DGDemux test version and then if you find it is usable for your needs, it could be released with appropriate warnings.

Now, the issue of linux support. From my vantage point there has been zero interest in the linux version of DGDemux, and that's why I stopped updating it quite some time ago. Believe me, maintaining a linux system and porting updates from the windows versions is highly odious for me. It's very hard to justify it for one person's needs. So my question for you is, if things never get ported to linux, is it worth proceeding?

Final point. EAC3TO remains an option. The only thing you said that seems to rule it out is "it has issues with eac3". That seems ironic to me, so can you please share what these issues are?

Your move.
User avatar
SomeHumanPerson
Posts: 96
Joined: Fri Mar 24, 2023 10:41 am

Manually select M2TS to demux

Post by SomeHumanPerson »

I'm butting into the conversation here because this subject interests me as well.

"it has issues with eac3"
If you try to demux DD+ (a.k.a. E-AC-3) with eac3to, it throws a warning:

Code: Select all

MKV, 1 video track, 1 audio track, 1:33:29, 24p
1: h264/AVC, English, 1280x688 24p (37:20)
2: EAC3, English, 5.1 channels, 48kHz
v01 The video framerate is correct, but rather unusual.
Bitstream parsing for track 2 failed.
Demuxing this track may still produce correct results - or not.
I don't know what happens if it produces "incorrect results" because I've never bothered, and I also don't know how much of an issue this is when dealing with Blu-rays because I can't remember ever seeing DD+ on a Blu-ray.


Back on the main subject, I'm failing to see how hubblec4's scenario ever works correctly unless MKVToolNix is somehow handling the sync problems by itself (I have no idea on the technical aspect of how that might be done), in which case your "possible approach" of adding the file input list to DGDemux and allowing the user to disable (or leave enabled) gaps processing should be workable, sidestepping the DGIndexNV issues (Nvidia + mandatory gaps processing).

I would be highly interested in that feature for a handful of discs I've found where "skip first" or "skip last" were not sufficient to get rid of trash M2TSs in a playlist (i.e. more than one at start or end). Different use case, but this solution would still work.


Linux support is great when available, but it's understandably a ton of extra work, and I think most people just use VMs to get around platform limitations these days anyway.
Post Reply