DGDemux development

DAE avatar
dragon
Posts: 2
Joined: Fri Nov 15, 2019 3:11 am

Re: DGDemux development

Post by dragon »

Rocky wrote:
Sun Dec 15, 2019 11:19 am
Got the garbage skipping working for CROUPIER. I'd like to ask users whether they think garbage skipping should be optional, i.e., should there be an option to not skip garbage? I always prefer not to have unnecessary options. Can anyone see a use case where it might be desirable to retain content outside the playlist in<-->out times? After this is decided I'll make a release.

BTW, I also enhanced the GUI to do an automatic re-browse when the skip first or skip last boxes are checked/unchecked after an MPLS has been loaded. Much more friendly!
By garbage skipping do you mean cases were "skip first" is applicable? Like Kino Lorber discs that have 11.010s m2ts clip at the beginning that causes audio/subtitle out-of-sync issue because intro m2ts just consists of the video stream and not the audio/subtitle streams?
If that's what you mean, then I'd say that it's not always an issue. Sometimes intro/outro content is useful and skipping them is not a good option.

I do agree with you though, I also prefer for there not to be any unnecessary options. I'm not a huge fan of -sf and -sl options. They kind of bloat the program in my view.

As far as understand, this particular issue arises only when m2ts files that in the playlist don't consist of the same streams. That particular situation raises not only the out-of-sync problem, but other problems such as when some of the m2ts files have 640kbps ac3 audio for example, and some have 448kbps, in the same playlist. What is the output in that case? The way eac3to deals with it is it re-encodes the audio into 640kbps using libAften encoder that it comes with.

Both problems stem from non-consistent list of streams. I think one decent way of dealing with the first problem (out-of-sync issue due to intro) would be for dgdemux to scan m2ts files before demuxing starts, if it discovers that list of streams in all m2ts files of the playlist aren't consistent, then it asks you something like this:

Code: Select all

dgdemux -i MOVIE\BDMV\PLAYLIST\00000.mpls -o output\

Code: Select all

dgdemux
The streams in 00000.mpls [00001.m2ts + 00003.m2ts + 00006.m2ts] aren't consistent:

00001.m2ts		00003.m2ts
1011: Video [AVC]	1011: Video [AVC]		✔
			1100: DTS HDMA L+R 48 1536	✘
			1101: PGS subtitles [eng]	✘
Do you want demux 00000.mpls entirely or skip first m2ts? [d] - demux, [sf] - skip first, [sl] - skip last: ...
Something similar or same could be done for skip last. I think this is a better way then adding command line options that are applicable for less then 1% of blu-ray discs.
Especially when sometimes, 00001.m2ts may not cause issues if audio/video tracks in 00003.m2ts have delay added to them, some studios do this, and in cases like that, demuxing the entire mpls even when first m2ts has no audio/subtitle tracks causes no issues as long as dgdemux respects the delay of the playlist on those tracks.

The second issue, where the audio bit rate/type changes between m2ts files is more difficult to solve I think. I haven't tested how dgdemux handles those, I assume it won't re-encode like eac3to does, and it can't "combine" 2 different bit rate audios so I assume it will demux first one, and skip the second one.
for example, if mpls is 00001.m2ts+00002.m2ts+00003.m2ts, and if 00001.m2ts & 00002.m2ts both have AC3@640kbps audio, but 00003.m2ts has AC3@448kbps audio, I assume dgdemux will demux audio just from first two m2ts files and skip the last one, or it will have an error. But again, I haven't tested it so I don't know. There should be a warning in any case, like the above warning when there are inconsistent streams in m2ts files, so that user is aware that demuxed audio is not viable, then he can decide how to handle it. (hopefully in future, there will be a detailed log where all this will be logged.)
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Garbage refers to content in a playlist that is outside the range defined by the IN_time of the first M2TS and the OUT_time of the last M2TS. This is distinct from -sf and -sl. The IN_time of an M2TS need not correspond to the first video access unit of an M2TS. For example, CROUPIER (one M2TS in the playlist) has almost a full minute of colors bars and blackness before the IN_time, and then some more blackness after the OUT_time.

I appreciate your detailed info and requests, dragon. However, I'm not going to revisit things we have already discussed and decided. -sf and -sl are staying. There are too many weird cases and corner cases to try to automate things. Always prefer the human mind. Also I'm not going to do anything based on just your theory and experience. You will need to cite specific disks with links to purchase and say how they are being misprocessed in your view.
User avatar
Bullwinkle
Posts: 338
Joined: Thu Sep 05, 2019 6:37 pm

Re: DGDemux development

Post by Bullwinkle »

"Always prefer the human mind."

Snort! Moose minds work way better. Moose are awesome!

https://www.youtube.com/watch?v=9T27xFSsxxs
https://www.youtube.com/watch?v=JS2MNeUSDa8

Natasha told me she could help me become a moderator. She knows Forum Mr. Big!
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

That's great, Bullwinkle, but we really need a GUI concept for the playlist scanning. Let's go!
User avatar
Boris
Posts: 92
Joined: Sun Nov 10, 2019 2:55 pm

Re: DGDemux development

Post by Boris »

Pottsylvanian coders have already solved your problems. Natasha is explaining it to Bullwinkle right now. He is very excited.

Will it be curtains for Bullwinkle, or we all learn to code?
User avatar
redbtn
Posts: 44
Joined: Sat Nov 16, 2019 2:35 pm

Re: DGDemux development

Post by redbtn »

Does Bullwinkle already know how to do this?
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Haven't seen him for a while. Hope he's OK!
User avatar
Bullwinkle
Posts: 338
Joined: Thu Sep 05, 2019 6:37 pm

Re: DGDemux development

Post by Bullwinkle »

Sorry, Rock, here I am. Natasha was going to help me become a moderator, but then she tried to bite me! So I gave her a little stomping and she limped away. Think about it!

The design is all worked out for playlist scanning. You're gonna like it!
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

That's great, Bullwinkle! I have to get a DGDemux release out for CROUPIER. Then Xunkar found some weird things in DGIndexNV program stream parsing. Then whee!
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Bullwinkle gave me a nice GUI concept for playlist scanning and I was able to get it working. Here is how it looks. I still have a few little things to do and then I will make a release. Time difference from BDINFO is likely due to accumulated rounding of each playlist item. No big deal for now; I'll fix that later. Playlists shorter than 10 minutes, or ones that have duplicated items are suppressed.

Image
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Playlists shorter than 10 minutes
In future can this be made user selectable?
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Sure it can be done. Is there a good reason for it?
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Series discs, by adjusting the time we can filter out most of the extras and leave the actual episodes
Average 1 hr slot episode is 45 to 50 min so one could use 35 minutes
1/2 hr time slot episode is 20 to 25 minutes so we could use 15 minutes
That is the basic idea
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

All those times are greater than 10 minutes. BDINFO uses 20 seconds by default. Maybe I should do that?
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

The idea is to not show the extras in episodic discs, so making the cut off length closer to the episode time would make it easier to identify the playlists that point to the episodes
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Doesn't the MPLS duration give you the information you need?

I am very reluctant to add what appear to me to be superfluous options.
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Yes, but for example
Episodic disc has 4 episodes from a series > 4 mpls plus 1 for play all
It can also have a bunch of extras, cast commentaries, deleted scenes etc.
Could be a large number of mpls files
The easiest way to find the required mpls files is by filtering by a minimum time or sorting by mpls duration
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

I don't mind sorting by duration.
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Okay, that works too.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Good! I'm making the threshold 5 minutes and sorting by descending order of duration. Already implemented and tested. :D
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Here is DGDemux/DGDemuxGUI 1.0.0.14:

* Added playlist scanning functionality. Playlists are listed in descending order by duration.

* Made open dialogs modal.

* Fixed logic of -sf and -sl such that the file list cannot be made empty.

* Refresh when the skip checkboxes are changed. You no longer have to re-open the input.

* Fix crash for very large M2TS list.

* GUI reorganization.

* Documents updated.

Note that I have not yet pulled the trigger on garbage skipping. It is coded but it needs extensive testing.

http://rationalqm.us/dgdemux/DGDemux_1.0.0.14.rar
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Looks good
You can see the six episodes (about 43 min)
series.png
and the movie mpls is obvious
movie.png
User avatar
redbtn
Posts: 44
Joined: Sat Nov 16, 2019 2:35 pm

Re: DGDemux development

Post by redbtn »

Thank you Rocky! Nice release! I'm about to test it.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Thank you, guys!
User avatar
redbtn
Posts: 44
Joined: Sat Nov 16, 2019 2:35 pm

Re: DGDemux development

Post by redbtn »

I found a problem with space allocation. I'm demuxing BD which is 40gb. If on ssd not enough space for allocate all files (for example I have 43gb, and I checked 4 audio streams and video) free space is gone while demuxing and DGDemux says "Done", but video stream is only 3gb. No errors are displayed.
I guess there should be an error. And there is have to be a way to demux 40gb BD, when you have less than 75+10*4=115gb free space. I think 4*10+30(video stream size) is enough to demux without problems (I guess audio streams allocate first, so 40gb is used and then error to allocate 75gb, continue without allocation, and if on ssd enough space for that (30gb), it can work well). But maybe we can have an option to disable allocation?
I hope you will understand, my English is very bad, haha)
Post Reply