Problem with multi-edition MKV creation

[hubblec4]

Hi Curly

eac3to works only for the english DTS track, not for the german EAC3 track.
The EAC3 track has the same audio issue, a loud hum.

If you also work on this issue for eac3to, I can open a separate thread in the eac3to section if you like.

[Curly]

Oh, that changes things. The idea to just compare DGDemux and eac3to won't be as useful. Let's see.

[Shoespooner]

Oh, that changes things. The idea to just compare DGDemux and eac3to won't be as useful. Let's see.

I believe that it could be quite useful. We need to differentiate between editions and audio tracks. The key to understanding is that every edition contains all audio tracks. That is, the German edition contains the English and the German audio tracks, and the English edition contains the Englisch and the German audio tracks as well.

A summary of the situation:

In the English edition, there are no problems, neither in the English nor in the German audio track, if DGDemux is used. In the English edition, there is also no problem in the English audio track if Eac3to is used. The German audio track in the English edition cannot be tested with Eac3to, because the track is E-AC3 which Eac3to seems to have problems with.

In the German edition, there are problems with the English as well as with the German audio track if DGDemux is used. But in the German edition, there are no problems with the English audio track if Eac3to is used. As with the English edition, the German audio track cannot be tested with Eac3to in the German edition.

So we know at least the following: In the German edition, the English audio track shows no problems if Eac3to is used, but shows problems if DGDemux is used.

That may make it worth to compare the output from DGDemux and Eac3to in this respect.

Best regards, and thank you very much!

[Rocky]

@hubblec4

Are we concerned only with PIDs 1100 and 1101?

EDIT: I don't have any problem extracting the German audio from the English edition (what hubs calls edition 1, 00800.mpls?) using eac3to 3.46_test. It probably works in 3.45 too. Are you using 3.36 or something? Don't, because I cannot support that and don't have code for it. I only have 3.36+ first supplied to me by madshi, from which I forked all the later versions.

[Shoespooner]

Are we concerned only with PIDs 1100 and 1101?

Yes, correct.

The audio track PIDs in question are 4352 (English audio, DTSHD-MA) and 4353 (German audio, E-AC3), in decimal notation. In hexadecimal notation, the PIDs are 1100 and 1101.

There are other additional audio tracks as well, but I don't care about them and would like to remove them, because they are either Italian (which I don't speak and currently don't plan to learn), quality-reduced versions of other existing streams (why should I keep them if I already keep the "full" versions?), or commentaries (that I am not interested in).

EDIT: I don't have any problem extracting the German audio from the English edition (what hubs calls edition 1, 00800.mpls?) using eac3to 3.46_test. It probably works in 3.45 too. Are you using 3.36 or something? Don't, because I cannot support that and don't have code for it. I only have 3.36+ first supplied to me by madshi, from which I forked all the later versions.

To be honest (and this is quite embarrassing for me), I personally didn't test different versions of Eac3to. Instead, because I have no clue about all this, I have just believed what hubble4c has said and what I have read in a few other posts. There were statements that Eac3to has problems with E-AC3 tracks.

@hubblec4

Could you please give some details regarding the exact problem that occurs when Eac3to 3.46 demuxes E-AC3 tracks? Do you have example data (command line + media file) that enables @Rocky to reproduce the problem?

Thank you very much, and best regards!

[Rocky]

I've duplicated it in DGDemux. It's a short low-frequency pop, though, not a loud hum. Investigating...

[Rocky]

I see from an earlier post hubblec4 says the German EAC3 track demuxes fine with 3.46. So I guess there are no issues for multi-edition with eac3to. OK.

[Rocky]

Testing eac3to for edition 2 English audio, the pop does NOT occur, as claimed. OK. Now there is an acorn to gnaw on.

[Rocky]

I've tested all cases:

1. EAC3TO edition 1 eng OK
2. EAC3TO edition 1 ger OK
3. EAC3TO edition 2 eng OK
4. EAC3TO edition 2 ger POP
5. DGDemux edition 1 eng OK
6. DGDemux edition 1 ger OK
7. DGDemux edition 2 eng POP
8. DGDemux edition 2 ger POP

So it's complex because it's not just DGDemux versus EAC3TO. I need to try to find the cause. So I am going to focus on case 3 versus 7 for now.

The video streams have different lengths but I have to check if that is significant or not.

[Rocky]

I MKV-muxed the DGDemux video and German audio but with the eac3 English audio. The POP is no longer there when playing edition 2 English. So at least for this case it's something different about the two English audio tracks. I'm going to binary diff them and try to see why they produce different results.

[Rocky]

I tried disabling gaps correction on DGDemux. It had no effect.

This is going to be difficult. I'll have to think for a while about how to proceed. Just diffing the audio is not so insightful because I don't really know whether the differences I see are significant or are just due to differences in the gaps correction. And without knowing what the players are doing at the switch to one of the extra M2TSs I'm groping in the dark.

We do know that eac3to is not completely innocent.

Hey, three POPs for the whole movie is not so bad.

[hubblec4]

I think we don't need talk longer about eac3to3.36, because it is the old unsupported version.
The new eac3to version from Curly works for this issue same as the old eac3to version.

Edition 1 is the 00800.mpls and is the English movie version.
Edition 2 is the 00801.mpls and is the German movie version.

For Edition 1 all streams are fine, because the used m2ts files for this movie version in the correct order.
For Edition 2 is only the English track working when eac3to is used(in my test I have used version 3.45, now I could use the newest version 3.46, but I guess the result will be the same). The German EAC3 track is not working(means has the audio POP issue) for eac3to, and DGDemux has issues with both tracks.

I hope this helps a bit.

[hubblec4]

I've duplicated it in DGDemux. It's a short low-frequency pop, though, not a loud hum. Investigating...

Sounds good Rocky.

[hubblec4]

I tried disabling gaps correction on DGDemux. It had no effect.

This is going to be difficult. I'll have to think for a while about how to proceed. Just diffing the audio is not so insightful because I don't really know whether the differences I see are significant or are just due to differences in the gaps correction. And without knowing what the players are doing at the switch to one of the extra M2TSs I'm groping in the dark.

I can well imagine that this isn't all that easy, and I myself would only like to understand this properly.

Because I think that in the end it might be easier to remove the excess audio data from an mkv file, since everything is stored there much more simply, not as complicated as in m2ts format (ok, that's just my humble opinion because I don't so detailed m2ts understand).

And yes, the player can also play an important role, although you also have to look at how the jump is made in the mkv being played and how the correct timestamps are accessed.
However, I have already created so many multi-edition mkvs and they have no audio issues, so I trust that the good players will do it right.

We do know that eac3to is not completely innocent.

Yes.

[hubblec4]

I've tested all cases:

1. EAC3TO edition 1 eng OK
2. EAC3TO edition 1 ger OK
3. EAC3TO edition 2 eng OK
4. EAC3TO edition 2 ger POP
5. DGDemux edition 1 eng OK
6. DGDemux edition 1 ger OK
7. DGDemux edition 2 eng POP
8. DGDemux edition 2 ger POP

A perfect overview, great.

[Shoespooner]

I've duplicated it in DGDemux. It's a short low-frequency pop, though, not a loud hum. Investigating...

Thank you very much for all your tests!

I am still sure that the pop is not random noise. Although it may seem like a weird low-frequency sound, it is the last few milliseconds from the audio at the end of chapter 3. In the multi-edition MKV, chapter 3 (in edition 2) physically ends at 02:07:12.124, where chapter 22 (in edition 2) begins. This enables us to verify the source of the pop the following way:

Using a MKV player that does not honor ordered chapters, load the resulting MKV and skip to 02:07:00, and watch the movie. At this point, there should be a scene that shows a battleship; the predominant sound at the end of the scene comes from the battleship's engines. Then there is a cut, and a new scene begins.

That new scene is the problematic scene with the the pop. But this time, you won't notice the pop, because the pop is just the end of the audio of the (physically) previous scene. If you're young and concentrate extremely, you instead may notice that the battleship engine sound continues a few milliseconds after the cut, before the sound that belongs to the new scene actually begins.

You hear the pop only when playing the second edition in a player that honors ordered chapters. When watching the second edition, the audio at the end of the scene that the player shows before the problematic scene is totally different from the battleship engine sound (the environment and background noise is relatively quiet, and O. and L. have just finished their talk, so there is practically no sound there). Then there is the cut to the scene where H. runs into G., and at the begin of that scene, there is a few ms of the battleship engine sound which you notice as a pop, because it neither fits to the scene shown before nor to the new scene.

Regarding the test mentioned above:

I don't know many MKV players, but you could use AVIDemux (despite its name) to play MKV files if you do not want to honor ordered chapters. If you don't want to use another player, an alternative would be to remove the chapters from the MKV file. This should make your player play it linearly so that you can conduct the test.

Best regards, and thank you very much!

[Shoespooner]

The method is called "Ordered Chapters" because the player plays the chapters literally in the order they are in the chapters file. Notably, the player does not try to sort chapters according to their start time. This can make the player jump wildly around in the MKV file, which is perfectly acceptable.

However, usually that stuff is arranged so that the first edition is completely linear in the MKV, and the M2TS of the other edition(s) are appended at the end of the MKV file in the order they are later used. This is also the case for the movie we are discussing here. However, that's only a convention; it is not technically necessary.

I needed a while to understand how ordered chapters work. It is very hard to learn it from looking into chapters.xml directly. But it was getting easier when I looked to the chapters in a decent GUI. I have just verified that you can load the chapters.xml into MKVToolnixGUI, which then shows a nice overview.

The interesting thing here is the second edition, where the entries even show the source M2TS files.

I don't believe that there is a delay before the new M2TS is ready. Actually, there is no physical switch between files when playing the MKV; it is just jumps between different places in the MKV file. These are cheap seek operations that usually don't impose any problem, given that players read ahead and buffer large amounts of data in advance.

I have played around with that stuff a lot in the past days and could track down the problem to be either (a) that the player pulls a few audio frames from the wrong place when starting a new scene, or (b) an audio overlap. For testing, I have re-computed some chapter start and end times in the chapter file manually and could make the pop disappear. This was only for testing, because it had undesired side effects (that I had expected), but it shows that the seek operations or the preparation of new clusters after a jump is not the underlying issue.

The general underlying problem of course is that the video and the audio tracks have unrelated frame rates. Hence, if you take the video as reference, you either must remove the last audio frame before a gap (not good, because this leads to silence), or the last audio frame ends after the last video frame before the gap (usual method, but also not good, because it leads to overlaps).

Perhaps the only "perfect" way around this would be to re-encode the audio, of course only at the gap (sometimes called "smart cut" or "smart rendering"). But that's not a (de)muxer's task, and is beyond of what a (de)muxer can do.

[Rocky]

@hubblec4

OK, let's try some rodent logic.

I made the DGDemux version omitting the chapter file. For edition 2 the chapter file wants to find the chapter 22 start time at <ChapterTimeStart>02:07:12.124422222. So it seems to me as a first step that we need to be able to demux from the MKV (with high precision) starting exactly there and check that the audio is clean. I'm not aware of any utility to do that but I could write my own as I have MKV demuxing code in DGIndexNV.

If that audio is clean, then we could demux from chapter 21 until its end time and check that the audio is clean at the end (i.e., nothing from the last chapter end with the explosions). I think it would be more likely that the first case would be bad.

If both are clean then it must be a player problem. If one of them is not clean, we try to find out why not.

Seem reasonable?

[Shoespooner]

From what I have read so far, it is a problem with purely linear files as well:

As far as I have understood, we need to demux audio tracks as soon as we want to generate a MKV (without ordered chapters) from more than one M2TS file. Only if we want to convert one single M2TS file to a MKV file, it doesn't need to be demuxed. In case of multiple M2TS files, the demuxing is necessary to remove excess audio at the end of each M2TS file. Without removing the excess audio, the delay between video and audio could sum up to an unbearable amount if there are many input files.

A MKV file with ordered chapters is absolutely nothing special. The (de)muxer doesn't know about anything about chapters. It just should demux / concatenate every M2TS file (in our case, 6 M2TS files for the first edition plus 3 M2TS files that are unique to the second edition) and produce a MKV from it. That is in no way different from demuxing / concatenating 9 "normal" M2TS files.

That is, if there is a noticeable problem at all, that problem is a delay between video and audio. That delay exists regardless of whether or not there are ordered chapters. This can eventually be tested with the method described in one of my previous posts (remove the chapters.xml from the MKV or play the MKV in a player that does not honor ordered chapters).

That delay may be hard to notice when playing the file linearly, because it probably is a few ms or a few hundredth of a second. If there is an overlap during a cut, your ear will normally not notice it, because it is just the sound from the scene before the cut that continues a few ms too long, before the sound of the new scene starts.

But with ordered chapters, the same overlap can suddenly be noticed, because the audio in the overlap portion does neither fit to the previous scene nor to the new scene. Let's make a simplified example:

- In the physical layout of the MKV, there are five M2TS source files, called M1 to M5, in that order.
- M1 contains one scene where there is a loud scream at the end. M2 and M3 each contain one scene where there is continuous silence.
- The MKV has ordered chapters and contains two editions, called E1 and E2.
- E1 comprises M1, M2 and M3, in that order.
- M4 is literally the same file as M1 (doesn't make sense in real life, but makes the example more understandable, and means that there is the same loud scream at the end).
- M5 is literally the same file as M3, meaning that there is only silence.
- M1 (and thus, M4) have excess audio at the end (containing the last part of the loud scream).
- E2 comprises M4, M2 and M5, in that order.

Now, when playing E1, we see M1, M2 and M3, in that order, where excess audio (the last part of the loud scream) is at the end of M1 and thus a few ms of the scream are in M2. However, you probably wouldn't notice that fact, because the scream already had begun before the cut from M1 to M2, and you won't notice that it lasts for a few ms while the first video frame of M2 is already presented.

The key point now is that M4 has the same excess audio (because it is the same file as M1). But the excess audio now becomes part of M5 in the MKV, simply because M5 is immediately after M4 in the physical file layout. That is, M5 begins with the last few ms of the loud scream from the end of M4. Now let's look what happens when we play the second edition:

First, M4 is played, ending with the loud scream. Then, M2 is played (audio is silent here as defined above). Next, M5 is played. This is a scene where the audio is also silent. But because there is the audio overlap from M4 to M5, there are a few ms of a loud scream at the begin of M5.

In other words, in E2, after M4, we have M2 (silence) and M5 (silence). That silence is interrupted by a few ms of the loud scream at the beginning of M5. This is a disturbance that everybody will notify.

This example shows why the same disturbance can be noticed in one edition, but won't be noticed by most people in the other edition.

Thank you very much, and best regards!

[Rocky]

I doubt that the POP I've heard is only a few milliseconds, or that a few milliseconds would be an issue. We detect the POP sound as a low tone, you claim to be able to recognize it as part of an explosion, but the shortest detectable tone, identifiable as a tone, would be on the order of 100 milliseconds.

https://sound.stackexchange.com/questio ... -human-ear

[hubblec4]

OK, let's try some rodent logic.

I made the DGDemux version omitting the chapter file. For edition 2 the chapter file wants to find the chapter 22 start time at <ChapterTimeStart>02:07:12.124422222. So it seems to me as a first step that we need to be able to demux from the MKV (with high precision) starting exactly there and check that the audio is clean. I'm not aware of any utility to do that but I could write my own as I have MKV demuxing code in DGIndexNV.

If that audio is clean, then we could demux from chapter 21 until its end time and check that the audio is clean at the end (i.e., nothing from the last chapter end with the explosions). I think it would be more likely that the first case would be bad.

If both are clean then it must be a player problem. If one of them is not clean, we try to find out why not.

Seem reasonable?

That sounds absolutely great. Because this will show what kind of audio data is in the mkv at this point and you can then safely draw conclusions from which m2ts this extra audio data comes from.

If you need help with Matroska in any way, please let me know, I'm very familiar with it.

One more thing I should mention about MKV and FPS.
Matroska doesn't know FPS, so no information like that is saved. All frames(video, audio, subs) have timestamps.
If there is excess audio data then the timestamps for video and audio would drift apart. For a certain timestamp you will either have the wrong video image or the wrong sound, depending on what the player prefers as a track.

I also created a new chapter.xml file. cE also offers the possibility of creating an edition where the chapters exactly match the playing times of the M2TS files.
It is the third edition, it also has ordered chapters, but the chapter times are continuously advanced from beginning to end without gaps, so it is as if normal chapters were used.

This should make testing easier and saves you having to change chapter files or play them in a player that cannot play ordered chapters. The second Edition is still the default Edition and will be played after start.

[hubblec4]

I'm also learning about ordered chapters and figuring out the XML file.

Thank you, and I hope you have a bit fun.
Ordered chapters are so cool, and you can do many crazy things, and it's not magic, it's Matroska.

[Rocky]

I don't understand things well enough yet to follow what this new chapter files does or how it is calculated. Let's back up a little. For the first chapter file, can you explain in detail how cE determines the timestamps that it uses for the chapters?

[Rocky]

Made some progress. All below is for the DGDemux files.

I determined that the length of the audio artifact is about 75 milliseconds, which is consistent with my earlier claim that it couldn't be just a few milliseconds, and consistent with the information at the site I linked.

So I made a new chapters file that starts 00912 and 00913 100 milliseconds later than what the original chapters file from cE specified. Note that these correspond to 2 of the extra segments. I didn't do the first one (00910) because it's the first segment. Maybe I didn't need to do 00913 also, as it is a "hidden" chapter, whatever that means. Here is the modified chapter file:

https://rationalqm.us/misc/chapters_ch3 ... s100ms.xml

This plays clean for both editions and both audio streams. It does not affect audio sync. You do start the extra segments a bit late but as these are chapter points they are highly likely to be scene changes so nothing is noticeable. So a possible heuristic would be to add 100 milliseconds to the start of all of the extra segments. You could omit the first extra if it happens to be the first played segment. Maybe it could be an option for cE: turn on artifact prevention and specify a start delay in milliseconds. But see the last paragraph here.

IIRC, it was Shoespooner that earlier referred to such tweaking but claimed there are other problems with it. I'd be very curious to know what he was referring to. Our goal is to make a clean multi-edition MKV, and I argue that this heuristic does that.

I'm going to continue trying to find out why the extra audio is so large when it should be only 1/2 of an audio frame on average, and one frame maximum. I'll look at segment start times and lengths to see if the substitutions for edition 2 are really the same for start time and length. If they differed it wouldn't matter for playing the bluray as it would use PTS timestamps, which are lost when demuxing.

[hubblec4]

I don't understand things well enough yet to follow what this new chapter files does or how it is calculated. Let's back up a little. For the first chapter file, can you explain in detail how cE determines the timestamps that it uses for the chapters?

The procedure is always the same for all three chapter files(which I have uploaded).
The first edition is the simplest because all the times are always in a row so there are no jumps and therefore no hidden chapters. Only the timestamps from the chapters(found in the mpls) are used. For the last chapter the endtime get the timestamp of the sum of all used m2ts duration.

In the second edition, hidden chapters have to be inserted if necessary for jumps in certain places.

All timestamps are always calculated in relation to the m2ts playing duration. (OUT time minus IN time from an m2ts file.)

The chapter times are specified and is in the mpls file.
The 00801.mpls is required to calculate the timestamps for the 2nd edition.
Since the playing time of an m2ts is known and also where the m2ts is stored in the mkv, the timestamps can be calculated with nanosecond precision.

I'll give a minimal example.
This time we have 2 m2ts files for each edition.
Edition 1 = M1 + M2
Edition 2 = M3 + M2

Multi-Segment List = M1 + M2 + M3
Each M2TS file is exactly 60 seconds long.
M1 starts at 0s
M2 starts at 60s
M3 starts at 120s

Then we have two chapters for each edition and these are identical for both editions.
Chapter 1 = 0s
Chapter 2 = 90s

Edition 1 now has the following ordered chapters
Chapter 1 = start 0s - end 90s
Chapter 2 = start 90s - end 120s

Chapter 1 is composed of the entire M1 plus half of M2. Chapter 2 then has the remaining half of M2.

Edition 2 now has the following ordered chapters
Chapter 1 = start 120s - end 180s
Chapter 2 = start 60s - end 90s -> hidden
Chapter 3 = start 90s - end 120s

Because Chapter 2 is a hidden chapter, its playing time is added to the previous chapter without this chapter being visible in the timeline.