Problem with multi-edition MKV creation

[hubblec4]

@hubblec4

Would you consider the need for this adjustment more a deficiency of the Matroska specification or of the Matroska players?

I would say neither.
I think it's more due to the muxing tool.
So MTX could make sure that when the input file changes, the timestamps for the audios are also adjusted to the timestamp of the video.

But I know that MTX already has a mega heuristic and these additional specifications would inflate the heuristic even further.

There was one thing I did differently than Rocky during the test. I don't have the video stream demuxed, so MTX then has to access the M2TS file to get the video frames.
However, the video is 100% identical as if it had been extracted with DGDemux. (A great proof that MTX can handle multi m2st segments really well, at least for the video track)

My plan for a workaround is the following:
The timestamps when a new M2TS file begins are known and cE also sets 100% correct timestamps in the chapters.
After the file is created, the audio timestamps could be shifted a little.
I don't mean ALL timestamps here, just the timestamps from the first audio frame in this cluster.

In our case, the timestamps only need to be shortened to 4ms. This can be done very quickly as only two bytes in the mkv need to be overwritten.

[hubblec4]

hubblec4 wrote:

By “original chapter file” which file do you mean?

It just means the multi-edition chapter file you first made and gave me for use with the A_NEW_HOPE 00800/00801 playlists.

OK, fine, then I translated and understood it correctly, and my answer was appropriate.

If hubblec4 wants the adjustment in DGDemux, we could easily do it.

If an adjustment is made, it should not be automatic in nature. A switch to control the behavior would be a good option.

On the other hand, I think that this does not necessarily have to be built into DGDemux, since it is not a DGDemux problem.

It's more of a problem with the audio, how these codecs are structured, and also in the Matroska structure, which isn't 100% perfect either.

[Rocky]

Interesting. I'm sure you will take into account the maximum audio frame size that might be encountered per SomeHumanPerson. Probably that will be automatic if you do per-cluster adjustments. Please keep us informed of your progress.

Based on your preference I will not do any adjustment in DGDemux.

cE could do the track checking/unchecking with right click as done in DGDemuxGUI.

I took the liberty of replying in the reddit thread. It's not showing up, though. Maybe they have to approve it because it has a link back to here.

[SomeHumanPerson]

Thanks for the more generalized adjustment figure, Rocky.

I'd have a brownie if they're on offer. Polite pass on Boris' Granny's herbal enhancement, though.

[Rocky]

Sure, you earned it! Boris' Granny made that. I'd engage a taster. The lolli may be safer.

[hubblec4]

Interesting. I'm sure you will take into account the maximum audio frame size that might be encountered per SomeHumanPerson. Probably that will be automatic if you do per-cluster adjustments. Please keep us informed of your progress.

I then thought again about moving the entire timestamp using a delay. Although it is only one audio frame, this affects the entire film because there is an audio frame offset to every video frame. But without this wonderful approach, I would certainly never have thought of manipulating the audio timestamp at the relevant point.

I've now been able to do a few tests and now have a wonderful solution (I think).

Setting the audio timestamp exactly to the Cluster timestamp (and thus also the video timestamp) was unsuccessful.
Then I thought about the fact that the first audio frame in the Cluster itself is "disturbing". Therefore I changed the audio timestamp 1ms before the Cluster timestamp.

The Audio POP can no longer be heard.

I find this all very interesting at the moment, because now I understand how the player plays it.
Apparently 1ms is enough for the first audio frame to be skipped. The player seems to only orientate itself on the video track. If an audio frame is behind a video frame(timestamp), it will no longer be used.

I still want to find out one thing. The audio frame that is skipped of the Cluster, whether this is needed elsewhere.

EDIT:
I don't know exactly whether all the following audio timestamps have to or should be changed.
In any case, only the audio timestamps that belong to the respective M2TS, which are included as extras at the end in the MKV file.
The data there is never played back linearly.

The sound sounds clean and no gaps or silences are audible.

cE could do the track checking/unchecking with right click as done in DGDemuxGUI.

This is not so easy because the right-click is already in use. But I will modify the right-click behavior.

I took the liberty of replying in the reddit thread. It's not showing up, though. Maybe they have to approve it because it has a link back to here.

I don't know. I'm not a big fan of Reddit. I upvoted your comment, maybe this helps.

[Rocky]

I'm back in the nest early (feeling a bit under the weather after our extended sub-zero deg F spell) so I will digest and cogitate on your new findings in the morning. Mousie is here with me and we're keeping each other warm. DG keeps us well-supplied with high-energy chow and soft bedding. There's not enough room on top of the Crosley Super 8 for both of us. Anyway, Britney is visiting but she's pathologically scared of rodents, otherwise we would hang out in the house. Getting swatted with a broom is no fun. Rat poison is even worse. Hey, at least I don't eat my own poop.

Yeah, I agree about reddit. I figured out the problem. Comments are all collapsed by default. I prefer to have all comments expanded by default. Apparently there is a setting in the posting configuration to do that, but I don't have the energy or time to bother with it. Thanks for the upvote!

cE is a really great application. Congratulations!

So you're a Pascal guy, huh? Curly wants to port eac3to to the latest Lazarus. He might be asking your advice about that.

[Britney]

I did NOT eat my own poop!

[Rocky]

@hubblec4

I don't know exactly whether all the following audio timestamps have to or should be changed.

I don't see why they would have to be. Once you have seeked, playing from there will be linear. In other words, those adjustments are only important for seeking.

Don't forget that this "bug" also applies for seeking in normal MKVs (single edition), so theoretically every cluster should be adjusted.

[hubblec4]

cE is a really great application. Congratulations!

Thank you for kind words.

So you're a Pascal guy, huh? Curly wants to port eac3to to the latest Lazarus. He might be asking your advice about that.

Yes, I'm a Lazarus (FreePascal) guy
Best programming language in my opinion.

I'm happy to help with Lazarus, I only recently installed the latest version.

I don't know exactly whether all the following audio timestamps have to or should be changed.

I don't see why they would have to be. Once you have seeked, playing from there will be linear. In other words, those adjustments are only important for seeking.

That was also my first thought, that the other audio frames would then be played linearly and no problems should arise.

Don't forget that this "bug" also applies for seeking in normal MKVs (single edition), so theoretically every cluster should be adjusted.

understand what you mean, but I'm not entirely sure if that's necessarily the case for a single edition.
I don't have any knowledge of how this is related to the audio.
It was only through this thread that I gained some further insight.

[Rocky]

Give me some time. I'm going to try to demonstrate it to you.

I'll let Curly know you'd be willing to help him.

[Rocky]

OK, I'm going to blow your mind now. Accept my apologies in advance.

Demux the streams as if you wanted to make a multi-edition MKV. Load the streams in MTX but do not load the chapters file. Then just complete the multiplex to create the MKV without chapters. Now open the MKV in MPC-HC and hit stop so it is not playing. Now do Navigate/Go To... and enter 02:07:12.124. Hit Go! You will still be stopped with the correct frame visible. Now hit Play. You will hear the POP.

There is nothing special about this MKV. It's just that at this transition there is async and the difference in audio is very great before and after the transition, and that's why you hear it. Normally, if you seek to a random point the audio before and after is going to be similar. Even with most chapter seek points the audio is similar or transitions across the seek point, so you don't perceive anything. In the 2nd edition of a multi-edition MKV it is much more likely that you will encounter a transition with a big change of audio, because at such a transition the audio before the transition is not from linearly before that point. If the difference is big enough you will hear it.

Please think it through carefully, because I believe it explains everything. The bottom line is simple. When you seek to a point with appreciable async, you will play audio from before the seek point (we explained why earlier in the thread). If it is similar to the following audio everything is fine. If it is not, you will hear it. I remember you earlier observed seeing something like that when playing the first edition. It was just an unlucky seek point with async and a big audio difference.

The link I posted earlier showed that the larger the difference in audio volume, the smaller is the length of time needed for it to be heard. In our case the difference is very great. It's just an unfortunate confluence of circumstances that arises for the specific case we are discussing.

We've seen several possible "fixes". I leave it to you to choose the one that satisfies archiving needs and is not too difficult to implement.

[Sherman]

I did NOT eat my own poop!

OK, whose poop did you eat?

[Britney]

That's it, I'm going back to Mississipi right now! Let the rodents take over, I don't care.

[hubblec4]

OK, I'm going to blow your mind now. Accept my apologies in advance.

.....

We've seen several possible "fixes". I leave it to you to choose the one that satisfies archiving needs and is not too difficult to implement.

Yes you are right, the Audio for the extras is not linear stored and for this Cluster the timestamps have to be adjusted.
There is a switch for mkvmerge "-timestamp" to manipulate the timestamps while muxing the MKV.
Modifying the timestamps on the bit-level would be faster and needs only micro seconds, but cE's code is to old for implementing my own EBML and Matroska libs, and unfortunately I can't(or with a huge amount of work) upgrade to the newest Lazarus version.

But to change this 2 Bytes, maybe I could write a small piece of code of that job.

[Rocky]

I think you missed my point that it can happen anywhere in any MKV file, not just multi-edition files, and not just for the extra segments.

[hubblec4]

That's right, there's a piece of text missing somehow.

In principle, this is the case with almost every Cluster because the audio timestamps never match the video timestamps exactly.
However, it is not a problem when playing linearly, and there may be annoying noises during jumps, but the user has forced them himself.
I think it wouldn't be right to adjust all audio timestamps because, as I said, they are correct for the linear playback.

[Rocky]

Ah good, now we're on the same page. Because a jump when playing a second edition was not forced by the user, we have to adjust things, otherwise, hey, too bad, you asked for it.

I'm still interested to know your final solution.

[hubblec4]

I'm still interested to know your final solution.

At the moment I haven't reached a final solution yet, I only have rough ideas in my head as to how and where I could incorporate this into cE.

I have two methods in mind:
1. The chapter times are shifted by one video frame.
2. The audio timestamps are adjusted

to 1)
This might be quite easy to do since everything is prepared for the chapters in cE. A simple selection for the problematic jumps could be displayed and the user should check whether everything is OK there. If not, cE edits the chapters and saves them in the MKV and the error should be fixed.

to 2)
It won't be that easy and a lot of new parser/writer code will have to be created. But the selection for the problematic jumps remains the same.

I'm still working on the select-all-tracks feature. (And I have found a bug for creating the .mtxcfg settings file for this disc.)

[Rocky]

Thanks. I like 1. applied only at the extra segment start clusters. I would just do it unconditionally and avoid all the checking/selection.

[hubblec4]

Yes, I also think that I will install the first variant first, since it does not change the data in the MKV.
And you won't notice the "loss" of one video frame when watching the film.

But doing the whole thing automatically and all the time isn't necessarily always a good thing. Because I already have a lot of multi-edition MKVs that didn't cause any problems.
Likewise, an audio problem doesn't necessarily have to occur with every jump.

But there are other things on the TODO list at the moment, so I still have some time to think about it.

[hubblec4]

Yesterday I had a little time and looked at everything again and asked myself whether this was actually the only part (Chapter 22) that was causing problems.

Here are my latest findings:
There is a hidden chapter after chapter 3(second edition). I then made this visible (and all other hidden chapters). Unfortunately there is also a "problem" there, but it only has something to do with the video.
In MPC-HC I jump to the "hidden" chapter (it is now visible) and go back one video frame. The correct frame is then not displayed. Only when you go back one frame is it correct again.
After a while of checking the cluster timestamps and the chapter times I found the "error".

By default, MTX uses a value of 1,000,000 for the TimeStampScale and this means that there is only millisecond precision for the timestamps.
For this reason, the chapter timestamps no longer match exactly (chapters use nanosecond precision). And the result is that the player plays one video frame too many before the "hidden" chapter is reached.

I then changed the TimeStampScale value to 1000 which means microsecond precision.
This then fixed the error. However, the clusters now look strange because a BlockGroup is now used for each video frame. Even positive and negative reference timestamps are used.


--------
my other issue from my todo list is strange and has for the moment priority, because I have never seen such an mpls structure before, and I have seen a lot.

[Rocky]

By default, MTX uses a value of 1,000,000 for the TimeStampScale and this means that there is only millisecond precision for the timestamps.
[etc.]

Very interesting, thank you. 1ms granularity does seem too large to me.

my other issue from my todo list is strange and has for the moment priority, because I have never seen such an mpls structure before, and I have seen a lot.

Can you share details on that? Or is there a cE issues list somewhere we could look at? Thanks!

[hubblec4]

By default, MTX uses a value of 1,000,000 for the TimeStampScale and this means that there is only millisecond precision for the timestamps.
[etc.]

....1ms granularity does seem too large to me.

Yes, I feel that way too. It has always bothered me a little that the chapters are handled with nanosecond precision but the tracks are only handled with millisecond precision.

However, I would now like to understand why mkvmerge.exe now uses these BlockGroups.
Likewise, 1000 seems to be the smallest value that can be used for the TimeStampScale. This has to do with the 2 bytes (signed integer) that are used for the time difference in the block structure.

my other issue from my todo list is strange and has for the moment priority, because I have never seen such an mpls structure before, and I have seen a lot.

Can you share details on that? Or is there a cE issues list somewhere we could look at? Thanks!

At the moment I'm still waiting for a test from the user that will show me whether DGDemux also has a problem with this specific mpls.

It is an mpls that is structured like these special menu mpls. An m2ts is used more than once. However, not the entire m2ts is used every time, but only a certain period of time. The m2ts file is used 20 times and the IN and OUT times increase, but are not in a continuous series, there are gaps.
This mpls file simulates ordered chapters

There is another mpls file on the disc, which uses the m2ts in full and has a playing time of 1h 34m.

[Rocky]

Thanks for the information and let me know if you need me to do any testing.