Problem with multi-edition MKV creation

User avatar
hubblec4
Posts: 269
Joined: Tue May 02, 2023 6:03 pm

Problem with multi-edition MKV creation

Post by hubblec4 »

Rocky wrote:
Wed Jan 17, 2024 9:11 am
Well I figured out cE. Looks like cE always puts the extra files at the end. :cry:
Yes, this was/is necessary so that the first edition plays normally in all players which don't understand ordered chapters.
Rocky wrote:
Wed Jan 17, 2024 9:11 am
I am worried that the timestamps for the extras are beyond the claimed duration of the MKV. Could it confuse the player?
Actually, the playing time of the MKV should have the highest value of all the tracks used. and I think that this is also the case here and therefore the timestamps for the extras should be correct.

Apart from that, any good Matroska player ignores the duration when using ordered chapters.
Rocky wrote:
Wed Jan 17, 2024 9:11 am
In the track selection dialog, is there a way to deselect all tracks so I don't have to uncheck all of them individually?
Unfortunately not, but I'll put it on the TODO list.
Rocky wrote:
Wed Jan 17, 2024 9:11 am
Is there a way to make just the chapter file and not do the demuxing and MKV creation?
The chapters are created immediately after the mpls is activated. And in multi-edition mode after the multi-editions have been prepared (the button with the green arrow).
The chapters are also loaded directly in the chapter editor.
Rocky wrote:
Wed Jan 17, 2024 11:16 am
In Info tool for the multi MKV I see that the cues do not include the extra segments. So seeking to an extra segment may be an issue.
I hadn't checked that yet and it's quite strange since all video frames from the m2ts files are used.
The cues are very helpful for a player and make jumping much easier, but a good Matroska player should be able to jump easily even without the Cues. The timestamps for each cluster can be read very quickly.
User avatar
hubblec4
Posts: 269
Joined: Tue May 02, 2023 6:03 pm

Problem with multi-edition MKV creation

Post by hubblec4 »

Rocky wrote:
Wed Jan 17, 2024 2:59 pm
The method:

1. Demux 00800.mpls with the three extras using DGDemux as before.

2. Load the demuxed streams into MTX.

3. Set a delay of -5 ms for both of the audio streams. This not enough to be perceptible.

4. Load the chapter file from cE.

5. Run the multiplex operation.

That's it. Have fun. I have to go do some coaching. When I come back I'll explain the theory.
There's one thing that MTX does when you use negative delays on the tracks.
Unfortunately, the timestamps are not moved but the first frames are cut off and lost.
The timestamps are only shifted for positive values.

I didn't know this for a long time, but since then I've never used negative delays again.

The reversal of your suggestion would be to use a positive delay for the video track.

But it sound great and I'm curious what the theory is.
User avatar
Rocky
Posts: 3754
Joined: Fri Sep 06, 2019 12:57 pm

Problem with multi-edition MKV creation

Post by Rocky »

Here's what is going on.

The audio and video frames have different granularities, so at a gap between files you can have up to one audio frame more audio than video at the end of the first file. This will vary along the whole movie. The extra audio pushes into the next segment. With TS you have separate timestamps for audio and video so on a seek, you can start playing both at their correct times. But in MKV we have only the one cluster timestamp.

We see in the timestamps (using Info tool) for your MKV at chapter 22:

cluster timestamp 2:07:12.124
video timestamp 2:07:12.124
audio 1 timestamp 2:07:12.128
audio 2 timestamp 2:07:12.128

The offset just represents the sync condition at that point. Theoretically a player could use the audio and video timestamps individually. But they do not do that! They seek to 2:07:12.124 using the cluster timestamp and then start reading frames, both audio and video. Therefore, they will start playing audio from an earlier point. So we shift the audio back so that it starts playing the correct audio.

The amount to shift back only has to be the maximum audio frame duration. In this case I chose -5 milliseconds. This fortunately is not perceptible. All you lose if 5 milliseconds at the start of the movie only.

So Sherlock was correct, the players are too simplistic. The workaround just corrects for that deficiency.

I don't think there is anything better to be done if we want to work with the existing players.

This was an interesting problem. Thank you for challenging me with it. :salute:
User avatar
Rocky
Posts: 3754
Joined: Fri Sep 06, 2019 12:57 pm

Problem with multi-edition MKV creation

Post by Rocky »

hubblec4 wrote:
Wed Jan 17, 2024 4:53 pm
There's one thing that MTX does when you use negative delays on the tracks.
Unfortunately, the timestamps are not moved but the first frames are cut off and lost.
The timestamps are only shifted for positive values.
We don't want to shift the timestamps. The chapters file does not change as you see. We just slide the entire audio stream back. The only loss is 5 milliseconds at the start of the movie. That's the beauty of this method. :wow:

At first I tried adjusting by changing the timestamps in the chapters file. But it did not work, because you can't shift only the audio. Then I had the brainstorm to let MKV shift the stream when muxing. It works great for me. Nobody is going to miss 5 milliseconds of silence at the beginning.
The reversal of your suggestion would be to use a positive delay for the video track.
Yes, you would shift positive by one video frame time. But the timestamps would have to change. For the -5 millisecond audio shift, MTX just throws away the first audio frame. Also, there may be a problem of having no video while the audio starts. I'd have to experiment. I think the negative audio shift is best.
User avatar
Britney
Posts: 153
Joined: Sun Aug 09, 2020 3:24 pm

Problem with multi-edition MKV creation

Post by Britney »

OMG Rocky. :?

Image
User avatar
Natasha
Posts: 150
Joined: Wed Nov 20, 2019 11:11 am

Problem with multi-edition MKV creation

Post by Natasha »

Even a blind squirrel finds an acorn once in a while.
User avatar
Bullwinkle
Posts: 345
Joined: Thu Sep 05, 2019 6:37 pm

Problem with multi-edition MKV creation

Post by Bullwinkle »

Way to go, Rock!

Image
User avatar
Sherman
Posts: 633
Joined: Mon Jan 06, 2020 10:19 pm

Problem with multi-edition MKV creation

Post by Sherman »

@Britney

Are rabbits rodents?

The rabbit was classified under Rodentia (rodents) until early in the 20th century. This family includes rats, squirrels, mice and marmots. However, you’ll start to notice the differences between rabbits and rodents once you take a closer look.

Rabbits have different digestive systems

The cecum sits at the head of the large intestine and houses beneficial bacteria that break down and ferment cellulose in plants. Because rabbits need to maximize digestion of vegetation, they essentially eat their food twice.

Some rodents also have a cecum. After a rabbit dines on plant material, the material passes through the digestive tract and comes out of the body as a soft black pellet (a caecotroph). They then eat the caecotroph, re-chew and re-digest it before it exits as the hard round pellets rabbit owners are used to seeing. :wow:

Rabbits are almost exclusively herbivorous

While plant matter is the sustenance of choice for both rabbits and rodents, rodents have a more varied diet of roots, grains, seeds, bugs, etc.

Rabbits have four incisor teeth

Rabbits have four incisor teeth, while rodents only have two. Rabbits have a smaller set of peg-like teeth behind their top incisors. These are about ¼ the size of their first set of incisors, and many believe they may help rabbits bite through vegetation more easily.
Sherman Peabody
Director of Linux Development
User avatar
hubblec4
Posts: 269
Joined: Tue May 02, 2023 6:03 pm

Problem with multi-edition MKV creation

Post by hubblec4 »

Rocky wrote:
Wed Jan 17, 2024 5:36 pm
Here's what is going on.
......
Wow, I'm at a loss for words. What a great analysis and the nice thing is, I know exactly what you're talking about. Just Matroska language :-)

You're absolutely right, there is only one Cluster timestamp and all tracks are based on that, and the Cluster timestamp is always assigned to the "most valuable track". In this case the video track.

----
In the Matroska team we also have many film archivists who work for large institutes. All data is always stored without loss, and I could imagine that they don't even want to miss out on a single frame.
But for me as a private user, something like that would be perfectly fine.

But as I said, we could also set the video delay to positive 5 ms. I'll try it.

Anyway, it's great anyway.
There are already three solutions.
1. you live with the audio POP
2. the chapter times are adjusted a little at certain points
3. use a negative audio delay when muxing
(4. positive video delay -> still to be tested)

to 2.)
I tested this with the modified chapter file you created. It was clear that only a single frame would “skip”. You shifted the chapter times by 100ms and after your great analysis I suspect that maybe 41ms is enough. Just one video frame.

As always, Rocky is great to work with you. Many thanks. :hat:
User avatar
Rocky
Posts: 3754
Joined: Fri Sep 06, 2019 12:57 pm

Problem with multi-edition MKV creation

Post by Rocky »

I'd personally go with the audio shift, but as long as you test things thoroughly I suppose the video shift could be acceptable. You know it, but I say it for others: You have to shift just the video while muxing. Never just adjust the chapters file as that would affect both video and audio. Why it seemed to work earlier is probably just because it shifted things to a point in the stream where the sync was close. It's theoretically wrong. Your 3 and 4 are good, don't use 1 and 2.
In the Matroska team we ...
What team is that and are you on it?

Thank you for your answers to several of my earlier questions.
User avatar
Britney
Posts: 153
Joined: Sun Aug 09, 2020 3:24 pm

Problem with multi-edition MKV creation

Post by Britney »

@Sherman

Does that mean they eat their own sh i t? Ewww. :|
User avatar
hubblec4
Posts: 269
Joined: Tue May 02, 2023 6:03 pm

Problem with multi-edition MKV creation

Post by hubblec4 »

Rocky wrote:
Wed Jan 17, 2024 7:05 pm
In the Matroska team we ...
What team is that and are you on it?
I am a member of the Matroska team, we develop Matroska. So the specifications also include the EBML and Matroska libs.
The whole thing is then “reported” to the IETF team.
https://github.com/ietf-wg-cellar/matro ... cification
User avatar
Sherman
Posts: 633
Joined: Mon Jan 06, 2020 10:19 pm

Problem with multi-edition MKV creation

Post by Sherman »

Wow, Mr hubblec4. We never knew that you were a big-wig. Lucky we were nice to you, huh?
Sherman Peabody
Director of Linux Development
User avatar
DG
Posts: 94
Joined: Thu Dec 31, 2020 9:55 am

Problem with multi-edition MKV creation

Post by DG »

You should try it, Britney. You might like it. You have to try everything at least once.

Rocky never gives up as long as there remains a sliver of hope. Bravo!

Thank you hubblec4 for one of the most amazing threads we've ever had.
User avatar
Rocky
Posts: 3754
Joined: Fri Sep 06, 2019 12:57 pm

Problem with multi-edition MKV creation

Post by Rocky »

Thank you, DG.

@hubblec4

"Actually, the playing time of the MKV should have the highest value of all the tracks used."

Should, hehe. I made an MKV by just doubling the streams so it was 4 hours+. Then I applied the original chapters file and opened it in MPC-HC. The reported duration is always the same as the end timestamp of the last chapter. If I edit that in the chapters file, the reported duration changes accordingly. Did you know that?

Without cues wouldn't seeking have to linearly search a multi-GB file? Or is there some kind of binary search with file seeking? Or can you skip millisecond by millisecond, as clusters seem to come at that interval?
User avatar
hubblec4
Posts: 269
Joined: Tue May 02, 2023 6:03 pm

Problem with multi-edition MKV creation

Post by hubblec4 »

Sherman wrote:
Wed Jan 17, 2024 7:24 pm
Wow, Mr hubblec4. We never knew that you were a big-wig. Lucky we were nice to you, huh?
Thank you very much. But I'm just a tiny light compared to Mosu and Steve.

I feel very comfortable here with you.
User avatar
hubblec4
Posts: 269
Joined: Tue May 02, 2023 6:03 pm

Problem with multi-edition MKV creation

Post by hubblec4 »

DG wrote:
Wed Jan 17, 2024 7:31 pm
Thank you hubblec4 for one of the most amazing threads we've ever had.
You are welcome, and I do what I can.
And all this is only the beginning of Matroska Chapters, there are Linked-Ordered-Chapters and Linked-Edition-Ordered-Chapters :-)
User avatar
hubblec4
Posts: 269
Joined: Tue May 02, 2023 6:03 pm

Problem with multi-edition MKV creation

Post by hubblec4 »

Rocky wrote:
Wed Jan 17, 2024 7:44 pm
@hubblec4

"Actually, the playing time of the MKV should have the highest value of all the tracks used."

Should, hehe. I made an MKV by just doubling the streams so it was 4 hours+. Then I applied the original chapters file and opened it in MPC-HC. The reported duration is always the same as the end timestamp of the last chapter. If I edit that in the chapters file, the reported duration changes accordingly. Did you know that?
I'm not entirely sure if I got everything translated correctly.

By “original chapter file” which file do you mean?
If there were ordered chapters and the chapter times are all consecutive, then the last end timestamp is always the play duration.
Whenever ordered chapters are used, the player must create a new virtual timeline and then calculate a total playing time from all chapter durations.

You could also place an ordered chapter in the 4h video, which should only play 10 minutes of a certain part.
Then the player only shows 10 minutes as the playing time.

That's why I love these ordered chapters so much. I can always tailor the content of the films or homemade videos to make them fun.
Without any hassle cutting or re-muxing.

Just change chapters, that's all.

Rocky wrote:
Wed Jan 17, 2024 7:44 pm
Without cues wouldn't seeking have to linearly search a multi-GB file? Or is there some kind of binary search with file seeking? Or can you skip millisecond by millisecond, as clusters seem to come at that interval?
I'm currently working on my own EBML and Matroska lib for FreePascal.
Finding the next cluster is super quick and the player can do this in the background while the video starts.
In VLC there is an option to preload all clusters.

This is more or less the nature of Matroska, parsing is very fast, because only a few bytes are read out and a parser knows where is the next Element.

EDIT:
There is no fix interval for Clusters. Only recommended restriction of 5mb or 5 seconds, MTX would then write a new Cluster at the latest.

If a large jump is requested, the player can do a "guess-jump" calculated of the current position.
Find a Cluster and check the timestamp. If it's close to the requested timestamp then the next Clusters can be parsed quickly.
User avatar
hubblec4
Posts: 269
Joined: Tue May 02, 2023 6:03 pm

Problem with multi-edition MKV creation

Post by hubblec4 »

I'm finish with the 5ms delay for the video.
I can't hear any difference between the -5ms Audio delay MKV.

The first video frame is now behind the Audio frame, but this is not a problem for a good Matroska player.
If I remember correct there are test files on the Matroska GitHub for this case.
User avatar
SomeHumanPerson
Posts: 96
Joined: Fri Mar 24, 2023 10:41 am

Problem with multi-edition MKV creation

Post by SomeHumanPerson »

This was amazing detective work, Rocky. Well done. :bravo:

Is the -5ms audio delay a value that should work for any audio format? I'm not familiar with the frame lengths of various audio codecs so I don't know how your value from this example would relate to any other situation.
User avatar
Rocky
Posts: 3754
Joined: Fri Sep 06, 2019 12:57 pm

Problem with multi-edition MKV creation

Post by Rocky »

You always ask great questions, SomeHumanPerson. Our gaps correction needs to know the frame durations for each audio type and I remember a case statement to set that. But I'm holed up in the nest right now. In the morning when I get to the office I'll have a look and report back.

Actually, as long as gaps correction is performed when demuxing, the maximum async is actually 1/2 a frame, because when it exceeds 1/2 frame a full frame is removed.
User avatar
Rocky
Posts: 3754
Joined: Fri Sep 06, 2019 12:57 pm

Problem with multi-edition MKV creation

Post by Rocky »

The largest frame duration is AC3 at 32 milliseconds. So 1/2 of that is 16 milliseconds, which should always be our shift (if we don't want to track and calculate the maximum asyncs across all clusters). That is still less than the human perceptible desync threshold of 20 milliseconds. The Big Squirrel in the Sky is watching over us.

Of course, the needed adjustment depends on the actual async at the seeked-to cluster.

Thank you for bringing this up, SomeHumanPerson. We have such amazing members here. Lollipop or brownie?
User avatar
new_guy
Posts: 60
Joined: Fri Jan 15, 2021 11:12 am

Problem with multi-edition MKV creation

Post by new_guy »

@hubblec4

Would you consider the need for this adjustment more a deficiency of the Matroska specification or of the Matroska players?

I note that this mechanism applies to all MKV streams, not only multi-edition ones. Rocky showed that empirically.

@Rocky

Could/should this adjustment be an option for DGDemux?

I'll take the brownie as long as it is made without seeds oils. Beggars can't be choosers, but I am not a beggar. ;)
User avatar
Rocky
Posts: 3754
Joined: Fri Sep 06, 2019 12:57 pm

Problem with multi-edition MKV creation

Post by Rocky »

hubblec4 wrote:
By “original chapter file” which file do you mean?
It just means the multi-edition chapter file you first made and gave me for use with the A_NEW_HOPE 00800/00801 playlists.

@new_guy

Honestly, without a mouth, what are you going to do with a brownie?

If hubblec4 wants the adjustment in DGDemux, we could easily do it.
User avatar
Boris
Posts: 93
Joined: Sun Nov 10, 2019 2:55 pm

Problem with multi-edition MKV creation

Post by Boris »

This is all a giant cluster-duck. But we have relief for you in the form of Granny's traditional herbal formulation. Increases blood flow to critical areas. Send PM for special low pricing for forum members. Granny knows best.

Onward to the victory!
Post Reply