Seamlessly branching multi-disc movies with Atmos

[SomeHumanPerson]

Edit: tested it, same issue unfortunately. The files play perfectly in VLC on my PC, but when playing Atmos through my AVR I get the audio dropout at the joint.

What are you using for playback?

I'm led to understand that there is a problem with Plex on the NVidia Shield (and maybe other devices) that doesn't handle joined TrueHD properly. Remuxes of seamless-branching discs demuxed with recent DGDemux will work perfectly in most PC software (MPC-HC, VLC, mpv) but cause problems in Plex for unknown reasons.


Edit: clarification

[Rocky]

After reviewing old posts I remembered that you can detect the errors without an AVR by transcoding to flac with ffmpeg:

ffmpeg -i file.thd file.flac

It spits out errors when things are not kosher. I'm going to test using that for these cases:

1. The two untrimmed THD streams individually.
2. The concatenated stream without trimming.
3. The two trimmed THD streams individually.
4. The concatenated THD stream with trimming.

The disks have a single M2TS but I'll also disable gaps correction when demuxing to be sure to get the raw untouched streams. This hopefully will point to the problem and allow me to figure things out. Hope springs eternal.

For those interested in the history here of THD compressed domain gaps correction, etc., start reading here:

https://www.rationalqm.us/board/viewtop ... 841#p10841

[Rocky]

Results with untrimmed streams:

Tested individually, there are no errors/warnings. Tested combined, we get a "Lossless check failed" warning at the joint.

Results with trimming 0 5 for disk 1, 5 0 for disk 2:

Same as for untrimmed!

Note that this "Lossless check failed" warning has been widely regarded as inconsequential. More serious errors occur for mangling of the streams, violating the frame deleting rules, etc. I looked into this years ago but the spec was not complete enough (missing sections) to understand the cause. Now I wonder if the cause is the resetting of the timestamps at the joint. I'll look into that and test correcting them to be continuous when appending, and see if that eliminates the warning. If so, we can have horseshowers test it on his AVR.

We do need to know from horseshowers if the dropout occurs when joining the untrimmed streams. My guess is that it will occur. If so, it is nothing to do with trimming but may be related to the lossless check warning.

Also not to be ruled out are SomeHumanPerson's observations and questions above.

[SomeHumanPerson]

Note that this "Lossless check failed" warning has been widely regarded as inconsequential.

I've encountered that error on a small number of single-M2TS TrueHD streams when decoding to WAV with ffmpeg, so it's not limited to potential joining or gaps correction issues.

(I verified file integrity right back to source with muliple passes when this happened, so I believe it was not related to a damaged source/demux.)

[Rocky]

Re-testing with latest ffmpeg to see if the warning is possibly an ffmpeg issue.

[Rocky]

OK, with the latest ffmpeg the warning is gone for the combined.thd but still present for the combined_trimmed.thd! So that gives me something to go on. Investigating why the trimming introduces an error...

EDIT: Looks like trimming at the end works fine but trimming at the beginning is hosed. Investigating...

EDIT2: Here is the start of trimmed2.thd:

+ 44673 994
- 44752 264
- 44792 264
- 44832 248
...

That first timestamp is wrong. It should never be odd and it should be 40 less than 44752. And the size looks dubious too. Should be fairly easy to find out why this is happening. Probably my frame iteration code is getting confused.

[horseshowers]

Tell me your command lines used for the trimming so I can duplicate what you did.

I don't have an AVR so if it is not something obvious then we're probably at the end of what I can do.

thdtrim d1.thd d1out.thd 0 5.005

thdtrim d2.thd d2out.thd 1.96 0

What I think it happening is that the AVR notices it's a different track, and has to set something up in order to decode it. This happens every time I switch an audio track in any file.

When cutting and splicing the same Atmos audio track it doesn't happen, so there must be something in the stream that's suddenly different. I don't understand how the formats work on the code level so I don't know exactly, but given it's only an issue with splicing Atmos tracks I would assume it's something in the metadata.

[horseshowers]

EDIT: If you just join the streams without trimming, does it drop out?

I will try that and report back.

EDIT: no, no issue when combining uncut files.

EDIT2: How long is the dropout? Does audio recover?

About 1s, the audio recovers shortly after.

What are you using for playback?

Tested on both an Nvidia Shield through Plex and a Dune HD Real Vision 4K played directly.

I'm led to understand that there is a problem with Plex on the NVidia Shield (and maybe other devices) that doesn't handle joined TrueHD properly. Remuxes of seamless-branching discs demuxed with recent DGDemux will work perfectly in most PC software (MPC-HC, VLC, mpv) but cause problems in Plex for unknown reasons

I have noticed that issue on the Shield when playing Apocalypse Now, I believe it's an Exoplayer bug. Playing the same film on my Dune works perfectly.

EDIT: did some more testing. The audio actually gets delayed by 1s and stays delayed until I skip ahead or back a bit, at which point it jumps back in sync. This is the same exact thing that happens with seamless discs on the Shield, which makes me think there may be a way to mitigate this in the player. Of course fixing it in the files would be ideal.
I didn't notice this because it takes a while before there's any dialogue after the joint and I never watched that far during testing.

[Rocky]

Thank you for the information and testing. I am investigating everything.

[Rocky]

Possible idea. The second stream starts with a double major, but when I trim the second stream it now has only a single major. Maybe that first double major is important. When you combine the untrimmed streams you keep that double major. We'll see. I'm going to try various patterns of manual deletion.

[Rocky]

Here is the start of trimmed2.thd:

+ 44673 994
- 44752 264
- 44792 264
- 44832 248
...

That first timestamp is wrong. It should never be odd and it should be 40 less than 44752. And the size looks dubious too.

This was a false alarm. It's fine, because that is also seen for the untrimmed stream, which gives no errors/warnings with ffmpeg.

Cutting off false pathways is a form of progress too.

[Rocky]

Hehe, things are getting interesting. Trims 0 5 for disk 1 and 5 0 for disk 2:

disk1 (untrimmed) + trimmed2 => OK
trimmed1 + trimmed 2 => FAILS

This suggests it is the cut at the end of disk 1 that is the problem. Looking at the spec
I see 'terminatorA' in the last access unit, which signals end of stream, and which we may guess
resets the decoder. So loss of the terminator may likely be the cause. This is not a problem for
our gaps correction because it is all one original stream so there is no terminator to be lost
at the gaps.

Now investigating how to mitigate this. Try to add a terminator? Delete all of the end cut access
units except the last access unit? Stay tuned...

[Rocky]

Deleting just the last minor of disk 1 causes the failure. That confirms that the lost terminator is probably the cause. Looking into how to add a terminator.

[SomeHumanPerson]

This board needs a popcorn emoji.

(I'm not being sarcastic, I find this kind of detail very interesting and keep checking in for new tidbits.)

[Rocky]

This board needs a popcorn emoji.

Post link to one or send one to me via PM and it will be installed. Glad that you are entertained.

[Rocky]

I started making a general purpose THD stream parser/dumper/editor. I'll need it to progress further on this problem.

Following is an incomplete dump output for disk 1. Still need to add substream stuff. But you can see that it is iterating and parsing correctly.

Code: Select all

----------
Access unit 1
check_nibble = 0xf
access_unit_length = 340
input_timing = 65496
format_sync = 0xf8726fba [major]
        format_info = 0x0017804f
        signature = 0x0000b752
        flags = 0x00001000
        variable_rate = 0x1
        peak_data_rate = 1664
        substreams = 4
        extended_substream_info = 0x3
        substream_info = 0xfc
        2ch_control_enabled = 0x0
        6ch_control_enabled = 0x0
        8ch_control_enabled = 0x0
        drc_startup_gain = 64
        2ch_dialog_norm = 37
        2ch_mix_level = 29
        6ch_dialog_norm = 31
        6ch_mix_level = 35
        6ch_source_format = 0x0
        8ch_dialog_norm = 31
        8ch_mix_level = 35
        8ch_source_format = 0x0
        extra_channel_meaning_present = 0x1
                extra_channel_meaning_length = 0x1
                16ch_dialog_norm = 31
                16ch_mix_level = 35
                16ch_channel_count = 11
                dyn_object_only = 0x1
                lfe_present = 0x1
        major_sync_info_CRC = 0xb013
----------
Substream directory 0
----------
Substream directory 1
----------
Substream directory 2
----------
Substream directory 3

[horseshowers]

Hehe, things are getting interesting. Trims 0 5 for disk 1 and 5 0 for disk 2:

disk1 (untrimmed) + trimmed2 => OK
trimmed1 + trimmed 2 => FAILS

I did a bit more testing and found untrimmed1 + trimmed2 and trimmed1 + untrimmed2 work correctly. Unfortunately I can't know for sure if there's a dropout at the joint of trimmed1 + untrimmed 2 as there's a couple seconds of silence anyway, but it didn't cause the 1s delay of trimmed1 + trimmed2.

This indicates to me that there's something at the end of D1 or at the start of D2 that's missing and causing the issue.

[horseshowers]

This board needs a popcorn emoji.

Post link to one or send one to me via PM and it will be installed. Glad that you are entertained.

I don't know if unicode is implemented on the browser level or in the forum code, but works on my machine.

[Rocky]

I did a bit more testing and found ... trimmed1 + untrimmed2 works correctly. Unfortunately I can't know for sure if there's a dropout at the joint of trimmed1 + untrimmed 2 as there's a couple seconds of silence anyway, but it didn't cause the 1s delay of trimmed1 + trimmed2.

That combination produces the lossless check error with ffmpeg. disk1 + trimmed2 is the only combination with trimming that does not produce an error, so it points to the end of disk1 as the culprit.

Thank you for the popcorn emoji. It's almost not identifiable on my small laptop so I'm going to look around.

[Rocky]

Popcorn smilie is now available.



SomeHumanPerson

[Rocky]

A 'lossless check' occurs at every major. It's a check over the entire stream from the beginning of decoding up to that major. Any deletion of minors prior to that major will violate this check. AVRs can produce undefined results when the lossless check is violated. For transcoding it's usually unimportant.

We do not want lossless violations! Deleted data can be replaced with a single major containing appropriate data.

There is still much left to contemplate.

[SomeHumanPerson]

A 'lossless check' occurs at every major. It's a check over the entire stream from the beginning of decoding up to that major. Any deletion of minors prior to that major will violate this check. AVRs can produce undefined results when the lossless check is violated. For transcoding it's usually unimportant.

When you say "entire stream", this must be limited per individual piece/file, right? So your gap correction technique has avoided any lossless check violations up to this point because you trim at the end of the previous piece/file, and then the next piece/file "starts over" in terms of the lossless check?

We do not want lossless violations! Deleted data can be replaced with a single major containing appropriate data.

Do you mean here that you know exactly what the lossless check entails and can re-calculate to "forge" a new major with the corrected result? Does the check use a simple hash of some kind?

[Rocky]

When you say "entire stream", this must be limited per individual piece/file, right?

It's not clear. The calculation must be reset at the joints, but it seems that the check of all previous data is made at the double major before it is reset. Exactly how the seamlessness works is still unclear to me. In some CRC implementations initialization involves setting the sum to 0 and then updating twice before going into on-the-fly single updates at the single majors. Here's where it is interesting: there is a double major at each cut point, so it looks analogous. I'm still looking into this, but not too seriously because I think these failures are not the root cause for the Atmos dropouts with AVRs.

So your gap correction technique has avoided any lossless check violations up to this point because you trim at the end of the previous piece/file, and then the next piece/file "starts over" in terms of the lossless check?

No. The gaps correction does cause lossless check errors. I may have been unclear about that. But it is not an issue for transcoding. And now I believe it is not the root cause of the dropouts with the AVRs, but rather loss of data in the 4th (Atmos) substream. Without knowing more intimate details about Atmos encoding and decoding, I'm pretty much stymied. The reason I no longer think it's the lossless check failures is because they do not seem to affect decoding (e.g., with VLC etc.), and because horseshower's dropouts occur only with Atmos and not simple TrueHD.

Do you mean here that you know exactly what the lossless check entails and can re-calculate to "forge" a new major with the corrected result? Does the check use a simple hash of some kind?

The parsing and handling of the lossless check can be seen in Ian Caulfield's ffmpeg support for TrueHD. His code is also valuable in filling in missing info in the MLP spec. The check is a running CRC-32 down-converted to 8 bits and compared to the lossless check field in the major frame's restart header. That much is clear. Ian's decoder reports the failures but that's all; it doesn't stop decoding or invoke any recovery process. Yes, the forging was an idea but I no longer think the lossless check failures are the root cause. Note that Ian's code ignores the 4th substream so it does not support Atmos for transcoding. Anyway, you can't put Atmos into (say) FLAC.

I see all over the web cases of Atmos dropouts, even for linear play with no cutting. I'm ready to admit defeat. The thdtrim utility may be useful for transcoding in some use cases, but the cutting would be more appropriately done in the uncompressed domain, given that we don't know how to salvage the Atmos.

Well, that's my current thinking. I don't see any forward path at this point. But prove me wrong!

[SomeHumanPerson]

Ah, I did misunderstand a couple of things. Thank you for the clarifications.

There are a few people working on full Atmos decoding/re-encoding and allegedly making good progress, so maybe this will become moot in the not-too-distant future where we can just fully decode/extract Atmos and re-encode it again using the "available" Dolby Encoding Engine or Media Encoder tools.

I was just hopeful that you might be able to end-run that whole process, but it's understandably difficult without access to Dolby's specifications.



Atmos dropouts do seem to be a widespread thing, makes me wonder about bad hardware decoder implementations, or bugs in Dolby's official tools for studios/publishers. Either are entirely possible.

[horseshowers]

Well, that's my current thinking. I don't see any forward path at this point. But prove me wrong!

Given untrimmed1+trimmed2 and trimmed2+untrimmed1 works, I wonder what would happen if you were able to keep just the first bit or two of untrimmed2 (or the last of untrimmed1) when trimming it. I tried doing something like this with MKVToolnix but I don't think that tool can cut precisely enough, as it broke the audio stream completely in VLC. If you add an argument to thdtrim that lets me customize how much of the very start is kept, I can do some testing with various values and see if any of them work.