DGDemux development

DAE avatar
domy
Posts: 28
Joined: Fri Mar 20, 2020 10:50 am

Re: DGDemux development

Post by domy »

The software is called Audacity, it's free.

I suppose that begs the question how we define out of sync. I consider "in sync" to be each segment's (.m2ts) raw audio stream, concatenated together with duplicated frames at the tips overlapping, regardless of the video signal. This only holds if we assume that .m2ts files are in sync and that there aren't any duplicated video frames (I'm not sure how to verify the latter). In that case the total length is 1:47:12.635. DGDemux's TrueHD stream duration is 1:47:12.6367. Both are perfectly in sync up until the end of segment #1. By the third segment, DGDemux's output (top track) is 80 samples late - this is 8 seconds before the film ends:

moana-dgdemux-total-sync-annotated.png

By that definition, I'd say this is out of sync.

Sidenote: If I count the video frames of the movie, I get a total length of 1:47:12.6345: 154,229 * (24000 / 1001):

Code: Select all

$ ffmpeg -i "Moana.mkv" -map 0:v:0 -c copy -f null -
154,229
If you already have any Disney/Pixar UHD blu-ray, tell me which and I'll most likely find the same issue. I chose Moana for this example because it's only three segments and you can quite easily see the duplicated waveforms. I can also try this with a disc that has many more segments and see how much it accumulates, if you want.

Oh and since I forgot to mention it before, I used DGDemux 1.0.0.20.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

I define it as audio within 50 ms of the video at all points in the movie. There should be no perceptible async at any point. The desync you show appears to be 1ms if I read the app correctly.

Let's continue with CARS_2 for now (I will buy Moana). I also have TOY_STORY_4_3D. I will get Audacity.

I do not currently do any gaps processing for THD but only for the embedded AC3 (when it is demuxed to a separate file). That is because the THD frames are so short that they do not accumulate to an out of sync condition (per my definition), even for a large number of M2TS files. Nevertheless, I do not rule out doing processing for THD. It may also be possible to combine methods 1 and 2 for gaps correction, and achieve what you call perfect sync.
DAE avatar
domy
Posts: 28
Joined: Fri Mar 20, 2020 10:50 am

Re: DGDemux development

Post by domy »

So I just took a look at Toy Story 4 (US, UHD)'s THD situation. It's a ridiculous disc with 61 segments for the 00800 playlist. All but three segment boundaries have duplicated audio frames. (segments #76-77, #81-82 and #109-110 did not overlap)

Movie length: 143,939 video frames, 1:40:03.4557916666...
THD tracks end-to-end aligned: 288,168,120 samples, 1:40:03.5025
THD tracks aligned with dupe frames overlapping: 288,165,840 samples, 1:40:03.455

So by the end, the audio is 47.5 ms late. That meets your goal of <50 ms, and Toy Story 4 is one of the most segmented UHD blu-rays out there that I know of, so this is definitely an extreme case ...

Until I discovered that Monsters University (US, UHD) has 135 segments, wow. It appears that it suffers from the same issue: the movie is 1:43:48.389, but DGDemux's THD track of that disc is 109ms longer, coming in at 1:43:48.498. After muxing DGDemux's THD and HEVC stream back together, I've measured a 92ms delay (+/- 20ms or so*) at the very end (the pixar lamp scene) against the reference 00160.m2ts. It is noticeable if you know what to look/listen for. This one exceeds the 50ms threshold around midway through the film.

Aside from that:
  • I don't get any pops/cracks during THD playback, unlike with MakeMKV, which is great :)
  • The GUI crashes when selecting a playlist if the disc's path contains a ] (closing square bracket).
(* It's a comparison of 120fps slow-motion recordings taken of the movie overlayed with Windows's volume mixer - kinda jank but works).
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Thank you, domy, for your test results. That Monsters University is pretty insane with 135 segments. I'm going to have to implement gaps processing for THD. I'll buy the disk and get on it when I finish up HEVC seeking.

Let's calculate: 135 x 0.8ms (size of a THD frame) = 108 milliseconds. That is close to your 92ms and 109ms, so this is theoretically understood and thus fixable.

I have the ] bug fixed locally and will include it in the next slipstream.

Stay safe and well, my friend!
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

zqslzwzw wrote:
Sat Mar 21, 2020 7:37 am
I would like to report two minor issues of the latest DGDemux 1.0.0.20 with regard to the option 'Do not split THD'.
1) With the latest DGDemux, no matter whether the option is checked, the THD stream(s) will be demuxed.
2) If the option is checked, the language property of the generated .thd file is missed, such as '00800 PID 1100.thd'.
I just noticed this as it got scrolled off. I'll look into it tomorrow. Thank you for your report.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

zqslzwzw wrote:
Sat Mar 21, 2020 7:37 am
1) With the latest DGDemux, no matter whether the option is checked, the THD stream(s) will be demuxed.
I have duplicated this for do_not_split and have it fixed locally. But I can't duplicate it for when do_not_split is not checked. Can you try that again please?
2) If the option is checked, the language property of the generated .thd file is missed, such as '00800 PID 1100.thd'.
The name should be rewritten after the file is closed (when demuxing finishes) with full info in the name. Is that not happening for you?
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

zqslzwzw hasn't been here for a while so I am going to release the fixes that I have:

* Opening MPLS with [] characters.

* Fix always demuxes THD even when disabled for do not split mode.

The second one also needs to be done for DGIndexNV.

After that, back to DGHDRtoSDR stuff.
User avatar
Bullwinkle
Posts: 338
Joined: Thu Sep 05, 2019 6:37 pm

Re: DGDemux development

Post by Bullwinkle »

Rocky wrote:
Sun Mar 22, 2020 11:16 am
That Monsters University is pretty insane with 135 segments.
Still waiting for the disk to arrive. Not an essential product? GTFOOH. STOMP!
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Monsters disc is here and being ripped right now.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

@domy

How did you get Audacity to open your thd file? I have Audacity 2.3.3 and ffmpeg 2.2.2, and I have set it in Preferences/Library. But still, when I try to open my thd file, it says cannot and suggests installing ffmpeg.

Audacity log:

11:32:08: File name is D:\tmp\DGDemuxGUI test\00801 PID 1100 48000 8ch eng DELAY 0ms.thd
11:32:08: Mime type is *
11:32:08: Opening with libsndfile
11:32:08: Opening with liboggvorbis
11:32:08: Opening with libflac
11:32:08: Opening with lof
11:32:08: Opening with libav
11:32:15: Error: FFmpeg: avformat_find_stream_info() failed for file D:\tmp\DGDemuxGUI test\00801 PID 1100 48000 8ch eng DELAY 0ms.thd
11:32:15: Error: can't flush file descriptor 5 (error 5: Access is denied.)
11:32:15: Error: Importer::Import: Opening failed.

Note that import of the separate embedded AC3 succeeds. Also, when opening Audacity it shows that ffmpeg was loaded.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

So it will open some sample files I downloaded but fails on anything from a UHD bluray. Any ideas?
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Did you try to Import as Raw Data?
Worked on my system
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

It may appear to work. But raw data is uncompressed and raw with no metadata. A THD file is not that. You can read anything as raw!
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Well I found this
https://forum.audacityteam.org/viewtopic.php?t=88387
Seems there is a size issue, apparently it will only open smaller thd (size) files
Also ffmpeg plugin 2.2.2 does not support "thd atmos"
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Quick test
BluRay thd track with no atmos 2.38 GB opens fine
UHD thd track with atmos 4.00 GB does not open
UHD thd track with atmos 3.49 GB does not open
Issue does appear to be atmos
User avatar
MeteorRain
Posts: 130
Joined: Fri Jul 25, 2014 4:04 pm
Location: New Jersey

Re: DGDemux development

Post by MeteorRain »

Kindly remind that you mis-wrote someone as me in the binary update ;)
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

MeteorRain wrote:
Wed Apr 01, 2020 5:24 am
Kindly remind that you mis-wrote someone as me in the binary update ;)
Fixed. Thank you for alerting me.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Yes, my research confirms it is caused by the atmos. I wonder if what domy did was convert to flac using a recent ffmpeg and then rename it as file.thd_. You see that _ in his audacity screenshot. I will try it this way.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Or maybe he used ffmpeg to just strip the atmos. I know that can be done but don't have a command line for it. Anybody?
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

OK, I found a working command line:

ffmpeg -i file.thd -c:a copy -bsf:a truehd_core out.thd

Now out.thd can be opened by audacity (with ffmpeg installed). Here are the timings I get for Monsters:

video: 6258.425 seconds
embedded ac3 (corrected by gaps processing): 6258.432 (good sync to video, 7ms async by end of stream)
truehd: 6258.528 (bad async of 103ms by end of stream)

The truehd is out of sync and so it is clear that I need to implement gaps processing for THD. It may also be interesting to add an option to DGDemux to strip the ATMOS on demuxing.
DAE avatar
domy
Posts: 28
Joined: Fri Mar 20, 2020 10:50 am

Re: DGDemux development

Post by domy »

I used eac3to to rip the THD from the m2ts and convert it to mono flac, so that I could open it up in Audacity and easily compare multiple segments/streams. For example:

Code: Select all

eac3to "BDMV\STREAM\00055.m2ts" 2: 1_55.flac -mono
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

I see, thank you. Try my method. It's much faster and doesn't re-encode.

Working on THD gaps correction. It's not straightforward due to the embedded AC3 and the major/minor frames distinction.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

I have it working but have to double check that the embedded AC3 is not getting hit due to a small technical matter. The durations I get are now:

video: 6258.418 seconds
thd: 6258.420 seconds

Sync is spot on all the way through. So that looks pretty good. :D
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Released THD gaps processing in slipstream 22. Makes mincemeat out of MONSTERS_UNIVERSITY!
User avatar
Bullwinkle
Posts: 338
Joined: Thu Sep 05, 2019 6:37 pm

Re: DGDemux development

Post by Bullwinkle »

OK guys, Rocky is busy on something or other so I'm gonna finish off the garbage skipping feature. Our test disk is CROUPIER reported by mini-moose. Love that nick by the way. Nobody calls me mini! :lol:
Post Reply