DGDemux development

[Bullwinkle]

Ah, thought y'all might want to see the actual cutlist for MONSTERS 00800.mpls:

trim 0 =149.608333 =149.609167 =246.039167 =246.040000 =387.390000 =387.391667 =435.522500 =435.523333 =464.760833 =464.761667 =486.366667 =486.367500 =521.652500 =521.653333 =602.735000 =602.735833 =656.706667 =656.708333 =709.093333 =709.094167 =732.785000 =732.786667 =746.925000 =746.925833 =782.920833 =782.921667 =803.567500 =803.569167 =817.707500 =817.708333 =850.407500 =850.408333 =893.577500 =893.579167 =932.242500 =932.243333 =972.909167 =972.910833 =993.973333 =993.974167 =1024.921667 =1024.922500 =1043.065833 =1043.067500 =1117.265833 =1117.266667 =1142.375000 =1142.375833 =1230.297500 =1230.298333 =1267.544167 =1267.545833 =1282.435000 =1282.435833 =1387.875000 =1387.875833 =1401.890000 =1401.890833 =1417.990000 =1417.990833 =1432.672500 =1432.674167 =1470.711667 =1470.712500 =1498.282500 =1498.284167 =1510.962500 =1510.963333 =1804.548333 =1804.549167 =1868.488333 =1868.489167 =1883.755000 =1883.756667 =1916.998333 =1917.000000 =1987.904167 =1987.905833 =2000.459167 =2000.460000 =2007.300000 =2007.300833 =2018.812500 =2018.813333 =2060.813333 =2060.814167 =2079.833333 =2079.834167 =2363.285000 =2363.286667 =2374.964167 =2374.965000 =2400.324167 =2400.325833 =2414.172500 =2414.173333 =2431.440833 =2431.441667 =2454.798333 =2454.799167 =2467.311667 =2467.312500 =2620.757500 =2620.758333 =2634.439167 =2634.440833 =2639.570000 =2639.570833 =2661.050833 =2661.051667 =2672.730000 =2672.730833 =2694.835833 =2694.836667 =2713.522500 =2713.523333 =2893.703333 =2893.704167 =2905.007500 =2905.009167 =3053.323333 =3053.324167 =3066.629167 =3066.630000 =3214.110833 =3214.111667 =3225.956667 =3225.957500 =3231.045833 =3231.046667 =3320.636667 =3320.637500 =3341.909167 =3341.910833 =3390.334167 =3390.335833 =3671.408333 =3671.410000 =3753.365833 =3753.366667 =3765.670833 =3765.671667 =3853.885000 =3853.885833 =3882.247500 =3882.248333 =3900.058333 =3900.060000 =3939.432500 =3939.434167 =3951.904167 =3951.905000 =4069.147500 =4069.148333 =4094.465000 =4094.465833 =4127.832500 =4127.833333 =4195.067500 =4195.068333 =4208.206667 =4208.207500 =4214.880833 =4214.881667 =4233.734167 =4233.735833 =4260.470000 =4260.470833 =4274.776667 =4274.777500 =4298.593333 =4298.594167 =4374.795833 =4374.797500 =4416.464167 =4416.465833 =4531.580833 =4531.582500 =4605.030000 =4605.030833 =4619.253333 =4619.254167 =4783.543333 =4783.544167 =4797.558333 =4797.559167 =5357.035000 =5357.035833 =5369.131667 =5369.133333 =5421.643333 =5421.644167 =5438.119167 =5438.120000 =5510.817500 =5510.818333 =5557.115000 =5557.115833 =5643.201667 =5643.202500 =5664.807500 =5664.808333 =6202.011667 =6202.012500

[Rocky]

Wow, Bullwinkle! You really pulled out a rabbit this time.

If anybody has a sox.exe with w64 support they can duplicate the adjustment for MONSTERS using the cutlist as you described. Of course, we will be integrating all this to make it transparent to the user.

You want someone to misbehave? OK, you are a very poopy smelly moose that is useless for anything, except moose stew. Yummy!

[Bullwinkle]

Watch me stomp this miscreant! ... Arggh, one mod can't stomp another. Woe is me, unto Matilda.

[Mr. Peabody]

Reference there, I suppose, is to https://en.wikipedia.org/wiki/Matilda_(1996_film).

"When Matilda's parents refuse to enroll her into school, she retaliates against them, first by adding hydrogen peroxide bleaching agent to her father's hair tonic and then by gluing his hat to his head. When Harry rips up the library's copy of Moby-Dick (which he finds appalling because of the title) and forces Matilda to watch television instead, Matilda becomes increasingly enraged until the television mysteriously explodes."

[renols]

Hi.

Sounds exciting.

Where does that leave ones that want to use the thd file though?

We will have to choose between 1, file being out of sync, or 2, file in sync but may cause problems due to file not being according to specs anymore?

mkvmerge doesn't complain about the thd/atmos file from those disks with 80+ m2ts files when muxing, so not sure if it even matters.

It is great that we are getting to the bottom of this, so we know what options we have.

Much appreciated

renols

[Bullwinkle]

Top of the mornin' to you, renols.

Sounds exciting.

Yes, Matilda is a great flick.

We will have to choose between 1, file being out of sync, or 2, file in sync but may cause problems due to file not being according to specs anymore?

Yes, for now. But don't forget the third option. Most people will want to go to FLAC anyway. It's way smaller than THD. For example, 7 GB -> 2.5 GB.

mkvmerge doesn't complain about the thd/atmos file from those disks with 80+ m2ts files when muxing, so not sure if it even matters.

You talking about the gaps adjusted file? Sure, and players don't complain either, but ffmpeg can't transcode them.

And as Rocky said we will revisit doing gaps correction on the THD while keeping it spec-legal and while keeping it from bombing ffmpeg's transcoding to FLAC. And we have given a way to make 1ms-synced FLAC so maybe you are right and it doesn't matter. Still, would be good to get to the bottom even if only for theoretical understanding.

Your kindness has earned you total exemption from future stompings for one year.

[Sherman]

Musicians always keep things in perspective.

https://youtu.be/krxU5Y9lCS8?t=155

This one too, Levi!

https://youtu.be/nrIPxlFzDi0?t=14

[Levi]

You are a fine boy, Sherman. Study your scales and sing from the heart!

[renols]

Hi.

Thanks for the explanation.

As said earlier the technical side of things are not really my strong side

Been trying to do some reading, and am I correct that the Atmos part will be lost when doing a FLAC file?

I think it is great news that there is now a possiblilty, to get an audiofile that is perfectly in sync.

If there is a way to get TrueHD/Atmos file somehow to be in perfect sync, or close to it, it would be awesome.

As you said yourself eac3to is completely hopeless when heavy seamless branching is there, and as domy wrote Makemkv is also prodcucing a thd file which is still more than the 25 ms out of sync.

I haven't tried tsmuxer to see what it does and what the result from using that would be. Looks like tsmuxer is now open source, and quite some changes has been made to the program recently. Maybe I should try and demux one of the disks I have. I don't think I have any disks with more than maybe 80 m2ts files.

This seems like quite an obstacle to get past.

renols

[renols]

Hi.

Just tried to demux Cars 2 4k disk using tsmuxer.

The log shows this:

tsMuxeR version git-cfa580e. github.com/justdan96/tsMuxer
Decoding TRUE-HD stream (track 1): AC3 core+TRUE-HD. Peak bitrate: 6894Kbps (core 640Kbps) Sample Rate: 48KHz Channels: 7.1
TRUE-HD stream (track 1): overlapped frame detected at position 00:01:48,319. Remove frame.
TRUE-HD stream (track 1): overlapped frame detected at position 00:03:51,680. Remove frame.
TRUE-HD stream (track 1): overlapped frame detected at position 00:08:34,751. Remove frame.
TRUE-HD stream (track 1): overlapped frame detected at position 00:09:56,447. Remove frame.
.
.
.
.
.
TRUE-HD stream (track 1): overlapped frame detected at position 01:45:51,743. Remove frame.
Flushing write buffer
Demux complete.

When trying to use eac3to to create a flac file from the file created by tsmuxer it shows this:

[truehd @ 026b2ce0] End of stream indicated.
[truehd @ 026b2ce0] Lossless check failed - expected 00, calculated b5.
[truehd @ 026b2ce0] End of stream indicated.
[truehd @ 026b2ce0] Lossless check failed - expected 00, calculated 45.
[truehd @ 026b2ce0] End of stream indicated.
[truehd @ 026b2ce0] Lossless check failed - expected 00, calculated 2e.
[truehd @ 026b2ce0] End of stream indicated.
[truehd @ 026b2ce0] Lossless check failed - expected 00, calculated 82.
[truehd @ 026b2ce0] End of stream indicated.

An goes on with messages like this.

Guess tsmuxer just removes frames. Not sure how to get the length of the thd file, so I am not sure wether the resulting thd file is actually in sync or not.

renols.

[Bullwinkle]

Been trying to do some reading, and am I correct that the Atmos part will be lost when doing a FLAC file?

Yes, of course. But how many people really have elevated speakers and upward blasting sound bars?

I think it is great news that there is now a possiblilty, to get an audiofile that is perfectly in sync.

Sure it's great but it is a reality, not a possibility.

If there is a way to get TrueHD/Atmos file somehow to be in perfect sync, or close to it, it would be awesome.

Not yet determined and we will see.

I haven't tried tsmuxer to see what it does and what the result from using that would be. Looks like tsmuxer is now open source, and quite some changes has been made to the program recently. Maybe I should try and demux one of the disks I have. I don't think I have any disks with more than maybe 80 m2ts files.

That would be great if you can do that for us. Use the 80-M2TS disk.

[posts crossed]

[Bullwinkle]

Those warnings are unimportant. Standby, I will give you a small utility to get the length of the THD.

[Bullwinkle]

http://rationalqm.us/misc/truehd.exe

Syntax:

truehd file.thd

E.g.,

D:\Don\Programming\C++\TrueHD\x64\Release>truehd "d:\tmp\DGDemuxGUI Test\00800_raw.thd"
58693 major, 7415504 minor, 7474197 total, 6228.497500 duration

There is no error checking.

You can also run it on the raw demux from DGDemuxGUI and we can compare the two to see the adjustment it made.

Please post results and the number of frames and frame rate of the video. Also check that ffmpeg can transcode it as well.

ffmpeg -i file.thd file.flac

Thank you, renols.

If the sync looks good and ffmpeg transcodes it, then I will look at the tsmuxer source code to see how they are deleting frames. If it is usable, it still probably won't have 1ms-sync performance. So then we will have these options:

1. Demux THD raw. Keeps ATMOS.
2. Demux to THD with clean gaps correction but low granularity. Keeps ATMOS.
3. Demux to FLAC with 1ms granularity. Loses ATMOS.

If it does sync to 1ms granularity then we could drop option 3. A THD frame is 0.8ms so theoretically it is possible. We always have option 3 as a fallback. Let's see.

Wow, man. I am totally psyched!

[renols]

How long time should it take to proccess the thd file.

Have "truehd.exe 00004.track_4352.thd" running for more than 5 minutes now, and nothing seems to happen.

renols

[Bullwinkle]

Be patient. How big is the THD file? It could hang if there is a spec violation. Can you upload me the file if I give you Rocky's FTP?

Could take quite a while if you are not on an SSD or you have a Pentium III.

[renols]

Hi.

It is still just sitting there.

I can upload the file to your ftp, no problem.

Am doing a demux with dgdemux 23 now.

The file from tsmuxer is thd+ac3 though. That might be why the truehd program is hanging?

Not sure how to split them. tsmuxer doesn't have that option.

The thd+ac3 file from tsmuxer is 6,5 GB, but no problem for me to send it. Might have to rar it up, to make sure the transfer doesn't introduce errors.

Let me know what to do and I will be happy to help.

As said I am using cars 2 4k disk and 00004.mpls

renols

[Bullwinkle]

Yes, probably the AC3 is screwing it up. I always split them when demuxing for my testing.

OK, I have CARS_2. Please tell me how to demux it with tsmuxer. I know nothing about tsmuxer so please be thorough.

Another way to get the length is to load it in Audacity and then cursor to end of stream to see the time. Transport/Cursor to/Track End. Then look at the time at the bottom.

[renols]

Hi.

I am using tsmuxer w64-nightly-2020-03-16--01-11-36.zip

There are versions which are newer than this, but I have only tried with that version since I already had it on my drive.

The same version can be downloaded from:

https://bintray.com/justdan96/tsMuxer/t ... 1-36#files

When unpacked run tsMuxerGUI.exe

Under Input click "add" top right corner. Open 00004.mpls of Cars 2 disk and a list of its content will be shown in the "tracks:" box.

Select just the TrueHD file.

In the Output section choose output location and select "Demux".

Then simply click "Start demuxing" at the bottom of the screen, and wait for it to get done.

As the file gets proccessed, you should see the same lines in the log window that I pasted, where it says each time it removed a frame.

For some reason the resulting file will be named .ac3

renols

[Bullwinkle]

Cool. Thank you so much, renols. No-stomping exemption extended by 5 more years.

Speaking of which... had the idea to have a new member rank called "Moose Approved". You may be one of them.

[Bullwinkle]

Bummer. Audacity doesn't accept it with ATMOS, and ffmpeg won't strip ATMOS if the file has embedded AC3. ffmpeg also won't transcode it to flac. Trying to find a way to get its length... Maybe fix my utility?

EDIT: OK, stripped AC3 with eac3to and now my utility works. Standby for comparison to video...

EDIT2: OMG! Here are the numbers:

video: 6375.16046 seconds
thd: 6375.16000

So it is in-sync with THD frame granularity. And ffmpeg can transcode the AC3-stripped THD to FLAC.


tsmuxer should add the abilty to split the THD and AC3, IMHO. It's not that difficult, even Rocky was able to do it.

Off to study the tsmuxer source code... Oh, happy Day!

[admin]

renols, you are now Moose Approved. Along with all our other strong contributors/generally awesome members, neither of which appears to apply to me.

[Rocky]

Oy, take a little nap and look what happens. Move over, Bullwinkle, I'm back. And what about Squirrel Approved?

[Natasha]

And what about Squirrel Approved?

Here, have a peanut and be happy.

[Bullwinkle]

Natasha, you are NOT Moose Approved.

admin, I'm still deciding about you.

I approved myself. Duh!

[Curly]

I love how when you go to the members list and sort on rank it shows Moose Approved above Site Admin.