Support forum for DG Tools

hydra3333 wrote: ↑

Sat Nov 16, 2019 10:15 pm

If there was a like button on the forum, I'd click it

It seems you got your wish.

viewtopic.php?f=5&t=859

Levi is Moose Approved with only 3 posts. Is that fair?

You'll understand one day...if you grow up. Bwahahahahaha!

Hey Natasha

Sherman wrote: ↑

Fri Apr 24, 2020 3:46 pm

Is that fair?

It's about being a strong contributor/generally awesome. So far it's not looking good for you. Levi, on the other hand, is way beyond awesome. He is a God. He may well be the reincarnation of Apollo.



Great image there, gonca. Love it! Here's what I would say to Natasha: Does it hurt?

I would remind her that Rocky is a flying squirrel, that could easily be her nose

When Rocky gets in a bad mood...



Does it hurt?

You guys are pushing your luck.

Hmm. Looks like when DGDemux does gaps correction it mangles the major_sync_info() field, which makes the CRC wrong and explains why ffmpeg says invalid data. Weird. OK, let's find out why this is happening.

I don't think tsmuxer is doing anything special; it's just not mangling the major_sync_info().

Rocky wrote: ↑

Fri Apr 24, 2020 8:40 pm

OK, let's find out why this is happening.

Great plan. Who'd a thunk?

Shut up, Natasha. gonca is right, you're a nobody.

Be nice to the ladies, Sherman. One day you will understand why.

Here are two things I just discovered that threw me for loops.

1. When stripping the AC3 core with eac3to you have to give the -keepDialnorm option. This is why I thought I was mangling the major sync info. When you do this they are the same again. domy has asked for an option to remove it in DGDemux but it's not there yet.

2. This one is huge! If you demux only the THD stream with tsmuxer, gaps processing is not performed. You have to also demux the video at the same time to get gaps processing. Without video, it says it is deleting frames but it apparently isn't. To see that just compare the two THD files, one demuxed with video and one without. They are different! EDIT: See the next post for the real reason the files are different.

Hopefully knowing all these things now my debugging can get back on track.

Oh...My...God!

tsmuxer is not doing proper gaps processing at all. It is just truncating the audio when the video runs out! That is why the THD streams differ depending on if you also demux video.

I demuxed the raw THD with DGDemux (no gaps processing and with splitting the THD and AC3). Resulting THD duration is 6375.219167. Then I demuxed using tsmuxer with video and stripped the AC3 core keeping dialnorm. Resulting THD duration is 6375.160000. It thus looks like tsmuxer is doing gaps processing. Comparison of these two files should show all the frame deletions made by tsmuxer. But when you compare them (you can use fc /b) the only difference is that the tsmuxer one has truncated all the audio beyond the end of the video. There are no individual frame deletions! So tsmuxer is producing equal duration video and audio, but it is not actually doing any sync correction. And looking at the tsmuxer source code, it claims to only delete a single transport packet, which would be wrong because a THD frame spans many packets. tsmuxer is a fraud (at least as far as sync correction is concerned)!

So tsmuxer is useless for our purposes. I'll have to go back to MakeMKV for clues about how to delete individual frames in the compressed THD domain. I can make an MKV and then demux the THD for comparison to DGDemux. Let's hope MakeMKV is not also a fraud.

Oh what a tangled web!

Using MakeMKV to rip CARS_2 to MKV...

Aha! Everything is clear now.

MakeMKV is not a fraud. It is deleting only major frames and not performing any other adjustments, e.g., to the timestamps. Thus, I would expect ffmpeg to bomb when converting the THD stream demuxed from the MKV to FLAC. And it does! It fails exactly as does DGDemux. I suppose it is not surprising as there was never an answer to Mike's post asking how to properly adjust timestamps (last post here: https://github.com/Nevcairiel/LAVFilters/issues/282). Thus, MakeMKV is implementing option 2 below. tsmuxer is implementing option 1 but truncating the audio to the length of the video when the video is also demuxed along with the audio.

Therefore, unless one of our esteemed members finds something to challenge this, it looks like we will have to go with our solution for creating a clean in-sync flac file. To remind you, the options are proposed to be:

1. Demux the stream raw. Keeps ATMOS but no sync correction and does not bomb ffmpeg FLAC conversion. Use when not too many M2TS files are in the playlist.

2. Demux with gaps correction. Keeps ATMOS and does sync correction but bombs ffmpeg FLAC conversion. Use when you have too many M2TS files. You can use the THD file directly in (say) mkvtoolnix, as players don't care about the spec violation.

3. Demux raw and correct sync in the uncompressed domain. Does sync correction and does not bomb ffmpeg but loses ATMOS. Not a big deal because any conversion to FLAC would lose ATMOS. Use when you want a sync-corrected FLAC file.

I do not see any way to make a valid sync-corrected THD file that retains the ATMOS. Maybe it could be possible by re-encoding the clean FLAC from 3 to THD and re-adding the ATMOS metadata. Not something that is going to be feasible absent a full-featured THD encoder and some technology to save and restore ATMOS metadata. Forget about it!

I conclude that method 3 (successfully concept tested) is the ONLY way to make a sync-corrected FLAC file. Your feedback on this conclusion and proposal will be greatly appreciated. domy? renols? gonca?

Personally I just keep the THD file + Atmos, but for those that transcode to other formats, option 3 is the way to go.
The movies I do don't seem to have to many m2ts files, normally.

gonca wrote: ↑

Sat Apr 25, 2020 11:06 am

Personally I just keep the THD file + Atmos, but for those that transcode to other formats, option 3 is the way to go.
The movies I do don't seem to have to many m2ts files, normally.

So option 1 for you. And if you do have too many M2TS files then you can use option 2. If you need FLAC, then option 3.

Even so, there is no guarantee that option 2 files will not be found to be problematic somewhere, although tested players don't seem to mind it. And for archiving, you wouldn't want option 2.

Correct.
As long as DGDemux retains the 3 options all bases are covered.
Until someone figures out how to correct the timestamps this is as close to perfection as we are going to get

Yes, but it's not even clear that correcting timestamps will be enough, because the buffering model will be broken, as explained earlier. We would have to become a THD encoder in effect.

Alright, we have gonca's endorsement. One down! Let's see if the others want to weigh in before going full steam ahead.

have to become a THD encoder in effect

which is way past the scope of DGDemux.
As long as we have the 3 options all bases are covered.
DGDemux is probably one of the few solutions out there for this issue, without using Dolby's encoding tools ($$$)

probably? I would say definitely the only solution for clean, transcodable sync correction without THD re-encoding. OK, quibbling on your language, I know.

Rocky and Bullwinkle rule!

Gonna Moose Approve some more members that I missed yesterday.

Hi.

I am also totally fine with the three options.

One thing with ragard to makemkv. the version you have used is the most recent one? (1.15.1). I ask because someone wrote this in an earlier post:

"MAKEMKV does not produce errors in FFMPEG now in the latest version, whereas the previous version did. I guess this is like you suspected Rocky, they remove the major frames what we see comparing DGDEMUX and MAKEMKV output."

If no software, other than some TrueHD encoder can produce a thd file without "errors", then so be it. I guess up to 25 m2ts files, it would still be OK to use the thd file without gap removal, as that would give a 20 ms delay, according to the 0,8 ms per m2ts file you gave us earlier.

So, three options it is, until some other solution may come along.

Mike might also want to keep his cards close, so maybe he isn't interested in sharing what he does, if he does something different. Don't know if you have communicated with him in the past.

Again thanks for the effort put into this. Just makes a great program even better.

renols

renols wrote: ↑

Sat Apr 25, 2020 1:51 pm

"MAKEMKV does not produce errors in FFMPEG now in the latest version, whereas the previous version did.

Oh, thank you for pointing out my oversight! I have been using 1.14.7. Maybe Mike figured something out. Need to repeat the testing ASAP. After my lunch.

I had read of seamless branching issues with 1.15.1 so didn't upgrade. You can google it.