DGDemux development

User avatar
Curly
Posts: 712
Joined: Sun Mar 15, 2020 11:05 am

Re: DGDemux development

Post by Curly »

This domy guy is pretty sharp. Am I wrong?
Curly Howard
Director of EAC3TO Development
User avatar
Natasha
Posts: 150
Joined: Wed Nov 20, 2019 11:11 am

Re: DGDemux development

Post by Natasha »

Wrong would be a gross understatement. Curly, you are a waste of oxygen. And neurotransmitters, if you have any.
User avatar
Sherman
Posts: 576
Joined: Mon Jan 06, 2020 10:19 pm

Re: DGDemux development

Post by Sherman »

Stalking? :scratch:
Sherman Peabody
Director of Linux Development
DAE avatar
renols
Posts: 149
Joined: Tue Feb 22, 2011 2:34 am

Re: DGDemux development

Post by renols »

Hi.

I have several disks here I can test with, but I need some way to meassure sync other than just looking at the muxed mkv afterwards.

Could you explain the correct steps to be able to check if audio is actually in sync, so that I can check the titles I have available?

renols
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

I usually check that the duration is correct and then do spot checks playing in MPC-HC. You can also use MPC-HC delay adjustment. Also, you can use an Avisynth filter to overlay the audio waveform on the video. Or use Audacity like domy. Maybe he will chime in to help.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

It's turning out to be quite challenging to implement this on-the-fly for DGDemux/DGIndexNV. We definitely want to do the processing on-the-fly rather than as a second pass at the end.

I have to:

1. malloc/free data for the gaps processing (THD frame buffers, etc.).

2. Convert file writes by transport packet to writes by THD frame.

3. Buffer the previous 2 frames and write behind by 2 frames. Output the unwritten 2 frames at end of stream. Shift the frames along using pointer games rather than memcpy's.

4. Parse the major/minor and detect double majors.

5. Suppress the minor frame preceding a double major.

So far I have 1 and 2 completed and tested. The changes correctly demux the raw stream. That is verified so onto steps 3-5. It's probably gonna take a few days.
DAE avatar
domy
Posts: 28
Joined: Fri Mar 20, 2020 10:50 am

Re: DGDemux development

Post by domy »

Oooohh, tell me more about the avisynth filter, Rocky; I've been looking for a way to show a waveform during playback.

My method of checking AV sync is very ad-hoc and janky, but here goes:

So, in order for us to check if some AV stream is in sync, we first need a reference - that is, an AV stream of the same scene that we know is in sync, which are the original blu-ray m2ts files. What I did is:
  • Pick a scene with sharp rises in volume; think gunshots, etc. Pixar films start and end with the classic lamp bounce scene, which also works very well. Ideally, enable an overlay in your player that shows you what video frame is being displayed. I use mpv, toggle the overlay with [Shift]+[O], and have this in my mpv.conf:

    Code: Select all

    osd-status-msg=${playback-time/full} / ${duration} (${percent-pos}%)\nframe: ${estimated-frame-number} / ${estimated-frame-count}
    
    This isn't necessary, but makes it easier to determine the exact position in the movie on a sub-frame level.
  • Open the Windows volume mixer (I'm not sure if macOS or Linux has anything similar) next to the video, and make sure the audio level of the player (not the system one, as it can lag behind a bit) is visible.
  • Now that we have our scene picked and player set up, play the reference version.
  • Record the screen somehow, either with your phone in high framerate mode (what I did) or with a screenrecorder. The screenrecorder needs to be able to record at the native display frame rate. Make sure both the volume level and video (with the overlay, if enabled) are visible.
  • Find the same scene in your target AV stream, which is probably your demuxed movie mkv. Repeat the recording process with that.
  • Compare the recordings. What you want to see is that the volume level reacts (i.e. rises sharply) on the exact same movie video position in both recordings. If it doesn't, you can roughly (see below) calculate the desync by counting how many recording video frames the Windows volume level is delayed by.
The accuracy of this is better than your eyes/ears, but still rather limited and depends on your display refresh rate. On a 60 Hz display, this'll give you an error of about +/- 8.3 ms - meaning, if the sync between reference and target is within ~17ms, you'll see it as perfectly in sync. You would presumably get a narrower error window on a 144 Hz screen (~7 ms), but I don't have such a screen so I can't confirm that (it's possible the Windows volume mixer only updates at 60 Hz regardless).

If there's a way to show a waveform during playback, that would be a much more accurate way of determining sync though :)
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Google for "avisynth show audio waveform filter". Looks like there are at least three of them. Please let us know which you like best.

domy, are you a coder/developer?
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

3. Buffer the previous 2 frames and write behind by 2 frames.
Step 3 completed and tested. Still outputting the correct THD raw stream. Should be all downhill from here.

But right now it's time for brekkies and watching the Magnus Carlsen invitational tournament. :P
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Well golly gee, Magnus lost his match with Ding Liren. :wow: His opening in the first game was just STOOPID. He still made the finals starting tomorrow but what the ... ?

Got steps 4 and 5 done and successfully demuxed MONSTERS with sync correction using DGDemux. Gotta review code and test thoroughly but things are looking good. Totally nontrivial coding for this. Buffering inside of buffering, and trying to make it fit the current design. They don't call me Rocky the Coding Squirrel for nothing.

Coming to you later today, boys and girls.
User avatar
Natasha
Posts: 150
Joined: Wed Nov 20, 2019 11:11 am

Re: DGDemux development

Post by Natasha »

Rocky wrote:
Thu Apr 30, 2020 11:34 am
Coming to you later today
If we're lucky it won't be one of those "after my lunch" deals.
User avatar
Bullwinkle
Posts: 338
Joined: Thu Sep 05, 2019 6:37 pm

Re: DGDemux development

Post by Bullwinkle »

You're funny, Natasha, but too predictable. Break the chains that bind you, and fly with the butterflies and zebras.

Awesome, Rock, can't wait!
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

All right, guys and gals, here is a test version:

* Fixed THD demuxing for UHD seamless branching such that the demuxed stream is in-sync throughout and is compatible with ffmpeg THD decoding.

http://rationalqm.us/misc/DGDemux_test.exe

You have to rename it to DGDemux.exe and put it with DGDemuxGUI.exe, as usual.

I have tested it with CARS_2, TOY_STORY, and MONSTERS_UNIVERSITY.
DAE avatar
jj666
Posts: 2
Joined: Tue Jan 21, 2020 6:15 pm

Re: DGDemux development

Post by jj666 »

Oops, the infamous Monsters University playlist 800 :-).

Cheers,

-jj-
DAE avatar
stRAf3r
Posts: 4
Joined: Mon Mar 30, 2020 1:15 pm

Re: DGDemux development

Post by stRAf3r »

got a quick question:

I'm myself running DGDemux on Windows Server 2019 Standard with an i9 9900K and it's working great! a friend of mine, who also bought a license, runs the same setup - all good.
he tried to install it on his secondary setup, which is same OS, but different CPU (Xeon E3-1270v2) which comes without iGPU - could this be the reason DGDemux not starting at all? there is no error message... so I'm not sure if maybe OS is missing something (he accesses the server by RDP).
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Could be the RDP is the problem. Can he try it with direct access?

Welcome to the forum!
DAE avatar
stRAf3r
Posts: 4
Joined: Mon Mar 30, 2020 1:15 pm

Re: DGDemux development

Post by stRAf3r »

thx - and thank you very much for your work on DGDemux! :bow:

could be RDP - but I'm also accessing server by RDP - but maybe remote access is still handled in a different way due to igpu - he can't direct access - but he should try with VNC or something similar too... I'll let you know.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Thank you, stRAf3r. I don't see how the iGPU could be relevant. DGDemux does not use the iGPU or video card GPU. I'll look for the WMI tester I made a while ago and give you that. It tests whether the machine satisfies the APIs required by DGDemux security.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

jj666 wrote:
Thu Apr 30, 2020 10:55 pm
4. The EAC3/Dolby Digital Plus track seems somehow incorrect being almost 4 seconds shorter (and totally out of sync throughout).
I just demuxed MONSTERS with the video and EAC3 tracks selected (nothing else). Everything was fine with correct duration and in-sync throughout. Perhaps there is an interaction when demuxing everything at once. I'll try that now. Meanwhile, perhaps you can try just demuxing the video and EAC3.

How are you determining that the audio is 4 seconds too short?

It's PID 1101. That's the one you are working with, yes?
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

stRAf3r, here is the WMI tester:

http://rationalqm.us/misc/MBSerialNumber.exe

Please run in a CMD window on the server (via RDP) and post the output here.

WGZ, can you test with this also? Thank you.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Rocky wrote:
Fri May 01, 2020 7:07 am
Perhaps there is an interaction when demuxing everything at once. I'll try that now.
Nope, no difference. Bottom line, I cannot reproduce the issue. It would be great if you could tell us exactly what you did.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Please re-download the MBSerialNumber tester. I had the wrong one up there initially.
DAE avatar
stRAf3r
Posts: 4
Joined: Mon Mar 30, 2020 1:15 pm

Re: DGDemux development

Post by stRAf3r »

he gets:
C:\Users\Administrator>C:\Users\Administrator\Desktop\DGDemux\MBSerialNumber.exe
Connected to ROOT\CIMV2 WMI namespace
returned 1, count 0
No objects found
Cleaning up...
Cleanup successfull

he couldn't add/create serial on that machine, since DGDemux never started...

he had a message before about missing MSVCR120.dll - but did a reinstall of the latest vcredist_x64.exe
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

That output is abnormal. I'll check the source code and tell you why shortly. Meanwhile...

Please talk about DGDemuxGUI not starting, otherwise you confuse me! I mean, please properly distinguish DGDemuxGUI and DGDemux.

Please try running just DGDemux without parameters:

DGDemux

Post the output.
User avatar
Rocky
Posts: 3557
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

That trace indicates a WMI failure. I'll try some googling to see why WMI may not be working on a server.

EDIT: There are a few good threads about this here. Search for "WMI".

EDIT2:

https://docs.microsoft.com/en-us/window ... leshooting
https://support.microsoft.com/en-us/hel ... server-201
https://community.spiceworks.com/topic/ ... mi-failure
Post Reply