HEVC with DV and cE + MTX-v81

[hubblec4]

Hi Rocky

I have a little problem and wanted to exchange a few thoughts, from developer to developer.

MKVToolNix(MTX) has made an update for HEVC streams in v81.
https://gitlab.com/mbunkus/mkvtoolnix/-/issues/3660
MTX is now able to write the HEVC and DV stream directly to an mkv file. MTX no longer offers the DV stream to the user, which means that this DV stream is also missing in cE.
This means I can no longer supply DGDemux with both stream PIDs.

To continue to use the DV streams in cE, but not for MTX, would require some effort.

Since it is no longer absolutely necessary to demux these two streams with DGDemux, I could simply prohibit the user from doing so. But I would still like to keep the demux function for HEVC + DV stream.

Would it be possible for DGDemux to only receive the PID for the HEVC stream and then search for the DV stream itself?
Both streams are demuxed and then treated with the further options by DGDemux -> merge process.

Please don't feel obligated to do anything, it's just a few thoughts and maybe you'll give me a different perspective on the whole thing.

Best regards and I hope you have a nice time with your family.

[Rocky]

Hi hubblec4. Thank you for your wishes and same to you and yours.

I don't understand how MKV is giving you any PIDs at all, as it is a different container entirely. Can you describe your workflow and then how the MKV change affects that?

[hubblec4]

The PIDs comes from the mpls file not from an mkv, I may have expressed myself incorrectly.

Here is cE's workflow:

cE depends on what information mkvmerge.exe provides, because this is also the data that is used for muxing.

To get information about a file use the "-J" switch

mkvmerge.exe -J path/to/00389.mpls

Output v80 -> mtx.txt
Output v81 -> mtx81.txt

In the output of MTX-v81 you can see that the entry for the PID 1015 (4117) is missing. This track was still displayed in v80 and later made available to the user in the GUI and in cE in the track list.

cE basically loads all mpls files on a Blu-ray and analyzes them itself. Then MTX is used to analyze to know what MTX recognizes and can process. And the output of DGDemux is also used for the mpls files.

From all three analyses, it is now decided what can be used.
First up is MTX, everything it has recognized is now compared with the output of DGDemux.
With v81, a DV stream (PID 1015) is no longer provided and cE no longer offers this track.

Makemkv was the first tool that could write the HEVC and DV streams directly into an mkv file without having to be muxed together beforehand. MTX now does the same and the DV stream is used internally automatically.

Although MTX v81 has changed something in the output of the mpls analysis, cE still works as intended when it comes to creating the mkv file.

However, cE can no longer initiate a demux process for the DV stream internally because, as I said, it is no longer in the list.

It is completely clear to me that DGDemux continues to do everything as it was programmed and has no errors.

But without the PID for the DV stream, DGDemux will not demux it, nor will it merge with the HEVC stream.

My knowledge of the Blu-ray structure is limited and I doubt that I quickly understand how I can always reliably determine the PID for the DV stream in order to continue to supply DGDemux with this PID internally.

[Rocky]

Still not clear to me. Why not just use DGDemux to get the PIDs you need? Why do you have to compare MTX and DGDemux etc.?

[hubblec4]

Why do you have to compare MTX and DGDemux etc.?

The first thing is, that my internal cE track list must be the same like MTX.
I can't(atm) add a stream with a unknown PID for MTX. Yes I could to that but this will fail for the creation of the mux.

If I were to use all the PIDs that DGDemux offers me, creating the mkv would no longer work because an "unknown" track should be used. The track is of course known to MTX, but no longer needs it as input.

And secondly, I use all file names from the DGDemux output for the display. I do this for all track editing tools so that the user then has the same track names as if they had used the tool directly.

Why not just use DGDemux to get the PIDs you need?

Getting to the PID(s) isn't necessarily the problem. I even think that my mpls parser also finds the PID for the DV stream.

But unfortunately I don't know how to get from the PID for the HEVC stream to the PID for the DV stream.

Since MTX no longer provides me with a PID for the DV stream, I would now have to check for myself whether there is still a DV stream available.

I don't think there is any indication in the DGDemux output that these two streams belong together, right?

In the DGDemuxGUI there is a note that both streams must be activated in order to start a merge process.

If I have compared all PIDs from MTX with DGDemux and notice that not all PIDs have been processed, what should I do now?

In this case the PID 1015 would remain.

[Rocky]

I just can't make any sense out of what you are saying. You say one thing and two sentences later contradict it (in my understanding). And things that simply are meaningless to me. E.g., "If I have compared all PIDs from MTX with DGDemux and notice that not all PIDs have been processed". Processed by who and for what? I'm sure it's my problem of understanding.

Let's try it this way: What is the simplest thing I could do in DGDemux to make you happy? You mentioned automatically finding and demuxing the DV EL stream. I could do that with one complication: In DGDemuxGUI you can turn on or off the demuxing of the DV EL. How would I control that in your scenario? E.g., should I change DGDemuxGUI so that the DV EL is always demuxed if the BL is enabled?

Technical note: DGDemux assumes 1) there is only one pair of main/DV streams, 2) a video stream with PID >= 0x1015 and <= 0x1b00 is the DolbyVision stream corresponding to the main video stream. You could simply look for a video stream in that PID range. This is a heuristic based on examining UHD disks. Theoretically, the PMT registration descriptor could be used to identify a stream as the DV EL.

[hubblec4]

I just can't make any sense out of what you are saying. You say one thing and two sentences later contradict it (in my understanding). And things that simply are meaningless to me. E.g., "If I have compared all PIDs from MTX with DGDemux and notice that not all PIDs have been processed". Processed by who and for what? I'm sure it's my problem of understanding.

Maybe it's just me because I'm not really good at explaining things.

MTX has changed it's input behavior and this is the reason why I don't have the PID for the DV stream. And I don't need to use the DV PID(stream) as input for MTX. Only the HEVC PID(stream) is now used as input by MTX.
This means I can not use all PIDs from my mpls-Parser or from DGDemux as input for MTX.
If I use the PID from the DV stream as input, MTX will fail while muxing the mkv.

But without the DV PID, DGDemux will not perform a correct demuxing and muxing the HEVC+DV stream into a single file.

Let's try it this way: What is the simplest thing I could do in DGDemux to make you happy? You mentioned automatically finding and demuxing the DV EL stream. I could do that with one complication: In DGDemuxGUI you can turn on or off the demuxing of the DV EL. How would I control that in your scenario? E.g., should I change DGDemuxGUI so that the DV EL is always demuxed if the BL is enabled?

Many thanks for this offer.
Yes indeed it would very nice if I only have to send the HEVC PID to DGDemux (CLI) and the DV stream is automatically used and demuxed. I guess when a user wants to demux the HEVC stream, also the DV stream is enabled for demuxing. (Maybe 0,01% don't want to keep the DV stream. But I can't find a reason why.)

MTX makes the same now, only the HEVC stream PID is used as input. The DV stream is now handled internally.

For cE I don't use the DGDemux GUI, only the CLI DGDemux tool is used.

Technical note: DGDemux assumes 1) there is only one pair of main/DV streams, 2) a video stream with PID >= 0x1015 and <= 0x1b00 is the DolbyVision stream corresponding to the main video stream. You could simply look for a video stream in that PID range. This is a heuristic based on examining UHD disks. Theoretically, the PMT registration descriptor could be used to identify a stream as the DV EL.

Very nice, that are great info for me, thank you.

1) what is when there are more streams which are related -> DGDemux will fail?
2) nice to know that there is a range of PIDs which could be used as DV streams

And "PMT registration descriptor" sounds very interesting: is this a table with info for the streams, I guess.

Many thanks Rocky that you teach me.

[Rocky]

Maybe it's just me because I'm not really good at explaining things.

Maybe both of us.

MTX has changed it's input behavior and this is the reason why I don't have the PID for the DV stream. And I don't need to use the DV PID(stream) as input for MTX. Only the HEVC PID(stream) is now used as input by MTX.
This means I can not use all PIDs from my mpls-Parser or from DGDemux as input for MTX.
If I use the PID from the DV stream as input, MTX will fail while muxing the mkv.

But without the DV PID, DGDemux will not perform a correct demuxing and muxing the HEVC+DV stream into a single file.

OK, starting to get clearer. Why can't you just use both PIDs for DGDemux but only the HEVC PID for MTX? You could just delete PIDs in the range 1015-1b00 for passing to MTX. Seems like the simplest solution to me, and then I wouldn't have to change anything.

Yes indeed it would very nice if I only have to send the HEVC PID to DGDemux (CLI) and the DV stream is automatically used and demuxed. I guess when a user wants to demux the HEVC stream, also the DV stream is enabled for demuxing. (Maybe 0,01% don't want to keep the DV stream. But I can't find a reason why.)

I'll start looking into that but I'm still interested in the answer to my question above.

1) what is when there are more streams which are related -> DGDemux will fail?

Well, for UHD disks there is always only the one BL and one EL, and DGDemux only accepts disks.

And "PMT registration descriptor" sounds very interesting: is this a table with info for the streams, I guess.

Have a look here. There is also the video descriptor.

https://professional.dolby.com/siteasse ... x-v1.2.pdf

Have a happy new year!

[hubblec4]

...
Why can't you just use both PIDs for DGDemux but only the HEVC PID for MTX? You could just delete PIDs in the range 1015-1b00 for passing to MTX. Seems like the simplest solution to me, and then I wouldn't have to change anything.

Yes, that was my first thought too, but I know that I don't really know much about the Blu-ray structure to be sure that I'm processing the correct PIDs. When it comes to things like this, I prefer to trust developers who are more familiar with it, like you or Mosu.

At the moment it is more likely that I would have to keep the PID(s) for the EL separately so that I can send both PIDs to DGDemux in cE. It is important for cE that the track list is exactly as MTX wants it to be. A lot of internal cE processes depend on this.
In MTX-v80 was the track list equal to DGDemux, eac3to, TSMuxeR. But MTX-v81 is now different to all other tools.

I also think that it is not difficult to keep the PIDs that DGDemux provides, but not MTX.
The problem is more in the correct assignment to the BL stream.

1) what is when there are more streams which are related -> DGDemux will fail?

Well, for UHD disks there is always only the one BL and one EL, and DGDemux only accepts disks.

And "PMT registration descriptor" sounds very interesting: is this a table with info for the streams, I guess.

Have a look here. There is also the video descriptor.

https://professional.dolby.com/siteasse ... x-v1.2.pdf

Many thanks for this pdf, many new info for me.

If I understood correctly, then it is very easy to assign the EL PID, since there is only one and only one BL.

If that is the case, then it should not be a problem to assign the corresponding EL PID from the DGDemux output to the BL.

Only the BL track would then be displayed in the track list, as MTX-v81 does.
And if the user wants to perform a demux, I could ensure that both PIDs are then sent to DGDemux.

I knew it was a good idea to ask you.

If I install it like this in cE, then nothing would have to be changed in DGDemux.
Unless you want to add this as a new feature in DGDemux.

[Rocky]

If I install it like this in cE, then nothing would have to be changed in DGDemux.
Unless you want to add this as a new feature in DGDemux.

Let's try it this way and keep open the option to modify DGDemux in the unlikely case that any issues arise. Please keep me informed.

[hubblec4]

Yes I will do it. Maybe I can start this night.

And many thanks for your help.

[Rocky]

You're welcome. It's a deal then. I'll mark this resolved for now, subject to re-opening later if needed.

[hubblec4]

I'll mark this resolved for now, ....

That's fine. I could make some progress and the extracting works already so far.
Now I have to change a lot, but I think everything will working.

[Rocky]

Sweet! It's challenging use cases like yours that keep us on our toes.

[hubblec4]

That's exactly how it is.