DGDemux development
Re: DGDemux development
DGIndexNV allows more than one running instance
Any plans to add that capability to DGDemux?
Any plans to add that capability to DGDemux?
Re: DGDemux development
Never thought about that. We'll see. Right now everything is focused on the speedup.
Re: DGDemux development
Thanks for considering it
It would as a sort of batch/queue system
It would as a sort of batch/queue system
Re: DGDemux development
Thought I'd give an update on the speedup project.
I have all the video (main, secondary, and MVC), AC3 audio, and LPCM audio converted and working. Should be able to knock out the rest of the formats and PGS subtitles pretty quick. Then some odds and ends like garbage stripping, preallocation, DELAY values, filler NALU stripping, episode demuxing. I think that's it. Still seeing 300-350% speedups.
I found one thing that had me all hot and bothered for a while. I found that everything is being demuxed with _write() instead of fwrite(). That is unbuffered and so is theoretically quite slower. I erroneously thought at first it was just these unbuffered _write()'s responsible for everything but testing refuted that. You see, in practice, for video, it's written by NALU and most of the NALUs are very big, so the penalty is small. For the audio the total demuxed sizes are fairly small compared to video so the penalty again is not great. The case of uncompressed audio is significant though. Some of those can get up to the 10GB level on some disks. So I converted everything to fwrite(). We'll get a speedup from that and, of course, from the new architecture which avoids all the nested buffering and parsing.
I have all the video (main, secondary, and MVC), AC3 audio, and LPCM audio converted and working. Should be able to knock out the rest of the formats and PGS subtitles pretty quick. Then some odds and ends like garbage stripping, preallocation, DELAY values, filler NALU stripping, episode demuxing. I think that's it. Still seeing 300-350% speedups.
I found one thing that had me all hot and bothered for a while. I found that everything is being demuxed with _write() instead of fwrite(). That is unbuffered and so is theoretically quite slower. I erroneously thought at first it was just these unbuffered _write()'s responsible for everything but testing refuted that. You see, in practice, for video, it's written by NALU and most of the NALUs are very big, so the penalty is small. For the audio the total demuxed sizes are fairly small compared to video so the penalty again is not great. The case of uncompressed audio is significant though. Some of those can get up to the 10GB level on some disks. So I converted everything to fwrite(). We'll get a speedup from that and, of course, from the new architecture which avoids all the nested buffering and parsing.
Sherman Peabody
Director of Linux Development
Director of Linux Development
Re: DGDemux development
Awesome, Sherman. Really looking forward to the final product.
_write() instead of fwrite()... Wondering how that happened. The original DVD2AVI and DGIndex both have fwrite()'s so what made me change that? Here is what I think happened. When I made DGSplit I found _write() faster (and DGSplit is blazingly fast) but in that use there are very large write sizes, so the lack of buffering was inconsequential. I must have wrongly got into my head that _write() is therefore all-around better. Another hang-my-head-in-shame moment. Cut me some slack. Way back then I was still learning to code competently. The upside of course is that it is easily fixable, as you have discovered. And as you point out, most of the gains are coming from the new architecture.
In case you are wondering, we can use _read() on the input side because reads are buffered internally. And of course DGSource() is not affected because it does not write any files.
_write() instead of fwrite()... Wondering how that happened. The original DVD2AVI and DGIndex both have fwrite()'s so what made me change that? Here is what I think happened. When I made DGSplit I found _write() faster (and DGSplit is blazingly fast) but in that use there are very large write sizes, so the lack of buffering was inconsequential. I must have wrongly got into my head that _write() is therefore all-around better. Another hang-my-head-in-shame moment. Cut me some slack. Way back then I was still learning to code competently. The upside of course is that it is easily fixable, as you have discovered. And as you point out, most of the gains are coming from the new architecture.
In case you are wondering, we can use _read() on the input side because reads are buffered internally. And of course DGSource() is not affected because it does not write any files.
Re: DGDemux development
Excuses, excuses. What a loser.
Re: DGDemux development
It can't be too bad, Natty. DG tools have made me a very well-off little squirrel. It's not just about acorns.
Re: DGDemux development
You're to young for that sort of thing ShermanI found one thing that had me all hot and bothered
Re: DGDemux development
Pretty sure he meant:
2. (idiomatic, colloquial, slang) aggravated or irritated
2. (idiomatic, colloquial, slang) aggravated or irritated
Re: DGDemux development
...there's no progress bar? Also, DGDemux seems to be creating .sup files that are invalid and can't be read by MKVToolNix/BDSup2Sub.
Re: DGDemux development
Please bear with me, this is my first post and it may sound like a very newbie question.
I've been using DGDemux for a couple of months now and I'm loving it. Thanks Rocky, fantastic software!
Now, on the updates/fixes log for slipstream 30, it's stated:
"Fixed file gaps processing for THD audio"
What does it mean in pratical terms?
For exampled, I've demuxed Toy Story 4 UHD that has a DTHD Atmos track with slipstream 29 and muxed it back with MKVToolnix to include the UHD video, saind DTHD Atmos track, plus the BD DTS-HD MA track.
Does it mean this THD track on my mux is incorrect somehow? Does it have any sync issues?
Thanks a lot!
I've been using DGDemux for a couple of months now and I'm loving it. Thanks Rocky, fantastic software!
Now, on the updates/fixes log for slipstream 30, it's stated:
"Fixed file gaps processing for THD audio"
What does it mean in pratical terms?
For exampled, I've demuxed Toy Story 4 UHD that has a DTHD Atmos track with slipstream 29 and muxed it back with MKVToolnix to include the UHD video, saind DTHD Atmos track, plus the BD DTS-HD MA track.
Does it mean this THD track on my mux is incorrect somehow? Does it have any sync issues?
Thanks a lot!
Re: DGDemux development
No, it is fine. That fix was for uncommon streams that did not start all major frames with payload_unit_start_indicator=1. TOY_STORY does not do that and it was one of the disks that I had tested with.BDgeek2 wrote: ↑Tue Jul 07, 2020 10:58 pmI've demuxed Toy Story 4 UHD that has a DTHD Atmos track with slipstream 29 and muxed it back with MKVToolnix to include the UHD video, saind DTHD Atmos track, plus the BD DTS-HD MA track.
Does it mean this THD track on my mux is incorrect somehow? Does it have any sync issues?
Re: DGDemux development
There is a marque progress bar and the current m2ts is shown.
Regarding your subs issue, please tell the disk and PID and specify your exact process to show the issue. Also state the error that you encounter.
Re: DGDemux development
I just did some spot checks and cannot find any problems. The files are accepted by SubtitleEdit and MKVToolNix without any issues. Please specify the disk, stream, and your process that creates your issue. You don't want Bullwinkle thinking you are FUD'ing us.
- Bullwinkle
- Posts: 341
- Joined: Thu Sep 05, 2019 6:37 pm
Drag & drop
Would you please add drag & drop support?
Re: DGDemux development
Drag and drop what onto what?
Re: DGDemux development
Hi Rocky, and other devs.
I really appreciate your hard work on DG-Tools
I wish to make a small feature request (not sure if I did it in the right place though...): would it be possible to cut the files via the chapterlist (or by an external chapterlist). For the case of audio discs it would be a neat feature. I know it can only be done GOP/audio frame-wise, but still it would be great to have approximate cuts according to a timelist.
Thanks in advance,
Keep up the good work,
Cheers,
I really appreciate your hard work on DG-Tools
I wish to make a small feature request (not sure if I did it in the right place though...): would it be possible to cut the files via the chapterlist (or by an external chapterlist). For the case of audio discs it would be a neat feature. I know it can only be done GOP/audio frame-wise, but still it would be great to have approximate cuts according to a timelist.
Thanks in advance,
Keep up the good work,
Cheers,
Re: DGDemux development
Thanks a lot for the prompt reply Rocky!
BTW, I just read of a tool that claims to correctly demux THD tracks and it specifically mentions DGDemux under it's FAQ.
I don't understand about the technicals and if it's accurate at all, but it occured me to bring it to your attention Rocky.
https://github.com/domyd/mlp
Here's the note:
"[1]: MakeMKV 1.15.1 does eventually maintain A/V sync, but it does so by deleting larger groups of frames on fewer occasions. This isn't perfect, but it's good enough. DGDemux deletes a minor frame at every segment boundary, which is better still, but not perfect"
Later down the same page, there's a Special Thanks note acknowledging your contribution Rocky.
"Special Thanks
Rocky over at the DGDemux forum for figuring out where and how to cut TrueHD bitstreams without causing audio issues or decoder errors."
Could you please shed some light on this issue Rocky?
I mean, with the sole intention of understanding it and not meaning any disrespect, does DGDemux not take the best approach to THD demux?
Thanks again, I really appreciate all the work under DGDemux!
Re: DGDemux development
Hi CYPHER. What do you mean by an "audio disk". Is there a use case beyond that as well?CYPHER wrote: ↑Mon Jul 13, 2020 5:50 amI wish to make a small feature request (not sure if I did it in the right place though...): would it be possible to cut the files via the chapterlist (or by an external chapterlist). For the case of audio discs it would be a neat feature. I know it can only be done GOP/audio frame-wise, but still it would be great to have approximate cuts according to a timelist
Re: DGDemux development
Well, "best" depends on your criteria. domy is a respected Moose-Approved member here and we have had extensive discussions about all this, which you will see if you search a bit. Our solutions are different but the end results are within a few milliseconds of each other. domy seeks what he calls perfection, while I seek a balance between good sync and good performance with low complexity. Can my way be improved? It's possible. But do I want to make a complex mess that is bug prone to gain a few milliseconds in precision? domy has one thing to do while my demuxer is doing so much more. So I wouldn't get anxious over a few milliseconds when the average person can't even detect a 50-100 millisecond desync.
Re: DGDemux development
Thanks a lot Rocky, I really appreciated your answer!
I'll look for the debates you mentioned.
I'll look for the debates you mentioned.
Re: DGDemux development
Discussions, not debates!
Re: DGDemux development
By audio disc I mean live concert recordings, like this one: https://www.amazon.com/Hans-Zimmer-Live ... B0757G5PXWRocky wrote: ↑Tue Jul 14, 2020 7:13 pmHi CYPHER. What do you mean by an "audio disk". Is there a use case beyond that as well?CYPHER wrote: ↑Mon Jul 13, 2020 5:50 amI wish to make a small feature request (not sure if I did it in the right place though...): would it be possible to cut the files via the chapterlist (or by an external chapterlist). For the case of audio discs it would be a neat feature. I know it can only be done GOP/audio frame-wise, but still it would be great to have approximate cuts according to a timelist
Here each chapter is a different song/audio track, but they share the same playlist on the BD and it would be nice to extract the songs into different files. Furthermore, the bonus disc of e.g. Inception contains the OST of the film in the same manner.
Apart from these, for films/movies this feature is might not be so useful...
Re: DGDemux development
What is "OST"?