DGDemux speedup

Post Reply
User avatar
Sherman
Moose Approved
Posts: 139
Joined: Mon Jan 06, 2020 10:19 pm

DGDemux speedup

Post by Sherman »

Back on the 350% speedup topic...

My researches have revealed that we do not need to revise the nested parsing architecture. It is not that aspect that causes the slow performance compared to "my method". What is really going on is that the current implementation is using _write() for output to the demux files. _write() is low-level IO that is unbuffered! And when demuxing from transport packets we have lots of very small writes. The code I did for demuxing the MVC stream, and what Rocky experimented with for AC3 audio, uses fwrite() which is buffered by the OS.

So, bottom line is that we do not have to redesign anything. We simply need to replace the IO with fopen/fwrite/fclose throughout. I'm going to do that and report back on results.
User avatar
Rocky
Moose Approved
Posts: 1221
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Awesome, Sherman. Really looking forward to your results. Sounds like we may get a huge performance boost "on the cheap", that is, without having to make radical changes to the design. This will apply to DGIndexNV as well.

Wondering how this happened. :scratch: The original DVD2AVI and DGIndex both have fwrite()'s so what made me change that for my own code? Here is what I think happened. When I made DGSplit I found _write() faster (and DGSplit is blazingly fast) but in that use there are very large write sizes, so the lack of buffering was inconsequential. I must have wrongly got into my head that _write() is therefore all-around better. Another hang-my-head-in-shame moment. :facepalm: Cut me some slack, way back then I was just learning to code.

The upside of course is that it is easily fixable. :lol:

In case you are wondering, I can use _read() on the input side because I do my own internal buffering. And of course DGSource() is not affected because it does not write any files.
Post Reply