Feature Requests

[admin]

It's certainly possible but what do you mean by this:

"Without the normal in loop deblocking being done..."

[RedDwarf]

It's certainly possible but what do you mean by this:

"Without the normal in loop deblocking being done..."

It was what I remember H264 calling the de-blocking, but I might of got it wrong.

Basically when a video is played in a Media Player, de-blocking gets done according to the amount the AVC video says should be done. As far as I am aware DGDecNV doesn't do any, or at least from what I have seen it doesn't appear to do any. If the AVC video expects de-blocking to be done when the video is displayed, to me it doesn't seem right to not do it. In effect, the returned video isn't as it was meant to be.

For some video that might not be a problem. But for others such as the fairly low bitrate video used for HDTV in the UK, which is typically a 10 Mbit transport stream with 1440x1088 resolution giving just over 9Mbit for video. It means quite noticeable blocks where any movement occurs when de-blocking isn't done. The blocks aren't practically unnoticeable when normally de-blocked during playback in a media player.

It would be nice to have the video which is returned to be as it was meant to be displayed, rather than having to use separate de-blocking filters in addition.

[Didée]

The deblocking is performed. As you noted (with slight misspelling), H.264 is doing "in-loop deblocking". That means that deblocking is not done "after the fact" like you do when e.g. playing back Xvid videos. In H.264, the deblocking is an essential "inner" process of both encoding and decoding. If deblocking would not be done (it is possible to switch off), then you'd get an all-corrupted output.

So you don't need an option to switch it on. Be assured, it already IS on. - (Perhaps you could have an option to switch it *off*, but you don't really want that, anyway.)

However, with TV transmissions it is quite possible that they used too weak inloop deblocking from the start, or even had it switched off from the start. (In case of doubt, you can expect all kind of silly-ness from TV stations.) In such a case, the video stream can exhibit blocking problems indeed, and further postprocessing might be necessary. (For such sources, the Deblock() filter as found e.g. in dgdecode.dll, is appropriate. It uses the basic deblocking mechanisms of H.264 deblocking.)

[admin]

I concur with everything Didée said regarding DGDecNV deblocking. (I'd use the built-in deblocking (cpu option) of DGDecode in preference to Deblock(), but if that is not strong enough a dose of Deblock() can be added. That's neither here nor there for the subject of this thread, however.)

RedDwarf, if you have any evidence that DGDecNV is not performing deblocking correctly, please present it.

[Didée]

Still, shouldn't it be considered to add some sort of deblocking to DGDecodeNV ? In particular for Mpeg-2 sources. Fact is, the white bearded man DGDecode can do quantiser-adaptive deblocking on mpeg-1/2, but the young guy DGDecodeNV can do not. One could argue this is a regression.

[admin]

Sure, what you say is correct. The problem is that CUVID does not give me any access to the quants. But I'm not so sure the quants bring that much to the table anyway. I've not seen any evidence for it. It would be nice to have a GPU deblocker and that's what I will probably tackle first.

[RedDwarf]

The deblocking is performed. As you noted (with slight misspelling), H.264 is doing "in-loop deblocking". That means that deblocking is not done "after the fact" like you do when e.g. playing back Xvid videos. In H.264, the deblocking is an essential "inner" process of both encoding and decoding. If deblocking would not be done (it is possible to switch off), then you'd get an all-corrupted output.

It can be turned off in ffdshow Directshow decoder without any corruption, the picture looks sharper but slightly blockier. De-blocking can be de-activated in x264 encoding settings as well. So I don't see what you mean about your reference to deblocking being essential to avoid corruption.

So you don't need an option to switch it on. Be assured, it already IS on. - (Perhaps you could have an option to switch it *off*, but you don't really want that, anyway.)

However, with TV transmissions it is quite possible that they used too weak inloop deblocking from the start, or even had it switched off from the start. (In case of doubt, you can expect all kind of silly-ness from TV stations.) In such a case, the video stream can exhibit blocking problems indeed, and further postprocessing might be necessary. (For such sources, the Deblock() filter as found e.g. in dgdecode.dll, is appropriate. It uses the basic deblocking mechanisms of H.264 deblocking.)

That surprises me because the video from DGDecodeNV is considerably more blocky than I see when opening the video via DSS in AvsPMod. That is why I doubted that it was performing any de-blocking.
It's very noticeable where there is fast motion but not so noticeable when decoded via Directshow.
If I open a DGDecodeNV video in AvsPMod and the same video with Directshowsource, the DGDecodeNV video will look very blocky in places whereas the same frame does not have very noticeable blocks in the DS decoded video. That's using ffdshow AVC DS decoder.

........RedDwarf, if you have any evidence that DGDecNV is not performing deblocking correctly, please present it.

I will have to check through some videos when I have more time, stepping through frames to find something because it mainly appears where there is fast movement which can be missed when playing back at full speed. It was very noticeable in some places but not in others and it's finding it again. It's like some de-interlace line artefacts that appeared with the nVidia de-interlacing. They were noticeable in a few places in a video I looked at but I couldn't find any in others.

[admin]

That surprises me because the video from DGDecodeNV is considerably more blocky than I see when opening the video via DSS in AvsPMod.

That is ridiculous. DGDecNV is a bit perfect decoder just as all AVC decoders are that adhere to specifications. Again, please show your evidence and stop creating silly FUD.

I will have to check through some videos when I have more time

But you have enough time to spread FUD!

It's like some de-interlace line artefacts that appeared with the nVidia de-interlacing. They were noticeable in a few places in a video I looked at but I couldn't find any in others.

Which of course has nothing at all to do with deblocking.

If you're saying that one can get artifacts sometimes with a deinterlacer, well thank you, Captain Obvious.

[Didée]

I started writing a somewhat longish explanation about the what's and how's and why's of how AVC and deblocking is working ... but I lost mood. (But saved it ... if you insist by all means, I can show it)

Short report from the practical side: I can't reproduce.

Please show a source segment where DGDecodeNV gives blocky picture where other decoders don't.

[laserfan]

I'm one of those anal-retentive guys that keeps all of my logs from my encodings. I have noticed that the log created by DGDecNV does not include a version number, nor does the .dgi file i.e. I don't THINK that the NV file number is exclusive to a release e.g. 2036=12.

Is there a way to automatically note the version? Or could it be saved with either the .dgi file or the .log file?

[admin]

Is there a way to automatically note the version? Or could it be saved with either the .dgi file or the .log file?

I'll add it to the DGI file for the next release. Thanks for the suggestion.

[Guest 2]

More than a program feature request, this is a changelog feature request.

Please, please, please tell us when DGI version changes. Sometimes I upgrade versions and have to recreate DGI.

[admin]

I missed the notification of DGI file version NV12 for build 2034. I have added it to the change log in build 2036.

[laserfan]

Is there a way to automatically note the version? Or could it be saved with either the .dgi file or the .log file?

I'll add it to the DGI file for the next release. Thanks for the suggestion.

Thanks for indulging me--I oughta point-out that I haven't had any problems with "beta" versions of DGDecNV (requiring fallback to an earlier rev) since way before the tools' consolidation to DGDecNV! Seems like a very long time ago. But like I said, I'm anal (and a natural worrier) and if I do find any problems with encodings I make I like to keep all the records of what I'd done; "lessee my encoding was made July 10 so I musta used dgdecnv2019 which I'd downloaded on the 6th". Thanks again.

[Karyudo]

Sometimes I upgrade versions and have to recreate DGI.

Maybe it's not quite kosher, but instead of re-indexing, I often (OK, always) [REDACTED].

Seems to work fine...

[admin]

That will always cause *something* to not operate correctly, because I do not change the level when nothing is affected. Whether you notice the thing that is no longer correct depends on the particular change. For example, you may not notice that hints are wrong if you do not use them.

That is really bad advice you are giving and will cause support nightmares for me if generally adopted.

[Karyudo]

Best way to get good advice on the Internet? State loudly and unequivocally your own wrong way of doing it...

I was hoping you'd mention what the risks are -- increased support nightmares is one I'm very sorry I didn't think of. Doing what I suggested worked for the streams I have, but it's clear this is not what one is supposed to do, of course.

(If it's any consolation, if it didn't work I'd immediately re-index, rather than ask for support.)

[admin]

It's OK, I'm going to add a CRC check to the DGI file to thwart you.

[Karyudo]

For a given "non-broken" video stream, will there usually be a material difference between the index files produced by two different versions of DGIndexNV? I would assume not, but I don't *know*.

One of the reasons I avoid re-indexing is because I fear (perhaps irrationally) that any tricky Avisynth editing I've worked out will be borked in some very subtle way by the new version, and I'll have to spend time finding and fixing it. I guess I'm sort of a "better the devil you know..."-type person.

Would there be any way to "freshen" old DGI files with an updated version, faster than re-indexing from scratch? I'm guessing not, but one can always hope...

[admin]

The product is mature enough that at this point the likliehood of a reindexing with a higher version changing the delivered frame locations, trims, etc., is close to zero, and if it did, I would mention it prominently in the change log.

[Karyudo]

One might argue, then, that making a quick fix to the DGI (soon to be thwarted by a CRC check...) should also not introduce errors in delivered frame locations, trims, etc., between versions, and thus should perhaps be unofficially allowed (but not "supported") for those who know what to tweak where??

I'm thinking in particular of my most recent project, where I came across the MKV with compressed headers, and you created a new version to parse such streams. In the interim, I'd remuxed with uncompressed headers and indexed the result, and was editing from that DGI. When the new version came out, I installed it to be current -- but I now had a perfectly-good DGI from the previous version (made just the day before!) that ostensibly I couldn't use. One tiny fix later, and I was fine without having to reindex. But a CRC check would have added some level of annoyance...

I don't suppose you could check the CRC, and pop up a warning along the lines of:

Code: Select all

CRC Check Fail: This DGI was made with a previous version of DGIndexNV, and may not work correctly with the current version. It is recommended that the project be re-indexed with the current version. Proceeding without reindexing is done at the user's risk, and problems arising from ignoring this CRC check will not be supported.

Ignore CRC Check | Cancel and Re-index

But if we have to re-index, we have to re-index. I'm not the one that has to support/fix problems that users may bring upon themselves by not using your product the way it was meant to be used. I really can't get too bent out of shape by a very slight annoyance in an otherwise brilliant and bulletproof piece of software, and you'll hear no further whining/suggesting from me on this topic!

[admin]

One might argue, then, that ...

No, you can't argue that. I told you already that a file format up rev is guaranteed to mean that something will be wrong if you do not re-index to make a properly formatted DGI file that is compatible with the new software.

I'm thinking in particular of my most recent project, where I came across the MKV with compressed headers, and you created a new version to parse such streams. In the interim, I'd remuxed with uncompressed headers and indexed the result, and was editing from that DGI. When the new version came out, I installed it to be current -- but I now had a perfectly-good DGI from the previous version (made just the day before!) that ostensibly I couldn't use.

Who says you couldn't use it? The DGI file version did not go up from 2035 to 2036, and 2036 would not reject it.

One tiny fix later, and I was fine without having to reindex.

There was nothing to fix. Please tell us what your "tiny fix" was. Ha ha, busted!

And BTW, the CRC thing was a joke. There are some valid reasons to edit a DGI file, such as overriding the frame rate (although that can be done in your script too).

Can you imagine how messed up things would be if we didn't have a file format version number, which was designed to save you grief?

Now if you proposed that I add functionality to automatically generate a corrected new format DGI from existing ones, which is something I have thought about, then you might have a case. But that would be pretty low on my priority list, given that only one person has ever found this to be an irritant. It also would not always be possible to create a correct new file without re-scanning the source file(s).

[laserfan]

I don't get why anyone would bother to mess with old dgi files given how wicked-fast the indexing process has become?

[Karyudo]

I've found that indexing some MPEG-2 files is pretty slow, but I'd agree that indexing h264 stuff is quick.

admin, now that I think about it, it's quite likely I started the day with a version older than 2035, indexed a couple of files, then found that it wouldn't index an MKV with compressed headers, upgraded to 2035, and found that 2035 also wouldn't index MKVs with compressed headers. Then when 2036 was introduced and I tried to use a DGI I'd indexed just a day or two earlier, it didn't work. But clearly I'm still in the wrong, because I should have stayed up to date and used 2035 in the first place!

My new rule (eerily similar to your long-standing rule): If old DGI does not work with current version, re-index. No ifs, ands, or buts!

[kypec]

I don't get why anyone would bother to mess with old dgi files given how wicked-fast the indexing process has become?

I've found that indexing some MPEG-2 files is pretty slow, but I'd agree that indexing h264 stuff is quick.

I'm not sure what am I doing wrong but indexing MKV files (untouched stream ripped directly off bluray disc with MakeMKV, VC-1 or H.264) using DGIndex revision 2035 takes more than 40 minutes
for standard 2-hour movie.
MKV is located on external HDD (connected over eSATA), .dgi index is being written on local HDD.
My system: i7-920, 6GB RAM, Windows 7 x64, GeForce 8400GS 512MB
Any ideas?