[RESOLVED] DGIndexNV is dropping frames at the end

[stefan]

Hi,
I found out DGIndexNV is dropping frames: I recorded several short clips with a Canon EOS 550d (AVC Baseline profile, no B-Frames) and tried to open them with DGDecodeNV/AviSynth (demuxed to raw h264 before indexing), but found out it refuses to open more than 2 files in a script at the same time.
So I indexed all files together: The whole is less than the sum of it´s parts! With a total of 127 files, there are 9 frames missing, always the last frame of some files. The files are not corrupted, the last frame is perfectly viewable when file is indexed alone. In all DGI files is CODED==PLAYBACK, no warnings/errors in DGIndexNV.
Is this a bug or am I doing something wrong?
Regards,
Stefan

DGDecNV: Version 2038
Graphics: Nvidia Quadro 600

[admin]

Hi,
I found out DGIndexNV is dropping frames: I recorded several short clips with a Canon EOS 550d (AVC Baseline profile, no B-Frames) and tried to open them with DGDecodeNV/AviSynth (demuxed to raw h264 before indexing), but found out it refuses to open more than 2 files in a script at the same time.

The number of simultaneous DGSource() instantiations is limited by the amount of memory on your graphics card.

So I indexed all files together: The whole is less than the sum of it´s parts! With a total of 127 files, there are 9 frames missing, always the last frame of some files. The files are not corrupted, the last frame is perfectly viewable when file is indexed alone. In all DGI files is CODED==PLAYBACK, no warnings/errors in DGIndexNV.
Is this a bug or am I doing something wrong?

It's certainly unexpected. Can you give me a pair of files that I can use to see your problem? You can upload them to mediafire.com and post the links here. Or FTP, etc.

[stefan]

While looking for small files with the problem, I think I noticed the problem: The last frame will be dropped when it is an I-Frame!
I'm not sure, I just cut a huge raw h264 file with the problem to the second last IDR (it´s really simple with the binary offset from the DGI) and put the header in front of it. The file has 13 Frames, everything fine, if I load it two times, the video will have 25 Frames, not 26!
I do not have the permission to public release the file, so I send it to you via mail (it´s just 2.7MB).
Thanks for fast reply,
Stefan

[admin]

Thank you for the file via email. I have duplicated the issue and am investigating.

[admin]

When you cut the file so that there is just an IDR frame at the end, you will create an illegal H.264 stream if you append a file that starts with another IDR. The H.264 spec requires that two IDRs in a row must have different values of idr_pic_id. That might normally be done by the encoder if it creates an IDR-only stream. But if you manually do cutting that results in two consecutive IDRs with the same value of idr_pic_id, then parsing per spec will miss that picture boundary between the IDRs and will see only one IDR.

I could revise my indexing to pick up the picture boundary in a non-standard way but the problem is that I cannot so modify the CUVID parser, and things will not work right unless my parsing and the CUVID parsing agree.

So I'm afraid that you have shot yourself in the foot by cutting/editing such that you create illegal streams. Your options are to not cut after IDRs like that, or combine the files yourself and write a post-processor that adjusts the idr_pic_ids to be increasing.

This is yet another wart complicating compressed domain editing of H.264 video.

[stefan]

Thank you for analyzing the problem.

idr_pic_id identifies an IDR picture. The values of idr_pic_id in all the slices of an IDR picture shall remain unchanged.
When two consecutive access units in decoding order are both IDR access units, the value of idr_pic_id in the slices of
the first such IDR access unit shall differ from the idr_pic_id in the second such IDR access unit. The value of
idr_pic_id shall be in the range of 0 to 65535, inclusive.

Do I understand it correctly that the entire problem is that my camera always uses the same idr_pic_id and it is illegal to have two idr frames directly after each other with the same id (I cannot influence if it ends with idr, thats how the files come from my camera)? So the solution would be to change the first or last idr_pic_id of all files, right? The last would be easier, because it is not referenced, right, so basically just change one number (or two numbers, because frame number in slice header need to be changed too)?

Would it be possible for you to implement this feature, like "increasing idr_pic_id by one of last idr frame if it is the very last frame of the file"?

[admin]

Yes, you understand it correctly.

I'll look into implementing some kind of workaround. It's not easy to do what you say because the field is variable length coded, so if I change it from 0 I have to insert data in the bitstream; that's not a minor thing given my current architecture. I have to alter the bitstream in order for CUVID to do the right thing.

How are you getting clips that end with an IDR like that? You just stop the camera and the stream may end like that? Or are you editing with some application?

[stefan]

I'll look into implementing some kind of workaround. It's not easy to do what you say because the field is variable length coded, so if I change it from 0 I have to insert data in the bitstream; that's not a minor thing given my current architecture. I have to alter the bitstream in order for CUVID to do the right thing.

Variable length coded... that means I cannot just change a bit somewhere with hex-editor, I would need to parse the hole file (which would mean read&understand the hole H.264 spec, takes too much time...)

In the very most cases the last frame is not needed anyway, but it would be nice to know that it was dropped. Maybe you could just add a warning message when this problem occurs that the last frame of the files: a,b,c... had to be dropped, that shouldn't be too difficult?

How are you getting clips that end with an IDR like that? You just stop the camera and the stream may end like that? Or are you editing with some application?

Thats how the files come from the camera (in fact they are in mov container with pcm audio, I demuxed them with mp4box). The camera seems to work with a simple fixed GOP scheme of 12 frames, one I frame and the others are P-frames. When I press stop the chances are 1/12 that the last frame is an I-Frame. In fact that matches quite good with my initial problem: 127 files, 9 frames dropped.

[admin]

Variable length coding means you may have to expand or shrink the field's number of bits to change its value. There's a lot to love about AVC video, but also a lot to hate.

One possibility is to add the non-standard picture boundary detection to both my parsing and CUVID. I can ask Nvidia what they think about making such a change to CUVID.

[admin]

Canon should a) not stop a clip with an IDR, or b) increment idr_pic_id.

That is the correct resolution.

[stefan]

Canon should a) not stop a clip with an IDR, or b) increment idr_pic_id.

Canon most likely won´t do anything about that. Maybe the magic lantern guys can do something (they´re creating inofficial firmware extensions for enhanced functionality).
Because that are all different files, not one continuous stream separated to several files, the camera also doesn't do anything non-standard compatible.

The H.264 spec says that not two "access units" after each other should have the same idr_pic_id. Would it be possible to insert a useless dummy-"access unit" not containing any frame, just to separate the two files?
Or just warn if the last frame has to be dropped. I don´t know how you implemented the file loading process, but I would internally note where the last frame/access-unit if it is an I-frame starts and store it´s idr_pic_id. When next file is loaded and same idr_pic_id => drop last frame and warning, otherwise just go on.
I think in both cases nothing in CUVID has to be changed, only in parsing the h264 file, but since I don´t know your code I´m maybe completely wrong.

[admin]

In the very most cases the last frame is not needed anyway, but it would be nice to know that it was dropped. Maybe you could just add a warning message when this problem occurs that the last frame of the files: a,b,c... had to be dropped, that shouldn't be too difficult?

I have added a warning as you suggested. I looked at some possibilities for modifying the bitstream but it became a nightmare. What you will do is load all the streams and do save project. You will get an error popup at each instance of the problem that identifies the file name. You can then use a hex editor to delete the ending IDR frame from the file(s). Then you will have to reload the files and re-index. If you need help in using a hex editor to nuke the IDRs send me a PM. I hope to release a new version tonight.

I'll mark this resolved as I don't intend to do anything more than this error message.

[stefan]

Thanks!
But for what reason do I have to cut off the IDR frames in a hex editor? They are just dropped in the current version, so there will be no difference except the warning?

[admin]

If it works, then fine, but as that is not designed behavior I can't vouch for it.

[admin]

See here for latest developments regarding this issue:

http://neuron2.net/board/viewtopic.php?f=8&t=155

The bottom line is that the problem of this thread has a fix and will be implemented in the next release.