Getting weird artifacting at the same place when encoding

[squallmx]

I found a movie with the same problem, when the AVS script is loaded on VDub the image is perfect, but the encoded file is corrupted, happens in the same place regardless of the x264 settings.

Original stream on VDub:

Sample: http://www.mediafire.com/?6ph77azc5xxxd7z

1080p Re-encode stream on VDub (Blu-ray compilant):

Code: Select all

cabac=1 / ref=2 / deblock=1:-3:-2 / analyse=0x3:0x113 / me=hex / subme=6 / psy=1 / fade_compensate=1.00 / psy_rd=1.00:0.10 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-3 / threads=12 / sliced_threads=0 / slices=4 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=1 / constrained_intra=0 / fgo=0 / bframes=2 / b_pyramid=0 / b_adapt=1 / b_bias=0 / direct=3 / weightb=1 / open_gop=1 / weightp=1 / keyint=24 / keyint_min=2 / scenecut=40 / intra_refresh=0 / rc_lookahead=24 / rc=2pass / mbtree=1 / bitrate=18000 / ratetol=1.0 / qcomp=0.60 / qpmin=5 / qpmax=40 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=30000 / vbv_bufsize=30000 / nal_hrd=vbr / ip_ratio=1.10 / aq=1:1.00

720p Re-encode stream on VDub (DXVA compilant):

Code: Select all

cabac=1 / ref=4 / deblock=1:-3:-2 / analyse=0x3:0x113 / me=umh / subme=7 / psy=1 / fade_compensate=1.80 / psy_rd=0.90:0.09 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-3 / threads=12 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=1 / constrained_intra=0 / fgo=0 / bframes=3 / b_pyramid=1 / b_adapt=1 / b_bias=0 / direct=3 / weightb=1 / open_gop=1 / weightp=1 / keyint=48 / keyint_min=4 / scenecut=40 / intra_refresh=0 / rc_lookahead=48 / rc=2pass / mbtree=1 / bitrate=5000 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=44 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=15000 / vbv_bufsize=15000 / nal_hrd=vbr / ip_ratio=1.10 / aq=1:0.90

Sample: http://www.mediafire.com/?z5ox83xzs6urpps

Simple AVISynth script with only DGIndexNV and the resize filter (if needed)

If I only encode a small sample around the affected frames everything is Ok.

PD: I posted this on Doom9 too.

[admin]

I have to give the same answer again. It's a mystery. We have two or three people seeing this out of many thousands. I bought a bluray to try to duplicate it and could not. I just don't know what to do.

[Nick007]

It happened to me too. I tested it many days and figured out that it's really mind-boggling. I happens in the same exact place, looking exactly same and even after resizing 1080p to 720p, so that would indicate decoder's problem. But I found out if I won't use --colormatrix bt709 that it won't happen (with my usual settings --preset veryslow and stuff). But it did happen with/without --colormatrix on faster settings, I didn't understand why...
It didn't occur if I trimmed the source, it had to be from frame 0 to frame after where the glitch occurs. I could replicate it every time on my pc with GTX 570, but I couldn't replicate it on my laptop with GT 220M no matter what I tried. The most curious thing is that it did it in two sources so far and only when I used --colormatrix bt709 (which I had started using few months before the glitches). When I dropped --colormatrix, it haven't glitch so far, so I gave up on solving this mystery. Changing of drivers or x264 version (even 32-bit/64-bit) didn't help. It now works, so I'm happy. Just my 2 cents so that you know there are more people who witnessed this problem.

[admin]

Thanks, Nick007, for your valuable report. I'm looking out for the common factor between the few reports we have, so your detailed report is greatly appreciated.

[sandy_geser]

this sounds like what I'm having ( detailed in http://neuron2.net/board/viewtopic.php?f=8&t=174 ). I never used colormatrix bt709 though, at least not that I know of

I have to give the same answer again. It's a mystery. We have two or three people seeing this out of many thousands. I bought a bluray to try to duplicate it and could not. I just don't know what to do.

it's possible the problem is more common than you think, not everyone who makes an x264 transcode watch it right away, some people also just ignore tiny imperfections like that, and some just don't post on forums. I hope some breakthrough will happen eventually.

[NDM]

Yep.....
You can add me to the list also, at least I think.

Seems to be only certain Blu Rays (say, one in twenty-five) as far as I can fathom. It's happened to me at least twice that I know of, both fairly run of the mill AVC Blu Rays seemingly.

I've also never specified a 'colormatrix', unless it's changed by a filter's default settings, but I tend to use different filter combinations. The first BR that it happened with, I tried over and over again, eliminating filters etc. Corruption/artefacts of the above nature (OP's post), not exactly the same with each attempt, but within like a few seconds either way and very similar glitches each time.
I did try isolating the worst section (about a minute I think) and just encoding that, and that was even more variable - sometimes artefacts, sometimes not.

Had to switch back to using FFmpegSource, because I just don't know when or where this issue will strike again.
Like ^ Sandy ^ says... You won't always have watched your encodes all of the way through, In fact I rarely have.
The two I spotted were probably blind luck.
Particularly annoying that I switched to DGDecNV to avoid this very issue.

(Using PC / WinXP / MeGUI / AVIsynth 2.5.8 )

[sandy_geser]

Yep.....
Corruption/artefacts of the above nature (OP's post), not exactly the same with each attempt, but within like a few seconds either way and very similar glitches each time.
I did try isolating the worst section (about a minute I think) and just encoding that, and that was even more variable - sometimes artefacts, sometimes not.

interesting you had them in random. both op and myself had them repeating in the exact same spots, exact same glitching and could not recreate them using trimmed clips.

[NDM]

Just been fiddling about with another BR and it's happened again:

Code: Select all

http://www.mediafire.com/i/?bme9lqsrflmf96t

Yeah, there's definitely some variability. Out of six test runs with the above, it happened twice. Both times within frames of each other and a very similar corrupt rectangular strip.
Obviously, there are a lot of variables in video encoding, so maybe it's not even the exact same issue, although it is only happening with DGDecNV, and then only with selected Blu Ray streams.
It's the lack of certainty that stopped me from reporting the issue, 'til now.

(Video card is a GT520 btw.)

EDIT:
Just been looking more closely at squallmx's pictures in the original post.
What do you know....
Both of the corrupt frame screenshots have the same kind of segmentation/rectangular strips spanning the screen width.

[sandy_geser]

without pretending to be an expert (which I'm not) it seems to be something that happens on 4xx and 5xx nvidia series. op mentioned he couldn't replicate it on on a 2xx series card but has it on a 4xx. NDM has it on 5xx and I had it on 5xx and 2 different 4xx cards but not on a 2xx card. I know admin tried one of the sources I had issues with but I'm unsure which card he used. Might be a weak assumption, dunno.

[laserfan]

FWIW I experienced what was probably a one-frame glitch on one of my BD conversions on viewing recently. An hour & a half movie. I've not made any effort to t-shoot the glitch, except to say here that:

1. Keeping in mind it was (likely) a one-frame glitch out of 24/sec, it looked to me to be blockiness in the bottom third of the image, whereas the rest of the image was normal. I'll try to find it again (dunno if I can locate the scene) and report-back if I find anything different.

2. In my encoding process, the only thing "new" or different is latest x264 r2106; I never fiddle with settings other than bitrate in my 2-pass conversions.

3. I recently switched from an MSI N9600GT videocard to the N520GT (and newer driver 280.26 as well)

I don't do a lot of these, but this recent glitch was the ONLY one I've ever seen using DGNV tools, such that this one glitch and this thread is enough to send me back to my trusty 9600GT again.

In the next weeks or months I plan to build another PC with an i7 processor--when I do that I will utilize the 520GT once again and will look forward to helping t-shoot at that time (unless all y'all get it found & fixed before then of course).

[NDM]

Well, I tried encoding the OP's sample, and combining multiple copies to extend the length of it. Cropped or uncropped, I couldn't squeeze a single artefact out of it!

without pretending to be an expert (which I'm not) it seems to be something that happens on 4xx and 5xx nvidia series. op mentioned he couldn't replicate it on on a 2xx series card but has it on a 4xx. NDM has it on 5xx and I had it on 5xx and 2 different 4xx cards but not on a 2xx card. I know admin tried one of the sources I had issues with but I'm unsure which card he used. Might be a weak assumption, dunno.

The OP here didn't mention which card (or other hardware) he's using, I don't think.
Also, someone (7ekno) at that 'other forum' stated that they were having "decoding issues" using a 2xx card but not with a 4xx card, although I'm not clear whether he was just talking about casual video playback (which is of no real consequence), or whether it was in conjunction with encoding?

Probably best now to wait and see what lines DG is thinking along, or what information he needs from us.

[sandy_geser]

The OP here didn't mention which card (or other hardware) he's using, I don't think.

you are correct. it was Nick007 who mentioned GTX 570 and GT 220M. Hope OP will share his card info too. Both I think are vp4. I don't really know how the dgdecnv process works but I thought it's to do with the vp engine and from what I read there's no difference in vp between cards (i.e vp4 on one card would be the same on a different one with vp4). Could be more to it than that *shrug*

[nomolu]

I'm seeing the same problem with blockiness and artifacts when transcoding OTA ATSC HD MPEG2 videos to DVD resolutions. Initially I thought I was seeing corruption of the transport stream, but after seeing this thread and going straight to the "corrupt" part of the video, I realized that DGDecNV was producing the corruption. Looking at the DGI (index file), I found that the corruption stops at the first I frame following the corruption.

So far, I'm only seeing the corruption in 1280x720p, 59.94 FPS streams, and not 1920x1080i, 29.97 FPS streams. But that might just be because I've happened to convert more 1280x720p streams lately.

I've found that the problem only occurs when I use my GT520 card. I temporarily went switched back to my old GT220 card, and the problem does not occur with it.

I was using nVidia's 280.26 drivers, but after experiencing this problem, I upgraded to 285.58. But the new drivers didn't fix the problem.

For my testing I reduced my AviSynth scripts to something like the following, and I verified that the problem still occurs:
LoadPlugin("C:\WinApps\DGDecNV\DGDecodeNV.dll")
DGSource("D:\video.dgi")
Trim(0, 41999)

After much testing I found that if I changed the TRIM statement so that my conversion starts on, or after, the second I frame in the stream, then the problem doesn't occur. I tested a couple of other 1280x720p streams that had corruption and this was true for them as well.

Here's the kicker: the corruption only happens with the GT520 card when I use version 2041 of DGDecNV. When I switched back to version 2040, the corruption never happened! So the problem seems to be related to whatever changes were made to produce version 2041.

So, there are two workarounds for now:
1) Go back to using version 2040.
2) Use a script like the following to force the bulk of the decoding to start after the second I frame:
LoadPlugin("C:\WinApps\DGDecNV\DGDecodeNV.dll")
Part1 = DGSource("D:\video.dgi")
Part2 = DGSource("D:\video.dgi")
Trim(Part1, 0, 59) + Trim(Part2, 60, 0)

admin, if this isn't enough information for you to fix the problem, and the fact that you can't reproduce the problem with your hardware, then how about producing a debug version of DGDecNV? You know, a version which would log values of interest to you as the program goes about doing it's work that we could submit to you. Just an idea.

[admin]

Please set this in your INI file before opening DGIndexNV and then do the entire process:

Decode_Modes=0,1,0

2040 had this by default but now it is 0,0,0 because I thought that the underlying bug had been fixed in the driver, but maybe not.

If that does not help then post a link to the smallest source sample you can make that still shows the issue. Use DGSplit to cut the transport stream.

This does not sound like the problem of this thread.

[nomolu]

It sure looks like the problem mentioned in this thread. And isn't the fact that I can work around it with TRIM functions show that this is like the other poster, Nick007? I just narrowed down the "breaking point" to the second I frame in the elementary stream.

I just ran a test with "decode_modes=0,1,0" and the problem still occurs at the same spot. I don't have the original transport stream anymore. I'll see if a trimmed version of the elementary stream exhibits the problem. Would that be acceptable?

[admin]

As long as it is the demuxed but otherwise unprocessed source material I can look at it. As I keep saying, I can fix things only if I can duplicate them, and it always involves giving me a stream and process details. But don't bother posting a link if the stream cannot be used to create the problem. And I would strongly prefer that you do not post reports about streams you no longer have.

[nomolu]

I do have the streams that I reported on in my first post/report. You see I was reporting on, and all of my testing has been done on, elementary streams. I'm sorry if I left out the fact that I take the TS file that I capture OTA and then remove the commericals and what not using VideoReDo Plus to produce an elementary stream.

Anyhow, I've tried creating a small sample, but I can't reproduce the problem unless I go from the very beginning of the file to the point of the corruption which is at 11 minutes and 28 seconds in a 1280x720p file. This means the resulting file is 665MB. And mediafire.com only allows free hosting of up to 200 MB. Is there another free file transfer service that you recommend? Or should I split this sample into 4 files?

[Nick007]

You archive it with winrar/winzip and split into parts during the process.

[admin]

Or just split it with DGSplit and I can reassemble them. Nick007's idea is cleaner, though.

[nomolu]

Here are the links to my sample:

http://www.mediafire.com/?7k4jf6ay5k60ppx
http://www.mediafire.com/?37er3lfrcf7grnl
http://www.mediafire.com/?5vy5218b9217kkt

For me, and my GT520 video card, the corruption starts at frame 41295 and ends at frame 41328. One has to start at the beginning (frame 0) and go sequentially all the way thru frame 41328 to see the problem. If you go straight to frame 41295, you don't see any problem. And, if you do a TRIM(19, 0) when converting the stream, or use a value greater than 19, the corruption doesn't happen. Frame 19 is where the second I frame starts.

If you don't see the corruption, and remember, I don't see it when using a GT220 card, then maybe my idea of a "debug" version of DGDecNV could be considered.

[Penden]

Hello,

About a year ago, I was using WinXP 32bit and was doing bluray rip and got exactly the same problem as the thread starter and I could not figure out what was wrong. I tried to use different builds of x264 and different settings in the x264 and the results remained the same. Turned out I used back DGAVCIndex/DGAVCDecode 1.09 and got no problem.

In this year, I upgraded to Win 7 SP1 64bit with DirectX11 and I have a GTX460 using the latest driver(258.62 or 258.79). I was doing a bluray rip using DGIndexNV 2041 and got a weird problem. I checked the m2ts and found no problem but I deleted the video clips. I remember the problem was like that:
Assume I want to encode frame 0 to frame 5000 from a m2ts(more than 100,000 frames) and the problem happened at frame 4000.
Let say the frame sequence around frame 4000 in the m2ts is: a b c d e f g h i j k l m n o p q r s t
The encoded frame sequence around frame 4000 in the .264 will be: a b c d e f g h i j h k l m n o p q r s t
A duplicated ghost frame is produced behind.
If I use Trim(3000,5000) in the AviSynth script, there will be no problem.
If I use Trim(0,5000) in the AviSynth script, the problem happens at frame 4000
If I use Trim(0,3000) ++ Trim(3001,5000) in the AviSynth script, the problem happen at frame 4000
I could not figure out what was wrong. I tried to use different builds of x264 and different settings in the x264 and the results remained the same. I tried to use DGAVCIndex/DGAVCDecode 1.09 but it does not work because I am using all 64bit programs and I cannot find DGAVCIndex/DGAVCDecode x64. At last, I re-read all documents very carefully that I got from rationalqm.us/dgdecnv/dgdecnv.html and then changed one setting. Bingo! No problem for the encoded video. What I did was to change the default setting for "use_D3D" from false to true in the AviSynth script. (I have not tried this 'trick' on the bluray which I had problem a year ago). I used DGIndexNV for more than a year. From what I found, it really depends on the bluray sources. Most do not have any problem but very few do have weird problem and I still do not know which bluray sources will cause the weird problem until I encode the video and find out.

May I ask if it is completely safe to use "use_D3D=true" all the time? Does this setting depends on which graphic card and driver I am using? Is there any side effect or quality impact on the output video by using this settings?

Thank you

[admin]

Now we're getting somewhere. Thanks for the stream, nomolu. Thanks for your report, Penden.

Using nomolu's stream, I was able to create the issue as nomolu described.

I confirm that either one of these will fix the issue:

1. Use DGSource(...,use_D3D=true)

2. Use 2040 instead of 2041.

D3D mode should affect performance only. I'd have to test to give you the amount.

As to why 2040 works for this stream, I can't see any relevant differences in DGDecodeNV so I'll have to add the changes incrementally to see where the issue comes in.

I would like anyone having this issue to try with use_D3D=true and 2041 and report the results.

[admin]

The 2041 version of DGDecodeNV was not reading DECODE_MODES properly from the DGI file for MPG and VC1 streams. I have slipstreamed a fix in 2041 so re-download it and update both DGIndexNV and DGDecodeNV, set Decode_Modes=0,1,0 in your INI file, and remake your project.

Leave use_D3D=false.

With those changes, 2040 and 2041 decode nomolu's stream cleanly.

Now let's investigate the D3D connection for streams that are still broken even with the above changes. If you have a stream that is not fixed by the changes above, please try use_D3D=true and report your results.

[sandy_geser]

I tried with the h264 tvbluray I had a glitch problem with (described on a different thread). Used the fixed 2041, encoded to elementary stream. One encode with Decode_Modes=0,1,0 and D3D=0 and one with Decode_Modes=0,1,0 and D3D=1.
both contained the same glitch in the same spot. I should remind again that this one had the glitch only when resized in to 1280x720, in previous test leaving it as is didn't result in a glitch.
Also I got those with 4xx and 5xx series cards but not with a 2xx series one. The new test was done only with gt 520.

Next I will try both scenarios with the vc1 movie I have that had glitches resized and unresized.

[NDM]

Anyone else getting a "not recognised win32 executable" type error when trying to run the revised (2041) 'DGIndexNV.exe'?

Thanks for the progress all involved....
Just hope that my problem is the same issue being tackled here, and that there's a fix around the corner.
Can't do any proper testing myself until at least Sunday sometime.