[RESOLVED] Getting "Scan Lines" When Using fieldop

[AYColumbia]

I just got my 10-season collection of SG-1 on DVD and am re-encoding to AVC for my media server. They're mostly film, i.e., 95%+ so am using fieldop=1 in my AVS script. The result looks great (compared to using fieldop=0 with deinterlacing resulting in nasty jagged output) but I still get frames that look like this:

Is this expected or am I doing something wrong? The last two lines from the DGI file

Code: Select all

96.62% FILM
ORDER 2

I used "scan lines" to describe because I wasn't sure what the actual terminology is for what I'm seeing. Thanks for any guidance on this attempt.

[admin]

It's hard to say without seeing an unprocessed sample from the stream. Can you cut 50MB containing some frames like that and upload it somewhere for me. You can cut the VOB with DGSplit.

I could start speculating but I prefer to see the stream and go straight to the correct answer.

The usual term used is "combing".

[AYColumbia]

Thanks a lot for your help. I'm uploading the clip now. I don't have high upload speed so it'll be a bit. I'll PM you the link to download when it's ready. I use MakeMKV to generate files from DVD before re-encoding them, so I used mkvmerge to create the clip.

In case you want to know, here's my basic script:

Code: Select all

LoadPlugin("E:\MultiMediaTools\DGDecNV\DGDecodeNV.dll") 
DGSource("Stargate SG-1.S01E01.Children of the Gods (1 and 2).dgi",fieldop=1) 
converttoyv12()

Let me know if it'll help to have the entire DGI file and I can send it to you as well or any other info you may need. Thanks again.

[Nick007]

SG-1 sadly cannot be inverse telecined to 23.976 fps without any side effects. It would require variable framerate to remove all interlacing and keep all frames. VFR (playback at 60 Hz TV frequency) isn't very suitable for fluidness, though.
Plus it has a lot of grain and encodes would need huge bitrate, so I just remuxed it and with madVR's force film mode it works good enough.

[AYColumbia]

SG-1 sadly cannot be inverse telecined to 23.976 fps without any side effects. It would require variable framerate to remove all interlacing and keep all frames. VFR (playback at 60 Hz TV frequency) isn't very suitable for fluidness, though.
Plus it has a lot of grain and encodes would need huge bitrate, so I just remuxed it and with madVR's force film mode it works good enough.

Shoot, I hate to hear that. So you just kept the original mpeg-2. I do use madVR too but I only preview on the PC and stream to my Blu-ray player to watch on TV. I only re-encode to save space but I have noticed some DVD sources can be a real pain.

Here's what I found last night with these settings:

With deinterlace=1, use_pf=true I still get combing


With just deinterlace=1 I get no combing but not as smooth output, i.e., jagged edges per other images below


A frame from output with no visible combing but not as smooth as you can see the edges are rough compared to the last image. Very noticeable in the gold helmet.


Same frame as above output with deinterlace=1 and use_pf=true or using fieldop=1 which is nice and clean but I also get combing in the full output


I wish there was a setting that mixes the two. Actually, I wish I had the time to really delve into this stuff and understand it better.

[AYColumbia]

It's hard to say without seeing an unprocessed sample from the stream. Can you cut 50MB containing some frames like that and upload it somewhere for me. You can cut the VOB with DGSplit.

I could start speculating but I prefer to see the stream and go straight to the correct answer.

The usual term used is "combing".

Hey, have you had a chance to look at the clip? Any insight? Thanks.

[AYColumbia]

Hey all,
I think I found a solution for my problem. I did a little more research and found some posts pointing to HandBrake. I had never used this, but was able to install and determine it does have a CLI. I used a couple short clips to test the detelecine option and it works. I just finished encoding Children of the Gods (episodes 1 & 2) and it came out great. I basically took my current encoding command file and updated it to use the HandBrake CLI. Here's my command line:

Code: Select all

"Handbrakecli.exe" -i "Stargate SG-1.S01E01.Children of the Gods (1 and 2).mkv" -t 1 --angle 1 -o ".\output\Stargate SG-1.S01E01.Children of the Gods (1 and 2).mkv"  -f mkv  --detelecine  -w 720 --loose-anamorphic  --modulus 2 -e x264 -b 2909 -2  -T -r 24000/1001 --cfr -a 1 -E copy -6 auto -R Auto -B 0 -D 0 --gain 0 --audio-fallback ffac3 --x264-preset=veryslow  --x264-profile=main  --h264-level="4.1"  -x open-gop:colorprim=smpte170m:transfer=smpte170m:colormatrix=smpte170m --verbose=1

The key switch is --detelecine but am not sure how this is being used but it worked. It'll be interesting to see whether the remaining episodes will be as good as the first.

[EDIT] BTW, not a perfect solution as there is still some combing during some scene changes like the one below but I can live with these at this point.

[Aleron Ives]

Have you tried downloading Decomb from the main site? Telecide ought to handle all of those problems.

[AYColumbia]

Have you tried downloading Decomb from the main site? Telecide ought to handle all of those problems.

Yes, I tried all sorts of settings and combinations from doing the research on this and none of the results were acceptable. I was only able to accomplish one or the other but not both, i.e., combing gone but output is too jagged or output is smooth but a lot of combing.

[Aleron Ives]

I can take a stab at it I guess if you link me to your sample, since admin hasn't been online since Saturday.

[AYColumbia]

I can take a stab at it I guess if you link me to your sample, since admin hasn't been online since Saturday.

That would be great if you have the time. Let me upload a little bigger sample that includes the start of the episode. It's the first 3+ minutes and may be around 160MB. I'll upload it later when I get home.

[Aleron Ives]

I assume you haven't done any processing on the video since it's still MPEG-2, but generally you need to post unprocessed source samples (VOB, in this case), rather than a remuxed MKV.

Using Telecide, I was able to get rid of this:



and instead get this:



The preceeding frame is this:



so it seems that part of the phone and the face are bleeding through into the subsequent frame. Regarding the helmet, Telecide's debug information indicates that those scenes have a vthresh in the low 30s, so if you set vthresh to 40+, it won't catch the helmet anymore:



Here's the script I used:

Code: Select all

LoadPlugin("...\DGDecodeNV.dll")
LoadPlugin("...\Decomb.dll")
DGSource("sg1.dgi")
AssumeBFF()
Telecide(guide=1,post=2,vthresh=45)
Decimate()
Crop(8,0,-8,-0)

If I try AssumeBFF().SeparateFields(), the phone still bleeds through:



but the next "frame" is clean:



so it would seem to me that this isn't a problem caused by bad field matching; it looks like whoever mastered the DVD incorrectly blended the fields during scene changes. You would probably have to use special AviSynth plugins to try to clean that up, which would be a lot of extra work. Removing the combing is probably the best you can hope for, unless you want to just leave everything as interlaced and let your PC/TV deinterlace it for you during playback.

[AYColumbia]

Wow, that's good stuff. Thank you so much for taking the time and explaining. The MKV has the untouched elementary streams via MakeMKV. So there's no encoding as they're identical to what's on the DVD. The DVD is the original so the mastering is done by whomever MGM hired to do it. I bought the entire 10 seasons in the individual packaging so not the full 10-season set.

Looks like you hit the nail on the head with the vthresh parameter. My script looked similar to yours but my parameter didn't go that high. I think I had gone as high as 25. I really don't mind the bleed-through if it's clean, i.e., no combing. That typically happens on scene changes so not a big deal really. I'll take a look at Telecide's debug to see how that works because I had not used this feature.

BTW, I found one other issue in that I use MediaInfo to determine field order and it reports TFF, however, after looking at your scripts I noticed you use BFF. I checked the DGI file and sure enough it reports ORDER 2 (BFF). Not sure how MediaInfo determines this but it's wrong in this case. I'm going to have to update my encoding command file to look for the ORDER in the DGI parameter and set the order that way. Thanks a lot for your help.

[admin]

Thanks for helping out, Aleron. I've been rather busy lately. It looks like you have it nailed.

[AYColumbia]

...
Here's the script I used:

Code: Select all

LoadPlugin("...\DGDecodeNV.dll")
LoadPlugin("...\Decomb.dll")
DGSource("sg1.dgi")
AssumeBFF()
Telecide(guide=1,post=2,vthresh=45)
Decimate()
Crop(8,0,-8,-0)

Quick question: how were you able to load this script without the order parameter in Telecide without getting the error stating that you must set the order parameter? My encoding code sets the order parameter but when I removed it to mirror your script to preview it I received the error.

[Aleron Ives]

I'll take a look at Telecide's debug to see how that works because I had not used this feature.

It's a necessary step for determining what vthresh to use. You simply set:

Code: Select all

Telecide(post=3,show=true)

to show Telecide's statistics and display the white motion map on the areas it thinks has combing. The helmet has a vthresh of 31 - 35ish, so my usual vthresh=30 would be too low in this case.

BTW, I found one other issue in that I use MediaInfo to determine field order and it reports TFF, however, after looking at your scripts I noticed you use BFF. I checked the DGI file and sure enough it reports ORDER 2 (BFF). Not sure how MediaInfo determines this but it's wrong in this case.

I determined the field order using Don's advice in the Telecide manual: use AssumeTFF().SeparateFields() and step through a scene with motion to see if it moves smoothly or backwards and forwards. The scene where the officer walks up the ramp to the StarGate moved back and forth with TFF but only forward with BFF, so I knew BFF was correct. I'm not sure if DGIndexNV will always report the correct field order, so I wanted to be sure, as I hadn't looked at any SG-1 DVD clips before.

how were you able to load this script without the order parameter in Telecide without getting the error stating that you must set the order parameter?

I don't understand what you mean. AssumeTFF() and AssumeBFF() are AviSynth commands; Telecide doesn't have a field order parameter of its own. If you omit the field order, DGDecodeNV will tell Telecide what it thinks the field order is, and DGDecodeNV may or may not be correct, so it's best to check and set it yourself, AFAIK.

Thanks for helping out, Aleron. I've been rather busy lately. It looks like you have it nailed.

Thanks for the tutelage and documentation that have made my meager contribution possible.

[admin]

Earlier versions of Telecide() required the order parameter. The latest gets it from Avisynth.

Keep on keeping on, Aleron, you're doing fine.

[AYColumbia]

@Aleron,
Your contribution is not meager at all. You've helped me a great deal in just a couple posts.

@admin,
Do I need to get an updated decomb.dll? I did grab the latest from the main site, i.e., 5.2.4 which still requires the order parameter. To be sure I was using this latest version, I used LoadPlugin to load it from the specific location.

[Aleron Ives]

What's your full script? Are you using AviSynth 2.5 or 2.6? I'm using 2.6, which may or may not make a difference in this case.

[AYColumbia]

What's your full script? Are you using AviSynth 2.5 or 2.6? I'm using 2.6, which may or may not make a difference in this case.

I'm using 2.5.8 right now. I probably should upgrade to 2.6 but it's still in alpha right? Here's the full script (I added the hints parameter):

Code: Select all

LoadPlugin("E:\MultiMediaTools\megui32\tools\avisynth_plugin\ColorMatrix.dll")
LoadPlugin("E:\MultiMediaTools\megui32\tools\avisynth_plugin\decomb.dll")
LoadPlugin("E:\MultiMediaTools\DGDecNV\DGDecodeNV.dll") 
DGSource("Clip325.dgi") 
AssumeBFF()
Telecide(guide=1,order=0,post=2,hints=true,vthresh=45)
Decimate()
converttoyv12()

[admin]

There is no order parameter for Telecide() in 5.2.4. Remove it. If you get an error message without it then you are invoking a wrong DLL.

You can also remove the ConvertToYV12() call as it is already in YV12.

[AYColumbia]

Thanks. I went ahead and refreshed my DLL from the 5.2.4 zip file and now it's working. Not sure what version was there but it should have been 5.2.4 as it had the same date on it. Thanks for the tip on the YV12.

[admin]

Thanks for the status update and I'm glad you have that sorted out. Onward and upward.