DGDecomb

[jpsdr]

Ok, basicaly what you're doing follow the same idea than me, after reading your description. Instead me is a simple pixel difference, the denoising part is just done by removing the 2 LSB...
After, i do a lot of things, validate IVTC pattern (the 2 frames must be contiguous), if no data are validate enough (the decisions threshold are not meet) i keep the previous pattern to apply on the current sequence, etc...

[admin]

Thank you for the explanation. It seems to me that your solution is limited to fixed regular 3:2 pulldown and you cannot operate correctly on streams with irregular pulldown. Also, noise is not always limited to the two LSBs, but it's a reasonable thing to do.

BTW, the correct term for what you are doing is not "correlation" but "sum of absolute differences".

Did you like my results on your stream?

[jpsdr]

Didn't tested yet, i'll do this a little latter properly when more time (very probably this WE).
Yes, i think it's limited to 3:2, but honestly, don't remember exactly...
I've done this "especialy" for anime, which have "standard" 3:2 IVTC, but where the pattern changes at each scene, or even more during the same scene (very rare), but still stay a 3:2. It may be limitated, but increase the robustness of the detection.
And finaly, for scenes where nothing works, there is the "manual mode". I feed a text file to my plugin, telling on which frames i put the pattern. For exemple, i'll feed something like that :
1000 1500 3
5000 5030 1
10001 10033 4
....
Means :
Automatic detection until frame 999.
Force IVTC pattern to 3 between frames 1000 to 1500.
Automatic detection from frame 1001 to 4999
Force IVTC pattern to 1 between frames 5000 to 5030.
etc...
Because my purpose it to NOT deinterlace the result, i want it to be "perfect" progressive, without "bluring" because of a deinterlace.
But, on these files, very often fading are mess, mixing 2 IVTC pattern when between scene, or are field based instead of frame based on black/white fading, making them impossible to IVCT, whatever you do there is interlaced.
It's where the "manual mode" of mu deinterlace filter comes...
If feed it whith a text file the same way my IVTC filter, with for exemple :
100 110 1
1000 1020 2
.....
Means :
- Deinterlace between frames 100 to 110 with "mode 1".
- Deinterlace between frames 1000 to 1020 with "mode 2".
Between : Do nothing.
With these both tools and their manual mode, i'm able to get a full IVTC without deinterlacing result (except the fading parts).
But, the working time to get this result, is between 40 minutes to 2h for a 20 minutes episode.

[jpsdr]

Euh... I was too much curious and get the last 2053 version, but i have an error message telling me that there is not such function as dgtelecide...
I've opened an avs file using DGSource, it works fine, but only if i put the license.txt file in the avisynth plugin directory (otherwise, i have a green output with something small unreadable at the top left of the screen), it wasn't like that before...
Same result under Windows 7 SP-1 x64 with avs+ r2440 and under Windows 7 SP-1 x86 with avisynth 2.6.1.

[Sharc]

OK, Sharc, please re-download and re-test. Getting close to goodness now.

I was thinking of providing a way to dynamically change pthresh for different scenes. Something like a config file:

0: 1.5
100: 3.2
1050: 0.5

The pthresh value would change at the given frame number and remain until changed again. I'm not sure it's worth the bother, though.

All good now. Chroma problem solved. Thanks

P.S.
The config file for dynamically changing pthresh would have to be elaborated manually, by stepping through the file and inspecting the show=true result, right?

[admin]

Euh... I was too much curious and get the last 2053 version, but i have an error message telling me that there is not such function as dgtelecide...

It has not been slipstreamed yet, so you need the Beta.rar linked earlier.

http://rationalqm.us/misc/Beta.rar

I've opened an avs file using DGSource, it works fine, but only if i put the license.txt file in the avisynth plugin directory (otherwise, i have a green output with something small unreadable at the top left of the screen), it wasn't like that before...
Same result under Windows 7 SP-1 x64 with avs+ r2440 and under Windows 7 SP-1 x86 with avisynth 2.6.1.

Yes, I added protection to the DLL. You should read the Readme.txt file.

[admin]

The config file for dynamically changing pthresh would have to be elaborated manually, by stepping through the file and inspecting the show=true result, right?

Yes. Some people are picky enough that they would be willing to do it.

[Guest]

Thanks to D.G. and his CUDA tools I am getting the hang of very basic avisynth scripts, well kind of sort of.
So as I read the documentation I have some questions that arise, apart from what did he say?
If there are no objections I will ask the question in the appropriate sub forum in the hopes the more experienced users help me out.

If I may ask an opinion
Which would you recommend for DGTelecide
blend=true or blend=false

[admin]

Which would you recommend, coffee or tea? It depends a lot on personal taste. Here is my thinking. Blend hides aliasing (stairstepping artifacts) better, but ghosting is produced that can be visible for large motions. However, for me when you play at normal speeds the ghosting is not apparent (that's why we have blend decimation for hybrid content). So, in general, I prefer blend. Some people hate blending. That's why I made the default blend=false, so people would not hate on me. Or at least fewer people, haters gonna hate.

One final point, for IVTC, likely the number of postprocessed frames is small, and so it may not matter that much which mode you use.

Looking forward to your other questions.

[Guest]

Which would you recommend, coffee or tea?

Depends on whether I am going to mix in some good rum or not.

i know, let your eyes be the judge, just trying to get some pointers.
Thanks for the information

[admin]

Irish whiskey in the coffee.

[Sharc]

Just for my understanding:
The classic Decomb plugin includes the 3 functions
a. FieldDeinterlace(), an adaptive deinterlacer for interlaced video
b. Telecide(), for field matching of telecined footage with deinterlacing postprocessor
c. Decimate(), for decimation of 1 in every N frames

Now we have the GPU CUDA supported similar functions
d. deinterlacing or bobbing invoked via DGSource(), which is always acting on all frames, as I understand it.
e. DGTelecide(), a substitute for the classic b.
f. DGDecimate(), a substitute for the classic c.

A substitute for a. (adaptive deinterlacing of interlaced video) seems to be missing in CUDA world. I could perhaps try with DGDecomb(ptresh= ) postprocessing, however this always includes field matching which is probably not recommended for interlaced video. Of course I can always use the classic FieldDeinterlace() or similar if need be.

Or do I miss something?

[admin]

Correct, Sharc, I have not implemented DGFieldDeinterlace yet. I will do it fairly soon if not straight-away.

[jpsdr]

Ok, i've got the beta, i'll test this evening when back home.

Ohh... another shared idea i'm using to detect telecine (still oriented 3:2).
The following :
A B C D
0 1 2 3
will result after telecine in (one possible pattern) :
A/A B/B B/C C/D D/D
0 1 2 3 4
So, if both SoAD (and not "correlation" this time ) of (Top field of frame 1 and 2) and (Bottom field of frame 3 and 4) are "null" frame 2 and 3 are telecined.
This is something i'm checking in a second time, the B plan, if first tests (A plan) are not "good enough". "null" at noise or compression artifact threshold.

Another thing, as my IVTC filter is oriented 3:2 (but not sure limitated...), basicaly, it works by pack of 5 images at the time. So, decision algorithm is runned each pack of 5 images.
I can't wait to be back to check the result.
.......... Euh... I've just read the documentation, and... i've found out that your filter result is probably totaly different from mine, and why mine is more oriented 3:2.
Your filter deinterlace :

blend: bool (default: false)
When blend=true postprocessed frames are deinterlaced using field blending. When blend=false postprocessed frames are deinterlaced using field interpolation.

My filter reconstruct the frames.
Unless... I've misunderstood something.

Process the file i've provided to you in VDub with my filter with the following parameters :
Color format YV12 for input and output.
In the filter chain, put IVTC v6.2.0 and Remove frame v1.2.4 (with default settings for both).
Run/save the process, do not preview or anything, it will not work.
Or, i can put this evening when back home the result on my sever.

[admin]

jp, you have totally misunderstood. C'mon, I'm not an idiot. I do not perform IVTC by deinterlacing the frames. And my previous post to you clearly explained that I reconstruct the frames when possible. You even posted saying I am doing what you are doing.

Field matching is performed. If there is a good match, that is delivered. It is not deinterlaced. If there is no good match, that is, a clean frame cannot be reconstructed (possibly due to bad edit) then the frame is deinterlaced (if postprocessing is enabled). A "postprocessed frame" is one that did not find a good field match.

I suppose it was too much to think people would be familiar with Decomb, so I will copy over material from its documentation to the DGDecodeNV document.

[jpsdr]

A "postprocessed frame" is one that did not find a good field match.

Ok, that's what tricked me.
So, if i understand properly, and want to compare with my filter, i have to just use :
dgtelecide()
This will only perform the IVTC operation on frames detected telecine, and nothing more.
Perfect, just a few hours to wait to test...

[admin]

Yes, that is correct because pthresh defaults to 0.0.

Looking forward to your comparisons. What does your filter do with the interlaced fades in the stream you gave? Mine catches them with pthresh=2.5.

[hydra3333]

thank you for the release of DGTelecide and DGDecimate
viewtopic.php?f=4&t=463&p=6380#p6380

I'll be asking for them to be added to Vapoursynth. They'll ask me how many frames before and after that the filters calls so that they can hard-code it... I wonder if you could please confirm, and also for other filters when you (re)release them ?

[admin]

Please wait. I am still fixing some bugs in DGTelecide. I'll let you know.

[jpsdr]

What does your filter do with the interlaced fades in the stream you gave?

"Nothing" or some wrong mix-up is also possuble, but in any way there is no possibility to have a true progressive picture in that case, because two IVTC patterns are mixed up. So, I'll deinterlace them after with my deinterlace filter in manual mode, feeding it with a text file where i specify which frames i want it to deinterlace. I don't want to leave this step to an automatic method, because there is no way to be sure there is no miss or false detection. So, for this step, visual inspection and manual setting.
My IVTC method is a "long" several step process to achieve the result i want.

[Guest]

Hope you don't mind my asking an off topic question, but how is the gtx1080ti so far?

[admin]

Well, I'm loving it but I am still in the process of migrating to that machine, so won't start using it on a daily basis until my case arrives tomorrow. I installed most of the apps I rely on. I'm posting here from the new machine, still bread-boarded on the dining room table.

The 1080Ti overclocks very nicely. There's a strange thing. While running heavy CUDA there is a slight extra noise that is not from the fan. Weird.

Here's another weird thing. I have a local app called DGLicenseRSA that lets me make licenses without having to go to the online generator. It always had a little bug that cost me a few extra clicks and keystrokes. I transferred it over and opened it in VS 2013. Ran it without any changes and it did not work! The only difference is that the original EXE, which still works, was built with VS 2005. So apparently things have changed so as to hose my big digits code. One day, I'll look into it, but for now I'll just use the old EXE.

Last weird thing: while hunting for the deinterlacer bug I wasted many hours because I forgot that interlaced YV12 has chroma samples in different places than progressive. By going back to the Decomb code, I remembered all that.

Sorry for the off-topic.

[Guest]

Sorry for the off-topic.

You are the admin and owner of the forum, I think you are allowed to do it on occasion
Back on topic
After your remark about IVTCing interlaced sources to me, after my test of DGDecomb, I did some reading up.
Seems that just because mediainfo called my sample 480i it doesn't make it interlaced, it was actually hard telecined.
3 progressive frames and two interlaced
Guess even this old dog can still learn a thing or two

[admin]

gonca, my friend, just keep the inner child alive and always take the high road.

[Aleron Ives]

While running heavy CUDA there is a slight extra noise that is not from the fan. Weird.

Have you encountered the dreaded coil whine?