Problem decoding 12bit 4:4:4 HEVC

Selur · Post by **Selur** » Fri Sep 06, 2024 4:31 pm

I got an NVIDIA RTX 4080 which should decode this fine.

Using dgdecnv 255 (from 2024-07-30 08:23):

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/DGDecNV/DGDecodeNV.dll")
# Import scripts
import validate
# Source: 'G:\Output\test.mp4'
# Current color space: YUV444P12, bit depth: 12, resolution: 640x352, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg, format: HEVC
# Loading G:\Output\test.mp4 using DGSource
clip = core.dgdecodenv.DGSource("J:/tmp/mp4_dfa9774a755e37392a6138bbabde08d5_853323747.dgi")# 25 fps, scanorder: progressive
frame = clip.get_frame(0)
# setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)
# setting color transfer (vs.TRANSFER_BT601), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601)
# setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 25fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
# adjusting output color from: YUV444P16 to YUV444P12 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P12, range_s="limited", dither_type="error_diffusion")
# set output frame rate to 25fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# output
clip.set_output()

I get:

I first thought I made a mistake with the conversion somewhere, but using BestSource with 'hwdevice="cuda"':

Code: Select all

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/BestSource/BestSource.dll")
# Import scripts
import validate
# Source: 'G:\Output\test.mp4'
# Current color space: YUV444P12, bit depth: 12, resolution: 640x352, frame rate: 25fps, scanorder: progressive, yuv luminance scale: limited, matrix: 470bg, format: HEVC
# Loading G:\Output\test.mp4 using BestSource (NOT recommended and untested!)
clip = core.bs.VideoSource(source="G:/Output/test.mp4", cachepath="J:/tmp/test.json", track=0, hwdevice="cuda")
frame = clip.get_frame(0)
# setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)
# setting color transfer (vs.TRANSFER_BT601), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601)
# setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 25fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# making sure the detected scan type is set (detected: progressive)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
# set output frame rate to 25fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# output
clip.set_output()

instead of DGSource, works fine:

Source was created with x265, uploaded the source to my GoogleDrive.

Would be nice if this could be fixed.

Cu Selur

Post by **Rocky** » Fri Sep 06, 2024 6:23 pm

Works fine for Avisynth. I'll look into what's going wrong for Vapoursynth. Thank you for your report.

Post by **Rocky** » Fri Sep 06, 2024 7:18 pm

OK, I have it fixed. Please re-download build 255 and give it a taste. When you confirm tastiness I will make an announcement.

I missed setting Vapoursynth subsampling to 0,0 for 4:4:4.

Selur · Post by **Selur** » Fri Sep 06, 2024 11:16 pm

Works fine in Vapoursynth with the new version too.

Thanks!

Cu Selur

Post by **Rocky** » Sat Sep 07, 2024 12:24 am

Great! Thank you for your tasting.