[RESOLVED] Help identifying bottleneck

Support forum for DGDecNV
Post Reply
DAE avatar
darcmage
Posts: 5
Joined: Sun Oct 03, 2010 11:32 am

[RESOLVED] Help identifying bottleneck

Post by darcmage »

I need help figuring out why I can't get x264 to max my cpu instead of hovering around the 30% usage mark. I'm trying to convert 1080i video to 480p. I've been doing this since Oct 09 always using the latest version of DGDecNV. I've used both MeGUI and the x264 cli with the exact same results. I can max the cpu if I convert to 720p with more intensive x264 settings but the encoding fps stays the same. If I resize further down to 640x352, my cpu usage drops even more but the encoding fps stays the same.

My script is simple.

Code: Select all

DGSource("D:\1.dgi", deinterlace=1, resize_w=800, resize_h=448)
with

Code: Select all

--level 3.1 --ref 1 --vbv-bufsize 14000 --vbv-maxrate 17500 --no-mbtree
for the x264 settings which is the same as the DXVA SD profile without mbtree

Source:
  • Captured with Hauppauge HDPVR
  • MPEG-TS
  • Bit rate: 10.8 Mbps
  • Maximum bit rate: 20.0 Mbps
  • AVC
  • Main@L4.0
  • 1920x1080
  • 29.970 fps
  • Interlaced
  • Top Field First
Relevant info:
  • Windows 7 32-bit
  • Intel Core 2 Quad Q9400 @ 2.66GHz
  • 2.0GB Dual-Channel DDR2
  • 512MB GeForce 9800 GT (EVGA) ForceWare version 260.63
  • DGDecNV 2028
  • MeGUI 3.5.15
  • x264 v1724
I used to be able to max out my cpu but it only occurred under specific conditions that I've been unable to reproduce with the latest forceware and DGDecNV. With prior versions, sometimes when I begin an encoding job I would get a display adapter reset. Thankfully Win7 deals with these things gracefully and doesn't force a reboot but I would have to abort the encode process after the display adapter recovers. If I begin the same job again, it would run at 100% cpu usage without me changing any settings. I can't explain why and it wasn't easily reproducible. Whatever happened after the display adapter reset would stay that way until a reboot and then I wouldn't be able to max my cpu until I find to a way to force a display adapter reset. Usually I would do it by starting a encoding job and aborting it after a few minutes if there was no reset and trying again.

I've tried everything I could think of with the latest forceware and DGDecNV to crash the display adapter. I tried using D3D=1 in the ini. I tried running various GPU intensive applications and watching GPU accelerated video (which drops too many frames to be watchable) while encoding. Nothing works. This is one time where I wish things were less stable.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: Help identifying bottleneck

Post by admin »

Sorry, but this is really outside the purview of support of my tools. Why does x264.exe not use all the processor? I have no idea.

I suggest that you post this at Videohelp.
DAE avatar
darcmage
Posts: 5
Joined: Sun Oct 03, 2010 11:32 am

Re: Help identifying bottleneck

Post by darcmage »

My mistake, I guess I left out the part about maximizing cpu usage while using DGDecNV. Anyway the problem has been solved temporarily by reverting to the 197.45 drivers. My CPU usage is still not maxed but it is much higher now and the encoding fps is at ~49fps which is as high as I've ever been able to get it as opposed to the ~23fps it was earlier. No adapter resets yet but I'm sure they'll show up sooner or later.

I'd like to come back to this and do some proper troubleshooting when I have time during the offseason next summer if this topic is still in the archive.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: Help identifying bottleneck

Post by admin »

darcmage wrote:My mistake, I guess I left out the part about maximizing cpu usage while using DGDecNV.
I can't make any sense out of this. The whole point of DGDecNV is to *minimize* the CPU bandwidth used for decoding.

If you are transcoding and not fully utilizing the CPU, then you should look to the filtering/encoding part of your process, not to DGDecNV.
DAE avatar
darcmage
Posts: 5
Joined: Sun Oct 03, 2010 11:32 am

Re: Help identifying bottleneck

Post by darcmage »

neuron2 wrote:you should look to the filtering/encoding part of your process, not to DGDecNV.
Could you clarify?

I'm not sure what role DGDecNV plays in this problem if any but there was a bottleneck somewhere that was alleviated by reverting to an older driver set and nothing else. I don't think that would affect any part of the encoding process other than DGDecNV but that requires more testing to be sure.
DAE avatar
Audionut
Posts: 6
Joined: Mon Sep 20, 2010 11:20 pm

Re: Help identifying bottleneck

Post by Audionut »

With the command line you posted at 480p, you ain't going to max out x264.

As for your encoding speed. With a 9800gt, you're going to get about 30-40fps give or take decoding speed with 1080i.
Of course, x264 can encode faster than that, but it isn't magic, it can't encode faster than the decoder is delivering frames to it.

And as you're obviously de-interlacing, that will slow down the decode even more.
for the x264 settings which is the same as the DXVA SD profile without mbtree
No way. DXVA will handle alot more than that.

2 1/2 options,

For 1 pass CRF encoding,

--preset veryslow --level 3.1 --ref 10 --vbv-bufsize 14000 --vbv-maxrate 17500

Or for 2 pass bitrate encoding, encode to a lossless file first and then use whatever commandline you want.
Yes, you will end up doing 3 passes. 1 lossless, 2 x264. But the 2 x264 passes will be very quick when encoding from a lossless file.

The 1/2 option is to run a quick CRF check with the command line I posted and see what your CPU usage is. If you're CPU usage is high, and/or you have a slower encoding speed than you did previous with your command line, and you want to do 2 pass encoding, than you can skip the lossless encode first, as you are obviously maxing x264 out.

Oh, probably the biggest thing also, is that if you are doing a 2 pass encode, the first pass of x264 will automatically do a "fast first pass", which means it lowers alot of the options within itself to increase encoding speed. This WILL result in minimal CPU usage. Remember, even though it lowers options to increase encoding speed, it can't encode faster than the decoder is delivering frames.

edit: What I posted is way off topic for this forum. As admin pointed out, you're best served by seeking help related to x264 in the doom9 forum x264 section.
And as for an older driver increasing encoding speed. There would appear to be a problem with the latest nvidia drivers and your hardware setup. You should take that up with Nvidia.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: Help identifying bottleneck

Post by admin »

It seems to me that the OP should be focused on maximizing the transcoding frame rate at the required quality rather than maximizing the CPU utilization.

If an Nvidia driver issue is suspected, lets address that in terms of the decode frame rate (without transcoding).
DAE avatar
darcmage
Posts: 5
Joined: Sun Oct 03, 2010 11:32 am

Re: Help identifying bottleneck

Post by darcmage »

neuron2 wrote:If an Nvidia driver issue is suspected, lets address that in terms of the decode frame rate (without transcoding).
The playback in DGDecNV using the 260 driver set with maximum playback speed and single rate deinterlacing while the display is disabled was around 20fps. With the 197.45 drivers, DGDecNV played at 50fps.

Audionut is probably on the right track with this being a driver issue and I'll contact Nvidia when I have more time.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: Help identifying bottleneck

Post by admin »

Groucho2004 wrote:You're wasting people's time.
Not really, because it's important to make sure Nvidia knows about this before releasing the new drivers. I will be telling my Nvidia contact about this.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: Help identifying bottleneck

Post by admin »

@darcmage

Does this issue still occur with the latest release drivers?
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: [OPEN] Help identifying bottleneck

Post by admin »

Marking as resolved due to no-response by the OP.
DAE avatar
darcmage
Posts: 5
Joined: Sun Oct 03, 2010 11:32 am

Re: [RESOLVED] Help identifying bottleneck

Post by darcmage »

Apologies for not reporting in earlier. I replaced the 9800GT with a GTX460 and was waiting for enough time to pass by for me to definitively say the problem is resolved. I can do that now. I made no changes to the system other than replacing the card and updating to the newest drivers and everything is working great without me having to jump through hoops. It's still possible and likely the problem has something to do with my hardware config but I'm of the opinion that the CUDA implementation in the 9800GT was flaky.

Thanks again for this great piece of software that has saved me thousands of hours of encoding time.
Post Reply