[RESOLVED] Set consistent audio filename

[TiberiusT]

Have I missed a way to define the outputted audio filename using cli?

For my batch processing workflow it would be great to have it the same as the .dgi file so DGIndexNV would output clip1.dgi and it's corresponding clip1.ac3 etc etc

Tks
T

[admin]

You cannot specify the audio file name.

[TiberiusT]

OK. Is it possible to implement this as a feature then? I'll expand...

It’s primarily for 1080p/50fps .mts camcorder files from an HDC-SD700 which DGSourceNV handles perfectly. Other source filters have various issues and I get very inconsistent results with the audio inside avisynth, so I take it out first, process the video, and mux the audio back in at then end. AV sync issues with this method seem to be so minimal that they are irrelevant which is a tribute to DGNV, since everything else seems to have some sort of trouble with either the video or audio that this camcorder produces.

Typically, a camcorder event has about 25 scenes (files). I am automating the archiving process using batch files. I run avisynth batch scripter to create .avs files on a test file, then run a batch file which uses DGIndexNV to index, then x264 encodes to .H264 then MKVMerge muxes the .H264 file with the demuxed audio from DGIndexNV and appends the files together so at the end of the process I get one .mkv file per event.

I use the DOS ‘For’ command so every file in the directory is processed. This is fine if the files are consistently named, and a big bit missing is the matching demuxed audio filename from DGIndexNV, which I have to edit manually to match the relevant .dgi filename. Not a big deal, but it gets tiresome and more importantly it's very error prone when there’s 20 or more files to do. I am still finalsing the avs defaults and the x264 cl and I have over 300 original mts files on my HD waiting to be processed optimally.

Here's an example batch file

Code: Select all

prompt $d $t $_$P$G
set finalname=Ninemia
set workpath=\Video\Camcorder\Panny\Ninemia\
set size=1920x1080
set fps=50
set xn=mts
d:
cd %workpath%
for %%s in (*.%xn%) do "d:\video\AA Encode\dgnvtools\dgindexnv.exe" -i "%%s" -o %%~ns.dgi -a -e
for %%s in (*.avs) do "d:\Video\AA Encode\x264\x264.exe" --crf 20 --preset slow --sar 1:1 --output "%%~ns.h264" "%%s"
for %%s in (*.h264) do "d:\Video\AA Encode\MKVtoolnix\mkvmerge.exe" -o "%%~ns.mkv" "--display-dimensions" "0:%size%" "--default-duration" "0:%fps%fps" "--no-global-tags" "--no-chapters" "%%s" "--no-global-tags" "--no-chapters" "%%~ns.ac3"
mkvmerge -o %finalname%.mkv --title %finalname% 1.mkv +2.mkv +3.mkv +4.mkv +5.mkv +6.mkv +7.mkv +8.mkv +9.mkv +10.mkv +11.mkv +12.mkv +13.mkv +14.mkv +15.mkv +16.mkv +17.mkv +18.mkv +19.mkv +20.mkv +21.mkv +22.mkv
cmd

I also use a similar workflow for .ts HDTV 1080i captures with .ac3 5.1 audio. Life is not long enough to use a CPU based bob deinterlacer on those files, and again DGSourceNV handles the audio demux and video thru avisynth with 100pct consistency. Everything else seems to bork somewhere at sometime...(and no, I don't use Avisynth MT )

So, if it's possible to have the demuxed audio filename match the .dgi filename it would save me a lot of time and trouble. If it's a limitation then I suppose I could write another 'For' command to rename them, but the original files would have to be a consistent length to make that reliable.

Tks
T

[admin]

If you use a matching name instead of what I currently generate, how do you expect to get the delay information?

[TiberiusT]

Thanks for your response. My apologies, I'd given up on the reply and didn't come back here for almost a month.
On these files there's no significant audio delay so I ignore it. Here's the dir of the demuxed audio for the .MTS files used in the workflow above. They are representative of what the camcorder produces:

Code: Select all

05/09/2011  20:45    <DIR>          .
05/09/2011  20:45    <DIR>          ..
05/09/2011  20:42           646,656 1 PID 1100 3_2ch 48KHz 384Kbps DELAY 0ms.ac3
05/09/2011  20:42         2,304,000 10 PID 1100 3_2ch 48KHz 384Kbps DELAY 32ms.ac3
05/09/2011  20:43         2,443,776 11 PID 1100 3_2ch 48KHz 384Kbps DELAY 0ms.ac3
05/09/2011  20:43         7,835,136 12 PID 1100 3_2ch 48KHz 384Kbps DELAY 0ms.ac3
05/09/2011  20:43         1,959,936 13 PID 1100 3_2ch 48KHz 384Kbps DELAY 0ms.ac3
05/09/2011  20:43         1,889,280 14 PID 1100 3_2ch 48KHz 384Kbps DELAY 32ms.ac3
05/09/2011  20:43         3,479,040 15 PID 1100 3_2ch 48KHz 384Kbps DELAY 32ms.ac3
05/09/2011  20:43         2,188,800 2 PID 1100 3_2ch 48KHz 384Kbps DELAY 32ms.ac3
05/09/2011  20:43         4,769,280 3 PID 1100 3_2ch 48KHz 384Kbps DELAY 32ms.ac3
05/09/2011  20:43        11,105,280 4 PID 1100 3_2ch 48KHz 384Kbps DELAY 32ms.ac3
05/09/2011  20:43         4,654,080 5 PID 1100 3_2ch 48KHz 384Kbps DELAY 32ms.ac3
05/09/2011  20:43         1,451,520 6 PID 1100 3_2ch 48KHz 384Kbps DELAY 32ms.ac3
05/09/2011  20:43         1,497,600 7 PID 1100 3_2ch 48KHz 384Kbps DELAY 32ms.ac3
05/09/2011  20:43         6,153,216 8 PID 1100 3_2ch 48KHz 384Kbps DELAY 0ms.ac3
05/09/2011  20:43         9,008,640 9 PID 1100 3_2ch 48KHz 384Kbps DELAY 32ms.ac3

There's no noticeable lipsync issues at all.

Rgds
T

[admin]

Not all situations are as clean as yours. I could write the delay into the DGI file. OK, I'll do that and give you a CLI option for the audio filename. Will that be sufficient for you?

[TiberiusT]

Indeed. Checking other files I've noticed a max of 32ms.

Will that be sufficient for you?

Sufficient is an understatment. That would be truly outstanding bespoke support.

Rgds
T

[phenoboy]

Hi TiberiusT,

I'd be interested in your script. I have huge amounts of .mts files which I'd like to encode to x264 using NV Decoder. I was about to start do the scripting - but then I found this thread. I can see you have it pretty much handled.

Would you be willing to share your latest version of the script ?

[admin]

I put a test version here:

http://rationalqm.us/misc/TiberiusT.zip

Use -ad instead of -a to use filenames based on the DGI filename.

Please let me know if this is working for you.

[phenoboy]

-ad option seems to working just great.

does the delay mean that video is ahead of audio ? where does it actually come from ?

[admin]

Thank you for your test results.

The delay arises because demuxing elementary streams may not hit the same PTS values for the audio and video at that point in the stream. There is no guarantee in a stream that access units for audio and video with the same presentation times will be close together. So, for example, if we demux and the first video PTS is 7 and the first audio PTS is 3, then if we start playing both together immediately, we will have the video playing too early relative to the audio, so we need to move up the audio in time by -(7 - 3) = -4. So, a positive delay means you have to delay the audio, and a negative delay means you have to advance the audio.

[phenoboy]

EDIT: big thanks for explaining the delay. I need to add proper delay option for mp4box which is missing from code below. Actually delays seem to be so minimal in my case they don't matter.

Just did my first version of batch script for converting AVCHD's from Canon legria HF200. It's very nice to have CLI with dgindexnv.exe.

create.bat

Code: Select all

for %%s in (*.mts) do dgindexnv.exe -i "%%s" -o %%~ns.dgi -ad -e -at f:\template.avs
for %%s in (*.avs) do "c:\program files (x86)\mediacoder\codecs\x264.exe" --crf 20 --preset medium --sar 1:1 --output "%%~ns.h264" "%%s"
for %%s in (*.h264) do "c:\program files (x86)\mediacoder\tools\MP4Box.exe" -add %%s -add %%~ns.ac3 %%~ns.mp4

template.avs

Code: Select all

LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\NicAudio.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecodeNV.dll")

rem use nvidia purevideo with option 2 (double fps with bobbing, very fast and smooth deinterlacing with GPU) and GPU resize to 1280x720
video=DGMultiSource("__vid__",false,2,true,false,false,0,0,0,0,1280,720)
audio=NicAC3Source("__aud__")

AudioDub(video, audio)

Didn't bother to compress audio
I'm getting about 15fps with phenom II X4 905e, gtx 260 when x264 starts encoding - all cores around 100%.

If there's faster way to do this (1080i50 to 720p50 with high quality deinterlacing) - I'm all ears

[admin]

A VP5 engine will probably give a speed bump. I'll be getting one soon for some testing and also to implement selectable CUDA/CUVID for each video type. We know for example that MPEG2 is faster with CUDA on VP4 and below but is faster with CUVID on VP5. Since there is no way to query the VP engine version I thought I would make it configurable.

[TiberiusT]

@admin

Yes! It works perfectly. Thank you very much indeed - it's going to save me an age. I don't see the delay in the log file - but that's not important for me with this project.

I can see you have it pretty much handled

No! The batch workflow and basic archiving is indeed handled for the hundreds of files this camcorder produces every month, but optimal scripting/encoding I don't have handled at all. I can't get "x264 --crf 18 --preset veryslow" below 15000kbps even on moderately still/stabilised footage. On moving footage it can go well above 30000. Any substantially less bitrates using 2pass reveal obvious artifacts. Can't upload for some reason - my dodgy internet or mediafire is having issues.

I'm getting about 15fps with phenom II X4 905e, gtx 260 when x264 starts encoding - all cores around 100%.
If there's faster way to do this (1080i50 to 720p50 with high quality deinterlacing) - I'm all ears

Same graphics card here, but i7 at 4.0Ghz. If your CPU is maxed out and the script contains nothing else than GPU deinterlacing and resizing then yu can't go any faster. I just ran one of my Panny's 1080/50i thru:

Code: Select all

dgsource("D:\Video\Camcorder\Panny\6.dgi" , deinterlace=2, resize_w=1280, resize_h=720)
x264 --crf 20 --medium 

and got

Code: Select all

encoded 1632 frames, 39.85 fps, 8825.30 kb/s

But more importantly for me - it's the bitrate of 8825kbps', which IMHO is very high - what do yu get?






Rgds
T

[admin]

I don't see the delay in the log file

That's because it's at the bottom of the DGI file.

BTW, I'd appreciate you starting a new thread about your bitrates and performance, as it is OT in this thread. Thanks.

[phenoboy]

Same graphics card here, but i7 at 4.0Ghz. If your CPU is maxed out and the script contains nothing else than GPU deinterlacing and resizing then yu can't go any faster. I just ran one of my Panny's 1080/50i thru:

Code: Select all

dgsource("D:\Video\Camcorder\Panny\6.dgi" , deinterlace=2, resize_w=1280, resize_h=720)
x264 --crf 20 --medium 

and got

Code: Select all

encoded 1632 frames, 39.85 fps, 8825.30 kb/s

But more importantly for me - it's the bitrate of 8825kbps', which IMHO is very high - what do yu get?

Yep, my CPU is slow

for me the bitrate is quite high too. Here's one example:

Code: Select all

encoded 11256 frames, 15.11 fps, 10687.58 kb/s

[admin]

After further review I have decided not to release this feature.

The problem is that if the stream contains more than one audio track of the same type (e.g., AC3) then they all try to use the same filename, making a royal mess. I do not want to kludge things up any further in support of this. You'll have to do it in your batch file(s).

[laserfan]

The delay arises because demuxing elementary streams may not hit the same PTS values for the audio and video at that point in the stream. There is no guarantee in a stream that access units for audio and video with the same presentation times will be close together. So, for example, if we demux and the first video PTS is 7 and the first audio PTS is 3, then if we start playing both together immediately, we will have the video playing too early relative to the audio, so we need to move up the audio in time by -(7 - 3) = -4. So, a positive delay means you have to delay the audio, and a negative delay means you have to advance the audio.

TGIF to you sir! Today I needed to understand DELAY better, and found nothing in the DGIndexNV manual of value, but fortunately recalled this post. May I suggest you consider adding it to the manual, perhaps in the Audio Demux section.

Too good to leave hidden here IMO!

[admin]

laserfan, your wish is my command. I will add it as you suggest.

[laserfan]

laserfan, your wish is my command. I will add it as you suggest.

Must...resist...this feeling....of power...

Thank you from this delighted customer!

[admin]

I've slipstreamed into 2041 the DGIndexNV manual with the note about delays added.

[laserfan]

I've slipstreamed into 2041 the DGIndexNV manual with the note about delays added.

Ah yes, and I see that it is a thing of beauty:

Demuxed audio files will be named after the project name and additional parameters of the audio format. For example: "arte.hd PID 083 L2 2ch 48 128 DELAY -27ms.mp2".

The DELAY value reported is the required adjustment for the demuxed audio elementary stream to retain sync with the video. As it is not obvious why this delay exists and how to interpret the sign of the number, I offer the following explanation. The delay arises because demuxing elementary streams may not hit the same PTS (Presentation Time Stamp) values for the audio and video at the point in the stream where demuxing begins. (The PTS is a number that says when that audio or video access unit is to be displayed.) There is no guarantee in a stream that access units for audio and video with the same presentation times will be close together. So, for example, if we demux and find that the first video PTS is 7 and the first audio PTS is 3, then if we start playing both together immediately, we will have the video playing too early relative to the audio, so we need to move up the audio in time by -(7 - 3) = -4. So, a positive delay means you have to delay the audio, and a negative delay means you have to advance the audio.

I'm tempted now to chide you about renaming these slipstreamed versions, e.g. I've saved the latest file as degdecnv2041c.zip, but you probably have good reason, so I'll not press my luck! Thanks again!

[admin]

I do it on purpose. Learn to love it.

[TiberiusT]

After further review I have decided not to release this feature.

The problem is that if the stream contains more than one audio track of the same type (e.g., AC3) then they all try to use the same filename, making a royal mess. I do not want to kludge things up any further in support of this. You'll have to do it in your batch file(s).

I don't quite understand. Is this not implemented as an optional CL switch (–ad) only? So the user actively has to enable it in CLI mode? I would have thought that for CLI users it’s obvious that auto outputting a single matched audio filename is not going to work with anything more than 1 audio track and would indeed create a right royal mess. A warning to that effect could be added in the help text. I'm guessing that other camcorders output similar transport streams and given camcorders propensity to create hundreds of small files this could be a useful option to many Avisynth users. DGSource walked thru these 1080/50p files - everything else I tried borked somewhere.

Anyway, far be it from myself to suggest the way you implement features in your excellent program. I've caught up with my workload, I used the test version u uploaded and have processed over 500 files without a single error. There's no detectable audio sync issues anywhere. I shall continue to proudly use my own 'custom edition' DGIndexNV to this effect.

Cheers
T

[mastrboy]

Any reason why the Delay information is not added to WAV/LPCM audio when demuxed?

I have some vob files with LPCM that has this "problem", i tested with version 2040, 2041 and even good old DGIndex, and none of them adds the delay information...

Demuxed filename: VTS_01_1.VOB Ta0 48K 16bit 2ch.wav
but should be: VTS_01_1.VOB Ta0 48K 16bit 2ch Delay -64ms.wav

Mediainfo from the vob file:

Code: Select all

Audio
ID                               : 189 (0xBD)-160 (0xA0)
Format                           : PCM
Format settings, Endianness      : Big
Format settings, Sign            : Signed
Muxing mode                      : DVD-Video
Duration                         : 1mn 31s
Bit rate mode                    : Constant
Bit rate                         : 1 536 Kbps
Channel(s)                       : 2 channels
Channel positions                : Front: L R
Sampling rate                    : 48.0 KHz
Bit depth                        : 16 bits
Delay relative to video          : -64ms
Stream size                      : 16.8 MiB (21%)