Using Runtime.exec(), mencoder and cfthread

UPDATE: Finally I discovered how to successfully use mencoder or ffmpeg with cfexecute. You can find an example in Cfexecute, ffmpeg and mencoder - Mystery Solved. Though personally I still prefer using Runtime.exec(). The reason being it allows you to kill a process if it has run too long. As far as I know, you cannot do that with cfexecute.

In the entry using FFMPEG to convert video files to FLV format I talked about using Runtime.exec() from ColdFusion. Recently a poster on the abobe.com forums mentioned they tried to use this technique with mencoder, but it failed on the windows o/s and cfexecute did not work either. Though they did mention they eventually got it working by using cfexecute on Ubuntu.

But their comment got me to wondering why the technique worked with ffmpeg.exe but not with mencoder.exe. So I went back to the javaworld.com article about pitfalls of working with Runtime.exec(). I downloaded the sample code, changed the command string to use mencoder instead, and it worked perfectly. So why did it fail when called from ColdFusion?

When I reviewed the output I noticed mencoder appeared to write alternately to the output and error streams. Note, I am deliberately using parameters that write to the error stream.

OUTPUT>Quicktime/MOV file format detected.
OUTPUT>[mov] Video stream found, -vid 0
OUTPUT>VIDEO: [SVQ1] 320x240 24bpp 15.000 fps 0.0 kbps ( 0.0 kbyte/s)
OUTPUT>[V] filefmt:7 fourcc:0x31515653 size:320x240 fps:15.00 ftime:=0.0667
ERROR>** MUXER_LAVF *****************************************************************
ERROR>REMEMBER: MEncoder's libavformat muxing is presently broken and can generate
ERROR>INCORRECT files in the presence of B frames. Moreover, due to bugs MPlayer
ERROR>will play these INCORRECT files as if nothing were wrong!
ERROR>*******************************************************************************
OUTPUT>OK, exit
OUTPUT>Opening video filter: [expand osd=1]
...

That is when I realized you must use separate threads to process the streams. Otherwise they may still block or deadlock. Originally I thought this was optional, because ffmpeg seemed to work perfectly without it. But I think the reason it worked for ffmpeg is that all of its output went to one stream: the error stream (Strange, I know). Since the CF code read the error stream first, there was no blocking. As mencoder is writing to both streams alternately, that explains why processing the streams sequentially did not work.

Enter cfthread

So I took the code in my previous entry and modified it to use cfthread to process the streams. (Though I could also have used the StreamGobbler.class, from the javaworld.com example.)

Since I want to wait for the threads to finish, I first defined a timeout limit. In this example the code will wait 90 seconds for the two threads to finish, but you can change the value as needed. The "commandString" variable just contains a call to mencoder with a few parameters. The commandString and ProcessStream function are shown at the end of this entry.

<!--- do not wait more than 90 seconds for the threads to complete --->
<cfset millisecondsToWait = 90 * 1000>
...
<cfset runtime = createObject("java", "java.lang.Runtime").getRuntime()>
<cfset process = runtime.exec(commandString)>

<!--- process output stream in one thread --->
<cfthread action="run" name="processOutput">
  <cfset ProcessStream( process.getInputStream(), outLogPath )>
</cfthread>
<!--- process error stream in another thread --->
<cfthread action="run" name="processError">
  <cfset ProcessStream( process.getErrorStream(), errorLogPath )>
</cfthread>

I then used "join" to wait for the threads to finish, or until the allotted time has elapsed.

<!--- wait for the threads to complete --->
<cfthread action="join" name="processOutput,processError" timeout="#millisecondsToWait#" />

Next, I check the status of the two threads. If they have not finished processing within the given amount of time, I terminate them rather than letting them continue to process.

<cfset wasTerminated = false>
    
<!--- kill output thread if not completed in allotted time --->
<cfif cfthread.processOutput.Status IS "RUNNING" OR cfthread.processOutput.Status IS "NOT_STARTED">
  <cfthread action="terminate" name="processOutput" />
  <cfset wasTerminated = true>
</cfif>

<!--- kill error thread if not completed in allotted time --->
<cfif cfthread.processError.Status IS "RUNNING" OR cfthread.processError.Status IS "NOT_STARTED">
  <cfthread action="terminate" name="processError" />
  <cfset wasTerminated = true>
</cfif>

<!--- give the termination a chance to complete --->
<cfset sleep(500) >

If the two threads were terminated, I also terminate the parent process. Using the Server Monitor Tool, I discovered the child threads would continue to run unless I terminated the parent process as well.

<!--- if the threads were terminated, forcibly kill the parent process --->
<cfif wasTerminated>
  <cfset process.destroy()>
</cfif>

At this point the parent process should have completed normally or been forcibly terminated. So I obtain the final exit status by calling process.waitFor(). Technically I could call exitValue() instead, but I prefer to use waitFor(). Finally I use cfdump to display the results.

<!--- record the final status of the process --->
<cfset results.exitValue = process.waitFor()>
<cfset results.output = duplicate(cfthread.processOutput)>
<cfset results.error = duplicate(cfthread.processError)>

<!--- display results --->  
<cfdump var="#results#">

As they say "that's all she wrote". These are just my observations using Runtime.exec() on windows o/s. Your results may differ. As always, any comments, corrections or suggestions are welcome :)

Complete Code

The parameters in the commandString below were selected specifically because they generate a warning message. They are not intended as an example of how to use mencoder! For proper usage information you should consult the mencoder documentation ;)

<cfset mencoderPath = "c:\bin\MPlayer-1.0rc2\mencoder.exe">
<cfset inputFilePath = "c:\bin\input\tenMegFile.mp4">
<cfset ouputFilePath = "c:\bin\temp\tenMegFile.flv">
<cfset outLogPath = "c:\bin\temp\tenMegFile_result.log">
<cfset errorLogPath = "c:\bin\temp\tenMegFile_error.log">
<!--- do not wait more than 90 seconds for the threads to complete --->
<cfset millisecondsToWait = 90 * 1000>

<cfset results = structNew()>
<cftry>
  <!--- WARNING there is little rhyme or reason for these particular parameters --->
  <!--- Check the mencoder documenation for proper usage information --->
  <cfset commandString = mencoderPath &" "& inputFilePath & " -o "& ouputFilePath >
  <cfset commandString = commandString &" -quiet -of lavf -oac mp3lame -lameopts abr:br=56 -srate 22050 -ovc lavc">
  <cfset commandString = commandString &" -lavcopts vcodec=flv:vbitrate=500:mbd=2:mv0:trell:v4mv:cbp:last_pred=3">

<!--- do not wait more than 90 seconds for the threads to complete --->
<cfset millisecondsToWait = 90 * 1000>
  <cfset runtime = createObject("java", "java.lang.Runtime").getRuntime()>
  <cfset process = runtime.exec(commandString)>

  <!--- process output stream in one thread --->
  <cfthread action="run" name="processOutput">
      <cfset ProcessStream( process.getInputStream(), outLogPath )>
  </cfthread>
  <!--- process error stream in another thread --->
  <cfthread action="run" name="processError">
      <cfset ProcessStream( process.getErrorStream(), errorLogPath )>
  </cfthread>

  <!--- wait for the threads to complete --->
  <cfthread action="join" name="processOutput,processError" timeout="#millisecondsToWait#" />

  <cfset wasTerminated = false>
    
  <!--- kill output thread if not completed in allotted time --->
   <cfif cfthread.processOutput.Status IS "RUNNING" OR cfthread.processOutput.Status IS "NOT_STARTED">
      <cfthread action="terminate" name="processOutput" />
      <cfset wasTerminated = true>
  </cfif>

  <!--- kill error thread if not completed in allotted time --->
   <cfif cfthread.processError.Status IS "RUNNING" OR cfthread.processError.Status IS "NOT_STARTED">
      <cfthread action="terminate" name="processError" />
      <cfset wasTerminated = true>
  </cfif>

  <!--- Wait to make ensure the termination completes --->
  <cfset sleep(500) >

  <!--- if the threads were terminated, forcibly kill the parent process --->
  <cfif wasTerminated>
      <cfset process.destroy()>
  </cfif>

  <!--- record the final status of the process --->
  <cfset results.exitValue = process.waitFor()>
  <cfset results.output = duplicate(cfthread.processOutput)>
  <cfset results.error = duplicate(cfthread.processError)>

  <cfcatch>
      <cfdump var="#cfcatch#">  
  </cfcatch>
</cftry>

<!--- show the results --->  
<cfdump var="#results#">

ProcessStream Function

<cffunction name="processStream" access="public" output="true" returntype="void"
  hint="Used to drain input/output streams of Runtime.exec(). Optionally write the stream to a file">

  <cfargument name="in" type="any" required="true" hint="java.io.InputStream object">
  <cfargument name="logPath" type="string" required="false" default="" hint="Full path to log file">
  <cfset var Local = structNew()>

  <cfscript>
      Local.sendToLogFile = false;
    
      // if the results should be logged, create a writer to generate the output file
      if ( len(trim(arguments.logPath)) ) {
          Local.out = createObject("java", "java.io.FileOutputStream").init( arguments.logPath );
          Local.writer = createObject("java", "java.io.PrintWriter").init( Local.out );
          Local.sendToLogFile = true;
      }

      Local.reader = createObject("java", "java.io.InputStreamReader").init( arguments.in );
      Local.buffered = createObject("java", "java.io.BufferedReader").init( Local.reader );
      Local.line = Local.buffered.readLine();

      while ( structKeyExists(Local, "line") ) {
          // if logged, write each line of the stream to the output file
          if (Local.sendToLogFile) {
              Local.writer.println( Local.line );
          }
          Local.line = Local.buffered.readLine();
      }  

      // if logged, close the output file when finished
      if (Local.sendToLogFile) {
         Local.writer.flush();
         Local.writer.close();
      }
  </cfscript>
</cffunction>