Thursday, October 7, 2010

ColdFusion 9.0.1: Expand / Collapse All Groups in a CFGrid

Being late to the game, I just read about CF9's grouping feature with CFGrids. While it is a useful feature, I immediately noticed there was no option to expand or collapse all groups. After perusing the Ext forums, I discovered methods for expanding or collapsing all groups in the underlying GridView class. With the help of ajaxOnLoad(), I was able to create a grouped CFGrid, but this time with all groups collapsed by default.


<script type="text/javascript">
    function collapseGrid() {
        var grid = ColdFusion.Grid.getGridObject('MyGrid');
        grid.getView().collapseAllGroups();
    }
</script>

... create the grouped cfgrid ...

<cfset ajaxOnLoad("collapseGrid")>

Giddy with my small success, I decided to try creating a button that would Expand/Collapse all groups. Fortunately a great entry on Dan Vega's blog saved me from hurting myself with my rudimentary Ext skills. In no time at all, I had a new toolbar with a button.

Now since I wanted to pass the grid id into the handler function, I used createCallback() to bind the grid id as a function argument. Voila! A fully functional (albeit plain) toolbar button that toggles the expansion state of all groups.

If you are interested in styling the button, check out ColdFusion 8 Grid Custom Toolbars for an example.

Full Code (Requires CF 9.0.1)
<!--- sample data for grid --->
<cfset qGridData = queryNew("ID,Title,Area", "integer,varchar,varchar") />
<cfloop from="1" to="50" index="r">
    <cfset row = queryAddRow(qGridData, 1)>
    <cfset qGridData.ID[row] = r>
    <cfset qGridData.Title[row] = "Title "& numberFormat(r, "000")>
    <cfset qGridData.Area[row] = ceiling(r /10)>
</cfloop>    

<html>
<head>
<script type="text/javascript">
    function init(){
        // create a toolbar
        var tbar = ColdFusion.Grid.getTopToolbar('MyGrid');
        tbar.addButton({xtype: 'tbfill'});
        // add a button that expands/collapses all groups
        tbar.addButton({ text: "Expand/Collapse All", 
                tooltip: "Toggles the expansion state of all groups", 
                handler: toggleGridGroups.createCallback('MyGrid')
            });
        //show the toolbar 
        ColdFusion.Grid.showTopToolbar('MyGrid');

        //collapse all groups by default 
        var grid = ColdFusion.Grid.getGridObject('MyGrid');
        grid.getView().collapseAllGroups();
    }

    function toggleGridGroups(gridId){
        var grid = ColdFusion.Grid.getGridObject(gridId);
        grid.getView().toggleAllGroups();
    }
</script>
</head>
<body>
    <cfform format="html">
        <cfgrid name="MyGrid" format="html" query="qGridData" groupField="Area" width="500">
            <cfgridcolumn name="Title" header="Title" />
            <cfgridcolumn name="Area" header="Title Area" />
        </cfgrid>
    </cfform>
    <cfset ajaxOnLoad("init")>
</body>
</html>

...Read More
Posted by cfSearching at 10:18 PM ::  0 comments
Labels:

Sunday, September 19, 2010

CFHTTP + FILE + POST (Bug Byte ?)

A few years ago, I saw a post on the adobe forums about a strange problem with cfhttp. Posting a simple .zip file with <cfhttp> somehow ended up corrupting the file. So the resulting archive file could not be read by some tools, in particular <cfzip>. 


<!---
   With CF8 this code fails with the error:
   Unsupported File Format
   Ensure that the file is a valid zip file and it is accessible.
--->
<cfif structKeyExists(FORM, "myFile")>
    <cffile action="upload" destination="#ExpandPath('.')#" nameconflict="overwrite" />
    <cfzip action="list" file="#cffile.serverDirectory#/#cffile.serverFile#" name="result" />
    <cfdump var="#result#" label="Zip File Contents" />
<cfelse>
    <cfhttp url="http://127.0.0.1/cfhttpTest.cfm" method="post">
        <cfhttpparam name="myFile" type="file" file="c:/test/testFile.zip" mimetype="application/zip" />
    </cfhttp>
    <cfoutput>#cfhttp.fileContent#</cfoutput>
</cfif>

The Byte
So I decided to have a look and noticed something strange about the received file. It always seemed to be two (2) bytes bigger than the original. After a bit more experimentation, I discovered the problem disappears if you add a formfield (any formfield) directly after the file.  Placing it before the file does not work.

<!---
   This DOES work
--->
<cfif structKeyExists(FORM, "myFile")>
    <cffile action="upload" destination="#ExpandPath('.')#" nameconflict="overwrite" />
    <cfzip action="list" file="#cffile.serverDirectory#/#cffile.serverFile#" name="result" />
    <cfdump var="#result#" label="Zip File Contents" />
<cfelse>
    <cfhttp url="http://127.0.0.1/cfhttpTest.cfm" method="post">
        <cfhttpparam name="myFile" type="file" file="c:/test/testFile.zip" mimetype="application/zip" />
        <cfhttpparam name="BecauseTheFileIsCorruptedWithoutThisField" type="formfield" value="" />
    </cfhttp>
    <cfoutput>#cfhttp.fileContent#</cfoutput>
</cfif>

Now I just assumed it was an issue with cfhttp. As there was a work-around, I did not really explore it further. But a recent question on stackoverflow.com prompted me to revisit the issue and take a closer look. A poster named Kip mentioned something important I had missed the first time around. That extra two (2) bytes was a newline! So armed with that key piece of information, I decided to use Fiddler to see what was happening within the cfhttp request.

The Pest
If you are unfamiliar with multipart submissions, the w3c describes them as a ".. message contain(ing) a series of parts, each representing a successful control." In loose terms, a successful control is just a form field. (There is a little more to it than that.  But the details are not relevant to this particular case.) When a multipart form is submitted, the names and values of the various fields are submitted as parts, delineated by boundary markers.

Now in this case, there is only one (1) form field. But as you can see from the image below, the information is nested within boundary markers. Obviously the request contents are separated by a newline, at various points. But notice there is an extra newline just before the closing boundary marker? That is the extra two (2) bytes. Given that <cfhttp> is responsible for generating the request content, and the separating newlines, it certainly seemed like a problem <cfhttp>



Next I submitted the same file manually with a regular <form> post.  The results from Fiddler showed there was no extra newline in the content.

So I went back and tested the extra formfield hack and surprise, surprise.. that pesky newline was gone.


Another poster, Sergii, mentioned this issue does not occur with Railo 3.1.2. So I tested the same code under Railo. Sure enough the resulting archive was fine, no corruption. Fiddler confirmed there were no pesky newline problems when using Railo's <cfhttp>. That clinched it for me. I would say this is an ACF bug with <cfhttp> in CF8 and CF9. So if you ever experience something similar, now you know why.

(Note: I do not know if this issue applies to 9.0.1)

...Read More
Posted by cfSearching at 1:26 AM ::  0 comments
Labels:

Tuesday, September 7, 2010

My First Look at Diffie-Hellman (Merkle) Key Exchange - Part 2

After working through the basic formulas in Part 1, I felt I had a decent grasp of the overall process. So I decided to dive into a java example.  From what I have read, there are different implementations. So I looked over a few java examples before choosing what I felt was the simplest: the one from the (Sun) Java Cryptography Extension (JCE) Reference Guide. Make no mistake, there is definitely a lot more involved in establishing secure exchanges than is covered in the example. But as with most things, understanding the concepts and what the code is doing is a good start.


So when I first looked at the java code, I was a bit .. flummoxed. I honestly did not recognize much of anything, other than "DH". I suppose I naively expected I might see objects initialized with secret values, calculations of prime numbers and primitive roots. Ye-ah. Needless to say, I did not find anything like that. At least not in the basic "intro" example.

The more I thought about it, I realized how silly that would be. Remember I mentioned typical exchanges involve really, really big numbers? Well imagine having to calculate really large prime numbers or primitive roots (on your own) in order to use the library? Not fun. After reviewing the code more closely, I realized the SunJCE does indeed provide a class that does the grunt work for you.

First an AlgorithmParameterGenerator is used to generate the initial parameters. In other words the prime number and primitive root agreed upon by Alice and Bob. In this example it is initialized with a key size 512 bits. A real exchange would probably use a larger key like 1024 bits.

As you can imagine, generating the values is an expensive operation and may take a few seconds. But when finished, the generator will return a DHParameterSpec object. It is simply an object containing the settings used for the exchange: prime number (p), primitive root (g) and key size (l). To view the values, use the methods getG(), getP() and getL().

<cfset generator = createObject("java", "java.security.AlgorithmParameterGenerator").getInstance("DH") />
<!--- Intialize the generator to create a 512 bit key pair (Testing only) --->
<!--- This is an expensive operation and may take a while --->
<cfset generator.init( javacast("int", 512) ) />
<cfset params = generator.generateParameters() />
<!--- Convert the parameters to the right type of object --->
<cfset DHParameterSpec = createObject("java", "javax.crypto.spec.DHParameterSpec")/>
<cfset parameterSpec   = params.getParameterSpec(DHParameterSpec.getClass()) />

Next the code passes those parameters into a KeyPairGenerator . The generator returns a KeyPairobject which contains Alice's private and public keys. While so far things may not seem very familiar, the generator is actually doing the same thing we did in Part 1. Except it automatically selects Alice's secret number, and calculates her public number internally using the given parameters (ie prime number and primitive root).

<cfset KeyPairGenerator = createObject("java", "java.security.KeyPairGenerator") />
<cfset alicesKeyPairGenerator = KeyPairGenerator.getInstance("DH") />
<cfset alicesKeyPairGenerator.initialize( parameterSpec ) />
<cfset alicesKeyPair     = alicesKeyPairGenerator.generateKeyPair() />

How do you know this? Well if you display Alice's keys, you will see several parameter names followed by a long string of hexadecimal. Those values are just very large numbers, encoded as hex. The "x" represents Alice's private number and the "y" her public number. The "p" and "g" are our prime number and primitive root values. Again, encoded as hex.

Starting to seem familiar now? 




<cfoutput>
<strong>Alice's Values</strong>
<pre>
    Format    = #alicesKeyPair.getPublic().getFormat()#
    Base (g)  = #alicesKeyPair.getPublic().getParams().getG()#
    Prime (p) = #alicesKeyPair.getPublic().getParams().getP()#
</pre>    

<pre>    
#alicesKeyPair.getPrivate()#
</pre>    
<pre>    
#alicesKeyPair.getPublic()#
</pre>
</cfoutput>

Before Alice sends Bob her public number, she creates a KeyAgreement. Basically this object will be used to complete the final step: calculating the shared key. So it is first initialized with Alice's private key. Then Alice sends her public key off to Bob, and waits to receive his in return.

<!--- Alice creates and initializes her DH KeyAgreement object with her Private Key --->
<cfset KeyAgreement = createObject("java", "javax.crypto.KeyAgreement").getInstance("DH") />
<cfset alicesKeyAgreement = KeyAgreement.getInstance("DH") />
<cfset alicesKeyAgreement.init( alicesKeyPair.getPrivate() ) />
<!--- Alice encodes her public key, and sends it over to Bob --->
<cfset alicesPublicKeyBytes = alicesKeyPair.getPublic().getEncoded() />

When Bob receives Alice's public key, it is encoded in x509 format. So it has to be decoded it first. Bob then uses that object to create his own keys. Finally, he encodes his public key and sends it back to Alice. 

<!---
    Bob gets the DH parameters associated with Alice's public key. 
    He must use the same parameters when he generates his own key pair.
--->
<cfset dhParamSpec = publicKeyFromAlice.getParams() />

<!--- Bob creates his own DH key pair --->
<cfset KeyPairGenerator = createObject("java", "java.security.KeyPairGenerator") />
<cfset bobsKeyPairGenerator = KeyPairGenerator.getInstance("DH") />
<cfset bobsKeyPairGenerator.initialize( dhParamSpec ) />
<cfset bobsKeyPair = bobsKeyPairGenerator.generateKeyPair() />
<!--- Bob encodes his public key, and sends it over to Alice --->
<cfset bobsPublicKeyBytes = bobsKeyPair.getPublic().getEncoded() />

Just like Alice's values, Bob's keys will be encoded as hex.

<cfoutput>
<strong>Bob's Values</strong>
<pre>
    Base (g)  = #dhParamSpec.getG()#
    Prime (p) = #dhParamSpec.getP()#

</pre>
<pre>
#bobsKeyPair.getPrivate()#
</pre>

<pre>
#bobsKeyPair.getPublic()#
</pre>
</cfoutput>

Bob is now ready to calculate the shared key. So he too creates a KeyAgreement, and initializes it with his private key. Finally he plugs in Alice's public key and calculates the shared value. Now internally this class is using the same formulas as we did in Part 1. But it is all done for you auto-magically.

<!--- Bob creates his DH KeyAgreement and initializes it with his private key --->
<cfset bobsKeyAgreement = createObject("java", "javax.crypto.KeyAgreement").getInstance("DH") />
<cfset bobsKeyAgreement.init( bobsKeyPair.getPrivate() ) />
<cfset bobsKeyAgreement.doPhase(publicKeyFromAlice, true) />
<cfset bobsSharedSecret = bobsKeyAgreement.generateSecret() />
<cfset bobsSharedSecretAsHex = binaryEncode(bobsSharedSecret, "hex") />

Once Alice receives Bob's public key, she decodes it and plugs the value into her KeyAgreement object. Then calculates the shared value.

<cfset alicesKeyFactory = createObject("java", "java.security.KeyFactory").getInstance("DH") />
<cfset X509EncodedKeySpec = createObject("java", "java.security.spec.X509EncodedKeySpec") />
<cfset x509KeySpec = X509EncodedKeySpec.init( bobsPublicKeyBytes ) />
<cfset publicKeyFromBob = alicesKeyFactory.generatePublic(x509KeySpec) />
<cfset alicesKeyAgreement.doPhase( publicKeyFromBob, true ) />
<cfset alicesSharedSecret = alicesKeyAgreement.generateSecret() />
<cfset alicesSharedSecretAsHex = binaryEncode(alicesSharedSecret, "hex") />

Thanks to the wonders of mathematics, Alice and Bob both arrive at the same shared value.


<cfset areSecretsTheSame = alicesSharedSecretAsHex eq bobsSharedSecretAsHex />
<cfoutput>
<strong>Are Alice and Bob's secrets the same?</strong> #areSecretsTheSame#

<p class="result">alicesSharedSecretAsHex</p>
<p>#alicesSharedSecretAsHex#</p>

<p class="result">bobsSharedSecretAsHex</p>
<p>#bobsSharedSecretAsHex#</p>
</cfoutput>

By itself, the shared value cannot be used for encryption. However Alice and Bob can use the shared value to generate a secret key which can be used for encryption.

The java documentation includes a simple example using DES. Obviously outdated, but it does demonstrate that Alice and Bob can successfully can encrypt/decrypt each other's values, starting only with the shared key.


Bob Encrypts
<!--- 
     The previous call to bobsKeyAgreement.generateSecret resets the key
     agreement object, so we call doPhase again prior to another generateSecret call
--->
<cfset bobsKeyAgreement.doPhase(publicKeyFromAlice, true) />
<cfset bobsDESKey         = bobsKeyAgreement.generateSecret("DES") />

<!--- Encrypt the text with a simple encryption --->
<cfset Cipher               = createObject("java", "javax.crypto.Cipher") />
<cfset bobsCipher          = Cipher.getInstance("DES/ECB/NoPadding") />
<cfset bobsCipher.init( Cipher.ENCRYPT_MODE, bobsDESKey ) />
<cfset textToEncrypt      = "StandAndDeliver!" />
<cfset bytesToEncrypt       = CharsetDecode(textToEncrypt, "utf8") />
<cfset encryptedBytes       = bobsCipher.doFinal(bytesToEncrypt) />
<cfset encryptedText     = BinaryEncode(encryptedBytes, "hex")  />

<!--- results --->
<strong>Bob's Values:</strong>
<cfoutput>
<p><strong>DES Key</strong>  : #BinaryEncode(bobsDESKey.getEncoded(), "hex")# </p>
<p><strong>Encrypted Text</strong>  : #encryptedText# </p>
</cfoutput>

Alice Decrypts
<!--- 
     The previous call to bobsKeyAgreement.generateSecret resets the key
     agreement object, so we call doPhase again prior to another generateSecret call
--->
<cfset alicesKeyAgreement.doPhase(publicKeyFromBob, true) />
<cfset alicesDESKey = alicesKeyAgreement.generateSecret("DES") />
<cfset Cipher               = createObject("java", "javax.crypto.Cipher") />
<cfset alicesCipher         = Cipher.getInstance("DES/ECB/NoPadding") />
<cfset alicesCipher.init( Cipher.DECRYPT_MODE, alicesDESKey ) />
<cfset bytesToDecrypt     = BinaryDecode(encryptedText, "hex") />
<cfset unencryptedBytes  = alicesCipher.doFinal( bytesToDecrypt ) />

<!--- results --->
<strong>Alice's Values:</strong>
<cfoutput>
<p><strong>DES Key</strong>  : #BinaryEncode(alicesDESKey.getEncoded(), "hex")# </p>
<p><strong>Unencrypted text</strong>  : #CharsetEncode(unencryptedBytes, "utf8")# </p>
</cfoutput>

Obviously there is a lot more to consider before entering into this type of exchange. Guarding against man-in-the-middle attacks is definitely one issue. So you will want to do a lot more reading first. But hopefully this entry helped de-mystify Diffie-Hellman key exchanges ... a little.

As always, any comments or corrections are welcome!

...Read More
Posted by cfSearching at 10:52 PM ::  0 comments
Labels: ,

My First Look at Diffie-Hellman (Merkle) Key Exchange - Part 1

While I have seen references to Diffie-Hellman before, I honestly knew very little about it until this week. After seeing it mentioned on stackoverflow.com, I decided to do some research. Now I am pretty sure this protocol is not available in the standard edition of Adobe ColdFusion, and contrary to popular opinion, I have my doubts about its availability in Enterprise version as well. Though admittedly, I could not find any solid references one way or the other. So I could be wrong. Anyway, since there is a plethora of implementations in the java world, I decided to explore that route.


Disclaimers
First let me say this entry is not intended to be a "how to guide". In the world of encryption, I am most definitely a novice. But I am a curious novice. So in an effort to better understand the Diffie-Hellman (Merkle) exchange, I decided to take one of the simpler java implementations, deconstruct it, and put the results in a CF context.  If you are already familiar with it, this beginner level entry will probably be one big yawn. But any corrections or clarifications are definitely welcome.  Just go easy lest my brain implode.

What is it?
Wikipedia describes Diffie-Hellman as a key exchange protocol

"...that allows two parties that have no prior knowledge of each other to jointly establish a shared secret key over an insecure communications channel. This key can then be used to encrypt subsequent communications using a symmetric key cipher."

My novice interpretation would be that instead of exchanging a key, two parties exchange "other", transitory, pieces of information instead. Then use those "other" pieces of information to derive the key (independently) without actually sending the key itself over an open channel.

What are those "other" pieces of information? In short, they are really, really large numbers. The protocol uses several numbers in a series of simple mathematical formulas to eventually calculate the secret key. I will not go into details about how and why those numbers are selected. The wikipedia entry describes it far better than I ever could. But suffice it to say they are not arbitrary. Each has specific characteristics that directly affect both the results and the security of the exchange. So I would highly recommend you read the article.  But to paraphrase the salient points (without formulas):
  1. Two parties (Alice and Bob) agree upon two (2) numbers to be used in their calculations (a prime number and a primitive root) Note: These are public values, known by both Alice and Bob
  2. Then Alice and Bob each pick a private number. Note: Alice should not know Bob's value, and Bob should not know Alice's value.
  3. Using their private numbers (plus the prime and primitive root) Alice and Bob each calculate a public number. They then exchange public numbers with each other. Note: These are public values, known to both Alice and Bob.
  4. After exchanging public values, Alice and Bob now have enough information to calculate the shared secret key (using another formula) Note: They both arrive at the same secret key value, independently, without ever sending that value over an open channel.


Math 101
Now jumping straight into a java example from here felt a bit like sending a student driver onto a five lane highway after receiving only five minutes of instruction. So I decided to test the basic formulas from the wikipedia example first.  It gave me a better understanding of the overall process, and also provided a good basis of comparison for the java results.

Now before someone corrects me, the overview below is conceptual only.  When I finally did run the simple java example, the actual steps were slightly different. But overall the process was the same.

(On a side note, this whole thing felt a bit like something you would read about in a spy novel. But I suppose that cannot be helped...)

Step 1) Preparing for the Meeting
First, a prime number and primitive root are selected. These are considered public values, known to both Alice and Bob.   Now you may notice I am using java objects. That was necessary because the calculations involved produce very large numbers. Even using small test values like 23 and 5 the results exceeded the capacity of a basic CF integer.

<cfset prime = createObject("java", "java.math.BigInteger").init( 23 ) />
<cfset base = createObject("java", "java.math.BigInteger").init( 5 ) />
<strong>Values known to both Alice and Bob</strong>
<cfoutput>
    <p> ie prime  = #prime# AND  base   = #base# </p>
</cfoutput>

Step 2) Secret Code Words
Next, Alice and Bob each select a private number, which they do not share with each other. 
<cfset alicesPrivateValue    = createObject("java", "java.math.BigInteger").init( 6 ) />
<cfset bobsPrivateValue        = createObject("java", "java.math.BigInteger").init( 15 ) />

Step 3) The Public Exchange
Alice and Bob then use their private numbers to calculate a public number using the formula: base ^ private MOD prime. They then exchange public numbers. Again, their private numbers are never exchanged.


<cfset alicesPublicValue     = base.modPow( alicesPrivateValue, prime) />
<strong>Alice's values: </strong>
<span class="code">alicesPublicValue = base ^ alicesPrivateValue MOD prime </span>
<cfoutput>
    <p>ie #alicesPublicValue# = #base# ^ #alicesPrivateValue# MOD #prime# </p>
</cfoutput>

<cfset bobsPublicValue     = base.modPow( bobsPrivateValue, prime) />
<strong>Bob's values: </strong>
<div class="code">bobsPublicValue = base ^ bobsPrivateValue MOD prime </div>
<cfoutput>
    <p>ie #bobsPublicValue# = #base# ^ #bobsPrivateValue# MOD #prime#</p>
</cfoutput>

Step 4) Finding the Key
Alice an Bob now have enough information to derive the shared secret value, independently. Alice uses the formula: alicesSharedKey = bobsPublicValue ^ alicesPrivateValue MOD prime .

<strong>Alice uses Bob's value to compute the shared key (B <sup>a</sup> MOD p)</strong>
<div class="code">alicesSharedKey = bobsPublicValue ^ alicesPrivateValue MOD prime </div>
<cfset alicesSharedKey    = bobsPublicValue.modPow(alicesPrivateValue, prime ) />
<cfoutput>
    <p class="result">Alice's shared key is <strong>#alicesSharedKey#</strong></p>
    <p>ie   #alicesSharedKey# = #bobsPublicValue# ^ #alicesPrivateValue# MOD #prime#</p>
</cfoutput>

Whereas Bob uses the formula: bobsSharedKey = alicesPublicValue ^ bobsPrivateValue MOD prime

<strong>Bob uses Alice's value to compute the shared key (A<sup>b</sup> MOD p)</strong><br/>
<div class="code">bobsSharedKey = alicesPublicValue ^ bobsPrivateValue MOD prime </div>
<cfset bobsSharedKey    = alicesPublicValue.modPow( bobsPrivateValue, prime ) />
<cfoutput>
    <p class="result">Bob's shared key: #bobsSharedKey#</p>
    <p>ie   #bobsSharedKey# = #alicesPublicValue# ^ #bobsPrivateValue# MOD #prime#</p>
</cfoutput>

If everything was done correctly, they will both calculate the same value (ie 2). This value can then be used to create a secret key (DES, etcetera) with which to encrypt and decrypt data. 





...Read More
Posted by cfSearching at 3:09 PM ::  4 comments
Labels: ,

Wednesday, August 18, 2010

OT: Random Image from Flickr

I came across this image on flickr. It made me laugh for some reason ...

Typo Poster / Batman 02

;)

...Read More
Posted by cfSearching at 8:26 PM ::  0 comments
Labels: ,

Thursday, May 27, 2010

CF9 Question: Do Spreadsheet Functions Support "Blank Cells"?

I saw a question about spreadsheets this week that had me scratching my head. When you create a new worksheet in Excel, all of the cells are blank. So if you enter a long string in any cell, the text will overflow into the adjacent cell. (If the adjacent cell is blank.)




However, if you do something similar in ColdFusion 9, the text does not overflow into the next cell. Notice how the extra text in cell A1 is hidden, even though the adjacent cell is technically empty.



<cfscript>
    cfSheet = SpreadsheetNew("foo");
    SpreadsheetAddRow(cfSheet, "this text should overflow,,short text", 1);
    SpreadsheetSetColumnWidth(cfSheet, 1, 15); 
    SpreadsheetSetColumnWidth(cfSheet, 2, 15); 
    SpreadsheetSetColumnWidth(cfSheet, 3, 15); 
    
    saveToFile = ExpandPath("test.xls");
    SpreadsheetWrite(cfsheet, saveToFile, true);
    
    // how about a WriteLine() function folks ... ;) ?
    WriteOutput("Saved file: "& saveToFile &"<hr>");
</cfscript>

<cfheader name="Content-Disposition" value="inline; filename=#saveToFile#">
<cfcontent type="application/vnd.ms-excel" file="#saveToFile#" deleteFile="true" />


In the underlying POI library there is a special cell type for blank cells: CELL_TYPE_BLANK. In loose terms, it represents a raw cell that has never had a value. This is different from a cell whose value is set to an empty string. The value may be an empty string, but the cell still has a value. So it has a type of CELL_TYPE_STRING.

If you loop over the test file created earlier, you will see all of the cells are CELL_TYPE_STRING. Which at least explains why the first cell's text does not overflow into the next cell.

<cfscript>
    // open the workbook
    source = ExpandPath("test.xls");
    input  = createObject("java", "java.io.FileInputStream").init( source );
    wb     = createObject("java", "org.apache.poi.ss.usermodel.WorkbookFactory").create( input );
    // get the first sheet
    sheet  = wb.getSheetAt(0);

    // loop over each row in the sheet
    rows = sheet.rowIterator();
    while(rows.hasNext()) {
        // get the current row
        r = rows.next();

        // loop over each cell in the sheet
        cells = r.cellIterator();
        while (cells.hasNext()) {
            // get the current cell
            c = cells.next();

            // check the cell type (ingore error handling for brevity)
            if (c.getCellType() == c.CELL_TYPE_BLANK) {
                type = "CELL_TYPE_BLANK";
            }
            else if (c.getCellType() == c.CELL_TYPE_STRING) {
                type = "CELL_TYPE_STRING";
            }
            else {
                type = "other";
            }

            // display the position, cell type / value
            // note: Adding +1 because POI indexes are 0-based
            WriteOutput("["& r.getRowNum()+1 &"]["& c.getColumnIndex()+1 &"]=");
            WriteOutput(type &" / "& c.toString() &"<hr>");
        }
    }
    // clean up 
    input.close();
</cfscript>

But this raises the question, does ColdFusion 9 support "blank" cells, and if so how do you create one? I am honestly not sure. You could work around it by tapping into the undocumented getWorkBook() method of ColdFusion spreadsheet objects. It returns a reference to the underlying POI workbook. Using the workbook, you can then grab the desired rows and cells and change the cell type to CELL_TYPE_BLANK. 

<cfscript>
    cfSheet = SpreadsheetNew("foo");
    SpreadsheetAddRow(cfSheet, "this text should overflow,,short text", 1);
    SpreadsheetSetColumnWidth(cfSheet, 1, 15); 
    SpreadsheetSetColumnWidth(cfSheet, 2, 15); 
    SpreadsheetSetColumnWidth(cfSheet, 3, 15); 
    
    // work around to make cell B1 a "blank" cell   
    // POI row/cell indexes are 0-based !!
    poiSheet = cfSheet.getWorkBook().getSheet("foo");
    poiCell = poiSheet.getRow( 0 ).getCell( 1 ); 
    poiCell.setCellType( poiCell.CELL_TYPE_BLANK );
    
       
    saveToFile = ExpandPath("newTest.xls");
    SpreadsheetWrite(cfsheet, saveToFile, true);
    WriteOutput("Saved file: "& saveToFile &"<br>");
</cfscript>
                 
<cfheader name="Content-Disposition" value="inline; filename=#saveToFile#">
<cfcontent type="application/vnd.ms-excel" file="#saveToFile#" deleteFile="true" />

However I have not found a documented way to do this. So does anyone know the answer to the million dollar question: does ColdFusion 9 provide a documented method for creating blank cells? My guess would be no. But I would be very happy to be proven wrong. Any spreadsheet function gurus out there?

...Read More
Posted by cfSearching at 10:47 PM ::  3 comments
Labels: ,

Wednesday, May 26, 2010

Experiment Extracting Windows Thumbnails (XP and Windows 2003 only)

Windows thumbnails have existed for eons. But in all that time I never really used the windows explorer thumbnail view. Yes, pictures are nice. But invariably I found myself needing the detail view with its listing of file types, sizes and dates.

Well, this week I came across an old stackoverflow thread which mentioned a C# tool for reading and extracting images from the windows thumbnail cache (ie thumbs.db). At least on older XP and Windows 2003 systems. (Vista and later use a slightly different format.) While there are tools galore in this category, the idea of a small DLL that could be called from CF was appealing. So I decided to give it a quick whirl with CF8, under XP.

After compiling the source with Visual C# Express, I changed my explorer settings so I actually had a thumbnails file to test. Next, I created an instance of the ThumbDB class and initialized it by passing in the path my thumbnails database. Once initialized, I used the GetThumbFiles() method to grab an array of all file names within that database.

<cfset util = createObject(".net", "ThumbDBLib.ThumbDB", "c:/test/ThumbDBLib.dll")>
<cfset util.init( "C:/docs/thumbs.db" )>
<cfset fileNames = util.GetThumbFiles()>
<cfdump var="#fileNames#" label="Top 25 Files in Thumbs.db" top="25">

Next, I selected one of the file names and used the GetThumbData() method to retrieve the image bytes. I was hoping to create a CF image object from the bytes and display it in my browser. But every time I called ImageNew(), ColdFusion kept complaining about my argument type.

<!--- this does NOT work --->
<!---  grabbing an arbitrary file from the listing for testing ...--->
<cfset thumbnailName = fileNames[1] />
<cfset bytes = util.GetThumbData( thumbnailName )>
<cfset img = ImageNew(bytes) />

That is when it hit me. A byte in C# is not the same as byte in Java. Unlike Java, C# has signed and unsigned types. So where C# has two data types, sbyte (-128 to 127) and byte (0 to 255), java has only one, byte (range -128 to 127). So according to the data type conversion matrix, the C# byte array was being converted to a short array.

Thanks to a tip from BlackWasp.com, I added a new method called GetJavaThumbData(), which converts the C# byte array into one that is compatible with Java. Using the new method, I was then able to display the thumbnail perfectly.

C# Code:
public sbyte[] GetJavaThumbData(string fileName)
    {
        byte[] data = GetThumbData(fileName);
        if (data != null)
        {
        sbyte[] jvData  = Array.ConvertAll(data, delegate(byte b) { return unchecked((sbyte)b); });
        return jvData;
        }
        return null;
    }

ColdFusion Code:
<!---  grabbing an arbitrary file from the listing for testing ...--->
<cfset thumbnailName = fileNames[1] />
<cfset bytes = util.GetJavaThumbData( thumbnailName )>
<cfset img = ImageNew( bytes )>
<cfimage action="writeToBrowser" source="#img#">

It is worth noting the ThumbDB class also has a method named GetThumbnailImage(), which returns a C# image object. You can use its properties to display things like height, width, resolution. The class also has several methods for saving the image to a file. Out of curiosity, I tried several of the overloaded save() methods. But I only had success with the simplest form: Image.save(filepath). I am not sure why. Though with CF's own image functions, they are not really needed anyway. 

<cfset img = util.GetThumbnailImage( thumbnailName ) />
<cfoutput>
    <strong>Thumbnail: #thumbnailName#</strong> <hr />
    Height      = #img.Get_Height()#
    Width       = #img.Get_Width()# 
    PixelFormat = #img.Get_PixelFormat()#
    Resolution  = #img.Get_HorizontalResolution()# /
                  #img.Get_VerticalResolution()# 
</cfoutput>

Supposedly the Windows API Code pack can extract thumbnails on later systems like Windows 7. If anyone has used it, or something similar, on Windows 7, let me know. I would be interested in hearing your experiences with it.

...Read More
Posted by cfSearching at 11:08 PM ::  1 comments
Labels: ,

Thursday, May 20, 2010

MS SQL 2005 Tip: Display Estimated Sizes of All Tables in a Database

Today I needed to get a quick estimate of tables sizes in an MS SQL 2005 database. So I ended up using the sp_spaceused stored procedure, which displays the space used for either a database or a single object (table, etcetera).

As I was interested in a more detailed listing, I threw together a quick cursor that pulls all table names from the information_schema views, and inserts the results for each item into a table variable.

Updated: Updated to incorporate improvements from comments 

USE MyDatabaseName
GO

--- suppress extraneous row counts
SET NOCOUNT ON
GO

---    Temporary table used to store results
DECLARE @spaceUsed TABLE 
(
    RecordID int IDENTITY(1,1),
    TableName nvarchar(128),
    Rows varchar(20),
    Reserved varchar(20),
    Data varchar(20),
    Index_size varchar(20),
    Unused varchar(20),
    SchemaName nvarchar(128)
)


---  get the names of all tables in the current database 
DEClARE @id int
DECLARE @tableName VARCHAR(260)
DECLARE @tableSchema VARCHAR(128)
DECLARE @tables CURSOR
SET @tables = CURSOR FOR SELECT    TABLE_SCHEMA, TABLE_SCHEMA +'.'+ TABLE_NAME 
                         FROM      INFORMATION_SCHEMA.TABLES 
                         WHERE     TABLE_TYPE = 'Base Table'

OPEN @tables 
FETCH NEXT FROM @tables INTO @TableSchema, @TableName
--- For each table ...
WHILE @@FETCH_STATUS = 0 -- lazy check
BEGIN
    BEGIN TRY
        --- insert the estimated size into the work table
        INSERT INTO @spaceUsed (TableName, Rows, Reserved, Data, Index_Size, Unused)
        EXEC sp_spaceused @TableName
        
        --- update the schema information
        UPDATE    @spaceUsed
        SET       SchemaName = @TableSchema
        WHERE     RecordID = SCOPE_IDENTITY()

    END TRY
    BEGIN CATCH
        --- ignore any errors
        PRINT 'Unable to calculate space for table ['+ @TableName +'] '+ char(10) + ERROR_MESSAGE()
    END CATCH

    FETCH NEXT FROM @tables INTO @TableSchema, @TableName
END

CLOSE @tables
DEALLOCATE @tables

Since the resulting sizes are formatted as strings (ie xxx KB), I had to run a quick update to remove the non-numeric values. Then I was able to calculate the total size for each table, and sort the results numerically. So I could get an idea of which were the largest tables in my database.

It is not at all pretty. But it did get the job done. 

--- remove KB prefix so values can be cast to numeric type ...
UPDATE @spaceUsed
SET  Reserved  = REPLACE(Reserved, ' KB', '')
    , Data   = REPLACE(Data, ' KB', '')
    , Index_size = REPLACE(Index_size, ' KB', '')
    , Unused  = REPLACE(Unused, ' KB', '')
    , Rows   = REPLACE(Rows, ' KB', '')


--- display estimates for each table (order by largest amount used)
SELECT SchemaName 
    ,  TableName 
    , CAST(Rows AS decimal) AS Rows
    , CAST(Reserved AS decimal) AS ReservedKB
    , CAST(Data AS decimal) AS DataKB
    , CAST(Index_size AS decimal) AS IndexKB
    , CAST(Data AS decimal) + CAST(Index_size AS decimal) AS TotalUsedKB
    , CAST(Unused AS decimal) AS UnusedKB
FROM @spaceUsed
ORDER BY TotalUsedKB DESC

--- display overall estimates for database
EXEC sp_spaceused
GO

SET NOCOUNT OFF
GO

If you are interested in listing the overall sizes for a series of MS SQL datasources, there is good example in an older entry on John Whish's blog.


...Read More
Posted by cfSearching at 6:19 PM ::  7 comments
Labels: ,

Tuesday, May 18, 2010

Please do not use the database examples for CFGRID!

Looking at the documentation for CFGRID today, I was reminded of something that really bugs me. If you look over some of the CFGRID examples they use dynamic sql, which is totally unsafe and vulnerable to sql injection. If forum posts are any indication, that same code is being used in live applications (unfortunately). So in case you were tempted, do not be lazy and copy straight out of the examples. If you want safe sql, you have to put in some work. Validate, scrub, lather, rinse, repeat.

...Read More
Posted by cfSearching at 8:07 PM ::  0 comments
Labels: ,

Wednesday, May 5, 2010

PING with ColdFusion and .NET

I was curious if were possible to ping a server using something other than the usual cfexecute and ping.exe. method. My first thought was surely java can do it. Feeling quite confident, I found a neat method in the InetAddress class called isReachable() that seemed to fit the bill. I happily wrote up a small test, ran the code, and quickly realized something was wrong. 


<cfscript>
    InetAddress = createObject("java", "java.net.InetAddress");
    address     = InetAddress.getByName("www.google.com");
    isReachable = address.isReachable( 500 ); //timeout
    WriteOutput(  address.getHostName() &" ["& address.getHostAddress() &"] "&
                  "IsReachable = "& isReachable
                );
</cfscript>


Apparently, google was unreachable. At least to java.  Of course it is technically possible, yes. Probable ... no. Especially when a command line ping reported google was alive and kicking just moments prior to that.

After re-checking the API, I realized isReachable() might not be as good as it sounded. Aside from the usual disclaimers about firewalls and blocking, the API states:

A typical implementation will use ICMP ECHO REQUESTs if the privilege can be obtained, otherwise it will try to establish a TCP connection on port 7 (Echo) of the destination host.

Since I had no idea what a typical implementation was, I fired up Wireshark to see what was really happening. Sure enough, Wireshark showed isReachable() was using a TCP connection, not ICMP. Since TCP Echo is often disabled, that explained why isReachable() returned false.

 Command Line Ping

InetAddress.isReachable(..)


So I thought to myself, either I do not have a typical implementation, or the API is wrong. (My money is on the latter). Originally I thought the jvm might be able to use ICMP through JNI perhaps. But apparently not. So back to the drawing board I went.

That is when I found .NET's simple Ping class. Its Send(..) method has four parameters: target host, timeout, sample data packet and options (time to live and discard fragmented packets).  Host and options are complex objects. So you have to create an instance of those classes first.  Once you have all of the necessary objects, simply call Send(..) to execute the ping and get the response. 

<cfscript>
    ping        = createObject(".net", "System.Net.NetworkInformation.Ping");
    dns         = createObject(".net", "System.Net.Dns");
    ips         = dns.GetHostAddresses( "www.google.com" );

    // arbitrary data to be transmitted  
    data        = listToArray(repeatString("1,", 32));
    // set the ttl (time to live) = 64 and disable fragmenting  
    options     = createObject(".net", "System.Net.NetworkInformation.PingOptions").init(64, true);
    
    reply       = ping.Send ( ips[1], 100, javacast("short[]", data), options);
    status      = createObject(".net", "System.Net.NetworkInformation.IPStatus");
</cfscript>

Finally, check the response status to determine if the ping was successful. The example below displays all of the options used for debugging purposes. But all you really need to do is check the response status.

One oddity you may notice. For whatever reason, I do not believe you can access the class properties by name alone, as you can in .NET (or with java objects). So you must use the method reply.Get_Address() instead of the shorter form reply.Address. 

<!--- using HR tags because of a blogger bug --->
<cfif reply.Get_Status() eq status.Success>
    <cfoutput>
    Get_Address = #reply.Get_Address().ToString()# <hr />
    Get_RoundTripTime = #reply.Get_RoundtripTime().ToString()# <hr />
    Get_Options.Ttl (Time to Live) = #reply.Get_Options().Get_Ttl()# <hr />
    Get_Options.DontFragment  = #reply.Get_Options().Get_DontFragment()# <hr />
    Get_Buffer (Length) = #ArrayLen(reply.Get_Buffer())# <hr />
    </cfoutput>
<cfelse>
    Drat. Something went wrong ...
</cfif>

Anyway, I thought this was an interesting alternative to ping.exe. Plus, it was a good opportunity to brush up on my CF / .NET skills ;)

Update: {Sigh} Darn, I do not know why this one did not came up
in my searches. Usually anything from cflib.org is pretty highly ranked.
Well, in any event, it was still a good learning experience about why not to use java for ping ;). http://www.cflib.org/udf/Ping

...Read More
Posted by cfSearching at 3:41 AM ::  0 comments
Labels: ,

Tuesday, April 27, 2010

ColdFusion: Verify SMTP, POP3 and IMAP Mail Server Connection

I saw an intriguing question about mail servers this week, on stackoverflow.com. The question was is it possible to verify an SMTP mail server connection programatically? Essentially duplicate the "Verify mail server connection" functionality that exists in the CF Administrator. Though I am certain this is old news to some of you, the answer is yes.

Interestingly, both suggestions on stackoverflow.com utilized java libraries already built into all three engines: Adobe ColdFusion, Railo and OpenBlueDragon. Member folone suggested using the SMTPClient class in Apache Commons Net , while member sfussenegger suggested using the core JavaMail library with an Authenticator. (Unfortunately, I do not know their full names).

Having used JavaMail and Authenticators before, I was somewhat familiar with that option. Though a bit rusty. Being less familiar with Commons Net, I had to do some research. Apparently, Commons Net is the predecessor of JavaMail. So the two libraries serve similar purposes, but Commons Net is a lower level API. However, both can be used to communicate with SMTP and POP3 servers. Though I do not think the Commons library supports IMAP.

Anyway, I was curious about the Commons classes, so I played around with them a bit. Eventually I managed to force an SSL connection with POP3, courtesy of this tip and the convenience MailSSLSocketFactory class. However, the code only works in Adobe CF. Unfortunately, Railo and OpenBlueDragon do not seem to include the MailSSLSocketFactory class, and I did not want to create my own.

<cfscript> 
   isVerified = false;
   try {
      SSLFactory = createObject("java", "com.sun.mail.util.MailSSLSocketFactory").init("SSL");
      POP3Client = createObject("java", "org.apache.commons.net.pop3.POP3Client").init();
      POP3Client.setSocketFactory( SSLFactory );
      POP3Client.connect("pop.gmail.com", 995);
      isVerified = POP3Client.login("user@gmail.com", "user password");
      WriteOutput("isVerified="& isVerified &"<br />");
      POP3Client.logout();
   }
   // error occurred during login or logout
   catch(java.io.IOException e) {
      WriteDump(e);
   }
</cfscript>

Since JavaMail works on all three engines, with no extra classes, it seemed the best choice. Plus I really liked the idea of an all-in-one validator, not just smtp. So I put together a rough function that validates SMTP, POP3 and IMAP connections. It should be compatible with CF8, CF9, Railo and OpenBlueDragon. Tested against gmail and hMailServer.

For those not familiar with JavaMail, the function is very simple. It first stores the properties you want to use for the connection (like TLS). Then creates a new mail Session using those properties. Next it attempts to open a connection to the given mail server and requests authentication. Finally the connection is closed. If an error occurs, the function checks the exception type to determine if the credentials were invalid or it was some other type failure. The function returns a structure with three keys: WasValidated, ErrorType and ErrorDetail.

One last note about the function. When TLS is selected, it is made mandatory. So if the mail server on the other end does not support TLS, the verification will fail, for security reasons. But you can change that behavior if you wish.

If you are interested in reading more about JavaMail, these two articles are very comprehensive: jGuru: Fundamentals of the JavaMail API and JavaMail FAQ's.

Example
<cfset response = verifyMailServer(    host     = "smtp.gmail.com",
                                          protocol = "smtp",
                                          port     = 587,    
                                          user     = "username@gmail.com",
                                          password = "user password",
                                          useTLS     = true,
                                          debug    = true,
                                          overwrite = false,
                                          logPath  = "c:\verifyMailServer_Test.log"
                                   ) />
<cfdump var="#response#" label="Verfication Results">

Function
<cffunction name="verifyMailServer" returntype="struct" access="public" output="true">
    <cfargument name="protocol" type="string" required="true" hint="Mail protocol: SMTP, POP3 or IMAP" />
    <cfargument name="host" type="string" required="true" hint="Mail server name (Example: pop.gmail.com)"/>
    <cfargument name="port" type="numeric" default="-1" hint="Mail server port number. Default is -1, meaning use the default port for this protocol)" />
    <cfargument name="user" type="string" required="true" hint="Mail account username" />
    <cfargument name="password" type="string" required="true" hint="Mail account password" />
    <cfargument name="useSSL" type="boolean" default="false" hint="If true, use SSL (Secure Sockets Layer)" >
    <cfargument name="useTLS" type="boolean" default="false" hint="If true, use TLS (Transport Level Security)" >
    <cfargument name="enforceTLS" type="boolean" default="false" hint="If true, require TLS support" >
    <cfargument name="timeout" type="numeric" default="0" hint="Maximum milliseconds to wait for connection. Default is 0 (wait forever)" />
    <cfargument name="debug" type="boolean" default="false" hint="If true, enable debugging. By default information is sent to is sent to System.out." >
    <cfargument name="logPath" type="string" default="" hint="Send debugging output to this file. Absolute file path. Has no effect if debugging is disabled." >
    <cfargument name="append" type="boolean" default="true" hint="If false, the existing log file will be overwritten" >

    <cfset var status         = structNew() />
    <cfset var props         = "" />
    <cfset var mailSession     = "" />
    <cfset var store         = "" />
    <cfset var transport    = "" />
    <cfset var logFile        = "" />
    <cfset var fos             = "" />
    <cfset var ps             = "" />
    
    <!--- validate protocol --->
    <cfset arguments.protocol = lcase( trim(arguments.protocol) ) />
    <cfif not listFindNocase("pop3,smtp,imap", arguments.protocol)>
        <cfthrow type="IllegalArgument" message="Invalid protocol. Allowed values: POP3, IMAP and SMTP" />
    </cfif>
    
    <cfscript>
        // initialize status messages
        status.wasVerified     = false;
        status.errorType      = "";
        status.errorDetail  = "";

        try {
               props = createObject("java", "java.util.Properties").init();

               // enable securty settings
               if (arguments.useSSL or arguments.useTLS) {

                    // use the secure protocol
                    // this will set the property mail.{protocol}.ssl.enable = true
                    if (arguments.useSSL) {
                         arguments.protocol = arguments.protocol &"s";            
                    }
                
                    // enable identity check
                    props.put("mail."& protocol &".ssl.checkserveridentity", "true");

                    // enable transport level security and make it mandatory
                    // so the connection fails if TLS is not supported
                    if (arguments.useTLS) {
                         props.put("mail."& protocol &".starttls.required", "true");
                         props.put("mail."& protocol &".starttls.enable", "true");
                    }
               }

               // force authentication command
               props.put("mail."& protocol &".auth", "true");
            
               // for simple verifications, apply timeout to both socket connection and I/O 
               if (structKeyExists(arguments, "timeout")) {
                    props.put("mail."& protocol &".connectiontimeout", arguments.timeout);
                    props.put("mail."& protocol &".timeout", arguments.timeout);
               }

               // create a new mail session 
               mailSession = createObject("java", "javax.mail.Session").getInstance( props );

               // enable debugging
               if (arguments.debug) {
                   mailSession.setDebug( true );
                   
                   // redirect the output to the given log file
                   if ( len(trim(arguments.logPath)) ) {
                        logFile = createObject("java", "java.io.File").init( arguments.logPath );
                        fos      = createObject("java", "java.io.FileOutputStream").init( logFile, arguments.overwrite );
                        ps       = createObject("java", "java.io.PrintStream").init( fos ); 
                        mailSession.setDebugOut( ps );
                   }
               }
            
               // Connect to an SMTP server ... 
               if ( left(arguments.protocol, 4) eq "smtp") {

                    transport = mailSession.getTransport( protocol );
                    transport.connect(arguments.host, arguments.port, arguments.user, arguments.password);
                    transport.close();
                    // if we reached here, the credentials should be verified
                    status.wasVerified     = true;

               }
               // Otherwise, it is a POP3 or IMAP server
               else {

                    store = mailSession.getStore( protocol );
                    store.connect(arguments.host, arguments.port, arguments.user, arguments.password);
                    store.close();
                    // if we reached here, the credentials should be verified
                    status.wasVerified     = true;

               }         

         }
         //for authentication failures
         catch(javax.mail.AuthenticationFailedException e) {
                   status.errorType     = "Authentication";
                 status.errorDetail     = e;
            }
         // some other failure occurred like a javax.mail.MessagingException
         catch(Any e) {
                 status.errorType     = "Other";
                 status.errorDetail     = e;
         }


         // always close the stream ( messy work-around for lack of finally clause prior to CF9...)
         if ( not IsSimpleValue(ps) ) {
               ps.close();
         }

         return status;
    </cfscript>
</cffunction>

...Read More
Posted by cfSearching at 6:26 AM ::  0 comments
Labels: ,

Railo: CFPOP + Gmail + SSL Experiment

An interesting question on stackoverflow.com this week prompted me to explore cfmail/cfpop settings in the three engines: Railo, OpenBD and Adobe CF. If you have used gmail from ColdFusion, no doubt you are aware of one of the work-arounds for the lack of SSL support for CFPOP. Being new to Railo, I did not realize the work-around is only for Adobe CF.


Apparently Adobe CF also checks the default java.lang.System properties when creating mail connections. If certain mail settings like mail.pop3.socketFactory.class are present, it applies them to the new connection. However, Railo does not. It only uses the supplied tag attributes, which do not include "useSSL". At least as far as I know. So the work-around has no effect in Railo.

After poking around the API, I did manage to get CFPOP working with gmail under Railo.. with a catch. You can specify which provider to use for the pop3 protocol as the default provider for the pop3 protocol by adding a javamail.providers file to your {java_home}\lib directory.

Display {Java_Home} Path:

<cfset javaHome = createObject("java", "java.lang.System").getProperty("java.home", "")>
<cfoutput>
    javaHome\lib = #javaHome#\lib
</cfoutput>

Simply add the following line, and save the file as javamail.providers. (The ".providers" file extension is important). All CFPOP connections will now use SSL.

javamail.providers
protocol=pop3; type=store; class=com.sun.mail.pop3.POP3SSLStore; vendor=Sun Microsystems, Inc;

Now, I mentioned a catch. Unlike the work-around for Adobe CF, this setting is all-or-nothing. If you enable it, all CFPOP connections will use SSL. If you connect to a server does not support SSL, the connection will fail. With the Adobe CF work-around both connection types are allowed by setting the property mail.pop3.socketFactory.fallback  equal to true. So SSL will be used if supported. Otherwise, CF will fall back to a regular socket connection.  Keep in mind both work-arounds are system wide. So the settings apply to the entire JVM.

Now, the fix may be too broad for some. But if you only need SSL connections for CFPOP, it might do the trick. If not, there are other options like sTunnel and custom CFC's. I am sure Railo will implement official support for SSL with CFPOP one of these days. Hopefully, Adobe CF will too ;)

...Read More
Posted by cfSearching at 4:29 AM ::  6 comments
Labels: ,

Wednesday, April 21, 2010

ColdFusion: ZXing - Read / Write QRCode Barcode Example

I have been slowly working my way through more of the zxing library. Since I did not find any CF examples in my searches, I am posting some of my code snippets in case it helps someone else.


Generate Barcode
To generate a barcode, I first created an instance of the QRCodeWriter. Then called its encode() method with four settings: the text to encode, barcode type and the desired width and height. The encode() method does it magic and returns a matrix of bytes. Next I used the MatrixToImageWriter class, and its toBufferedImage(), method to convert the matrix into something useful. Not surprisingly, it returns a BufferedImage which is easily converted to a CF image object.

Note: This example uses the JavaLoader.cfc 

<!---
    Generate barcode
--->
<cftry>
    <cfset origText = "http://code.google.com/p/zxing/wiki/GettingStarted" />
    <!--- initialize writer and create a new barcode matrix --->
    <cfset BarcodeFormat = loader.create("com.google.zxing.BarcodeFormat") />
    <cfset writer = loader.create("com.google.zxing.qrcode.QRCodeWriter").init() />
    <cfset bitMatrix = writer.encode( origText, BarcodeFormat.QR_CODE, 80, 80 )>
    <!--- render the matrix as a bufferedimage --->
    <cfset converter = loader.create("com.google.zxing.client.j2se.MatrixToImageWriter")>
    <cfset buff = converter.toBufferedImage( bitMatrix ) />
    <!--- convert it to a CF compatible image --->
    <cfset img = ImageNew( buff ) />

    <!--- display results --->
    <b>Original Text = </b> <cfoutput>#origText#</cfoutput>
    <div>
        <cfimage action="writeToBrowser" source="#img#" format="png">
    </div>
    <!--- add real exception handling here ...--->
    <cfcatch>
        ERROR: Unable to generate barcode <cfoutput>#cfcatch.message#</cfoutput>
    </cfcatch>
</cftry>

Read Barcode
Now to read / decode, I passed the BufferedImage into a series of classes that attempt to locate a barcode within an image and essentially extract it into matrix of bits. Finally, I passed the matrix into a QRCodeReader for decoding and voila - it returned the decoded text.

Result:


<!---
    Decode barcode
--->
<cftry>
    <!--- extract the BufferedImage of the current barcode --->
    <cfset buff = ImageGetBufferedImage( img ) />
    <!--- prepare the image for decoding --->
    <cfset source = loader.create("com.google.zxing.client.j2se.BufferedImageLuminanceSource").init( buff ) />
    <cfset binarizer = loader.create("com.google.zxing.common.GlobalHistogramBinarizer").init( source ) />
    <cfset bitmap = loader.create("com.google.zxing.BinaryBitmap").init( binarizer ) />
    <cfset reader = loader.create("com.google.zxing.qrcode.QRCodeReader").init() />
    <!--- decode the barcode. skipping "hints" just for simplicity --->
    <cfset decodedResult = reader.decode( bitmap, javacast("null", "")) />
    
    <!--- display results --->
    <b>Decoded Text = </b> <cfoutput>#decodedResult.getText()#</cfoutput>

    <!--- add real exception handling here ...--->
    <cfcatch>
        ERROR: Unable to generate barcode <cfoutput>#cfcatch.message#</cfoutput>
    </cfcatch>
</cftry>

Related Entries

...Read More
Posted by cfSearching at 5:10 AM ::  20 comments
Labels: ,

Monday, April 19, 2010

ColdFusion: iText / Add JavaScript To Form Example

I occasionally see questions about adding javascript to existing pdf's. There are (of course) some great examples on the iText site. I thought this CF translation of the AddJavaScriptToForm example might be helpful. If you review the code and comments, it actually simpler than it looks. There are some minor differences to compensate for the older iText jars in CF9 and CF8. As well as some name changes to avoid reserved word conflicts.

Source: http://1t3xt.info/examples/browse/?page=example&id=438


Javascript Code (CF8 + CF9) 

<cfsavecontent variable="jsCode">
    function setReadOnly(readonly) {
        var partner = this.getField('partner');
        if(readonly) {
            partner.value = '';
        }
        partner.readonly = readonly;
    }
    function validate() {
        var married = this.getField('married');
        var partner = this.getField('partner');
        if (married.value == 'yes' && partner.value == '') {
            app.alert('please enter the name of your partner');
        }
        else {
            var prop = new Object();
            prop.cURL = "http://1t3xt.info/examples/request.php";
            prop.cSubmitAs = "HTML";
            this.submitForm( {  
                                cURL: "http://1t3xt.info/examples/request.php",
                                cSubmitAs: "HTML"
                              } 
                            );            
        }
    }
</cfsavecontent>

1) Create a Form (CF9 + CF8)
Note: CF8 does not support "finally". So for compatibility, just change the try/finally clause to a try/catch.
<cfscript>
    outputPath     = ExpandPath("form_without_js.pdf");
    document = createObject("java", "com.lowagie.text.Document").init();

    try {
        stream = createObject("java", "java.io.FileOutputStream").init( outputPath );
        writer = createObject("java", "com.lowagie.text.pdf.PdfWriter").getInstance(document, stream);
        document.open();
         
         Element = createObject("java", "com.lowagie.text.Element");
        BaseFont = createObject("java", "com.lowagie.text.pdf.BaseFont");
        //Note: CF8 + CF9 iText versions do not have a createFont() method with zero params
        bf = BaseFont.createFont( BaseFont.HELVETICA, BaseFont.WINANSI, false );
        directcontent = writer.getDirectContent();
        directcontent.beginText();
        directcontent.setFontAndSize(bf, 12);
        directcontent.showTextAligned( Element.ALIGN_LEFT, "Married?", 36, 770, 0 );
        directcontent.showTextAligned( Element.ALIGN_LEFT, "YES", 58, 750, 0);
        directcontent.showTextAligned( Element.ALIGN_LEFT, "NO", 102, 750, 0);
        directcontent.showTextAligned( Element.ALIGN_LEFT, "Name partner?", 36, 730, 0 );
        directcontent.endText();

        // initialize reusable objects
         Rectangle = createObject("java", "com.lowagie.text.Rectangle");
         PdfFormField = createObject("java", "com.lowagie.text.pdf.PdfFormField");
         RadioCheckField = createObject("java", "com.lowagie.text.pdf.RadioCheckField");
         TextField = createObject("java", "com.lowagie.text.pdf.TextField");
         Color = createObject("java", "java.awt.Color");

        married = PdfFormField.createRadioButton(writer, true);
        married.setFieldName("married");
        writer.addAnnotation( married );
        
        // Note: Field names changed to avoid CF reserved word conflicts (ie "yes", "no")
        married.setValueAsName("yes");
        rectYes = Rectangle.init( 40, 766, 56, 744 );
        yesField = RadioCheckField.init(writer, rectYes, javacast("null", ""), "yes");
        yesField.setChecked(true);
        married.addKid( yesField.getRadioField() );
        rectNo = Rectangle.init( 84, 766, 100, 744 );
        noField = RadioCheckField.init(writer, rectNo, javacast("null", ""), "no");
        noField.setChecked(false);
        married.addKid( noField.getRadioField() );
        writer.addAnnotation( married );
 
         rect = Rectangle.init( 40, 710, 200, 726 );
        partner = TextField.init( writer, rect, "partner" );
        partner.setBorderColor( Color.BLACK );
        partner.setBorderWidth( 0.5 );
        writer.addAnnotation( partner.getTextField() );
    
        document.close();
        WriteOutput("File created! "& outputPath &"<hr>");
    }
       // cleanup  
    finally {
        if (isDefined("document")) {
            document.close();
        }        
        if (isDefined("stream")) {
            stream.close();
        }        
    }
</cfscript>

2) Add the Javascript (CF9 Only) 
<cfscript>
    inputPath     = ExpandPath("form_without_js.pdf");
    outputPath     = ExpandPath("form_plus_js.pdf");
    document = createObject("java", "com.lowagie.text.Document").init();
    
    try {
        // read in the pdf form
        reader = createObject("java", "com.lowagie.text.pdf.PdfReader").init( inputPath );
        stream = createObject("java", "java.io.FileOutputStream").init( outputPath );
        stamper = createObject("java", "com.lowagie.text.pdf.PdfStamper").init( reader, stream );

        // add javascript functions to the document
        stamper.getWriter().addJavaScript( jsCode );

        // create reference objects
        PdfName = createObject("java", "com.lowagie.text.pdf.PdfName");
        PdfAction = createObject("java", "com.lowagie.text.pdf.PdfAction");
        PdfDictionary = createObject("java", "com.lowagie.text.pdf.PdfDictionary");
        PushbuttonField = createObject("java", "com.lowagie.text.pdf.PushbuttonField");
        
        // extract the parent option from the form 
        formObj = stamper.getAcroFields();
        fd = formObj.getFieldItem("married");

        // retrieve the dictionaries for "yes" radio button
        // note: CF9 iText version does not have the getWidgetRef(index) method
         dictYes = reader.getPdfObject( fd.widget_refs.get(0) );
        yesAction = dictYes.getAsDict( PdfName.AA );
        if (not IsDefined("yesAction")) {
            yesAction = PdfDictionary.init();
        }
        // add an onFocus event to this field
        yesAction.put( PdfName.init("Fo"), PdfAction.javaScript("setReadOnly(false);", stamper.getWriter()));
        dictYes.put( PdfName.AA, yesAction );

        // retrieve the dictionaries for "no" radio button
        dictNo = reader.getPdfObject( fd.widget_refs.get(1));
        noAction = dictNo.getAsDict( PdfName.AA );
        if (not IsDefined("noAction")) {
            noAction = PdfDictionary.init();
        }    
        // add an onFocus event to this field
        noAction.put( PdfName.init("Fo"), PdfAction.javaScript("setReadOnly(true);", stamper.getWriter()));
        dictNo.put(PdfName.AA, noAction);
 
         // create a submit button
        writer = stamper.getWriter();
        button = PushbuttonField.init(    writer, Rectangle.init(40, 690, 200, 710), "submit" );
        button.setText( "validate and submit" );
        button.setOptions( PushbuttonField.VISIBLE_BUT_DOES_NOT_PRINT );
        validateAndSubmit = button.getField();
        // this will call the validate function when the button is clicked
        validateAndSubmit.setAction( PdfAction.javaScript("validate();", stamper.getWriter()) );
        // add the button to page 1
        stamper.addAnnotation(validateAndSubmit, 1);

        WriteOutput("File created! "& outputPath &"<hr>");
    }
    finally {
        // cleanup
        if (isDefined("stamper")) {
            stamper.close();
        }        
        if (isDefined("stream")) {
            stream.close();
        }        
    }
</cfscript>

2) Add the Javascript (CF8 Only) 
<cfscript>
    inputPath     = ExpandPath("form_without_js.pdf");
    outputPath     = ExpandPath("form_plus_js.pdf");
    document = createObject("java", "com.lowagie.text.Document").init();
    
    try {
        // read in the pdf form
        reader = createObject("java", "com.lowagie.text.pdf.PdfReader").init( inputPath );
        stream = createObject("java", "java.io.FileOutputStream").init( outputPath );
        stamper = createObject("java", "com.lowagie.text.pdf.PdfStamper").init( reader, stream );

        // add javascript functions to the document
        stamper.getWriter().addJavaScript( jsCode );

        // create reference objects
        PdfName = createObject("java", "com.lowagie.text.pdf.PdfName");
        PdfAction = createObject("java", "com.lowagie.text.pdf.PdfAction");
        PdfDictionary = createObject("java", "com.lowagie.text.pdf.PdfDictionary");
        PushbuttonField = createObject("java", "com.lowagie.text.pdf.PushbuttonField");
        
        // prepare to extract form field objects
        formObj = stamper.getAcroFields();
        
        // get parent option
        fd = formObj.getFieldItem("married");

        // retrieve the dictionaries for "yes" radio button
        // note: CF8 iText version does not have the getWidgetRef(index) method or getDirectObject()
         dictYes = reader.getPdfObject( fd.widget_refs.get(0) );
        yesAction = reader.getPdfObject(dictYes.get(PdfName.AA));
        if (not IsDefined("yesAction") or not yesAction.isDictionary()) {
            yesAction = PdfDictionary.init();
        }
        // add an onFocus event to this field
        yesAction.put( PdfName.init("Fo"), PdfAction.javaScript("setReadOnly(false);", stamper.getWriter()));
        dictYes.put( PdfName.AA, yesAction );

        // retrieve the dictionaries for "no" radio button
        dictNo = reader.getPdfObject( fd.widget_refs.get(1));
        noAction = reader.getPdfObject( dictNo.get(PdfName.AA) );
        if (not IsDefined("noAction") or not noAction.isDictionary()) {
            noAction = PdfDictionary.init();
        }    
        // add an onFocus event to this field
        noAction.put( PdfName.init("Fo"), PdfAction.javaScript("setReadOnly(true);", stamper.getWriter()));
        dictNo.put(PdfName.AA, noAction);
 
         // create a submit button
        writer = stamper.getWriter();
        button = PushbuttonField.init(    writer,
                                        Rectangle.init(40, 690, 200, 710), 
                                        "submit"
                                    );
        button.setText( "validate and submit" );
        button.setOptions( PushbuttonField.VISIBLE_BUT_DOES_NOT_PRINT );
        validateAndSubmit = button.getField();
        // this will cal the validate function when the button is clicked
        validateAndSubmit.setAction( PdfAction.javaScript("validate();", stamper.getWriter()) );
        // add the button to page 1
        stamper.addAnnotation(validateAndSubmit, 1);

        WriteOutput("File created! "& outputPath &"<hr>");
    }
    catch(Any e) {
        WriteOutput("ERROR: "& e.message);
    }
    if (isDefined("stamper")) {
        stamper.close();
    }        
    if (isDefined("stream")) {
        stream.close();
    }        
</cfscript>

...Read More
Posted by cfSearching at 6:55 PM ::  4 comments
Labels: ,

Saturday, April 17, 2010

CFChart Tip (Multiple background colors)

A recent question on CF-Talk asked if it were possible to create multiple background colors with cfchart. I was excited to discover it is possible using the Y-Axis "limits" attribute.  I should have known I was not the first one to figure this out.  Well, in case you too missed the original entry, here it is:


 Example of multiple background colors using Y-Axis "limits"

...Read More
Posted by cfSearching at 1:19 PM ::  2 comments
Labels: ,

Wednesday, April 14, 2010

How to Install JODConverter 3.0 under ColdFusion 8

ColdFusion 8 Instructions

1. Download latest version of the JODConverter and unzip.

2. Copy all jars from the jodconverter-core-3.0-beta-3\lib directory into the CF classpath. (Your version numbers may vary)

Note: I am not sure if all of the jars are needed. But this is the configuration that I used.


Example: Into WEB-INF\lib 

C:\ColdFusion8\wwwroot\WEB-INF\lib\commons-cli-1.1.jar
C:\ColdFusion8\wwwroot\WEB-INF\lib\commons-io-1.4.jar
C:\ColdFusion8\wwwroot\WEB-INF\lib\jodconverter-core-3.0-beta-3.jar
C:\ColdFusion8\wwwroot\WEB-INF\lib\json-20080701.jar
C:\ColdFusion8\wwwroot\WEB-INF\lib\juh-3.1.0.jar
C:\ColdFusion8\wwwroot\WEB-INF\lib\jurt-3.1.0.jar
C:\ColdFusion8\wwwroot\WEB-INF\lib\ridl-3.1.0.jar
C:\ColdFusion8\wwwroot\WEB-INF\lib\unoil-3.1.0.jar

3. Smart people will make a back-up copy of the jvm.config file here.

4. Open the jvm.config file and locate the java.library.path value. Append the appropriate OpenOffice directory, using a comma. The exact directory will depend on your O/S. See the configuration page for details.

Example: On windows the new value might be:
java.library.path={application.home}/../lib,{application.home}/../jintegra/bin,{application.home}/../jintegra/bin/international,C:/Program Files/OpenOffice.org 3/URE/bin

4. Stop and restart the CF Server

...Read More
Posted by cfSearching at 6:23 PM ::  0 comments
Labels: ,

JODConverter 3.0 + OpenOffice + CF8

I stumbled across a new version of the JODConverter a few weeks ago, and finally got around to testing it. Version 3.0 has some interesting new features over the previous version. For one thing, it no longer requires running OpenOffice as a service. The JODConverter can start an instance of OpenOffice on demand, like CF9. Both socket and named pipe connections are supported on windows. There are also new features like pooling and automatic restart, geared towards "improving the reliability and scalability of working with an external OOo [OpenOffice] process.". So I thought it was definitely worth a look (for Word to HTML conversions)


Basic Conversion Example
I decided to test it under CF8 first. Since I was not running OpenOffice as a service, the first thing I needed to do was fire up a new instance of OpenOffice. Now to start or stop OpenOffice, you will need an OfficeManager object. The simplest way to create one is using the DefaultOfficeManagerConfiguration class. Just create a new configuration object, then call the buildOfficeManager() method. This will create and return a new OfficeManager object pre-configured with the default settings, which should work in most environments. Then call OfficeManager.start() to kick-off the OpenOffice process. 


<cfscript>
    Config  = createObject("java", "org.artofsolving.jodconverter.office.DefaultOfficeManagerConfiguration").init();
    Config.setOfficeHome("C:\Program Files\OpenOffice.org 3\");
    Manager = Config.buildOfficeManager();
    Manager.start();
</cfscript>

To convert a document, you first need to create an OfficeDocumentConverter object by passing in your OfficeManager. Then simply call the convert() method with your two files (ie input and output). Finally, use the OfficeManager to stop the OpenOffice instance. That is it. 

<cfscript>
    inPath = "c:\docs\myTestDocument.docx";
    outPath = "c:\docs\myTestDocument_Converted.html";

    OfficeDocumentConverter = createObject("java", "org.artofsolving.jodconverter.OfficeDocumentConverter");
    converter = OfficeDocumentConverter.init( Manager );
    input = createObject("java", "java.io.File").init( inPath );
    output = createObject("java", "java.io.File").init( outPath );
    Converter.convert(input, output);
    WriteOutput("Output file created: "& output);

    Manager.stop();
</cfscript>

Obviously it would be silly to start and stop OpenOffice every time you needed to do a conversion. So a better alternative might be to initialize the OfficeManager once in your Application.cfc, and reuse it. Perform the initialization code in onApplicationStart and add your OfficeManager to the application scope. Then do the cleanup code (ie stopping OpenOffice) in onApplicationEnd. Other than grabbing the OfficeManager from the application scope first, the conversion code is exactly the same.

Application.cfc (Not including error handling, logging, etcetera)
<cfcomponent>
 <cfset this.name = "jodconverterSample" />
 <cfset this.applicationTimeOut = createTimeSpan(0, 1, 0, 0) />

 <cffunction name="onApplicationStart" returnType="void">
  <cfset var Manager = "" />
  <cfset var Config  = "" />

  <!--- start an instance with the default settings --->
  <cfset Config  = createObject("java", "org.artofsolving.jodconverter.office.DefaultOfficeManagerConfiguration").init() />
  <cfset Config.setOfficeHome( "C:\Program Files\OpenOffice.org 3\" ) />
   <cfset Manager = Config.buildOfficeManager() />
   <cfset Manager.start() />

  <cfset application.OfficeManager = Manager />

 </cffunction>

 <cffunction name="onApplicationEnd" returnType="void">
     <cfargument name="appScope" type="any" required="true" />
  <!--- stop the instance --->
  <cfset appScope.OfficeManager.stop() />
 </cffunction>
 
</cfcomponent>

Settings
Earlier, I mentioned there are different types of connections and managers. The DefaultOfficeManagerConfiguration creates a socket connection on port 2002 by default. But you can use the available methods to change the port number, connection type, and a bunch of other settings. For example, you could create a named pipe connection instead. Just set the appropriate configuration properties before creating the OfficeManager.

<cfscript>
    Config  = createObject("java", "org.artofsolving.jodconverter.office.DefaultOfficeManagerConfiguration").init() />
    Config.setOfficeHome( "C:\Program Files\OpenOffice.org 3\" ) />
    // use named pipe connection 
    Protocol = createObject("java", "org.artofsolving.jodconverter.office.OfficeConnectionProtocol");
    Config.setConnectionProtocol( Protocol.PIPE );
    Config.setPipeName( "myApp_jod_pipe"  );
    // kill any task that takes longer than 2 minutes 
    Config.setTaskExecutionTimeout((2 * 60 * 1000));
    Manager = Config.buildOfficeManager();
    // ....
</cfscript>

You could also connect to an external instance that is already running using the ExternalOfficeManagerConfiguration class instead. Though when working with an external process (ie controlled elsewhere ), obviously you do not start() or stop() it. Just connect to it.

Web Application / Servlet
Though JODConverter can be used strictly as a java library, there is also a sample web application/servlet available. It is not currently part of the distribution jar, but you can find it under the project source tab. Just be aware the web application does not fully handle conversions to HTML out-of-the-box. The servlet will produce a single html file, minus any images. As mentioned in the FAQ's, that is by design. The best way to handle images really "..depends on your particular requirements". So the implementation of image handling is deliberately left up to you. For more details see the FAQ's.

OpenOffice Quirks
Obviously, OpenOffice has some quirks of its own. Though it does a pretty good job with most documents, it is not perfect. It almost certainly will not be able to convert everything you throw at it. So any conversion code should definitely incorporate some solid error handling.

CF9 Quirks
I was very curious to see how well all of the pieces worked together under CF9. As I expected, there were a few quirks.

For whatever reason, things only worked smoothly when the JODConverter's instance of OpenOffice was started after CF9 started its instance. In other words, I had to run a small Word to Pdf conversion first, to force CF to start its OpenOffice process. Then afterward start the JODConverter's instance. When they were not started in that order, all sorts of errors ensued. Both from CF and the JODConverter.

While I had success with socket and external connections, I had zero luck getting a separate named pipe connection to work alongside CF9's instance. Initially I thought it should be possible. But I am not very well versed in UNO or named pipes. So that could just be ignorance on my part. If anyone does know the answer, one way or the other, let me know.

Conclusions
All in all, I was pleased with my initial tests. Though I am still not completely comfortable with the automatic restart feature. I can definitely see its value. OpenOffice can, and on occasion, does crash. But anything that automatically revives itself after death, tends to make me think of zombies ;) So I think I will need to study it (and how to best manage OOO instances) further.

...Read More
Posted by cfSearching at 4:27 PM ::  8 comments
Labels: ,
Subscribe to: Posts (Atom)

  © Blogger templates The Professional Template by Ourblogtemplates.com 2008

Header image adapted from atomicjeep