Tuesday, November 24, 2009

ColdFusion 8: Blowfish Encryption (Something smells fishy .. and it is not the algorithm)

A while back I saw a thread on the adobe forums about an issue with Blowfish encryption. The issue being the results from ColdFusion's encrypt function seemed to differ from other languages. Hardly uncommon given the "delicate" nature of encryption. What grabbed my attention was that the results from ColdFusion and Java were also different. Not what I would have expected.

Take the simple example below. It sure seems like the same text and key are used for both CF and Java encryption. Yet the results are different. However, if you compare the java results with those from the online tools (mentioned in the adobe thread) they are all the same. The odd one out here is ColdFusion.

Note: These examples were chosen for simplicity. They are NOT a good example of strong encryption (usage of ECB, etcetera ...)

ColdFusion

<cfset myText        = "guess this!1 2 3" />
<cfset myKey         = generateSecretKey("Blowfish", 32) />
<cfset cfEncrypted   = Encrypt(myText, myKey, "Blowfish/ECB/NoPadding", "Hex") />

<!--- results (using 

tags because the blogger eats line breaks!) ---> <strong>ColdFusion:</strong><hr /> <cfoutput> <p><strong>MyText</strong> : #myText#</p> <p><strong>MyKey</strong> : #myKey#</p> <p><strong>CF Encrypted</strong> : #cfEncrypted#</p> </cfoutput>



Java
<!---  generate a secret key --->
<cfset SecretKeySpec     = createObject("java", "javax.crypto.spec.SecretKeySpec") />
<cfset myKeyBytes         = CharsetDecode(myKey, "utf8") />
<cfset keySpec             = SecretKeySpec.init(myKeyBytes, "Blowfish") />
<!--- get a cipher instance for encrypting --->
<cfset Cipher             = createObject("java", "javax.crypto.Cipher") />
<cfset encryptor         = Cipher.getInstance("Blowfish/ECB/NoPadding") />
<cfset encryptor.init(Cipher.ENCRYPT_MODE, keySpec) />
<!--- do the encryption --->
<cfset myTextBytes         = CharsetDecode(myText, "utf8") />
<cfset javaEncrypted     = encryptor.doFinal(myTextBytes) />

<!--- results --->
<cfoutput>
<p><strong>Java Encrypted</strong>  : #BinaryEncode(javaEncrypted, "hex")# </p>
</cfoutput>

Sample Results:


Since I am definitely not an expert on encryption, I just figured I was missing something. Eventually I did get the CF and java results to match by converting the secret key to binary and then back to base64.

<cfset myText        = "guess this!1 2 3" />
<cfset myKey         = generateSecretKey("Blowfish", 32) />
<cfset magicKey      = BinaryEncode(CharsetDecode(myKey, "utf8"), "base64") />
<cfset cfEncrypted   = Encrypt(myText, magicKey, "Blowfish/ECB/NoPadding", "Hex") />

The new results clearly showed my magicKey now matched the java keySpec value. So of course the encrypted strings also matched.


But I am not sure why that the extra step necessary. Supposedly the value returned from GenerateSecretKey() is already base64 encoded. Also, it seems to work correctly in reverse (ie when the secret key is generated from the javax.crypto.KeyGenerator).

So am I missing something here, or is this a "feature"?


Update: Yep! I was obviously staring at this for too long. I missed something obvious. I should have used BinaryDecode(myKey, "base64") to get the java key bytes and vice versa. Now it make sense! ;)

As mentioned in the comments, using CBC mode is a bit different. As it requires an IV (Initialization Vector). Here is a basic example using CBC mode. Though in practice, you probably would not use the same IV each time as I am doing here. I also included a lot of extra debugging output. So you can verify the values used by both ColdFusion and Java are the same.

<!---
    ENCRYPT WITH COLDFUSION
--->
<cfset algorithm      = "Blowfish/CBC/PKCS5Padding">
<cfset myText        = "can you guess this?" />
<cfset myKey         = generateSecretKey("Blowfish", 32) >
<cfset myIV             = CharsetDecode("12345678", "utf8")>
<cfset cfEncrypted   = Encrypt(myText, myKey, algorithm, "base64", myIV) />

<!--- results (using <p> tags because the blogger eats line breaks!) --->
<strong>ColdFusion (Encrypted):</strong><hr />
<cfoutput>
    <p><strong>MyText</strong> : #myText#</p> 
    <p><strong>MyKey</strong> : #myKey#</p>  
    <p><strong>MyIV</strong> : <cfdump var="#myIV#"></p>  
    <p><strong>CF Encrypted</strong> : #cfEncrypted#</p> 
</cfoutput>
</p>

<!---
    ENCRYPT WITH JAVA
--->
<cfset SecretKeySpec       = createObject("java", "javax.crypto.spec.SecretKeySpec") />
<cfset myKeyBytes          = BinaryDecode(myKey, "base64") />
<cfset keySpec             = SecretKeySpec.init(myKeyBytes, "Blowfish") />
<cfset ivSpec               = createObject("java", "javax.crypto.spec.IvParameterSpec").init( myIV )>
<cfset Cipher              = createObject("java", "javax.crypto.Cipher") />
<cfset encryptor           = Cipher.getInstance( algorithm ) />
<cfset encryptor.init(Cipher.ENCRYPT_MODE, keySpec, ivSpec) />
<cfset myTextBytes         = CharsetDecode(myText, "utf8") />
<!--- convert encrypted byte array to base64 string --->
<cfset javaEncrypted       = BinaryEncode( encryptor.doFinal(myTextBytes), "base64") />

<!--- results (using <p> tags because the blogger eats line breaks!) --->
<strong>Java Encrypted:</strong><hr />
<cfoutput>
    <p><strong>myTextBytes (utf8)</strong> : #CharsetEncode(myTextBytes, "utf8")#</p>  
    <p><strong>keySpec</strong> : #BinaryEncode(keySpec.getEncoded(), "base64")#</p> 
    <p><strong>ivSpec (raw)</strong> : <cfdump var="#ivSpec.getIV()#"></p>  
    <p><strong>ivSpec (utf8)</strong> : #CharsetEncode(ivSpec.getIV(), "utf8")#</p>  
    <p><strong>Java Encrypted</strong>  : #javaEncrypted# </p>
</cfoutput>

Now just to prove the results are compatible, you can decrypt the ColdFusion results with Java (and vice versa).

<!---
    DECRYPT *COLDFUSION* RESULT WITH JAVA
--->
<cfset SecretKeySpec     = createObject("java", "javax.crypto.spec.SecretKeySpec") />
<cfset myKeyBytes        = BinaryDecode(myKey, "base64") />
<cfset keySpec           = SecretKeySpec.init(myKeyBytes, "Blowfish") />
<cfset ivSpec             = createObject("java", "javax.crypto.spec.IvParameterSpec").init( myIV )>
<cfset Cipher            = createObject("java", "javax.crypto.Cipher") />
<cfset decryptor         = Cipher.getInstance( algorithm ) />
<cfset decryptor.init(Cipher.DECRYPT_MODE, keySpec, ivSpec) />
<!--- convert our encrypted (base64 encoded) string to a byte array --->
<cfset encryptedBytes     = BinaryDecode( cfEncrypted , "base64") />
<!--- the decrypted bytes will be in UTF8 --->
<cfset javaDecrypted      = CharsetEncode( decryptor.doFinal( encryptedBytes ), "utf8") />

<cfoutput>
    <p><strong>Java Decrypted</strong>  : #javaDecrypted# </p>
</cfoutput>

<!---
    DECRYPT *JAVA* RESULT WITH COLDFUSION
--->
<!--- decrypt (base64 encoded) encrypted string --->
<cfset cfDecrypted   = Decrypt(javaEncrypted, myKey, algorithm, "base64", myIV) />

<cfoutput>
    <p><strong>ColdFusion Decrypted</strong>  : #cfDecrypted# </p>
</cfoutput>

24 comments:

inj November 25, 2009 at 1:34 AM  

Thank you, very interesting and usefull.

ianus March 10, 2010 at 2:18 AM  

I'm running a ColdFusion 7 server and am having exactly this problem. I was thrilled to find your solution, as I need to communicate with a 3rd party API and they can't decode the info I'm sending them.

Your solution (the last 4 line code snippet on the page) sadly doesn't work for me :(

The GenerateKey() function in CF7 only accepts one param; the algorithm, so instead I've used a 32 character string as the key.

I'm getting this error: There has been an error while trying to encrypt or decrypt your input string: Unsupported keysize or algorithm parameters.

which is thrown out when Encrypt() is called.

Do you have any idea what's going on here?

cfSearching March 18, 2010 at 2:01 PM  

@ianus,

What version of 7? Some additional parameters for the Encrypt() function were added in 7.0.1.

http://kb2.adobe.com/cps/546/e546373d.html

Also, did you try using the java method above? That should work for any version. At least with a 32 bit key.

-Leigh

ianus March 19, 2010 at 1:31 AM  

Hi Leigh, thanks for your answer - interesting that the additional parameters should be accepted; we're using CF v7.0.2.142559 but I get this error: "The function takes 1 parameter".

Anyway, that aside, the java alternative (which I was sure would give me what I wanted) throws the error "Unsupported keysize or algorithm parameters" for line 08.

Am I right in saying that this has something to do with the "Unrestricted JCE Policy files"?

We're toying with the idea of upgrading to CF9, but again, thanks for any light you can shed on this :)

cfSearching March 19, 2010 at 8:18 AM  

@ianus,

It might, I do not recall the limits off-hand. I think I have MX 7 around somewhere. Let me try a few things.

-Leigh

cfSearching March 19, 2010 at 9:04 AM  

@ianus,

With a little more caffeine in my brain, yes the "unsupported key size.." error does sound like the unlimited strength files might not installed. I tried the java method with MX7 and it worked fine for me.

Of course it could be other things too. But if you are using a larger than default key size, that is the likely culprit.

-Leigh

ianus March 29, 2010 at 3:15 AM  

@Leigh, thanks a lot for your help; I haven't cracked it yet, but as soon as I've figured out what exactly is wrong with the server setup I'll let you know :)

Anonymous,  July 23, 2010 at 1:21 PM  

If you change the text you are encrypting to something that is NOT in multiples of 8 bytes, the example breaks. It works only because you have 16 characters in the string you are encrypting.

Changing it from nopadding to pkcs5padding resolves that for me.

"Blowfish/ECB/pkcs5padding"

I'm no encryption guru either so that solution may or may not be a horrible idea. I'm just trying to put something together and working heavily off of your example.

Anonymous,  July 23, 2010 at 1:30 PM  

The keysize (number 32) in the generateSecretKey function does not mean that the resulting key will have that many bytes or characters. It is the number of BITS. so generateSecretKeyt("blowfish", 32) would actually be an 8 byte request, not 32. (1 byte = 8 bits). Hope that helps with

cfSearching July 24, 2010 at 3:51 PM  

@Anonymous,

If you change the text you are encrypting to something that is NOT in multiples of 8 bytes, the example breaks...

Yes, that is what "NoPadding" means. With block ciphers, the text length (in bits) must be an exact multiple of the cipher's block size. So when you use NoPadding, you are basically saying "..I understand the text must be a certain length, and I've taken care of it myself".

Since most people do not want to handle padding themselves, they use a setting like "Pkcs5padding". It automatically pads the input string to the correct length, using some pre-established algorithm. So using "Pkcs5padding" (or some other padding mode) is the right thing to do ;)

Though as I mentioned in the entry, ECB is not very secure. So using another mode, like CBC, is preferable.


generateSecretKey... It is the number of BITS

Yep. That actually is in the documentation. Though I misread it too the first time around. I thought the keysize was in bytes too.. it is not ;)

-Leigh

Anonymous,  July 29, 2010 at 8:07 AM  

Thanks for the clarification on padding. That makes complete sense now.

I tried your example but changed ECB to CBC and the resulting encrypted strings were different again.

My biggest issue right now is key size.

If I use generateSecretKey("Blowfish") CF and Java both bark at the size of the key claiming it is an illegal size.

Since the user will be providing their own blowfish keys that they already use in their processes, I'm trying to not limit the keysize in anyway I don't have to.

Anonymous,  July 29, 2010 at 11:44 AM  

@ianus

This is what I did to resolve the key size issues. Hopefully it helps.

https://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_Developer-Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=jce_policy-6-oth-JPR@CDS-CDS_Developer

cfSearching July 29, 2010 at 12:00 PM  

@Anonymous 1,

Well the key sizes usually have to be a multiple of the block size, and have a limited range. So you cannot just use any value (See the allowable lengths in the "Block size table" section)


Also, for the really large key sizes you must install the unlimited encryption policies. Someone kindly posted the link for those files just above this post.

As far as the values not matching, it may be due to a difference in IV values. I believe CBC uses an IV (Initialization Vector). If you do not specify one explicitly, I think CF (and java) create one automatically. My guess would be the auto-generated IV's are random and it is doubtful the two will match. So that might explain the difference you are seeing.


-Leigh

Bobby H July 30, 2010 at 8:33 AM  

Thanks again for further clarification Leigh. You have been a huge help as this is all pretty new ground for me.

You'd think CF's generateSecretKey() would be smart enough to generate keys of valid sizes though. After installing the unlimited key strength policy jars, no keysizes below 32 works in generatesecretkey("blowfish", XX).

All of the "Anonymous" posts prior to your last post were me by the way. I was just too lazy to create an account :-)

~Bobby H.

Anonymous,  July 30, 2010 at 11:13 AM  

If you have sample code to decrypt with java, that would be very useful too!

cfSearching July 30, 2010 at 1:51 PM  

@Bobby H,

After installing the unlimited key strength policy jars, no keysizes below 32 works in generatesecretkey("blowfish", XX).

I do not remember what the behavior is without the unlimited jars installed. But is it possible CF just ignored key sizes less than 32, and returned 32 by default? Because I was under the impression (perhaps mistaken) that the allowable range for Blowfish keys was something like 32/40 to 448 bits.

All of the "Anonymous" posts prior to your last post were me by the way. I was just too lazy to create an account :-)

Ah! I figured all of them were you, except maybe the last one. But I know exactly what you mean. Too many accounts to keep track of ..

Let me throw up an encrypt/decrypt example of CBC with an IV.

-Leigh

cfSearching July 30, 2010 at 2:45 PM  

@Bobby H,

Done! I updated the entry to include a CBC encrypt/decrypt example.

-Leigh

Bobby H July 30, 2010 at 3:19 PM  

You may be right about defaults for generateSecretKey under 32. If i recall, they all looked about the same length. I'll have to try another server that I haven't installed the policy files on yet.

Thanks for the decryption example. Very much appreciated. BinaryDecoding the key appears to be where I was screwing it up.

Thanks again for the examples, the clarifications and the help. :-)

cfSearching July 30, 2010 at 4:20 PM  

BinaryDecoding the key appears to be where I was screwing it up.

... exactly what happened to me too ;) Anyway, I am glad I could help!

-Leigh

ianus October 25, 2010 at 3:27 AM  

Hi Leigh,

This has turned into a really useful thread! I was able to fix all problems I came across, so thanks for your help.

I now have a new 'issue', which is semi-related so I'd be interested to hear your opinion.

I have a string (comprised of a userID and a date/time stamp), which I then encrypt using Encrypt(inputString, myKey, "Blowfish/ECB/PKCS5Padding", "Hex").

In order to interface with a 3d party I have to perform the following:

I have to convert each character pair within the resultant string into a HEX value. Each HEX value is then represented as an integer before being output as an ASCII character.

All the ASCII characters combine to form the Bytestring which is then converted to Base64, URL encoded and then sent off (phew!)

It all works seamlessly, APART FROM when the original CFEncrypted string contains a "00".

The HEX value 00 translates as the integer (via function InputBaseN) 0 which then refuses to translate correctly into an ASCII character!

The resultant url string is messed up and the 3d party is unable to decipher it.

**

It's worth mentioning that I do declare: at the top of the page.

If anyone can help that would be great! If not (sorry for the length of this comment) then tell me to push off to StackOverflow..

cfSearching October 25, 2010 at 4:29 AM  

Hi @ianus,

Without much caffeine in my system .. a "0" in ASCII is usually a null byte (not the same as a java null). That might be the problem ..

-Leigh

ianus October 25, 2010 at 4:58 AM  

OK, now we're getting somewhere.. The resultant Base64 string on any string where "00" is present contains 2 padding characters.

If all's well, there should be just 1. I guess the null generated by the 00 means the byteString is one character short.

Is there any way of avoiding having this 00 appearing in the CF Encrypted string? It crops up consistently if the datestamp is October, so I'm getting plenty of errors for the time being!

cfSearching October 25, 2010 at 5:14 AM  

While I can see how null bytes might be a problem, I am not clear on your actual vs. expected results. Can you post a small example?

cfSearching October 25, 2010 at 5:31 AM  

If you prefer, feel free to email it @ cfsearching / yahoo.

  © Blogger templates The Professional Template by Ourblogtemplates.com 2008

Header image adapted from atomicjeep