In The Zone With Time

The account-expires attribute in Active Directory carries a value of time in 100 nanosecond units since January 1, 1601 00:00 UTC. In fact quite a lot of Microsoft products are now using this epoch. Java and Unix use January 1, 1970 00:00 UTC as their epoch.

This is important information if you’re ever going to develop code in ColdFusion that needs to, for example, figure out whether or not a domain account has expired.

Another piece of information that’s handy to have is an understanding of how ColdFusion handles large numbers. Numeric values are handled as signed 32-bit values that can range from -2,147,483,648 to 2,147,483,647. ColdFusion can handle much larger numbers, but it does so by converting numeric values into strings and then using its own library to handle math operations based on these strings. And it stores these large values in scientific notation. The problem with this is you start to lose precision. For example, if you run the following CF snippet:

<cfoutput>#( 2147483648 * 2147483648 )#</cfoutput><br>
<cfoutput>#( 2147483648 * 2147483648 + 1)#</cfoutput>

The output is the same for both operations, “4.61168601843E+018“. This loss of precision makes it impossible to calculate the exact time account-expires refers to. So what else can we do?

One trick that sort of works is to divide the value in account-expires
by 10000000 to convert from 100 nanoseconds to seconds. This will significantly shorten the number. So let’s try that on an account-expires value of “129247307050000000” (that’s 7/27/10 @2:58.25 EDT). The line of CF looks like this:

<cfoutput>#DateAdd( "s", ( 129247307050000000 / 10000000 ), "01/01/1601 12:00 AM")#</cfoutput>

Try to run this code and you get the error message “Cannot convert the value 1.2924730705E10 to an integer because it cannot fit inside an integer.”. It turns out that DateAdd() can’t handle a number larger than 2147483647 (the limit of a 32-bit signed integer). 2,147,483,647 seconds is about 70 years of time. To use DateAdd() we need to make the account-expires value much smaller. We could, for example, calculate the account-expires value with respect to a different epoch, one that’s within 70 years of the time referenced by account-expires. We could use any date we want, but I’m going to stick with Java’s epoch of January 1, 1970 to keep things in-line with what’s probably the most common epoch among computer systems.

The number of seconds between January 1, 1601 and January 1, 1970 is 11,644,473,600. If we plug this into the code above we get this:

<cfoutput>#DateAdd( "s", ( 129247307050000000 / 10000000 ) - 11644473600, "01/01/1970 12:00 AM")#</cfoutput>

Run this and we get a result! Specifically {ts '2010-07-27 19:58:25'}. Remember this is in UTC, we’ll need to convert to our local timezone. We’ll do that in just a moment. But first…

You should have recognized by now that this solution won’t work forever. It will fail whenever the data in account-expires is after January 19, 2038. Perhaps this doesn’t bother you since whatever applications you’re developing now will surely be either obsolete or running on a more modern ColdFusion system where DateAdd() can deal with larger numbers before account-expires with that large a value are in use.

Perhaps. But I hate making assumptions like that. Plan for the future! Assume nothing!

If we want lasting code free of the 2038 limitation we’ll need to steer clear of ColdFusion date and time functions. We can do this by using Java objects, specifically java.util.Calendar to manage date and time, and java.math.BigInteger to handle the very large numbers we’ll be working with.

The process is fairly straightforward, but there are a few things to watch out for. Java time is measured in milliseconds, not 100 nanoseconds, so we’ll have to do some conversion. Java and Microsoft’s epochs are different. Calendar objects with a date and time earlier than Java’s epoch will have a negative millisecond value. And we need to calculate this date/time with respect to UTC (GMT timezone) not the local timezone.

First create a BigInteger object. Initialize it to the value of account-expires, and then convert it to milliseconds as that’s the unit of time Java likes to work with.

01: <cfset variables.bigInt = CreateObject( "java", "java.math.BigInteger" ) />
02: <cfset variables.expTime = variables.bigInt.init( JavaCast( "String", "#arguments.accountExpires#" )) />
03: <cfset variables.expTime = variables.expTime.divide( variables.bigInt.valueOf( "10000" )) />

You can initialize BigInteger objects with strings and numerical values. I’m specifically casting account-expires as a string so that ColdFusion doesn’t treat it like an integer and try to put it into scientific notation. Note in line 3 the use of BigInteger’s valueOf() function. This takes a string and returns a BigInteger value. Math functions for BigInteger objects require BitInteger arguments, thus we need to pass all values (even static ones) through valueOf() instead of JavaCast().

Next we set up the calendar object. This means creating a java.util.TimeZone object, a java.util.Calendar object, and then initializing the calendar to the GMT timezone. Once the calendar object is initialized we set it to Microsoft’s epoch.

04: <cfset variables.tz = CreateObject( "java", "java.util.TimeZone" ) />
05: <cfset variables.Calendar = CreateObject( "java", "java.util.Calendar" ) />
06: <cfset variables.gCal = variables.Calendar.getInstance( tz.getTimeZone( "GMT" )) />
07: <cfset variables.gCal.set( 1601, 0, 1, 0, 0, 0 ) />

Note that the second argument of the calendar object’s set() function is 0. Java’s calendar object’s month index begins with 0 for January instead of 1. This is something to be aware of whenever dealing with Java calendar objects.

This next step requires a little explanation. variables.expTime is now in milliseconds thanks to line 3. I’m going to add the value of the calendar, in milliseconds, to variables.expTime. Because the calendar is set to a date before Java’s epoch it will actually be a negative number. The addition operation is actually a subtraction operation. The resulting value of variables.expTime will be the number of milliseconds since Java’s epoch.

08: <cfset variables.expTime = variables.expTime.add( variables.bigInt.valueOf( "#variables.gCal.getTimeInMillis()#" )) />

All we need to do now is reset the calendar to the value we now have in variables.expTime, then convert it to our local timezone and we’re done!

09: <cfset variables.gCal.setTimeInMillis( variables.expTime.longValue() ) />
10: <cfset variables.gCal.setTimeZone( tz.getDefault() ) />

Last, but not least, we convert the calendar object into a ColdFusion date/time object with ColdFusion’s CreateDateTime() function.

11: <cfset variables.expDateTime = CreateDateTime(
variables.gCal.get( variables.gCal.YEAR ),
variables.gCal.get( variables.gCal.MONTH ) + 1 ,
variables.gCal.get( variables.gCal.DAY_OF_MONTH ),
variables.gCal.get( variables.gCal.HOUR_OF_DAY ),
variables.gCal.get( variables.gCal.MINUTE ),
variables.gCal.get( variables.gCal.SECOND )
) />

Note that I’m adding 1 to the month variable because in Java 0 = January, but in ColdFusion 1 = January.

And there you have it. Converting from Microsoft’s 01/01/1601 100 nanosecond based timestamps to something a bit more usable in ColdFusion.

Of course you’ll need a reverse of this process as well, and I’ve already got that for you:

01:  <cfset variables.tz = CreateObject( "java", "java.util.TimeZone" ) />
02:  <cfset variables.Calendar = CreateObject( "java", "java.util.Calendar" ) />
03:  <cfset variables.gCal = variables.Calendar.getInstance( tz.getDefault() ) />
04:  <cfset variables.gCal.set(
Year( arguments.Date ),
Month( arguments.Date ) - 1,
Day( arguments.Date ),
Hour( arguments.Date ),
Minute( arguments.Date ),
Second( arguments.Date )
) />
05: <cfset variables.bigInt = CreateObject( "java", "java.math.BigInteger" ) />
06: <cfset variables.expTime = variables.bigInt.init( JavaCast( "String", "#variables.gCal.getTimeInMillis()#" )) />
07: <cfset variables.expTime = variables.bigInt.divide( variables.bigInt.valueOf( "1000" )) />
08: <cfset variables.expTime = variables.expTime.add( variables.bigInt.valueOf( "11644473600" )) />
09: <cfset variables.expTime = variables.expTime.multiply( variables.bigInt.valueOf( "10000000" )) />
10: <cfreturn variables.expTime.longValue() />

You should be able to follow along based on my explanation earlier in the article. However there are a couple extra bits I’d like to point out.

The calendar is initialized to the local timezone. We don’t have to worry about converting back to UTC/GMT because the calendar function getTimeInMillis() will give you the number of milliseconds from Java’s epoch, which is in UTC.

On line 8 I’m adding the number of seconds from Microsoft’s epoch to Java’s epoch. You’ll also notice the line before that I’m dividing by 1000 to convert time from milliseconds to seconds and after I add the epoch difference I’m converting to 100 nanoseconds by multiplying by 10000000. So why not skip the conversion to seconds and simply add three more zeroes to the end of the epoch difference (making the time in milliseconds) and multiplying by 10000?

The reason is that there’s a problem with precision and the resulting value will vary by a few hundred milliseconds. Since the times I’m working with are in seconds, those extra milliseconds could be ignored. But by dividing, adding, and then multiplying like this I don’t get the millisecond variances. I’m sure there’s better stuff on the ‘net to explain precision with BigInteger objects if you’re inclined to investigate further.

Before you go I have one piece of information I learned while working on this particular topic.

ColdFusion treats all date and time values as if they are in the local timezone. Specifically, if your timezone has daylight savings then all dates are treated as if they have daylight savings, including GMT/UTC times which do NOT have daylight savings time. DateConvert() will not protect you from this issue. There’s a deeper explanation here.

My solution is to check if GetTimeZoneInfo().isDSTon is TRUE. If it is then I need to add (or subtract) 1 hour from my UTC time. Note that this is only needed when converting between UTC and local timezones with ColdFusion date and time functions. The above code specifically stays away from ColdFusion date and time functions so this isn’t a problem with my example. But it is something to keep in the back of your head when you start to notice dates are being calculated an hour off the time they should be.

A bit of vinegar to go with the SOAP.

Not long ago I wrote a 3-part series on using SOAP over HTTPS with ColdFusion. My final solution was to create Java objects directly, bypassing ColdFusion’s CFHTTP tag.

I have since found a subtle flaw with this implementation.

It’s not in the code, but in the  JVM. ColdFusion 8 ships with an older JVM. I recently upgraded our JVM to a more current version in an attempt to resolve a timezone bug. In doing so my SOAP application stopped working.

A little research led me to this article about a TLS bug in Java that could lead to a man-in-the-middle exploit. It appears the way I’m performing my SOAP operation triggers a TLS/SSL renegotiation when it receives a response from the external server.

The short answer is to add the following line to ColdFusion’s JVM arguments:

-Dsun.security.ssl.allowUnsafeRenegotiation=true

This does resolve the problem, but it apparently leaves the JVM vulnerable to MITM attacks. There is another bit of code in that article which shows how to change the allowUnsafeRenegotiation flag on-the-fly. I added this to my ColdFusion code, but changing the flag didn’t appear to have any effect.

If anyone else has played around with this particular problem I’d love to hear about it.

For now I”ve left the JVM in its vulnerable state as we only make HTTP requests from the JVM for a couple of applications and neither of them carry personally identifiable information.

ColdFusion Regex Backreferences

I want to take a string that contains HTML and increment all heading tags by 1. For example, I want to turn <h3>…</h3> into <h4>…</h4>.

I might be able to accomplish this with a long ReplaceList command. However I decide against this as I’ve run into problems with ReplaceList not replacing items in the order it claims to in the documentation. Plus I would have to hard-code every heading tag (both opening and closing) and that would just look ugly. So let’s try and be clever.

This sounds like a job for regular expressions (regex). I opt for the REReplace function over REReplaceNoCase because REReplaceNoCase seems lazy to me, and it forces me to specifically take case into consideration. I like that. Deal with potential problems with input now instead of waiting for it to become a problem later.

So my search string will be something like “<(/?[hH])([1-6])“. This will search for both opening and closing heading tags and I can use the number of the heading as a backreference (that is, I can refer to it in my replacement string). I’m only search for H1 through H6 because the HTML spec clearly states there are only 6 headings, H7 and so on aren’t valid HTML tags.

So my REReplace line might look something like REReplace( variables.input_string, “<(/?[hH])([1-6])”, “\1#Val( \2 + 1 )#”, “ALL” ).

But this will not work.

The reason being that ColdFusion will evaluate all CF variables before performing regex operations. So the backreference to the number of the heading (the “\2” in the example above) will be interpreted as the literal string “\2”. This will trigger a parsing error.

So the workaround is a bit ugly, but still more elegant than just one big ReplaceList function, and that is to loop from 1 to 5 and perform the regex operations one heading at a time. Except there’s another gotcha here. If my loop starts at 1 and works up to 5, I will replace all H1 tags to H2. In the second loop, all H2 tags, including the former H1 tags, will become H3 tags. In the end my HTML will be all H6 tags. The fix is simple, just loop backwards, from 5 to 1. Easy! The code would look something like this:

<cfloop from=”5″ to=”1″ index=”variables.h” step=”-1″>
<cfset variables.input_string = REReplace( variables.input_string, “<(\/?[hH])#variables.h#”, “<\1#Val( variables.h + 1 )#”, “ALL” ) />
</cfloop>

Simple. Except this code won’t work. All the heading tags get dropped for some reason.”<h4>” becomes “<>”. What’s going on?

Refer back to how ColdFusion will evaluate all variables before performing the regex operation. In the replace string I use the backreference “\1”. But ColdFusion evaluates the Val() operation before performing the regex operation, so the backreference the regex operation sees (for an H1 replacement) is “\12”. The regex operation looks for a 12th backreference, which doesn’t exist, so the replaced string is empty.

In proper regex engines, this can be easily fixed in a few different ways. I could explicitly specify the base of the number (regex engines typically accept octal, hex, and decimal base numbers) with something like \x01 (hex-base). Or I could wrap the number in curly brackets like \{1}. But ColdFusion doesn’t have a proper regex engine, but a rather bastardized regex engine.

I need to separate my backreference (\1) from my replacement variable (#Val( variables.h + 1 )#). A space won’t work because my H4 tag becomes “H 4”, which isn’t an HTML tag. Can’t use HTML entities as this isn’t text to be output to the user, it’s HTML that the HTML engine needs to interpret. So what can I do?

Well, here’s the trick I’ve decided to go with.

Looking at ColdFusion’s “Using backreferences” page I see references to special characters used to make letters uppercase (\u) or lowercase (\l). I also see I can perform this case change over a string of characters by inserting \U or \L at the start of the case change, and then insert a \E where the case change should end. This is what I need. These are single, alpha-character special regex characters that do not generate output by themselves. I can use a “\U\E” sequence to separate my backreference from my variable! So my final code looks something like this:

<cfloop from=”5″ to=”1″ index=”variables.h” step=”-1″>
<cfset variables.input_string = REReplace( variables.input_string, “<(\/?[hH])#variables.h#”, “<\1\U\E#Val( variables.h + 1 )#”, “ALL” ) />
</cfloop>

A quick test shows me I don’t even need the \U, I can stick with just a \E to get the job done. But I’ll keep the \U for completeness.

The reason I start with 5 and not 6 is that I don’t want to create H7 tags (which is not valid HTML) so H6 tags will remain H6 tags.

And now we come to the end of my post. This is where someone posts a comment and points out that my solution is, in fact, unnecessary because there IS a mechanism in ColdFusion’s regex engine to handle such a situation and I just didn’t RTFM as closely as I should have. Well, I truly welcome such a revelation should it exist. If not, perhaps Adobe could look into beefing up their regex engine a bit.

Washing Client Certs in ColdFusion with SOAP – Part 3

Continuing from part 2, I’ve got my SOAP request working, but the solution is limited to the OS platform and I don’t like the security implications. What I would really like is a solution that works without the need of a custom tag that is OS dependent. If only CFHTTP wasn’t broken.

I decided to look at what I already had available; namely Java. ColdFusion runs on top of Java and it’s possible to create Java objects from within ColdFusion.

I’ll spare you the grueling details, but I did get caught up at one point playing around with the Java keystore and importing my client certificate into that keystore. Using things like keytool and OpenSSL I tore apart and reconstructed my client certificate and my Java keystore. Ultimately I didn’t need to do any of that and should have just left the keystore alone.

The path to enlightenment began when I found an example of Java socket operations using  the javax.net.ssl.SSLContext object. The example code I was looking at used this object to create regular sockets, not SSL-enabled sockets. I looked into this object and found a few more examples using it to create SSL connections. A bit of back and forth between these examples and the online Java API spec resulted in the discovery that I could load a PKCS12 file as a keystore and pass that to the SSLContext object then create sockets from that SSLContext. This looked promising.

After a bit of trial and error I finally had something that worked!

Here’s the relevant portion of code. I’ll go line-by-line afterwards.

<cfscript>
  // 01: create input file stream from certificate
  variables.ksf = CreateObject( "java", "java.io.FileInputStream" ).init( variables.cert_file );

  // 02: create keystore object from certificate
  variables.ks = CreateObject( "java", "java.security.KeyStore" ).getInstance( "PKCS12" );
  variables.ks.load( variables.ksf, JavaCast( "String", variables.cert_password ).toCharArray() );

  // 03: create key manager factory out of keystore
  variables.kmf = CreateObject( "java", "javax.net.ssl.KeyManagerFactory" ).getInstance( "SunX509" );
  variables.kmf.init( variables.ks, JavaCast( "String", variables.cert_password ).toCharArray() );

  // 04: create SSL context object using key manager factory
  variables.sslc = CreateObject( "java", "javax.net.ssl.SSLContext" ).getInstance( "TLS" );
  variables.sslc.init( kmf.getKeyManagers(), JavaCast( "null", "" ), JavaCast( "null", "" ) );

  // 05: get SSL socket from SSL context
  variables.factory = sslc.getSocketFactory();

  // 06: connect to the server via SSL
  variables.sock = variables.factory.createSocket( variables.server_address, variables.server_port);
  variables.sock.startHandshake();

  // 07: create input and output streams to read from and write to the socket
  variables.sout = variables.sock.getOutputStream();
  variables.out = createObject( "java", "java.io.PrintWriter" ).init( variables.sout );
  variables.sinput = variables.sock.getInputStream();
  variables.inputStreamReader = createObject( "java", "java.io.InputStreamReader" ).init( variables.sinput );
  variables.input = createObject( "java", "java.io.BufferedReader" ).init( variables.InputStreamReader );

  // 08: send the HTTP request over the socket
  variables.out.println( Trim( variables.request_header ) );
  variables.out.println();
  variables.out.println( Trim( variables.soap_envelope ) );
  variables.out.println();
  variables.out.flush();

  // 09: read the response from the server
  variables.result = "";
  do {
    variables.line = input.readLine();
    variables.lineCheck = IsDefined( "variables.line" );
    if ( variables.lineCheck ) {
      variables.result = variables.result & variables.line & Chr(13) & Chr(10);
    }
  } while ( variables.lineCheck );

  // 10: close the connection
  variables.sock.close();
</cfscript>

So let’s work through this.

01: The variable variables.cert_file contains the full path to the client certificate. This line opens that file for reading.

02: This creats a java.security.KeyStore object that expects a PKCS12 format and then loads the client certificate into the keystore. Note that the second parameter of the KeyStore.load() method is the password of the PKCS12 file. You must pass this as a character array and not as a string, which is why the password must be cast as a String object whose toCharArray() method is called to obtain that character array.

03: This creates a javax.net.ssl.KeyMangerFactory object which is used by the SSLContext object to deal with the certificates in the keystore. Note that the second parameter of the init() is the password to the certificate in character array format. You must provide the password twice, once when creating the keystore, and then again creating the key manager factory.

04: Here we create the javax.net.ssl.SSLContext object. The getInstance() call tells what protocol to use with this object, in this case “TLS“. (Note: “SSLv3” also works. No idea which one I should be using. If one fails, try the other.) Once the object is instantiated we initialize it with a key manager from the key manager factory. The second and third parameters of the init() method could point to a trust manager and a source of randomness. It doesn’t appear that either is needed (this is what I saw in other examples) so I pass null values. Note that you have to use JavaCast() to pass a null.

05: Get a socket factory object from the SSLContext object. This factory will be used to create individual sockets using the parameters established with the SSLContext object.

06: Create a single socket from the socket factory by passing the server address (variables.server_address) and port (variables.server_port) we want to connect to. Then call startHandshake() which will create the connection to the server using SSL.

07: Create objects needed to read from and write to the socket we’ve created.

08: Here we send our SOAP envelope to the server. Note that since we’re doing everything by hand we also have to send the proper HTTP headers. Both the variables.request_header and variables. soap_envelope variables were created using the CFSAVEDCONTENT tag as shown in part 1. You want to create the envelope first so you know the correct content-length value for the HTTP header. Since I decided to stick with SOAP 1.1 the header also contains the SOAPAction field and the content-type is set to text/xml.

Here is an example of how I created the headers:

<cfsavecontent variable="variables.request_header">
POST <cfoutput>#variables.server_request_path#</cfoutput> HTTP/1.0
Host: <cfoutput>#variables.server_address#</cfoutput>
User-Agent: SOAP Washer/1.0
SOAPAction: "<cfoutput>#variables.soap_action#</cfoutput>"
Content-Type: text/xml
Content-Length: <cfoutput>#Len( Trim( variables.soap_envelope ))#</cfoutput>
</cfsavecontent>

Standard HTTP/1.0 headers. You can make the User-Agent field just about whatever you want.

Also worth mentioning is that I didn’t try to combine the two (HTTP headers and envelope) into a single variable. This is because how CFSAVEDCONTENT handles newline characters is different between ColdFusion 8 and ColdFusion 9. By keeping them separate this code is able to operate under either version.

The empty println() commands are there to send a newline. And the flush() call simply sends anything left in the buffers, thus completing the sending of our request.

09: This is where we read the response from the server. The response can only be read one line at a time, so we need a loop to keep reading one line at a time and appending it to the variable where we’ll store the entire response (variables.result).

A tricky thing working between ColdFusion and Java objects is that if you assign the return value of a Java object method to a ColdFusion variable and that return value is null then the ColdFusion variable is unset or deleted. Since the readLine() method returns null when there’s nothing more to read, the only way to know we’ve completed reading the response is to test if variables.line, where we store the return value of readLine(), still exists. If it doesn’t, we’ve reached the end of the response.

10: Close the connection

The ColdFusion variable variables.result will contain the entire response from the server, including the response HTTP headers. Meaning if you want to get a proper XML object from xmlParse() you’re going to need to first strip away those headers.

<cfset variables.temp = REFindNoCase( "content-length: ([0-9]+)", variables.result, 1, "True" )>
 <cfif variables.temp.len[1] GT 0>
  <cfset variables.length = Val( Mid( variables.result, variables.temp.pos[2], variables.temp.len[2] ))>
  <cfset variables.xmlResult = Trim( Right( RTrim( variables.result ), variables.length ))>
</cfif>

This should do the trick. Although you may want to throw in some code to strip out the status code to ensure a good transaction.

And there you have it. Handling transactions with a third-party web site using client certificates under ColdFusion using only native ColdFusion objects.

Washing Client Certs in ColdFusion with SOAP – Part 2

In part 1 I introduced you to basic SOAP consumption in ColdFusion. Let’s see where things go from there.

The task at hand was to integrate our system with a third-party site. I authenticate users locally, request a token from the third-party site that allows the user to access said site, and then redirect the user to that site with the token passed on the URL.

But how does the third-party site know to trust my requests and not the requests from other people? After all you don’t want people forging requests to access the third-party site as some other user. The solution for this third-party site is client certificates. A client certificate is either issued by the third-party site or you provide your public certificate to the third-party site and the third-party’s site is configured to trust the certificate. SOAP requests are then made over SSL using this certificate to confirm your identity and to encrypt the channel.

Starting with ColdFusion 8 CFHTTP accepts two parameters to afford the use of client certs, clientCert and clientCertPassword. The clientCert parameter points to a PKCS12 formatted file containing your public and private keys and, possibly, the certificate chain from the root certificate authority (such as VeriSign or Thawte) down to whoever issued your client certificate. The clientCertPassword parameter contains the password with which the PKCS12 file is encrypted.

A note to ColdFusion developers:
The PKCS12 must be encrypted with a password! For whatever reason (I believe a limitation in the underlying java.security.KeyStore object) your cert must have a password. This is never explicitly stated in the ColdFusion 8 documentation.

I obtained my client certificate and set out to start writing code to talk with this third-party.

The first problem I encountered was I would not be able to use the CreateObject() method covered in part 1. The reason being that there was no way to provide my client certificate to the object. So it’s back to the CFHTTP method.

The second problem I encountered had nothing to do with ColdFusion and everything to do with the documentation provided by the third-party. Turns out the header for the SOAP request changed considerably during development. I had been given some early development documents that did not reflect the current header structure. Once I realized the problem and obtained the current documentation I was able to correctly construct my SOAP request’s envelope header and…

I got another error.

But this time it wasn’t a normal SOAP request error. The CFHTTP object’s filecontent value contained nothing more than “Connection failure”. But the HTTP status code was 200, which indicates a successful request. Previous SOAP request errors would return a 403 status code. This was odd.

In searching the Adobe forums and the internet in general I found sparse comments about possible problems with CFHTTP handling SSLv3 sessions, although there wasn’t any sort of official comment or response to the few reports of this problem. I loaded up the developer edition of ColdFusion 9 on my own computer to see if perhaps this problem had been resolved with the latest copy of ColdFusion. It had not.

To confirm this as a problem with ColdFusion and not my client cert, or my SOAP request, I installed Apache and PHP locally and ran the equivalent PHP code. The PHP code worked perfectly. I started trying to do as much comparison between the two platforms as I could. I event went so far as to run a packet capture on the PHP and CF requests (pointing them at a dummy, local page that wasn’t encrypted so I could see the requests) and compared them to make sure everything was the same, which they were.

Eventually I found a post online that mentioned CFHTTP wasn’t up to the job, but a third-party custom tag written in C++ did work just fine. That custom tag is CFX_HTTP5. I downloaded a demo copy and installed it locally. How CFX_HTTP5 handles client certs is different from CFHTTP. Rather than simply pointing the tag at the client cert, I had to install the client cert into the Windows certificate store and then point the tag at the store. There is a bit of work involved with it, although nothing too difficult and it’s all covered in the CFX_HTTP5 documentation.

Once I had the tag and the certificate imported into the local Windows certificate store, I rewrote the CFHTTP call using CFX_HTTP5 and it worked! The SOAP envelope was the same, the headers were the same, the only difference between the two was the logic underlying the tags.

Something is broken with CFHTTP and it can’t be used to do some operations using client certificates. But at least there is an alternative.

However I didn’t like the alternative.

First, it’s a Windows-only solution. We’re out of luck if we’re running ColdFusion on a Linux machine.

Secondly, the client cert must be stored in a place that the ColdFusion process has permissions to access, and it is accessed without needing to know the client cert password. The result is that any person with permission to create CFM scripts that are executed under this process could authenticate against this third-party web site. In a shared hosting environment this can create a serious security issue. The only solution is to separate the process, but that probably means a separate server and a new OS license and hardware costs. If that option isn’t available you have some problems. You might register the CFX with a name that contains random characters and hope none of the users in the shared environment know how to enumerate registered custom tags. I’m not sure if that is possible, but I’m willing to bet it is.

Long story short, I don’t like the CFX_HTTP5 solution.

So, in Part 3, I go back to the drawing board.

Washing Client Certs in ColdFusion with SOAP – Part 1

Recently I was asked to look into integrating our systems with an external application via the third-party’s single sign-on system. The way it works is simple enough. We would have an application that authenticates the user through our system and then sends a request to the third-party asking for a token to sign the user into their system. The third-party would return the token that I then give to the end-user and redirect them to the third-party web site. This token is how the third-party would authenticate the user into their system. (The token is nothing more than a long string of characters that is passed on the URL of the redirect.)

The method of obtaining the token is also fairly simple. The application submits a SOAP request over an SSL session to the third-party’s authentication server and that server would respond with the token. SOAP, for all the technical specs and other crap, is very straightforward. It’s a simple XML document consisting of a root element called ENVELOPE which contains two children called HEADER and BODY. The header isn’t always required and the body typically contains elements with the names of various fields the SOAP function your calling requires with each element containing the value of that parameter. Very straightforward.

A SOAP request in ColdFusion couldn’t be simpler, especially with the CFSAVECONTENT tag. Simply construct your envelope inside a CFSAVECONTENT tag and then use CFHTTP to submit the request. It looks a little something like this:

<cfsavecontent variable="variables.soap">
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <GetCurrentTime xmlns="http://ws.historicaloptiondata.com/" />
  </soap:Body>
</soap:Envelope>
</cfsavecontent>

<cfhttp
  url="http://ws.historicaloptiondata.com/Service.asmx"
  method="POST"
>
  <cfhttpparam type="header" name="SOAPAction" value="""http://ws.historicaloptiondata.com/GetCurrentTime""" />
  <cfhttpparam type="header" name="Content-Length" value="#Len( Trim( variables.soap ))#" />
  <cfhttpparam type="xml" value="#Trim( variables.soap )#" />
</cfhttp>

<cfdump var="#xmlParse( cfhttp.filecontent )#" />

You should be able to plug this code into a CFM file and run it without having to touch a thing and you should see the CFDUMP of an XML object. (I say “should” because xmlParse() seems to first try and open a file with the name of the content of the passed variable and, when that fails, treat the passed value as an XML document itself. This can trigger errors and make it unusable if you employ any sort of file operation restrictions on your server. In which case modify the code to remove the xmlParse() call and just dump the cfhttp.filecontent.)

A few notes about this code SOAP in general.

• This is an example of a SOAP 1.1 request. There is another, slightly different format known as SOAP 1.2. The major differences between the two are that the content-type for 1.1 is text/xml, but for 1.2 it is application/soap+xml. Also the SOAPAction HTTP header is no longer needed in 1.2.
• The SOAPAction HTTP header’s value must be wrapped in double-quotation marks. And sometimes the first character after the open quotes will need to be a pound (#) symbol. This means ColdFusion programmers will need to be certain they escape these special characters in their values.
• If the CFHTTPPARAM type “xml” is present, CFHTTP automatically sets the content-type to text/xml. I am not sure if it’s possible to override this, but I believe not, therefore you’re almost always going to have to stick with SOAP 1.1 if you’re using CFHTTP for your SOAP requests.
• You must Trim() the variables.soap variable! The newline at the beginning of the value, which exists because there is a newline immediately after after CFSAVEDCONTENT tag (for visual formatting purposes) will make the XML document you’re sending an invalid XML document and result in errors.

Now comes the WSDL file. A WSDL file is an XML documents that describes the functions and parameters of said functions available through a SOAP service. The CreateObject() function has a “webservice” object type which will consume a WSDL file and create an object with all the available functions offered by the SOAP service. All the stuff with XML and ENVELOPES and CFHTTP becomes transparent and, as it turns out, SOAP can be simpler than my previous example. The above code can be reduced to the following using CreateObject():

<cfset ws = CreateObject( "webservice", "http://ws.historicaloptiondata.com/Service.asmx?WSDL" ) />
<cfset ws.getCurrentTime() />
<cfdump var="#GetSOAPResponse( ws )#" />

This makes this life much simpler for ColdFusion programmers. No need to worry about what version of SOAP you’re using or what URLs you need to submit your request to, the formatting of your SOAP envelope, the SOAPAction HTTP header variable, etc. It’s all taken care of for you by ColdFusion.

So this integration I was asked to do should be a piece of cake, right?

Wrong.

Simple SOAP is simple. Complex SOAP… well, you’ll see in Part 2.

cf_buffer

My knowledge of ColdFusion 8 (or hell, even 7 for that matter) isn’t terribly broad. So if what I’m about to cover can be accomplished with an existing CF function or tag please let me know.

Output Buffering.

It’s something I first got use to with PHP. What it does is allow you to set a start and end point in your program and any output generated between those two points gets put into a variable which you can then manipulate further before outputting the content.

Recently I was asked if this was possible to do with ColdFusion. I couldn’t think of any existing feature that does this so I put together a (surprisingly) short custom tag that would do just that.

Here it is:

<cfif CompareNoCase( thistag.ExecutionMode, "start" ) EQ 0>
  <cfparam name="attributes.buffer" default="variables.buffer">
<cfelse>
  <cfset "caller.#attributes.buffer#" = thisTag.GeneratedContent>
  <cfset thisTag.GeneratedContent = "">
</cfif>


I put this into a file named buffer.cfm. Now anything that occurs between an opening and closing CF_BUFFER tag will be stored in a variable which can then be further manipulated however you want.

I did find something similar using getPageContext().getOut().getString(), but nothing simpler.

You might argue that this feature shouldn’t be needed in the first place. That whatever the need is it can be solved by redesigning the application. That might be true. But it’s a handy feature to play around with.

ColdFusion: CFQUERY and Evaluate()

Been going back and forth with a friend who does CF development for some company. Anyways, he rings me up to talk about some SQL injection attacks that have been reported on an application he maintains. I take a look at the code and, sure enough, there are. Problem is they’re buried a bit and not easily found. Furthermore, one of the two exploits I’m about to cover is something not many CF developers will ever notice, even if they are aware of the dangers of SQL injection. This is because ColdFusion, on some levels, sucks.

Case 1: Evaluate

Here’s the code:

<cfset session.isadmin = false>

<cfparam name="url.x" default="1">

<cfset variables.msg1 = "Welcome To Our Site">
<cfset variables.msg2 = "Site Administration">

<h1><cfoutput>#Evaluate( "variables.msg#x#" )#</cfoutput></h1>

<cfif session.isadmin is true>
  <p>You may now administer the site as you see fit.</p>
<cfelse>
  <p>Buy our product.</p>
</cfif>

This is now how you write CF, this is just a very simplified example to show you the problem that was discovered; in this case, it’s the Evaluate() line. Variable X is being passed on the URL in this example for simplification. In reality, there are any number of ways user-provided data could find it’s way into an Evaluate() statement. I’m just making it easy and obvious here.

Evaluate() will evaluate (and execute) the contents of the string that is passed to the function and return the result. The intention of the example above is to return the relevant message associated with the user’s level of access. Since X is supplied by the user it is possible to inject something into that Evaluate() line. But what?

Long story short, calling this script with the following URL will alter the isadmin session variable to change the user’s access level.
vulnerable.cfm?x=2+eq+'a'+or+SetVariable(session.isadmin,true)

This creates the following Evaluate() statement:
Evaluate( "variables.msg2 eq 'a' or SetVariable(session.isadmin,true)" )

The string now represents a boolean statement. Instead of returning the value of the string variables.msg2 it now will return TRUE or FALSE. And before it does that it will execute every expression in the statement, including a call to the function SetVariable() which alters the isadmin session variable. With that, the user is now an admin and will have full control of the application. What’s worse, more function calls could could be passed to the application which essentially give the user the same level of access to the machine as whatever user ColdFusion is running under. Especially under ColdFusion MX where CreateObject() can be used to create JAVA objects that provide system-level access.

Case 2: Evaluate in a CFQUERY

<cfsetting showdebugoutput="Yes">
<cfset variables.hack = "' OR 1=1 OR Name='">
<cfquery datasource="misc" name="test">
  SELECT *
  FROM Test
  WHERE Name = '#Evaluate( "variables.hack" )#'
</cfquery>

Again, I’m oversiplifying this a bit, but I’ll show you where this would happen in the real world once you understand what’s going on here.

variables.hack represents the point of injection. We assume this variable has a legit purpose but, at this point in the code execution, it’s value has been altered by a malicious user to what it’s set as in the code. The relevant portion of the SQL injection is OR 1=1 OR which will trigger the database to return all records instead of just the one record intended. There are many other things you can do when you’ve got a SQL injection point like this, but this isn’t about SQL injection it’s about how Evaluate() creates problems.

In ColdFusion, any variable in a CFQUERY block that is wrapped in single quotes ('#variables.string#') will be automatically escaped. In other languages like PHP you have to do this maually, but CF does it automatically. Escaping these strings prevents malicious users from injecting SQL commands into the query. It makes the database treat the string as just a string and not as a series of commands that’s part of the SQL statement.

If you were to run this script on a CF server you would discover that the string in variables.hack is NOT escaped. This allows SQL to be injected into the query. Why is this? The Evaluate() statement is wrapped in single quotes, so what gives?

Well.. actually, ColdFusion is working as intended. There are several passes of the SQL statement made by ColdFusion. The first pass handles the escaping of variables. The second pass handles executing functions and after that the SQL is passed on to the database. If you had something like:


<cfset variables.msg="hack'd">
<cfquery datasource="misc" name="test">
  SELECT *
  FROM Test
  WHERE Name = '#Evaluate( "variables.#msg#" )#'
</cfquery>


You would find the Evaluate() statement becomes:
Evaluate( "variables.hack''d" )

Two single-quotes is the escape sequence for a single quot e in SQL. The contents of the msg variable are escaped. After this escape, Evaluate() is executed. In this case it’d return an error since variable names can’t contain single-quotes.

So you see, the escaping of variables occurs before Evaluate() is processed. This means the contents of whatever Evaluate() will not be escaped and provides a point of injection for an attacker.

The solution is to set the results of the Evaluate() call to a temp variable and use that temp variable inside the CFQUERY.


<cfset variables.test = Evaluate( "variables.hack" )>
<cfquery datasource="misc" name="test">
  SELECT *
  FROM Test
  WHERE Name = '#variables.test#'
</cfquery>


Where is this used in the real world? Arrays. Something where you loop through a number set, say 1 to 20, where you insert the Nth value in an array into the database. Something like SET data = '#Evaluate( "userdata[#X#]" )#' where userdata is an array that stores data provided by the user (form data, data pulled from a database that was set through another application, etc.)

This problem exists for other functions used in a CFQUERY.

That is, any function that performs an evaluation of a string will have the same problem. IIF() is one such function. I’m not sure what others there are off the top of my head, but it’s definately something to think about.

So the point to remember is to never pass any data that, at any time could be set by the user, through Evaluate() and IIF().

And there you go. A couple exploit vectors (as the security guys say) in ColdFusion that you need to be aware of if you’re a CF developer.

CF: cflock

CFLOCK is used whenever there’s a chance that more than one process will try to manipulate a given resource at the same exact time. You might use cflock with a database so that if a read operation occurs during a write, the read operation will have to wait for the write to finish. That way the read operation will have the most up-to-date information possible. Normally you don’t need to worry about stuff like this. I use this in a room selection application (where students can pick the room they live in for the following year). I cflock an entire block of logic that first checks to see if the room being selected is available and, if it is, record the room selection. This way I don’t risk someone else selecting the room between the read to check for availability and the write to record the selection. If I didn’t do that, there’s a chance rooms could become double-booked.

When working with a file that is used in an application, you almost always use cflock. Now I don’t mean you cflock whenever someone is uploading a file or you’re going to read from some random htm file (like a template system) because your application isn’t going to be altering the file’s contents (more than once, in the case of writing an uploaded file). But if you’ve got a file your application is going to read and write to throughout the application’s life, you need to protect against the chance two writes will occur at the same time. For example, in a user account claim application I have to write usernames to two separate files. One gets picked up by a process that creates e-mail accounts, another that creates LMS accounts. There’s a chance two people will try to claim an account at the same time. So I’ll cflock the file during the write, so no other CF process tries to write to the file at the same time. If that happened, you can wind up with a corrupt or empty file. Guestbooks, blog comments, etc are examples of applications where you’d cflock file access (assuming you’re using files and not a database).

Now why CFLOCK session variables?

Because you can’t assume a user will only make one connection at a time. When a user requests a page, the web browser begins to make several simulatneous connections to download the images, CSS files, javascript, etc.. on top of the HTML for the page. If you’ve got one or more of those files setup as a CFM (a dynamic image, dynamic stylesheet, who knows what) you’re application.cfm will run as well. If you’ve got logic in your application.cfm that manipulates session variables, you run the risk of having your session variables being changed in mid-process, creating either corrupted data (less likely) or incorrect data being acted upon (more likely).

For example, let’s say you’ve got a voting system. The voting system uses a session variable to set whether or not you’ve voted. A person logs into this system, makes their vote selection, and double-clicks the “vote” button. Your CF server now has 2 separate vote processes from the same person that it will process. The application logic is:

1. Check if allowed to vote
2. Record vote
3. Flag user as having voted

Step 2 is a database operation. In computer time, that step is going to take forever to process. While process 1 is working on step 2, process 2 comes along and passes the check in step 1, and gets in line for step 2. Process 1 moves on to step three and records the user has voted, but only after process 2 has started recording the vote for a second time.

Your user has now voted twice because of a race condidtion with session variables.

To fix this process you can do a couple things. You could put all three steps inside a single cflock block. You could swap steps 2 and 3 and then put steps 1 & 2 (check/record allowed to vote flag) then do your database options. The latter option frees the lock sooner to help keep resources to a minimum and is a potential speed increase but you might lose votes. You could wrap the database operation in a cftry/catch block and reset the flag if needed, but now you’re getting overly complicated in a system where just wrapping the 3 steps in a cfblock works fine.

So why not wrap every page in a cflock?

Because you will have pages that take a few seconds to process. If a user double-clicks a button, like in the example above, they will have to wait twice as long for the results to be displayed. If they get bored/angry at the wait, they might press that button 10 or 20 more times thinking it’ll go quicker, when in reality it’s only slowing things down. At that point, you’ve got 20+ processes waiting for that lock to open up. CFMX recommends your number of simultaneous processes allowed in CF be 3 or 4 times the number of CPUs in the machine. I’ll tell you that we have ours set to 12. (3 * 4 (2 P4s, each of which act like 2 separate processors)). So at 20+ processes, each waiting for that lock, each on the stack of running processes, you’re entire site (or your CF applications at least) grind to a halt. Now every user (not just the one) starts clicking on that button to speed things up. You eventually wind up with a really nasty situation where you’ve got hundreds (even thousands) of processes in the queue waiting to be processed by CF. Your site becomes unusable for minutes, maybe even hours.

That’s why you need to be very very efficient in your use of cflock. They can be a source of severe bottlenecking. I have a cf_sleep custom tag that gets CF to hang for a few seconds. I don’t use it much (if ever) but the way it works is by nesting cflocks on the same resource. Create a page with a 20 second sleep, reload it 20 times, and you’ll shut down the site for 20+ seconds. Very nasty.

If you can, set yourself up with a performance monitor (this is a Windows thing. start->run->perfmon) set on your CF server and you can see this in action yourself. (Assuming you’re setup, monitoring all the CF related monitors.) Turn on highlighting in your performance monitor (CTRL+H or click the lightbulb icon in the top toolbar). Then select the “running requests” monitor from the list in the bottom section of the performance monitor window. The highlighted (white) line you see shows you how many current requests are being processed.

Create a script with a 20 second sleep in it. Load the page then check your performance monitor and you’ll see that there is 1 process running. Now hold down CTRL+R in your browser for a few seconds. You’ll get maybe a couple hundred of these processes going. Now check out the performance monitor. The running requests will max out at 12 (which is what I set it to as mentioned earlier). Now check out the “queued requests” monitor and see how that spikes up.

The site is essentially useless while you wait for those processes to finish. All this because of 1 user holding down CTRL+R in a browser for a few seconds. That’s the downside of cflock (and any slow ColdFusion page). Try changing the page so the sleep is only a second and do it again. The server takes a little longer to go unresponsive, and it recovers more quickly. But you start to see how CF can be exploited.

There’s a configuration option to kill any process that runs over X seconds long in the administrator interface. That offers some protection from prolonged denial-of-service attacks, but not much. (I typically set it to 30 seconds, but 5-10 seconds might be better for most people.)

Increase the number of simultaneous requests? That only prolongs the inevitable and when you hit that max, it takes much longer to recover because your server is doing a lot more than it can handle.

So be careful of bottlenecks in your CF code, CFLOCK being potentially one of the biggest in your application.

CF: Session Hijacking

ColdFusion uses two unique values to keep track of user session information. These values are CFID and CFTOKEN. They are stored as cookies but can also be passed along the URL and inside POST data.

Session variables are a place to store information specific to the user and to the current session (such as whether or not a user is logged in).

It is possible to hijack a user’s session by supplying the correct CFID and CFTOKEN values to the server, either on the URL, or wherever else you want.

The two numbers combined represent a space of 10^15 numbers. Average brute force will take half that amount, so 10^15/2. Figure 100 attempts per second, and the average time it would take to brute force is in the neighborhood of 150,000 years.

There’s the 1 in a jillion chance someone might guess a correct CFID and CFTOKEN, but that doesn’t really worry me much.

Your more likely to see someone hack your application by doing a little packet sniffing (or looking over someone’s shoulder) and capturing the CFID and CFTOKEN that way.

Packet sniffing you can curb by going over SSL with your application. Over-the-shoulder attacks can be stopped by not passing the CFID and CFTOKEN values on the URL (which CF does with cflocation tag by default… go figure).

If the user has a virus on their machine passing out their cookies, well that user has a bigger problem than having their session hijacked.

So how do you protect against session hijacking? You store the IP address as a session variable. Compare the IP in the session variable to the user’s IP (stored in cgi.remote_addr) and if they don’t match, you’ve got a hijacking attempt.

… But there’s a catch.

AOL, for example, uses a proxy server for their packaged browser. This means everyone comes from the same IP address. Not cool. Now AOL users can simply go into their browser settings and kill the proxy config and they’ll surf using their own IP, but can you really ask users to do that for every little application we have using session variables?

Also AOL users won’t be the only ones behind a proxy server.

And if you have session timeouts set to days, dial-up users and any other user on a network with shared IP addresses will eventually get an IP address of a former user. And they might be able to get into the application that way.

So what can you do? Not much. You won’t ever be 100% certain in your security. It’s all about managing risk. In this case, you’re at a fairly low risk with hijacking if you’re comparing IP addresses.

But here’s what I do to take it 1 step further.

Combine the user’s IP address and browser string (cgi.http_user_agent) into a single string. Then MD5 hash the thing. Store that hash as a session variable. Recalculate and compare hashes as the first step in any user request (in other words: put this logic in your application.cfm file, and put it up at the top). And that should protect you well enough. The browser string provides a little extra security in the event of proxy users hitting the site.

Also keep your session timeouts to a low value (30minutes.. 2 hours MAX, unless security isn’t a big issue for your application).

When you detect a hijack attempt, you might not want to kill the session because the legit user also gets locked out. You can reset CFID and CFTOKEN on the user with the CFCOOKIE tag then redirect the user to the enterance page.