In The Zone With Time

The account-expires attribute in Active Directory carries a value of time in 100 nanosecond units since January 1, 1601 00:00 UTC. In fact quite a lot of Microsoft products are now using this epoch. Java and Unix use January 1, 1970 00:00 UTC as their epoch.

This is important information if you’re ever going to develop code in ColdFusion that needs to, for example, figure out whether or not a domain account has expired.

Another piece of information that’s handy to have is an understanding of how ColdFusion handles large numbers. Numeric values are handled as signed 32-bit values that can range from -2,147,483,648 to 2,147,483,647. ColdFusion can handle much larger numbers, but it does so by converting numeric values into strings and then using its own library to handle math operations based on these strings. And it stores these large values in scientific notation. The problem with this is you start to lose precision. For example, if you run the following CF snippet:

<cfoutput>#( 2147483648 * 2147483648 )#</cfoutput><br>
<cfoutput>#( 2147483648 * 2147483648 + 1)#</cfoutput>

The output is the same for both operations, “4.61168601843E+018“. This loss of precision makes it impossible to calculate the exact time account-expires refers to. So what else can we do?

One trick that sort of works is to divide the value in account-expires
by 10000000 to convert from 100 nanoseconds to seconds. This will significantly shorten the number. So let’s try that on an account-expires value of “129247307050000000” (that’s 7/27/10 @2:58.25 EDT). The line of CF looks like this:

<cfoutput>#DateAdd( "s", ( 129247307050000000 / 10000000 ), "01/01/1601 12:00 AM")#</cfoutput>

Try to run this code and you get the error message “Cannot convert the value 1.2924730705E10 to an integer because it cannot fit inside an integer.”. It turns out that DateAdd() can’t handle a number larger than 2147483647 (the limit of a 32-bit signed integer). 2,147,483,647 seconds is about 70 years of time. To use DateAdd() we need to make the account-expires value much smaller. We could, for example, calculate the account-expires value with respect to a different epoch, one that’s within 70 years of the time referenced by account-expires. We could use any date we want, but I’m going to stick with Java’s epoch of January 1, 1970 to keep things in-line with what’s probably the most common epoch among computer systems.

The number of seconds between January 1, 1601 and January 1, 1970 is 11,644,473,600. If we plug this into the code above we get this:

<cfoutput>#DateAdd( "s", ( 129247307050000000 / 10000000 ) - 11644473600, "01/01/1970 12:00 AM")#</cfoutput>

Run this and we get a result! Specifically {ts '2010-07-27 19:58:25'}. Remember this is in UTC, we’ll need to convert to our local timezone. We’ll do that in just a moment. But first…

You should have recognized by now that this solution won’t work forever. It will fail whenever the data in account-expires is after January 19, 2038. Perhaps this doesn’t bother you since whatever applications you’re developing now will surely be either obsolete or running on a more modern ColdFusion system where DateAdd() can deal with larger numbers before account-expires with that large a value are in use.

Perhaps. But I hate making assumptions like that. Plan for the future! Assume nothing!

If we want lasting code free of the 2038 limitation we’ll need to steer clear of ColdFusion date and time functions. We can do this by using Java objects, specifically java.util.Calendar to manage date and time, and java.math.BigInteger to handle the very large numbers we’ll be working with.

The process is fairly straightforward, but there are a few things to watch out for. Java time is measured in milliseconds, not 100 nanoseconds, so we’ll have to do some conversion. Java and Microsoft’s epochs are different. Calendar objects with a date and time earlier than Java’s epoch will have a negative millisecond value. And we need to calculate this date/time with respect to UTC (GMT timezone) not the local timezone.

First create a BigInteger object. Initialize it to the value of account-expires, and then convert it to milliseconds as that’s the unit of time Java likes to work with.

01: <cfset variables.bigInt = CreateObject( "java", "java.math.BigInteger" ) />
02: <cfset variables.expTime = variables.bigInt.init( JavaCast( "String", "#arguments.accountExpires#" )) />
03: <cfset variables.expTime = variables.expTime.divide( variables.bigInt.valueOf( "10000" )) />

You can initialize BigInteger objects with strings and numerical values. I’m specifically casting account-expires as a string so that ColdFusion doesn’t treat it like an integer and try to put it into scientific notation. Note in line 3 the use of BigInteger’s valueOf() function. This takes a string and returns a BigInteger value. Math functions for BigInteger objects require BitInteger arguments, thus we need to pass all values (even static ones) through valueOf() instead of JavaCast().

Next we set up the calendar object. This means creating a java.util.TimeZone object, a java.util.Calendar object, and then initializing the calendar to the GMT timezone. Once the calendar object is initialized we set it to Microsoft’s epoch.

04: <cfset variables.tz = CreateObject( "java", "java.util.TimeZone" ) />
05: <cfset variables.Calendar = CreateObject( "java", "java.util.Calendar" ) />
06: <cfset variables.gCal = variables.Calendar.getInstance( tz.getTimeZone( "GMT" )) />
07: <cfset variables.gCal.set( 1601, 0, 1, 0, 0, 0 ) />

Note that the second argument of the calendar object’s set() function is 0. Java’s calendar object’s month index begins with 0 for January instead of 1. This is something to be aware of whenever dealing with Java calendar objects.

This next step requires a little explanation. variables.expTime is now in milliseconds thanks to line 3. I’m going to add the value of the calendar, in milliseconds, to variables.expTime. Because the calendar is set to a date before Java’s epoch it will actually be a negative number. The addition operation is actually a subtraction operation. The resulting value of variables.expTime will be the number of milliseconds since Java’s epoch.

08: <cfset variables.expTime = variables.expTime.add( variables.bigInt.valueOf( "#variables.gCal.getTimeInMillis()#" )) />

All we need to do now is reset the calendar to the value we now have in variables.expTime, then convert it to our local timezone and we’re done!

09: <cfset variables.gCal.setTimeInMillis( variables.expTime.longValue() ) />
10: <cfset variables.gCal.setTimeZone( tz.getDefault() ) />

Last, but not least, we convert the calendar object into a ColdFusion date/time object with ColdFusion’s CreateDateTime() function.

11: <cfset variables.expDateTime = CreateDateTime(
variables.gCal.get( variables.gCal.YEAR ),
variables.gCal.get( variables.gCal.MONTH ) + 1 ,
variables.gCal.get( variables.gCal.DAY_OF_MONTH ),
variables.gCal.get( variables.gCal.HOUR_OF_DAY ),
variables.gCal.get( variables.gCal.MINUTE ),
variables.gCal.get( variables.gCal.SECOND )
) />

Note that I’m adding 1 to the month variable because in Java 0 = January, but in ColdFusion 1 = January.

And there you have it. Converting from Microsoft’s 01/01/1601 100 nanosecond based timestamps to something a bit more usable in ColdFusion.

Of course you’ll need a reverse of this process as well, and I’ve already got that for you:

01:  <cfset variables.tz = CreateObject( "java", "java.util.TimeZone" ) />
02:  <cfset variables.Calendar = CreateObject( "java", "java.util.Calendar" ) />
03:  <cfset variables.gCal = variables.Calendar.getInstance( tz.getDefault() ) />
04:  <cfset variables.gCal.set(
Year( arguments.Date ),
Month( arguments.Date ) - 1,
Day( arguments.Date ),
Hour( arguments.Date ),
Minute( arguments.Date ),
Second( arguments.Date )
) />
05: <cfset variables.bigInt = CreateObject( "java", "java.math.BigInteger" ) />
06: <cfset variables.expTime = variables.bigInt.init( JavaCast( "String", "#variables.gCal.getTimeInMillis()#" )) />
07: <cfset variables.expTime = variables.bigInt.divide( variables.bigInt.valueOf( "1000" )) />
08: <cfset variables.expTime = variables.expTime.add( variables.bigInt.valueOf( "11644473600" )) />
09: <cfset variables.expTime = variables.expTime.multiply( variables.bigInt.valueOf( "10000000" )) />
10: <cfreturn variables.expTime.longValue() />

You should be able to follow along based on my explanation earlier in the article. However there are a couple extra bits I’d like to point out.

The calendar is initialized to the local timezone. We don’t have to worry about converting back to UTC/GMT because the calendar function getTimeInMillis() will give you the number of milliseconds from Java’s epoch, which is in UTC.

On line 8 I’m adding the number of seconds from Microsoft’s epoch to Java’s epoch. You’ll also notice the line before that I’m dividing by 1000 to convert time from milliseconds to seconds and after I add the epoch difference I’m converting to 100 nanoseconds by multiplying by 10000000. So why not skip the conversion to seconds and simply add three more zeroes to the end of the epoch difference (making the time in milliseconds) and multiplying by 10000?

The reason is that there’s a problem with precision and the resulting value will vary by a few hundred milliseconds. Since the times I’m working with are in seconds, those extra milliseconds could be ignored. But by dividing, adding, and then multiplying like this I don’t get the millisecond variances. I’m sure there’s better stuff on the ‘net to explain precision with BigInteger objects if you’re inclined to investigate further.

Before you go I have one piece of information I learned while working on this particular topic.

ColdFusion treats all date and time values as if they are in the local timezone. Specifically, if your timezone has daylight savings then all dates are treated as if they have daylight savings, including GMT/UTC times which do NOT have daylight savings time. DateConvert() will not protect you from this issue. There’s a deeper explanation here.

My solution is to check if GetTimeZoneInfo().isDSTon is TRUE. If it is then I need to add (or subtract) 1 hour from my UTC time. Note that this is only needed when converting between UTC and local timezones with ColdFusion date and time functions. The above code specifically stays away from ColdFusion date and time functions so this isn’t a problem with my example. But it is something to keep in the back of your head when you start to notice dates are being calculated an hour off the time they should be.

A bit of vinegar to go with the SOAP.

Not long ago I wrote a 3-part series on using SOAP over HTTPS with ColdFusion. My final solution was to create Java objects directly, bypassing ColdFusion’s CFHTTP tag.

I have since found a subtle flaw with this implementation.

It’s not in the code, but in the  JVM. ColdFusion 8 ships with an older JVM. I recently upgraded our JVM to a more current version in an attempt to resolve a timezone bug. In doing so my SOAP application stopped working.

A little research led me to this article about a TLS bug in Java that could lead to a man-in-the-middle exploit. It appears the way I’m performing my SOAP operation triggers a TLS/SSL renegotiation when it receives a response from the external server.

The short answer is to add the following line to ColdFusion’s JVM arguments:

-Dsun.security.ssl.allowUnsafeRenegotiation=true

This does resolve the problem, but it apparently leaves the JVM vulnerable to MITM attacks. There is another bit of code in that article which shows how to change the allowUnsafeRenegotiation flag on-the-fly. I added this to my ColdFusion code, but changing the flag didn’t appear to have any effect.

If anyone else has played around with this particular problem I’d love to hear about it.

For now I”ve left the JVM in its vulnerable state as we only make HTTP requests from the JVM for a couple of applications and neither of them carry personally identifiable information.

Americans with Disabilities Act (ADA) and the Web

The United States Department of Justice has announced that it plans to create rules that apply the ADA to the web.

I’d like to begin by pointing you specifically to the section titled “Barriers to Web Accessibility“. It is a very good read with clear and specific examples of how web content can be inaccessible to users with disabilities.

I think this is a Very Good Thing. Not for any humanitarian reason, but for the very selfish reason that it will force developers to create better web sites. It will force developers to think “how will this affect users with disabilities” before they implement a web site design.

For example, there are quite a lot of stylesheets out there that make heavy use of !important rules. These rules override anything else that exists to style a given element, including user-defined stylesheets applied to web pages by users who have difficulty with low-contrast web pages (think gray text on a white background). !important rules are almost always a product of lazy developers who don’t take the time to learn the cascading order of CSS and resort to !important when they can’t figure out why their style won’t apply like it should.

However the are stickier areas that we’ll all have to deal with. For example the use of CAPTCHAs; those little scrambled words that you have to type into some box before you can submit a form. CAPTCHAs typically rely on images which are inaccessible to blind users. reCAPTCHA employs an aleternative, audio-based CAPTCHA along side it’s image CAPTCHA for such users. I’m a big fan of reCAPTCHA and suggest it to all web developers.

Another problem will be video content. Blind and deaf users won’t be able to access the full content of the video, however providing video captions or (more correct) a transcript of the video will solve the issue. It’s not a technological hurdle, just a tedious one. This web site specifically talks about YouTube and captioning as one way to solve this problem.

Mouse-driven events are yet one more problem area we’ll need to deal with. I myself make heavy use of drop-down menus with the CSS :hover pseudoclass. However, try tabbing through a web page yourself and you’ll see those drop-down menus don’t trigger. My approach to this issue has always been that the top-level items (those accessible to users who can only tab through the page) should link to pages from which the items in the drop-down are accessible. The drop-down provides a shortcut, but you are not limiting access to information for disabled users.

There are other areas to cover, but I’m not here to cover them all. In fact  I’m going to assume new areas will be created as technology progresses. The trick is to develop the mindset that as you develop a web site, or some web-based resource, to constantly ask yourself “is this accessible?”. If the answer is ever “no”, you need to find a way to make that answer “yes”. And, most importantly, follow through to make it a “yes” with vigor rather than apathy as I tend to believe developer apathy is the cause for the majority of inaccessible web sites out there right now.

Obfuscated Javascript Spam

Recently I’ve been receiving phishing-spam in the form of official-looking Amazon.com invoices. Curiosity got the better of me and I clicked on the phishing link. The page that came up was blank. A quick source view revealed a bunch of obfuscated javascript.

I wanted to see how it worked.

Here is a sample line of the code:

mGdujq[‘euvLaulm'[VvIf](/[muzLc]/g, EWgUi)] \
(ltY(mGdujq[[‘uhnKehsKcKaKpleo'[VvIf](/[oKhlE]/g, EWgUi)]](IuO)));

Now what’s going on here?

Well, plain as day in the source I see a couple very important lines that will help decode this. The lines are:

var EWgUi = ”;
var mGdujq = this;
var VvIf = ” + ‘replace’;

Armed with this information the line decodes easily before our eyes to

this[‘euvLaulm'[replace](/[muzLc]/g,”) \
(ltY(this[[‘uhnKehsKcKaKpleo'[replace](/[oKhlE]/g,”)]](IuO)));

What’s left to decode is the use of shorthand regular expressions. For example let’s look at this piece of code

‘euvLaulm'[replace](/[muzLc]/g,”)

‘euvLaulm’ is just a regular old string. You can call the string’s replace function in many different ways such as:

var str = ‘euvLaulm’; str.replace();
‘euvLaulm’.replace();
‘euvLaulm'[replace]();

The regex /[muzLc]/g simply matches any character within the square brackets. The full line of code calls for every match to be replaced with ” (an empty string) or in other words, to delete those characters from the string.

euvLaulm

The result is the string ‘eval’.

So the fully interpreted line of javascript reads as follows:

this[eval](ltY(this[[unescape]](IuO)));

Or in code more readable to my own eyes:

this.eval( ltY( this.unnescape( IuO )));

Strewn throughout the javascript are lots of variable declarations that create strings of seemingly random letters and numbers. Upon close inspection you might notice that there’s a pattern to the strings; they consist of alternating hex and non-hex characters. (A hex character or value is 0-9 and a-f).

Near the end of the code all these strings are concatenated and a series of replace operations are performed to replace all the non-hex characters with ‘%<hex character>’. The result is a string of URL escape sequences (a percent symbol followed by 2 hex characters). This string is stored in the variable IuO.

The URL escaped data is then unescaped to create an array of bytes (aka, a string, except the bytes aren’t all printable characters, so I can’t call it a string). This data is passed to the ltY function which performs a ( <byte> XOR 13 ) operation on each byte of data. The result is a string of HTML that creates a hidden iframe to some porn referral page and a META refresh that redirects the user to a male supplements web site after 4 seconds.

That was fun. A little sleuthing and puzzle solving. But what is there to take away from all this?

First was myself learning new ways to use and abuse javascript syntax such as ‘string'[function].

Also curious was the large amount of superfluous statements in the code. Variables would be created without initialization. They they’d be initialized to an empty string. Then they’d be set to their real value. Three statements to perform an operation that could be done in one. I imagine this, along with the use of random upper and lowercase letters used as variables AND data make the code more difficult to parse by hand (or by eye). But a few minutes and a bit of perseverance will overcome that. However that shows these types of scams are designed with the user who tries to perform a cursory inspection of the underlying code in mind.

The multiple decoding steps to arrive at the final “attack” HTML indicates to me the code is designed to circumvent string-comparison spam filters. That there was more than one decoding step and that there’s a bunch of extra, useless javascript (if/else blocks with one assignment to an unused variable) makes me wonder if this code was also created to circumvent spam filters that are javascript-aware. It’s working so far. My institution’s normally solid spam filtering software has let this one slip by twice in the last week.

And it’s nice to see what kinds of tricks spammers have up their sleeves.

ColdFusion Regex Backreferences

I want to take a string that contains HTML and increment all heading tags by 1. For example, I want to turn <h3>…</h3> into <h4>…</h4>.

I might be able to accomplish this with a long ReplaceList command. However I decide against this as I’ve run into problems with ReplaceList not replacing items in the order it claims to in the documentation. Plus I would have to hard-code every heading tag (both opening and closing) and that would just look ugly. So let’s try and be clever.

This sounds like a job for regular expressions (regex). I opt for the REReplace function over REReplaceNoCase because REReplaceNoCase seems lazy to me, and it forces me to specifically take case into consideration. I like that. Deal with potential problems with input now instead of waiting for it to become a problem later.

So my search string will be something like “<(/?[hH])([1-6])“. This will search for both opening and closing heading tags and I can use the number of the heading as a backreference (that is, I can refer to it in my replacement string). I’m only search for H1 through H6 because the HTML spec clearly states there are only 6 headings, H7 and so on aren’t valid HTML tags.

So my REReplace line might look something like REReplace( variables.input_string, “<(/?[hH])([1-6])”, “\1#Val( \2 + 1 )#”, “ALL” ).

But this will not work.

The reason being that ColdFusion will evaluate all CF variables before performing regex operations. So the backreference to the number of the heading (the “\2” in the example above) will be interpreted as the literal string “\2”. This will trigger a parsing error.

So the workaround is a bit ugly, but still more elegant than just one big ReplaceList function, and that is to loop from 1 to 5 and perform the regex operations one heading at a time. Except there’s another gotcha here. If my loop starts at 1 and works up to 5, I will replace all H1 tags to H2. In the second loop, all H2 tags, including the former H1 tags, will become H3 tags. In the end my HTML will be all H6 tags. The fix is simple, just loop backwards, from 5 to 1. Easy! The code would look something like this:

<cfloop from=”5″ to=”1″ index=”variables.h” step=”-1″>
<cfset variables.input_string = REReplace( variables.input_string, “<(\/?[hH])#variables.h#”, “<\1#Val( variables.h + 1 )#”, “ALL” ) />
</cfloop>

Simple. Except this code won’t work. All the heading tags get dropped for some reason.”<h4>” becomes “<>”. What’s going on?

Refer back to how ColdFusion will evaluate all variables before performing the regex operation. In the replace string I use the backreference “\1”. But ColdFusion evaluates the Val() operation before performing the regex operation, so the backreference the regex operation sees (for an H1 replacement) is “\12”. The regex operation looks for a 12th backreference, which doesn’t exist, so the replaced string is empty.

In proper regex engines, this can be easily fixed in a few different ways. I could explicitly specify the base of the number (regex engines typically accept octal, hex, and decimal base numbers) with something like \x01 (hex-base). Or I could wrap the number in curly brackets like \{1}. But ColdFusion doesn’t have a proper regex engine, but a rather bastardized regex engine.

I need to separate my backreference (\1) from my replacement variable (#Val( variables.h + 1 )#). A space won’t work because my H4 tag becomes “H 4”, which isn’t an HTML tag. Can’t use HTML entities as this isn’t text to be output to the user, it’s HTML that the HTML engine needs to interpret. So what can I do?

Well, here’s the trick I’ve decided to go with.

Looking at ColdFusion’s “Using backreferences” page I see references to special characters used to make letters uppercase (\u) or lowercase (\l). I also see I can perform this case change over a string of characters by inserting \U or \L at the start of the case change, and then insert a \E where the case change should end. This is what I need. These are single, alpha-character special regex characters that do not generate output by themselves. I can use a “\U\E” sequence to separate my backreference from my variable! So my final code looks something like this:

<cfloop from=”5″ to=”1″ index=”variables.h” step=”-1″>
<cfset variables.input_string = REReplace( variables.input_string, “<(\/?[hH])#variables.h#”, “<\1\U\E#Val( variables.h + 1 )#”, “ALL” ) />
</cfloop>

A quick test shows me I don’t even need the \U, I can stick with just a \E to get the job done. But I’ll keep the \U for completeness.

The reason I start with 5 and not 6 is that I don’t want to create H7 tags (which is not valid HTML) so H6 tags will remain H6 tags.

And now we come to the end of my post. This is where someone posts a comment and points out that my solution is, in fact, unnecessary because there IS a mechanism in ColdFusion’s regex engine to handle such a situation and I just didn’t RTFM as closely as I should have. Well, I truly welcome such a revelation should it exist. If not, perhaps Adobe could look into beefing up their regex engine a bit.

Washing Client Certs in ColdFusion with SOAP – Part 3

Continuing from part 2, I’ve got my SOAP request working, but the solution is limited to the OS platform and I don’t like the security implications. What I would really like is a solution that works without the need of a custom tag that is OS dependent. If only CFHTTP wasn’t broken.

I decided to look at what I already had available; namely Java. ColdFusion runs on top of Java and it’s possible to create Java objects from within ColdFusion.

I’ll spare you the grueling details, but I did get caught up at one point playing around with the Java keystore and importing my client certificate into that keystore. Using things like keytool and OpenSSL I tore apart and reconstructed my client certificate and my Java keystore. Ultimately I didn’t need to do any of that and should have just left the keystore alone.

The path to enlightenment began when I found an example of Java socket operations using  the javax.net.ssl.SSLContext object. The example code I was looking at used this object to create regular sockets, not SSL-enabled sockets. I looked into this object and found a few more examples using it to create SSL connections. A bit of back and forth between these examples and the online Java API spec resulted in the discovery that I could load a PKCS12 file as a keystore and pass that to the SSLContext object then create sockets from that SSLContext. This looked promising.

After a bit of trial and error I finally had something that worked!

Here’s the relevant portion of code. I’ll go line-by-line afterwards.

<cfscript>
  // 01: create input file stream from certificate
  variables.ksf = CreateObject( "java", "java.io.FileInputStream" ).init( variables.cert_file );

  // 02: create keystore object from certificate
  variables.ks = CreateObject( "java", "java.security.KeyStore" ).getInstance( "PKCS12" );
  variables.ks.load( variables.ksf, JavaCast( "String", variables.cert_password ).toCharArray() );

  // 03: create key manager factory out of keystore
  variables.kmf = CreateObject( "java", "javax.net.ssl.KeyManagerFactory" ).getInstance( "SunX509" );
  variables.kmf.init( variables.ks, JavaCast( "String", variables.cert_password ).toCharArray() );

  // 04: create SSL context object using key manager factory
  variables.sslc = CreateObject( "java", "javax.net.ssl.SSLContext" ).getInstance( "TLS" );
  variables.sslc.init( kmf.getKeyManagers(), JavaCast( "null", "" ), JavaCast( "null", "" ) );

  // 05: get SSL socket from SSL context
  variables.factory = sslc.getSocketFactory();

  // 06: connect to the server via SSL
  variables.sock = variables.factory.createSocket( variables.server_address, variables.server_port);
  variables.sock.startHandshake();

  // 07: create input and output streams to read from and write to the socket
  variables.sout = variables.sock.getOutputStream();
  variables.out = createObject( "java", "java.io.PrintWriter" ).init( variables.sout );
  variables.sinput = variables.sock.getInputStream();
  variables.inputStreamReader = createObject( "java", "java.io.InputStreamReader" ).init( variables.sinput );
  variables.input = createObject( "java", "java.io.BufferedReader" ).init( variables.InputStreamReader );

  // 08: send the HTTP request over the socket
  variables.out.println( Trim( variables.request_header ) );
  variables.out.println();
  variables.out.println( Trim( variables.soap_envelope ) );
  variables.out.println();
  variables.out.flush();

  // 09: read the response from the server
  variables.result = "";
  do {
    variables.line = input.readLine();
    variables.lineCheck = IsDefined( "variables.line" );
    if ( variables.lineCheck ) {
      variables.result = variables.result & variables.line & Chr(13) & Chr(10);
    }
  } while ( variables.lineCheck );

  // 10: close the connection
  variables.sock.close();
</cfscript>

So let’s work through this.

01: The variable variables.cert_file contains the full path to the client certificate. This line opens that file for reading.

02: This creats a java.security.KeyStore object that expects a PKCS12 format and then loads the client certificate into the keystore. Note that the second parameter of the KeyStore.load() method is the password of the PKCS12 file. You must pass this as a character array and not as a string, which is why the password must be cast as a String object whose toCharArray() method is called to obtain that character array.

03: This creates a javax.net.ssl.KeyMangerFactory object which is used by the SSLContext object to deal with the certificates in the keystore. Note that the second parameter of the init() is the password to the certificate in character array format. You must provide the password twice, once when creating the keystore, and then again creating the key manager factory.

04: Here we create the javax.net.ssl.SSLContext object. The getInstance() call tells what protocol to use with this object, in this case “TLS“. (Note: “SSLv3” also works. No idea which one I should be using. If one fails, try the other.) Once the object is instantiated we initialize it with a key manager from the key manager factory. The second and third parameters of the init() method could point to a trust manager and a source of randomness. It doesn’t appear that either is needed (this is what I saw in other examples) so I pass null values. Note that you have to use JavaCast() to pass a null.

05: Get a socket factory object from the SSLContext object. This factory will be used to create individual sockets using the parameters established with the SSLContext object.

06: Create a single socket from the socket factory by passing the server address (variables.server_address) and port (variables.server_port) we want to connect to. Then call startHandshake() which will create the connection to the server using SSL.

07: Create objects needed to read from and write to the socket we’ve created.

08: Here we send our SOAP envelope to the server. Note that since we’re doing everything by hand we also have to send the proper HTTP headers. Both the variables.request_header and variables. soap_envelope variables were created using the CFSAVEDCONTENT tag as shown in part 1. You want to create the envelope first so you know the correct content-length value for the HTTP header. Since I decided to stick with SOAP 1.1 the header also contains the SOAPAction field and the content-type is set to text/xml.

Here is an example of how I created the headers:

<cfsavecontent variable="variables.request_header">
POST <cfoutput>#variables.server_request_path#</cfoutput> HTTP/1.0
Host: <cfoutput>#variables.server_address#</cfoutput>
User-Agent: SOAP Washer/1.0
SOAPAction: "<cfoutput>#variables.soap_action#</cfoutput>"
Content-Type: text/xml
Content-Length: <cfoutput>#Len( Trim( variables.soap_envelope ))#</cfoutput>
</cfsavecontent>

Standard HTTP/1.0 headers. You can make the User-Agent field just about whatever you want.

Also worth mentioning is that I didn’t try to combine the two (HTTP headers and envelope) into a single variable. This is because how CFSAVEDCONTENT handles newline characters is different between ColdFusion 8 and ColdFusion 9. By keeping them separate this code is able to operate under either version.

The empty println() commands are there to send a newline. And the flush() call simply sends anything left in the buffers, thus completing the sending of our request.

09: This is where we read the response from the server. The response can only be read one line at a time, so we need a loop to keep reading one line at a time and appending it to the variable where we’ll store the entire response (variables.result).

A tricky thing working between ColdFusion and Java objects is that if you assign the return value of a Java object method to a ColdFusion variable and that return value is null then the ColdFusion variable is unset or deleted. Since the readLine() method returns null when there’s nothing more to read, the only way to know we’ve completed reading the response is to test if variables.line, where we store the return value of readLine(), still exists. If it doesn’t, we’ve reached the end of the response.

10: Close the connection

The ColdFusion variable variables.result will contain the entire response from the server, including the response HTTP headers. Meaning if you want to get a proper XML object from xmlParse() you’re going to need to first strip away those headers.

<cfset variables.temp = REFindNoCase( "content-length: ([0-9]+)", variables.result, 1, "True" )>
 <cfif variables.temp.len[1] GT 0>
  <cfset variables.length = Val( Mid( variables.result, variables.temp.pos[2], variables.temp.len[2] ))>
  <cfset variables.xmlResult = Trim( Right( RTrim( variables.result ), variables.length ))>
</cfif>

This should do the trick. Although you may want to throw in some code to strip out the status code to ensure a good transaction.

And there you have it. Handling transactions with a third-party web site using client certificates under ColdFusion using only native ColdFusion objects.

Washing Client Certs in ColdFusion with SOAP – Part 2

In part 1 I introduced you to basic SOAP consumption in ColdFusion. Let’s see where things go from there.

The task at hand was to integrate our system with a third-party site. I authenticate users locally, request a token from the third-party site that allows the user to access said site, and then redirect the user to that site with the token passed on the URL.

But how does the third-party site know to trust my requests and not the requests from other people? After all you don’t want people forging requests to access the third-party site as some other user. The solution for this third-party site is client certificates. A client certificate is either issued by the third-party site or you provide your public certificate to the third-party site and the third-party’s site is configured to trust the certificate. SOAP requests are then made over SSL using this certificate to confirm your identity and to encrypt the channel.

Starting with ColdFusion 8 CFHTTP accepts two parameters to afford the use of client certs, clientCert and clientCertPassword. The clientCert parameter points to a PKCS12 formatted file containing your public and private keys and, possibly, the certificate chain from the root certificate authority (such as VeriSign or Thawte) down to whoever issued your client certificate. The clientCertPassword parameter contains the password with which the PKCS12 file is encrypted.

A note to ColdFusion developers:
The PKCS12 must be encrypted with a password! For whatever reason (I believe a limitation in the underlying java.security.KeyStore object) your cert must have a password. This is never explicitly stated in the ColdFusion 8 documentation.

I obtained my client certificate and set out to start writing code to talk with this third-party.

The first problem I encountered was I would not be able to use the CreateObject() method covered in part 1. The reason being that there was no way to provide my client certificate to the object. So it’s back to the CFHTTP method.

The second problem I encountered had nothing to do with ColdFusion and everything to do with the documentation provided by the third-party. Turns out the header for the SOAP request changed considerably during development. I had been given some early development documents that did not reflect the current header structure. Once I realized the problem and obtained the current documentation I was able to correctly construct my SOAP request’s envelope header and…

I got another error.

But this time it wasn’t a normal SOAP request error. The CFHTTP object’s filecontent value contained nothing more than “Connection failure”. But the HTTP status code was 200, which indicates a successful request. Previous SOAP request errors would return a 403 status code. This was odd.

In searching the Adobe forums and the internet in general I found sparse comments about possible problems with CFHTTP handling SSLv3 sessions, although there wasn’t any sort of official comment or response to the few reports of this problem. I loaded up the developer edition of ColdFusion 9 on my own computer to see if perhaps this problem had been resolved with the latest copy of ColdFusion. It had not.

To confirm this as a problem with ColdFusion and not my client cert, or my SOAP request, I installed Apache and PHP locally and ran the equivalent PHP code. The PHP code worked perfectly. I started trying to do as much comparison between the two platforms as I could. I event went so far as to run a packet capture on the PHP and CF requests (pointing them at a dummy, local page that wasn’t encrypted so I could see the requests) and compared them to make sure everything was the same, which they were.

Eventually I found a post online that mentioned CFHTTP wasn’t up to the job, but a third-party custom tag written in C++ did work just fine. That custom tag is CFX_HTTP5. I downloaded a demo copy and installed it locally. How CFX_HTTP5 handles client certs is different from CFHTTP. Rather than simply pointing the tag at the client cert, I had to install the client cert into the Windows certificate store and then point the tag at the store. There is a bit of work involved with it, although nothing too difficult and it’s all covered in the CFX_HTTP5 documentation.

Once I had the tag and the certificate imported into the local Windows certificate store, I rewrote the CFHTTP call using CFX_HTTP5 and it worked! The SOAP envelope was the same, the headers were the same, the only difference between the two was the logic underlying the tags.

Something is broken with CFHTTP and it can’t be used to do some operations using client certificates. But at least there is an alternative.

However I didn’t like the alternative.

First, it’s a Windows-only solution. We’re out of luck if we’re running ColdFusion on a Linux machine.

Secondly, the client cert must be stored in a place that the ColdFusion process has permissions to access, and it is accessed without needing to know the client cert password. The result is that any person with permission to create CFM scripts that are executed under this process could authenticate against this third-party web site. In a shared hosting environment this can create a serious security issue. The only solution is to separate the process, but that probably means a separate server and a new OS license and hardware costs. If that option isn’t available you have some problems. You might register the CFX with a name that contains random characters and hope none of the users in the shared environment know how to enumerate registered custom tags. I’m not sure if that is possible, but I’m willing to bet it is.

Long story short, I don’t like the CFX_HTTP5 solution.

So, in Part 3, I go back to the drawing board.

Washing Client Certs in ColdFusion with SOAP – Part 1

Recently I was asked to look into integrating our systems with an external application via the third-party’s single sign-on system. The way it works is simple enough. We would have an application that authenticates the user through our system and then sends a request to the third-party asking for a token to sign the user into their system. The third-party would return the token that I then give to the end-user and redirect them to the third-party web site. This token is how the third-party would authenticate the user into their system. (The token is nothing more than a long string of characters that is passed on the URL of the redirect.)

The method of obtaining the token is also fairly simple. The application submits a SOAP request over an SSL session to the third-party’s authentication server and that server would respond with the token. SOAP, for all the technical specs and other crap, is very straightforward. It’s a simple XML document consisting of a root element called ENVELOPE which contains two children called HEADER and BODY. The header isn’t always required and the body typically contains elements with the names of various fields the SOAP function your calling requires with each element containing the value of that parameter. Very straightforward.

A SOAP request in ColdFusion couldn’t be simpler, especially with the CFSAVECONTENT tag. Simply construct your envelope inside a CFSAVECONTENT tag and then use CFHTTP to submit the request. It looks a little something like this:

<cfsavecontent variable="variables.soap">
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <GetCurrentTime xmlns="http://ws.historicaloptiondata.com/" />
  </soap:Body>
</soap:Envelope>
</cfsavecontent>

<cfhttp
  url="http://ws.historicaloptiondata.com/Service.asmx"
  method="POST"
>
  <cfhttpparam type="header" name="SOAPAction" value="""http://ws.historicaloptiondata.com/GetCurrentTime""" />
  <cfhttpparam type="header" name="Content-Length" value="#Len( Trim( variables.soap ))#" />
  <cfhttpparam type="xml" value="#Trim( variables.soap )#" />
</cfhttp>

<cfdump var="#xmlParse( cfhttp.filecontent )#" />

You should be able to plug this code into a CFM file and run it without having to touch a thing and you should see the CFDUMP of an XML object. (I say “should” because xmlParse() seems to first try and open a file with the name of the content of the passed variable and, when that fails, treat the passed value as an XML document itself. This can trigger errors and make it unusable if you employ any sort of file operation restrictions on your server. In which case modify the code to remove the xmlParse() call and just dump the cfhttp.filecontent.)

A few notes about this code SOAP in general.

• This is an example of a SOAP 1.1 request. There is another, slightly different format known as SOAP 1.2. The major differences between the two are that the content-type for 1.1 is text/xml, but for 1.2 it is application/soap+xml. Also the SOAPAction HTTP header is no longer needed in 1.2.
• The SOAPAction HTTP header’s value must be wrapped in double-quotation marks. And sometimes the first character after the open quotes will need to be a pound (#) symbol. This means ColdFusion programmers will need to be certain they escape these special characters in their values.
• If the CFHTTPPARAM type “xml” is present, CFHTTP automatically sets the content-type to text/xml. I am not sure if it’s possible to override this, but I believe not, therefore you’re almost always going to have to stick with SOAP 1.1 if you’re using CFHTTP for your SOAP requests.
• You must Trim() the variables.soap variable! The newline at the beginning of the value, which exists because there is a newline immediately after after CFSAVEDCONTENT tag (for visual formatting purposes) will make the XML document you’re sending an invalid XML document and result in errors.

Now comes the WSDL file. A WSDL file is an XML documents that describes the functions and parameters of said functions available through a SOAP service. The CreateObject() function has a “webservice” object type which will consume a WSDL file and create an object with all the available functions offered by the SOAP service. All the stuff with XML and ENVELOPES and CFHTTP becomes transparent and, as it turns out, SOAP can be simpler than my previous example. The above code can be reduced to the following using CreateObject():

<cfset ws = CreateObject( "webservice", "http://ws.historicaloptiondata.com/Service.asmx?WSDL" ) />
<cfset ws.getCurrentTime() />
<cfdump var="#GetSOAPResponse( ws )#" />

This makes this life much simpler for ColdFusion programmers. No need to worry about what version of SOAP you’re using or what URLs you need to submit your request to, the formatting of your SOAP envelope, the SOAPAction HTTP header variable, etc. It’s all taken care of for you by ColdFusion.

So this integration I was asked to do should be a piece of cake, right?

Wrong.

Simple SOAP is simple. Complex SOAP… well, you’ll see in Part 2.

Acrobat Open Parameters

Recently I was tasked with updating the college catalog. Not the content, that was already generated for us and delivered in PDF format by the publisher. I needed to update the web site and the links into the PDFs. Most of the links went to just the PDF itself, but a few pointed to named destinations inside the PDF. These bookmarks don’t exist in the documents we receive from the publisher — we have to create them ourselves.

But I don’t have Acrobat Pro installed on my computer. If the college orders me a copy today we’ll probably have the install DVD  next week, but I don’t want to wait that long.

So what options are at my disposal? Is it possible to link to a specific location inside a PDF without having to create bookmarks inside the PDF?

After scouring the interwebs a bit I find this document on Acrobat Open Parameters.

I can link to a specific page by simply appending to the PDF’s url #page=2 and, magically, Acrobat Reader will load the PDF starting on page 2.

Cool.

But some of the items I’m linking to inside the PDF start halfway down the page. A user clicking on the link might not see the section they’re being linked to within the top area of the page that is visible on their screen and immediately assume they’ve clicked on the wrong thing.

So can we do something more with these URL parameters?

Yes!

The VIEW parameter allows you to define how the PDF is scaled in the browser (fit to window, fit horizontally to window, fit vertically to window, etc.) as well as where on the page to start based on a x,y coordinate system.

In this case I want to set VIEW to FitH (Fit Horizontally) meaning the PDF will size itself so the width of the document is the same as the width of the browser’s viewport. This also allows us to specify a Y component as to the position of the page displayed when the PDF is loaded. Combine this with the PAGE parameter and you can define where in a given page the PDF will start when loaded.

So a link like document.pdf?page=7&view=FitH,300 will load the PDF on page 3 at 300 units down the page.

Now what unit are we talking about? And is 300 units on my screen the same as 300 units on someone else’s screen? I have no idea. The document is light on details, but I believe the coordinate system is based on the parameters of the PDF itself and not the end-user’s screen. So with a little bit of trial-and-error I find just the right number to use with the FitH view and voilà!

I can now link to the Masters of Arts in Teaching program within the Secondary Education section of the catalog. And I did it without having to pay for some bloated Adobe software.

Granted, with named destinations updating the catalog next year wouldn’t require that I touch those links (just the PDFs we get from the publisher) and maybe it’s better from a semantics standpoint, but this was quick and easy.

Will it work with PDF readers other than Acrobat? I don’t know. If you’ve got one installed try it out and let me know. The fact that these “open parameters” have been around since at least Acrobat 6 it’s possible other PDF viewers have incorporated this feature into their application.

But, hey, it works with the most popular and free PDF reader and it didn’t require expensive software to accomplish. That’s good enough for me.

Equal height columns with borders (and a gap).

I was recently reading an article over at Smashing Magazine about new CSS techniques. One of them was about columns with equal heights and a border. The solution presented requires a background image to be used to fake the gap between the two columns. As I prefer to stay away from images and control the layout with straight CSS, I took a shot at this without any images. This is what I came up with.

Basically I’m using an empty DIV that’s absolutely positioned and with an absurd height value (999,999 pixels) to fake the gap between the two columns rather than use a background image. I then wrap everything in another DIV and set the overflow value to hidden. This hides everything but the bit of that very tall DIV needed to produce the effect. I’m basically clipping the very tall DIV and the clipping is done dynamically (based on the height of the content). The advantage with this solution is that I can control the color of the borders and background with just CSS — I don’t have to create a new image every time I change my color scheme. The downside is you need a couple star-HTML hacks for IE 5 and 6, plus you have to add extra markup in your HTML document that serves no purpose other than to create visual layout objects (which HTML purists feel is wrong, and I kind of agree with them).

Why do we have to fake the visual appearance of two columns and why is it so difficult?

Good question.

HTML block elements that have no height value explicitly defined (such as via the height CSS property) will be take whatever height is needed to contain all its children. So if you’ve got a very large image inside a block element, the element will grow very big — just enough to contain the image. If a small image is placed inside a block element, the element will only grow a little bit, just enough to fit the small image.

So create two block elements that will be our columns. Float them left so they sit next to each other. Now put a border around them. What you will see is that the block elements will only be as tall as the content (the text) inside each column. The columns will not be equal height (unless you’ve put the same content into both block elements).

If you want to create the appearance of two columns with the same height, you’ll have to fake it.

How?

Put your columns inside another block element. Your left and right columns will still have different heights, but the block element that contains the columns will grow as tall as it needs to contain both columns, in other words it will grow to be as tall as the tallest column. With this you now have an element you can apply some CSS to which you know will always be as tall as the tallest column in your web page.

What CSS is that?

Depends. A common approach is to set the background color of the block element that contains the two columns (let’s call it a wrapper since it wraps around the columns) to the color you want your left-hand column’s background to be. Then create a border on the wrapper set to the width of the right-hand column and set the color of that border to the color you want the right-hand column’s background to be. You can then use CSS to position that right-hand column over the border of the wrapper. To the viewer it appears as if there are two separate columns with different backgrounds. You can see an example of this in action in my Skidoo Redux layout.

The trick I presented earlier in this post can then be used to add a border around each column.

But Skidoo already has borders around its columns.

Yes, but those columns share borders with other columns. The trick I present here allows each column to have its own set of borders, not shared with any other column (in other words, there’s a gap between the columns). That’s what makes this trick different.