There you will find Amazon taking the opportunity to tell you how awesome Silk is and then direct you to the Amazon Silk Terms and Conditions if you’re still interested in this silly privacy nonsense. However there is one small, but important piece before we move on to the ToC:

What about handling secure (https) connections?

We will establish a secure connection from the cloud to the site owner on your behalf for page requests of sites using SSL (e.g. https://siteaddress.com).

Amazon Silk will facilitate a direct connection between your device and that site. Any security provided by these particular sites to their users would still exist.

If you use Amazon Silk then Amazon will be able to see everything you do over HTTPS. This is bad. Very, horribly, bad. And should be a red flag for anyone considering purchasing a Kindle Fire. Now about that ToC document.

Straight away they give you the section on privacy and tell you that the Silk ToC includes the Amazon Privacy Notice. They’ve even gone and underlined it for you to make sure you see it. Curiously they don’t actually link to the document. Isn’t that a bit odd? I’ll help you out in a way Amazon.com won’t and point you to the Amazon.com Privacy Notice here.

The basic idea of the Silk ToC is that they keep your information for “generally” no longer than 30 days and that everything you do over the web using Silk Amazon.com sees. Couple that with the privacy notice and Amazon has given themselves the right to look at and analyze everything you do over the web with Silk.

However, there is one piece of news in the ToC:

You can also choose to operate Amazon Silk in basic or “off-cloud” mode. Off-cloud mode allows web pages generally to go directly to your computer rather than pass through our servers. As such, it does not take advantage of Amazon’s cloud computing services to speed-up web content delivery.

So it will be possible to use Silk without putting Amazon between you and the web. It’s unfortunate this option is buried in Amazon’s documentation and I suspect this option will be just as buried in Silk, but it is there. And that’s good, because the Kindle Fire does not support Android Market which makes installing an alternative browser a bit problematic.

If I were in the market for an Android tablet I don’t think Silk would turn me off the Kindle Fire, however I am the type who likes to have as much control over my own hardware as possible and I don’t feel I would have complete control over a Kindle Fire in much the same way I don’t feel I’d have complete control over an iPad. Therefore I would probably look elsewhere.

But included with the Fire is a new web browser called Amazon Silk. And Amazon would like you to know this is a new and different browser from anything you’ve seen before.

So what’s the big deal about Silk? All you web requests go through Amazon’s servers which will handle retrieving all the content for you and optimize it wherever possible (e.g. a 3mb jpeg that’s sized to 300×200 pixels will be resized by Amazon before being sent to you as a 50k jpg). Because Amazon’s servers are sitting on some of the fattest pipes on the interwebs, it’ll be able to pull down and deliver all this content to your Kindle significantly faster and more efficiently than if your browser was doing the work all by itself.

Sounds pretty nice, but I’m a bit of a pessimist.

I wonder if the image resize example given in the above video might have unexpected consequences. For example there may be web applications that purposely load a large image into your browser and allow you to zoom in and out and move around the image. Will Amazon’s services understand that scenario and know to not shrink the image? I can think of a few examples that use image clipping and revealing the full image when hovering over the clipped area using CSS. Will Amazon know that the clipped area is not the only part of the image being displayed? Perhaps only known situations are optimized rather than Amazon using software to guess.

My biggest worry, however, is that all your web browsing is now going through a third-party. If Amazon is making requests on your behalf it will need to preset session cookies to those sites your browsing. What happens when you need to log into a system over SSL? Does Silk make the HTTPS request through Amazon’s? Does that mean all your passwords will be, at some point, on Amazon’s servers? What happens if they’re ever compromised? Does Amazon log and can they track your browsing history? What happens when I try to go to Barnes & Noble to buy something online through Amazon’s servers? Some web sites use session-hijacking prevention by comparing looking at things like your IP address. Amazon’s servers, as they point out in the video, are all over the world. Will my IP address stay the same throughout a session or will it change as requests are routed through different Amazon servers? Some web applications might break because of that.

Amazon must be logging your browsing with Silk. Imagine a scenario where someone posts some illegal material through Amazon Silk. Authorities will track down the IP which will lead them back to Amazon. Amazon must then have some mechanism to identify the user who posted the illegal material otherwise Silk becomes a giant anonymous proxy machine.

I’m very wary of Amazon Silk. I do not think I would never use it unless forced into a situation where no alternative was available. I don’t want some third-party sitting between me and the web sites I interact with, watching and recording everything I do.

Now that’s out of the way, let’s talk about a situation that came up over the weekend. I was looking at a particular layout I’ve developed and lamented that the solid color background felt a little too empty. What it needed was some kind of texture to make it more visually interesting, but no so much that it takes attention away from the actual content of the page. What immediately came to mind were the linoleum tiles of an old grocery store I went to years ago which had solid color tiles with little dots of black and white color. I thought something like that might just pull off the trick of making things a little more visually interesting without taking away focus from the content of the page. So I needed to make some dots.

Creating Texture

I knew how I wanted to execute the texture: dots and short lines in colors of black, white, and a couple shades of gray with a transparent background. This way I could use this texture over any color I want and the range of colors for the dots would provide contrast with just about any background color I wanted. Perfect!

I fired up Inkscape which has quickly become a favorite application of mine. On a fresh and empty canvas I created eight lines of varying length and made them all the same color. I then selected them, copied and pasted, and with the new group of lines selected, picked a different color. I repeated this process two more times to arrive at four groups of lines with the varying colors I wanted. At that point it was a simple matter of dragging the dots around and mixing them up over a relatively large, square area. Once I felt happy with the spread I selected them all, copied and pasted, and rotated the dots around to increase the density of the dots. I did some more arranging and by the time I was done I had a blob of dots on my screen.

I then broke out the select tool and selected an area within the blob and used Inkscape’s export bitmap feature (under the FILE menu) to export the selection to a PNG file. I then created a simple web page with the background color of the layout I was working with and then set this PNG as the background image. I loaded this page up in a web browser to see how things looked. It wasn’t bad, but the dots stood out a bit too much and the tiling of the image created an obvious pattern that stood out as well. These problems needed to be remedied.

To keep the dots from standing out so much I simply selected all the dots and set their opacity to 50%, making them semi-transparent. This cuts down on the strength of the color and the half-opacity allows the dots to blend with any background color. I exported a new PNG and reloaded the test web page. The colors were more muted, but still carried the texture.

To fix the tiling problem I stared at the page and let my eyes unfocus a bit and picked out what features were standing out. For example there were a group of white dots that, together, especially stood out. I broke those dots up, exported the bitmap, and reloaded the page. The tiling pattern was less obvious, but it was still noticeable. This time there were some black dots on the horizontal edges of the exported bitmap that appeared together when the image was tiled. This grouping stood out and made the tiling pattern obvious, so I move one of the dots off and exported the PNG one more time. Things were looking good. The only problem left was a small, but obvious gap of open space with no dots. I copied a couple existing dots, stuck it into the white-space, exported, reloaded, and everything was jake.

After applying the new PNG to the layout I thought some other elements might do with a bit of texture, especially the supplemental boxes that carry the site’s menus, so I applied the dots to those boxes as well and things looked pretty good, but the background color of menu items when the mouse was over one of them was hiding the underlying texture. I wondered if it would be possible to get the dots to show through.

Transparent Maths

The problem was I quite liked the color being used for the menus’ hover state. If I was going to make this color transparent I needed it to be transparent in such a way that it blended with the background color to produce the same color I was using now without transparency.

Could that be done?

After a bit of internet searching I came across a simple formula used when blending two colors where the bottom color is fully opaque.

Final Color = Top Color * Top Alpha + Background Color * ( 1 - Top Alpha )
or
Fc = TcTa + Bc(1-Ta)

The menu background color (background color or Bc) was #f7ead5 or rgb(247,234,213). The hover state background color for menu items (the color I want to end up with or final color or Fc) was #eecc33 or rgb(238,204,51). All I needed to do was pick a suitable transparency (or alpha) value and plug the values into the formula and solve for Tc. With an alpha value of, say, 50%, the formula could be arranged to look something like this:

Tc = (Fc - (Bc - (Bc * 0.5))) / 0.5

For example, solving for red, the formula looks like:

Tc = (238 - (247 - (247 * 0.5))) / 0.5 = 229

The new transparent color will have a red value of 229. Solving for green gives us a value of 174. However solving for blue gives us a value of -111. We can’t do negative color values, valid values are between 0 and 255. What happens if we set blue to 0?

You can see that the transparent version (using CSS3′s rgba() color function) is a bit muted in color when compared to the original. In the layout I was working with this was especially noticeable as all the colors in the layout’s palette are especially vibrant.

So what can we do? The first thing I did was rework the formula to solve for the alpha value that results in a blue value of 0.

Ta = (Fc-Bc / Tc-Bc) = (51-213) / (0-213) = 0.76

So an alpha value of 0.76 is the most transparent we can get to achieve the intended color of rgb(238,204,51). Plugging this new alpha value into the formula gives me a final RGBA value of rgba(235,195,0,0.76). When we use this value we see this:

That looks spot-on. I could further tinker with the alpha value a little to see how much lower I can make it before the color becomes too washed out, but for now I’m leaving it right where it is.

If you start playing around with overlaying transparencies like this you’ll want to watch out for your color limits. I lucked out with in this situation that I didn’t encounter any upper limits, but there are sure to be color combinations that will. My suggestion is to first solve the formulas for the transparency value you want and if any value goes beyond the 0-255 limit start playing with the formulas to find what values will keep you within limits. You may find there simply is no alpha value that will keep you within limits in which case you’re either out of luck or you need to rethink your color palette.

Good luck.

The attack requires the attacker be in position to intercept and inject into the data between the client and server, otherwise known as a man-in-the-middle attack. The attack appears to use javascript to send out known plaintext over an encrypted channel and allow attackers to then crack the encryption key using that known plaintext.

The simplest solution to protect yourself is to disable javascript.

As a web developer I look at things like noscript, cross-site scripting, and exploits like this as reasons to believe that users of my web site may not have javascript enabled. Therefore it is vital that javascript not be a critical component to using any site I manage. That doesn’t mean never use it, it only means that your web site still be usable without it. Treat it like Flash, or Java, or any other function that isn’t explicitly defined in the HTML or CSS specs. These things can make life easier for the user, but should never be a requirement.

This latest exploit, with the immediate message to end-users being disable javascript until a fix is ready, puts those web developers who rely on javascript to provide critical function to their site in a bad position. Don’t be one of those developers.

Eventually the practice of at least a white-list approach (see noscript) to Flash, Javascript, etc., if not completely removing them from the browser, will become the norm for even basic web users. It may not happen this year, or even in the next five years, but it’s very likely to happen as attacks on the web layer continue to grow.

I recently did a bit of contract work which inspired me to work on a new layout which then inspired me to do some spring cleaning and share with you a bunch of old designs I’ve started over the last five years, but never got around to finishing. So hang on, there’s a bunch of new stuff to get through.

Simplify

Simplify is the new layout that I put together over the last week. I never intend for the layouts I create to be so damn boxy, but inevitably that happens. Furthermore most of the layouts I’ve been working with have multiple stylesheets and that can be a bit overwhelming especially to those relatively new to CSS. Simplify is something of an answer to those feelings of “I don’t like that” (which seems to be a primary motivator for me).

The layout is still a bit boxy, there are corners, but there are no edges to the page for the purpose of making the layout feel more open. The styling is intentionally plain (given the name you should expect that) with just black text on a white background.

There’s a famous Dutch artist named <a href="https://secure.wikimedia.org/wikipedia/en/wiki/Piet_Mondrian"Piet Mondrian who got famous for painting a few boxes in solid, primary colors. I never understood that and when I enrolled in an introductory art class a few years ago my single goal was to come away with an understanding of Mondrian and I think I was successful. The idea is Mondrian was breaking down art into it’s most basic elements, a bit like scientists trying to find the most basic form of matter (atoms, quarks, etc.) From that breaking down we look to define the most basic properties of what stuff is, be it art or matter, and hopefully find a fuller understanding of the whole.

I think Simplifyis a step in that direction. I’m not down to quarks; probably closer to particles of matter with atoms, protons, and quarks still to come.

Portal

Portal is a layout I originally created for an application that would act as a transitional phase between and old portal system and a new, yet undefined system. Unfortunately the pseudo-portal application I wrote never caught, but I can at least share the layout. There’s a lot of generic styles defined in the stylesheet to be used in configuring both the layout of the page as well as individual portlets. I like the color scheme I had picked out (which is a rare thing for me) and I’d hate to see this thing go to waste so I’m sharing it in the hopes that others might find it useful.

Experimentation Chamber

The Experimentation Chamber has been around for quite a while, but you could only get into it if you ever noticed the little door icon in the upper-right corner of the layouts page (which very few did). So now I’m not only drawing attention to it, but also to the fact that I’ve added quite a lot to it.

Specifically you’ll want to check out the Abandoned/Unfinished Layouts section where you’ll find crap I started on and eventually forgot about over the years. The Flux layout is actually used right now in a production environment, but all the tweaks and changes made to that version haven’t been carried over to the version on the layouts site which is why it’s under the unfinished category.

There are a couple layouts in the wild that I produced which I can tell you a little about. One is for a blog called Team Dick and another is for Gargoyles on Station Eight. The S8 one is a couple years old, but I think has held up pretty well. I may clean that up, change the drop-down menu shadows from images to CSS3 box-shadows and release it into the public domain. The Team Dick layout won’t be released into the public domain for a while as it’s relatively new and I feel it still needs a bit of tweaking.

The rest of the chamber is worth going through, but the layouts are probably what most people are interested in.

Xanadu

Speaking of Team Dick, there’s a post on there that might actually interest people reading this blog. It’s a post on Xanadu which is a life-long project of Ted Nelson. Xanadu has some really interesting ideas about it that set it apart from the world wide web. It’s an interesting read if you’re into that sort of thing.

That’s about it for my biannual blog post.

Have fun!

Responsive web design is something that’s been around since the first web designs came out utilizing percent values to define the widths of objects, such as TABLE, DIV, and IMG elements. The idea was that a person on a 640×480 resolution screen can view the web site, but a person with a larger resolution like 1024×768 would be able to see more content on their screen as the layout would expand to the edges of the larger viewport (the area in which a web page is rendered). This idea was somewhat radical back in the early 1990s as most layouts would simply set a fixed width for the layout (usually 780 or 1000 pixels) and state quite plainly on the web site that it was designed to be “best viewed in 1024×768″.

Things got more interesting with CSS. Web designs could now use percentages for margins, padding, and placement of elements within a page. Layouts that would expand and contract with the viewport could be more complicated. Especially handy were the CSS max-width and min-width attributes. These allowed you to set limits on just how much your layout would expand or contract before forcing the viewport to stop resizing the layout. Now layouts could expand and contract, but not so much that they became unusable.

Enter CSS3 media queries.

But first, a small caveat to web developers.

The CSS3 spec is not yet final. In fact there isn’t (or is, depending on your point of view) really a CSS3 spec at all. CSS3 is a collection of modules and therefore somewhat open-ended. You can see a list of some of the CSS3 modules and their curren status here. Some CSS3 modules are currently in an early format and not recommended for implementation, while others have reached a state where it’s not quite final, but browser authors are encouraged to implement these modules and supply feedback to help refine the spec before it becomes final. Media queries are in such a state and as such several modern browsers have already implemented them including the latest releases of Firefox, Chrome, Safari and Opera. So web developers can start making use of them now and those users who run modern web browsers will be able to enjoy your use of CSS3.

At the same time, I do not recommended you rely heavily on CSS3 modules in any designs for the near future, or that if you do, you do so while making sure to check compatibility of the layout against older browsers. While the general public appears to be more capable at keeping up to date with their browser (mainly thanks to the automated update features of several OSes and the increased proliferation of broadband internet connections),  there’s still a significant number of users (over 50% according to this site) who are using Internet Explorer 8 or earlier, which does not support media queries (or much else in the way of CSS3). IE9 will support media queries.

Now that that’s out of the way, what can media queries do for us? Simply stated, you can apply styles based on many attributes of the viewport. To keep things brief-ish and to the point, I’m going to specifically focus on width-related media queries. Let’s see what one looks like:

#frame { width: 1000px; margin: 0 auto; }
@media screen and ( max-width: 480px ) {
  #frame {
    width: 100%;
  }
}

The above example will make an element with the id of “frame” to be 1,000 pixels wide and will center (via the margin value auto for the left and right margin properties) it in the middle of the viewport if the viewport is wider than 1,000 pixels. On the second line is a media query. It begins with @media screen (in place of “screen” you can use “all” for any media, or “print” for print media, etc.) which lets the browser know that the following block of CSS should only be applied to the screen. This is followed by and ( max-width: 480px ) which is the media query. It states that the following block of rules should also only be applied when the viewport is at or less than 480 pixels in width. You can also use min-width to tell the browser the block of rules should only be applied when the viewport is at least 480 pixels wide.

Why would you want such a thing? If a viewport is that narrow and you have wide elements or you have multiple columns, chances are those elements would be very usable at such a narrow width. With a media query you can change the layout to something more usable on a small screen. For example, consider a layout that has multiple columns. At 480 pixels your columns will barely fit a few words per line, if any at all. Reading text becomes very difficult. With a media query you can drop your multiple columns into a single column layout. Now each column will have the full width of the viewport to render its content, making it much more usable.

In the example above I am removing the fixed-width on the #frame element and resetting its width to 100% of the viewport. Smartphone users will be able to view the content of the layout much more easily and without having to scroll horizontally to read each line.

Excellent! Now go and design for the masses!

But… not quite yet. There’s one problem. Some smartphones are a bit too smart. They understand not many web designs are out there that take advantage of these features. Instead these smartphones will scale the rendering of the page. While the screen may only have 480 pixels of width, it’ll render web pages as if it had 1024 pixels of width and then scale it down to fit the screen. Media queries alone won’t fix this problem.

However, there is a solution.

The viewport meta tag allows you to tell browsers how to scale the web page. Most mobile browsers understand this tag including the mobile version of Safari found on may Apple devices and the mobile version of Opera which is found just about everywhere else. So to fix the mobile browser scaling issues, just include the following META tag into your web page:

<meta name="viewport" content="width=device-width; initial-scale=1.0;">

This tag tells the browser to render the web page with a 1-to-1 pixel scaling. Meaning 1 pixel of your web site is 1 pixel on the device’s screen. There are many other attributes you can include in this META tag such as maximum and minimum scaling values or whether the user is allowed to scale the page. Disabling user scaling effectively disables the pinch-zoom feature of many mobile devices. I recommend against it.

One last thing to touch upon before I give you a working example. It’s a simple block of CSS that will help make everything run much more smoothly on a mobile browser:

img { max-width: 100%; }

This little CSS rule allows images to shrink as needed as their containing element shrinks. So a 600 pixel wide image will shrink enough to fit on a 480 pixel wide browser. It’s a cheap, but very useful trick. However I would suggest that you don’t use the exact CSS rule given above. To apply such a thing to all images may have unexpected negative effects on images that you don’t want to have shrink as needed. Instead use a more specific CSS selector such as

#frame .blog-post .post-content img { max-width: 100%; }

The more specific you are, the less surprises you’ll have down the road.

As promised, I have put together a demonstration web page that utilizes all these techniques. Grab a modern browser that isn’t Internet Explorer, open the layout, and start shrinking and growing your browser’s window to see the content and layout change to fit within the width of your window. Pop it open on your smartphone’s browser as well!

MARS: Responsive Layout Demo

The stylesheet is embedded in the source to encourage you to take a look at the whole source for the page. It has also been given some CSS hacks so that the layout will work even in IE 7 and IE 6 (in theory), but all the media query stuff will obviously not work.

One final note.

At home I have a very wide (16:10) monitor, while at work I have a standard (4:3) monitor. I like to set max-widths to my layouts because keeping the distance from end to end of a given line of text small helps reduce eyestrain and the whitespace on either side of the layouts is more visually pleasing. But on a very wide monitor, the whitespace on the sides is too much and it makes the layout look very small. With media queries I can fix this problem by increasing the max-width of my layout frame for those screens with a very large width (over 1280 pixels). I could also increase the font size to help with readability. The point is that media queries aren’t just for keeping your pages usable on a smartphone, but they also help keep them usable on very large screens as well.

There are a few things worth talking about such as the situation with the HTML spec. W3C maintains one HTML spec, while WHATWG maintains another. They were playing nicely, but there’s been a small shake-up recently with WHATWG deciding to drop version numbers from the spec, while the W3C will continue to call the latest spec HTML5. How the hell did we wind up in this situation? Well WHATWG has a great piece in their HTML spec that gives a brief history of the HTML spec. It’s not too long and very much worth reading to get an idea of where HTML came from and where it’s going.

News web sites tend to have a template ready to go whenever there’s “breaking news”. It’s often a big, brightly colored box with bold text usually stating “BREAKING NEWS”. This is quite an attention getter. A fact that is not lost on modern news web site developers. Lately “BREAKING NEWS” web stingers are being used to report extremely mundane news. The reason they are using these stingers at all is to simply grab your attention and get you to click through to read the full story. The problem is these news sites are quickly diluting the news value of their “BREAKING NEWS” stingers and sooner rather than later we’ll start to ignore them alltogether. So what will news web sites do when real breaking news happens? I have some thoughts on that which I’ll save for a later blog post, but it’s something to consider.

A small gift to those who have read this far. Here is a template I put together last week to test palette options. This isn’t a layout. You could use it for a web site, but I haven’t done anything to make it compatible with older browsers. It’s more just to test potential color palettes for web sites. And a great place to find palettes is COLOURlovers.

Now that I’ve given away one of my design secrets, I’m off. I expect there will be less of a delay between this and my next post.

To the individual this means being able to manage your content from any location that provides some form of internet access. Things like gmail, Google Docs, Flickr, WordPress.com and YouTube are examples.

To the corporation it’s a form of outsourcing. Gone are the days of large data centers to manage corporate information. Now the information is stored in the cloud. Now it’s someone else’s nightmare to manage.

It’s also a security nightmare for those who dare to take a moment to consider the security costs rather than the monetary costs associated with “the cloud”. You lose control of your information. You’re putting it into the hands of a third-party. You may have a contract with them that makes them responsible for any security breaches. In fact some managers prefer the cloud specifically because there’s a security contract that, in legal form at least, takes responsibility off their heads. But that doesn’t mean a security breach won’t happen. And when it does, when the genie is out of the bottle, what becomes more significant, that the information is out there or that you’re not going to get fired?

Sadly, I think most IT managers would answer the latter.

Privacy is one aspect of security, but it’s a concept that’s slowly starting to catch on. The public is slowly and painfully becoming aware that putting all their information out on third-party sites, that probably doesn’t have the individual’s best interests in mind, is a bad idea. The recent stir surrounding Facebook’s privacy issues is one example of this.

And as much as I would love to see this catch hold and become a driving force that tears apart “The Cloud” and everything “2.0″, it won’t. People will be quick to forgive or forget or to tolerate those privacy issues in return for easy access to information and entertainment.

“The Cloud” is probably not going to go away. In fact, it’s probably going to be the future. And it will be incredibly attractive.

I’m going to focus mainly on “the Cloud” from the individual perspective.

Imagine an iTunes subscription. You could stream any music you want to any device you have, be it your phone while on the way to work or a tablet while you’re at home reading a book on it or a set-top box at your friend’s house who needs some background music for his or her party. Or from your hotel room while on a trip. You don’t have to carry any actual electronic device with you. Your music is in “The Cloud” and you can stream it from any electronic device.

Now apply the same idea to NetFlix or Hulu. Actually, NetFlix and Hulu (in many ways) already do exactly this.

E-mail? Photos? Documents? Gmail. Flickr. Google Docs.

All your information is in “The Cloud”. Ready for you to pull it up on any internet-enabled device, from your iPhone to a computer in an internet cafe halfway around the world.

This type of access already exists, but the interface is too clunky. I imagine a near future with some sort of set top device that you plug into a television that would provide all of this for you. It would become a common feature at most hotels and certainly everyone from grandma on down would have one in their house. No real computer needed anymore, just a thin client with a web browser and maybe a hardware video decoder.

And for the high-end user there would be the portable device. Something like an iPhone4 but with extra features like a micro projector to watch movies on the wall (who needs a TV?) and a small suction-cup device that turns any flat surface (window, table, etc) into a large speaker to provide clear sound.

There would be movie parties. Where everyone would sign into a private room on NetFlix and could watch the same movie and talk to each other, while each person sits in their own room separated by hundreds of miles.

Friendships would no longer need face-to-face meetings. Everyone becomes an avatar. A projected personality that may or may not relate to the individual’s physical presence.

Soon meetings would be conducted in a similar fashion. No longer do we need a boardroom, simply start up your telepresence app and everyone sees and hears each other without ever having to leave their cubicle.

Of course in such a world a room full of cubicles with dozens of separate conversations going on would create quite a bit of background noise and interference. Which is why cubicles would become something more like miniature offices with sound proofs walls and no windows.

You don’t really need to see outside. Just start up the local weather app to project an image of what it looks like outside; tailor-made to your preferred surroundings of a wooded area or urban setting.

Eventually such archaic devices like projectors, speakers, and microphones will be made obsolete with brain implants. Telepresence becomes more real. You don’t just see, you feel and smell and taste and touch. Physical contact with others can be achieved even though your thousands of miles apart.

The porn industry reaches its full potential. It’s not prostitution anymore, it’s all virtual. Every depraved and deviant fetish is now catered to, and it’s all virtual, it’s all fake. But it will feel very real. And what is real, but signals processed by your brain. It is real.

And one day some alien race might finally find our planet. Perhaps touch down and have a look around. They’ll walk the halls of large buildings filled with small, personal-refrigerator sized cubicles. Automated machines keep everything running smoothly. Perhaps interest gets the better of them. They peek inside one of the refrigerators and find a curious gray mass in a container of goo with some probes.

Say hello, then, to your grandchildren a million years from now.

This is important information if you’re ever going to develop code in ColdFusion that needs to, for example, figure out whether or not a domain account has expired.

Another piece of information that’s handy to have is an understanding of how ColdFusion handles large numbers. Numeric values are handled as signed 32-bit values that can range from -2,147,483,648 to 2,147,483,647. ColdFusion can handle much larger numbers, but it does so by converting numeric values into strings and then using its own library to handle math operations based on these strings. And it stores these large values in scientific notation. The problem with this is you start to lose precision. For example, if you run the following CF snippet:

<cfoutput>#( 2147483648 * 2147483648 )#</cfoutput><br>
<cfoutput>#( 2147483648 * 2147483648 + 1)#</cfoutput>

The output is the same for both operations, “4.61168601843E+018“. This loss of precision makes it impossible to calculate the exact time account-expires refers to. So what else can we do?

One trick that sort of works is to divide the value in account-expires
by 10000000 to convert from 100 nanoseconds to seconds. This will significantly shorten the number. So let’s try that on an account-expires value of “129247307050000000″ (that’s 7/27/10 @2:58.25 EDT). The line of CF looks like this:

<cfoutput>#DateAdd( "s", ( 129247307050000000 / 10000000 ), "01/01/1601 12:00 AM")#</cfoutput>

Try to run this code and you get the error message “Cannot convert the value 1.2924730705E10 to an integer because it cannot fit inside an integer.”. It turns out that DateAdd() can’t handle a number larger than 2147483647 (the limit of a 32-bit signed integer). 2,147,483,647 seconds is about 70 years of time. To use DateAdd() we need to make the account-expires value much smaller. We could, for example, calculate the account-expires value with respect to a different epoch, one that’s within 70 years of the time referenced by account-expires. We could use any date we want, but I’m going to stick with Java’s epoch of January 1, 1970 to keep things in-line with what’s probably the most common epoch among computer systems.

The number of seconds between January 1, 1601 and January 1, 1970 is 11,644,473,600. If we plug this into the code above we get this:

<cfoutput>#DateAdd( "s", ( 129247307050000000 / 10000000 ) - 11644473600, "01/01/1970 12:00 AM")#</cfoutput>

Run this and we get a result! Specifically {ts '2010-07-27 19:58:25'}. Remember this is in UTC, we’ll need to convert to our local timezone. We’ll do that in just a moment. But first…

You should have recognized by now that this solution won’t work forever. It will fail whenever the data in account-expires is after January 19, 2038. Perhaps this doesn’t bother you since whatever applications you’re developing now will surely be either obsolete or running on a more modern ColdFusion system where DateAdd() can deal with larger numbers before account-expires with that large a value are in use.

Perhaps. But I hate making assumptions like that. Plan for the future! Assume nothing!

If we want lasting code free of the 2038 limitation we’ll need to steer clear of ColdFusion date and time functions. We can do this by using Java objects, specifically java.util.Calendar to manage date and time, and java.math.BigInteger to handle the very large numbers we’ll be working with.

The process is fairly straightforward, but there are a few things to watch out for. Java time is measured in milliseconds, not 100 nanoseconds, so we’ll have to do some conversion. Java and Microsoft’s epochs are different. Calendar objects with a date and time earlier than Java’s epoch will have a negative millisecond value. And we need to calculate this date/time with respect to UTC (GMT timezone) not the local timezone.

First create a BigInteger object. Initialize it to the value of account-expires, and then convert it to milliseconds as that’s the unit of time Java likes to work with.

01: <cfset variables.bigInt = CreateObject( "java", "java.math.BigInteger" ) />
02: <cfset variables.expTime = variables.bigInt.init( JavaCast( "String", "#arguments.accountExpires#" )) />
03: <cfset variables.expTime = variables.expTime.divide( variables.bigInt.valueOf( "10000" )) />

You can initialize BigInteger objects with strings and numerical values. I’m specifically casting account-expires as a string so that ColdFusion doesn’t treat it like an integer and try to put it into scientific notation. Note in line 3 the use of BigInteger’s valueOf() function. This takes a string and returns a BigInteger value. Math functions for BigInteger objects require BitInteger arguments, thus we need to pass all values (even static ones) through valueOf() instead of JavaCast().

Next we set up the calendar object. This means creating a java.util.TimeZone object, a java.util.Calendar object, and then initializing the calendar to the GMT timezone. Once the calendar object is initialized we set it to Microsoft’s epoch.

04: <cfset variables.tz = CreateObject( "java", "java.util.TimeZone" ) />
05: <cfset variables.Calendar = CreateObject( "java", "java.util.Calendar" ) />
06: <cfset variables.gCal = variables.Calendar.getInstance( tz.getTimeZone( "GMT" )) />
07: <cfset variables.gCal.set( 1601, 0, 1, 0, 0, 0 ) />

Note that the second argument of the calendar object’s set() function is 0. Java’s calendar object’s month index begins with 0 for January instead of 1. This is something to be aware of whenever dealing with Java calendar objects.

This next step requires a little explanation. variables.expTime is now in milliseconds thanks to line 3. I’m going to add the value of the calendar, in milliseconds, to variables.expTime. Because the calendar is set to a date before Java’s epoch it will actually be a negative number. The addition operation is actually a subtraction operation. The resulting value of variables.expTime will be the number of milliseconds since Java’s epoch.

08: <cfset variables.expTime = variables.expTime.add( variables.bigInt.valueOf( "#variables.gCal.getTimeInMillis()#" )) />

All we need to do now is reset the calendar to the value we now have in variables.expTime, then convert it to our local timezone and we’re done!

09: <cfset variables.gCal.setTimeInMillis( variables.expTime.longValue() ) />
10: <cfset variables.gCal.setTimeZone( tz.getDefault() ) />

Last, but not least, we convert the calendar object into a ColdFusion date/time object with ColdFusion’s CreateDateTime() function.

11: <cfset variables.expDateTime = CreateDateTime(
variables.gCal.get( variables.gCal.YEAR ),
variables.gCal.get( variables.gCal.MONTH ) + 1 ,
variables.gCal.get( variables.gCal.DAY_OF_MONTH ),
variables.gCal.get( variables.gCal.HOUR_OF_DAY ),
variables.gCal.get( variables.gCal.MINUTE ),
variables.gCal.get( variables.gCal.SECOND )
) />

Note that I’m adding 1 to the month variable because in Java 0 = January, but in ColdFusion 1 = January.

And there you have it. Converting from Microsoft’s 01/01/1601 100 nanosecond based timestamps to something a bit more usable in ColdFusion.

Of course you’ll need a reverse of this process as well, and I’ve already got that for you:

01:  <cfset variables.tz = CreateObject( "java", "java.util.TimeZone" ) />
02:  <cfset variables.Calendar = CreateObject( "java", "java.util.Calendar" ) />
03:  <cfset variables.gCal = variables.Calendar.getInstance( tz.getDefault() ) />
04:  <cfset variables.gCal.set(
Year( arguments.Date ),
Month( arguments.Date ) - 1,
Day( arguments.Date ),
Hour( arguments.Date ),
Minute( arguments.Date ),
Second( arguments.Date )
) />
05: <cfset variables.bigInt = CreateObject( "java", "java.math.BigInteger" ) />
06: <cfset variables.expTime = variables.bigInt.init( JavaCast( "String", "#variables.gCal.getTimeInMillis()#" )) />
07: <cfset variables.expTime = variables.bigInt.divide( variables.bigInt.valueOf( "1000" )) />
08: <cfset variables.expTime = variables.expTime.add( variables.bigInt.valueOf( "11644473600" )) />
09: <cfset variables.expTime = variables.expTime.multiply( variables.bigInt.valueOf( "10000000" )) />
10: <cfreturn variables.expTime.longValue() />

You should be able to follow along based on my explanation earlier in the article. However there are a couple extra bits I’d like to point out.

The calendar is initialized to the local timezone. We don’t have to worry about converting back to UTC/GMT because the calendar function getTimeInMillis() will give you the number of milliseconds from Java’s epoch, which is in UTC.

On line 8 I’m adding the number of seconds from Microsoft’s epoch to Java’s epoch. You’ll also notice the line before that I’m dividing by 1000 to convert time from milliseconds to seconds and after I add the epoch difference I’m converting to 100 nanoseconds by multiplying by 10000000. So why not skip the conversion to seconds and simply add three more zeroes to the end of the epoch difference (making the time in milliseconds) and multiplying by 10000?

The reason is that there’s a problem with precision and the resulting value will vary by a few hundred milliseconds. Since the times I’m working with are in seconds, those extra milliseconds could be ignored. But by dividing, adding, and then multiplying like this I don’t get the millisecond variances. I’m sure there’s better stuff on the ‘net to explain precision with BigInteger objects if you’re inclined to investigate further.

Before you go I have one piece of information I learned while working on this particular topic.

ColdFusion treats all date and time values as if they are in the local timezone. Specifically, if your timezone has daylight savings then all dates are treated as if they have daylight savings, including GMT/UTC times which do NOT have daylight savings time. DateConvert() will not protect you from this issue. There’s a deeper explanation here.

My solution is to check if GetTimeZoneInfo().isDSTon is TRUE. If it is then I need to add (or subtract) 1 hour from my UTC time. Note that this is only needed when converting between UTC and local timezones with ColdFusion date and time functions. The above code specifically stays away from ColdFusion date and time functions so this isn’t a problem with my example. But it is something to keep in the back of your head when you start to notice dates are being calculated an hour off the time they should be.

I have since found a subtle flaw with this implementation.

It’s not in the code, but in the  JVM. ColdFusion 8 ships with an older JVM. I recently upgraded our JVM to a more current version in an attempt to resolve a timezone bug. In doing so my SOAP application stopped working.

A little research led me to this article about a TLS bug in Java that could lead to a man-in-the-middle exploit. It appears the way I’m performing my SOAP operation triggers a TLS/SSL renegotiation when it receives a response from the external server.

The short answer is to add the following line to ColdFusion’s JVM arguments:

-Dsun.security.ssl.allowUnsafeRenegotiation=true

This does resolve the problem, but it apparently leaves the JVM vulnerable to MITM attacks. There is another bit of code in that article which shows how to change the allowUnsafeRenegotiation flag on-the-fly. I added this to my ColdFusion code, but changing the flag didn’t appear to have any effect.

If anyone else has played around with this particular problem I’d love to hear about it.

For now I”ve left the JVM in its vulnerable state as we only make HTTP requests from the JVM for a couple of applications and neither of them carry personally identifiable information.