How do we do Continuous Integration to test our frontend on Internet Explorer + Firefox + Chrome + Safari

tl;dr

Create an EC2 instance. Write a script. Enjoy!

The early ages

We are used to do Javascript testing for our frontend for ages using a headless Chrome on a Linux box and JsTestDriver.

Basically we just need to run:

/usr/bin/Xvfb :42 -ac &
DISPLAY=:42 java -jar JsTestDriver.jar --tests all --port 1234 --browser /usr/bin/google-chrome --reset

It fires up Chrome running the test suite using JsTestDriver’s server listening on localhost:1234.

This setup works pretty well but the major drawback is we can’t run our test suite on Internet Explorer as we’re not insane enough to develop under Windows. What’s more, testing is not automated.

Set up a Amazon instance with Windows

Create the instance

Let’s start by launching a new EC2 instance. We’ll choose Windows 2008 as the OS and select a Security Group (which is somehow a firewall policy in AWS’s terminology) that will let us connect through SSH and RDP. SSH will be used to do all the magic whereas RDP will be used to install and configure all the browsers (and possibly help us to debug).

Configure the instance

At this point, only an Administrator account is created, I made the mistake to create a regular user at first. Indeed, we’re going to install Cygwin that will provide us OpenSSH server and a decent shell. It turns out that Cygwin is not happy if it doesn’t have administrator privileges. So let’s stick to the Administrator account (nobody cares anyway, it’s Windows after all ;)).

Install whatever browsers you want to run tests on

Probably the most boring part, download and install them. Be sure to launch every browsers at least once to untick “Always check to see if #{browser} is the default browser on startup” and others “first-run” popups.

Tips

The hook

Before being able to run the tests automatically whenever we push new code, we need a way to launch the tests on the Amazon instance. This is the role of a little Batch script - you know, this wonderful programing language with plenty of evil goto.

The magic script

All the magic happens in those ~200 LOC. Basically, here’s what it does:

Prepare the stuff

First of all, we create a Ruby object representing our AWS instance. In order to achieve that, we’ll use the official RubyGem provided by Amazon: aws-sdk. This will save us a lot of time because we won’t have to reimplement all those API calls we need since Amazon has already done it for us - DRY.

First thing is to start the instance unless it’s already running. We also requested an Elastic IP (i.e. a static IP address in AWS’s wording) to allow us to connect to this instance more easily. So next thing is to associate this elastic IP to the instance. Finally, we just need to wait for Windows’ account to be loaded and OpenSSH server to be listening for incoming connections using the network Swiss Army knife aka netcat.

Upload the hook

rsync provides us a convenient -c flag which does, quoting rsync’s man page:

before-the-transfer “Does this file need to be updated?” check

This allows us to know whether or not we need to restart this hook. Why don’t restarting it every times? Because I didn’t manage to find a not-too-dirty way to do it quickly. Indeed, the current solution simply reboots the instance! The new hook will be run automatically on reboot as it’s located in Administrator’s Startup folder.

If anyone knows a way to do it in a more cleaner way, i.e. to launch a Batch script as a Windows process through Cygwin, I’d be really interested ;).

Run the tests!

First thing here is to do a mirror copy of the code used to run tests under the headless Chrome under Linux. Why? Because most of our code is in CoffeScript including tests. We use Sass and Mustache as well and obviously, all these pretties require some RubyGems to get compiled. So this will save us from installing a Ruby environment on the Windows box.

Once again, we’ll use the very good rsync command with -a (as archive) and --delete to delete old files.

We’re now ready to send to green flag to our remote control by simply creating a file named ‘go’ containing the path to the project we want to build.

At this point, the hook will run tests through all browsers outputting logs and exit codes into files.

Once all tests are finished, we get back logs, do a sum of all exit codes then send it back to our CI solution: CruiseControl.rb. Oh and there’s also a little check to be sure JsTestDriver didn’t timeout or fail for any reason.

The tweaking bits

Cool, it works but there’s still something wrong: there’s no point at keeping this Windows running when we don’t need it. This is somehow the second purpose of this script.

A Cron job is executed every ten minutes calling this script with the --shutdown-instance-if-needed mode.

The logic here is quite simple: if it’s working time, we shut it down unless the last build was less than one hour ago (to avoid repetitive reboots); otherwise, we stop it as soon as the Cron task is executed.

Fork it!

You may want to have a look there: https://gist.github.com/1422833. Yeah it’s slightly filthy in some places but it’s a first version that does the trick for now.

What could be improved:

Conclusion

Now you know we work hard to make our game working perfectly fine on plenty of browsers, it’s time to give it a try: picklive.com. But it’s definitively not an excuse to still be using Internet Explorer because you won’t get the full amazing Picklive’s experience anyway with IE!

Posted on 02 December 2011 by C├ędric @infertux.
blog comments powered by Disqus