The Picklive Tech Blog

How do we do Continuous Integration to test our frontend on Internet Explorer + Firefox + Chrome + Safari

tl;dr

Create an EC2 instance. Write a script. Enjoy!

The early ages

We are used to do Javascript testing for our frontend for ages using a headless Chrome on a Linux box and JsTestDriver.

Basically we just need to run:

/usr/bin/Xvfb :42 -ac &
DISPLAY=:42 java -jar JsTestDriver.jar --tests all --port 1234 --browser /usr/bin/google-chrome --reset

It fires up Chrome running the test suite using JsTestDriver’s server listening on localhost:1234.

This setup works pretty well but the major drawback is we can’t run our test suite on Internet Explorer as we’re not insane enough to develop under Windows. What’s more, testing is not automated.

Set up a Amazon instance with Windows

Create the instance

Let’s start by launching a new EC2 instance. We’ll choose Windows 2008 as the OS and select a Security Group (which is somehow a firewall policy in AWS’s terminology) that will let us connect through SSH and RDP. SSH will be used to do all the magic whereas RDP will be used to install and configure all the browsers (and possibly help us to debug).

Configure the instance

At this point, only an Administrator account is created, I made the mistake to create a regular user at first. Indeed, we’re going to install Cygwin that will provide us OpenSSH server and a decent shell. It turns out that Cygwin is not happy if it doesn’t have administrator privileges. So let’s stick to the Administrator account (nobody cares anyway, it’s Windows after all ;)).

Install whatever browsers you want to run tests on

Probably the most boring part, download and install them. Be sure to launch every browsers at least once to untick “Always check to see if #{browser} is the default browser on startup” and others “first-run” popups.

Tips

The hook

Before being able to run the tests automatically whenever we push new code, we need a way to launch the tests on the Amazon instance. This is the role of a little Batch script - you know, this wonderful programing language with plenty of evil goto.

The magic script

All the magic happens in those ~200 LOC. Basically, here’s what it does:

Prepare the stuff

First of all, we create a Ruby object representing our AWS instance. In order to achieve that, we’ll use the official RubyGem provided by Amazon: aws-sdk. This will save us a lot of time because we won’t have to reimplement all those API calls we need since Amazon has already done it for us - DRY.

First thing is to start the instance unless it’s already running. We also requested an Elastic IP (i.e. a static IP address in AWS’s wording) to allow us to connect to this instance more easily. So next thing is to associate this elastic IP to the instance. Finally, we just need to wait for Windows’ account to be loaded and OpenSSH server to be listening for incoming connections using the network Swiss Army knife aka netcat.

Upload the hook

rsync provides us a convenient -c flag which does, quoting rsync’s man page:

before-the-transfer “Does this file need to be updated?” check

This allows us to know whether or not we need to restart this hook. Why don’t restarting it every times? Because I didn’t manage to find a not-too-dirty way to do it quickly. Indeed, the current solution simply reboots the instance! The new hook will be run automatically on reboot as it’s located in Administrator’s Startup folder.

If anyone knows a way to do it in a more cleaner way, i.e. to launch a Batch script as a Windows process through Cygwin, I’d be really interested ;).

Run the tests!

First thing here is to do a mirror copy of the code used to run tests under the headless Chrome under Linux. Why? Because most of our code is in CoffeScript including tests. We use Sass and Mustache as well and obviously, all these pretties require some RubyGems to get compiled. So this will save us from installing a Ruby environment on the Windows box.

Once again, we’ll use the very good rsync command with -a (as archive) and --delete to delete old files.

We’re now ready to send to green flag to our remote control by simply creating a file named ‘go’ containing the path to the project we want to build.

At this point, the hook will run tests through all browsers outputting logs and exit codes into files.

Once all tests are finished, we get back logs, do a sum of all exit codes then send it back to our CI solution: CruiseControl.rb. Oh and there’s also a little check to be sure JsTestDriver didn’t timeout or fail for any reason.

The tweaking bits

Cool, it works but there’s still something wrong: there’s no point at keeping this Windows running when we don’t need it. This is somehow the second purpose of this script.

A Cron job is executed every ten minutes calling this script with the --shutdown-instance-if-needed mode.

The logic here is quite simple: if it’s working time, we shut it down unless the last build was less than one hour ago (to avoid repetitive reboots); otherwise, we stop it as soon as the Cron task is executed.

Fork it!

You may want to have a look there: https://gist.github.com/1422833. Yeah it’s slightly filthy in some places but it’s a first version that does the trick for now.

What could be improved:

Conclusion

Now you know we work hard to make our game working perfectly fine on plenty of browsers, it’s time to give it a try: picklive.com. But it’s definitively not an excuse to still be using Internet Explorer because you won’t get the full amazing Picklive’s experience anyway with IE!

Posted on 02 December 2011 by Cédric @infertux.

Long delays on NameError in Rails views

After upgrading to Rails 3.1, whenever we made a typo in a method name in the view, we experienced a long delay before the error was displayed, or the request eventually timed out. If we killed the server, we had a cryptic error message:

Mysql2::Error: This connection is still waiting for a result, try again once you have the result: SHOW TABLES

After some ruby-prof-ing I found out that most processing takes place under NameError#message in ActionDispatch::Routing::RouteSet.

This is what happened:

To add insult to injury, the result of inspect was shortened to display:

"undefined local variable or method idontexist' for #<#<Class:0x1078ad628>:0x10789d368>

If you include <%= self.inspect %> in your view, you get the same result.

If you override ActionDispatch::Routing::RouteSet#inspect with a simple method, you don’t get any delay.

I have created a gem called short_inspect that makes sure that inspect will not be called recursively. Follow the link to see how to use it.

Please let us know in the comments if you have experienced this problem and if this gem fixed it or you have found a better solution.

Posted on 23 November 2011 by Levente @leente.

Making knife ssh work with chef 0.10

Different version, different behaviour

A while ago we updated Chef to the current release. It had some fixes we needed and no one likes to use old smelly software versions, right?

One thing we ran into is that the current version of knife ssh doesn’t read ssh configuration from ~/.ssh/config any more. Like a lot of people we use a non-standard SSH port and have dedicated SSH keys, so this was a huge pain. You can specify the ssh port with -p and the identity file with -i, but using those all the time makes the command line ugly.

Not very well documented configuration

A bunch of searching this morning led me to the solution: you can specify these options in knife.rb, it’s just not documented anywhere but a JIRA ticket. To help signal boost this bit of hidden knowledge, here’s the lines I added to my ~/.chef/knife.rb to make knife ssh tolerable again:

knife[:identity_file] = '/Users/john/.ssh/picklive_identity_file'
knife[:ssh_port] = 1234

(Details changed from actual values, of course.)

Posted on 15 November 2011 by John @semanticist.

Simple slide in content with pure CSS3

The problem

We had some great new characters made for our game screen. It seemed a lot more fun to bring them to life by having them pop into the page and then slide back out. We decided to do this with CSS only.

Our new character in the game

We tried a few different ways of making this work cleanly and degrade to a simple show and hide in old browsers such as IE6 and 7. A couple of problems we faced at first were divs sliding off the page causing scrollbars to appear in old browsers, and using CSS animations when in the end we found the transitions to be simpler and more effective for this task.

One of the keys to making this work was to avoid display: none to show and hide anything. Transitions don’t work for elements that are display: none at the start, so it’ll requires a lot more code and extra elements. In the end by setting the height at 0px it appears to be ‘hidden’ but is actually still on the page, the transform will then work on this value.

This is a just simple post and some code for you to play with as we found it to be really useful.

The code

The wrapper selector is the overall area in which our content will live in. Setting both the open and closed state at 500px wide means that it will slide in neatly from the bottom. If you set the width to anything lower than 500px on the initial wrapper it will ‘grow’ out from a point in both directions, which is also a nice effect if you prefer that.

You can also make the initial height 500px and width 0, this will cause the element to slide in from the left side.

This degrades nicely into IE6 and 7 as a simple show and hide, it also has a relatively small amount of code. Check out the demo to see it in action and grab the code.

Don’t forget you can also adjust the time of the transition to make it faster or slower, I decided to go with .5 seconds. You can also play around with the timing function to ease in and out, or have simple linear transition. There’s a fun little bezier creator you can use for this, and for a basic insight into CSS3 transitions A List Apart has a nice article.

.wrapper {
  display: block;
  position: absolute;
  width: 500px;
  height: 0;
  bottom: 0;
  left: 0;
  overflow: hidden;
  -webkit-transition: all 0.5s;
  -moz-transition: all 0.5s;
}

.wrapper.shown {
  width: 500px;
  height: 500px;
}

.wrapper.shown .content {
  -webkit-transform: translate(0px, 0px);
  -moz-transform: translate(0px, 0px);
}

.content {
  width: 480px;
  height: 480px;
  padding: 10px;
  position: absolute;
  bottom: 0;
  background: red;
  color: #FFF;
  -webkit-transform: translate(0px, 500px);
  -webkit-transition: all 0.5s;
  -moz-transform: translate(0px, 500px);
  -moz-transition: all 0.5s;
}

I’m sure there are many ways to reproduce this same task - so let us know if this is was helpful to you, or if you have any better techniques!

Posted on 09 September 2011 by Tim P & Tim R @timparker.

Chef on steroids

The problem

At Picklive, we love Ruby and therefore we love Chef. But Chef has two little glitches: he’s a bit too talkative and he doesn’t like SSL.

Since our main activity is gambling, we have to pass strict security audits and then displaying full backtraces of Chef errors to everyone on somewhere.picklive.com is not seen in a good light. Unfortunately, Chef logging level is not very configurable and it could be really verbose - even running in production mode. And hacking it is not an option since it would makes it hard to maintain up-to-date. So we have to filter error pages as HTTP 4xx and 5xx. It’s now that another great piece of software comes into play: Nginx.

Let’s proxy Chef through Nginx and return a custom error page instead of Chef backtraces. While we’re at it, we could put all Chef traffic over SSL to avoid sending potential sensitive information over the hazardous Internet.

Set up Nginx as proxy

Just let’s add this to our host configuration:

location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X_FORWARDED_PROTO $scheme;
    proxy_set_header Host $host;

    proxy_connect_timeout 4;
    proxy_read_timeout 30;
    proxy_send_timeout 30;

    proxy_redirect off;
    proxy_max_temp_file_size 0;

    access_log /var/log/nginx/somewhere.picklive.com-upstream_log upstream;
    proxy_pass http://localhost:4000;
}

Finally we say to chef to listen only on localhost with -h localhost and then restart the services.

Everything is broken!

Just try to upload a cookbook to test whether the proxy is working:

$ knife cookbook upload picklive -l debug
…
DEBUG: /var/folders/ti/tik1tgFzHlBl3tMqXZGAaE+++TQ/-Tmp-/chef-picklive-build20110825-1460-1wrs8m6-0/picklive/recipes/application.rb has not changed
DEBUG: Committing sandbox
DEBUG: Signing the request as xxx
DEBUG: Sending HTTP Request via PUT to somewhere.picklive.com:80/sandboxes/c864e4cc8178469b96c987f213acab27
/Users/xxx/.rvm/rubies/ree-xxx/lib/ruby/x.x/net/http.rb:xxxx:in `error!': 405 "Not Allowed" (Net::HTTPServerException)
…

Debugging

Obviously, we didn’t expect this weird 405 "Not Allowed" error. But if we look the line just above, we can spot what’s the problem. Yes, I’m talking about this somewhere.picklive.com:80 whereas we were expecting it in port 4000 - the port upon which Chef is listening. So let’s launch Wireshark and see what’s going on. It starts with some classic GET/POST requests to end up on a 201 Created followed by the PUT request above. The problem is that Chef sends us an URL to a new resource without the :4000:

"http://somewhere.picklive.com/sandboxes/c864e4cc8178469b96c987f213acab27"

So the problem is on the server-side.

Let’s sniff the traffic between Nginx and Chef with ngrep - a tcpdump-like but more suitable to analyse HTTP traffic.

$ sudo ngrep -W byline -d lo port 4000

T 127.0.0.1:59923 -> 127.0.0.1:4000 [A]
PUT /cookbooks/picklive/0.1.0 HTTP/1.0.
X-Real-IP: xxx.xxx.xxx.xxx.
X-Forwarded-For: xxx.xxx.xxx.xxx.
X_FORWARDED_PROTO: http.
Host: somewhere.picklive.com
Connection: close.
Accept: application/json.
…

T 127.0.0.1:4001 -> 127.0.0.1:59923 [AP]
HTTP/1.1 201 Created.
Location: http://somewhere.picklive.com/sandboxes/49e918e530a047f09b7b90c66ba0d729.
Content-Type: application/json; charset=utf-8.
Connection: close.
Server: thin x.x.x codename Flaming Astroboy.

As we can see, the :4000 is not removed by Nginx thus it’s Chef which doesn’t set it at the beginning of the chain. Why?! Well, just take a look at our Nginx configuration again before using some dirty hack like http://wiki.nginx.org/NginxHttpProxyModule#proxy_redirect.

Illumination!

Oh wait, maybe if we set the right port here:

proxy_set_header Host $host:4000;

Oh yeah! It was that, we are now able to use knife again.

The actual work

Now, filtering error messages and putting traffic over SSL is a piece of cake.

Filter error messages

Simply use the error_page directive offered by Nginx:

# Return a dummy page for all standard errors because chef-server is too verbose.
proxy_intercept_errors on;
error_page 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 495 496 497
           500 501 502 503 504 505 506 507 /50x.html;
location = /50x.html {
    root  /var/www/nginx-default;
}

SSL

Keep it simple, stupid.

ssl on;
ssl_certificate     /etc/ssl/<%= node[:chef][:server][:certificate][:file] %>;
ssl_certificate_key /etc/ssl/<%= node[:chef][:server][:certificate][:key] %>;

Voilà!

Posted on 02 September 2011 by Cedric @infertux.