Remote Pairing in the Cloud

I had my first experience of remote pairing in 2002. Remote pairing has all the usual conventions of pair programming (two people working simultaneously on a single task with frequent switches of ‘driver’ and ‘navigator’) except that, rather than the two developers being sat side-by-side at the desk working with one monitor, one keyboard and one mouse, they work in two physically separate locations, often from home.

Back in 2002 the technology limitations made a lot of our choices for us. We used:

  • Microsoft Net Meeting for desktop sharing
  • Each developer had the same development environment separately maintained on their local machine (Eclipse IDE, JVM, infrastructure products, etc.)
  • Speaker phone for voice
  • 512/1MBps ADSL internet connections

This worked but there were a number of ‘hassles’ ranging from slow swapping of driver and navigator (latency on Net Meeting was such that the driver had to be using their local machine with the navigator observing remotely and making the odd click if necessary), to dealing with firewall issues, to keeping two development environments perfectly in sync and ensuring that all tests pass on both machines. John Daniels and I presented on remote pairing as part of a session on dispersed development at OT2004 and I found that the slides from that session are still available online.

Recently I found myself remote pairing again and was surprised to find that many individual aspects of the set-up have improved without things really getting any better:

  • Broadband download speeds are way up but upload speeds (crucial in peer-to-peer desktop sharing) are unchanged
  • There are many more technologies for desktop sharing – including Skype, iChat (I now use Macs rather than Wintel), Adobe Acrobat  Connect, GoToMeeting, and so on – but they all have varying approaches to viewing vs. controlling with the ones that give the best viewing experience invariably not supporting taking control, meaning we still have to swap ‘host’ every time we want to swap driver.
  • Using Skype for voice is far better quality and, more importantly, far cheaper than using a landline but contributes to connection saturation
  • Developing Cloud-based solutions for removes some, but not all, of the local environment problems but the fact that each file save and each test run uses the Internet connection also contributes to connection saturation

Seven years on the whole experience was actually worse than in 2002. Not only were the bandwidth limitations of two fast, urban ASDL connections causing poor refresh rates on the shared desktop but the times to save a file, access a web page and run our tests were soaring: unacceptable for development.

What we really needed was an environment where we weren’t limited by upload speeds, where the Skype and desktop sharing connections didn’t contribute to the save, browse and test times in building the application and where we had one environment to set up and maintain. And the answer was actually pretty obvious: develop for the cloud in the cloud.

Currently our setup is:

  • A 64-bit CentOS installation running as an image in Amazon’s EC2. Into this we’ve installed the development environment, our testing tools, a browser and a freenx server – the only things we really need. This has been serialised onto Amazon’s S3 storage.
  • NoMachine NX clients on local machines to access the remote desktop (using ‘shadowing’ to create a shared session). This is crucial as NX is the first technology I’ve seen that makes a remote desktop session feel like it is running on the local machine. It’s not quite there but the differences are pretty subtle and for most of what you do when pairing – file editing and testing as opposed to graphic image manipulation or fast context switching between lots of windows – its better than adequate.
  • Skype connection for voice (your bog-standard peer-to-peer, non-cloud variety)

Yesterday was the first full day of development in this environment and I have to say it went better than expected (and certainly felt a few years on from my first experiences of remote pairing). We got the benefits we expected – desktop sharing latency is now determined by 4MBps+ download speed rather than 256KBps upload speed with Skype not having a noticeably detrimental effect – and a few we didn’t. The fact that the development environment is physically located in the same country as the data centre means our saving/browsing/testing times are lower than they are when working locally in the UK. The nature of development means we don’t need to worry about re-serialising the EC2 image at the end of every day, we simply place any assets under version control and then terminate the image meaning we start with a fresh, clean environment every morning. And the nature of EC2 means that, when we need to get another pair working, we simply run up two instances of our dev image rather than one!

The next step is to provision a Windows environment under EC2 or one of the dedicated Windows cloud providers so we can run multi-browser testing; once we have that we’ll have a complete, cloud based version of our local development environments for remote pairing. The step after that is probably to ditch the local environments altogether.


4 thoughts on “Remote Pairing in the Cloud

    1. pauldyson Post author

      Instructions direct from my wiki. Assumes CentOS but otherwise just substitute yum for whatever package manager your variant of Linux supports. Derived from although I left SSH password authentication enabled (otherwise you can only connect with NX).

      Launch the RightScale x86_b4 CentOS 5.2 image (ami-155cba7c) using the EC2 keypair
      Connect using ssh
      Install X Windows and GNOME: 

      yum groupinstall 'X Window System' 'GNOME Desktop Environment'

      Install nx and freenx:

      yum install nx freenx

      In /etc/nxserver

      cp node.conf.sample node.conf
      vi node.conf
      service sshd reload
      nxserver --adduser <USERNAME>
      nxserver --password <USERNAME>

      Copy the contents of /etc/nxserver/client.id_dsa.key to the NX Client

  1. Willem van den Ende

    ubuntu / debian install is slightly more involved (freenx is not in default repositories it seems), but doable.

    It seemed at first ssh password authentication was obligatory, I could not find a working tunnelling solution, and that would be a dealbreaker for me. After having the experience of being hacked it is ssh key authentication only for me…

    In the documentation I read so far I missed the enable_passdb_authentication option. Turns out the NoMachine version has something similar, but different: options EnableUserDB = “1” and EnablePasswordDB=”1″ give the same result (and adding, as in your solution my user to nxserver, only it’s called –adduser instead of –useradd to make things less confusing 😉 ). Package installation for debian seemed to be slightly less involved with the NoMachine version, I may try freenx later.

    It seems quite a bit faster. Thanks for the tips!

  2. Piran

    The final piece of the puzzle, at least for those on Macs, is Caffeine: . A quick click on this will at least stop your screen from sleeping while your pair is driving.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s