I’ve been trying for a while to set up a workable development environment in the cloud, for several reasons:

  • Renting the computer capacity I need, for just as long as I need it, seems less environmentally damaging and wasteful than buying a new, expensive laptop every couple of years.

  • A VM in the cloud has much better internet bandwidth than most of the locations I work from, so running npm install, bundle install, or pulling docker images is less frustrating.

  • Maintaining a development VM will encourage me to keep everything about my development setup defined, and therefore documented, as code.

  • It removes the possibility of messing up my development environment by installing incompatible versions of libraries or other software, since I’ll only install whatever I need for the particular project I’m working on.

I’ve been toying with these ideas for a while, and made a few attempts to switch to this way of working, but it’s only since I’ve been stuck using rather crappy mobile internet for the last few months (long story), that I’ve really made a serious attempt.

This article describes my current setup, which has been working quite well for me.

This setup works if you use a unix-like environment (I’m using a Mac), and a terminal-mode editor (I use vim). If you mainly use a GUI-based editor like Atom or VS Code, this approach might not work for you.

Objectives

This is what I want from a cloud-based development environment:

  • Zero manual setup

I want to run a single command to (re)create my development machine. No manual steps required – just fire off a command and come back when it’s done and start work. So, any software installation and configuration should already be done.

  • As little sensitive data as possible

As far as possible, I want sensitive data such as SSH keys or API credentials only on my laptop, for security reasons.

  • Ephemeral infrastructure

I want to create the VM when I need it, and destroy it completely when I’m done. There should be no cloud resources that persist between sessions. This is partly for cost – I only want to pay for compute/storage for the time I’m actually using it – but mainly for security. I don’t want a machine living in the cloud with any of my credentials or data on it (e.g. API tokens, or code from private repos).

  • All of my customisations

I’ve developed/acquired a lot of shell scripts, and customised my editor and shell to my own preferences over the years. I want all of that stuff in place on my development machine before I start working.

Pre-requisites

For this setup, you will need:

  • A DigitalOcean hosting account, and an API token with read & write permissions.

DigitalOcean is my go-to for quick, simple and cheap hosting. Personally, I wouldn’t run any production infrastructure on it, but it’s great for quickly throwing together experiments and prototypes.

  • Terraform

I’m using version 0.14.5

  • Ansible

I’m using version 2.10.5

  • An SSH key defined on your GitHub account

  • An SSH key to restrict access to the development VM

This can be the same SSH key as your GitHub account, but I’d recommend having separate keys.

  • ssh-agent (or equivalent) running locally

This lets you use your GitHub SSH key to manage your repositories without having to copy the private key to the development VM. I’m using a Mac laptop, so OSX handles this for me, and I just have to run the following whenever I reboot:

ssh-add ~/.ssh/my-github-private-ssh-key

Creating/destroying the VM with Terraform

I use terraform to build and destroy the VM. To do this, terraform needs our DigitalOcean API token, which we can supply as an environment variable:

export TF_VAR_do_token=your api token

Now we can define our VM in terraform. I’m putting everything in a single main.tf file:

First some header information to specify the versions of terraform and the DigitalOcean provider:

terraform {
  required_version = ">= 0.14.5, < 0.15"

  required_providers {
    digitalocean = {
      source  = "digitalocean/digitalocean"
      version = "~> 2.4"
    }
  }
}

We initialize the DigitalOcean provider using the API token from our TF_VAR_do_token environment variable like this:

variable "do_token" {}

provider "digitalocean" {
  token = var.do_token
}

We have to define a public SSH key on DigitalOcean containing the public part of the SSH keypair we’ll use to SSH onto our VM:

variable "private_key_file" {
  default = "~/.ssh/digitalocean_buildvm"
}

resource "digitalocean_ssh_key" "buildvm" {
  name       = "Build VM"
  public_key = file("${var.private_key_file}.pub")
}

I’ve defined the variable as private_key_file because we’ll use the private key when we run ansible later to configure our VM, but the private key never leaves our local machine.

Now we’re ready to create our VM (which DigitalOcean calls a "droplet"):

resource "digitalocean_droplet" "buildvm" {
  image      = "ubuntu-20-04-x64"
  name       = "buildvm"
  region     = "sgp1"
  size       = "s-4vcpu-8gb" # $40/month, $0.06/hour
  monitoring = true
  ssh_keys   = [digitalocean_ssh_key.buildvm.fingerprint]
}

The ssh_keys parameter tells DigitalOcean to configure the VM so that we can use the private key to SSH onto it as root.

I’m creating my VM in the Singapore region (sgp1) with 4 virtual CPUs and 8GB of RAM. You can get the available options for region, droplet size, and base image via the DigitalOcean API.

At this point, you can build the VM by running:

terraform init
terraform apply

Connecting via SSH

We’ll use Ansible to configure our base Ubuntu 20.04 VM as a development environment by installing and configuring all the software we need.

To do this, Ansible will need to be able to access the new VM via SSH. The easiest way to do this is via its IP number, which we can get from Terraform like this:

output "droplet_ip" {
  value = digitalocean_droplet.buildvm.ipv4_address
}

With this in our main.tf file, we can get the IP number like this:

$ terraform output
droplet_ip = "188.166.217.252"

We need that IP number quite often, and running terraform output is a little slow, so I prefer to cache the IP number in a local file called .ip using terraform’s "local_file" resource:

resource "local_file" "ip-address" {
  content  = digitalocean_droplet.buildvm.ipv4_address
  filename = "${path.module}/.ip"
}

With the IP number and our private SSH key, we can connect to the VM as root like this:

ssh -i /.ssh/digitalocean_buildvm root@$(cat .ip)

Every time we recreate the VM, it will have a new identity as far as SSH is concerned, so you’ll get warnings about it being a new host, and errors about ssh host fingerprints. We can suppress these errors like this:

ssh -i /.ssh/digitalocean_buildvm -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@$(cat .ip)

We will also want the -A flag to tell SSH to use "agent forwarding" so that any SSH keys we have in our local ssh-agent are available if we run ssh from the development VM.

Makefile

I use make to simplify running (and remembering) these commands. Create a file called makefile containing:

SSH_KEY := ~/.ssh/digitalocean_buildvm
SSH_OPTIONS := -A -i $(SSH_KEY) -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no

buildvm:
  . .env; terraform init; terraform apply -auto-approve

destroy:
  . .env; terraform destroy

ssh:
	ssh $(SSH_OPTIONS) root@$$(cat .ip)

You must use tabs, not spaces, to indent lines in your makefile. Also, be careful with the double $ symbol in the ssh: section. Make interprets a single $ as referring to a local make variable (like $(SSH_KEY)), so you need $$ if you want the output of a shell command, as in $$(cat .ip)

Now, we can build our VM using make buildvm, SSH onto it with make ssh, and destroy it with make destroy

In the next post in this series, I’ll describe how I use Ansible to install and configure my development environment.

Don’t forget to destroy your VM when you’re not using it, so that you don’t incur unnecessary charges

One thought on “Developing in the cloud with terraform and ansible

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s