Building Machine Images with Packer

Earlier this year we wrote about adopting Vagrant and Terraform in our steady march toward Infrastructure as Code. We recently added a new tool to this list, HashiCorp’s Packer. Packer automates building machine images, and with a single set of provisioners, creates images for multiple builders (such as VirtualBox, DigitalOcean, and Google Cloud).

The Packer use case

Let’s say you're planning a project that requires a local development environment, a staging environment, and a production environment. You have a distributed team using different operating systems so your development environment must be portable and virtualized. Due to external constraints, the staging environment must be hosted on Cloud Provider A and the production environment must be deployed to Cloud Provider B. You're expecting a lot of traffic and need a load balancer in front of several servers. You've done this before and have a very specific stack in mind.

Enter Packer.

Packer allows you to build machine images for multiple platforms simultaneously. You define a set of builders, which are the platforms you want to export images to, and a set of provisioners, which can range from shell to Ansible, Chef, and Puppet. Packer creates machines on each platform, runs provisioners on each such machine, shuts down the machines, exports platform-specific images, and destroys the machines. What's left is a set of platform-specific images that can be used to create multiple instances on each platform.

This means that we can use Packer to define our local, staging, and production platforms; provision identical images using tools like Ansible, Chef, or just plain bash scripts; and then deploy the images -- either manually or with tools like Vagrant and Terraform.

A basic example

We're going to build three identical machine images:

• The first will be for the local development server. This needs to be centralized but portable, so we're going to build a Vagrant box using VirtualBox.
• The second will be for the DigitalOcean staging server.
• The third will be for the Google Cloud production server.

The plan is to use Packer to automate creating and configuring these images. Via Packer, we'll also call Ansible and shell scripts to provision our images.

You can follow along with this example by cloning https://github.com/mugoweb/packer_configs. Just remember that you’ll need to make sure you have Packer, Vagrant, VirtualBox, and Ansible installed on your computer. In addition, you'll need to replace gcloud.json and do.json with your own Google Cloud credentials and DigitalOcean API token: { "do_token": "<< YOUR TOKEN >>" }.

.
├── README.md
├── ansible
│ ├── main.yml
│ └── requirements.yml
├── http
│ └── preseed.cfg
├── iso
│ └── ubuntu-18.04.1-server-amd64.iso
├── scripts
│ ├── ansible.sh
│ ├── cleanup.sh
│ ├── init.sh
│ └── init_vagrant.sh
├── secrets
│ ├── do.json
│ └── gcloud.json
├── ubuntu18041.json
└── vagrant
  └── Vagrantfile

Almost everything in the above layout is optional except the main Packer configuration file, (named ubuntu18041.json in the example), which defines the builders and provisioners. Unless instructed otherwise, Packer will run all provisioners against all builders.

The ansible folder houses our Ansible playbook and a requirements file that can include arbitrary roles from Ansible Galaxy.

The http folder contains a file named preseed.cfg, which contains a set of directives for the Ubuntu installer (because the VirtualBox image is built using a Ubuntu 18.04.1 ISO, the installer is actually run against the virtual machine).

The iso folder contains a local copy of the Ubuntu 18.04.1 ISO that is used to build the VirtualBox image.

The scripts folder contains all of the shell provisioning scripts (such as init_vagrant.sh which is run against VirtualBox to configure and export a Vagrant box).

The secrets folder contains credentials for builders that require them (for instance, DigitalOcean and Google Cloud). This allows you to version control your Packer configuration while omitting secret credentials.

Finally, the vagrant folder stores the exported Vagrant box and a minimal Vagrantfile to test it with.

To validate your Packer configuration, run

packer validate -var-file=secrets/do.json ubuntu18041.json

To build your images, run

packer build -var-file=secrets/do.json ubuntu18041.json

To build images only for a specific builder -- for instance, VirtualBox -- run

packer build -only virtualbox-iso -var-file=secrets/do.json ubuntu18041.json

Once built, to add an image to Vagrant, run

vagrant box add packer-ubuntu-18041-amd64 vagrant/packer-ubuntu-18041-amd64.box

And then from within the vagrant folder, run

vagrant up

The next steps

Once your machine images have been generated, you’re ready to deploy them to their respective platforms. As we’ve discussed in the past, this step can also be automated, which saves time, guarantees consistency, and reduces the risk of accidental misconfiguration.

Are you considering migrating your infrastructure to code? Contact Mugo for a free consultation!