Skip to main content Skip to Footer

BLOG


February 23, 2017
Immutable infrastructure with Chef, Packer, Consul and Jenkins
By: Laura Woo

Treating your infrastructure as code can be seen as a foundation for DevOps, applying core principles such as automated testing, version control and continuous integration. Servers evolve over time and configuration drift leads to the creation of servers each as unique as snowflakes, introducing fragility and unpredictability to environments. The use of configuration management tools (e.g., Chef, Ansible and Puppet) help to avoid “snowflake servers” by automating the specification of how servers should be configured. However, repeatedly applying configuration to bring a server up to date is not enough to completely prevent configuration drift, which is where immutable images come into play.

Having an immutable infrastructure in place enables you to tear down existing servers and replace them with new ones, along with an updated configuration—all without the need to ever modify existing instances. The aim is to take infrastructure as code to the next level, striving towards:

  • Higher scalability and availability

  • More consistent deployments across environments

  • Decreased operations intervention

Below is an approach taken to achieving an immutable infrastructure, and an overview of the software used to automate immutable image builds:

Tool

Function

Chef

Chef is an automation platform that is used for infrastructure configuration management and deployment across cloud, on-premises or hybrid environments.

Packer

Packer is a tool that provides the capability to build container and machine images for a variety of platforms from a single source configuration. It supports configuration management tools such as Chef, Puppet, Ansible, Salt, etc., and is able to build images for multiple platforms such as AWS, Azure, Google Cloud, Docker, VirtualBox, etc.

Jenkins

Jenkins is an open source CI (Continuous Integration)/CD (Continuous Delivery) server. It can be used for the automation of image builds and provisioning by having CI/CD jobs that pull from repositories.

Consul

Consul is a high availability service discovery and configuration tool. Services are able to register and discover other services via Domain Name System (DNS) or Hypertext Transfer Protocol (HTTP) interface. Dynamic configuration can be stored in the key/value store.


What is an immutable image?

Click to Tweet. This opens in a new window.

An immutable image has all the software components required baked into the image. When a virtual instance is launched from an image, no packages are downloaded and no software installed. It results in an instance that is ready on boot.

The approach to configuring an immutable image will depend on your requirements. You may choose to include all the configuration in your image or use an external configuration service. Our approach was to decouple static and dynamic configuration and refactor existing Chef recipes to cater to the distinction. The reasoning behind using Consul for dynamic configuration was driven by the need for a higher availability, more resilient and scalable tool than the Chef Master.

Configuration Type Tool Function
Static Chef
  • Installation of software

  • Application configuration

  • Hardening of the image (the process to reduce vulnerabilities that systems are exposed to)

Dynamic Consul and Consul templates
  • Environment specific configuration

  • Networking variables


Why the immutable approach

Click to Tweet. This opens in a new window.

Immutable infrastructure is an approach to infrastructure deployment and configuration, where infrastructure artefacts (the output of image builds) are deployed, unchanged, to different environments. Infrastructure reliability is increased, as the approach allows for identical server images to be deployed across environments with the use of automated pipelines. Development-production parity is achieved, as immutable images make it easier to mirror production in lower environments. This, in turn, improves testability and ought to result in errors being found at an earlier stage.

The immutable approach prevents configuration drift; for example, instead of rolling out a security patch and potentially missing a few instances, the patching can be baked into a new image and old instances replaced with patched versions. Rebuilding instances from a base image enable an instance’s configuration to be set to a known state. This decreases operational complexity when outages occur or roll backs are required.

Implementation of automated image builds

Click to Tweet. This opens in a new window.

Building images and provisioning them onto platforms of your choice should be carried out by an automation tool such as Jenkins. Static images are created using Chef and Packer templates that define how the images are built. A Jenkins job will then poll the repository where the Packer templates are stored and build the image with Packer. How you decide to run Packer from Jenkins will be dependent on your set up. Some might be interested in using the Packer plugin for Jenkins, which allows you to choose the installation binary type and specify a system-wide packer template to be used on a slave node.

The image artefacts from the CI job will be subjected to automated integration testing before being versioned and stored in a repository. A separate provisioning job will enable users to select the image version they wish to provision, with Packer and the data center to deploy the instance. Dynamic configuration can then be applied at run time, once the instance has been provisioned in the target environment.

Popular Tags

    Archive