Configuration Management Terraform Infrastructure as Code
October 3, 2022
Throughout almost all of Sysadmin and Infrastructure Engineer history, golden images saved a ton of time. Whether for a Windows desktop, a Windows server, or a Linux box, having a golden image ready to be deployed in time of need was an excellent measure to take in terms of putting in the time to have said golden image.
It's no different in the DevOps and Platform Engineer realm.
That's why AMIs exist in the first place.
In this blog post, you'll walk through from a hands-on and theoretical perspective what Packer is, why use CloudTruth and Packer, and how to get it all up and running.
Packer is a tool for creating golden images.
A golden image could be anything from:
and any other type of AMI.
If you're unfamiliar with the term "golden image," essentially think about it as a clone of your server or workstation. For example, you need an exact copy of an EC2 instance or an Azure Virtual Machine. You can take a copy of it and save it from being used later on if you need to deploy the same exact EC2 instance or Azure Virtual Machine with the same settings, applications installed, dependencies, etc.
At this point, you may be thinking, why would I use Packer if I already have tools like Terraform and Ansible? and that's a very good question. Let's break down two common scenarios.
The first is more high-level from an infrastructure perspective. Let's say you have an application running on EC2 instances in AWS. The application will need to scale at some point. For that infrastructure to scale, it'll need one or more EC2 instances. If, in this example, over 5,000 users all of a sudden reach out to your application and the infrastructure needs to scale fast, otherwise it'll go down, Terraform will need to kick into action to create the infrastructure, and then Ansible will have to install/configure the servers. The problem is that this process (although automated) can be slow, and infrastructure may not be provisioned correctly. Every engineer that has used Ansible has run into the scenario that Ansible lost connection, or couldn't connect, to the specified server, resulting in some of the configuration not occurring.
Let's dive into a more specific scenario - a Kubernetes environment on-prem. Say you have three worker nodes running in your Kubernetes environment. All of a sudden, several thousand users reach out to your app. The Pods need to scale horizontally, but the three worker nodes are already at 85% capacity from a CPU and memory perspective. Because of that, another worker node needs to come online. At this point, you can't wait for Terraform and Ansible to do the job of creating and configuring a new worker node. Instead, you need a worker node that's ready to be deployed right then and there. That's when having an AMI/golden image ready to go is helpful.
These scenarios are why golden image tools like Packer still exist.
You'll learn more about how to use Templates and CloudTruth in this blog post, but to keep things theoretical, let's just look at the Template below and see what's happening before deploying anything.
packer {
required_plugins {
amazon = {
version = ">= 0.0.2"
source = "github.com/hashicorp/amazon"
}
}
}
source "amazon-ebs" "ubuntu" {
ami_name = "learn-packer-linux-aws"
instance_type = "t2.micro"
region = "us-west-2"
source_ami_filter {
filters = {
name = "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*"
root-device-type = "ebs"
virtualization-type = "hvm"
}
most_recent = true
owners = ["099720109477"]
}
ssh_username = "ubuntu"
}
build {
name = "learn-packer"
sources = [
"source.amazon-ebs.ubuntu"
]
}
There is a lot of configuration data here:
That's a lot of data to have to update, remember, configure, and then finally store in source control, which you'll then have to manage.
With CloudTruth, you get a UI and a straightforward CLI to manage all parameters and configuration data. You don't have to worry about storing all of this data in environment variable files and pushing it up to source control, only to be managed later. Instead, utilize CloudTruth to store the configuration data and parameters for each environment (dev, staging, prod).
When you need to create fast pace infrastructure, including a Kubernetes environment, you need a way to quickly scale up on-demand. Terraform or another IaC/ConfigMgmt tool, will be slower than using a golden image/AMI. For example, if you have to create Terraform code to create the infrastructure and then run Ansible code to configure it/install all of the resources, an app may be down because the needed infrastructure wasn't yet present.
That's why engineers still use tools like Packer. To have a ready-to-go AMI/golden image that's needed to quickly get infrastructure back up and running.
In this scenario, let's learn how to create an AMI in AWS for an EC2 instance.
Now that you understand the theory behind Packer and utilizing CloudTruth with Packer, let's review how the two tools work together. In this blog post section, you can take a hands-on approach to creating an AMI in AWS with Packer and CloudTruth. If you don't have an AWS account, you can still follow along and see how it would be done in, for example, Azure or GCP. The overall workflow (other than the Packer code) wouldn't change regardless of what cloud you're using.
The Packer installation methods for each platform are here.
For example, the Homebrew (package manager for macOS), commands are :
brew tap hashicorp/tap
brew install hashicorp/tap/packer
Once Packer is installed, you need to create a Template. The Template will consist of:
First, create the main.pkr.hcl configuration file to create the AMI in AWS.
packer {
required_plugins {
amazon = {
version = ">= 0.0.2"
source = "github.com/hashicorp/amazon"
}
}
}
source "amazon-ebs" "ubuntu" {
ami_name = var.AMI_Name
instance_type = var.AMI_Instance_Type
region = var.region
source_ami_filter {
filters = {
name = "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*"
root-device-type = "ebs"
virtualization-type = "hvm"
}
most_recent = true
owners = ["099720109477"]
}
ssh_username = "ubuntu"
}
build {
name = "learn-packer"
sources = [
"source.amazon-ebs.ubuntu"
]
}
Next, create the variables.pkr.hcl file and store the following variables. Notice how there's a function called env() used. This is because the Parameters used from CloudTruth are stored as environment variables. Packer uses a special env() function to call upon environment variables.
variable "AMI_Name" {
type = string
default = env("AMI_Name")
}
variable "AMI_Instance_Type" {
type = string
default = env("AMI_Instance_Type")
}
variable "Region" {
type = string
default = env("Region")
}
Save both in a specific location and ensure that you're in the directory where you stored the configuration files when you run the packer commands, which are coming up in one of the next sections.
Now that you have your configuration and variables, you'll need to create the parameters in CloudTruth that match the variables.
As you can see in the screenshot below, there are three parameters that match up with the three variables in the Packer configuration.
The parameters can be utilized with the CloudTruth CLI, which you'll see coming up next.
Once the Packer configuration is complete and the parameters exist in CloudTruth, you can utilize the CloudTruth CLI and the packer CLI to run the configuration.
The CloudTruth CLI calls out to the parameters and stores them as a sort of "local variable" or "environment variable" when deploying the Packer configuration.
cloudtruth --project "Packer" --env default run -- packer init .
Next, validate that the Packer configuration is accurate.
cloudtruth --project "Packer" --env development run -- packer validate .
The configuration is valid.
The last step is to run the packer build command, which calls upon the Parameters in CloudTruth to supply to the Packer configuration.
cloudtruth --project "Packer" --env default run -- packer build .
==> Wait completed after 3 minutes 2 seconds
==> Builds finished. The artifacts of successful builds are:
--> learn-packer.amazon-ebs.ubuntu: AMIs were created:
us-east-1: ami-06e68bee45ee7a52d
You should see an output on the terminal of a successful build, and if you log into AWS, you'll see the AMI listed in the screenshot below.
Congrats! You have successfully used Packer and CloudTruth to build an AMI.
Here's a quick screencast of the process:
Our bite-sized newsletter with DevSecOps industry tips and security alerts to increase pipeline velocity and system security.
