Terraform for Dummies Part 2: Launch an Instance with a static website on AWS

Intro

This has become a habit so far to explore different ways of automated provisioning for each cloud provider. This time, I will try Terraform on AWS and reproduce the same deployment I have completed on Oracle Cloud, and as usual, we won’t just deploy an instance but also configure a website linked to its public IP. I’ll end this post with some notes on OCI/AWS differences.

Note: I have done a similar task with my bash scripts and AWS CLI which was very helpful to understand the logic behind each IAC interaction during the resource creation (More details >here).

Here’s a direct link to my GitHub repo linked to this lab =>: terraform-examples/terraform-provider-aws

Content :
I. Terraform setup
IV. Partial deployment
 V. Full deployment
Tips  & Conclusion

 

Overview and Concepts

Topology

The following illustration shows the layers involved between your workstation and Oracle Cloud Infrastructure while running the terraform commands along with the instance attributes we will be provisioning.

Besides describing my GitHub repo before starting this tutorial, I’ll just briefly discuss some principles.

  • Infrastructure As Code Manages and provisions cloud resources using a declarative code (i.e Terraform)  and definition files avoiding interactive configuration. Terraform is an immutable Orchestrator that creates and deletes all resources in the proper sequence. Each Cloud vendor has what we call a provider that Terraform uses in order to convert declarative texts into API calls reaching the Cloud infrastructure layer.

 

  • Terraform Files

– Can be a single file or split into multiple tf or tf.json files, any other file extension is ignored.
– Files are merged in alphabetical order but resource definition order doesn’t matter (subfolders are not read).
– Common configurations have 3 types of tf files and a statefile.

1- main.tf:  terraform declaration code (configuration) . The file name can be anything you choose

2- variables.tf: Resource variables needed for the deploy

3- outputs.tf:  displays the resources detail at the end of the deploy

4- terraform.tfstate: keeps track of the state of the stack(resources) after each terraform apply run

 

  • Terraform resource declaration syntax looks like this:
Component "Provider_Resource_type" "MyResource_Name" { Attribute1 = value .. 
                                                       Attribute2 = value ..}
  • Where do I find a good AWS deployment sample?

The easiest way is to create/locate an instance from the console and then use the import function from Terraform to generate each of the related components in HCL format (vpc,instance,subnet,etc..) based on their id.

Example for a VPC >>
1-  Create a shell resource declaration for the vpc ina  file called vpc.tf 
2-  Get the id of the vpc resource from your AWS Console
3-  Run the Terraform import then run Terraform show to extract the vpc’s full declaration from AWS in the same file (vpc.tf)
4- Now you can remove the id attribute with all non required attributes to create a vpc resource (Do that for each resource) 

1- # vi vpc.tf 

  provider "aws" {     region = "us-east-1"    }
  resource "aws_vpc" "terra_vpc" {
   }
2- # terraform import aws_vpc.terra_vpc vpc-0091141e28608813c
3- # terraform show -no-color > vpc.tf 

Note:
If you want to import all the existing resources in your account in bulk mode (not one by one) there is a tool called Terraformingwhich can import both code and state from your AWS account automatically.

Terraform lab content: I have deliberately split this lab into 2:

  • VPC Deployment: To grasp the basics of single resource deployment.
  • Instance Deployment: includes the instance provisioning (with above vpc) with a hosted web server.

 

I.Terraform setup

Since I’m on windows I  tried the lab using both Gitbash and WSL (Ubuntu) terminal clients (same applies for Mac).

Windows: Download and run the installer from their website (32-bit ,64-bit)

Linux: Download, unzip and move the binary to the local bin directory

$ wget https://releases.hashicorp.com/terraform/0.12.28/terraform_0.12.28_linux_amd64.zip
$ unzip terraform_0.12.18_linux_amd64.zip
$ mv terraform /usr/local/bin/
  • Once installed run the version command to validate your installation

    $ terraform --version
      Terraform v0.12.24

AWS authentication

To authenticate with your AWS account, Terraform will need to provide both  access_key_id & secret_access_key . This can be done either by sharing the authentication parameters with aws-cli or by Including the access_key and key_id within the Terraform config (i.e. variables.tf)

Assumptions

– I will assume that either of the two below authentication options is present/configured in your workstation:

  • AWS CLI default profile configured with your AWS credentials (Access keys). Refer to my blog post for more details
$ aws configure list
Name                    Value            Type    Location
----                    ---------        ----    --------
profile                 <not set>        None    None
access_key     ****************J2WA shared-credentials-file
region                us-east-1      config-file    ~/.aws/config
  • Or AWS credentials embedded in provider-specific config section on one of your terraform files (See vpc.tf)
provider "aws" {
# access_key = "${var.aws_access_key}" – uncomment & replace accordingly 
# secret_key = "${var.aws_secret_key}" – uncomment & replace accordingly
region = var.aws_region --- uncomment and replace accordingly}

– I’ll also assume the presence of an ssh key pair to attach to your ec2 instance. If not here is a command to generate a PEM-based key pair.  

$  ssh-keygen -P "" -t rsa -b 2048 -m pem -f ~/id_rsa_aws
     Generating public/private rsa key pair.
 Your identification has been saved in    /home/brokedba/id_rsa_aws.
 Your public key has been saved in        /home/brokedba/id_rsa_aws.pub.

 

II. Clone the repository

 

III. Provider setup

1. INSTALL AND SETUP THE AWS PROVIDER

  • Cd Into the subdirectory terraform-provider-aws/create-vpc where our configuration resides (i.e vpc)
GitBash $ cd /c/Users/brokedba/aws/terraform-examples/terraform-provider-aws/create-vpc

ubuntu $ cd /mnt/c/Users/brokedba/aws/terraform-examples/terraform-provider-aws/create-vpc 
  • AWS provider plugin is distributed by HashiCorp hence it will be automatically installed by terraform init.
$ terraform init
  Initializing the backend...

  Initializing provider plugins...
  - Checking for available provider plugins...
  - Downloading plugin for provider "aws" (hashicorp/aws) 3.11.0...
  * provider.aws: version = "~> 3.11"
$ terraform --version
  Terraform v0.12.24
  + provider.aws v3.11.0   ---> the provider is now installed
  
  • Let’s see what’s in the create-vpc directory. Here, only *.tf files matter (click to see content)
$ tree
  .
  |-- outputs.tf        ---> displays resources detail after the deploy
  |-- variables.tf      ---> Resource variables needed for the deploy   
  |-- vpc.tf            ---> Our vpc terraform declartion 

 

IV. Partial Deployment

DEPLOY A SIMPLE VPC

  • Once the authentication configured (access_key_id/secrete set) , we can run terraform plan the command to create an execution plan (quick dry run to check the desired state/actions).
    $ terraform plan
       Refreshing Terraform state in-memory prior to plan... 
      ------------------------------------------------------------------------
      An execution plan has been generated and is shown below.
        Terraform will perform the following actions:
    
        # aws_internet_gateway.terra_igw will be created
        + resource "aws_internet_gateway"  "terra_igw" 
        {..}
        # aws_route_table.terra_rt will be created
        + resource "aws_route_table" "terra_rt" {
        {..} 
        # aws_route_table_association.terra_rt_sub will be created
        + "aws_route_table_association" "terra_rt_sub" 
        {..}
        # aws_security_group.terra_sg will be created
        + resource "aws_security_group" "terra_sg" {
        + arn                    = (known after apply)
        + description            = "SSH ,HTTP, and HTTPS"
        + egress                 =[{...}]
        + id                     = (known after apply)
        + ingress                = [
            + {... rules for HTTP/SSH/HTTPS ingress access}
        # aws_subnet.terra_sub will be created
        + resource "aws_subnet" "terra_sub" {
        ...
        + cidr_block             = "192.168.10.0/24”
       ...}
        # aws_vpc.terra_vpc will be created
        + resource "aws_vpc" "terra_vpc" {
        ...
        + cidr_block             = "192.168.0.0/16”
        ...
        + tags                   = {
            + "Name" = "Terravpc"         }     }
      Plan: 6 to add, 0 to change, 0 to destroy.
    

    – The output being too verbose I deliberately kept only relevant attributes for the VPC resource plan

  • Next, we can finally run terraform deploy to apply the changes required to create our VPC (listed in the plan)
$ terraform apply -auto-approve
aws_vpc.terra_vpc: Creating...
...
Apply complete! Resources: 6 added, 0 changed, 0 destroyed.

Outputs:

Subnet_CIDR = 192.168.10.0/24
Subnet_Name = terrasub
internet_gateway_Name = terra-igw
map_public_ip_on_launch = true
route_table_Name = terra-rt
vpc_Name = Terravpc
vpc_id = vpc-09d491059eb740562
vpc_CIDR = 192.168.0.0/16
vpc_dedicated_security_group_Name = terra-sg
vpc_dedicated_security_ingress_rules = 
["Inbound HTTP access :  80 , CIDR: 0.0.0.0/0",
  "Inbound HTTPS access :  443 , CIDR: 0.0.0.0/0",
  "Inbound RDP access :  3389 , CIDR: 0.0.0.0/0",
  "Inbound SSH access:  22 , CIDR: 0.0.0.0/0",]

Observations:

– The deploy started by loading the resources variables in variables.tf which allowed the execution of vpc.tf
– Finally terraform fetched the attributes of the created resources listed in outputs.tf

Note: We’ll now destroy the vpc as the next instance deploy contains the same vpc specs.

$ terraform destroy -auto-approve

Destroy complete! Resources: 6 destroyed.

 

V. Full deployment (Instance)

1. OVERVIEW

$ tree ./terraform-provider-aws/launch-instance
.
|-- cloud-init           ---> SubFolder
|   `--> vm.cloud-config ---> script to config a webserver & add a HomePage
|-- compute.tf    ---> Instance related terraform configuration
|-- outputs.tf    ---> displays the resources detail at the end of the deploy
|-- variables.tf  ---> Resource variables needed for the deploy   
|-- vpc.tf        ---> same vpc terraform declaration deployed earlier

Note: As you can see we have 2 additional files and one Subfolder. compute.tf is where the compute instance and all its attributes are declared. All the other .tf files come from my vpc example with some additions for variables.tf and output.tf

  • Cloud-init: is a cloud instance initialization method that executes tasks upon instance startup by providing the user_data entry of the aws_instance resource definition (See below).
...variable "user_data" { default = "./cloud-init/vm.cloud-config"} 
    
 $ vi compute.tf
resource "aws_instance" "terra_inst" {
...
user_data                    = filebase64(var.user_data)
...     
  • In my lab, I used cloud-init to install nginx and write an HTML page that will be the server’s HomePage at Startup.
  • Make sure your public ssh key is in your home directory or just modify the path below (see variables.tf)
 resource "aws_key_pair" "terra_key" {    key_name   = var.key_name   public_key = file("~/id_rsa_aws.pub")  } ## Change me

2. LAUNCH THE INSTANCE

  • Once in “launch-instance” directory, you can run the plan command to validate the 10 resources required to launch the EC2 Instance (end-state). The output has been truncated to reduce verbosity
$ terraform plan
   Refreshing Terraform state in-memory prior to plan... 
  ------------------------------------------------------------------------
  An execution plan has been generated and is shown below.
    Terraform will perform the following actions:

  ... # VPC declaration (see previous vpc deploy 
  ...
  # aws_key_pair.terra_key will be created
   + resource "aws_key_pair" "terra_key" {
      + key_name    = "demo_aws_KeyPair"
      + key_pair_id = (known after apply)
      + public_key  = "ssh-rsa AAAAB3Nz…"
      ...}
  # aws_ebs_volume.terra_vol[0] will be created
  + resource "aws_ebs_volume" "terra_vol" {...}
  
 # aws_instance.terra_inst will be created
  + resource "aws_instance" "terra_inst" {
    + ...
    + ami                      = "ami-01861c2f0a2adfdb7"  
    + availability_zone        = "us-east-1a"   
    + instance_type            = "t2.micro" 
    + key_name                 = "demo_aws_KeyPair"
    +  private_ip              = "192.168.10.51"
    + tags                     = {
     ----+ "Name" = "TerraCompute"
        }
     + user_data                = "c8c701575f9c76db131ccf77cf352da"
     + ebs_block_device {
     + network_interface {
     + root_block_device {
     + ...
     + ...}
   
  # aws_volume_attachment.terra_vol_attach[0] will be created
  + resource "aws_volume_attachment" "terra_vol_attach" {
      + device_name = "/dev/xvdb"
      ...}
   ...
  }
  Plan: 10 to add, 0 to change, 0 to destroy.
  • Let’s launch our CENTOS7  instance using terraform apply (I left a map of different OS ami’s in the variables.tf you can choose)
$ terraform apply -auto-approve
...
aws_vpc.terra_vpc: Creating...
aws_key_pair.terra_key: Creation complete after 0s [id=demo_aws_KeyPair]
aws_security_group.terra_sg: Creation complete after 3s [id=sg-0b04564e8b]
aws_instance.terra_inst: Still creating... [40s elapsed]
aws_instance.terra_inst: Creation complete after 44s [id=i-046f1b1406bff]
aws_ebs_volume.terra_vol[0]: Creation complete after 11s [id=vol-037c1a9f9]aws_volume_attachment.terra_vol_attach[0]: Creation complete after 21s [.. 
...
Apply complete! Resources: 10 added, 0 changed, 0 destroyed.

Outputs:
...
vpc_Name = Terravpc
vpc_CIDR = 192.168.0.0/16
Subnet_CIDR = 192.168.10.0/24
SecurityGroup_ingress_rules = [
  "Inbound HTTP access :  80 , CIDR: 0.0.0.0/0",
  "Inbound HTTPS access :  443 , CIDR: 0.0.0.0/0",
  "Inbound RDP access :  3389 , CIDR: 0.0.0.0/0",
  "Inbound SSH access:  22 , CIDR: 0.0.0.0/0",
]
SSH_Connection = ssh connection to instance TerraCompute ==> sudo ssh -i ~/id_rsa_aws centos@35.173.222.166
private_ip = "192.168.10.51"
public_ip  = "54.91.27.50"

  • Once the instance is provisioned, just copy the public IP address(54.91.27.50) in your browser and Voila!
  • Here I just embedded a video link into the index page but you can adapt the cloud-init file to your own liking (with new content in the write_file section.
  • You can also tear down this configuration by simply running terraform destroy from the same directory

 

Tips

  • You can fetch any of the specified attributes in outputs.tf  using the Terraform output command i.e:  
$ terraform output SSH_Connection
ssh connection to instance TerraCompute ==> sudo ssh -i ~/id_rsa_aws centos@54.91.27.50
  • Terraform Console:
    Although Terraform is a declarative language, there are still myriads of functions you can use to process strings/numbers/lists/mappings etc. In my case I had to do some tests to get the right syntax for outputting the  ingress SG rules:
$ terraform console
> formatlist("%s: %s" ,aws_security_group.terra_sg.ingress[*].description,aws_security_group.terra_sg.ingress[*].to_port)
[ "Inbound HTTP access : 80", "Inbound HTTPS access : 443",
"Inbound SSH access: 22",]
  • There is an excellent all in one script with examples of most terraform functions >> here 

Differences between OCI/AWS & things I wish AWS provider had

  • Unlike OCI, most of AWS resources don’t have a display-name attribute, hence a tag is necessary each time you want to create a resource in AWS 
  • If you are used to DNS-labels for Subnet/VCNs in OCI, this is not something you’ll find in AWS
  • There is no direct way of setting the instance hostname like in OCI’s hostname_label attribute, you will have to do it via user-data
  • Bummer: I wish AWS had a data source for aws_key_pair as we can’t check if a key pair exists while launching an instance. This is a drawback for cases where a dev team needs to share a key. The only option is to give it a different key name for each deployment even if the key is the same, which will generate a lot of unnecessary duplicates. 

 

CONCLUSION

  • We have demonstrated in this tutorial how to quickly deploy an instance using terraform in AWS and leverage Cloud-init to bootstrap an instance into a webserver.
  • Remember that all user attributes in this exercise can be modified in the variables.tf file.
  • Improvement: In my next blog post, I will look to improve the vpc.tf code to leverage for_each function on map collections to make security group rules creation dynamic and conditional.
    stay tuned
Share on: