OCI FortiGate HA Cluster – Reference Architecture: Code Review and Fixes
Introduction OCI Quick Start repositories on GitHub are collections of Terraform scripts and configurations provided by Oracle. These repositories ... Read More
Learn more about why Eclipsys has been named the 2023 Best Workplaces in Technology and Ontario, Certified as a Great Place to Work in Canada and named Canada’s Top SME Employer!
Learn more!This has become a habit so far to explore different ways of automated provisioning for each cloud provider. This time, I will try Terraform on AWS and reproduce the same deployment I have completed on Oracle Cloud, and as usual, we won’t just deploy an instance but also configure a website linked to its public IP. I’ll end this post with some notes on OCI/AWS differences.
Note: I have done a similar task with my bash scripts and AWS CLI which was very helpful to understand the logic behind each IAC interaction during the resource creation (More details >here).
Here’s a direct link to my GitHub repo linked to this lab =>: terraform-examples/terraform-provider-aws
Content :
I. Terraform setup
IV. Partial deployment
V. Full deployment
Tips & Conclusion
Topology
The following illustration shows the layers involved between your workstation and Oracle Cloud Infrastructure while running the terraform commands along with the instance attributes we will be provisioning.
Besides describing my GitHub repo before starting this tutorial, I’ll just briefly discuss some principles.
– Can be a single file or split into multiple tf or tf.json files, any other file extension is ignored.
– Files are merged in alphabetical order but resource definition order doesn’t matter (subfolders are not read).
– Common configurations have 3 types of tf files and a statefile.
1- main.tf: terraform declaration code (configuration) . The file name can be anything you choose
2- variables.tf: Resource variables needed for the deploy
3- outputs.tf: displays the resources detail at the end of the deploy
4- terraform.tfstate: keeps track of the state of the stack(resources) after each terraform apply run
Component "Provider_Resource_type" "MyResource_Name" { Attribute1 = value ..
Attribute2 = value ..}
The easiest way is to create/locate an instance from the console and then use the import function from Terraform to generate each of the related components in HCL format (vpc,instance,subnet,etc..) based on their id.
Example for a VPC >>
1- Create a shell resource declaration for the vpc ina file called vpc.tf
2- Get the id of the vpc resource from your AWS Console
3- Run the Terraform import then run Terraform show to extract the vpc’s full declaration from AWS in the same file (vpc.tf)
4- Now you can remove the id attribute with all non required attributes to create a vpc resource (Do that for each resource)
1- # vi vpc.tf
provider "aws" { region = "us-east-1" }
resource "aws_vpc" "terra_vpc" {
}
2- # terraform import aws_vpc.terra_vpc vpc-0091141e28608813c
3- # terraform show -no-color > vpc.tf
Note:
If you want to import all the existing resources in your account in bulk mode (not one by one) there is a tool called Terraforming, which can import both code and state from your AWS account automatically.
Terraform lab content: I have deliberately split this lab into 2:
Since I’m on windows I tried the lab using both Gitbash and WSL (Ubuntu) terminal clients (same applies for Mac).
Windows: Download and run the installer from their website (32-bit ,64-bit)
Linux: Download, unzip and move the binary to the local bin directory
$ wget https://releases.hashicorp.com/terraform/0.12.28/terraform_0.12.28_linux_amd64.zip
$ unzip terraform_0.12.18_linux_amd64.zip
$ mv terraform /usr/local/bin/
Once installed run the version command to validate your installation
$ terraform --version
Terraform v0.12.24
AWS authentication
To authenticate with your AWS account, Terraform will need to provide both access_key_id & secret_access_key . This can be done either by sharing the authentication parameters with aws-cli or by Including the access_key and key_id within the Terraform config (i.e. variables.tf)
Assumptions
– I will assume that either of the two below authentication options is present/configured in your workstation:
$ aws configure list
Name Value Type Location
---- --------- ---- --------
profile <not set> None None
access_key ****************J2WA shared-credentials-file
region us-east-1 config-file ~/.aws/config
provider "aws" {
# access_key = "${var.aws_access_key}" – uncomment & replace accordingly
# secret_key = "${var.aws_secret_key}" – uncomment & replace accordingly
region = var.aws_region --- uncomment and replace accordingly}
– I’ll also assume the presence of an ssh key pair to attach to your ec2 instance. If not here is a command to generate a PEM-based key pair.
$ ssh-keygen -P "" -t rsa -b 2048 -m pem -f ~/id_rsa_aws
Generating public/private rsa key pair.
Your identification has been saved in /home/brokedba/id_rsa_aws.
Your public key has been saved in /home/brokedba/id_rsa_aws.pub.
$ git clone https://github.com/brokedba/terraform-examples.git
Note: As explained earlier you will find 2 directories inside the repository which will make things easier:
terraform-provider-aws/create-vpc/
To grasp how we deploy a single VPC.terraform-provider-aws/launch-instance/
For the final instance deploy.
1. INSTALL AND SETUP THE AWS PROVIDER
terraform-provider-aws/create-vpc
where our configuration resides (i.e vpc)GitBash $ cd /c/Users/brokedba/aws/terraform-examples/terraform-provider-aws/create-vpc
ubuntu $ cd /mnt/c/Users/brokedba/aws/terraform-examples/terraform-provider-aws/create-vpc
terraform init
.$ terraform init
Initializing the backend...
Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "aws" (hashicorp/aws) 3.11.0...
* provider.aws: version = "~> 3.11"
$ terraform --version
Terraform v0.12.24
+ provider.aws v3.11.0 ---> the provider is now installed
create-vpc
directory. Here, only *.tf
files matter (click to see content)$ tree
.
|-- outputs.tf ---> displays resources detail after the deploy
|-- variables.tf ---> Resource variables needed for the deploy
|-- vpc.tf ---> Our vpc terraform declartion
DEPLOY A SIMPLE VPC
terraform plan
the command to create an execution plan (quick dry run to check the desired state/actions).
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Terraform will perform the following actions:
# aws_internet_gateway.terra_igw will be created
+ resource "aws_internet_gateway" "terra_igw"
{..}
# aws_route_table.terra_rt will be created
+ resource "aws_route_table" "terra_rt" {
{..}
# aws_route_table_association.terra_rt_sub will be created
+ "aws_route_table_association" "terra_rt_sub"
{..}
# aws_security_group.terra_sg will be created
+ resource "aws_security_group" "terra_sg" {
+ arn = (known after apply)
+ description = "SSH ,HTTP, and HTTPS"
+ egress =[{...}]
+ id = (known after apply)
+ ingress = [
+ {... rules for HTTP/SSH/HTTPS ingress access}
# aws_subnet.terra_sub will be created
+ resource "aws_subnet" "terra_sub" {
...
+ cidr_block = "192.168.10.0/24”
...}
# aws_vpc.terra_vpc will be created
+ resource "aws_vpc" "terra_vpc" {
...
+ cidr_block = "192.168.0.0/16”
...
+ tags = {
+ "Name" = "Terravpc" } }
Plan: 6 to add, 0 to change, 0 to destroy.
– The output being too verbose I deliberately kept only relevant attributes for the VPC resource plan
terraform deploy
to apply the changes required to create our VPC (listed in the plan)$ terraform apply -auto-approve
aws_vpc.terra_vpc: Creating...
...
Apply complete! Resources: 6 added, 0 changed, 0 destroyed.
Outputs:
Subnet_CIDR = 192.168.10.0/24
Subnet_Name = terrasub
internet_gateway_Name = terra-igw
map_public_ip_on_launch = true
route_table_Name = terra-rt
vpc_Name = Terravpc
vpc_id = vpc-09d491059eb740562
vpc_CIDR = 192.168.0.0/16
vpc_dedicated_security_group_Name = terra-sg
vpc_dedicated_security_ingress_rules =
["Inbound HTTP access : 80 , CIDR: 0.0.0.0/0",
"Inbound HTTPS access : 443 , CIDR: 0.0.0.0/0",
"Inbound RDP access : 3389 , CIDR: 0.0.0.0/0",
"Inbound SSH access: 22 , CIDR: 0.0.0.0/0",]
Observations:
– The deploy started by loading the resources variables in variables.tf which allowed the execution of vpc.tf
– Finally terraform fetched the attributes of the created resources listed in outputs.tf
Note: We’ll now destroy the vpc as the next instance deploy contains the same vpc specs.
$ terraform destroy -auto-approve
Destroy complete! Resources: 6 destroyed.
1. OVERVIEW
terraform-provider-aws/launch-instance/
$ tree ./terraform-provider-aws/launch-instance
.
|-- cloud-init ---> SubFolder
| `--> vm.cloud-config ---> script to config a webserver & add a HomePage
|-- compute.tf ---> Instance related terraform configuration
|-- outputs.tf ---> displays the resources detail at the end of the deploy
|-- variables.tf ---> Resource variables needed for the deploy
|-- vpc.tf ---> same vpc terraform declaration deployed earlier
Note: As you can see we have 2 additional files and one Subfolder. compute.tf is where the compute instance and all its attributes are declared. All the other .tf files come from my vpc example with some additions for variables.tf and output.tf
...variable "user_data" { default = "./cloud-init/vm.cloud-config"}
$ vi compute.tf
resource "aws_instance" "terra_inst" {
...
user_data = filebase64(var.user_data)
...
resource "aws_key_pair" "terra_key" { key_name = var.key_name public_key = file("~/id_rsa_aws.pub") } ## Change me
2. LAUNCH THE INSTANCE
launch-instance”
directory, you can run the plan command to validate the 10 resources required to launch the EC2 Instance (end-state). The output has been truncated to reduce verbosity$ terraform plan
Refreshing Terraform state in-memory prior to plan...
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Terraform will perform the following actions:
... # VPC declaration (see previous vpc deploy
...
# aws_key_pair.terra_key will be created
+ resource "aws_key_pair" "terra_key" {
+ key_name = "demo_aws_KeyPair"
+ key_pair_id = (known after apply)
+ public_key = "ssh-rsa AAAAB3Nz…"
...}
# aws_ebs_volume.terra_vol[0] will be created
+ resource "aws_ebs_volume" "terra_vol" {...}
# aws_instance.terra_inst will be created
+ resource "aws_instance" "terra_inst" {
+ ...
+ ami = "ami-01861c2f0a2adfdb7"
+ availability_zone = "us-east-1a"
+ instance_type = "t2.micro"
+ key_name = "demo_aws_KeyPair"
+ private_ip = "192.168.10.51"
+ tags = {
----+ "Name" = "TerraCompute"
}
+ user_data = "c8c701575f9c76db131ccf77cf352da"
+ ebs_block_device {
+ network_interface {
+ root_block_device {
+ ...
+ ...}
# aws_volume_attachment.terra_vol_attach[0] will be created
+ resource "aws_volume_attachment" "terra_vol_attach" {
+ device_name = "/dev/xvdb"
...}
...
}
Plan: 10 to add, 0 to change, 0 to destroy.
$ terraform apply -auto-approve
...
aws_vpc.terra_vpc: Creating...
aws_key_pair.terra_key: Creation complete after 0s [id=demo_aws_KeyPair]
aws_security_group.terra_sg: Creation complete after 3s [id=sg-0b04564e8b]
aws_instance.terra_inst: Still creating... [40s elapsed]
aws_instance.terra_inst: Creation complete after 44s [id=i-046f1b1406bff]
aws_ebs_volume.terra_vol[0]: Creation complete after 11s [id=vol-037c1a9f9]aws_volume_attachment.terra_vol_attach[0]: Creation complete after 21s [..
...
Apply complete! Resources: 10 added, 0 changed, 0 destroyed.
Outputs:
...
vpc_Name = Terravpc
vpc_CIDR = 192.168.0.0/16
Subnet_CIDR = 192.168.10.0/24
SecurityGroup_ingress_rules = [
"Inbound HTTP access : 80 , CIDR: 0.0.0.0/0",
"Inbound HTTPS access : 443 , CIDR: 0.0.0.0/0",
"Inbound RDP access : 3389 , CIDR: 0.0.0.0/0",
"Inbound SSH access: 22 , CIDR: 0.0.0.0/0",
]
SSH_Connection = ssh connection to instance TerraCompute ==> sudo ssh -i ~/id_rsa_aws centos@35.173.222.166
private_ip = "192.168.10.51"
public_ip = "54.91.27.50"
$ terraform output SSH_Connection
ssh connection to instance TerraCompute ==> sudo ssh -i ~/id_rsa_aws centos@54.91.27.50
$ terraform console
> formatlist("%s: %s" ,aws_security_group.terra_sg.ingress[*].description,aws_security_group.terra_sg.ingress[*].to_port)
[ "Inbound HTTP access : 80", "Inbound HTTPS access : 443",
"Inbound SSH access: 22",]
Differences between OCI/AWS & things I wish AWS provider had
variables.tf
file.Introduction OCI Quick Start repositories on GitHub are collections of Terraform scripts and configurations provided by Oracle. These repositories ... Read More
Introduction So far, I have used Oracle AutoUpgrade, many times in 3 different OS’. Yet the more you think you’ve seen it all and reached the ... Read More