Infrastructure Automation With Terraform

Github repository




AWS Setup:

Make sure you have installed the AWS CLI

  • if you use my vagrant devops-box it is included by default:  
  • https://github.com/jasvinder-aulakh/devops-box.git

Useful Commands

$ terraform plan                                  # plan
$ terraform apply                                 # shortcut for plan & apply - avoid this in production
$ terraform plan -out out.terraform      # terraform plan and write the plan to out file
$ terraform apply out.terraform            # apply terraform plan using out file
$ terraform show                                  # show current state
$ cat terraform.tfstate                           # show state in JSON format

Reference Documentation



<<<<<<<<<<<< What is Terraform>>>>>>>>>>
  

1. Infrastructure as a Code.
2. Automation of your infrastructure
3. Keep your infrastructure in a certain state.
4. Make your infrastructure auditable
5. Terraform can automate the provisioning of the infrastructure itself.




<<<<<<<<<<<Installing Terraform>>>>>>>>>>>

https://www.terraform.io/downloads.html

Download and install it in your system.

after installation should be run like:

$ terraform
Usage: terraform [--version] [--help] <command> [args]

The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.

Common commands:
    apply              Builds or changes infrastructure
    console            Interactive console for Terraform interpolations
    destroy            Destroy Terraform-managed infrastructure
    env                Workspace management
    fmt                Rewrites config files to canonical format
    get                Download and install modules for the configuration
    graph              Create a visual graph of Terraform resources
    import             Import existing infrastructure into Terraform
    init               Initialize a Terraform working directory
    output             Read an output from a state file
    plan               Generate and show an execution plan
    providers          Prints a tree of the providers used in the configuration
    push               Upload this Terraform module to Atlas to run
    refresh            Update local state file against real resources
    show               Inspect Terraform state or plan
    taint              Manually mark a resource for recreation
    untaint            Manually unmark a resource as tainted
    validate           Validates the Terraform files
    version            Prints the Terraform version
    workspace          Workspace management

All other commands:
    debug              Debug output management (experimental)
    force-unlock       Manually unlock the terraform state
    state              Advanced state management 



<<<<<<<<<<< Terraform installation using vagrant>>>>>>>>>>>>


Vagrant is a tool for building and managing virtual machine environments in a single workflow. With an easy-to-use workflow and focus on automation, Vagrant lowers development environment setup time, increases production parity, and makes the "works on my machine" excuse a relic of the past


1. First install the virtual box in your system: from

https://www.virtualbox.org/wiki/Downloads

2.  Second install Vagrant from:

https://www.vagrantup.com/downloads.html

3. Create a new VM.

do clone devops-box from github:

After cloning there are 3 files:


-rw-r--r--   1 jass-devops  staff   137B May 20 07:42 README.md
-rw-r--r--   1 jass-devops  staff   388B May 20 10:17 Vagrantfile
drwxr-xr-x   3 jass-devops  staff    96B May 20 10:17 scripts/


Write Vagrant file as per your required VM, written for ubuntu as given below:

## cat Vagrantfile

Vagrant.configure(2) do |config|
config.vm.define "devops-box" do |devbox|
devbox.vm.box = "ubuntu/xenial64"
    #devbox.vm.network "private_network", ip: "192.168.199.9"
    #devbox.vm.hostname = "devops-box"
      devbox.vm.provision "shell", path: "scripts/install.sh"
    devbox.vm.provider "virtualbox" do |v|
      v.memory = 2048
      v.cpus = 2
    end
end
end

## In scripts folder, write install.sh bash script to do terraform, awscli and packer installation:

## cat install.sh

#!/bin/bash
set -x

if [ -e /etc/redhat-release ] ; then
  REDHAT_BASED=true
fi

TERRAFORM_VERSION="0.11.1"
PACKER_VERSION="0.10.2"
# create new ssh key
[[ ! -f /home/ubuntu/.ssh/mykey ]] \
&& mkdir -p /home/ubuntu/.ssh \
&& ssh-keygen -f /home/ubuntu/.ssh/mykey -N '' \
&& chown -R ubuntu:ubuntu /home/ubuntu/.ssh

# install packages
if [ ${REDHAT_BASED} ] ; then
  yum -y update
  yum install -y docker ansible unzip wget
else
  apt-get update
  apt-get -y install docker.io ansible unzip
fi
# add docker privileges
usermod -G docker ubuntu
# install pip
pip install -U pip && pip3 install -U pip
if [[ $? == 127 ]]; then
    wget -q https://bootstrap.pypa.io/get-pip.py
    python get-pip.py
    python3 get-pip.py
fi
# install awscli and ebcli
pip install -U awscli
pip install -U awsebcli

#terraform
T_VERSION=$(/usr/local/bin/terraform -v | head -1 | cut -d ' ' -f 2 | tail -c +2)
T_RETVAL=${PIPESTATUS[0]}

[[ $T_VERSION != $TERRAFORM_VERSION ]] || [[ $T_RETVAL != 0 ]] \
&& wget -q https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip \
&& unzip -o terraform_${TERRAFORM_VERSION}_linux_amd64.zip -d /usr/local/bin \
&& rm terraform_${TERRAFORM_VERSION}_linux_amd64.zip

# packer
P_VERSION=$(/usr/local/bin/packer -v)
P_RETVAL=$?

[[ $P_VERSION != $PACKER_VERSION ]] || [[ $P_RETVAL != 1 ]] \
&& wget -q https://releases.hashicorp.com/packer/${PACKER_VERSION}/packer_${PACKER_VERSION}_linux_amd64.zip \
&& unzip -o packer_${PACKER_VERSION}_linux_amd64.zip -d /usr/local/bin \
&& rm packer_${PACKER_VERSION}_linux_amd64.zip

# clean up
if [ ! ${REDHAT_BASED} ] ; then
  apt-get clean
fi



## vagrant up  ( To create vm, which defined in vagrant file with install.sh setup)

## vagrant ssh ( To login in new created vm)


Now you have done vagrant setup and created new vm with terraform and awscli installed.

<<<<<<<<<<<<< Terraform- spinning up  an instance on AWS >>>>>>>>>>>>>>

Process:

1. Open AWS account
2. Create IAM  admin user
3. Create terraform file to spin up t2.micro instance
4. Run terraform apply


<<<<<<<<<<<< Create terraform file >>>>>>>>>>>>

# vim instance.tf

provider "aws" {
  access_key = "ACCESS_KEY_HERE"
  secret_key = "SECRET_KEY_HERE"
  region     = "us-east-1"
}

resource "aws_instance" "example" {
  ami           = "ami-0d729a60"
  instance_type = "t2.micro"
}




<<<<<<<<<<< Variables in terraform >>>>>>>>>>>>

Everything in one file is not great.
Use variables to hide secrets
Use variables for elements that might change
Use variables to make it yourself easier to reuse terraform files

Create 4 files:

instance.tf, provider.tf, vars.tf and terraform.tfvars


# cat instance.tf

resource "aws_instance" "example" {
  ami           = "${lookup(var.AMIS, var.AWS_REGION)}"
  instance_type = "t2.micro"
}

#  cat provider.tf

provider "aws" {
    access_key = "${var.AWS_ACCESS_KEY}"
    secret_key = "${var.AWS_SECRET_KEY}"
    region = "${var.AWS_REGION}"
}


# cat vars.tf


variable "AWS_ACCESS_KEY" {}
variable "AWS_SECRET_KEY" {}
variable "AWS_REGION" {
  default = "eu-west-1"
}
variable "AMIS" {
  type = "map"
  default = {
    us-east-1 = "ami-13be557e"
    us-west-2 = "ami-06b94666"
    eu-west-1 = "ami-0d729a60"
  }
}



# cat terraform.tfvars

AWS_ACCESS_KEY=""
AWS_SECRET_KEY=""

Then execute terraform apply


<<<<<<<<<<<<<<<<<<<<<Software Provisioning>>>>>>>>>>>>>>>>>>>>>>

There are 2 ways to software provisioning on your instance:

1. You can build your own custom AMI and bundle your software with the image

2. Another way is to boot standardized AMIs, and then install the software on it, you need



First, define: 5 files- instance.tf, vars.tf, provider.tf, terraform.tfvars and script.sh
then create a key as name of "mykey"

# vim instance.tf

 resource "aws_key_pair" "mykey" {
  key_name = "mykey"
  public_key = "${file("${var.PATH_TO_PUBLIC_KEY}")}"
}

resource "aws_instance" "example" {
  ami = "${lookup(var.AMIS, var.AWS_REGION)}"
  instance_type = "t2.micro"
  key_name = "${aws_key_pair.mykey.key_name}"

  provisioner "file" {
    source = "script.sh"
    destination = "/tmp/script.sh"
  }
  provisioner "remote-exec" {
    inline = [
      "chmod +x /tmp/script.sh",
      "sudo /tmp/script.sh"
    ]
  }
  connection {
    user = "${var.INSTANCE_USERNAME}"
    private_key = "${file("${var.PATH_TO_PRIVATE_KEY}")}"
  }
}



# vim vars.tf

variable "AWS_ACCESS_KEY" {}
variable "AWS_SECRET_KEY" {}
variable "AWS_REGION" {
  default = "ap-south-1"
}
variable "AMIS" {
  type = "map"
  default = {
    ap-south-1 = "ami-0189d76e"
    us-west-2 = "ami-06b94666"
    eu-west-1 = "ami-844e0bf7"
  }
}

variable "PATH_TO_PRIVATE_KEY" {
  default = "mykey"
}
variable "PATH_TO_PUBLIC_KEY" {
  default = "mykey.pub"
}
variable "INSTANCE_USERNAME" {
  default = "ubuntu"
}


# vim provider.tf

provider "aws" {
    access_key = "${var.AWS_ACCESS_KEY}"
    secret_key = "${var.AWS_SECRET_KEY}"
    region = "${var.AWS_REGION}"
}


# vim terraform.tfvars

 AWS_ACCESS_KEY=""
AWS_SECRET_KEY= ""

# vim script.sh

#!/bin/bash

# sleep until instance is ready
until [[ -f /var/lib/cloud/instance/boot-finished ]]; do
  sleep 1
done

# install nginx
apt-get update
apt-get -y install nginx

# make sure nginx is started
service nginx start


then run terraform apply

the result should like - aws_instance.example: Creation complete after 58s (ID: i-0f0e050c4cf2a3bfe)

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

# check your nginx with IP-





































Comments

Post a Comment

Thank you for visiting my blog.

Popular posts from this blog

Defining Audit Rules

The Audit system operates on a set of rules that define what is to be captured in the log files. There are three types of Audit rules that can be specified: Control rules — allow the Audit system's behavior and some of its configuration to be modified. File system rules — also known as file watches, allow the auditing of access to a particular file or a directory. System call rules — allow logging of system calls that any specified program makes.  Audit rules can be specified on the command line with the auditctl utility (note that these rules are not persistent across reboots), or written in the /etc/audit/audit.rules file. The following two sections summarize both approaches to defining Audit rules.   Auditing goals By using a powerful audit framework, the system can track many event types to monitor and audit the system. Examples include: Audit file access and modification See who changed a particular file Detect unauthorized cha...
Read more

AIX Install packages, upgrade, patching commands

lslpp -L                             displays info about all installed filesets or fileset updates lslpp -L <fileset>                   displays info about that fileset lslpp -h <fileset>                   shows the history of the fileset lslpp -l | grep <fileset>           shows if it is installed or not lslpp -lc| grep <fileset>           shows the state (Aplp., Comm., Broken..) in /etc/objrepos and in /usr/lib/objrepos of a fileset lslpp -f <fileset>                   shows all files that are installed with  a particular fileset lslpp -w /usr/local/bin/lsof        shows the fileset which contains the giv...
Read more

Oracle Database Quick Installation steps 11g Release 2 for Linux x86-64

This article is a comprehensive steps for installing Oracle Database 11 g Release 2 (11.2.0.1) on the Red Hat Enterprise Linux 6 (RHEL6)  operating environment. Both 32-bit (x86) and 64-bit (x86_64) architectures are covered in this guide. Unless otherwise noted, the installation steps are the same for either. Having said that, one of the first decisions to make before continuing with this guide is which architecture you will be using. Both Oracle and Linux must be installed on the same operating system architecture. For example, 32-bit Oracle is only supported to run on 32-bit Linux OS and 64-bit Oracle is only supported to run on 64-bit Linux OS. Install Required Linux Packages for Oracle: After installing the Linux OS, the next step is to verify and install all packages required for Oracle Database 11 g Release 2. The Oracle Universal Installer (OUI) performs checks on the machine during installation to verify that it meets the...
Read more