Lets talk about Cloud

As mentioned in my post before, it is no so easy as a beginner to get everything realized in Terraform. The challenge was, deploy to a web site in Azure which is able to scale out behind a load balancer. After demonstrating the way be using virtual machine scale sets, I would like to show the way I found with Azure App Services as the service to go to.

If you take a look at the simple sample in the documentation, you see that it is very easy to deploy a simple website in azure. Out of this, my idea was, it could not be so complicated in Terraform to achieve the same.

So let’s get started with our script and define the provider and resource group:

## Lets start with an app service in Azure
provider "azurerm" {
    version = "~>2.3.0"
    features {}
}

### Resource Group
resource "azurerm_resource_group" "rg" {
  name     = "rg-appservice-test"
  location = "West Europe"
  tags = {
      App = "appservice"
      Source = "Terraform"
  }
}

That’s pretty much the same, as in the other posts. The only thing I changed was the version of the provider. You can always check the latest version in the GitHub repo releases section for the provider.

For an App Service, we need an “App Service Plan“.

### App Service plan
resource "azurerm_app_service_plan" "appservice" {
  name                = "azapp-plan-eastus-001"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name

  sku {
    tier = "Standard"
    size = "S1"
  }
}

Which SKU to choose depends on the features needed. You can check the possible options in the DOCs. As soon as we have an App Service Plan, we can define our App Service:

## The App Service 
resource "azurerm_app_service" "appservice" {
  name                = "azapp-appservice-test-001"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  app_service_plan_id = azurerm_app_service_plan.appservice.id
 }

Until this point, it was pretty much straight forward. My plan was now, to point to a simple web site that is provided by one of my GitHub repos. So use GitHub as the deployment source for my Web App. In the Azure portal it is very simple to configure:

Reading the documentation of the resource “azurerm_app_service”, I found out that there is the section of the site_config to configure the scm_type. But I did not find an option to define the “repo_url” and “branch”. It seems to be possible to read these attributes from an existing App Service, but I did not find a way to define it. After a little bit of research, I found out that this seems to be work in progress and not finished til now.

So I found another way to implement the connection to my GitHub hosted website code. There is an option to deploy an ARM template in a Terraform script. And that’s the way how I made my challenge.

Starting with the following ARM template to define the source code for my App Service deployment:

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "siteName": {
           "type": "string",
           "defaultValue": "[concat('WebApp-', uniqueString(resourceGroup().id))]",
            "metadata": {
                "description": "The name of you Web Site."
            }
        },
        "location": {
            "type": "string",
            "defaultValue": "[resourceGroup().location]",
            "metadata": {
                "description": "Location for all resources."
            }
        },
        "repoURL": {
            "type": "string",
            "defaultValue": "https://github.com/nielsophey/simplewebsite",
            "metadata": {
                "description": "The URL for the GitHub repository that contains the project to deploy."
            }
        },
        "branch": {
            "type": "string",
            "defaultValue": "master",
            "metadata": {
                "description": "The branch of the GitHub repository to use."
            }
        }
    },
    "variables": {},
    "resources": [
            {
                "type": "Microsoft.Web/sites",
                "apiVersion": "2018-02-01",
                "name": "[parameters('siteName')]",
                "location": "[parameters('location')]",
                "properties": {
                     },
                "resources": [
                    {
                        "type": "sourcecontrols",
                        "apiVersion": "2018-02-01",
                        "name": "web",
                        "location": "[parameters('location')]",
                        "dependsOn": [
                            "[resourceId('Microsoft.Web/sites', parameters('siteName'))]"
                        ],
                        "properties": {
                            "repoUrl": "[parameters('repoURL')]",
                            "branch": "[parameters('branch')]",
                            "isManualIntegration": true
                            }
                    }
                ]
            }
       ],
    "outputs": {}
}

I saved the template in the same folder as my Terraform script under appservice.json and added the following code to my Terraform script:

## Deploy the Deployment option by ARM Template
resource "azurerm_template_deployment" "appservice" {
    name                    = "arm-appservice-template"
    resource_group_name     = azurerm_resource_group.rg.name

    template_body = file("appservice.json")

    parameters = {
        "siteName" = azurerm_app_service.appservice.name
        "location" = azurerm_resource_group.rg.location
    }

    deployment_mode = "Incremental"
  
}

In the appservice.json is the definition of the GitHub repo and the needed branch. I set the two parameters as default so that I only need to define the App Service name and location for the ARM template to work probably.

So finally I got my web site up and running:

I totally agree that the chosen way is not the best way to deploy a simple App Services like this, but it was a great learning curve for me. I understand to check the issues, pull requests and release notes in GitHub for the Terraform Azure provider. As well as I learned how to read the documentation of the Terraform resources.

Also I hope, that the resource “azurerm_app_service_source_control” or something equal to that, will be available soon so that we do not need the way via ARM template anymore.

The two scripts can be found on my GitHub Repo for all these samples.

Now that we have one VM serving a web site, it is a common pattern to deploy not only one VM. Use multiple VMs to distribute the load. In Azure, this feature is called a virtual machine scale set (see the DOCs).

To build this in Terraform we need the azurerm_linux_virtual_machine_scale_set resource type. The documentation shows a sample on how to use it.

Please read first!

But CAUTION – I have done everything several times and tried a lot of possible parameters to deploy the scales set including the Apache webserver. I did not find out, why the configuration of the custom script extension does not work during the initial deployment. Only if you change the VM count after the deployment, the custom script will be deployed. You can see this issue here.

So I go through the whole sample and afterward I would like to show, how I would build the sample out of Yevgeniy Brikmann’s book by leveraging app services in Azure.

Let’s go first the way thru the virtual machine scale set:

We need a resource group to deploy everything to

### Resource Group
resource "azurerm_resource_group" "rg" {
  name     = "rg-vmssssample-test"
  location = "East US"
  tags = {
      App = "VMSS"
      Source = "Terraform"
  }
}

In this sample, we start using tags at the resource group level for the App we deployed, the source and what kind of environment we have. Also, I want to establish a naming convention based on the Microsoft best practices shared in this article.

So for a resource group, there is the suggested pattern
rg-<App or service name>-<Subscription type>-<### >

Next – the vNet

### Network
resource "azurerm_virtual_network" "vNet" {
  name                = "vnet-shared-eastus-001"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  address_space       = ["10.0.0.0/16"]
  tags = azurerm_resource_group.rg.tags
}

În the VNet we have to define the internal subnet for the VMs in the scale set

### Subnet
resource "azurerm_subnet" "sNet" {
  name                 = "snet-shared-vmsssample-001"
  resource_group_name  = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vNet.name
  address_prefix       = "10.0.2.0/24"
}

In my script, I add the following resource in front of the VM scale set definition. Because I want to add a FQDN to public IP assign to the load balancer. There is a helpful resource in Terraform to build a random String to be used for the FQDN:

### Random FQDN String
resource "random_string" "fqdn" {
 length  = 6
 special = false
 upper   = false
 number  = false
}

Implement a Loadbalancer into our script

The common design pattern is to deploy a load balancer in front of the VMs in the scale set. With this, the incoming traffic can be distributed between the virtual machines in the scale set. We add a load balancer definition to the script:

### Loadbalancer definition
resource "azurerm_lb" "vmsssample" {
 name                = "lb-vmsssample-test-001"
 location            = azurerm_resource_group.rg.location
 resource_group_name = azurerm_resource_group.rg.name

 frontend_ip_configuration {
   name                 = "ipconf-PublicIPAddress-test"
   public_ip_address_id = azurerm_public_ip.vmss-pip.id
 }

  tags = azurerm_resource_group.rg.tags
}

The load balancer needs some more configuration. We need to define a backend IP pool as well as a probe to check the health status of VMs in the backend pool:

### Define the backend pool
resource "azurerm_lb_backend_address_pool" "vmsssample" {
 resource_group_name = azurerm_resource_group.rg.name
 loadbalancer_id     = azurerm_lb.vmsssample.id
 name                = "ipconf-BackEndAddressPool-test"
}

### Define the lb probes
resource "azurerm_lb_probe" "vmsssample" {
 resource_group_name = azurerm_resource_group.rg.name
 loadbalancer_id     = azurerm_lb.vmsssample.id
 name                = "http-running-probe"
 port                = 80
}

The last step in the configuration is the rule for the load balancing – so which port should be balanced:

### Define the lb rule
resource "azurerm_lb_rule" "vmsssample" {
   resource_group_name            = azurerm_resource_group.rg.name
   loadbalancer_id                = azurerm_lb.vmsssample.id
   name                           = "http"
   protocol                       = "Tcp"
   frontend_port                  = 80
   backend_port                   = 80
   backend_address_pool_id        = azurerm_lb_backend_address_pool.vmsssample.id
   frontend_ip_configuration_name = "ipconf-PublicIPAddress-test"
   probe_id                       = azurerm_lb_probe.vmsssample.id
}

Now we have deployed the basic components of our architecture and can go ahead. As in our sample for a single VM it is important to define the network security group. But we do not need the SSH port been opened, we just need the port 80 on our webserver.

### Define the NSG
resource "azurerm_network_security_group" "vmsssample" {
    name                = "nsg-weballow-001"
    location            = azurerm_resource_group.rg.location
    resource_group_name = azurerm_resource_group.rg.name
    
     security_rule {
        name                       = "WebServer"
        priority                   = 1002
        direction                  = "Inbound"
        access                     = "Allow"
        protocol                   = "Tcp"
        source_port_range          = "*"
        destination_port_range     = "80"
        source_address_prefix      = "*"
        destination_address_prefix = "*"
     }
}

Finally, the VM scale set itself

### The VM Scale Set (VMSS)
resource "azurerm_linux_virtual_machine_scale_set" "vmsssample" {
  name                = "vmss-vmsssample-test-001"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  sku                 = "Standard_B2s"
  instances           = 1
  admin_username      = "adminuser"
  admin_password      = "Password1234!"
  disable_password_authentication = false
  tags = azurerm_resource_group.rg.tags

#### define the os image
 source_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "16.04-LTS"
    version   = "latest"
  }

#### define the os disk
  os_disk {
    storage_account_type = "Standard_LRS"
    caching              = "ReadWrite"
  }

#### Define Network
  network_interface {
      name    = "nic-01-vmsssample-test-001"
      primary = true

    ip_configuration {
        name      = "ipconf-vmssample-test"
        primary   = true
        subnet_id = azurerm_subnet.sNet.id
        load_balancer_backend_address_pool_ids = [azurerm_lb_backend_address_pool.vmsssample.id]
    }
    network_security_group_id = azurerm_network_security_group.vmsssample.id

  }
}

Now we can plan our script and apply it to our Azure Account. Now that we have out VM scale set up and running we need our Webserver in the machine again. To achieve this, we need to deploy a new resource – the “azurerm_virtual_machine_scale_setextension”. It is somehow the same kind of extension we used for the single VM – so our additional entry in the script will look like this:

### Add the Webserver to the VMSS
resource "azurerm_virtual_machine_scale_set_extension" "vmsssampleextension" {
  name                         = "ext-vmsssample-test"
  virtual_machine_scale_set_id = azurerm_linux_virtual_machine_scale_set.vmsssample.id
  publisher                    = "Microsoft.Azure.Extensions"
  type                         = "CustomScript"
  type_handler_version         = "2.0"
  auto_upgrade_minor_version   = true
  force_update_tag             = true
  
  settings = jsonencode({
      "commandToExecute" : "apt-get -y update && apt-get install -y apache2" 
    })
}

During my research on the web I found that with terraform version 0.12 the function jsoncode has been implemented. With this, it is easier to convert a given string to JSON. I used this function for the commandToExcecute attribute.

But

If we now deploy our script to azure we will have all components in place to have a virtual machine scale set with a web server installed. As mentioned at the beginning the custom script extension does not work as expected. If you go to the portal and change the number of deployed instances in the scaling option of the scale set, the custom script extensions will be deployed to the VMs. If we then browse to URL of the public IP – we will have the apache web server default website been presented.

So after scaling up – our script will show our desired state when browsing to the FQDN.

The next post will then show the deployment using Azure App Services to solve the same challenge and add a real website to that script.

The next iteration of the VM is to configure a Web Server running on the VM and add an auto-scaling function as well as a load balancer. Due to the point, that I’m not so aware of Linux, I took a little bit different approach to have a Web Server running on the VM. Yevgeniy uses in his book the following “user_data” option to have a web site been served by our VM.

user_date = <<-EOF
            #!/bin/bash
            echo "Hellom, World" > index.html
            nohup busybox httpd -f -p 8080 &
            EOF

I tried to get this as a script running in the VM just deployed. But I did not find out what will be the best way. So maybe this is a challenge for later, but take it the other way around, what is the normal Way in Azure to get something running in a VM just deployed. I normally use the custom script extensions to run a command in a machine. Especially in a Windows VM I would use any desired state configuration with this option. If you want to learn more about custom script extension focusing on a Linux VMs visit this DOCs article.

With this knowledge we now can add a section in our script to deploy a custom script extension:

resource "azurerm_virtual_machine_extension" "myFirstTerraform" {
  name = "myFirstTerraform-Script"
  virtual_machine_id = azurerm_linux_virtual_machine.myFirstTerraform.id
  publisher = "Microsoft.Azure.Extensions"
  type = "CustomScript"
  type_handler_version ="2.0"

  settings = <<SETTINGS
    {
      "commandToExecute" : "apt-get -y update && apt-get install -y apache2" 
    }
    SETTINGS
 }

The important configuration is made in the settings section. I added the command to install an apache web server on the machine and then we will have the standard website been served in port 80 on the Linux VM. The only trouble we get is, our network security group (NSG) we deployed, was only opening the ssh port. So we must add an additional rule in the NSG. So our NSG will look like this:

 resource "azurerm_network_security_group" "myFirstTerraform" {
    name                = "myFirstTerraform"
    location            = azurerm_resource_group.myFirstTerraform.location
    resource_group_name = azurerm_resource_group.myFirstTerraform.name

    security_rule {
        name                       = "SSH"
        priority                   = 1001
        direction                  = "Inbound"
        access                     = "Allow"
        protocol                   = "Tcp"
        source_port_range          = "*"
        destination_port_range     = "22"
        source_address_prefix      = "*"
        destination_address_prefix = "*"
     }

     security_rule {
        name                       = "WebServer"
        priority                   = 1002
        direction                  = "Inbound"
        access                     = "Allow"
        protocol                   = "Tcp"
        source_port_range          = "*"
        destination_port_range     = "80"
        source_address_prefix      = "*"
        destination_address_prefix = "*"
     }
}

If we now would run our script, we will be able to see the default apache web site on our Linux VM running in Azure:

To connect to this website it would be great to know on which public IP assigned to our Linux VM. As we learn in the book, we can use the output variables to achieve this. But there is one important difference. In Azure, a public IP is a resource on his own and will be attached to a network interface that then will be assigned to a VM. So we need to reference the IP in our output and not the VM.

What does that mean for our script:

output "public_ip" {
   value       = azurerm_public_ip.myFirstTerraform.ip_address
   description = "This is the asigned public ip to our VM"
}

If we have added this output to our script we can afterwards just get the ip after you apply your script again:

$ terraform apply

Outputs:

public_ip = 51.136.162.193

If you need the output of your latest terrafrom deployment again you just can call:

$ terraform output public_ip

51.136.162.193

So now you know the ip to browse to, if you want to see your apache 2 default webiste.

After reading Chapter 1 of the book it was time to get my machine ready for using Terraform to script the deployments in Azure. So I search the Microsoft Docs for a short guide and found this short description.

First Step – Install Terraform on my machine :

I decided to install Terraform in the Windows Subsystem for Linux (WSL) on my Windows10 machine. I’m on the fast-ring in the Windows10 insider program – so I’m already able to use WSL2. If you want to learn more about WSL2 visit the Microsoft page https://aka.ms/wsl2. To install Terraform in my WSL2 I opened the bash and entered the following command (use sudo of needed to have privileged rights):

# First - install unzip

apt-get install unzip

# Download the installation for Terraform (check for newest version before...)

wget https://releases.hashicorp.com/terraform/0.12.6/terraform_0.12.6_linux_amd64.zip

# Unzip the Terraform folder

unzip terraform_0.12.6_linux_amd64.zip terraform

# Move the folder to an accessible path for the system

mv terraform /usr/local/bin/

# test the Terraform version on your WSL machine

terraform --version

If everything worked fine, you will now have Terraform on your WSL to start using it.

Second Step – get Terraform connected

As soon as you want Terraform to apply your script in your Azure Tenant the best way is to deploy a service principal in your Azure Active Directory with the needed contributor rights. You can find these guide in the Microsoft Docs.

The last action is to define the environment variables for Terraform to use the just created credentials.

Last Step – deploy your first resource

Now that the environment is set up properly – we can deploy our first Azure resource. At what is the best first resource – yes, a resource group!

We need our first Terraform script. I made a new folder in the WSL and started my Visual Studio Code (VSCode) in this folder by just typing code . to start the UI in that folder and created my first script named main.tf and entered the following code:

# main.tf

provider "azurerm" {
  version = "~>2.0.0"
  features {}
}

resource "azurerm_resource_group" "rg" {
        name = "myFirstTerraform-RG"
        location = "westeurope"
        tags = {
            source = "Terraform"
        }
}

After saving the script I opened the terminal in VSCode and run my first Terraform command:

terraform init

If all environment variables are set correctly and we made no other typing error Terraform will interpret the provider statement and will download the needed Azure provider. This done we can apply our script to Azure by typing

terraform apply

We only have to type one final “yes” into the console and our first resource group is deployed in Azure. So let’s open the Azure portal and check:

That’s it – our first Terraform script was successfully been deploy to Azure!