Azure – Link Collection Cloud Adoption Framework for Azure

The Cloud Adoption Framework is the One Microsoft approach to cloud adoption in Azure, consolidating and sharing best practices from Microsoft employees, partners, and customers. The framework gives customers a set of tools, guidance, and narratives that help shape technology, business, and people strategies for driving desired business outcomes during their adoption effort.

Here is a short collection of links during a CAF related engagement:

Start with the CAF documentation in the Microsoft Docs.

CAF – Define Strategy, Ready and Plan

Find additional StrategyPlan and Ready documentation.

StrategyTools Strategy

Cloud Journey Tracker
Business Outcome Template
PlanTools Plan

Azure DevOps Demo Generator Template
Cloud adoption plan template
ReadyTools Ready

Cloud Adoption Framework – Azure Setup Guide
Cloud Adoption Framework Foundation Blueprint
Migration Landing Zone Blueprint
Naming and tagging tracking template

CAF – Adopt

Find additional Migrate and Innovate documentation.

AdoptTools Adopt

Strategic migration assessment and readiness tool (SMART)
Cloud Adoption Framework – Azure Migration Guide
Azure innovation guide

CAF -Govern and Manage

Find additional Govern and Manage documentation.

GovernTools Govern

Governance benchmarkGovernance process template
Cost Management process template
Deployment acceleration process template
Identity process template
Resource consistency process template
Security baseline process template
ManageTools Management

Microsoft Azure Well-Architected Review
Best practices source code
Operations management workbook

Links for Partner

Learn Terraform – Deploy an App Service instead of a scale set

As mentioned in my post before, it is no so easy as a beginner to get everything realized in Terraform. The challenge was, deploy to a web site in Azure which is able to scale out behind a load balancer. After demonstrating the way be using virtual machine scale sets, I would like to show the way I found with Azure App Services as the service to go to.

If you take a look at the simple sample in the documentation, you see that it is very easy to deploy a simple website in azure. Out of this, my idea was, it could not be so complicated in Terraform to achieve the same.

So let’s get started with our script and define the provider and resource group:

## Lets start with an app service in Azure
provider "azurerm" {
    version = "~>2.3.0"
    features {}
}

### Resource Group
resource "azurerm_resource_group" "rg" {
  name     = "rg-appservice-test"
  location = "West Europe"
  tags = {
      App = "appservice"
      Source = "Terraform"
  }
}

That’s pretty much the same, as in the other posts. The only thing I changed was the version of the provider. You can always check the latest version in the GitHub repo releases section for the provider.

For an App Service, we need an “App Service Plan“.

### App Service plan
resource "azurerm_app_service_plan" "appservice" {
  name                = "azapp-plan-eastus-001"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name

  sku {
    tier = "Standard"
    size = "S1"
  }
}

Which SKU to choose depends on the features needed. You can check the possible options in the DOCs. As soon as we have an App Service Plan, we can define our App Service:

## The App Service 
resource "azurerm_app_service" "appservice" {
  name                = "azapp-appservice-test-001"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  app_service_plan_id = azurerm_app_service_plan.appservice.id
 }

Until this point, it was pretty much straight forward. My plan was now, to point to a simple web site that is provided by one of my GitHub repos. So use GitHub as the deployment source for my Web App. In the Azure portal it is very simple to configure:

Reading the documentation of the resource “azurerm_app_service”, I found out that there is the section of the site_config to configure the scm_type. But I did not find an option to define the “repo_url” and “branch”. It seems to be possible to read these attributes from an existing App Service, but I did not find a way to define it. After a little bit of research, I found out that this seems to be work in progress and not finished til now.

So I found another way to implement the connection to my GitHub hosted website code. There is an option to deploy an ARM template in a Terraform script. And that’s the way how I made my challenge.

Starting with the following ARM template to define the source code for my App Service deployment:

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "siteName": {
           "type": "string",
           "defaultValue": "[concat('WebApp-', uniqueString(resourceGroup().id))]",
            "metadata": {
                "description": "The name of you Web Site."
            }
        },
        "location": {
            "type": "string",
            "defaultValue": "[resourceGroup().location]",
            "metadata": {
                "description": "Location for all resources."
            }
        },
        "repoURL": {
            "type": "string",
            "defaultValue": "https://github.com/nielsophey/simplewebsite",
            "metadata": {
                "description": "The URL for the GitHub repository that contains the project to deploy."
            }
        },
        "branch": {
            "type": "string",
            "defaultValue": "master",
            "metadata": {
                "description": "The branch of the GitHub repository to use."
            }
        }
    },
    "variables": {},
    "resources": [
            {
                "type": "Microsoft.Web/sites",
                "apiVersion": "2018-02-01",
                "name": "[parameters('siteName')]",
                "location": "[parameters('location')]",
                "properties": {
                     },
                "resources": [
                    {
                        "type": "sourcecontrols",
                        "apiVersion": "2018-02-01",
                        "name": "web",
                        "location": "[parameters('location')]",
                        "dependsOn": [
                            "[resourceId('Microsoft.Web/sites', parameters('siteName'))]"
                        ],
                        "properties": {
                            "repoUrl": "[parameters('repoURL')]",
                            "branch": "[parameters('branch')]",
                            "isManualIntegration": true
                            }
                    }
                ]
            }
       ],
    "outputs": {}
}

I saved the template in the same folder as my Terraform script under appservice.json and added the following code to my Terraform script:

## Deploy the Deployment option by ARM Template
resource "azurerm_template_deployment" "appservice" {
    name                    = "arm-appservice-template"
    resource_group_name     = azurerm_resource_group.rg.name

    template_body = file("appservice.json")

    parameters = {
        "siteName" = azurerm_app_service.appservice.name
        "location" = azurerm_resource_group.rg.location
    }

    deployment_mode = "Incremental"
  
}

In the appservice.json is the definition of the GitHub repo and the needed branch. I set the two parameters as default so that I only need to define the App Service name and location for the ARM template to work probably.

So finally I got my web site up and running:

I totally agree that the chosen way is not the best way to deploy a simple App Services like this, but it was a great learning curve for me. I understand to check the issues, pull requests and release notes in GitHub for the Terraform Azure provider. As well as I learned how to read the documentation of the Terraform resources.

Also I hope, that the resource “azurerm_app_service_source_control” or something equal to that, will be available soon so that we do not need the way via ARM template anymore.

The two scripts can be found on my GitHub Repo for all these samples.

Learn Terraform – Define a virtual machine scale set

Now that we have one VM serving a web site, it is a common pattern to deploy not only one VM. Use multiple VMs to distribute the load. In Azure, this feature is called a virtual machine scale set (see the DOCs).

To build this in Terraform we need the azurerm_linux_virtual_machine_scale_set resource type. The documentation shows a sample on how to use it.

Please read first!

But CAUTION – I have done everything several times and tried a lot of possible parameters to deploy the scales set including the Apache webserver. I did not find out, why the configuration of the custom script extension does not work during the initial deployment. Only if you change the VM count after the deployment, the custom script will be deployed. You can see this issue here.

So I go through the whole sample and afterward I would like to show, how I would build the sample out of Yevgeniy Brikmann’s book by leveraging app services in Azure.

Let’s go first the way thru the virtual machine scale set:

We need a resource group to deploy everything to

### Resource Group
resource "azurerm_resource_group" "rg" {
  name     = "rg-vmssssample-test"
  location = "East US"
  tags = {
      App = "VMSS"
      Source = "Terraform"
  }
}

In this sample, we start using tags at the resource group level for the App we deployed, the source and what kind of environment we have. Also, I want to establish a naming convention based on the Microsoft best practices shared in this article.

So for a resource group, there is the suggested pattern
rg-<App or service name>-<Subscription type>-<### >

Next – the vNet

### Network
resource "azurerm_virtual_network" "vNet" {
  name                = "vnet-shared-eastus-001"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  address_space       = ["10.0.0.0/16"]
  tags = azurerm_resource_group.rg.tags
}

În the VNet we have to define the internal subnet for the VMs in the scale set

### Subnet
resource "azurerm_subnet" "sNet" {
  name                 = "snet-shared-vmsssample-001"
  resource_group_name  = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vNet.name
  address_prefix       = "10.0.2.0/24"
}

In my script, I add the following resource in front of the VM scale set definition. Because I want to add a FQDN to public IP assign to the load balancer. There is a helpful resource in Terraform to build a random String to be used for the FQDN:

### Random FQDN String
resource "random_string" "fqdn" {
 length  = 6
 special = false
 upper   = false
 number  = false
}

Implement a Loadbalancer into our script

The common design pattern is to deploy a load balancer in front of the VMs in the scale set. With this, the incoming traffic can be distributed between the virtual machines in the scale set. We add a load balancer definition to the script:

### Loadbalancer definition
resource "azurerm_lb" "vmsssample" {
 name                = "lb-vmsssample-test-001"
 location            = azurerm_resource_group.rg.location
 resource_group_name = azurerm_resource_group.rg.name

 frontend_ip_configuration {
   name                 = "ipconf-PublicIPAddress-test"
   public_ip_address_id = azurerm_public_ip.vmss-pip.id
 }

  tags = azurerm_resource_group.rg.tags
}

The load balancer needs some more configuration. We need to define a backend IP pool as well as a probe to check the health status of VMs in the backend pool:

### Define the backend pool
resource "azurerm_lb_backend_address_pool" "vmsssample" {
 resource_group_name = azurerm_resource_group.rg.name
 loadbalancer_id     = azurerm_lb.vmsssample.id
 name                = "ipconf-BackEndAddressPool-test"
}

### Define the lb probes
resource "azurerm_lb_probe" "vmsssample" {
 resource_group_name = azurerm_resource_group.rg.name
 loadbalancer_id     = azurerm_lb.vmsssample.id
 name                = "http-running-probe"
 port                = 80
}

The last step in the configuration is the rule for the load balancing – so which port should be balanced:

### Define the lb rule
resource "azurerm_lb_rule" "vmsssample" {
   resource_group_name            = azurerm_resource_group.rg.name
   loadbalancer_id                = azurerm_lb.vmsssample.id
   name                           = "http"
   protocol                       = "Tcp"
   frontend_port                  = 80
   backend_port                   = 80
   backend_address_pool_id        = azurerm_lb_backend_address_pool.vmsssample.id
   frontend_ip_configuration_name = "ipconf-PublicIPAddress-test"
   probe_id                       = azurerm_lb_probe.vmsssample.id
}

Now we have deployed the basic components of our architecture and can go ahead. As in our sample for a single VM it is important to define the network security group. But we do not need the SSH port been opened, we just need the port 80 on our webserver.

### Define the NSG
resource "azurerm_network_security_group" "vmsssample" {
    name                = "nsg-weballow-001"
    location            = azurerm_resource_group.rg.location
    resource_group_name = azurerm_resource_group.rg.name
    
     security_rule {
        name                       = "WebServer"
        priority                   = 1002
        direction                  = "Inbound"
        access                     = "Allow"
        protocol                   = "Tcp"
        source_port_range          = "*"
        destination_port_range     = "80"
        source_address_prefix      = "*"
        destination_address_prefix = "*"
     }
}

Finally, the VM scale set itself

### The VM Scale Set (VMSS)
resource "azurerm_linux_virtual_machine_scale_set" "vmsssample" {
  name                = "vmss-vmsssample-test-001"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  sku                 = "Standard_B2s"
  instances           = 1
  admin_username      = "adminuser"
  admin_password      = "Password1234!"
  disable_password_authentication = false
  tags = azurerm_resource_group.rg.tags

#### define the os image
 source_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "16.04-LTS"
    version   = "latest"
  }

#### define the os disk
  os_disk {
    storage_account_type = "Standard_LRS"
    caching              = "ReadWrite"
  }

#### Define Network
  network_interface {
      name    = "nic-01-vmsssample-test-001"
      primary = true

    ip_configuration {
        name      = "ipconf-vmssample-test"
        primary   = true
        subnet_id = azurerm_subnet.sNet.id
        load_balancer_backend_address_pool_ids = [azurerm_lb_backend_address_pool.vmsssample.id]
    }
    network_security_group_id = azurerm_network_security_group.vmsssample.id

  }
}

Now we can plan our script and apply it to our Azure Account. Now that we have out VM scale set up and running we need our Webserver in the machine again. To achieve this, we need to deploy a new resource – the “azurerm_virtual_machine_scale_setextension”. It is somehow the same kind of extension we used for the single VM – so our additional entry in the script will look like this:

### Add the Webserver to the VMSS
resource "azurerm_virtual_machine_scale_set_extension" "vmsssampleextension" {
  name                         = "ext-vmsssample-test"
  virtual_machine_scale_set_id = azurerm_linux_virtual_machine_scale_set.vmsssample.id
  publisher                    = "Microsoft.Azure.Extensions"
  type                         = "CustomScript"
  type_handler_version         = "2.0"
  auto_upgrade_minor_version   = true
  force_update_tag             = true
  
  settings = jsonencode({
      "commandToExecute" : "apt-get -y update && apt-get install -y apache2" 
    })
}

During my research on the web I found that with terraform version 0.12 the function jsoncode has been implemented. With this, it is easier to convert a given string to JSON. I used this function for the commandToExcecute attribute.

But

If we now deploy our script to azure we will have all components in place to have a virtual machine scale set with a web server installed. As mentioned at the beginning the custom script extension does not work as expected. If you go to the portal and change the number of deployed instances in the scaling option of the scale set, the custom script extensions will be deployed to the VMs. If we then browse to URL of the public IP – we will have the apache web server default website been presented.

VMSS scaling Option

So after scaling up – our script will show our desired state when browsing to the FQDN.

The next post will then show the deployment using Azure App Services to solve the same challenge and add a real website to that script.

Learn Terraform – How can we make the Linux VM become a Web Server

The next iteration of the VM is to configure a Web Server running on the VM and add an auto-scaling function as well as a load balancer. Due to the point, that I’m not so aware of Linux, I took a little bit different approach to have a Web Server running on the VM. Yevgeniy uses in his book the following “user_data” option to have a web site been served by our VM.

user_date = <<-EOF
            #!/bin/bash
            echo "Hellom, World" > index.html
            nohup busybox httpd -f -p 8080 &
            EOF

I tried to get this as a script running in the VM just deployed. But I did not find out what will be the best way. So maybe this is a challenge for later, but take it the other way around, what is the normal Way in Azure to get something running in a VM just deployed. I normally use the custom script extensions to run a command in a machine. Especially in a Windows VM I would use any desired state configuration with this option. If you want to learn more about custom script extension focusing on a Linux VMs visit this DOCs article.

With this knowledge we now can add a section in our script to deploy a custom script extension:

resource "azurerm_virtual_machine_extension" "myFirstTerraform" {
  name = "myFirstTerraform-Script"
  virtual_machine_id = azurerm_linux_virtual_machine.myFirstTerraform.id
  publisher = "Microsoft.Azure.Extensions"
  type = "CustomScript"
  type_handler_version ="2.0"

  settings = <<SETTINGS
    {
      "commandToExecute" : "apt-get -y update && apt-get install -y apache2" 
    }
    SETTINGS
 }

The important configuration is made in the settings section. I added the command to install an apache web server on the machine and then we will have the standard website been served in port 80 on the Linux VM. The only trouble we get is, our network security group (NSG) we deployed, was only opening the ssh port. So we must add an additional rule in the NSG. So our NSG will look like this:

 resource "azurerm_network_security_group" "myFirstTerraform" {
    name                = "myFirstTerraform"
    location            = azurerm_resource_group.myFirstTerraform.location
    resource_group_name = azurerm_resource_group.myFirstTerraform.name

    security_rule {
        name                       = "SSH"
        priority                   = 1001
        direction                  = "Inbound"
        access                     = "Allow"
        protocol                   = "Tcp"
        source_port_range          = "*"
        destination_port_range     = "22"
        source_address_prefix      = "*"
        destination_address_prefix = "*"
     }

     security_rule {
        name                       = "WebServer"
        priority                   = 1002
        direction                  = "Inbound"
        access                     = "Allow"
        protocol                   = "Tcp"
        source_port_range          = "*"
        destination_port_range     = "80"
        source_address_prefix      = "*"
        destination_address_prefix = "*"
     }
}

If we now would run our script, we will be able to see the default apache web site on our Linux VM running in Azure:

Default apache website

To connect to this website it would be great to know on which public IP assigned to our Linux VM. As we learn in the book, we can use the output variables to achieve this. But there is one important difference. In Azure, a public IP is a resource on his own and will be attached to a network interface that then will be assigned to a VM. So we need to reference the IP in our output and not the VM.

What does that mean for our script:

output "public_ip" {
   value       = azurerm_public_ip.myFirstTerraform.ip_address
   description = "This is the asigned public ip to our VM"
}

If we have added this output to our script we can afterwards just get the ip after you apply your script again:

$ terraform apply

Outputs:

public_ip = 51.136.162.193

If you need the output of your latest terrafrom deployment again you just can call:

$ terraform output public_ip

51.136.162.193

So now you know the ip to browse to, if you want to see your apache 2 default webiste.

Learn Terraform – deploy the first VM

After terraform is installed on the computer you use to deploy your first cloud resources – the initial sample in chapter 2 is to deploy a VM. A Linux based VM. You can find all the samples out of the book from Yevgeniy under this repro in GitHub.

So how do we do this in azure?

Deploy your first VM in Azure

One main difference to aws is that in Azure we deploy resources always in a resource group (RG) – so the script in Terraform is not so easy as in aws.

    resource "azurerm_resource_group" "RG" {
        name = "myFirstTerraform-RG"
        location = "westeurope"
}

So having the resource group deployed we can use it to put the VM in there. that’s not all we need to deploy a VM in Azure. Let’s have a look at the documentation of the Azure provider at HashiCorp for the “azurerm_virtual_machine”. And as you read the introduction you see, there is another resource type in the provider to directly focus on a Linux VM – “azurerm_linux_virtual_machine”.

Going thru the documentation we will find, that we need the following CONFIG arguments for a Linux VM:

  • admin_username
  • location
  • name
  • network_interface_ids
  • os_disk
  • resource_group_name
  • size

and for sure one of these:

  • admin_password
    (if this option is used – be aware that _disable_passwordauthentication must be set to false)
  • admin_ssh_key

So if we compare this with a simple example in chapter 2 of the book – Azure seems to be more complex. For me it looks like that in aws you can a template and aws will add every configuration you needed with some default values (see page 43). So the Azure Resource Manager needs the configuration defined by the admin or better the script.

So let’s get back to our script.

We already have now the RG in it. Now we need to deploy a network:

resource "azurerm_virtual_network" "vnet" {
  name                = "myFirstVnet"
  address_space       = ["10.0.0.0/16"]
  location            = "myFirstTerraform-RG"
  resource_group_name = "westeurope"
}

In the Book, the concept of “DRY” is explained later, but I think it is good to think about it now. “DRY” means “don’t repeat yourself”. It would be no good idea to reference the location and the ressource_group_name by filling in the values defined in the script before. It is better to point to the values.

_{ressource}.{name}.{config}_

If we want to reference the name of the RG, we have to use:

azurerm_resource_group.myFirstTerraform-RG.name

and the same with the location used in the definition of the RG:

azurerm_resource_group.myFirstTerraform-RG.location

Taking this into the code leads to the following

resource "azurerm_virtual_network" "vnet" {
  name                = "myFirstVnet"
  address_space       = ["10.0.0.0/16"]
  location            = azurerm_resource_group.myFirstTerraform-RG.name
  resource_group_name = azurerm_resource_group.myFirstTerraform-RG.location
}

What is important to understand that terraform is now able to identify dependencies between the resources and can handle the order of deploying the needed resources (see Chapter 2 in the book page 51). The next steps in your terraform script would now to add all needed Resources including references as well as the VM itself.

The whole script would look like this:

## Deploy your first VM to Azure
## based on an Ubunto Image

## Define the provider
provider "azurerm" {
  version = "~>2.0.0"
  features {}
}

## define the resource gruop
resource "azurerm_resource_group" "myFirstTerraform" {
  name     = "myFirstTerraform-RG"
  location = "West Europe"
}

## define the network
resource "azurerm_virtual_network" "myFirstTerraform" {
  name                = "myFirstTerraform-vNet"
  address_space       = ["10.0.0.0/16"]
  location            = azurerm_resource_group.myFirstTerraform.location
  resource_group_name = azurerm_resource_group.myFirstTerraform.name
}

## define the subnet
resource "azurerm_subnet" "myFirstTerraform" {
  name                 = "internal"
  resource_group_name  = azurerm_resource_group.myFirstTerraform.name
  virtual_network_name = azurerm_virtual_network.myFirstTerraform.name
  address_prefix       = "10.0.2.0/24"
}

## define a public ip for the vm to ssh in
resource "azurerm_public_ip" "myFirstTerraform" {
  name                = "myFirstTerraform-pip"
  location            = azurerm_resource_group.myFirstTerraform.location
  resource_group_name = azurerm_resource_group.myFirstTerraform.name
  allocation_method   = "Dynamic"
}

## define the networkinterface
resource "azurerm_network_interface" "myFirstTerraform" {
  name                = "myFirstTerraform-nic"
  location            = azurerm_resource_group.myFirstTerraform.location
  resource_group_name = azurerm_resource_group.myFirstTerraform.name

  ip_configuration {
    name                          = "internal"
    subnet_id                     = azurerm_subnet.myFirstTerraform.id
    private_ip_address_allocation = "Dynamic"
    public_ip_address_id          = azurerm_public_ip.myFirstTerraform.id
  }
}

## define the VM
resource "azurerm_linux_virtual_machine" "myFirstTerraform" {
  name                = "myFirstTerraform-vm"
  resource_group_name = azurerm_resource_group.myFirstTerraform.name
  location            = azurerm_resource_group.myFirstTerraform.location
  size                = "Standard_B2s"
  computer_name = "myFirstLinuxVM"
  admin_username = "adminuser"
  admin_password = "Password1234!"
  disable_password_authentication = false
  network_interface_ids = [
    azurerm_network_interface.myFirstTerraform.id,
  ]

    os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Standard_LRS"
  }

  source_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "16.04-LTS"
    version   = "latest"
  }
}

CAUTION We do not have defined any firewall in front of the VM. In azure we need e.g. a network security group for this, so one more component should be added to your script and should be applied to the network înterface.

To define a network security group in the script:

resource "azurerm_network_security_group" "myFirstTerrafrom" {
    name                = "myFirstTerrafrom-NSG"
    location            = azurerm_resource_group.myFirstTerraform.location
    resource_group_name = azurerm_resource_group.myFirstTerraform.name

    security_rule {
        name                       = "SSH"
        priority                   = 1001
        direction                  = "Inbound"
        access                     = "Allow"
        protocol                   = "Tcp"
        source_port_range          = "*"
        destination_port_range     = "22"
        source_address_prefix      = "*"
        destination_address_prefix = "*"
    }
}

and we need the assignment to the network interface:

resource "azurerm_network_interface_security_group_association" "myFirstTerraform" {
    network_interface_id      = azurerm_network_interface.myFirstTerraform.id
    network_security_group_id = azurerm_network_security_group.myFirstTerraform.id
}

As soon as I have finished the whole chapter 2 – I will provide a GitHub Repro with all the information.