Azure – New Project Bicep

What is Bicep? Bicep is a Domain Specific Language (DSL) for deploying Azure resources declaratively. So, get out of ARM Templates and use a more declarative way to describe what you want to deploy to Azure. You can read more about the project on the GitHub page.

My first step was to deploy the tooling for Becip and the extension for Visual Studio Code. You find the installing instruction also in the GitHub project site. As soon as you have installed everything your VS Code will have support for *.becip files:

And as shown in the screenshot you get support for some snippets to get familiar with the syntax of Bicep. Taking the sample from the GitHub project page – here is a sample to declare a storage account:

resource sa 'Microsoft.Storage/storageAccounts@2019-06-01' = {
    name: 'satest12345no' // must be globally unique
    location: 'westeuope'
    kind: 'Storage'
    sku: {
        name: 'Standard_LRS'
    }
}

This looks very structured for me and as soon as you will build this *.bicep file – you will get an ARM template like this:

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {},
  "functions": [],
  "variables": {},
  "resources": [
    {
      "type": "Microsoft.Storage/storageAccounts",
      "apiVersion": "2019-06-01",
      "name": "satest12345no",
      "location": "eastwesteuopeus",
      "kind": "Storage",
      "sku": {
        "name": "Standard_LRS"
      }
    }
  ],
  "outputs": {}
}

What we get at the end, is a deployable ARM template which can be used to deploy the resources in Azure via the Azure Resource Manager. You can follow these steps on the GitHub page. I encourage to do it and learn more about the first release of this project.

In my point of view, it is a great start! I hope the project will go on fast and we will get a new option to build our infrastructure in Azure based on Bicep. At the moment it is much too early to build something based on Bicep in production, but if you look at the ideas (e.g. use of expressions) it looks like we will get a powerful tool!

What will be an interesting point to see, is how or even if Bicep will handle the current state of your environment in Azure. The “AzOps” approach out of the enterprise-scale landing zone would be interesting for this.

And for those of you, who would not like to install the “alpha” version on their machine – use the provided playground, to see Biceps in action.

So, stay tuned to the GitHub page – I will!

Azure – Link Collection Cloud Adoption Framework for Azure

The Cloud Adoption Framework is the One Microsoft approach to cloud adoption in Azure, consolidating and sharing best practices from Microsoft employees, partners, and customers. The framework gives customers a set of tools, guidance, and narratives that help shape technology, business, and people strategies for driving desired business outcomes during their adoption effort.

Here is a short collection of links during a CAF related engagement:

Start with the CAF documentation in the Microsoft Docs.

CAF – Define Strategy, Ready and Plan

Find additional StrategyPlan and Ready documentation.

StrategyTools Strategy

Cloud Journey Tracker
Business Outcome Template
PlanTools Plan

Azure DevOps Demo Generator Template
Cloud adoption plan template
ReadyTools Ready

Cloud Adoption Framework – Azure Setup Guide
Cloud Adoption Framework Foundation Blueprint
Migration Landing Zone Blueprint
Naming and tagging tracking template

CAF – Adopt

Find additional Migrate and Innovate documentation.

AdoptTools Adopt

Strategic migration assessment and readiness tool (SMART)
Cloud Adoption Framework – Azure Migration Guide
Azure innovation guide

CAF -Govern and Manage

Find additional Govern and Manage documentation.

GovernTools Govern

Governance benchmarkGovernance process template
Cost Management process template
Deployment acceleration process template
Identity process template
Resource consistency process template
Security baseline process template
ManageTools Management

Microsoft Azure Well-Architected Review
Best practices source code
Operations management workbook

Links for Partner

Learn Terraform – Deploy an App Service instead of a scale set

As mentioned in my post before, it is no so easy as a beginner to get everything realized in Terraform. The challenge was, deploy to a web site in Azure which is able to scale out behind a load balancer. After demonstrating the way be using virtual machine scale sets, I would like to show the way I found with Azure App Services as the service to go to.

If you take a look at the simple sample in the documentation, you see that it is very easy to deploy a simple website in azure. Out of this, my idea was, it could not be so complicated in Terraform to achieve the same.

So let’s get started with our script and define the provider and resource group:

## Lets start with an app service in Azure
provider "azurerm" {
    version = "~>2.3.0"
    features {}
}

### Resource Group
resource "azurerm_resource_group" "rg" {
  name     = "rg-appservice-test"
  location = "West Europe"
  tags = {
      App = "appservice"
      Source = "Terraform"
  }
}

That’s pretty much the same, as in the other posts. The only thing I changed was the version of the provider. You can always check the latest version in the GitHub repo releases section for the provider.

For an App Service, we need an “App Service Plan“.

### App Service plan
resource "azurerm_app_service_plan" "appservice" {
  name                = "azapp-plan-eastus-001"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name

  sku {
    tier = "Standard"
    size = "S1"
  }
}

Which SKU to choose depends on the features needed. You can check the possible options in the DOCs. As soon as we have an App Service Plan, we can define our App Service:

## The App Service 
resource "azurerm_app_service" "appservice" {
  name                = "azapp-appservice-test-001"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  app_service_plan_id = azurerm_app_service_plan.appservice.id
 }

Until this point, it was pretty much straight forward. My plan was now, to point to a simple web site that is provided by one of my GitHub repos. So use GitHub as the deployment source for my Web App. In the Azure portal it is very simple to configure:

Reading the documentation of the resource “azurerm_app_service”, I found out that there is the section of the site_config to configure the scm_type. But I did not find an option to define the “repo_url” and “branch”. It seems to be possible to read these attributes from an existing App Service, but I did not find a way to define it. After a little bit of research, I found out that this seems to be work in progress and not finished til now.

So I found another way to implement the connection to my GitHub hosted website code. There is an option to deploy an ARM template in a Terraform script. And that’s the way how I made my challenge.

Starting with the following ARM template to define the source code for my App Service deployment:

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "siteName": {
           "type": "string",
           "defaultValue": "[concat('WebApp-', uniqueString(resourceGroup().id))]",
            "metadata": {
                "description": "The name of you Web Site."
            }
        },
        "location": {
            "type": "string",
            "defaultValue": "[resourceGroup().location]",
            "metadata": {
                "description": "Location for all resources."
            }
        },
        "repoURL": {
            "type": "string",
            "defaultValue": "https://github.com/nielsophey/simplewebsite",
            "metadata": {
                "description": "The URL for the GitHub repository that contains the project to deploy."
            }
        },
        "branch": {
            "type": "string",
            "defaultValue": "master",
            "metadata": {
                "description": "The branch of the GitHub repository to use."
            }
        }
    },
    "variables": {},
    "resources": [
            {
                "type": "Microsoft.Web/sites",
                "apiVersion": "2018-02-01",
                "name": "[parameters('siteName')]",
                "location": "[parameters('location')]",
                "properties": {
                     },
                "resources": [
                    {
                        "type": "sourcecontrols",
                        "apiVersion": "2018-02-01",
                        "name": "web",
                        "location": "[parameters('location')]",
                        "dependsOn": [
                            "[resourceId('Microsoft.Web/sites', parameters('siteName'))]"
                        ],
                        "properties": {
                            "repoUrl": "[parameters('repoURL')]",
                            "branch": "[parameters('branch')]",
                            "isManualIntegration": true
                            }
                    }
                ]
            }
       ],
    "outputs": {}
}

I saved the template in the same folder as my Terraform script under appservice.json and added the following code to my Terraform script:

## Deploy the Deployment option by ARM Template
resource "azurerm_template_deployment" "appservice" {
    name                    = "arm-appservice-template"
    resource_group_name     = azurerm_resource_group.rg.name

    template_body = file("appservice.json")

    parameters = {
        "siteName" = azurerm_app_service.appservice.name
        "location" = azurerm_resource_group.rg.location
    }

    deployment_mode = "Incremental"
  
}

In the appservice.json is the definition of the GitHub repo and the needed branch. I set the two parameters as default so that I only need to define the App Service name and location for the ARM template to work probably.

So finally I got my web site up and running:

I totally agree that the chosen way is not the best way to deploy a simple App Services like this, but it was a great learning curve for me. I understand to check the issues, pull requests and release notes in GitHub for the Terraform Azure provider. As well as I learned how to read the documentation of the Terraform resources.

Also I hope, that the resource “azurerm_app_service_source_control” or something equal to that, will be available soon so that we do not need the way via ARM template anymore.

The two scripts can be found on my GitHub Repo for all these samples.

Learn Terraform – Define a virtual machine scale set

Now that we have one VM serving a web site, it is a common pattern to deploy not only one VM. Use multiple VMs to distribute the load. In Azure, this feature is called a virtual machine scale set (see the DOCs).

To build this in Terraform we need the azurerm_linux_virtual_machine_scale_set resource type. The documentation shows a sample on how to use it.

Please read first!

But CAUTION – I have done everything several times and tried a lot of possible parameters to deploy the scales set including the Apache webserver. I did not find out, why the configuration of the custom script extension does not work during the initial deployment. Only if you change the VM count after the deployment, the custom script will be deployed. You can see this issue here.

So I go through the whole sample and afterward I would like to show, how I would build the sample out of Yevgeniy Brikmann’s book by leveraging app services in Azure.

Let’s go first the way thru the virtual machine scale set:

We need a resource group to deploy everything to

### Resource Group
resource "azurerm_resource_group" "rg" {
  name     = "rg-vmssssample-test"
  location = "East US"
  tags = {
      App = "VMSS"
      Source = "Terraform"
  }
}

In this sample, we start using tags at the resource group level for the App we deployed, the source and what kind of environment we have. Also, I want to establish a naming convention based on the Microsoft best practices shared in this article.

So for a resource group, there is the suggested pattern
rg-<App or service name>-<Subscription type>-<### >

Next – the vNet

### Network
resource "azurerm_virtual_network" "vNet" {
  name                = "vnet-shared-eastus-001"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  address_space       = ["10.0.0.0/16"]
  tags = azurerm_resource_group.rg.tags
}

În the VNet we have to define the internal subnet for the VMs in the scale set

### Subnet
resource "azurerm_subnet" "sNet" {
  name                 = "snet-shared-vmsssample-001"
  resource_group_name  = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vNet.name
  address_prefix       = "10.0.2.0/24"
}

In my script, I add the following resource in front of the VM scale set definition. Because I want to add a FQDN to public IP assign to the load balancer. There is a helpful resource in Terraform to build a random String to be used for the FQDN:

### Random FQDN String
resource "random_string" "fqdn" {
 length  = 6
 special = false
 upper   = false
 number  = false
}

Implement a Loadbalancer into our script

The common design pattern is to deploy a load balancer in front of the VMs in the scale set. With this, the incoming traffic can be distributed between the virtual machines in the scale set. We add a load balancer definition to the script:

### Loadbalancer definition
resource "azurerm_lb" "vmsssample" {
 name                = "lb-vmsssample-test-001"
 location            = azurerm_resource_group.rg.location
 resource_group_name = azurerm_resource_group.rg.name

 frontend_ip_configuration {
   name                 = "ipconf-PublicIPAddress-test"
   public_ip_address_id = azurerm_public_ip.vmss-pip.id
 }

  tags = azurerm_resource_group.rg.tags
}

The load balancer needs some more configuration. We need to define a backend IP pool as well as a probe to check the health status of VMs in the backend pool:

### Define the backend pool
resource "azurerm_lb_backend_address_pool" "vmsssample" {
 resource_group_name = azurerm_resource_group.rg.name
 loadbalancer_id     = azurerm_lb.vmsssample.id
 name                = "ipconf-BackEndAddressPool-test"
}

### Define the lb probes
resource "azurerm_lb_probe" "vmsssample" {
 resource_group_name = azurerm_resource_group.rg.name
 loadbalancer_id     = azurerm_lb.vmsssample.id
 name                = "http-running-probe"
 port                = 80
}

The last step in the configuration is the rule for the load balancing – so which port should be balanced:

### Define the lb rule
resource "azurerm_lb_rule" "vmsssample" {
   resource_group_name            = azurerm_resource_group.rg.name
   loadbalancer_id                = azurerm_lb.vmsssample.id
   name                           = "http"
   protocol                       = "Tcp"
   frontend_port                  = 80
   backend_port                   = 80
   backend_address_pool_id        = azurerm_lb_backend_address_pool.vmsssample.id
   frontend_ip_configuration_name = "ipconf-PublicIPAddress-test"
   probe_id                       = azurerm_lb_probe.vmsssample.id
}

Now we have deployed the basic components of our architecture and can go ahead. As in our sample for a single VM it is important to define the network security group. But we do not need the SSH port been opened, we just need the port 80 on our webserver.

### Define the NSG
resource "azurerm_network_security_group" "vmsssample" {
    name                = "nsg-weballow-001"
    location            = azurerm_resource_group.rg.location
    resource_group_name = azurerm_resource_group.rg.name
    
     security_rule {
        name                       = "WebServer"
        priority                   = 1002
        direction                  = "Inbound"
        access                     = "Allow"
        protocol                   = "Tcp"
        source_port_range          = "*"
        destination_port_range     = "80"
        source_address_prefix      = "*"
        destination_address_prefix = "*"
     }
}

Finally, the VM scale set itself

### The VM Scale Set (VMSS)
resource "azurerm_linux_virtual_machine_scale_set" "vmsssample" {
  name                = "vmss-vmsssample-test-001"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  sku                 = "Standard_B2s"
  instances           = 1
  admin_username      = "adminuser"
  admin_password      = "Password1234!"
  disable_password_authentication = false
  tags = azurerm_resource_group.rg.tags

#### define the os image
 source_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "16.04-LTS"
    version   = "latest"
  }

#### define the os disk
  os_disk {
    storage_account_type = "Standard_LRS"
    caching              = "ReadWrite"
  }

#### Define Network
  network_interface {
      name    = "nic-01-vmsssample-test-001"
      primary = true

    ip_configuration {
        name      = "ipconf-vmssample-test"
        primary   = true
        subnet_id = azurerm_subnet.sNet.id
        load_balancer_backend_address_pool_ids = [azurerm_lb_backend_address_pool.vmsssample.id]
    }
    network_security_group_id = azurerm_network_security_group.vmsssample.id

  }
}

Now we can plan our script and apply it to our Azure Account. Now that we have out VM scale set up and running we need our Webserver in the machine again. To achieve this, we need to deploy a new resource – the “azurerm_virtual_machine_scale_setextension”. It is somehow the same kind of extension we used for the single VM – so our additional entry in the script will look like this:

### Add the Webserver to the VMSS
resource "azurerm_virtual_machine_scale_set_extension" "vmsssampleextension" {
  name                         = "ext-vmsssample-test"
  virtual_machine_scale_set_id = azurerm_linux_virtual_machine_scale_set.vmsssample.id
  publisher                    = "Microsoft.Azure.Extensions"
  type                         = "CustomScript"
  type_handler_version         = "2.0"
  auto_upgrade_minor_version   = true
  force_update_tag             = true
  
  settings = jsonencode({
      "commandToExecute" : "apt-get -y update && apt-get install -y apache2" 
    })
}

During my research on the web I found that with terraform version 0.12 the function jsoncode has been implemented. With this, it is easier to convert a given string to JSON. I used this function for the commandToExcecute attribute.

But

If we now deploy our script to azure we will have all components in place to have a virtual machine scale set with a web server installed. As mentioned at the beginning the custom script extension does not work as expected. If you go to the portal and change the number of deployed instances in the scaling option of the scale set, the custom script extensions will be deployed to the VMs. If we then browse to URL of the public IP – we will have the apache web server default website been presented.

VMSS scaling Option

So after scaling up – our script will show our desired state when browsing to the FQDN.

The next post will then show the deployment using Azure App Services to solve the same challenge and add a real website to that script.

Learn Terraform – How can we make the Linux VM become a Web Server

The next iteration of the VM is to configure a Web Server running on the VM and add an auto-scaling function as well as a load balancer. Due to the point, that I’m not so aware of Linux, I took a little bit different approach to have a Web Server running on the VM. Yevgeniy uses in his book the following “user_data” option to have a web site been served by our VM.

user_date = <<-EOF
            #!/bin/bash
            echo "Hellom, World" > index.html
            nohup busybox httpd -f -p 8080 &
            EOF

I tried to get this as a script running in the VM just deployed. But I did not find out what will be the best way. So maybe this is a challenge for later, but take it the other way around, what is the normal Way in Azure to get something running in a VM just deployed. I normally use the custom script extensions to run a command in a machine. Especially in a Windows VM I would use any desired state configuration with this option. If you want to learn more about custom script extension focusing on a Linux VMs visit this DOCs article.

With this knowledge we now can add a section in our script to deploy a custom script extension:

resource "azurerm_virtual_machine_extension" "myFirstTerraform" {
  name = "myFirstTerraform-Script"
  virtual_machine_id = azurerm_linux_virtual_machine.myFirstTerraform.id
  publisher = "Microsoft.Azure.Extensions"
  type = "CustomScript"
  type_handler_version ="2.0"

  settings = <<SETTINGS
    {
      "commandToExecute" : "apt-get -y update && apt-get install -y apache2" 
    }
    SETTINGS
 }

The important configuration is made in the settings section. I added the command to install an apache web server on the machine and then we will have the standard website been served in port 80 on the Linux VM. The only trouble we get is, our network security group (NSG) we deployed, was only opening the ssh port. So we must add an additional rule in the NSG. So our NSG will look like this:

 resource "azurerm_network_security_group" "myFirstTerraform" {
    name                = "myFirstTerraform"
    location            = azurerm_resource_group.myFirstTerraform.location
    resource_group_name = azurerm_resource_group.myFirstTerraform.name

    security_rule {
        name                       = "SSH"
        priority                   = 1001
        direction                  = "Inbound"
        access                     = "Allow"
        protocol                   = "Tcp"
        source_port_range          = "*"
        destination_port_range     = "22"
        source_address_prefix      = "*"
        destination_address_prefix = "*"
     }

     security_rule {
        name                       = "WebServer"
        priority                   = 1002
        direction                  = "Inbound"
        access                     = "Allow"
        protocol                   = "Tcp"
        source_port_range          = "*"
        destination_port_range     = "80"
        source_address_prefix      = "*"
        destination_address_prefix = "*"
     }
}

If we now would run our script, we will be able to see the default apache web site on our Linux VM running in Azure:

Default apache website

To connect to this website it would be great to know on which public IP assigned to our Linux VM. As we learn in the book, we can use the output variables to achieve this. But there is one important difference. In Azure, a public IP is a resource on his own and will be attached to a network interface that then will be assigned to a VM. So we need to reference the IP in our output and not the VM.

What does that mean for our script:

output "public_ip" {
   value       = azurerm_public_ip.myFirstTerraform.ip_address
   description = "This is the asigned public ip to our VM"
}

If we have added this output to our script we can afterwards just get the ip after you apply your script again:

$ terraform apply

Outputs:

public_ip = 51.136.162.193

If you need the output of your latest terrafrom deployment again you just can call:

$ terraform output public_ip

51.136.162.193

So now you know the ip to browse to, if you want to see your apache 2 default webiste.