How to Deploy a Web Application to a Private AKS Cluster With Azure DevOps

azure azure devops ci/cd kubernetes Oct 19, 2022

In a previous article, I shared how I was able to create a private AKS cluster with Terraform.

Today, we’ll use the Terraform script from the previous article to create the environment and we’ll deploy a web application to the cluster with Azure DevOps. To understand how to create the infrastructure, follow the steps from the previous article.

As a reminder, in a private AKS cluster the control plane has a private IP address and is not reacheable through internet. Because of this, we’ll use an Azure DevOps self-hosted agent. This agent has to be in a VNET that has access to the cluster.

This type of deployment where we push the configuration to AKS is called a push-based deployment. As opposed to a pull-based deployment where AKS is able to pull the configuration (check the article on ArgoCD). Today, we are focusing on push.

Below are 2 diagrams:

The first one describes the infrastructure. I circled in red the virtual machine that we’ll use as an agent. This VM is in the vnet-hub which is peered to the vnet-aks. Because of this peering, our VM will be able to contact the control plane to do the deployment.
The second one represents the actual CI/CD process with the different steps.

Prepare the virtual machine

First thing, we need an Azure DevOps account. If you do not already have one, go to dev.azure.com and create an account. With the Basic Plan, it is free for the first 5 users.

Then, to be able to run Azure DevOps pipelines on our virtual machine, we’ll have to register it as a self-hosted agent.

1. Create a personal access token (PAT)

A personal access token allows you to give permissions to the VM in Azure DevOps.

Go to Azure DevOps.
Open your user settings and select Personal access tokens.
Create a new token with scope Agent Pools (read, manage).
Once your PAT is created, save it somewhere safe.

2. Create a new agent pool

To do so:

Go to Azure DevOps > Organization Settings > Agent pools
Click on the “Add pool” button.
Pool type: self-hosted, Name: aks-private.
Click on the “Create” button.

Now that the agent pool is created, click on it then click on the “New agent” button. In the popup, copy the URL to download the agent as shown on the picture below.

Next, go to the Azure portal to connect to the VM via the bastion.

With Bastion, a remote desktop will start directly in your browser. Open Edge (follow the basic setup) and paste the URL that we copied earlier. Once the download is completed, open PowerShell and run the following script.

cd C:\
mkdir agent ; cd agent
Add-Type -AssemblyName System.IO.Compression.FileSystem ; [System.IO.Compression.ZipFile]::ExtractToDirectory("$HOME\Downloads\vsts-agent-win-x64-2.210.1.zip", "$PWD")
.\config.cmd

You’ll be asked the following:

Server URL. It will be something like this: https://dev.azure.com/{your project name}.
PAT. Copy your previously created PAT.
The agent pool name: aks-private.
The agent name. Leave the default vm-1.
The work folder. Leave the default.
If you want to run the agent as service. Choose Yes.
The rest, choose the default options.

Once the installation is done, go back to the agent pool details on Azure DevOps. You should see your new agent online in the agents tab.

3. Install Azure CLI

We’ll use az cli in our pipeline. We then have to install it on the agent.

From vm-1,

Open Edge.
Navigate to https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-windows?tabs=azure-cli and click on the button to download the latest release of Azure CLI
Run the installer

4. Install kubectl

We’ll use kubectl to communicate with AKS.

From vm-1,

Navigate to https://kubernetes.io/docs/tasks/tools/install-kubectl-windows/
Download the latest release of kubectl.
Add the binary to the system environment variable PATH.

Create a pipeline to deploy a simple web application

Now, we’ll create an Azure pipeline to deploy a web application. We’ll use a simple Asp.Net Core web application. The code is available on github.

1. ACR build tasks

To build our docker image, we’ll use Azure Container Registry tasks. This is convenient as we won’t have to install Docker on our self-hosted agent. As the Azure Container Registry is private, those tasks have to run from our VNET. Azure offers the possibility to run ACR tasks on a dedicated agent in our VNET. This agent will be managed by Azure inside our VNET. We just have to enable this feature on the ACR.

Add the following script to our existing Terraform script to activate a dedicated agent for our ACR in the VNET vnet-hub and subnet snet-global.

Run terraform apply.

# acr.tf
resource "azurerm_container_registry_agent_pool" "agentpool" {
  name                      = "myagentpool"
  resource_group_name       = azurerm_resource_group.rg.name
  location                  = azurerm_resource_group.rg.location
  container_registry_name   = azurerm_container_registry.acr.name
  virtual_network_subnet_id = azurerm_subnet.global.id
}

2. Connect to Azure from Azure DevOps

Open Azure DevOps, navigate to your project > Project Settings > Service Connections > Create service connection. Choose Azure Resource Manager. Service Principal (automatic). Then, select your Azure subscription. For the service connection name, enter azure. Then save.

3. Create the pipeline

In the pipeline, we’ll use a task called ReplaceTokens. Install it from Visual Studio marketplace. It is free. https://marketplace.visualstudio.com/items?itemName=qetza.replacetokens

Then, fork the repository on GitHub where the simple webapp is.

Once this is done, go to Azure DevOps > Select your project > Pipelines > New pipeline.

Choose GitHub. Follow the steps to authorize Azure DevOps to access to your GitHub. Choose the repository simple-webapp. Select Existing Azure Pipelines YAML file.

In the repository, there is a file named azure-pipelines.yml.

Save and run the pipeline. It will deploy the simple-webapp to the private AKS cluster.

4. Pipeline explanation

First, we define some variables that we’ll use later.

The name of the Azure Container Registry where the Docker image will be published.
The name of our AKS cluster.
The name of the resource group where the cluster is deployed.
The name of the Azure service connection that we created earlier.
The name of the repository of our app. We’ll use it also for the name of our artifact.

variables:
  acrName: acrpvakscac
  aksClusterName: aks-pvaks-cac-001
  azureResourceGroup: rg-pvaks-cac
  azureSubscriptionEndpoint: azure
  repository: simple-webapp

After declaring the variables, we add 2 stages:

1 to build the Docker image and publish it to ACR.
1 to deploy to AKS with kubectl.

Build stage

The Docker file contains all the information to build our app so we just use az cli to build the image. Remember, ACR will use the dedicated agent created earlier which is in our VNET. This is why we pass the argument —agent-pool with the name of the agent pool: myagentpool to the command.

Next, we copy the deployment.yml file to the artifact directory and publish the artifact. We’ll use that file in the next stage.

Deploy stage

Here, we first download the artifact.

Then, we do a replace tokens on the deployment.yml file. In the file, there is a token __Build.BuildId__ that will be replaced with the actual build ID. The variable Build.BuildId is a predefined variable. We need to do this replacement because we used this value in the build stage to tag our Docker image.

Finally, we execute kubectl to deploy the app to AKS.

## insert variables before

stages:
  - stage: Build
    jobs:
      - job: Build
        pool: aks-private
        steps:
          - task: AzureCLI@2
            displayName: Build with ACR
            inputs:
              azureSubscription: $(azureSubscriptionEndpoint)
              scriptType: ps
              scriptLocation: inlineScript
              inlineScript: |
                az acr login --name $(acrName)
                az acr build --agent-pool myagentpool --image $(repository):$(Build.BuildId) --registry $(acrName) --file $(Build.Repository.LocalPath)/Dockerfile $(Build.Repository.LocalPath)

          - task: CopyFiles@2
            inputs:
              sourceFolder: $(Build.Repository.LocalPath)
              contents: "deployment.yml"
              targetFolder: $(Build.ArtifactStagingDirectory)
              overwrite: true

          - publish: $(Build.ArtifactStagingDirectory)
            artifact: $(repository)

  - stage: Deploy
    jobs:
      - job: Deploy
        pool: aks-private
        steps:
          - task: DownloadBuildArtifacts@0
            inputs:
              artifactName: $(repository)

          - task: qetza.replacetokens.replacetokens-task.replacetokens@3
            displayName: "Replace tokens"
            inputs:
              rootDirectory: $(Pipeline.Workspace)\$(repository)
              targetFiles: |
                deployment.yml
              encoding: auto
              writeBOM: true
              escapeType: no escaping
              tokenPrefix: __
              tokenSuffix: __

          - task: Kubernetes@1
            displayName: "kubectl"
            inputs:
              connectionType: Azure Resource Manager
              azureSubscriptionEndpoint: $(azureSubscriptionEndpoint)
              azureResourceGroup: $(azureResourceGroup)
              kubernetesCluster: $(aksClusterName)
              command: apply
              useConfigurationFile: true
              configuration: $(Pipeline.Workspace)\$(repository)\deployment.yml

5. Deployment.yml file

Here is the full deployment.yml file that we use to deploy the app to AKS. Note the __Build.BuildId__ token in the container image name and the hostname simple-webapp.mydomain.com in the ingress.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: simple-webapp
spec:
  replicas: 2
  selector:
    matchLabels:
      app: simple-webapp
  template:
    metadata:
      labels:
        app: simple-webapp
    spec:
      nodeSelector:
        "kubernetes.io/os": linux
      containers:
        - name: simple-webapp
          image: acrpvakscac.azurecr.io/simple-webapp:__Build.BuildId__
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: 250m
            limits:
              cpu: 500m
---
apiVersion: v1
kind: Service
metadata:
  name: simple-webapp
spec:
  type: ClusterIP
  ports:
    - port: 80
  selector:
    app: simple-webapp
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: simple-webapp
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/use-private-ip: "true"
spec:
  rules:
    - host: simple-webapp.mydomain.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: simple-webapp
                port:
                  number: 80

Configure DNS

The last thing that we have to do is to create a private DNS for the domain mydomain.com and add a A record with the name simple-webapp and the IP of the Application Gateway.

To do this, we’ll just update our Terraform script.

Add the following code to the main.tf file and run terraform apply.

resource "azurerm_private_dns_zone" "mydomain" {
  name                = "mydomain.com"
  resource_group_name = azurerm_resource_group.rg.name
}

resource "azurerm_private_dns_zone_virtual_network_link" "mydomain" {
  name                  = "pdznl-mydomain-cac-001"
  resource_group_name   = azurerm_resource_group.rg.name
  private_dns_zone_name = azurerm_private_dns_zone.mydomain.name
  virtual_network_id    = azurerm_virtual_network.vnet_hub.id
}

resource "azurerm_private_dns_a_record" "simple-webapp" {
  name                = "simple-webapp"
  zone_name           = azurerm_private_dns_zone.mydomain.name
  resource_group_name = azurerm_resource_group.rg.name
  ttl                 = 300
  records             = ["10.1.0.4"]
}

Conclusion

Now you have it. A complete CI/CD to deploy a web application to AKS.

Once the build is completed, you can navigate to the app from VM-1. The URL of the app is http://simple-webapp.mydomain.com.

There are still a few manual steps remaining that we could automate, such as configuring the VM, but perhaps we can address that in another article.

Work With Me

Ready to take your Azure solutions to the next level and streamline your DevOps processes? Let's work together! As an experienced Azure solutions architect and DevOps expert, I can help you achieve your goals. Click the button below to get in touch.

Get In Touch