How To Avoid Regressions In Web Development With Azure DevOps

- It happened again, same bug.

- I thought we already fixed that.

- I deployed but I did not have time to test everything...

Those comments are still pretty common in companies today because dev teams are not using the tools available out there to improve testing and deployment. In this article, I'll show you a simple solution to automate testing in web development and make deployments easier and faster with Azure DevOps.

This article is for people who already have some knowledge in web development and are familiar with unit testing. The technologies used in this solution are C# for backend, Angular for frontend but the principles can be applied to any other language or framework.

Together we'll see:

1. How to create builds and run unit tests on Azure DevOps

2. The best way to automate your deployment in multiple environments

3. How to run end-to-end tests and be sure that everything is working before going live and AUTOMATE the process

4. Bonus: The easiest way to collect screenshot when end-to-end tests fail

All of this will be fully automated like that we can focus on what is important: creating features. We don't want to waste our time fixing bugs that used to work before.

Let's get started!

1. How To Create Builds And Run Unit Tests On Azure DevOps

If you don't already have an account on Azure DevOps, navigate to dev.azure.com and create an account. With the Basic Plan, it is free for the first 5 users.

Once your account is created, follow the steps to create a project and then navigate to Repos, clone the GIT repository to your machine and push the code that you want to build and deploy.

For the purpose of this article, I've created a simple web application with ASP.NET Core and Angular for you. The app is just a form where the user enters some basic information. There is one backend unit test written with NUnit and two Angular unit tests written with Jest.

Code is accessible here: https://dev.azure.com/remisolutions/_git/avoid_regressions_tutorial. Clone and push it to your repository to follow along.

Now that you have your app, the next step is to create the build. From Azure DevOps, navigate to Pipelines > Builds > New Pipeline.

Choose the classic editor to configure the build, then select where the code is. In our case: Azure Repos Git. Finally, select the ASP.NET Core template.

Edit the "Test" task to update the property "Path to project(s)" with the value "**/*UnitTests.csproj". This change allows you to only execute unit tests. Like that it won't execute the end-to-end tests that you'll add later.

You now have a build to execute your backend unit tests. Let's add frontend unit tests.

You need to export the result of javascript tests in a way that Azure DevOps understands it. Use jest-junit, which is a npm package I've installed in the solution.

Package.json looks like this.

Then, go back to Azure DevOps to add 3 new tasks to the build definition:

1. npm

2. npm

3. Publish Test Results

That's it. Backend unit tests and frontend unit tests are now automatically executed during the build and you benefit from a nice interface to validate that everything works as expected.

Let's see now how to deploy the build artifacts to different environments.

2. The Best Way To Automate Your Deployment In Multiple Environments

The goal is to deploy the same artifacts (from the same build) in multiple environments. Like that, you can be sure that what you test in staging is exactly what is going to be in production.

I've created the following environments on Azure:

• https://avoid-regression-staging.azurewebsites.net - STAGING

• https://avoid-regression-production.azurewebsites.net - PRODUCTION

To deploy to those environments, go to Azure DevOps: Pipelines > Releases > New pipeline.

Select the template Azure App Service deployment and name the stage "Staging". Then click on Add an artifact and choose the build that we've created earlier.

Navigate to the Tasks tab, choose your Azure subscription, authorize the connexion and select the staging App service.

Go back to the Pipeline tab, add a new stage with the same template Azure App Service deployment, name it Production. Do the same steps as the Staging stage but deploy to the production app service.

Your pipeline should look like this:

You can save the release pipeline and create a new release to trigger deployment. To enable continuous deployment every time a new build is available, click on the Continuous deployment icon the artifacts section and enable the trigger.

3. How To Run End-To-End Tests And Be Sure That Everything Is Working Before Going Live And AUTOMATE The Process

You'll run the end-to-end tests on the staging environment right before deploying in production. To do that, you'll create a new stage in the release pipeline between the Staging and Production. But first, you need to build the end-to-end test project.

Note: I've included a project in the solution, AvoidRegressions.Web.EndToEndTests, that uses specflow and selenium with chrome driver in order to test the application with Chrome. The goal is to execute the tests present in this assembly on a Azure DevOps agent. A version of Chrome and Chrome driver is already installed on Azure DevOps agent. The path to the Chrome driver is available through the environment variable %CHROMEWEBDRIVER%. You can check the code to see how I've configured selenium to be able to run successfully on the Azure agent.

To update the build definition, go to Pipelines > Select the build definition > Edit

After the dotnet publish task, add a new copy files task.

Save and queue a new build. The artifacts should look like this after the build.

Now that the end-to-end tests are included in the artifacts, you can edit the release pipeline definition. Go to Pipelines > Releases > Edit and add a new stage after Staging. Start with an empty job. Name it End-to-End Tests.

Update the Production stage to be triggered after the End-to-End stage.

Now, go to Tasks > End-to-End Tests to configuration the test execution.

First, you need to update the appsettings.json to change the BaseUrl to be the staging BaseUrl. To do that, install a task from the marketplace that will easily allow you to update the json. Search for Build & Release Tools and install it.

Once installed, add the task named Set string value in a json file

Lastly, add a Visual Studio Test task.

Save and create a release. To be sure that the tests are successfully executed, navigate to the logs of the End-to-End Tests stage > Tests tab.

4. Bonus: The Easiest Way To Collect Screenshot When End-to-End Tests Fail

When errors occur, having a screenshot can help you find the cause of the problem faster. Thanks to Selenium and NUnit you can easily do that with test attachments. By default, the VsTest task from Azure DevOps will be able to upload all test attachments available.

In order to make the end-to-end fails, update the BaseUrl in the End-to-End test stage to be https://google.com

Create a new release to re-deploy and re-execute the end-to-end tests. Observe the test fails. To see the screenshot, navigate to End-to-End Tests stage logs > Tests tab > select the failing test > Attachments tab

Here we go. Now you have a complete solution to automate the testing and deployment of any web application.