In the rapidly evolving landscape of cloud infrastructure, the Azure Bicep stands out as a declarative language and a tool designed to simplify the deployment of Azure resources.

In this post, we will explore the world of Bicep modules that are published to Microsoft’s Public Registry through the Azure Verified Modules (AVM) initiative and how they can be combined with private modules that organizations can develop to fit their specific needs.

At the beginning, there was a chaos

Sorry for starting in such a dramatic way, I simply couldn't resist. With the beginning I mean the time when Bicep introduced the posibility to create modules, a reusable infrastructure configuration that can be called from a deployment template (using the module keyword) or from another module.

It was a much-needed step for better collaboration, code reuse, and sharing. What Bicep didn't have from the start was the concept of public registry, where users can either find & consume official modules published by Microsoft or publish their own modules. Something that for instance HashiCorp had for Terraform for a long time.

Many engineering teams at Microsoft began to publish their infra configuration in various GitHub repositories with little to no commitment to keep that code up-to-date and provide support. In addition, the code you wished to use had to be cloned and 'merged' to your codebase. There wasn't any other streamlined way to 'consume it'.

AVM to the rescue

Eventually, a couple of engineers at Microsoft realized how this situation complicates the adoption of Bicep or infrastructure as code (IaC) for Azure in general, and started a new initiative called Azure Verified Modules.

It is a great initiative and the community behind it. I would encourage you to join it, attend community calls, contribute with your ideas on their GitHub repository, and more importantly, start using AVM modules :).

I will borrow their value proposition here because it is particularly important:

• Azure Verified Modules (AVM) is an initiative to consolidate and set the standards for what a good Infrastructure-as-Code module looks like.

• Modules will then align to these standards, across languages (Bicep, Terraform etc.) and will then be classified as AVMs and available from their respective language specific registries.

• AVM is a common code base, a toolkit for our Customers, our Partners, and Microsoft. It’s an official, Microsoft driven initiative, with a devolved ownership approach to develop modules, leveraging internal & external communities.

• Azure Verified Modules enable and accelerate consistent solution development and delivery of cloud-native or migrated applications and their supporting infrastructure by codifying Microsoft guidance (WAF), with best practice configurations.

Practically it means, that we as users are getting a 'single source of truth', one place, where we can get find a curated collection of well-designed, written, tested, and supported IaC modules for Bicep and Terraform.

There is one important distinction when we talk about IaC modules in a broad way: while HashiCorp allowed both organizations and individuals to publish their modules in the Terraform Registry for a long time, this isn't the case for Bicep Registry. Yes, there is an option to host your own private registry using Azure Container Registry (ACR) service, but it's far from being able to use a central place and publish your own configuration.

Public vs. private conundrum

You might be wondering; when Microsoft is giving me all those well-written and flexible modules, why would I want to use my private modules as well?

I see several reasons:

• you want to use some specific configuration, something unique to your organization that you don't want to publish externally.

• you cannot create new modules directly in the AVM repository and get them published to MCR (public registry). There is a process how one could contribute to this repo, but it has some strict requirements (like the ownership of modules).

On the other hand, you don't want to disregard AVM modules and maintain everything yourself! This would be too much of an overhead for you or your platform team.

In my head, this is about finding the right balance between consuming AVM modules and authoring your own modules. You can have such mix, because the AVM team ensures their modules are flexible, generalized, multi-purpose, and they integrate child and extension resources.

When we look at the module composition, there are two primary types of modules:

• resource modules - deploy a primary resource with WAF (Well-Architected Framework) high priority/impact best practice configurations set by default

• pattern modules - deploy multiple resources, usually using Resource Modules. They can be any size but should help accelerate a common task / deployment / architecture.

The most typical scenario would be to design your private pattern modules that are deploying a complete application or a landing zone archetype while referring to AVM resource modules as building blocks.

The following diagram shows, how you can combine these two types of assets:

• your platform team establishes a 'Private Modules Library', a code repository with CI/CD pipelines for verifying, testing, and publishing the modules to ACR. Those modules would be ideally written in accordance with AVM principles and specifications.

• your DevOps teams have access to a private registry (ACR) through Entra ID. they can refer to private modules from the registry in their bicep deployment files.

Building the Library

I have created a demo environment you can explore that will show you all code assets needed to build such a library.

First, let's look at the structure of this repo:

• infrastructure - contains a bicep configuration (template and parameter file) for deploying prerequisites (ACR, Web App). There is a workflow that gets triggered when you update the bicep file - .github/workflows/deploy-infrastructure.yaml.

• modules - a collection of your private modules that reference AVM modules from MCR. Each module has its own folder with deploy.bicep, README.md (auto-generated), and version.json file that determines, what tag will your module get when it is published to your private registry. Each module should have a corresponding workflow definition, for example .github/workflows/.github/workflows/module-ptn-avd-management-plane-publish.yaml).

• .github/actions - private actions responsible for testing, validating, and publishing the modules, as well as generating the documentation. The testing actions are leveraging PSRule for Azure, a great testing framework for Bicep / ARM templates.

• docs - a root folder for the Modules Web Catalog, a user-friendly web interface showing users all the private modules and their documentation. This Catalog is powered by PSDocs for Azure and MKDocs tools and hosted at Azure Static Webapps.

• scripts - a collection of PowerShell scripts that are used in the workflows or from the command line.

Try it out

There is a couple of things you need in place, if you want to test this solution using the code I published:

1. You need to fork the repo to your own GH account.

2. Create either a Service Principal or a Managed Identity with federated credentials. The service principal or managed identity used by the GitHub workflows needs to have the necessary permissions to deploy resources in the Azure subscription. The following permissions are needed:

   • Owner role assigned on the subscription level.

   • Acr Pull and Acr Push: in the Container Registry containing all bicep modules. This can be added after you provision the infrastructure in step 4.

3. Create the following repository-level secrets and variables on GH:

   • AZ_TENANT_ID - secret - your Entra ID tenant identifier

   • AZ_CLIENT_ID - secret - client ID of your SPN or MI

   • AZ_SUBSCRIPTION_ID - secret - Azure subscription identifier that will host the Modules Library

     

4. Modify the deploy.bicepparam file in the infrastructure directory and verify if the Actions workflow run was successful (check your Azure environment if those resources were provisioned). Focus on resource names to make them unique!

5. Modify bicepconfig.json in the repo root and update the ACR name. Create a new repository-level variable called AZ_MODULE_REGISTRY and add the name of your ACR registry without the .azurecr.io suffix.

6. Run the following command from the root of the repository from your PowerShell terminal: ./scripts/hydrate-registry.ps1 -moduleType 'ptn'

7. Stage, commit, and push the changes the script made to your upstream. This should trigger an Action workflow and publish one pattern module to your ACR registry. Check the status in the Azure Portal.

   

8. Open the Azure Portal and navigate to the Azure Static Web App service. Select correct app (e.g., moduleswebcatalog) and click Manage deployment token button. Copy the token value. Create a new GitHub repository secret with the name AZURE_STATIC_WEB_APPS_API_TOKEN and paste the token value.

9. Run the following command from your PowerShell terminal from the repository root: ./scripts/hydrate-docs.ps1 -registryName 'autlibbicepreg' Remember to update the registryName value. This script will (re)regenerate markdown files for all modules in the docs folder and respective modules folders.

   💡

   There is a mkdocs.yml file in the root of the repository that defines the structure of the Web Catalogue. You can modify this file to change the structure and modify its look and feel, like customer branding.

10. Stage all changed files and commit the changes to the repository. Push the changes to the remote repository. This will automatically trigger the 'Publish Modules Web Catalogue (publish-modules-web-catalogue.yml)' workflow and publish the Web Catalogue to the Azure Static Web App service.

11. Open the Azure Portal and navigate to the Azure Static Web App service. Select the correct app (e.g., moduleswebcatalog) and click the Browse button. You should see a similar view:

    

When you want to introduce a new private module, you could use a helper script that can scaffold needed files for you. Simply run the following command from the PowerShell terminal from the repo root:

./scripts/New-PrivateModule.ps1 -ModuleName 'my-new-module' -ModuleType 'ptn'

where:

• ModuleName is the name of your module. It will also become the name of the folder containing all needed files. Repository names can only include lowercase alphanumeric characters, periods, dashes, underscores, and forward slashes.

• ModuleType could either be res (resource module) or ptn (pattern module).

After you run the script, you should see a new folder under the modules directory (and correct subdirectory), so you can focus on authoring your Bicep module. Since that bootstrapping script also creates a workflow definition, you should see a GitHub Action run after you push your commit with the new module files to GitHub.

Lifecycle management of modules

As you can imagine, keeping your private modules up to date is not a trivial task. There are several triggers that will require changes in your modules:

• new feature requests from your users that will enhance the modules. Some will introduce a breaking change (major version) and some won't.

• bug fixes

• new versions of AVM modules published in MCR - could be both new features (updates to resources properties, service capabilities, etc.) and fixes

You might also need to temporarily fork some AVM modules to your Private Library to fix urgent issues if you cannot wait for a fix from Microsoft.

While you can fully control the code and versioning of your private modules, there are several challenges linked with AVM modules and MCR registry:

• there isn't any user-friendly web interface (as far as I know) that would allow you to browse modules in MCR and see what versions are available. You can use Bicep extension in Visual Studio Code to get such list when authoring your Bicep configuration:

• each module in the AVM repository has its own version.json file (for example the virtual-network module), but it only shows major and minor version. The patch version is auto generated by the publishing workflow.

• one way to get the overview of versions is by visiting the published modules index, where you can both see the latest version and get the list of all versions from MCR (for example for the virtual-network module).

  

    {
        "name": "bicep/avm/res/network/virtual-network",
        "tags": [
            "0.1.0",
            "0.1.1",
            "0.1.5",
            "0.1.6",
            "0.1.7",
            "0.1.8",
            "0.2.0"
        ]
    }
  

It would be great to have a change feed one could subscribe to and be notified if some modules got a latest version. This would help the platform team decide, if they want to 'bump up' the AVM modules references in their private modules and publish an updated version.

Callouts

The code used for this article was written in a collaboration with my great colleague Simen Østensen. You can check his LinkedIn profile and work done on GitHub.