Create a fully private AKS infrastructure with Bicep #AKS #biceplang

14 Apr

Intro

In this blog post I will describe a way to implement an architecture that includes a private Azure Kubernetes Service (AKS) Cluster by using Bicep, a declarative language for describing and deploying Azure resources.

According to project’s page on GitHub, Bicep is a Domain Specific Language (DSL) for deploying Azure resources declaratively and It aims to drastically simplify the authoring experience with a cleaner syntax, improved type safety, and better support for modularity and code re-use. 

The control plane or API server, in a fully private AKS cluster, has internal IP address and it communicates with the node pools through the private network, by leveraging the technology of Azure Private Link service. Furthermore, there is no need for the worker nodes to have a public IP assigned to a standard loadbalancer for egress the traffic as we are able to redirect the egress path through a Network Virtual Appliance or Azure Firewall.

In a nutshell, a fully private AKS cluster does not expose or use any public IP.

References

The initial idea to create a Bicep script that deploys an architecture like the one mentioned in the architectural diagram section, was taken from the following articles:

Architectural Diagram

Private AKS

Bicep script

The sources can be found at the following GitHub repository

https://github.com/vakappas/private-aks-bicep

The main Bicep script that deploys the architecture that is shown in the diagram, follows the modular approach and it has ‘subscription’ as its target scope.

// set the target scope to subscription
targetScope = 'subscription'


/ Create the hub vnet with all its components
module hubvnet './modules/hub-default.bicep' = {
  name: 'hub-vnet'
  scope: resourceGroup(hubrg.name)
  params: {
    location: location
    hubVnetName: hubvnetName
    hubFwName: 'hub-fw'
    tags: tags
  }
}

there is a folder called ‘modules’ that contains the smaller parts of the whole implementation such as:

  • The hub virtual network with all its components like the Azure Firewall and Azure Bastion
  • The Virtual Network for the AKS cluster
  • The Virtual Network for the Development Virtual Machines (agents)
  • The VNET peering and route tables
  • The private AKS cluster
  • The AKS’s private DNS zone links to hub and dev Virtual Networks
  • The dev VM which is an Ubuntu with Docker extension

Running the script

First you have to install the Bicep executable by following this guide:

Setup your Bicep development environment

Then, you will need to install either the Az CLI (2.20.0+) or the PowerShell Az module (v5.6.0+) that have Bicep support built-in.

After that, you will need to run the following commands in an Azure CLI or PowerShell console:

## Login to your Azure account
az login

## clone the repository
git clone https://github.com/vakappas/private-aks-bicep.git

## Start the deployment
az deployment sub create -f ./private-aks.bicep -l northeurope

As you see, you can use the standard deployment commands with your *.bicep files and the tooling will transpile the code and send it to ARM on your behalf.

3 Replies to “Create a fully private AKS infrastructure with Bicep #AKS #biceplang

      • It worked! 🙂

        Had some warnings while running it:
        C:\private-aks-bicep\modules\hub-default.bicep(222,39) : Warning BCP174: Type validation is not available for resource types declared containing a “/providers/” segment. Please instead use the “scope” property. See https://aka.ms/BicepScopes for more information.
        C:\private-aks-bicep\modules\hub-default.bicep(321,29) : Warning BCP174: Type validation is not available for resource types declared containing a “/providers/” segment. Please instead use the “scope” property. See https://aka.ms/BicepScopes for more information.
        C:\private-aks-bicep\modules\hub-default.bicep(712,27) : Warning BCP174: Type validation is not available for resource types declared containing a “/providers/” segment. Please instead use the “scope” property. See https://aka.ms/BicepScopes for more information.

        Also, had issues with running it with user who doesn’t have admin permissions (I’m not normally working as a local admin… but guess this is Bicep-related) – was getting an error saying “type object ‘datetime.datetime’ has no attribute ‘fromisoformat'” and deyployment didn’t work. When running with admin user, all worked just fine.

        Cheers,
        Tom

Leave a Reply to Vaggelis Kappas Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.