Access Azure VMware Solution Using Azure Route Server and VMware SD-WAN

By Trevor Davis March 31, 2021 Azure Route Server, Azure VMware Solution, Network, VMware SD-WAN 0 Comments

Written By: Trevor Davis (Microsoft), Eric Cheung-Young-Sen (VMware), Jeffrey Moore (VMware)

Recently worked with a customer who is migrating to Azure VMware Solution and has a very expansive VMware SD-WAN footprint.

The customer’s general architecture; from their on-premises datacenter, they have an ExpressRoute to Microsoft Azure. The remote offices connect back to the on-premises datacenter through VMware SD-WAN. Great architecture, but when workloads expand to Azure VMware Solution, bringing remote office traffic back to the on-premises datacenter to send the traffic to Azure isn’t optimal.

The optimal solution would be to take a more direct route from the remote offices as shown in the next graphic.

For remote offices to directly communicate with Azure and Azure VMware Solution, a VMware SD-WAN Edge appliance needs to be deployed into an Azure Virtual Network along with Azure Route Server (At the time this article was written Azure Route Server is in Public Preview). The result of this design is that all the remote sites will learn of the Azure and Azure VMware Solution networks from the VMware SD-WAN Edge in the Virtual Network. Azure VMware Solution will learn all the remote site network segments via BGP. All this because of the super cool Azure Route Server.

Why not just put a static route in Azure VMware Solution to the SD-WAN Edge in the Virtual Network or some other static routing trick? Or even just let the SD-WAN Edge advertise BGP routes without Azure Route Server. A couple of reasons. First, Azure VMware Solution to learn about anything outside of itself must learn it via BGP updates. The Azure Route Server facilitates this. It will send any routes it learns to Azure VMware Solution via the ExpressRoute Gateway, connecting the Azure VMware Solution ExpressRoute to the Azure Virtual Network. The ExpressRoute Gateway, without Azure Route Server, cannot learn routes through dynamic routing protocols.

In the following steps, we will be configuring the architecture shown in the graphic below. Notice that the BGP routes get shared between the remote sites, on-premises, and Azure, including Azure VMware Solution. The Azure Route Server is facilitating that. Without Azure Route Server, the BGP routes will not exchange between the SD-WAN Edge and Azure, including Azure VMware Solution. As a benefit, the VMware SD-WAN platform can supplement the Azure ExpressRoute connecting on-premises and Microsoft Azure.

NOTE: Using VMware HCX for migration over VMware SD-WAN is NOT a supported configuration; if migrations from on-premises to Azure VMware Solution are needed, this must be done via an ExpressRoute to Azure.

The configuration steps below put all three (Virtual Network Gateway, SD-WAN Edge, and Azure Route Server) items in the same Virtual Network.

NOTE: The SD-WAN Edge can be deployed into any Virtual Network; it does not need to co-exist with the Azure Route Server. The Azure Route Server can peer with virtual appliances across virtual networks.

Now, let’s walk through the steps to get this to work.

In the Virtual Network with the Virtual Network Gateway for the Azure VMware Solution ExpressRoute, create a subnet called RouteServerSubnet. This subnet must contain a network that has at least a /27 CIDR block. Details can be found here.

Go to https://aka.ms/routeserver to kick off the process of deploying the Azure Route Server. Details can be found here.

Configure the subscription, resource group, name, and region as appropriate for your environment (see screenshot below).

Virtual network: The Virtual Network where the RouteServerSubnet exists.
Subnet: The subnet will be the RouteServerSubnet which is within the Virtual network selected above.

The SD-WAN Edge needs to be deployed, which can be done using an ARM template found here. Of course, the deployment parameters need to be modified for your particular environment.

You can find detailed deployment guidance here on how to deploy an SD-WAN Edge appliance. For reference, here is an example of the deployment parameters for the SD-WAN Edge.

NOTE: In a production environment best practice would be to cluster at minimum two SD-WAN Edge appliances for high availability. The screenshot below shows a 2-node cluster. More information on SD-WAN edge clustering can be found here.

Now that the SD-WAN Edge deployment is complete, it’s time to create a Peer to the SD-WAN Edge from the Azure Route Server.

You will want to go to the Peers blade of the Azure Route Server and select the +Add button.

Name: Friendly name of your choice.
ASN: The ASN of the SD-WAN Edge.
Ipv4 Address: The IP Address of the SD-WAN Edge.

The BGP peer needs to be configured on the SD-WAN Edge to point back to the Azure Route Server. In the graphic below on the left is the Overview blade of the Azure Route Server. Notice the ASN and Peer IPs. The ASN is, of course, the ASN of the Azure Route Server, the Peer Ips are the IP addresses of the Azure Route Server.

The SD-WAN Edge needs BGP enabled and neighbor IP addresses added. The SD-WAN Edge configuration screen is shown on the right.

Turn the BGP Settings to On and edit the parameters. As shown, enter both Peer Ips of the Azure Route Server and the ASN (both can be gotten from the RouteServer blade) and set the Max-hop count to 2 and the Local ASN to whatever was established during the Peer configuration of the Azure Route Server. In our example, the ASN is 65111.

After the BGP peering configuration is completed on the SD-WAN Edge, the SD-WAN Edge BGP Settings should look similar to what is shown below.

Because the SD-WAN Edge has multiple NICs, it’s good practice to make sure it knows which path to use to reach the Azure Route Server. Below is an example of the configuration screen. The Azure Route Server IP addresses (10.3.9.4 and 10.3.9.5) are statically routed out of interface GE3. Review this link for details on how to apply this configuration.

The final step in the configuration is to Enable the Branch-to-branch setting in the Configuration blade of the Azure Route Server, as shown in the below graphic.

After configuration is complete, the BGP Edge Neighbor State of the SD-WAN Edge appliance should resemble what is shown below. Notice the Neighbor IP addresses are the ones from the Azure Route Server, and the State is ESTABLISHED.