There are scenarios when we would like to isolate routing in a legacy way – using old fashioned VRF right? This is what typically comes into mind to every old fashioned network engineer including myself.
In Aviatrix terminology there is similar concept to VRF – already discussed Network Domains. It is not fully a VRF but it is more than enough to provide traffic / spoke isolation. What if we need to extend it between onPrem and Aviatrix? For onPrem connectivity we can use cisco SDWAN which is fully capable of VRF. One way of connecting SDWAN to Aviatrix is through external IPSEC connections and BGP (BGPoverIPSEC).
No big deal right? Lets build more connections between them and assign them to different network domains individually.
However you can only build 1 ipsec tunnel between pair of public IPs.
On Aviatrix side you terminate all external connections on main interface eth0 which has only 1 public IP (BGPoverLAN terminates on dedicated int though but that doesn’t meet our purpose).
Lets add more int to SDWAN – also piece of cake. This approach has one drawback though, on a bigger scale might end up as quite costly solution. If we look at the instance size (i.e. Azure only)
Dv5 and Dsv5-series – Azure Virtual Machines | Microsoft Learn
- Do I need to use such a big monster (i.e. D16_v5) to get 8 connections / VRFs and extend them to Aviatrix?
- Can I go beyond that number?
- Does it make sense to run such big instances if I do not expect that much traffic? I just need routing isolation …
Secondary IP comes to rescue
Cisco i.e. supports up to 4 secondary IPs per interface so if we need 5 VRFs we are good with just 1 NIC on SDWAN. If we need more we can add more interfaces with public IPs.
Here with just 2 NICs assigned to SDWAN we are able to extend 10 VRFs to Aviatrix realm. Each private IP would have its own public IP.
Example - 2 SDWANs and 2 VRFs
Below is just example of IPSEC configuration with the use of secondary IP.
crypto ikev2 profile AVIATRIX-prof-VRF100
match identity remote address X.X.X.X 255.255.255.255
match identity remote address Y.Y.Y.Y 255.255.255.255
identity local address 10.198.234.4
authentication remote pre-share key TEST$123@321
authentication local pre-share key TEST$123@321
!
crypto ikev2 profile AVIATRIX-prof-VRF200
match identity remote address X.X.X.X 255.255.255.255
match identity remote address Y.Y.Y.Y 255.255.255.255
identity local address 10.198.234.5
authentication remote pre-share key TEST$123@321
authentication local pre-share key TEST$123@321
!
!
crypto ipsec profile IPSEC-PROF-vrf-100
set security-association lifetime kilobytes 102400000
set transform-set TSET TSET-aes256-sha256
set pfs group14
set ikev2-profile AVIATRIX-prof-VRF100
!
crypto ipsec profile IPSEC-PROF-vrf-200
set security-association lifetime kilobytes 102400000
set transform-set TSET TSET-aes256-sha256
set pfs group14
set ikev2-profile AVIATRIX-prof-VRF200
!
!
interface GigabitEthernet1
ip address 10.199.100.6 255.255.255.128 secondary
ip address 10.199.100.4 255.255.255.128
ip nat outside
negotiation auto
no mop enabled
no mop sysid
!
interface Tunnel101
ip vrf forwarding VRF100
ip address 10.198.252.146 255.255.255.252
ip mtu 1400
tunnel source 10.198.234.4
tunnel mode ipsec ipv4
tunnel destination X.X.X.X
tunnel protection ipsec profile IPSEC-PROF-vrf-100
!
interface Tunnel102
ip vrf forwarding VRF100
ip address 10.198.252.150 255.255.255.252
ip mtu 1400
tunnel source 10.198.234.4
tunnel mode ipsec ipv4
tunnel destination Y.Y.Y.Y
tunnel protection ipsec profile IPSEC-PROF-vrf-100
!
interface Tunnel201
ip vrf forwarding VRF200
ip address 10.198.253.146 255.255.255.252
ip mtu 1400
tunnel source 10.198.234.5
tunnel mode ipsec ipv4
tunnel destination X.X.X.X
tunnel protection ipsec profile IPSEC-PROF-vrf-200
!
interface Tunnel202
ip vrf forwarding VRF200
ip address 10.198.253.150 255.255.255.252
ip mtu 1400
tunnel source 10.198.234.5
tunnel mode ipsec ipv4
tunnel destination Y.Y.Y.Y
tunnel protection ipsec profile IPSEC-PROF-vrf-200
!
router bgp 4200000000
bgp log-neighbor-changes
!
address-family ipv4 vrf VRF100
neighbor 10.198.252.145 remote-as 4204500000
neighbor 10.198.252.145 activate
neighbor 10.198.252.145 send-community both
neighbor 10.198.252.149 remote-as 4204500000
neighbor 10.198.252.149 activate
neighbor 10.198.252.149 send-community both
exit-address-family
!
address-family ipv4 vrf VRF200
neighbor 10.198.253.145 remote-as 4204500000
neighbor 10.198.253.145 activate
neighbor 10.198.253.145 send-community both
neighbor 10.198.253.149 remote-as 4204500000
neighbor 10.198.253.149 activate
neighbor 10.198.253.149 send-community both
exit-address-family
!
########################## Transit 1 – sdwan01 #############################
resource “aviatrix_transit_external_device_conn” “transit1_sdwan01_vrf100” {
vpc_id = “${var.transit_1_vnet}:${var.transit_1_rg}:${var.transit_1_guid}“
connection_name = “TRANS01-SDWAN01-VRF-100”
gw_name = “transit-1”
remote_gateway_ip = “x.x.x.1” // public ip of nic0-ip0
phase1_remote_identifier = [“10.198.234.4”]
tunnel_protocol = “IPsec”
enable_ikev2 = true
connection_type = “bgp”
direct_connect = false
bgp_local_as_num = “4204500000”
bgp_remote_as_num = “4200000000”
ha_enabled = false
local_tunnel_cidr = “10.198.252.145/30,10.198.252.149/30”
remote_tunnel_cidr = “10.198.252.146/30,10.198.252.150/30”
custom_algorithms = true
phase_1_authentication = “SHA-512”
phase_1_dh_groups = “14”
phase_1_encryption = “AES-256-CBC”
phase_2_dh_groups = “14”
phase_2_authentication = “NO-AUTH”
phase_2_encryption = “AES-256-GCM-128”
pre_shared_key = data.azurerm_key_vault_secret.s2c[“sdwan02-transit1-vrf-100”].value
}
#——————————————————–
resource “aviatrix_transit_external_device_conn” “transit1_sdwan01_vrf200” {
vpc_id = “${var.transit_1_vnet}:${var.transit_1_rg}:${var.transit_1_guid}“
connection_name = “TRANS01-SDWAN01-VRF-200”
gw_name = “transit-1”
remote_gateway_ip = “x.x.x.2” // public ip of nic0-ip1 “
phase1_remote_identifier = [“10.198.234.5”]
tunnel_protocol = “IPsec”
enable_ikev2 = true
connection_type = “bgp”
direct_connect = false
bgp_local_as_num = “4204500000”
bgp_remote_as_num = “4200000000”
ha_enabled = false
local_tunnel_cidr = “10.198.253.145/30,10.198.253.149/30”
remote_tunnel_cidr = “10.198.253.146/30,10.198.253.150/30”
custom_algorithms = true
phase_1_authentication = “SHA-512”
phase_1_dh_groups = “14”
phase_1_encryption = “AES-256-CBC”
phase_2_dh_groups = “14”
phase_2_authentication = “NO-AUTH”
phase_2_encryption = “AES-256-GCM-128”
pre_shared_key = data.azurerm_key_vault_secret.s2c[“sdwan02-transit1-vrf-100”].value
}
########################## Transit 1 – sdwan02 #############################
resource “aviatrix_transit_external_device_conn” “transit1_sdwan02_vrf100” {
vpc_id = “${var.transit_1_vnet}:${var.transit_1_rg}:${var.transit_1_guid}“
connection_name = “TRANS01-SDWAN02-VRF-100”
gw_name = “transit-1”
remote_gateway_ip = “x.x.x.3” // public ip of nic0-ip0
phase1_remote_identifier = [“10.198.234.9”]
tunnel_protocol = “IPsec”
enable_ikev2 = true
connection_type = “bgp”
direct_connect = false
bgp_local_as_num = “4204500000”
bgp_remote_as_num = “4200000000”
ha_enabled = false
local_tunnel_cidr = “10.198.252.153/30,10.198.252.157/30”
remote_tunnel_cidr = “10.198.252.154/30,10.198.252.158/30”
custom_algorithms = true
phase_1_authentication = “SHA-512”
phase_1_dh_groups = “14”
phase_1_encryption = “AES-256-CBC”
phase_2_dh_groups = “14”
phase_2_authentication = “NO-AUTH”
phase_2_encryption = “AES-256-GCM-128”
pre_shared_key = data.azurerm_key_vault_secret.s2c[“sdwan02-transit1-vrf-100”].value
}
#——————————————————–
resource “aviatrix_transit_external_device_conn” “transit1_sdwan02_vrf200” {
vpc_id = “${var.transit_1}:${var.transit_1_rg}:${var.transit_1_guid}“
connection_name = “TRANS01-SDWAN02-VRF-200”
gw_name = “transit-1”
remote_gateway_ip = “x.x.x.4” // public ip of nic0-ip1
phase1_remote_identifier = [“10.198.234.10”] # use diff local id if terminated between
tunnel_protocol = “IPsec”
enable_ikev2 = true
connection_type = “bgp”
direct_connect = false
bgp_local_as_num = “4204500000”
bgp_remote_as_num = “4200000000”
ha_enabled = false
local_tunnel_cidr = “10.198.253.153/30,10.198.253.157/30”
remote_tunnel_cidr = “10.198.253.154/30,10.198.253.158/30”
custom_algorithms = true
phase_1_authentication = “SHA-512”
phase_1_dh_groups = “14”
phase_1_encryption = “AES-256-CBC”
phase_2_dh_groups = “14”
phase_2_authentication = “NO-AUTH”
phase_2_encryption = “AES-256-GCM-128”
pre_shared_key = data.azurerm_key_vault_secret.s2c[“sdwan02-transit1-vrf-100”].value
}