This section provides an example for configuring three-segment VXLAN to enable Layer 3 communication between VMs that belong to the different DCs.
In Figure 1, DC-A and DC-B reside in different BGP ASs. To allow intra-DC VM communication (VMa1 and VMa2 in DC-A, and VMb1 and VMb2 in DC-B), configure BGP EVPN on the devices in the DCs to create VXLAN tunnels between distributed gateways. To allow VMs in different DCs (for example, VMa1 and VMb2) to communicate with each other, configure BGP EVPN on Leaf2 and Leaf3 to create another VXLAN tunnel. In this way, three-segment VXLAN tunnels are established to implement DC interconnection (DCI).
Interfaces 1 through 3 in this example represent GE 0/1/0, GE 0/1/8, and GE 0/1/16, respectively.
Device Name |
Interface Name |
IP Address |
Device Name |
Interface Name |
IP Address |
---|---|---|---|---|---|
Device1 |
GE 0/1/0 |
192.168.50.1/24 |
Device2 |
GE 0/1/0 |
192.168.60.1/24 |
GE 0/1/8 |
192.168.1.1/24 |
GE 0/1/8 |
192.168.1.2/24 |
||
Loopback1 |
1.1.1.1/32 |
Loopback1 |
2.2.2.2/32 |
||
Spine1 |
GE 0/1/0 |
192.168.10.1/24 |
Spine2 |
GE 0/1/0 |
192.168.30.1/24 |
GE 0/1/8 |
192.168.20.1/24 |
GE 0/1/8 |
192.168.40.1/24 |
||
Loopback1 |
3.3.3.3/32 |
Loopback1 |
4.4.4.4/32 |
||
Leaf1 |
GE 0/1/0 |
192.168.10.2/24 |
Leaf4 |
GE 0/1/0 |
192.168.40.2/24 |
GE 0/1/8 |
- |
GE 0/1/8 |
- |
||
Loopback1 |
5.5.5.5/32 |
Loopback1 |
8.8.8.8/32 |
||
Leaf2 |
GE 0/1/0 |
192.168.20.2/24 |
Leaf3 |
GE 0/1/0 |
192.168.30.2/24 |
GE 0/1/8 |
- |
GE 0/1/8 |
- |
||
GE 0/1/16 |
192.168.50.2/24 |
GE 0/1/16 |
192.168.60.2/24 |
||
Loopback1 |
6.6.6.6/32 |
Loopback1 |
7.7.7.7/32 |
The configuration roadmap is as follows:
Assign an IP address to each interface.
Configure an IGP to ensure route reachability between nodes.
Configure static routes to achieve interworking between DCs.
Configure BGP EVPN on Leaf1 and Leaf2 in DC-A and Leaf3 and Leaf4 in DC-B to create VXLAN tunnels between distributed gateways.
Configure BGP EVPN on DC edge nodes Leaf2 and Leaf3 to create a VXLAN tunnel between DCs.
To complete the configuration, you need the following data:
VLAN IDs of the VMs
BD IDs
VXLAN network identifiers (VNIs) in BDs and VNIs in VPN instances
For configuration details, see Configuration Files in this section.
For configuration details, see Configuration Files in this section.
For configuration details, see Configuration Files in this section.
Run the display vxlan tunnel command on leaf nodes to check VXLAN tunnel information. The following example uses the command output on Leaf2. The command output shows that the VXLAN tunnels are Up.
[~Leaf2] display vxlan tunnel
Number of vxlan tunnel : 2
Tunnel ID Source Destination State Type Uptime
---------------------------------------------------------------------
4026531841 6.6.6.6 5.5.5.5 up dynamic 00:11:01
4026531842 6.6.6.6 7.7.7.7 up dynamic 00:12:11
Run the display ip routing-table vpn-instance vpn1 command to check IP route information. The following example uses the command output on Leaf1.
[~Leaf1] display ip routing-table vpn-instance vpn1
Route Flags: R - relay, D - download to fib, T - to vpn-instance, B - black hole route
------------------------------------------------------------------------------
Routing Table : vpn1
Destinations : 8 Routes : 8
Destination/Mask Proto Pre Cost Flags NextHop Interface
10.1.1.0/24 Direct 0 0 D 10.1.1.1 Vbdif10
10.1.1.1/32 Direct 0 0 D 127.0.0.1 Vbdif10
10.1.1.255/32 Direct 0 0 D 127.0.0.1 Vbdif10
10.20.1.0/24 IBGP 255 0 RD 6.6.6.6 VXLAN
10.30.1.0/24 IBGP 255 0 RD 6.6.6.6 VXLAN
10.40.1.0/24 IBGP 255 0 RD 6.6.6.6 VXLAN
127.0.0.0/8 Direct 0 0 D 127.0.0.1 InLoopBack0
255.255.255.255/32 Direct 0 0 D 127.0.0.1 InLoopBack0
After the configurations are complete, VMa1 and VMb2 can communicate with each other.
Spine1 configuration file
# sysname Spine1 # interface GigabitEthernet0/1/0 undo shutdown ip address 192.168.10.1 255.255.255.0 # interface GigabitEthernet0/1/8 undo shutdown ip address 192.168.20.1 255.255.255.0 # interface LoopBack1 ip address 3.3.3.3 255.255.255.255 # ospf 1 area 0.0.0.0 network 3.3.3.3 0.0.0.0 network 192.168.10.0 0.0.0.255 network 192.168.20.0 0.0.0.255 # return
Leaf1 configuration file
# sysname Leaf1 # evpn vpn-instance evrf1 bd-mode route-distinguisher 10:1 vpn-target 11:1 export-extcommunity vpn-target 11:1 import-extcommunity # ip vpn-instance vpn1 ipv4-family route-distinguisher 11:11 apply-label per-instance vpn-target 1:1 export-extcommunity vpn-target 11:1 export-extcommunity evpn vpn-target 1:1 import-extcommunity vpn-target 11:1 import-extcommunity evpn vxlan vni 5010 # bridge-domain 10 vxlan vni 10 split-horizon-mode evpn binding vpn-instance evrf1 # interface Vbdif10 ip binding vpn-instance vpn1 ip address 10.1.1.1 255.255.255.0 arp collect host enable vxlan anycast-gateway enable # interface GigabitEthernet0/1/0 undo shutdown ip address 192.168.10.2 255.255.255.0 # interface GigabitEthernet0/1/8 undo shutdown # interface GigabitEthernet0/1/8.1 mode l2 encapsulation dot1q vid 10 rewrite pop single bridge-domain 10 # interface LoopBack1 ip address 5.5.5.5 255.255.255.255 # interface Nve1 source 5.5.5.5 vni 10 head-end peer-list protocol bgp # bgp 100 peer 6.6.6.6 as-number 100 peer 6.6.6.6 connect-interface LoopBack1 # ipv4-family unicast undo synchronization peer 6.6.6.6 enable # ipv4-family vpn-instance vpn1 import-route direct advertise l2vpn evpn # l2vpn-family evpn undo policy vpn-target peer 6.6.6.6 enable peer 6.6.6.6 advertise irb peer 6.6.6.6 advertise encap-type vxlan # ospf 1 area 0.0.0.0 network 5.5.5.5 0.0.0.0 network 192.168.10.0 0.0.0.255 # return
Leaf2 configuration file
# sysname Leaf2 # evpn vpn-instance evrf1 bd-mode route-distinguisher 10:1 vpn-target 11:1 export-extcommunity vpn-target 11:1 import-extcommunity # ip vpn-instance vpn1 ipv4-family route-distinguisher 11:11 apply-label per-instance vpn-target 1:1 export-extcommunity vpn-target 11:1 export-extcommunity evpn vpn-target 1:1 import-extcommunity vpn-target 11:1 import-extcommunity evpn vxlan vni 5010 # bridge-domain 20 vxlan vni 20 split-horizon-mode evpn binding vpn-instance evrf1 # interface Vbdif20 ip binding vpn-instance vpn1 ip address 10.20.1.1 255.255.255.0 arp collect host enable vxlan anycast-gateway enable # interface GigabitEthernet0/1/0 undo shutdown ip address 192.168.20.2 255.255.255.0 # interface GigabitEthernet0/1/8 undo shutdown # interface GigabitEthernet0/1/8.1 mode l2 encapsulation dot1q vid 20 rewrite pop single bridge-domain 20 # interface GigabitEthernet0/1/16 undo shutdown ip address 192.168.50.2 255.255.255.0 # interface LoopBack1 ip address 6.6.6.6 255.255.255.255 # interface Nve1 source 6.6.6.6 vni 20 head-end peer-list protocol bgp # bgp 100 peer 5.5.5.5 as-number 100 peer 5.5.5.5 connect-interface LoopBack1 peer 7.7.7.7 as-number 200 peer 7.7.7.7 ebgp-max-hop 255 peer 7.7.7.7 connect-interface LoopBack1 # ipv4-family unicast undo synchronization peer 5.5.5.5 enable peer 7.7.7.7 enable # ipv4-family vpn-instance vpn1 import-route direct advertise l2vpn evpn # l2vpn-family evpn undo policy vpn-target peer 5.5.5.5 enable peer 5.5.5.5 advertise irb peer 5.5.5.5 advertise encap-type vxlan peer 5.5.5.5 import reoriginate peer 5.5.5.5 advertise route-reoriginated evpn ip peer 7.7.7.7 enable peer 7.7.7.7 advertise irb peer 7.7.7.7 advertise encap-type vxlan peer 7.7.7.7 import reoriginate peer 7.7.7.7 advertise route-reoriginated evpn ip # ospf 1 area 0.0.0.0 network 6.6.6.6 0.0.0.0 network 192.168.20.0 0.0.0.255 # ip route-static 7.7.7.7 255.255.255.255 192.168.50.1 ip route-static 192.168.1.0 255.255.255.0 192.168.50.1 ip route-static 192.168.60.0 255.255.255.0 192.168.50.1 # return
Spine2 configuration file
# sysname Spine2 # interface GigabitEthernet0/1/0 undo shutdown ip address 192.168.30.1 255.255.255.0 # interface GigabitEthernet0/1/8 undo shutdown ip address 192.168.40.1 255.255.255.0 # interface LoopBack1 ip address 4.4.4.4 255.255.255.255 # ospf 1 area 0.0.0.0 network 4.4.4.4 0.0.0.0 network 192.168.30.0 0.0.0.255 network 192.168.40.0 0.0.0.255 # return
Leaf3 configuration file
# sysname Leaf3 # evpn vpn-instance evrf1 bd-mode route-distinguisher 10:1 vpn-target 11:1 export-extcommunity vpn-target 11:1 import-extcommunity # ip vpn-instance vpn1 ipv4-family route-distinguisher 11:11 apply-label per-instance vpn-target 1:1 export-extcommunity vpn-target 11:1 export-extcommunity evpn vpn-target 1:1 import-extcommunity vpn-target 11:1 import-extcommunity evpn vxlan vni 5010 # bridge-domain 10 vxlan vni 10 split-horizon-mode evpn binding vpn-instance evrf1 # interface Vbdif10 ip binding vpn-instance vpn1 ip address 10.30.1.1 255.255.255.0 arp collect host enable vxlan anycast-gateway enable # interface GigabitEthernet0/1/0 undo shutdown ip address 192.168.30.2 255.255.255.0 # interface GigabitEthernet0/1/8 undo shutdown # interface GigabitEthernet0/1/8.1 mode l2 encapsulation dot1q vid 10 rewrite pop single bridge-domain 10 # interface GigabitEthernet0/1/16 undo shutdown ip address 192.168.60.2 255.255.255.0 # interface LoopBack1 ip address 7.7.7.7 255.255.255.255 # interface Nve1 source 7.7.7.7 vni 10 head-end peer-list protocol bgp # bgp 200 peer 6.6.6.6 as-number 100 peer 6.6.6.6 ebgp-max-hop 255 peer 6.6.6.6 connect-interface LoopBack1 peer 8.8.8.8 as-number 200 peer 8.8.8.8 connect-interface LoopBack1 # ipv4-family unicast undo synchronization peer 6.6.6.6 enable peer 8.8.8.8 enable # ipv4-family vpn-instance vpn1 import-route direct advertise l2vpn evpn # l2vpn-family evpn undo policy vpn-target peer 6.6.6.6 enable peer 6.6.6.6 advertise irb peer 6.6.6.6 advertise encap-type vxlan peer 6.6.6.6 import reoriginate peer 6.6.6.6 advertise route-reoriginated evpn ip peer 8.8.8.8 enable peer 8.8.8.8 advertise irb peer 8.8.8.8 advertise encap-type vxlan peer 8.8.8.8 import reoriginate peer 8.8.8.8 advertise route-reoriginated evpn ip # ospf 1 area 0.0.0.0 network 7.7.7.7 0.0.0.0 network 192.168.30.0 0.0.0.255 # ip route-static 6.6.6.6 255.255.255.255 192.168.60.1 ip route-static 192.168.1.0 255.255.255.0 192.168.60.1 ip route-static 192.168.50.0 255.255.255.0 192.168.60.1 # return
Leaf4 configuration file
# sysname Leaf4 # evpn vpn-instance evrf1 bd-mode route-distinguisher 10:1 vpn-target 11:1 export-extcommunity vpn-target 11:1 import-extcommunity # ip vpn-instance vpn1 ipv4-family route-distinguisher 11:11 apply-label per-instance vpn-target 1:1 export-extcommunity vpn-target 11:1 export-extcommunity evpn vpn-target 1:1 import-extcommunity vpn-target 11:1 import-extcommunity evpn vxlan vni 5010 # bridge-domain 20 vxlan vni 20 split-horizon-mode evpn binding vpn-instance evrf1 # interface Vbdif20 ip binding vpn-instance vpn1 ip address 10.40.1.1 255.255.255.0 arp collect host enable vxlan anycast-gateway enable # interface GigabitEthernet0/1/0 undo shutdown ip address 192.168.40.2 255.255.255.0 # interface GigabitEthernet0/1/8 undo shutdown # interface GigabitEthernet0/1/8.1 mode l2 encapsulation dot1q vid 20 rewrite pop single bridge-domain 20 # interface LoopBack1 ip address 8.8.8.8 255.255.255.255 # interface Nve1 source 8.8.8.8 vni 20 head-end peer-list protocol bgp # bgp 200 peer 7.7.7.7 as-number 200 peer 7.7.7.7 connect-interface LoopBack1 # ipv4-family unicast undo synchronization peer 7.7.7.7 enable # ipv4-family vpn-instance vpn1 import-route direct advertise l2vpn evpn # l2vpn-family evpn undo policy vpn-target peer 7.7.7.7 enable peer 7.7.7.7 advertise irb peer 7.7.7.7 advertise encap-type vxlan # ospf 1 area 0.0.0.0 network 8.8.8.8 0.0.0.0 network 192.168.40.0 0.0.0.255 # return
Device1 configuration file
# sysname Device1 # interface GigabitEthernet0/1/0 undo shutdown ip address 192.168.50.1 255.255.255.0 # interface GigabitEthernet0/1/8 undo shutdown ip address 192.168.1.1 255.255.255.0 # interface LoopBack1 ip address 1.1.1.1 255.255.255.255 # ip route-static 6.6.6.6 255.255.255.255 192.168.50.2 ip route-static 7.7.7.7 255.255.255.255 192.168.1.2 ip route-static 192.168.60.0 255.255.255.0 192.168.1.2 # return
Device2 configuration file
# sysname Device2 # interface GigabitEthernet0/1/0 undo shutdown ip address 192.168.60.1 255.255.255.0 # interface GigabitEthernet0/1/8 undo shutdown ip address 192.168.1.2 255.255.255.0 # interface LoopBack1 ip address 2.2.2.2 255.255.255.255 # ip route-static 6.6.6.6 255.255.255.255 192.168.1.1 ip route-static 7.7.7.7 255.255.255.255 192.168.60.2 ip route-static 192.168.50.0 255.255.255.0 192.168.1.1 # return