Configuring a Layer 3 VXLAN Gateway

Context

When distributed VXLAN gateways are deployed using BGP EVPN, Layer 3 VXLAN gateways must be configured to implement inter-subnet communication.

In distributed VXLAN gateway scenarios, inter-subnet communication between hosts requires Layer 3 forwarding. To allow this, Layer 3 VXLAN gateways must learn host routes. Perform the following operations on VXLAN gateways:

Configure a VPN instance whose routes can be installed into the routing table of the EVPN instance. This VPN instance is used to store host routes or network segment routes, differentiating tenants.
Bind the VPN instance to a Layer 3 VXLAN gateway, enable distributed gateway, and configure host route advertisement.
Configure the type of route to be advertised between VXLAN gateways. VXLAN gateways can send different routing information through different types of routes. If an RR is deployed on the network, only the type of route to be advertised between the RR and VXLAN gateways needs to be configured.

Figure 1 Layer 3 VXLAN gateway networking

When configuring a VXLAN Layer 3 gateway, choose configuration steps according to the Overlay network IP layer protocol.

When the Overlay network is an IPv4 network, you can choose Configuration of VXLAN Layer 3 Gateway for an IPv4 overlay network.
When the Overlay network is an IPv6 network, you can choose Configuration of VXLAN Layer 3 Gateway for an IPv6 overlay network.

Procedure

Configuration of VXLAN Layer 3 Gateway for an IPv4 overlay network:
1. Configure a VPN instance whose routes can be installed into the routing table of the EVPN instance.
  1. Run system-view
    
    The system view is displayed.
  2. Run ip vpn-instance vpn-instance-name
    
    A VPN instance is created and the VPN instance view is displayed.
    
    By default, no VPN instance is created.
  3. Run vxlan vni vni-id
    
    A VNI is created and mapped to the VPN instance.
    
    By default, a VNI is not bound to any VPN instance.
  4. Run ipv4-family
    
    The IPv4 address family is enabled for the VPN instance, and the VPN instance IPv4 address family view is displayed.
    
    By default, the IPv4 address family is not enabled for any VPN instance.
  5. Run route-distinguisher route-distinguisher
    
    An RD is configured for the VPN instance IPv4 address family.
    
    By default, no RD is configured for the VPN instance IPv4 address family.
  6. (Optional) Run vpn-target vpn-target &<1-8> [ both | export-extcommunity | import-extcommunity ]
    
    VPN targets are configured for the VPN instance IPv4 address family.
    
    By default, no VPN target is configured for the VPN instance IPv4 address family.
    
    A VPN target is the extended community attribute of BGP. It controls reception and advertisement of VPN routes. A maximum of eight VPN targets can be configured each time the vpn-target command is run. To configure more VPN targets for the VPN instance IPv4 address family, run the vpn-target command several times.
  7. Run vpn-target vpn-target &<1-8> [ both | export-extcommunity | import-extcommunity ] evpn
    
    VPN targets are configured for the VPN instance IPv4 address family for exchanging routes with the EVPN instance. vpn-target specified must be the same as the RT of the EVPN instance.
    
    The routes advertised by the VPN instance IPv4 address family to an EVPN instance do not carry the export VPN targets of the VPN instance IPv4 address family. Instead, the routes carry all VPN targets in the export VPN target list configured for the EVPN instance in the BD.
    
    The routes advertised by an EVPN instance can be added to the routing table of the VPN instance IPv4 address family only when the VPN targets of the routes are carried in the import VPN target list of the VPN instance IPv4 address family.
  8. (Optional) Run import route-policy policy-name evpn
    
    The VPN instance IPv4 address family is associated with an import route-policy that is used to filter routes imported from the EVPN instance to the VPN instance IPv4 address family.
    
    By default, an EVPN instance matches the export VPN targets of received routes against the import VPN targets of the VPN instance IPv4 address family to determine whether to import these routes. To precisely import routes advertised by an EVPN instance to the VPN instance IPv4 address family, perform this step to associate the VPN instance IPv4 address family with an import route-policy and set attributes for eligible routes.
  9. (Optional) Run export route-policy policy-name evpn
    
    The VPN instance IPv4 address family is associated with an export route-policy that is used to filter routes advertised from the VPN instance IPv4 address family to the EVPN instance.
    
    By default, the routes advertised by the VPN instance IPv4 address family to an EVPN instance carry all export VPN targets of the VPN instance IPv4 address family. To precisely import routes advertised by the VPN instance IPv4 address family to an EVPN instance, perform this step to associate the VPN instance IPv4 address family with an export route-policy and set attributes for eligible routes.
  10. Run quit
    
    Exit from the VPN instance IPv4 address family view.
  11. Run quit
    
    Exit from the VPN instance view.
2. Bind the VPN instance to a Layer 3 VXLAN gateway, enable distributed gateway, and configure host route advertisement.
  1. Run interface vbdif bd-id
    
    A VBDIF interface is created, and the VBDIF interface view is displayed.
    
    By default, no VBDIF interface is created.
    The number of the VBDIF interface must match an existing BD ID.
    
    For the S6720-EI and S6720S-EI switches, if the assign resource-mode command is run to set the resource mode to super-arp, the switches cannot forward VXLAN packets at Layer 3.
  2. Run ip binding vpn-instance vpn-instance-name
    
    A VPN instance is bound to the VBDIF interface.
    
    By default, no VPN instance is bound to a VBDIF interface.
  3. Run ip address ip-address { mask | mask-length } [ sub ]
    
    An IPv4 address is configured for the VBDIF interface.
    
    By default, no IP address is configured for a VBDIF interface.
  4. (Optional) Run mac-address mac-address
    
    A MAC address is configured for the VBDIF interface.
    
    By default, the MAC address of a VBDIF interface is the system MAC address.
    
    On a network with distributed VXLAN gateways that need to be simulated into one, you need to run the mac-address command to configure the same MAC address for the VBDIF interfaces of VXLAN Layer 3 gateways.
  5. Run arp distribute-gateway enable
    
    Distributed gateway is enabled.
    
    By default, distributed gateway is disabled.
    
    After distributed gateway is enabled on a Layer 3 gateway, the Layer 3 gateway discards network-side ARP packets and learns only user-side ARP packets.
  6. Run arp collect host enable
    
    Host route advertisement is configured for the VBDIF interface.
  7. Run quit
    
    Exit from the VBDIF interface view.
3. Configure the type of route to be advertised between VXLAN gateways. If an RR has been deployed, configure the type of route to be advertised between VXLAN gateways and the RR.
  - Configure IRB route advertisement.
    1. Run bgp as-number
      
      The BGP view is displayed.
    2. Run l2vpn-family evpn
      
      The BGP-EVPN address family view.
    3. Run peer ipv4-address advertise irb
      
      IRB route advertisement is configured.
    4. Run quit
      
      Exit from the BGP-EVPN address family.
    5. Run quit
      
      Exit from the BGP view.
    IRB routes are Type 2 BGP EVPN routes that carry hosts' MAC and IP addresses as well as Layer 2 and Layer 3 VNIs. IRB routes can be used to advertise host IP routes as well as ARP entries. After IRB route advertisement is configured, running the arp broadcast-suppress [ mismatch-discard ] enable command in BD view implements ARP broadcast suppression. In addition, host ARP entry advertisement allows VM migration in distributed gateway scenarios. As such, configuring IRB route advertisement is recommended.
  - Configure IP prefix route advertisement.
    1. Run bgp as-number
      
      The BGP is displayed.
    2. Run ipv4-family vpn-instance vpn-instance-name
      
      The BGP-VPN instance IPv4 address family view is displayed.
    3. Run import-route protocol [ process-id ] [ med med | route-policy route-policy-name ] ^*
      
      A type of route is imported to the BGP-VPN instance IPv4 address family view.
      
      If host IP route advertisement is required, configure direct in the command. If network segment route advertisement is required, use a dynamic routing protocol, such as OSPF. Then, configure the BGP-VPN instance IPv4 address family to import the routes of the dynamic routing protocol.
    4. Run advertise l2vpn evpn
      
      IP prefix route advertisement is configured.
    5. Run quit
      
      Exit from the BGP-VPN instance IPv4 address family view.
    6. Run quit
      
      Exit from the BGP view.
    IP prefix routes are Type 5 BGP EVPN routes that carry host IP addresses or network segment addresses as well as Layer 3 VNIs. IP prefix routes are used to advertise host IP routes as well as network segment routes to which the host IP routes belong. If a large number of specific host routes are available, configure IP prefix route advertisement so that the network segment routes can be imported to the BGP-VPN instance IPv4 address family, sparing the VXLAN gateways from storing all specific host routes.
    
    A VXLAN gateway can advertise network segment routes only if the network segments attached to the gateway are unique network-wide.
Configuration of VXLAN Layer 3 Gateway for an IPv6 overlay network:
1. Configure a VPN instance whose routes can be installed into the routing table of the EVPN instance.
  1. Run system-view
    
    The system view is displayed.
  2. Run ip vpn-instance vpn-instance-name
    
    A VPN instance is created and the VPN instance view is displayed.
    
    By default, no VPN instance is created.
  3. Run vxlan vni vni-id
    
    A VNI is created and mapped to the VPN instance.
    
    By default, a VNI is not bound to any VPN instance.
  4. Run ipv6-family
    
    The IPv6 address family is enabled for the VPN instance, and the VPN instance IPv6 address family view is displayed.
    
    By default, the IPv6 address family is not enabled for any VPN instance.
  5. Run route-distinguisher route-distinguisher
    
    An RD is configured for the VPN instance IPv6 address family.
    
    By default, no RD is configured for the VPN instance IPv6 address family.
  6. (Optional) Run vpn-target vpn-target &<1-8> [ both | export-extcommunity | import-extcommunity ]
    
    VPN targets are configured for the VPN instance IPv6 address family.
    
    By default, no VPN target is configured for the VPN instance IPv6 address family.
    
    A VPN target is the extended community attribute of BGP. It controls reception and advertisement of VPN routes. A maximum of eight VPN targets can be configured each time the vpn-target command is run. To configure more VPN targets for the VPN instance IPv6 address family, run the vpn-target command several times.
  7. Run vpn-target vpn-target &<1-8> [ both | export-extcommunity | import-extcommunity ] evpn
    
    VPN targets are configured for the VPN instance IPv6 address family for exchanging routes with the EVPN instance. vpn-target specified must be the same as the RT of the EVPN instance.
    
    The routes advertised by the VPN instance IPv6 address family to an EVPN instance do not carry the export VPN targets of the VPN instance IPv6 address family. Instead, the routes carry all VPN targets in the export VPN target list configured for the EVPN instance in the BD.
    
    The routes advertised by an EVPN instance can be added to the routing table of the VPN instance IPv6 address family only when the VPN targets of the routes are carried in the import VPN target list of the VPN instance IPv6 address family.
  8. (Optional) Run import route-policy policy-name evpn
    
    The VPN instance IPv6 address family is associated with an import route-policy that is used to filter routes imported from the EVPN instance to the VPN instance IPv6 address family.
    
    By default, an EVPN instance matches the export VPN targets of received routes against the import VPN targets of the VPN instance IPv6 address family to determine whether to import these routes. To precisely import routes advertised by an EVPN instance to the VPN instance IPv4 address family, perform this step to associate the VPN instance IPv6 address family with an import route-policy and set attributes for eligible routes.
  9. (Optional) Run export route-policy policy-name evpn
    
    The VPN instance IPv4 address family is associated with an export route-policy that is used to filter routes advertised from the VPN instance IPv6 address family to the EVPN instance.
    
    By default, the routes advertised by the VPN instance IPv6 address family to an EVPN instance carry all export VPN targets of the VPN instance IPv6 address family. To precisely import routes advertised by the VPN instance IPv4 address family to an EVPN instance, perform this step to associate the VPN instance IPv6 address family with an export route-policy and set attributes for eligible routes.
  10. Run quit
    
    Exit from the VPN instance IPv6 address family view.
  11. Run quit
    
    Exit from the VPN instance view.
2. Bind the VPN instance to a Layer 3 VXLAN gateway, enable distributed gateway, and configure host route advertisement.
  1. Run ipv6
    
    IPv6 packet forwarding is enabled.
  2. Run interface vbdif bd-id
    
    A VBDIF interface is created, and the VBDIF interface view is displayed.
    
    By default, no VBDIF interface is created.
  3. Run ip binding vpn-instance vpn-instance-name
    
    A VPN instance is bound to the VBDIF interface.
    
    By default, no VPN instance is bound to a VBDIF interface.
  4. Run ipv6 enable
    
    The IPv6 function is enabled on the interface.
    
    By default, the IPv6 function is disabled on an interface.
  5. Run ipv6 address { ipv6-address prefix-length | ipv6-address/prefix-length }
    
    An IPv6 global unicast address is manually configured.
    
    Alternatively, run ipv6 address { ipv6-address prefix-length | ipv6-address/prefix-length } eui-64
    
    An IPv6 global unicast address is generated in EUI-64 format.
    
    By default, no IPv6 address is configured for a VBDIF interface.
  6. (Optional) Run mac-address mac-address
    
    A MAC address is configured for the VBDIF interface.
    
    By default, the MAC address of a VBDIF interface is the system MAC address.
    
    On a network with distributed VXLAN gateways that need to be simulated into one, you need to run the mac-address command to configure the same MAC address for the VBDIF interfaces of VXLAN Layer 3 gateways.
  7. Run ipv6 nd distribute-gateway enable
    
    Distributed gateway is enabled.
    
    By default, distributed gateway is disabled.
    
    After distributed gateway is enabled on a Layer 3 gateway, the Layer 3 gateway discards network-side NS packets and learns only user-side NS packets.
  8. Run ipv6 nd collect host enable
    
    Host IPv6 route advertisement is configured for the VBDIF interface.
  9. Run quit
    
    Exit from the VBDIF interface view.
3. Configure the type of route to be advertised between VXLAN gateways. If an RR has been deployed, configure the type of route to be advertised between VXLAN gateways and the RR.
  - Configure IRB route advertisement.
    1. Run bgp as-number
      
      The BGP view is displayed.
    2. Run l2vpn-family evpn
      
      The BGP-EVPN address family view.
    3. Run peer ipv4-address advertise irbv6
      
      IRBv6 route advertisement is configured.
    4. Run quit
      
      Exit from the BGP-EVPN address family.
    5. Run quit
      
      Exit from the BGP view.
    IRB routes are Type 2 BGP EVPN routes that carry hosts' MAC and IP addresses as well as Layer 2 and Layer 3 VNIs. IRB routes can be used to advertise host IP routes as well as ARP entries. After IRB route advertisement is configured, running the ipv6 nd multicast-suppress [ mismatch-discard ] enable command in BD view implements ARP broadcast suppression. In addition, host ARP entry advertisement allows VM migration in distributed gateway scenarios. As such, configuring IRB route advertisement is recommended.
  - Configure IP prefix route advertisement.
    1. Run bgp as-number
      
      The BGP is displayed.
    2. Run ipv6-family vpn-instance vpn-instance-name
      
      The BGP-VPN instance IPv6 address family view is displayed.
    3. Run import-route protocol [ process-id ] [ med med | route-policy route-policy-name ] ^*
      
      A type of route is imported to the BGP-VPN instance IPv6 address family view.
      
      If host IP route advertisement is required, configure direct in the command. If network segment route advertisement is required, use a dynamic routing protocol, such as OSPF. Then, configure the BGP-VPN instance IPv6 address family to import the routes of the dynamic routing protocol.
    4. Run advertise l2vpn evpn
      
      IPv6 prefix route advertisement is configured.
    5. Run quit
      
      Exit from the BGP-VPN instance IPv4 address family view.
    6. Run quit
      
      Exit from the BGP view.
    IP prefix routes are Type 5 BGP EVPN routes that carry host IP addresses or network segment addresses as well as Layer 3 VNIs. IP prefix routes are used to advertise host IP routes as well as network segment routes to which the host IP routes belong. If a large number of specific host routes are available, configure IP prefix route advertisement so that the network segment routes can be imported to the BGP-VPN instance IPv4 address family, sparing the VXLAN gateways from storing all specific host routes.
    
    A VXLAN gateway can advertise network segment routes only if the network segments attached to the gateway are unique network-wide.

Follow-up Procedure

The S6720-EI and S6720S-EI switches can decapsulate received VXLAN packets and forward them at Layer 3 only after a VXLAN loopback interface is configured on them. As a result, you need to configure an Eth-Trunk interface as the VXLAN loopback interface when the S6720-EI and S6720S-EI switches function as the Layer 3 VXLAN gateway. Perform the configuration as follows:

Run interface eth-trunk trunk-id

The Eth-Trunk interface view is displayed.
Run service type vxlan-tunnel

The Eth-Trunk interface is configured as a VXLAN loopback interface.

By default, an Eth-Trunk interface is not a VXLAN loopback interface.
Run trunkport interface-type interface-number

A physical interface is added to the Eth-Trunk interface.
- After an Eth-Trunk is configured as a VXLAN loopback interface, STP is automatically disabled on the Eth-Trunk. The Eth-Trunk then does not support STP configuration commands. After the configuration is canceled, STP is automatically enabled on the Eth-Trunk.
- Only one Eth-Trunk on a switch can be configured as the VXLAN loopback interface. VXLAN packets from all VBDIF interfaces are encapsulated and decapsulated by this loopback interface.
- An Eth-Trunk containing member interfaces cannot be configured as a VXLAN loopback interface.
- The configurations allowed on an Eth-Trunk to be configured as a loopback interface include description, enable snmp trap updown, jumboframe enable, mixed-rate link enable, qos phb marking enable, set flow-stat interval, shutdown, local-preference enable, traffic-policy (interface view), and trust. If other configurations exist on the Eth-Trunk, the Eth-Trunk cannot be configured as a loopback interface.
- After an Eth-Trunk is configured as a loopback interface, the Eth-Trunk supports only the following configurations: authentication open ucl-policy enable, description, enable snmp trap updown, jumboframe enable, mixed-rate link enable, qos phb marking enable, set flow-stat interval, shutdown, local-preference enable, statistic enable (interface view), traffic-policy (interface view), vcmp disable, and trust.
- Before running the undo service type vxlan-tunnel command, delete all the member interfaces of the Eth-Trunk interface and all VBDIF interfaces on the device.