NFVI Distributed Gateway Function (BGP EVPN over E2E SR Tunnels)

The NFVI telco cloud solution uses the DCI+DCN networking. A large amount of mobile phone traffic is sent to vUGWs and vMSEs on the DCN. After being processed by the vUGWs and vMSEs, the IPv4 or IPv6 mobile phone traffic is forwarded over the DCN to destination devices on the Internet. The destination devices send traffic to mobile phones in similar ways. To achieve these functions and ensure traffic load balancing on the DCN, you need to deploy the NFVI distributed gateway function.

A vUGW is a unified gateway developed for Huawei's CloudEdge solution, which can be used for 3GPP access in GPRS, UMTS, and LTE modes. A vUGW can be a GGSN, an S-GW, or a P-GW, which meets the networking requirements of carriers in different stages and operations scenarios.

A vMSE is a virtual type of an MSE. The current carrier network includes multiple functional boxes, including the firewall box, video acceleration box, header enhancement box, and URL filter box. All of these functions are enabled through patch installation, causing a more and more complex network and difficult service provisioning and maintenance. To address the problems, vMSEs incorporate the functions of these boxes, uniformly manage these functions, and implement value-added service processing for the data service initiated by users.

The NFVI distributed gateway function supports service traffic transmission over SR or VXLAN tunnels. SR tunnels are classified as segmented SR tunnels or E2E SR tunnels. In E2E SR tunnel scenarios, PEs use BGP VPNv4/VPNv6 or BGP EVPN to connect to a DCN. The control-plane implementation varies according to the protocol used. This section describes the implementation principles in BGP EVPN scenarios.

Networking Introduction

Figure 1 shows the networking of an NFVI distributed gateway (BGP EVPN over E2E SR tunnels). DC-GWs, which are the border gateways of the DCN, exchange Internet routes with external devices over PEs. L2GW/L3GW1 and L2GW/L3GW2 are connected to VNFs. VNF1 and VNF2 that function as virtualized NEs are deployed to implement the vUGW functions and vMSE functions, respectively. VNF1 and VNF2 are each connected to L2GW/L3GW1 and L2GW/L3GW2 through IPUs.

Figure 1 NFVI distributed gateway networking

Function Deployment

On the network shown in Figure 1, the number of BDs needs to be planned based on the number of network segments corresponding to each IPU. An example assumes that the four IP addresses planned for four IPUs belong to four network segments. In this case, four BDs need to be planned. You need to configure the BDs and the corresponding VBDIF interfaces on all L2GW/L3GWs and bind all the VBDIF interfaces to the same L3VPN instance. In addition, the following functions need to be deployed on the DCN:

  • Establish BGP VPN peer relationships between VNFs and DC-GWs so that the VNFs can advertise mobile phone routes (UE IP) to DC-GWs.

  • On L2GW/L3GW1 and L2GW/L3GW2, configure static VPN routes with the IP addresses of VNFs as the destination addresses and the IP addresses of IPUs as next-hop addresses.

  • Deploy EVPN RRs which can be either a standalone device or a DC-GW. In this section, BGP EVPN peer relationships are established between all L2GWs/L3GWs, PEs, and DC-GWs, DC-GWs are deployed as RRs to reflect EVPN routes, and L2GW/L3GWs function as RR clients. L2GW/L3GWs can use EVPN RRs to synchronize MAC or ARP routes as well as the IP prefix routes carrying a VNF address as the destination address with IPUs.

  • Configure static default routes on PEs and use the EVPN RRs to reflect the static default routes to L2GW/L3GWs.

  • Deploy SR tunnels between PEs and L2GW/L3GWs and between DC-GWs and L2GW/L3GWs to carry service traffic.

  • The traffic transmitted between mobile phones and the Internet over VNFs is north-south traffic. The traffic transmitted between VNF1 and VNF2 is east-west traffic. To achieve load balancing of east-west traffic and north-south traffic, deploy the load balancing function on DC-GWs and L2GW/L3GWs.

Establishment of Forwarding Entries

In the networking of an NFVI distributed gateway (E2E EVPN over E2E SR tunnels), the procedure for establishing forwarding entries on each device is as follows:

  1. On L2GW/L3GWs, the number of BDs is planned based on the number of network segments corresponding to the IPUs, the BDs are bound to the links connecting to the corresponding IPUs, and VBDIF interfaces are configured as the gateways of IPUs. Static VPN routes are configured on L2GW/L3GWs so that the forwarding entries with the destination address as a VNF's address, the next-hop address as an IPU's IP address, and outbound interface as the VBDIF interface can be established on L2GW/L3GWs.

    Figure 2 Forwarding entries of static routes on L2GW/L3GWs

  2. After static VPN routes destined for VNFs are configured on L2GW/L3GWs, these static VPN routes are imported to the BGP EVPN routing table and IP prefix routes are generated. These routes are sent to DC-GWs and PEs based on BGP EVPN peer relationships. To prevent these routes from being transmitted to DC-GWs or PEs and recursed to VBDIF interfaces (DC-GWs and PEs do not have VBDIF interfaces) because they carry gateway addresses, configure an import route-policy on DC-GWs and PEs to delete gateway addresses from the received routes.

    Figure 3 Forwarding entries on PEs and L2GW/L3GWs

  3. An L2GW/L3GW learns the MAC addresses and ARP information of IPUs through the data plane. Such information is advertised to another L2GW/L3GW through EVPN routes and can be used for establishment of ARP entries and MAC entries for Layer 2 forwarding. Taking L2GW/L3GW1 as an example, the destination MAC address of the MAC entries on L2GW/3GW1 is an IPU's MAC address. For the IPU directly connected to L2GW/L3GW1, the IPU's interface is used as the outbound interface in MAC entries. For the IPU connected to another L2GW/L3GW, the MAC entries contain the outbound interface for SR tunnels and the IP address of the BGP EVPN peer of this L2GW/L3GW as the next-hop address.

    L2GW/L3GWs exchange routes with a VNF's IP address as the destination IP address. If two VNFs are connected to different L2GW/L3GWs, traffic is forwarded over routes. Otherwise, route loops may occur. Therefore, a route-policy needs to be configured on L2GW/L3GWs, so that the Gateway IP attribute is added to the routes exchanged between L2GW/L3GWs. The Gateway IP attribute is still the next-hop address, which is an IPU's IP address, and the outbound interface is the VBDIF interface. In this manner, L2GW/L3GWs forward traffic based on the MAC forwarding table to prevent route loops.

    Because multiple links and static routes exist between L2GW/L3GWs and VNFs, to achieve load balancing, the Add-Path function needs to be enabled during configuration of importing static routes to the BGP EVPN routing table.

    Figure 4 MAC entries on L2GW/L3GWs

  4. To establish BGP VPN peer relationships between DC-GWs and VNFs, DC-GWs need to advertise the routes destined for loopback addresses to L2GW/L3GWs. After BGP VPN peer relationships are established between VNFs and DC-GWs, VNFs can send mobile phone routes to DC-GWs, and DC-GWs send the mobile phone routes as IP prefix routes to L2GW/L3GWs based on the BGP VPN peer relationships. The Gateway IP attribute, which is the IP address of a VNF, is added to the IP prefix routes based on a route-policy. Upon receipt of IP prefix routes, L2GW/L3GWs generate VPN forwarding entries. L2GW/L3GWs then send EVPN routes to PEs based on EBGP EVPN peer relationships so that PEs can generate VPN forwarding entries with the destination address as the IP address of mobile phone routes and the next-hop address as a VNF's IP address. The routes are then recursed to VNFs, and the outbound interface is an SR tunnel.

    Figure 5 Forwarding entries on PEs and L2GW/L3GWs

  5. Devices on the DCN do not need to get aware of external routes. Therefore, route-policies need to be configured on PEs to allow PEs to send only default routes to L2GW/L3GWs.

    Figure 6 Forwarding entries on PEs and L2GW/L3GWs

  6. PEs can exchange information about Internet routes, such as the Internet server address, with external devices.

  7. To achieve load balancing of east-west traffic and north-south traffic, the load balancing function and Add-Path function need to be deployed on PEs and L2GW/L3GWs.

    • Load balancing of north-south traffic: Taking PE1 in Figure 1 as an example, PE1 can receive the VPN routes destined for VNF2 from L2GW/L3GW1 and L2GW/L3GW2. By default, after the load balancing function is configured, PE1 sends half of the traffic destined for VNF2 through L2GW/L3GW1 and the other half of the traffic through L2GW/L3GW2. However, L2GW/L3GW1 connects to VNF2 over one link and L2GW/L3GW2 connects to VNF2 over two links, causing the load balancing function failed to achieve the desired effect. Therefore, the Add-Path function needs to be deployed on L2GW/L3GWs. After the Add-Path function is deployed on L2GW/L3GWs, L2GW/L3GW2 sends two routes with the same destination address to DC-GW1, achieving load balancing.

    • East-west traffic load balancing: Taking L2GW/L3GW1 in Figure 1 as an example, because the Add-Path function is deployed on L2GW/L3GW2, L2GW/L3GW1 receives two EVPN routes from L2GW/L3GW2 and L2GW/L3GW1 has a static route with the next-hop address as IPU3's IP address. The destination addresses of all these routes are VNF2's IP address. Therefore, the load balancing function needs to be configured to balance traffic over static routes and EVPN routes.

Traffic Forwarding Process

As shown in Figure 7, the procedure for forwarding north-south traffic from mobile phones to the Internet is as follows:

  1. Mobile phone traffic is sent to base stations (Nodes) and encapsulated with a GPRS tunneling protocol (GTP) header. The destination IP address of the GTP tunnel is a VNF's IP address. PEs send the encapsulated packets to L2GW/L3GWs over SR tunnels based on the VPN routing table.

  2. Upon receipt of the encapsulated packets, L2GW/L3GWs look for the VPN routing table and find that the next-hop addresses of the forwarding entries corresponding to VNFs' IP addresses are IPUs' IP addresses and the outbound interface is the VBDIF interface. Therefore, routes destined for the network segment corresponding to the VBDIF interface are hit. L2GW/L3GWs search the MAC address that belongs to the network segment in an ARP table, look for the MAC forwarding table based on the ARP information, and forward traffic to VNFs based on the MAC forwarding table.

  3. Upon receipt of packets, VNFs decapsulate the GTP tunnel header, search the routing table based on the destination IP address in the decapsulated packets, and forward the packets to L2GW/L3GWs based on the default gateways of VNFs.

  4. L2GW/L3GWs search for the VPN routing table on L2GW/L3GWs. The default routes advertised by PEs to L2GW/L3GWs are recursed over SR tunnels and forwarded to PEs after being encapsulated with a VPN label.

  5. PEs use the VPN forwarding table to forward the packets to the Internet based on the VPN label.

Figure 7 North-south traffic forwarding from mobile phones to the Internet
As shown in Figure 8, the procedure for forwarding east-west traffic from the Internet to mobile phones over VNFs is as follows:

  1. Devices on the Internet send mobile phones the reply packets whose destination IP addresses are the IP addresses of mobile phones. This is because mobile phone routes are advertised by L2GW/L3GWs to VNFs based on BGP VPNv4/v6 peer relationships and then advertised to the Internet by PEs. Therefore, reply packets must be first transmitted to L2GW/L3GWs.

  2. Upon receipt of reply packets, PEs search the VPN routing table for the forwarding entries corresponding to mobile phone routes whose next-hop addresses are L2GW/L3GWs' IP addresses and the outbound interface is the one for SR tunnels. The reply packets are sent to L2GW/L3GWs after being encapsulated with a VPN label and an SR label.

  3. Upon receipt of reply packets, L2GW/L3GWs find the VPN forwarding entries based on the VPN label and the VBDIF interface based on the VPN forwarding entries. The packets are then forwarded to VNFs based on the MAC entries corresponding to the VBDIF interface.

  4. Upon receipt of the reply packets, VNFs search for the base stations corresponding to the destination IP addresses of mobile phones and add a tag of tunnel information with the destination IP address as the IP address of a base station. The reply packets are then forwarded to L2GW/L3GWs based on default gateways.

  5. Upon receipt of the reply packets, L2GW/L3GWs search the VPN routing table, and the default routes advertised by DC-GWs to L2GW/L3GWs are hit. The reply packets are then forwarded to DC-GWs over SR tunnels after being encapsulated with a VPN label.

  6. Upon receipt of reply packets, PEs use the VPN forwarding table to forward the packets to base stations based on the VPN label. The base stations decapsulate the packets before forwarding them to mobile phones.

Figure 8 East-west traffic forwarding from the Internet to mobile phones
Upon receipt of user packets, a VNF finds that the packets need to be sent to another VNF for value-added service processing. In this case, east-west traffic occurs. On the network shown in Figure 9, the difference between forwarding east-west traffic and forwarding north-south traffic lies in the processing of packets after they arrive VNF1.

  1. Upon receipt of user packets, VNF1 finds that the packets need to be processed by VNF2 and then adds a tunnel label with the destination IP address as VNF2's IP address. The user packets are sent to L2GW/L3GWs based on default routes.

  2. Upon receipt of user packets, an L2GW/L3GW searches the VPN forwarding table and finds that multiple load-balancing forwarding entries exist. In some entries, the outbound interface is an IPU or the next-hop address is the IP address of another L2GW/L3GW.

  3. If the traffic hits the path of another L2GW/L3GW, an EVPN label is added to the user packets and the routes are recursed to L2GW/L3GW2 over an SR tunnel. L2GW/L3GW2 searches for the BD and destination MAC address based on the EVPN label before forwarding the packets to VNF2.

  4. Upon receipt of the user packets, VNF2 processes and forwards the packets to the server. The subsequent forwarding follows the north-south traffic forwarding procedure.

Figure 9 East-west traffic forwarding from VNF1 to VNF2
Parent Topic: Application Scenarios for EVPN
Copyright © Huawei Technologies Co., Ltd.
Copyright © Huawei Technologies Co., Ltd.
< Previous topic Next topic >