BGP Additional Paths

BGP routers only advertise the best path to their neighbors. When a better path is found, it replaces the current path. Advertising a path and replacing it with a new path is called an implicit withdraw.

Since we only advertise the best path, a lot of other possible paths are unknown to some of the routers. We call this path hiding.

Path hiding has a couple of disadvantages:

  • You can’t use BGP multipath
  • Possible sub-optimal routing
  • MED oscillation
  • Next hop failure means BGP has to reconverge before traffic can be forwarded again

With the BGP additional paths feature, BGP advertises multiple paths for the same prefix so we have path diversity instead of path hiding. Additional paths is an extension to BGP and it adds a unique path identifier to each path. The path ID is similar to how the RD (Route Distinguisher) works in MPLS VPN, except the path IDs can be used in any BGP address family.

This feature can only be used for iBGP and there are three steps to take to implement it:

  1. Configure your routers so they can send and/or receive additional paths.
  2. Configure a global selection criterion on the router to define which additional candidate paths should be selected next to the best path.
  3. To each neighbor, advertise a set or sets of additional paths that we selected as candidate paths.

For the additional path selection, there are three options:

  • Best N: this is the best path and the second best path(s). The second best path(s) is chosen by eliminating the (next) best path and then selecting the next best path.
  • Group-best: the set of paths that are the best paths from each AS. This is best explained with a quick example:
    • AS 1 advertises three paths for prefix 1.1.1.1/32:
      • One with next hop 192.168.1.1
      • One with next hop 192.168.1.2
      • One with next hop 192.168.1.3
    • AS 2 advertises three paths for prefix 2.2.2.2/32:
      • One with next hop 192.168.2.1
      • One with next hop 192.168.2.2
      • One with next hop 192.168.2.3
    • AS 3 advertises three paths for prefix 3.3.3.3/32:
      • One with next hop 192.168.3.1
      • One with next hop 192.168.3.2
      • One with next hop 192.168.3.3
    • We choose the best path from each AS and that becomes our set. This could be:
      • 1.1.1.1/32 with next hop 192.168.1.1
      • 2.2.2.2/32 with next hop 192.168.2.1
      • 3.3.3.3/32 with next hop 192.168.3.1
  • All: all paths that have a unique next hop can be used as an additional path.

Let’s take a look at the additional paths feature in action.

Configuration



We’ll use the following topology:

Bgp Additional Paths Topology

We have five routers in AS 12345 where R2 is the RR (Route Reflector).  R6 has a loopback interface with prefix 6.6.6.6/32 that is advertised in BGP.

  • Configurations
  • R1
  • R2
  • R3
  • R4
  • R5
  • R6

Want to take a look for yourself? Here you will find the startup configuration of each device.

Let’s see what we have. R4 and R5 have learned about 6.6.6.6/32 from R6:

R4#show ip bgp | begin Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>   6.6.6.6/32       192.168.46.6             0             0 6 i
R5#show ip bgp | begin Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>   6.6.6.6/32       192.168.56.6             0             0 6 i
 * i                   4.4.4.4                  0    100      0 6 i

R4 and R5 advertise 6.6.6.6/32 to R2, our route reflector:

R2#show ip bgp | begin Network
     Network          Next Hop            Metric LocPrf Weight Path
 * i  6.6.6.6/32       5.5.5.5                  0    100      0 6 i
 *>i                   4.4.4.4                  0    100      0 6 i

R2 has both paths in its BGP table but installs the path to R4. It only advertises this best path to its clients. Let’s look at two scenarios where this might be an issue.

Load Balancing

Let’s take a look at R1:

R1#show ip bgp | begin Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>i  6.6.6.6/32       4.4.4.4                  0    100      0 6 i

Since R2 only advertises its best path, R1 only knows about the path through R4. How is the next hop resolved? Let’s have a look:

R1#show ip route 4.4.4.4
Routing entry for 4.4.4.4/32
  Known via "ospf 1", distance 110, metric 3, type intra area
  Last update from 192.168.12.2 on GigabitEthernet0/1, 01:15:01 ago
  Routing Descriptor Blocks:
  * 192.168.12.2, from 4.4.4.4, 01:15:01 ago, via GigabitEthernet0/1
      Route metric is 3, traffic share count is 1

4.4.4.4 resolves to R2 so R1 will never use the path through R3. Because R2 is “hiding” the path, we have two disadvantages here:

  • There is no load balancing, R1 could have used R3 to get to 6.6.6.6/32.
  • When next hop 4.4.4.4 fails, BGP has to reconverge before traffic can resume.

Sub-optimal Routing

Let’s check R3:

R3#show ip bgp | begin Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>i  6.6.6.6/32       4.4.4.4                  0    100      0 6 i

R3 also uses 4.4.4.4 as the next hop to get to 6.6.6.6. How does 4.4.4.4 get resolved?

R3#show ip route 4.4.4.4 | include via
  Known via "ospf 1", distance 110, metric 3, type intra area
    192.168.35.5, from 4.4.4.4, 01:13:48 ago, via GigabitEthernet0/3
  * 192.168.23.2, from 4.4.4.4, 01:16:15 ago, via GigabitEthernet0/2

Also, when 4.4.4.4 fails, BGP has to reconverge. What we have seen so far is just the way it is; this is how BGP works. Can we improve it though?

BGP Additional Paths

Let’s enable the additional-paths feature. I’ll start with R2. Let’s take a look at the command:

R2(config)#router bgp 12345
R2(config-router)#address-family ipv4
R2(config-router-af)#bgp additional-paths ?
  install  Additional paths to install into RIB
  receive  Receive additional paths from neighbors
  select   Selection criteria to pick the paths
  send     Send additional paths to neighbors

We enable this for the address-family, IPv4 in our case. There are a couple of options. Let’s start by enabling the extension itself:

R2(config-router-af)#neighbor 1.1.1.1 additional-paths ?
  disable  Disable additional paths for this neighbor
  receive  Receive additional paths from neighbors
  send     Send additional paths to this neighbor

For each neighbor, you can define if you want to send and/or receive additional paths. I’ll configure R2 to send additional paths to both R1 and R3:

R2(config-router-af)#neighbor 1.1.1.1 additional-paths send
R2(config-router-af)#neighbor 3.3.3.3 additional-paths send

The next thing to do is configure R2 how to globally select additional paths with the bgp additional-paths select command:

R2(config-router-af)#bgp additional-paths select ?
  all            Select all available paths
  backup         Select backup path
  best           Select best N paths
  best-external  Select best-external path
  group-best     Select group-best path

We’ll keep it simple and tell R2 to use all available paths as additional paths:

R2(config-router-af)#bgp additional-paths select all

After entering this command, there is a change in the BGP table:

R2#show ip bgp      
BGP table version is 11, local router ID is 2.2.2.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 * ia 6.6.6.6/32       5.5.5.5                  0    100      0 6 i
 *>i                   4.4.4.4                  0    100      0 6 i

Note the a for an additional path. This does not affect the routing table and/or forwarding table for R2:

R2#show ip route bgp

      6.0.0.0/32 is subnetted, 1 subnets
B        6.6.6.6 [200/0] via 4.4.4.4, 00:01:0

We still see only a single route here. Nothing gets installed. We are almost there, the last thing to do on R2 is tell it which additional paths (that we previously selected globally) we want to advertise to our neighbors:

R2(config-router-af)#neighbor 1.1.1.1 advertise additional-paths ?
  all         Select all available paths
  best        Select best N paths
  group-best  Select group-best paths

Above, you can see that we only have three options here. Let’s keep it simple and configure R2 to advertise all additional paths:

R2(config-router-af)#neighbor 1.1.1.1 advertise additional-paths all
R2(config-router-af)#neighbor 3.3.3.3 advertise additional-paths all

That’s all we have to do on R2.

R1 and R3 have to be configured to receive the additional paths:

R1 & R3
(config)#router bgp 12345 
(config-router)#neighbor 2.2.2.2 additional-paths receive

Let’s take a look at R1 again:

R1#show ip bgp

     Network          Next Hop            Metric LocPrf Weight Path
 *>i  6.6.6.6/32       4.4.4.4                  0    100      0 6 i
 * i                   5.5.5.5                  0    100      0 6 i

R1 has now two options so that tells us that R2 advertises two paths. R1 has selected 4.4.4.4 as the next hop but we could use 5.5.5.5 as well. Let’s take a closer look at the BGP table for prefix 6.6.6.6/32:

R1#show ip bgp 6.6.6.6
BGP routing table entry for 6.6.6.6/32, version 2
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Refresh Epoch 1
  6
    4.4.4.4 (metric 3) from 2.2.2.2 (2.2.2.2)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      Originator: 4.4.4.4, Cluster list: 2.2.2.2
      rx pathid: 0x0, tx pathid: 0x0
  Refresh Epoch 1
  6
    5.5.5.5 (metric 3) from 2.2.2.2 (2.2.2.2)
      Origin IGP, metric 0, localpref 100, valid, internal
      Originator: 5.5.5.5, Cluster list: 2.2.2.2
      rx pathid: 0x1, tx pathid: 0

Above, you can see the received path ID. These are two different values which make each path unique. You now could configure R1 to use multipath so that it used both paths, not a bad idea but since I already did this in the multipath lesson, we’ll try something else here. Let’s configure the path through R3 as a backup path.

How and why to use backup paths in BGP is explained in detail in the BGP PIC (Prefix Independent Convergence lesson).

Two commands are needed, first the bgp additional-paths select all command to tell R1 which additional paths to use and the second command to actually install the additional path:

R1(config)#router bgp 12345
R1(config-router)#address-family ipv4
R1(config-router-af)#bgp additional-paths select all
R1(config-router-af)#bgp additional-paths install

Let’s check the BGP table again:

R1#show ip bgp 6.6.6.6
BGP routing table entry for 6.6.6.6/32, version 13
Paths: (2 available, best #1, table default)
  Additional-path-install
  Path not advertised to any peer
  Refresh Epoch 1
  6
    4.4.4.4 (metric 3) from 2.2.2.2 (2.2.2.2)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      Originator: 4.4.4.4, Cluster list: 2.2.2.2
      rx pathid: 0x0, tx pathid: 0x0
  Path not advertised to any peer
  Refresh Epoch 1
  6
    5.5.5.5 (metric 3) from 2.2.2.2 (2.2.2.2)
      Origin IGP, metric 0, localpref 100, valid, internal, backup/repair, all
      Originator: 5.5.5.5, Cluster list: 2.2.2.2
      rx pathid: 0x1, tx pathid: 0x1

Above, we see that the second path is now a backup/repair path. We can also verify this in the CEF table:

R1#show ip cef 6.6.6.6 detail 
6.6.6.6/32, epoch 0, flags [rib only nolabel, rib defined all labels]
  recursive via 4.4.4.4
    nexthop 192.168.12.2 GigabitEthernet0/1
  recursive via 5.5.5.5, repair
    nexthop 192.168.13.3 GigabitEthernet0/2

This is very nice. When our next hop 4.4.4.4 fails, we can install 5.5.5.5 right away without having to wait for BGP to reconverge.

What about R3? Let’s have a look:

R3#show ip bgp 

     Network          Next Hop            Metric LocPrf Weight Path
 * i  6.6.6.6/32       4.4.4.4                  0    100      0 6 i
 *>i                   5.5.5.5                  0    100      0 6 i

Because R3 has received the additional path, it’s now able to make better choices. It installed 5.5.5.5 as the next hop.

It’s not a bad idea to configure R3 to use the other path as a backup/repair path:

R3(config)#router bgp 12345
R3(config-router)#address-family ipv4
R3(config-router-af)#bgp additional-paths select all
R3(config-router-af)#bgp additional-paths install

Let’s take a quick look at the BGP and CEF table:

R3#show ip bgp 6.6.6.6
BGP routing table entry for 6.6.6.6/32, version 20
Paths: (2 available, best #2, table default)
  Additional-path-install
  Path not advertised to any peer
  Refresh Epoch 2
  6, (Received from a RR-client)
    4.4.4.4 (metric 3) from 2.2.2.2 (2.2.2.2)
      Origin IGP, metric 0, localpref 100, valid, internal, backup/repair, all
      Originator: 4.4.4.4, Cluster list: 2.2.2.2
      rx pathid: 0x0, tx pathid: 0x1
  Path not advertised to any peer
  Refresh Epoch 2
  6, (Received from a RR-client)
    5.5.5.5 (metric 2) from 2.2.2.2 (2.2.2.2)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      Originator: 5.5.5.5, Cluster list: 2.2.2.2
      rx pathid: 0x1, tx pathid: 0x0
R3#show ip cef 6.6.6.6 detail
6.6.6.6/32, epoch 0, flags [rib only nolabel, rib defined all labels]
  recursive via 5.5.5.5
    nexthop 192.168.35.5 GigabitEthernet0/3
  recursive via 4.4.4.4, repair
    nexthop 192.168.23.2 GigabitEthernet0/2
    nexthop 192.168.35.5 GigabitEthernet0/3

That’s looking good, R5 is now our best path and R4 is our backup path.

Conclusion

You have now learned about BGP additional paths:

  • BGP only advertises the best path to neighbors, when a better path is found, it advertises the new path and replaces the old one. This is called implicit withdrawing.
  • Not advertising all paths is also called path hiding.
  • Path hiding has some disadvantages:
    • Can’t use BGP multipath
    • The possibility of sub-optimal routing
    • MED oscillation
    • Next hop failure means BGP has to reconverge
  • BGP additional paths let you advertise additional paths next to the best path.
  • This leads to path diversity instead of path hiding.
  • Each path gets a unique path identifier.
  • BGP additional paths are only available to iBGP.
  • There are three things needed for additional paths:
    • Configure the router to send and/or receive additional paths.
    • Configure a global selection criterion for additional paths.
    • To each neighbor, advertise a set or sets of additional paths that we selected as candidate paths.
  • There are three options for additional paths selection:
    • Best N
    • Group Best
    • All

I hope you enjoyed this lesson. If you have any questions feel free to leave a comment!

Unit 1: Introduction to BGP

Unit 2: BGP Neighbor Adjacency

Unit 3: BGP Attributes

Unit 4: BGP Communities

Unit 5: BGP Filtering

Unit 6: Advanced BGP Features

Unit 7: BGP Convergence