One of the more frustrating things that comes from studying BGP is that it isn’t always obvious why it is selecting one path over another. Fortunately, I came across a cool new BGP feature in modern IOS-XE that explains exactly that!
This can be particularly useful for CCIE study while your working through crazy practice labs.
Topology / Setup
In this post we will simply have 4 routers connected in a line with two connections per router. We will also add a few loopbacks on each router to give us some flexibility.
I’m using CSR1000v for my routers and they are on 16.11.1a firmware.
R01
R01(config)#interface GigabitEthernet2.12 R01(config-subif)# encapsulation dot1Q 12 R01(config-subif)# ip address 10.1.2.1 255.255.255.0 R01(config-subif)#interface GigabitEthernet2.21 R01(config-subif)# encapsulation dot1Q 21 R01(config-subif)# ip address 10.2.1.1 255.255.255.0 R01(config)#interface Loopback10 R01(config-if)# ip address 192.168.10.1 255.255.255.0 R01(config-if)#interface Loopback11 R01(config-if)# ip address 192.168.11.1 255.255.255.0 R01(config-if)#interface Loopback12 R01(config-if)# ip address 192.168.12.1 255.255.255.0 R01(config-if)#interface Loopback13 R01(config-if)# ip address 192.168.13.1 255.255.255.0 R01(config-if)#interface Loopback14 R01(config-if)# ip address 192.168.14.1 255.255.255.0 R01(config-if)#interface Loopback15 R01(config-if)# ip address 192.168.15.1 255.255.255.0 R01(config)#router bgp 65101 R01(config-router)#neighbor 10.1.2.2 remote 65102 R01(config-router)#neighbor 10.2.1.2 remote 65102 R01(config-router)# redistribute connected
R02
R02(config)#interface GigabitEthernet2.12 R02(config-subif)# encapsulation dot1Q 12 R02(config-subif)# ip address 10.1.2.2 255.255.255.0 R02(config-subif)#interface GigabitEthernet2.21 R02(config-subif)# encapsulation dot1Q 21 R02(config-subif)# ip address 10.2.1.2 255.255.255.0 R02(config-subif)#interface GigabitEthernet2.23 R02(config-subif)# encapsulation dot1Q 23 R02(config-subif)# ip address 10.2.3.2 255.255.255.0 R02(config-subif)#interface GigabitEthernet2.32 R02(config-subif)# encapsulation dot1Q 32 R02(config-subif)# ip address 10.3.2.2 255.255.255.0 R02(config)#interface Loopback20 R02(config-if)# ip address 192.168.20.1 255.255.255.0 R02(config-if)#interface Loopback21 R02(config-if)# ip address 192.168.21.1 255.255.255.0 R02(config-if)#interface Loopback22 R02(config-if)# ip address 192.168.22.1 255.255.255.0 R02(config-if)#interface Loopback23 R02(config-if)# ip address 192.168.23.1 255.255.255.0 R02(config-if)#interface Loopback24 R02(config-if)# ip address 192.168.24.1 255.255.255.0 R02(config-if)#interface Loopback25 R02(config-if)# ip address 192.168.25.1 255.255.255.0 R02(config)# router bgp 65102 R02(config-router)# bgp log-neighbor-changes R02(config-router)# neighbor 10.1.2.1 remote-as 65101 R02(config-router)# neighbor 10.2.1.1 remote-as 65101 R02(config-router)# neighbor 10.2.3.3 remote-as 65103 R02(config-router)# neighbor 10.3.2.3 remote-as 65103 R02(config-router)# redistribute connected
R03
R03(config)#interface GigabitEthernet2.23 R03(config-subif)# encapsulation dot1Q 23 R03(config-subif)# ip address 10.2.3.3 255.255.255.0 R03(config-subif)#interface GigabitEthernet2.32 R03(config-subif)# encapsulation dot1Q 32 R03(config-subif)# ip address 10.3.2.3 255.255.255.0 R03(config-subif)#interface GigabitEthernet2.34 R03(config-subif)# encapsulation dot1Q 34 R03(config-subif)# ip address 10.3.4.3 255.255.255.0 R03(config-subif)#interface GigabitEthernet2.43 R03(config-subif)# encapsulation dot1Q 43 R03(config-subif)# ip address 10.4.3.3 255.255.255.0 R03(config)#interface Loopback30 R03(config-if)# ip address 192.168.30.1 255.255.255.0 R03(config-if)#interface Loopback31 R03(config-if)# ip address 192.168.31.1 255.255.255.0 R03(config-if)#interface Loopback32 R03(config-if)# ip address 192.168.32.1 255.255.255.0 R03(config-if)#interface Loopback33 R03(config-if)# ip address 192.168.33.1 255.255.255.0 R03(config-if)#interface Loopback34 R03(config-if)# ip address 192.168.34.1 255.255.255.0 R03(config-if)#interface Loopback35 R03(config-if)# ip address 192.168.35.1 255.255.255.0 R03(config)#router bgp 65103 R03(config-router)# bgp log-neighbor-changes R03(config-router)# neighbor 10.2.3.2 remote-as 65102 R03(config-router)# neighbor 10.3.2.2 remote-as 65102 R03(config-router)# neighbor 10.3.4.4 remote-as 65104 R03(config-router)# neighbor 10.4.3.4 remote-as 65104 R03(config-router)# redistribute connected
R04
R04(config)#interface GigabitEthernet2.34 R04(config-subif)# encapsulation dot1Q 34 R04(config-subif)# ip address 10.3.4.4 255.255.255.0 R04(config-subif)#interface GigabitEthernet2.43 R04(config-subif)# encapsulation dot1Q 43 R04(config-subif)# ip address 10.4.3.4 255.255.255.0 R04(config)#interface Loopback40 R04(config-if)# ip address 192.168.40.1 255.255.255.0 R04(config-if)#interface Loopback41 R04(config-if)# ip address 192.168.41.1 255.255.255.0 R04(config-if)#interface Loopback42 R04(config-if)# ip address 192.168.42.1 255.255.255.0 R04(config-if)#interface Loopback43 R04(config-if)# ip address 192.168.43.1 255.255.255.0 R04(config-if)#interface Loopback44 R04(config-if)# ip address 192.168.44.1 255.255.255.0 R04(config-if)#interface Loopback45 R04(config-if)# ip address 192.168.45.1 255.255.255.0 R04(config)# router bgp 65104 R04(config-router)# bgp log-neighbor-changes R04(config-router)# neighbor 10.3.4.3 remote-as 65103 R04(config-router)# neighbor 10.4.3.3 remote-as 65103 R04(config-router)# redistribute connected
A Reminder on BGP Path Selection
BGP has a rather complex path selection, here is a reminder for how it works while we play with some examples.
- Prefer the path with the highest WEIGHT.
- Prefer the path with the highest Local Preference
- Prefer the path that was locally originated via a network or aggregate BGP subcommand or through redistribution from an IGP.
- Prefer the path with the shortest AS_PATH.
- Prefer the path with the lowest origin type.
- Prefer the path with the lowest Multi-Exit Discriminator
- Prefer eBGP over iBGP paths.
- Prefer the path with the lowest IGP metric to the BGP next hop.
- Determine if multiple paths require installation in the routing table for BGP Multipath. Continue, if bestpath is not yet selected.
- When both paths are external, prefer the path that was received first (the oldest one).
- Prefer the route that comes from the BGP router with the lowest router ID.
- If the originator or router ID is the same for multiple paths, prefer the path with the minimum cluster list length.
- Prefer the path that comes from the lowest neighbor address.
The Fun Stuff
Let’s have a look at the BGP table and focus on the 192.168.4x.0/24 routes
R01#show bgp ipv4 unicast 192.168.40.0/21 longer-prefixes BGP table version is 42, local router ID is 192.168.255.1 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, x best-external, a additional-path, c RIB-compressed, t secondary path, L long-lived-stale, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found Network Next Hop Metric LocPrf Weight Path * 192.168.40.0 10.2.1.2 0 65102 65103 65104 ? *> 10.1.2.2 0 65102 65103 65104 ? * 192.168.41.0 10.2.1.2 0 65102 65103 65104 ? *> 10.1.2.2 0 65102 65103 65104 ? * 192.168.42.0 10.2.1.2 0 65102 65103 65104 ? *> 10.1.2.2 0 65102 65103 65104 ? * 192.168.43.0 10.2.1.2 0 65102 65103 65104 ? *> 10.1.2.2 0 65102 65103 65104 ? * 192.168.44.0 10.2.1.2 0 65102 65103 65104 ? *> 10.1.2.2 0 65102 65103 65104 ? * 192.168.45.0 10.2.1.2 0 65102 65103 65104 ? *> 10.1.2.2 0 65102 65103 65104 ?
To use the fancy best-path-reason feature, we simply need to add it to the end of the bgp command, yes you can also use it if you are still using show ip bgp!
To try this out we’ll look at the 192.168.40.0/24 route
We can see that 10.1.2.2 is the best path, if we look at the 10.1.2.1 we can see it lost because the route had a higher peer address which is the 13th step in the path selection.
R01#show bgp ipv4 unicast 192.168.40.0/24 best-path-reason BGP routing table entry for 192.168.40.0/24, version 33 Paths: (2 available, best #2, table default) Advertised to update-groups: 1 Refresh Epoch 1 65102 65103 65104 10.2.1.2 from 10.2.1.2 (192.168.255.2) Origin incomplete, localpref 100, valid, external rx pathid: 0, tx pathid: 0 Updated on Jun 4 2019 05:35:52 UTC Best Path Evaluation: Higher peer address Refresh Epoch 1 65102 65103 65104 10.1.2.2 from 10.1.2.2 (192.168.255.2) Origin incomplete, localpref 100, valid, external, best rx pathid: 0, tx pathid: 0x0 Updated on Jun 4 2019 05:35:52 UTC Best Path Evaluation: Overall best path
We’ll work through a few examples so we can see how the feature works!
Weight
We might as well start with the HEAVIEST metric (not sorry!)
This will require a route-map that matches a loopback, I’ll use 192.168.40.0/24 and apply it to the 10.1.2.2 neighbor on R01
R01(config)#ip prefix-list PL_NET40 permit 192.168.40.0/24 R01(config)#route-map RM_BGP_IN permit R01(config-route-map)#match ip address prefix PL_NET40 R01(config-route-map)#set weight 500 R01(config-route-map)#route-map RM_BGP_IN permit 20 R01(config-route-map)#exit R01(config)#router bgp 65101 R01(config-router)#neighbor 10.1.2.2 route-map RM_BGP_IN in R01(config-router)#end R01# *Jun 4 06:04:57.876: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (192.168.106.1) R01#clear ip bgp * soft
Now when we look at the BGP output again, we can see that 10.2.1.2 is losing because it has a lower weight!
R01#show bgp ipv4 unicast 192.168.40.0/24 best-path-reason BGP routing table entry for 192.168.40.0/24, version 43 Paths: (2 available, best #2, table default) Advertised to update-groups: 1 Refresh Epoch 2 65102 65103 65104 10.2.1.2 from 10.2.1.2 (192.168.255.2) Origin incomplete, localpref 100, valid, external rx pathid: 0, tx pathid: 0 Updated on Jun 4 2019 06:05:26 UTC Best Path Evaluation: Lower weight Refresh Epoch 2 65102 65103 65104 10.1.2.2 from 10.1.2.2 (192.168.255.2) Origin incomplete, localpref 100, weight 500, valid, external, best rx pathid: 0, tx pathid: 0x0 Updated on Jun 4 2019 06:05:26 UTC Best Path Evaluation: Overall best path
Local Preference
Next up is my personal preference (ok sorry!)
R01(config)#ip prefix-list PL_NET_41 permit 192.168.41.0/24 R01(config)#no route-map RM_BGP_IN R01(config)#route-map RM_BGP_IN permit 10 R01(config-route-map)#match ip address prefix PL_NET_41 R01(config-route-map)#set local R01(config-route-map)#set local-preference 5000 R01(config-route-map)#route-map RM_BGP_IN permit 20 R01(config-route-map)#exit R01(config)#do clear ip bgp * soft R01(config)#^Z R01# *Jun 4 06:16:10.026: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (192.168.106.1)
This time we can see 10.2.1.2 is losing because of a lower local preference.
R01#show bgp ipv4 unicast 192.168.41.0/24 best-path-reason BGP routing table entry for 192.168.41.0/24, version 45 Paths: (2 available, best #2, table default) Advertised to update-groups: 1 Refresh Epoch 3 65102 65103 65104 10.2.1.2 from 10.2.1.2 (192.168.255.2) Origin incomplete, localpref 100, valid, external rx pathid: 0, tx pathid: 0 Updated on Jun 4 2019 06:16:05 UTC Best Path Evaluation: Lower local preference Refresh Epoch 4 65102 65103 65104 10.1.2.2 from 10.1.2.2 (192.168.255.2) Origin incomplete, localpref 5000, valid, external, best rx pathid: 0, tx pathid: 0x0 Updated on Jun 4 2019 06:16:05 UTC Best Path Evaluation: Overall best path
Origin
Maybe I should have done this one first!
R01(config)#ip prefix-list PL_NET_43 permit 192.168.43.0/24 R01(config)#no route-map RM_BGP_IN R01(config)#route-map RM_BGP_IN R01(config-route-map)#match ip address prefix-list PL_NET_43 R01(config-route-map)# set origin igp R01(config-route-map)#route-map RM_BGP_IN permit 20 R01(config-route-map)#do clear ip bgp * soft
This time round, 10.2.1.2 loses because the origin is incomplete since I redistributed connected routes.
R01(config-route-map)#do show bgp ipv4 unicast 192.168.43.0/24 best-path-reason BGP routing table entry for 192.168.43.0/24, version 52 Paths: (2 available, best #2, table default) Advertised to update-groups: 1 Refresh Epoch 13 65102 65103 65104 10.2.1.2 from 10.2.1.2 (192.168.255.2) Origin incomplete, localpref 100, valid, external rx pathid: 0, tx pathid: 0 Updated on Jun 4 2019 06:30:54 UTC Best Path Evaluation: Less favorable origin Refresh Epoch 18 65102 65103 65104 10.1.2.2 from 10.1.2.2 (192.168.255.2) Origin IGP, localpref 100, valid, external, best rx pathid: 0, tx pathid: 0x0 Updated on Jun 4 2019 06:30:54 UTC Best Path Evaluation: Overall best path
AS-Path
At this point we are on the path to working through all the steps!
R01(config)#ip prefix-list PL_NET_42 seq 5 permit 192.168.42.0/24 R01(config)#no route-map RM_BGP_IN R01(config)#route-map RM_BGP_IN R01(config-route-map)#match ip address prefix-list PL_NET_42 R01(config-route-map)#set as-path prepend last-as 3 R01(config)#route-map RM_BGP_IN permit 20 R01(config-route-map)#do clear ip bgp * soft
This time it loses because of the longer AS path.
R01(config-route-map)#do show bgp ipv4 unicast 192.168.42.0/24 best-path-reason BGP routing table entry for 192.168.42.0/24, version 76 Paths: (2 available, best #2, table default) Advertised to update-groups: 1 Refresh Epoch 21 65102 65102 65102 65102 65103 65104 10.1.2.2 from 10.1.2.2 (192.168.255.2) Origin incomplete, localpref 100, valid, external rx pathid: 0, tx pathid: 0 Updated on Jun 4 2019 06:43:20 UTC Best Path Evaluation: Longer AS path Refresh Epoch 14 65102 65103 65104 10.2.1.2 from 10.2.1.2 (192.168.255.2) Origin incomplete, localpref 100, valid, external, best rx pathid: 0, tx pathid: 0x0 Updated on Jun 4 2019 06:43:20 UTC Best Path Evaluation: Overall best path
MED
I’m gonna have to go on meds after all these puns!
R01(config)#ip prefix-list PL_NET_44 seq 5 permit 192.168.44.0/24 R01(config)#no route-map RM_BGP_IN R01(config)#route-map RM_BGP_IN R01(config-route-map)#match ip address prefix PL_NET_44 R01(config-route-map)#set metric 50 R01(config-route-map)#route-map RM_BGP_IN permit 20 R01(config-route-map)#do clear ip bgp * soft
By default, a missing metric is the best metric so us adding a low metric in the route-map actually did the opposite of what you might have expected.
R01(config-route-map)#do show bgp ipv4 unicast 192.168.44.0/24 best-path-reason BGP routing table entry for 192.168.44.0/24, version 109 Paths: (2 available, best #2, table default) Advertised to update-groups: 1 Refresh Epoch 24 65102 65103 65104 10.1.2.2 from 10.1.2.2 (192.168.255.2) Origin incomplete, metric 50, localpref 100, valid, external rx pathid: 0, tx pathid: 0 Updated on Jun 4 2019 06:50:05 UTC Best Path Evaluation: Higher MED Refresh Epoch 15 65102 65103 65104 10.2.1.2 from 10.2.1.2 (192.168.255.2) Origin incomplete, localpref 100, valid, external, best rx pathid: 0, tx pathid: 0x0 Updated on Jun 4 2019 06:44:04 UTC Best Path Evaluation: Overall best path
We can flip it around by setting med to consider no value as the worst metric.
R01(config-route-map)#router bgp 65101 R01(config-router)#bgp bestpath med missing-as-worst R01(config-router)# R01(config-router)#do show bgp ipv4 unicast 192.168.44.0/24 best-path-reason BGP routing table entry for 192.168.44.0/24, version 156 BGP Bestpath: med Paths: (2 available, best #1, table default) Advertised to update-groups: 1 Refresh Epoch 25 65102 65103 65104 10.1.2.2 from 10.1.2.2 (192.168.255.2) Origin incomplete, metric 50, localpref 100, valid, external, best rx pathid: 0, tx pathid: 0x0 Updated on Jun 4 2019 06:50:47 UTC Best Path Evaluation: Overall best path Refresh Epoch 16 65102 65103 65104 10.2.1.2 from 10.2.1.2 (192.168.255.2) Origin incomplete, metric 4294967295, localpref 100, valid, external rx pathid: 0, tx pathid: 0 Updated on Jun 4 2019 06:50:47 UTC Best Path Evaluation: Higher MED
The Rest?
After MED, BGP really just wants to pick a winner such as the peer IP we saw in the first example.
Wrapping up
For this post we looked at the best-path-reason feature and it is pretty neat!