Determining exact route when using ECMP

Many enterprise Cisco networks take advantage of multiple, redundant routed paths between routers or layer-3 switches. Certain routing protocols make it easy to not only achieve redundancy, but also higher throughput by utilizing both links via equal cost multipath (ECMP). This means that the routing table contains multiple entries, with same administrative distance and same metric, for the same network. The router then balances traffic between the two links using CEF (Cisco Express Forwarding). OSPF, paired with a symmetrical design, lends itself well to ECMP links, whereas EIGRP may require variance to be configured in order to allow for ECMP.

This works great, but because of this, the normal troubleshooting commands to find the deterministic path of a packet are somewhat ambiguous. Traditionally in a non-ECMP environment, even if there are multiple routes to a destination, the best route wins and is inserted into the routing table. Therefore, a simple "show ip route" can give you the path over which a particular flow is leaving the router. However, when there are more than one path in the routing table for a particular network, "show ip route" will show both. If you need to know for sure which link is being used, there is a way to do this.

In IOS, the way to determine the exact route a particular flow is taking, is to utilize the command "show ip cef exact-route". This command will take a source IP, destination IP, and give the outgoing interface from which the packet will leave. Below is an example:

TestLab-6500#show ip route 172.16.0.5 Routing entry for 172.16.0.5/32   Known via "ospf 1", distance 110, metric 21, type intra area   Last update from 10.0.35.74 on TenGigabitEthernet4/1, 11:16:48 ago   Routing Descriptor Blocks:   * 10.0.35.74, from 172.16.0.5, 11:16:48 ago, via TenGigabitEthernet1/1       Route metric is 21, traffic share count is 1     10.0.35.72, from 172.16.0.5, 11:16:48 ago, via TenGigabitEthernet2/1       Route metric is 21, traffic share count is 1 TestLab-6500# TestLab-6500#show ip cef exact-route 10.0.0.10 172.16.0.5 10.0.0.10 -> 172.16.0.5 => IP adj out of TenGigabitEthernet1/1, addr 10.0.35.74 TestLab-6500#

For those of you who have found the Nexus line to be fitting for L3 switching, the same basic functionality is there in NX-OS. The command syntax has changed slightly -- "show routing hash":

N7K-R1# show ip route 172.16.0.5 IP Route Table for VRF "default" '*' denotes best ucast next-hop '**' denotes best mcast next-hop '[x/y]' denotes [preference/metric] 172.16.0.5/32, ubest/mbest: 2/0     *via 10.0.4.121, Eth2/1, [110/31], 5d05h, ospf-1, intra     *via 10.0.8.121, Eth1/1, [110/31], 15w4d, ospf-1, intra N7K-R1# N7K-R1# N7K-R1# show routing hash 10.0.0.10 172.16.0.5 Load-share parameters used for software forwarding: load-share mode: address source-destination port source-destination Universal-id seed: 0x9617c10 Hash for VRF "default" resulting hash: 0x01 path '>' 172.16.0.5/32 unicast forwarding path(s) 2   *via 10.0.4.121%Ethernet2/1 > *via 10.0.8.121%Ethernet1/1 Route: N7K-R1#

While most of the time, it's acceptable to just know that it is taking one of several redundant paths, sometimes it is necessary to know the specific path a packet is taking. Hopefully this will serve as a quick reference during such events.