How it works?
If you have two routers / two Layer3 switches connected with two L3 links (two paths) you can route with two equal static routes towards the same prefix and the router will load balance traffic across both links.
The idea is to make two same static routes on the same router but with different next-hops. The question was: Which link or which route will be used? And if the traffic will be load balanced, which mechanism will be used to share the traffic across both of links.
ip route 10.0.0.0 255.0.0.0 192.168.10.2 ip route 10.0.0.0 255.0.0.0 192.168.11.2
If both routes have the same destination prefix and no different Administrative Distance is configured, both routes will get installed in the routing table. Routing table will then leave to the switching process the job of load-sharing. That is, route-cache mechanisms, CEF in case of Cisco device will do load-share per session using source-destination IP.
More about that
CEF Load-Balancing Overview
CEF – Cisco Express Forwarding load balancing is by default using source and destination IP to calculate the hash and distribute traffic over multiple paths.
There are two methods of path selection, per-destination being default one:
- load balancing per-destination
- load balancing per-packet
Per-Destination Load Balancing
Router will take packet source and destination IP to calculate the hash and use multiple paths to load share traffic. Packets from one source destined towards same destination will always take the same path. Traffic destined for different source-destination pairs tend to go across different paths.
CEF is enabled by default on Cisco router so Per-destination load balancing is enabled by default to. It is fairly logical that Per-destination load balancing is default one because is ensuring that packets for a given host pair have the best chance to arrive in order which can not always be true with Per-Packet Load Balancing
Per-Packet Load Balancing
Router will use round-robin to send successive data packets over different links. If you have destinations that are available with single path at the end it will work fine for other situations Per-packet load balancing can reorder packets and affect performance of TCP stack. So it’s not really the right way to go if you are forwarding VoIP or using any kind of stream on your network.
Configuration
CEF is on by default, this is the command to enable it if you find some device with CEF off:
router# ip cef
This is how you see if CEF is working:
router# show ip cef
This is how you see CEF entry of some prefix:
router# show ip cef 10.10.10.0
This is how you change load-balancing from Per-Destination to Per-Packet:
ip load-sharing per-packet
Thanks for the article!
I have just several additions, if you don’t mind 🙂
When you have several routes in RIB towards one destination:
1. If you do trace from local router the traffic will be always per-packet load-balanced. And the reason for that is because locally-originated traffic is process-switched, not CEF switched – only traffic through the router is CEF-switched.
2. You can verify out of which interface+NH the router will send the packet towards using:
show ip cef exact-route
Basically how the load balancing with CEF is done is not very complicated. CEF uses special hash function, where source and destination IPs are input to this function and the output is number of “bucket”. CEF process takes all traffic share counts from routing table for the routes towards the same destination, decides how many “buckets” to allocate (for example, 16) and then distributes NH to buckets so that ratio is more or less coincides with traffic share count ratio. For example, if you have ECMP with two routes, you allocate 8 buckets for one NH and 8 buckets for another. If you have unequal cost multiple path, one NH will get more buckets while another NH will get less.
It is obvious that with this per-destination load-balancing you can have traffic polarization – when one link is chosen all the time, while another one is underutilized. There isn’t much you can do. If your hardware supports hashing/load-balancing using source/destination port you can turn it on, but it is hardware limitation. Also there is special universal ID – random value generated during router boot up, this ensures that different routers produces different hashes to the same input pair.
More details about CEF polarization can be found here:
http://www.cisco.com/c/en/us/support/docs/ip/express-forwarding-cef/116376-technote-cef-00.html
Thank a lot both of you