Multiple Routes to Same Destination

Now that you have set up the security policies using the Policy Routing route structure, you turn to the setup on the core router. Recently your company has obtained two Internet connections from two different service providers. Each connection is a T1 with an independent router and an independent assigned address scope. You want to set up load balancing for the Internet traffic.

The global information you will need is about the two different ISPs, and you will set up the multiple addresses you need on your router's external interface, eth1, as shown here:

ISP #1:
   Router Interface = 1.1.1.30/27

ISP #2:
   Router Interface = 2.2.2.30/27

Your router eth1 is:
ip addr add 1.1.1.1/27 dev eth1 brd +
ip addr add 2.2.2.1/27 dev eth1 brd +

Even though you have two different routes to the Internet, you would think that you can only have one default route. But you can have as many default or other routes as you would like. There are several different ways to code multiple routes to the same destination. Each method depends on the behavior you would like to have.

The first method is to use a per-packet method of multiple default routes. Under this scenario each packet entering the router will go out a different route. The main drawback to this format is that the paths to the final destination may vary in transit time enough to cause problems with packet reassembly queuing, especially with certain server types. But this is a very simple method to implement.

The route subcommand of the ip utility contains the methods allowing for multiple routers. This is coded using the equalize and nexthop commands. The nexthop command itself defines multiple gateways to send packets to and can take an optional weight command, which allows packets to be differentially balanced. The equalize command tells the route structure to send on a per-packet basis.

For example, if you decide to send each packet independently through each router, you would use the following command:

ip route add equalize default \
   nexthop via 1.1.1.30 dev eth1 \
   nexthop via 2.2.2.30 dev eth1

This will send each packet out through a different router. The first packet will go to 1.1.1.30, the second to 2.2.2.30, the third to 1.1.1.30, and so on ad nauseaum.

What if the router 1.1.1.30 was two T1s and the router 2.2.2.30 was a 512K fractional T1? Then you would want to weight the routes so as to send 4 packets to 1.1.1.30 for every 1 packet sent to 2.2.2.30. The easy way is to use the packet counts as weights. You would then use the following version of the command:

ip route add equalize default \
   nexthop via 1.1.1.30 dev eth1 weight 4 \
   nexthop via 2.2.2.30 dev eth1 weight 1

Now another way you might want to load balance is to allow each traffic flow sequence to go by one of the routes. But you do not want to inspect packets or code half the addresses one way and half the other. Instead you simply remove the equalize modifier from your multiple hop default route. Now traffic will be routed to one or the other route on a per-flow basis rather than a per-packet basis. Again you can use weights in this sense to load balance the flows themselves. Note that the per-flow is for tcp sessions while udp is treated per packet.