O men, when you start to write about BGP it is probably the time then you seriously start questioning yourself where did I go with myself. That is probably the moment in which you realize that there is a network geek sitting somewhere inside you. At least that is what happened to me when I finished to write this huge post. Don’t be scared, it’s fun to know about this thing below.
Table of Contents
Simple start
Every local network is managed by his own network administrator. If the network become big enough and there are more than few sub-segments inside that network there will probably be some kind of routing protocol running inside. That routing protocol will be IGP or interior gateway protocol more probably OSPF as it’s vendor independent.
When we want to connect our network to other networks across the world, we are trying to connect it to the internet. The Internet is the network connecting most of the networks today and in that way it became the biggest inter-networking system in the world. To be able to get that huge network to function and get our LANs to act jointly there must be a routing protocol that enables it.
BGP – Border Gateway Protocol is that one.
Every individual network has his own policies that are enabling that network to behave as the administrator want. When connection networks to the internet network all those policies need to be tied together with BGP protocol in order to influence outside communication entering the local network and communications initiated from the local network going outside somewhere on the internet. This is done using more that few different BGP attributes. Those attributes are forwarded across specific prefixes. Sometimes those attributes are not only forwarded but also modified on the way, one of which is the community attribute.
As BGP communities are not decently illustrated across the Internet resources I will try to do it here hopefully whit some success.
Community attributes shortly called communities are mostly used in service provider part of the network but there are some good use cases inside our corporate network also. Those cases when they are used inside corporate network can be very creative and interesting to follow in order to get more knowledge about communities and BGP.
There are so many service providers on the world today and some of them are peered together, of course, to give the interconnection to all users. The number of providers is so big that controlling which way will some communication be directed towards the destination can be very difficult. Service providers need to have agreements so that they are able to get all the data across the Internet flowing without problems. When we summarize those agreements something like this can be said: Customers are paying to ISP to get all the prefixes so that they can access all the corners of the Internet space. Customers are also advertising their own prefix to the provider and PEERS are in agreement that they will exchange their customer prefixes to each other.
There are different kinds of methods to get to this result inside ISP BGP configuration to implement policies like that. AS path filters, Prefix filters, communities.
For the first two BGP methods of policing the prefixes (prefix and AS path filters) ISP is able to doo all of that. The thing that is causing the issue is scalability. For every net customer ISP must ensure that every new customers prefix and AS Number is added to the filters on all of the BGP edge routers. All of the BGP edge routers of that one ISP of course, but that is still probably to big number of routers.
There are TcL scripts and things like that that are ably to help and automate one portion of the configuration but that is not really a solution of this issue. The more routers there are the probability of an error, be that human or some other error increases.
Communities give us a sophisticated yet straightforward solution.
The BGP Community Attribute
Inside the Autonomous system all BGP routers are running iBGP. They are probably configured in a full mesh in order not to have routing loops. In that config all BGP-speaking routers are sending their prefixes to each of its iBGP neighbors. Okay, you can use route reflectors so that one is the chief who catches and sends the prefixes to others but the idea is the same all routers get all the info.
When we peek outside the AS – BGP Autonomous system. Routers that are connection our AS with some other AS are configured as eBGP peers.
BGP routers exchange the info about their prefixes. It means that they are telling to their neighbors about network with subnet mask, and BGP attributes. With all that info best-path selection algorithm can be performed on every router to give him the means to decide about witch routes are best to use. When update with the prefix is sent to the neighbor AS, attribute AS-PATH is updated with the number of that sending ASN. The AS-PATH is in that way used to prevent routing loops. You see, if some AS eBGP router receives a prefix update with his own ASN inside AS-PATH attribute string it will see that this prefix is trying to go in the loop as it was once before advertised across this router.
We can add community BGP attribute to every advertised prefix. For community we can say that it is a BGP transitive optional attribute. It means that other BGP autonomous systems (eBGP) routers upon receipt of some prefix with community attribute inside may not be able to recognize what it means but they will however be able to decide whether to transport it through the AS or pass it to other AS.
Community attribute is simply a 32bit value that can be glued to any prefix.
32-bit value can be written as one instance or it can be divided into two halves. First 2 bytes for your ASN and the other 2 for something else. Values 0x00000000 through 0x0000FFFF and 0xFFFF0000 through 0xFFFFFFFF are reserved for something that does not matter here for us. Today’s routers use communities in this format ASN:SOMETHING. In this convention the communities 1:0 through 65534:65535 are free for use. It is a convention that you should use your ASN number as the leading 16 bits for your own communities and communities that you are sending to other AS.
There are some communities that are defined as standard. RFC 1997 does that and it is saying that in BGP implementations those are:
- NO-EXPORT 0xFFFFFF01
- NO-ADVERTISE 0xFFFFFF02
- NO-ADVERTISE-SUBCONFED 0xFFFFFF03
The details
NO-EXPORT is used if you want to stop your eBGP routers to send some particular prefix out to others outside your AS. One reason can be if you are sending a subnet of a large address block in order to influence external AS best-path selection with longest match. If that is what you do there are simple many subnets that you create and only few need to get outside your AS. All other you can mark with NO-EXPORT and they will get nowhere.
NO-ADVERTISE will stop a BGP router in sending tagged prefix to any BGP router including iBGP neighbors inside his AS.
NO-ADVERTISE-SUBCONFED is used to stop a prefix within a confederation.
Ok, what is the confederation?
You can make confederations of few routers inside AS and they are then like in some sort of subAS. Like if you will group the routers to AS inside AS. Normally there will be at least two confederation inside one AS and they are boring and not even used so much in provider network so there is no need to get into it here if you are not CCIE candidate. If you are, thay are not so heavy to understand, don’t worry.
Extended Community
There is another transitive optional attribute type, the Extended Community. It is Type 16.
Extended Community is a structured 8 octet long value. The first octet specifies the type. With that value we are defining the structure given to the remaining octets. Type field uses the first bit to differentiate whether this community is registered in Internet Assigned Numbers Authority (IANA) or in Internet Engineering Task Force (IETF). Second bit defines if the community is Transitive or Non-Transitive. It means that you can see from second bit if the community attribute will be passed between AS or not. There are other community types defined for the first octet prepared in advance for the use as standardized. One of them is the Route Target Community. This one is used in Multiprotocol Label Switching Virtual Private Networks – MPLS VPNs. You remember: ip vrf VPN_A rd 100:1 route-target import 100:1 route-target export 100:1
Route Target Community identifies which routers may receive some specific prefix. In the MPLS VPN world you need this to select which prefixes are part of which VPN if there is more than one customer using MPLS from service provider and usually it is.
There is also another one called Link Bandwidth Community. With this one ISP can influence the best path selection. When eBGP router on our AS learns from some other AS about a prefix with this attribute attached, this router will send to every other router inside our AS that the link bandwidth to that prefix is such and such. As it’s a Non-Transitive one it’s limited to our local AS.
Intra-Autonomous System Communities
We can make a lot of things using this kind of communities. We can take ISP three basic types of neighbors and see what to do. Normally customers of a service provider want to send their prefixes only and not their peers’ prefixes. It’s complicated a little, let’s make distinction between peer, customer and transit prefix.
We will advertise each prefix to a customer, peer, or transit provider by matching communities associated with the correct policy. Look at the picture above. To all prefixes received from customers we are putting an ingress tag 100:10. From peers we are putting the tag 100:20. Prefixes from transit provider are tagged with 100:30.
And now we can control what prefix goes where using this internal community attribute attached to every prefix that we received from somewhere.
Customers by definition want to go everywhere and they are expecting from their ISP to get all the prefixes. Each BGP session that goes to our customer will be configured to send all prefixes matching 100:10, 100:20, 100:30. They will receive everything that we learned. Peer is on the other side someone who wants to get only our customers prefixes, other thing he is learning from some other peering. We will send to the peer only prefixes tagged with 100:10.
We can do a lot more using this community coding, we can also use more communities each one for something else. To get more complex vendors enable us for pattern matching on routers for specific values, ranges or even logical operators OR and NOT. There is also the possibility to match community attribute with regular expressions.
Inter-Autonomous System Communities
Using Inter-Autonomous System Communities we can influence the traffic flows and make traffic engineering to a next level without much effort. We are able to prepend AS numbers inside our advertised prefixes and also use Multi-Exit Discriminators. Multi-Exit Discriminators are not supported by all ISP-s but if they are they are enabling us to configure from which ISP will our traffic be forwarded back to our AS if we have more ISPs (multihoming). We can then decide which pipe to use for incoming (download) traffic for specific session, app etc. We can furthermore decide to advertise some particular prefix or not announce prefixes at all, modify the origin type, or use other’s ISP’s communities designed for specific use.
There are more traffic engineering signaling possibilities. One of them is to simply force adjacent AS to prepend its ASN a specific number of times for some prefix sent to customers or peers. The other example includes a request for the neighbor to drop all traffic to a specific prefix.
Used for DoS mitigation
Last example shows communities in network security role. Denial of Service – DoS attacks may break customer access to resources across the internet by sending to much connection request to their servers so they became unable to serve ordinary requests. The attack may be focused on one or more hosts and not on the whole network (basically it can be focused to particular server that attacker wants to break.
This is fancy stuff, look at this:
Administrator on that network is able to tag individual server route. When administrator detects DoS attack to his server it can simply define a prefix of /32 that is basically representing single server IP address (192.168.1.100 255.255.255.255). Administrator can use that /32 prefix with Inter-Autonomous System Communities attribute set to signal to the provider to drop all traffic that is destined for that server/prefix.
ISP upon receipt of that /32 prefix with attribute as described above selects that single IP address and routes all traffic destined for it to the NULL interface on each BGP router. This can be the best ever technique to immediately stop DoS attack to your service but it must be used with caution as from this example is clear that if that is the only ISP you are using, the requests for that server service will be dropped even for valid requests. If there is multihoming involved then is better but keep in mind that you should try to drop only the traffic that is sourced from specific source that is detected as attacker IP or attacker subnet. In DDoS attacks this can be difficult as the sources are many but this technique will stop the attack immediately even then, hopefully the attackers will then decide to attack somewhere else where fancy BGP attributes are not used J
Sometimes the ISP is also sending some communities to the customers giving them the choice to use them for some purpose if they want to. If for example, a customer has the same provider for all offices on different continents those communities can be used to deduce where the prefix is coming from. So we can see where the prefix was originated. By using this community information customer can configure his BGP to prefer, for example, UK connection rather than Greece connection for traffic destined to USA. If you look at the backbone Internet map there is logic in that UK is directly connected under the sea with USA.
Conclusion
If you are an administrator with rather small network environment it does not mean that you cannot benefit from BGP communities. You just need to be careful and start with a clean and structured community design and you can get most of it. Here are some suggestions, some of them from myself, some cached on different resources across the network.
For the internal communities you need to choose whether to use them or not based on the network size and the whole setup of your network. For external communities is a little more useful maybe, ISPs are offering different sort and number of communities. Some will provide nothing, some enough to tag the primary and backup links to the Internet, and others will get you a big list with all sorts of influence possibilities.
You should document all communities that you are using and be aware of all the communities that you are transporting across your AS for others if you are a transit.