This lab continues from BGP path attributes and path selection. Both previous labs relied on the full IBGP mesh built in the first BGP lab. Here we will remove one session from that mesh and confirm the resulting loss of the prefix against the IBGP advertisement rule. Then we will replace the mesh with a route reflector. We will then apply the same approach to policy instead of naming a prefix in a route filter, we will tag it with a community at the source and match the tag downstream.
Overview
We will add two things to the previous topology. R1 gets a customer LAN, which provides a prefix originated inside AS 65100 for the reflector to distribute, and R4 gets a second external neighbor in AS 65300, which provides a downstream AS for no-export to work.
We will perform the following tasks:
- Add the customer LAN behind R1 and the third AS behind R4, and confirm what R5 does and does not receive.
- Remove the R1-R3 IBGP session and confirm that R3 loses
203.0.113.0/24completely. - Turn R2 into a route reflector
- Confirm that reflection does not change the next-hop.
- Use no-advertise and no-export, and see each propagation at a different boundary.
- Tag
198.51.100.0/24with a custom community on R4 and usefrom communityon R3 to raise local preference, moving the AS exit by class instead of by prefix.
Topology
Building the lab
R1 (the customer LAN)
set interfaces ge-0/0/3 unit 0 family inet address 203.0.113.1/24set policy-options policy-statement ADV-CUSTOMER-LAN term lan from protocol directset policy-options policy-statement ADV-CUSTOMER-LAN term lan from route-filter 203.0.113.0/24 exactset policy-options policy-statement ADV-CUSTOMER-LAN term lan then acceptset protocols bgp group ibgp-65100 export [ NHS ADV-CUSTOMER-LAN ]R5 (the third AS)
set interfaces ge-0/0/0 unit 0 family inet address 172.16.45.2/30set interfaces lo0 unit 0 family inet address 10.255.0.5/32set routing-options router-id 10.255.0.5set routing-options autonomous-system 65300set protocols bgp group ext-as65200 type externalset protocols bgp group ext-as65200 peer-as 65200set protocols bgp group ext-as65200 neighbor 172.16.45.1R4 (peer with R5)
set interfaces ge-0/0/2 unit 0 family inet address 172.16.45.1/30set protocols bgp group ext-as65300 type externalset protocols bgp group ext-as65300 peer-as 65300set protocols bgp group ext-as65300 neighbor 172.16.45.2H2
ifconfig eth0 203.0.113.10 netmask 255.255.255.0 uproute add default gw 203.0.113.1Verifying the starting state
R4 now transits between AS 65100 and AS 65300. We commit and check what R5 learned:
show route protocol bgpshow route 198.51.100.0/24R5 receives AS 65100's loopbacks with AS path 65200 65100, because the default BGP export behavior re-advertises active BGP-learned routes. R5 has no route to 198.51.100.0/24, even though that subnet is directly connected to R4 and R4 advertises it to AS 65100. A direct route requires an explicit export policy for each group it should be advertised to, and R4's ADV-CUSTOMER is applied to the ext-as65100 group only.
The mesh is still configured at this stage, so on R3:
show route 203.0.113.0/24traceroute 203.0.113.10 source 10.0.0.3R3 has the customer prefix directly from R1, next-hop 10.0.0.1, and the traceroute to H2 succeeds through R1.
Removing a session from the mesh
Route reflection is used because the full mesh does not scale. We will remove one session and see what happens.
R1:
delete protocols bgp group ibgp-65100 neighbor 10.0.0.3R3:
delete protocols bgp group ibgp-65100 neighbor 10.0.0.1R2 is now the only router both borders still peer with, and R2 is an ordinary IBGP speaker. It has 203.0.113.0/24 from R1 and will not re-advertise an IBGP-learned route to another IBGP peer.
On R3:
show route 203.0.113.0/24show route receive-protocol bgp 10.0.0.2 203.0.113.0/24traceroute 203.0.113.10 source 10.0.0.3The prefix is gone. R2 advertises nothing to R3 and the traceroute to H2 fails with no route to host. In the previous labs we fixed this by having a full mesh but IBGP mesh does not scale 20 IBGP speakers require 190 sessions, and every additional speaker has to peer with every existing one that's too much.
Route reflectors
We will configure R2 as a route reflector:
set protocols bgp group ibgp-65100 cluster 10.0.0.2The cluster statement makes every IBGP neighbor in that group a client of this reflector.
On R3:
show route 203.0.113.0/24show route 203.0.113.0/24 detailtraceroute 203.0.113.10 source 10.0.0.3The prefix is present again and the traceroute to H2 succeeds, this time over a reflected route. The detail output carries the two attributes reflection adds:
Cluster list: 10.0.0.2is R2's cluster ID, carried in the CLUSTER_LIST attribute. This is the loop-prevention mechanism. A reflector that receives a route already containing its own cluster ID rejects it.Originator ID: 10.0.0.1is the router ID of R1, which first advertised the prefix inside the AS.
The AS path field also reads I (Originator) rather than plain I, which is how Junos flags a route that reached this router through reflection.
The next-hop in the same output is still 10.0.0.1, R1's loopback. Reflection does not change the next-hop. It resolves here because R1 originated the prefix and already runs next-hop self, so the next-hop is a loopback carried by OSPF. If a reflected route carried an unresolvable next-hop, the client would install it as hidden.
Session count and redundancy
The R1-R3 session has been removed and reachability is retained. With one reflector each client needs one session to the reflector, which reduces the client peering topology from O(n²) to O(n).
One reflector is a single point of failure in the control plane, so a production cluster commonly uses two. When both reflectors serve the same client cluster, they are configured with the same cluster ID and every client peers with both. Each reflector receives the client routes directly, and the shared cluster ID prevents a route reflected by one reflector from being accepted and reflected again by the other. The clients still have redundant sessions and independent copies of their routes.
Different cluster IDs identify independent clusters. They are also required when a redundant reflector is expected to accept and reflect routes learned from another reflector into other clusters.
Well-known communities
A community is a tag carried with the route. Well-known communities have standardized meanings that every BGP speaker honors. We will look at two of them in this lab.
no-advertise
no-advertise means the route is not advertised to any peer:
set policy-options community NO-ADVERTISE members no-advertiseset policy-options policy-statement ADV-CUSTOMER-LAN term lan then community add NO-ADVERTISECommit and check R2, which now holds the tagged route:
show route 203.0.113.0/24 detailshow route advertising-protocol bgp 10.0.0.3 203.0.113.0/24 detailThe route is active on R2 and shows community no-advertise. The advertising-protocol command returns nothing for R3.
On R3:
show route 203.0.113.0/24The route is gone because no-advertise overrides route reflection: the reflector installs the route for its own use and does not propagate it to any peer.
We remove the tag and the unused community definition before the next test:
delete policy-options policy-statement ADV-CUSTOMER-LAN term lan then community add NO-ADVERTISEdelete policy-options community NO-ADVERTISEno-export
no-export is less restrictive. The receiving AS may carry the route through its own IBGP, but can't advertise it past its external boundary. On R1:
set protocols bgp group ext-as65200 export [ ADV-CUSTOMER-LAN ADV-INTERNAL ]On R4:
show route 203.0.113.0/24 detailshow route advertising-protocol bgp 172.16.45.2 203.0.113.0/24 detailR4 has the route from R1 and advertises it to R5. On R5:
show route 203.0.113.0/24 detailR5 has it with AS path 65200 65100, so the prefix has crossed two AS boundaries. We now tag it on R1:
set policy-options community NO-EXPORT members no-exportset policy-options policy-statement ADV-CUSTOMER-LAN term lan then community add NO-EXPORTADV-CUSTOMER-LAN is shared by R1's IBGP and EBGP export chains, so both copies leave R1 tagged. We commit and check the three routers:
On R3:
show route 203.0.113.0/24 detailR3 keeps the reflected route and displays community no-export.
On R4:
show route 203.0.113.0/24 detailshow route advertising-protocol bgp 172.16.45.2 203.0.113.0/24 detailR4 installs the route and shows the community, but doesn't advertise to R5.
On R5:
show route 203.0.113.0/24The route is gone. no-advertise stopped the route at the receiving router. no-export permitted propagation through IBGP and stopped the route at AS 65200's external boundary toward AS 65300.
We remove the tag, its definition, and the temporary EBGP export:
delete policy-options policy-statement ADV-CUSTOMER-LAN term lan then community add NO-EXPORTdelete policy-options community NO-EXPORTdelete protocols bgp group ext-as65200 export ADV-CUSTOMER-LANCustom communites
Custom communities are arbitrary ASN:value tags with no meaning. A route is tagged once at the source and the tag is matched anywhere downstream, which allows policy to be written for a class of routes instead of a list of prefixes.
We can use communities to change the local preference. R4 tags its customer prefix:
set policy-options community CUST-100 members 65200:100set policy-options policy-statement ADV-CUSTOMER term origin then community add CUST-100On R1 and R3:
show route 198.51.100.0/24 detailBoth borders now see 198.51.100.0/24 carrying community 65200:100. R3 matches the tag and raises local preference on it:
set policy-options community CUST-100 members 65200:100set policy-options policy-statement COMM-LP term match from community CUST-100set policy-options policy-statement COMM-LP term match then local-preference 200set policy-options policy-statement COMM-LP term match then acceptset protocols bgp group ext-as65200 import COMM-LPWe commit and check the reflector:
show route 198.51.100.0/24 detailshow route receive-protocol bgp 10.0.0.3 198.51.100.0/24 detailR2's active path is via R3, with local preference 200 and community 65200:100. The receive-protocol output confirms that both attributes arrived from R3 rather than being set locally. By default the reflector reflects its single best path, so it sends the R3 path to R1.
On R1:
show route 198.51.100.0/24 detailshow route receive-protocol bgp 10.0.0.2 198.51.100.0/24 detailR1 prefers the reflected local preference 200 path through R3 over its own EBGP path at 100. This is the same comparison as in the previous lab, with the path arriving from the reflector rather than from a direct IBGP session with R3.
Data plane, from R2:
traceroute 198.51.100.10 source 10.0.0.2The path is R2, R3, R4, H1. If R4 tagged 50 prefixes with 65200:100, this single policy would steer all 50 without any change to R3's configuration.
We remove the community configuration. On R3:
delete protocols bgp group ext-as65200 import COMM-LPdelete policy-options policy-statement COMM-LPdelete policy-options community CUST-100On R4:
delete policy-options policy-statement ADV-CUSTOMER term origin then community add CUST-100delete policy-options community CUST-100Key points:
- Route reflection relaxes one rule. A reflector re-advertises IBGP-learned routes to its clients, which an ordinary IBGP speaker does not do. This is allows the full mesh to be removed.
- The configuration is one statement on the reflector:
set protocols bgp group <group> cluster <cluster-id>. Every IBGP neighbor in that group becomes a client, and the clients need no reflection-specific configuration. - The CLUSTER_LIST attribute prevents reflection loops and ORIGINATOR_ID names the router that first advertised the route inside the AS. A reflector drops a route that already carries its own cluster ID. In
show route ... detailthese read asCluster list:andOriginator ID:. - Reflection does not change the next-hop. If the next-hop on a reflected route is not resolvable through the IGP, the client installs the route as hidden.
- Two reflectors serving one client cluster share a cluster ID, and every client peers with both. Different cluster IDs identify independent clusters or support multi-cluster reflection between reflectors.
- A reflector advertises its own best path only, not every path it holds.
- no-advertise stops a route at the receiving router, including IBGP peers and reflector clients. no-export permits IBGP propagation but blocks advertisement past the external boundary. no-export-subconfed blocks advertisement to EBGP peers in other member ASes of the same confederation.
- Custom communities (
ASN:value) carry no meaning. They are matched withfrom communityto act on a class of routes instead of a list of prefixes. The route is tagged once at the source and acted on anywhere downstream.
