>_nexthopreachable

JNCIP-ENT Mega Lab Part 1: Internet Edge and Core

Ali Bahadır Coşkun19 min read

This is the first part of my JNCIP-ENT mega lab series. The goal is to build one enterprise network that includes every topic in the certification. In this part, we build the Internet edge and headquarters core. There are two providers, two enterprise border routers and two core routers, and everything is dual-stack. OSPFv2 and OSPFv3 run in area 0, and the two cores work as route reflectors for iBGP. After this is working, the enterprise must make two decisions: which provider outbound traffic will use, and through which provider inbound traffic will arrive. For the first decision, we will use local preference and for the second one, we will use AS path prepend.

Overview

DeviceRole
ISP-AAS 65100, originates 198.51.100.0/24 and 2001:db8:a::/64
ISP-BAS 65200, originates 203.0.113.0/24 and 2001:db8:b::/64
HQ-EDGE01border toward ISP-A, iBGP client
HQ-EDGE02border toward ISP-B, iBGP client
HQ-CORE01core, route reflector, cluster ID 10.0.0.5
HQ-CORE02core, route reflector, same cluster
APP-Apublic endpoint behind ISP-A
APP-Bpublic endpoint behind ISP-B

In this part, we will do the following tasks:

  • Assign IPv4 and IPv6 addresses to every link and bring up OSPFv2 and OSPFv3 in area 0.
  • Add MD5 authentication to inter-core OSPFv2 adjacency and use an IPsec SA for OSPFv3.
  • Configure both providers so each one originates a public prefix and also advertises the other provider's prefix.
  • Build dual-stack eBGP on both border routers.
  • Build iBGP using the cores as reflectors, change the next hop on the borders, and check the cluster list and originator ID on one client.
  • Originate enterprise aggregates for return traffic and use an export policy which rejects everything else.
  • Select ISP-B for outbound traffic with local preference, and influence inbound traffic with an AS path prepend toward ISP-A.
  • Add communities to the routes coming from both providers.

Topology

The backbone

We run OSPFv2 and OSPFv3 in area 0 on internal links and loopbacks. Internal links are point-to-point, and loopback interfaces are passive. We do not include provider transit links inside OSPF.

HQ-EDGE01

junos
set interfaces ge-0/0/0 unit 0 family inet address 172.16.1.1/30set interfaces ge-0/0/0 unit 0 family inet6 address 2001:db8:172:1::1/64set interfaces ge-0/0/1 unit 0 family inet address 10.1.1.1/30set interfaces ge-0/0/1 unit 0 family inet6 address 2001:db8:1:0::1/64set interfaces ge-0/0/2 unit 0 family inet address 10.1.1.5/30set interfaces ge-0/0/2 unit 0 family inet6 address 2001:db8:1:4::1/64set interfaces ge-0/0/3 unit 0 family inet address 10.1.1.9/30set interfaces ge-0/0/3 unit 0 family inet6 address 2001:db8:1:8::1/64set interfaces lo0 unit 0 family inet address 10.0.0.3/32set interfaces lo0 unit 0 family inet6 address 2001:db8:0::3/128set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2pset protocols ospf area 0.0.0.0 interface ge-0/0/2.0 interface-type p2pset protocols ospf area 0.0.0.0 interface ge-0/0/3.0 interface-type p2pset protocols ospf area 0.0.0.0 interface lo0.0 passiveset protocols ospf3 area 0.0.0.0 interface ge-0/0/1.0 interface-type p2pset protocols ospf3 area 0.0.0.0 interface ge-0/0/2.0 interface-type p2pset protocols ospf3 area 0.0.0.0 interface ge-0/0/3.0 interface-type p2pset protocols ospf3 area 0.0.0.0 interface lo0.0 passiveset routing-options router-id 10.0.0.3set routing-options autonomous-system 65000

HQ-CORE01

junos
set interfaces ge-0/0/0 unit 0 family inet address 10.1.1.6/30set interfaces ge-0/0/0 unit 0 family inet6 address 2001:db8:1:4::2/64set interfaces ge-0/0/1 unit 0 family inet address 10.1.1.14/30set interfaces ge-0/0/1 unit 0 family inet6 address 2001:db8:1:12::2/64set interfaces ge-0/0/2 unit 0 family inet address 10.1.1.21/30set interfaces ge-0/0/2 unit 0 family inet6 address 2001:db8:1:20::1/64set interfaces lo0 unit 0 family inet address 10.0.0.5/32set interfaces lo0 unit 0 family inet6 address 2001:db8:0::5/128set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 interface-type p2pset protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2pset protocols ospf area 0.0.0.0 interface ge-0/0/2.0 interface-type p2pset protocols ospf area 0.0.0.0 interface lo0.0 passiveset protocols ospf3 area 0.0.0.0 interface ge-0/0/0.0 interface-type p2pset protocols ospf3 area 0.0.0.0 interface ge-0/0/1.0 interface-type p2pset protocols ospf3 area 0.0.0.0 interface ge-0/0/2.0 interface-type p2pset protocols ospf3 area 0.0.0.0 interface lo0.0 passiveset routing-options router-id 10.0.0.5set routing-options autonomous-system 65000

We apply a load-balancing policy on all four routers for the equal-cost paths:

junos
set policy-options policy-statement LB term 1 then load-balance per-packetset routing-options forwarding-table export LB

Verification

On HQ-CORE01:

junos
show ospf neighborshow ospf3 neighborshow route 10.0.0.6show route 2001:db8:0::6

All three OSPFv2 adjacencies and all three OSPFv3 adjacencies must be in the Full state. Also, every other loopback must be reachable in both address families.

OSPF authentication

OSPFv2 authentication works per interface. Both sides need to use the same key ID and key. We apply it on the inter-core link for both HQ-CORE01 and HQ-CORE02:

junos
set protocols ospf area 0.0.0.0 interface ge-0/0/2.0 authentication md5 1 key "MegaLab-OSPF"
junos
show ospf interface ge-0/0/2.0 detail

The adjacency stays in the Full state, and key ID 1 is seen on both sides.

OSPFv3 does not have an authentication field. Instead, it uses an IPsec security association. This association must be manual, bidirectional and use transport mode:

junos
set security ipsec security-association OSPFv3-SA mode transportset security ipsec security-association OSPFv3-SA manual direction bidirectional protocol ahset security ipsec security-association OSPFv3-SA manual direction bidirectional spi 256set security ipsec security-association OSPFv3-SA manual direction bidirectional authentication algorithm hmac-sha1-96 key ascii-text "MegaLab-OSPFv3-Key01"set protocols ospf3 area 0.0.0.0 interface ge-0/0/2.0 ipsec-sa OSPFv3-SA

The two providers

Each provider originates one IPv4 and one IPv6 prefix. They peer with each other over eBGP to exchange these prefixes.

ISP-A

junos
set interfaces ge-0/0/0 unit 0 family inet address 172.16.1.2/30set interfaces ge-0/0/0 unit 0 family inet6 address 2001:db8:172:1::2/64set interfaces ge-0/0/1 unit 0 family inet address 172.16.0.1/30set interfaces ge-0/0/1 unit 0 family inet6 address 2001:db8:172:0::1/64set interfaces ge-0/0/2 unit 0 family inet address 198.51.100.1/24set interfaces ge-0/0/2 unit 0 family inet6 address 2001:db8:a::1/64set interfaces lo0 unit 0 family inet address 10.0.0.1/32set interfaces lo0 unit 0 family inet6 address 2001:db8:0::1/128set protocols router-advertisement interface ge-0/0/2.0 prefix 2001:db8:a::/64set routing-options router-id 10.0.0.1set routing-options autonomous-system 65100set policy-options policy-statement ADV-CUST term v4 from protocol directset policy-options policy-statement ADV-CUST term v4 from route-filter 198.51.100.0/24 exactset policy-options policy-statement ADV-CUST term v4 then acceptset policy-options policy-statement ADV-CUST term v6 from protocol directset policy-options policy-statement ADV-CUST term v6 from route-filter 2001:db8:a::/64 exactset policy-options policy-statement ADV-CUST term v6 then acceptset protocols bgp group ENT-V4 type externalset protocols bgp group ENT-V4 peer-as 65000set protocols bgp group ENT-V4 family inet unicastset protocols bgp group ENT-V4 neighbor 172.16.1.1set protocols bgp group ENT-V4 export ADV-CUSTset protocols bgp group ENT-V6 type externalset protocols bgp group ENT-V6 peer-as 65000set protocols bgp group ENT-V6 family inet6 unicastset protocols bgp group ENT-V6 neighbor 2001:db8:172:1::1set protocols bgp group ENT-V6 export ADV-CUSTset protocols bgp group PEER-V4 type externalset protocols bgp group PEER-V4 peer-as 65200set protocols bgp group PEER-V4 family inet unicastset protocols bgp group PEER-V4 neighbor 172.16.0.2set protocols bgp group PEER-V4 export ADV-CUSTset protocols bgp group PEER-V6 type externalset protocols bgp group PEER-V6 peer-as 65200set protocols bgp group PEER-V6 family inet6 unicastset protocols bgp group PEER-V6 neighbor 2001:db8:172:0::2set protocols bgp group PEER-V6 export ADV-CUST

The endpoints are VPCS nodes. On APP-A, we configure these addresses:

text
ip 198.51.100.10/24 198.51.100.1ip 2001:db8:a::10/64

VPCS does not accept a static IPv6 default gateway. For this reason, it learns the IPv6 default router from Router Advertisement.

Verification

On ISP-A:

junos
show bgp summaryshow route 203.0.113.0/24show route 2001:db8:b::/64

The session toward ISP-B must be Established for both families, and the ISP-B prefixes must arrive with AS path 65200.

eBGP at the borders

Each border has one provider connection with two BGP sessions. One session runs over IPv4 and carries inet NLRI. The other runs over IPv6 and carries inet6 NLRI.

HQ-EDGE01

junos
set protocols bgp group EXT-ISPA-V4 type externalset protocols bgp group EXT-ISPA-V4 peer-as 65100set protocols bgp group EXT-ISPA-V4 family inet unicastset protocols bgp group EXT-ISPA-V4 neighbor 172.16.1.2set protocols bgp group EXT-ISPA-V6 type externalset protocols bgp group EXT-ISPA-V6 peer-as 65100set protocols bgp group EXT-ISPA-V6 family inet6 unicastset protocols bgp group EXT-ISPA-V6 neighbor 2001:db8:172:1::2

HQ-EDGE02

junos
set protocols bgp group EXT-ISPB-V4 type externalset protocols bgp group EXT-ISPB-V4 peer-as 65200set protocols bgp group EXT-ISPB-V4 family inet unicastset protocols bgp group EXT-ISPB-V4 neighbor 172.16.2.2set protocols bgp group EXT-ISPB-V6 type externalset protocols bgp group EXT-ISPB-V6 peer-as 65200set protocols bgp group EXT-ISPB-V6 family inet6 unicastset protocols bgp group EXT-ISPB-V6 neighbor 2001:db8:172:2::2

Verification

On HQ-EDGE01:

junos
show bgp summaryshow route protocol bgp table inet.0show route protocol bgp table inet6.0

Both sessions must be Established, and both public prefixes must exist in both address families.

iBGP and route reflection

A route reflector re-advertises an iBGP-learned route to another iBGP peer. A normal iBGP router cannot advertise a route learned from one iBGP peer to another iBGP peer.

Both cores work as reflectors with the same cluster ID, 10.0.0.5. A reflector puts this ID into the cluster list of every route it reflects. If a route already has its own cluster ID, it drops the route. Because of this, each core drops the copy reflected by the other core.

HQ-CORE01

junos
set protocols bgp group RR-CLIENTS-V4 type internalset protocols bgp group RR-CLIENTS-V4 local-address 10.0.0.5set protocols bgp group RR-CLIENTS-V4 family inet unicastset protocols bgp group RR-CLIENTS-V4 cluster 10.0.0.5set protocols bgp group RR-CLIENTS-V4 neighbor 10.0.0.3set protocols bgp group RR-CLIENTS-V4 neighbor 10.0.0.4set protocols bgp group RR-CLIENTS-V6 type internalset protocols bgp group RR-CLIENTS-V6 local-address 2001:db8:0::5set protocols bgp group RR-CLIENTS-V6 family inet6 unicastset protocols bgp group RR-CLIENTS-V6 cluster 10.0.0.5set protocols bgp group RR-CLIENTS-V6 neighbor 2001:db8:0::3set protocols bgp group RR-CLIENTS-V6 neighbor 2001:db8:0::4set protocols bgp group RR-PEER-V4 type internalset protocols bgp group RR-PEER-V4 local-address 10.0.0.5set protocols bgp group RR-PEER-V4 family inet unicastset protocols bgp group RR-PEER-V4 neighbor 10.0.0.6set protocols bgp group RR-PEER-V6 type internalset protocols bgp group RR-PEER-V6 local-address 2001:db8:0::5set protocols bgp group RR-PEER-V6 family inet6 unicastset protocols bgp group RR-PEER-V6 neighbor 2001:db8:0::6

HQ-CORE02 uses the same configuration with its own local addresses. It uses the same cluster ID, 10.0.0.5 and its peer group points back to HQ-CORE01.

The clients do not need any special configuration for reflection. On HQ-EDGE01, we configure normal iBGP groups:

junos
set protocols bgp group IBGP-V4 type internalset protocols bgp group IBGP-V4 local-address 10.0.0.3set protocols bgp group IBGP-V4 family inet unicastset protocols bgp group IBGP-V4 advertise-externalset protocols bgp group IBGP-V4 neighbor 10.0.0.5set protocols bgp group IBGP-V4 neighbor 10.0.0.6set protocols bgp group IBGP-V6 type internalset protocols bgp group IBGP-V6 local-address 2001:db8:0::3set protocols bgp group IBGP-V6 family inet6 unicastset protocols bgp group IBGP-V6 advertise-externalset protocols bgp group IBGP-V6 neighbor 2001:db8:0::5set protocols bgp group IBGP-V6 neighbor 2001:db8:0::6set policy-options policy-statement NHS then next-hop selfset protocols bgp group IBGP-V4 export NHSset protocols bgp group IBGP-V6 export NHS

HQ-EDGE02 has the same configuration, using 10.0.0.4 and 2001:db8:0::4.

local-address. The iBGP peers use loopbacks, so we must tell the session which local address it will use as its source.

NHS. When a border learns a prefix over eBGP, the next hop is inside the provider transit subnet. We did not add the transit links to OSPF, so this next hop does not exist in the IGP. If the border sends the route into iBGP without changing the next hop, the other routers cannot resolve it and the route becomes hidden. NHS changes the next hop to the border router's loopback, which OSPF can resolve.

advertise-external. By default, BGP advertises only its best path. After local preference makes the ISP-B path best for every route, HQ-EDGE01 normally stops advertising its ISP-A path to the cores. In this situation, the cores can see only one provider. With advertise-external, each border continues to advertise its own best external path, even when this path is not the overall best one.

Verification

On HQ-CORE01:

junos
show bgp summaryshow route 198.51.100.0/24 detailshow route 2001:db8:a::/64 detailshow route hiddenshow route receive-protocol bgp 10.0.0.3 198.51.100.0/24

All iBGP sessions must be Established. Both public prefixes must be active in both families, with next hops pointing to the border loopbacks. There must not be any hidden route because of an unresolved next hop. The received route also shows us the changed next hop.

Reflection attributes

The cluster list and originator ID are added only during route reflection. They are not seen on the reflector's own path learned directly from a client. For this reason, we must check them on a client that receives the reflected route. On HQ-EDGE01, we look at the route originated by ISP-B that entered through HQ-EDGE02:

junos
show route receive-protocol bgp 10.0.0.5 203.0.113.0/24 detail

The cluster list includes 10.0.0.5, and the originator ID is 10.0.0.4, which is the router ID of HQ-EDGE02. HQ-EDGE02 originated the route into iBGP, and after that HQ-CORE01 reflected it. These two attributes show us this process.

Return reachability

For a traceroute sourced from a loopback, the public side also needs a return route toward this loopback. Therefore, the enterprise originates two aggregate routes and exports only these two. On both border routers:

junos
set routing-options aggregate route 10.0.0.0/16set routing-options rib inet6.0 aggregate route 2001:db8:0::/64set policy-options policy-statement ADV-ENT term agg-v4 from protocol aggregateset policy-options policy-statement ADV-ENT term agg-v4 from route-filter 10.0.0.0/16 exactset policy-options policy-statement ADV-ENT term agg-v4 then acceptset policy-options policy-statement ADV-ENT term agg-v6 from protocol aggregateset policy-options policy-statement ADV-ENT term agg-v6 from route-filter 2001:db8:0::/64 exactset policy-options policy-statement ADV-ENT term agg-v6 then acceptset policy-options policy-statement ADV-ENT term reject-all then reject

On HQ-EDGE01:

junos
set protocols bgp group EXT-ISPA-V4 export ADV-ENTset protocols bgp group EXT-ISPA-V6 export ADV-ENT

We apply the same export policy on HQ-EDGE02 for its two ISP-B groups.

On both providers, we verify that the summary routes arrived:

junos
show route 10.0.0.0/16 exactshow route 2001:db8:0::/64 exact

After this, we verify on HQ-EDGE01 that no other route is advertised outside:

junos
show route advertising-protocol bgp 172.16.1.2 10.0.0.5/32show route advertising-protocol bgp 172.16.1.2 203.0.113.0/24show route advertising-protocol bgp 2001:db8:172:1::2 2001:db8:0::5/128show route advertising-protocol bgp 2001:db8:172:1::2 2001:db8:b::/64

Choosing the outbound provider

Now, each core has two paths for every public prefix. For 198.51.100.0/24, the direct path has AS path 65100, and the indirect path has 65200 65100. Local preference is 100 on both paths. Because of this, the shorter AS path becomes active, and for now each destination is reached through its own provider.

We want to use ISP-B as the exit for both destinations. We can do this with local preference because BGP checks it before AS path. On HQ-EDGE02:

junos
set policy-options policy-statement LP-ISPB term all from protocol bgpset policy-options policy-statement LP-ISPB term all then local-preference 200set policy-options policy-statement LP-ISPB term all then acceptset protocols bgp group EXT-ISPB-V4 import LP-ISPBset protocols bgp group EXT-ISPB-V6 import LP-ISPB

This is an import policy, and we apply it at the point where routes enter our AS. Local preference is carried inside iBGP. Therefore, the core routers and HQ-EDGE01 see value 200 on these paths. At the eBGP boundary, local preference is removed, so the providers cannot see this value.

Verification

On HQ-CORE01:

junos
show route 198.51.100.0/24 detailrun traceroute 198.51.100.10 source 10.0.0.5run traceroute 203.0.113.10 source 10.0.0.5

The ISP-B path becomes active with local preference 200 for both destinations. Even traffic toward 198.51.100.0/24 now selects the longer AS path. Both traceroutes leave the enterprise through HQ-EDGE02.

Choosing the inbound provider

Local preference controls where traffic leaves, and this attribute never goes outside AS 65000. It cannot affect how external networks reach the enterprise because this decision is made by the neighbouring AS. The enterprise can only influence this decision. For this, we can use AS path prepend.

We want the ISP-B entrance to be preferred, but ISP-A must stay as an alternative. Therefore, we make the AS path longer for the aggregates advertised from HQ-EDGE01 to ISP-A. On HQ-EDGE01:

junos
set policy-options policy-statement ADV-ENT term agg-v4 then as-path-prepend "65000 65000"set policy-options policy-statement ADV-ENT term agg-v6 then as-path-prepend "65000 65000"

Now ISP-A has two ways to reach 10.0.0.0/16. The direct path through HQ-EDGE01 has an AS path with three entries, and the path through ISP-B is shorter.

Verification

On ISP-A:

junos
show route 10.0.0.0/16 exact detailshow route 2001:db8:0::/64 exact detail

The active path must point toward ISP-B. From APP-A, we test it with a trace:

text
trace 10.0.0.5

The trace enters the enterprise through HQ-EDGE02.

Communities

A community is like a label attached to a route. By using it, policy can work on one class of routes instead of checking a prefix list. We add the tag where the route enters the AS, and after this it can be matched at any point inside the network.

On HQ-EDGE01:

junos
set policy-options community FROM-ISPA members 65000:100set policy-options policy-statement TAG-ISPA term all from protocol bgpset policy-options policy-statement TAG-ISPA term all then community add FROM-ISPAset policy-options policy-statement TAG-ISPA term all then acceptset protocols bgp group EXT-ISPA-V4 import TAG-ISPAset protocols bgp group EXT-ISPA-V6 import TAG-ISPA

HQ-EDGE02 already uses an import policy on its ISP-B groups. For this reason, we add the community action inside the existing policy:

junos
set policy-options community FROM-ISPB members 65000:200set policy-options policy-statement LP-ISPB term all then community add FROM-ISPB

On HQ-CORE01:

junos
show route 198.51.100.0/24 detailshow route 203.0.113.0/24 detail

Every path learned from ISP-A carries 65000:100. Every path learned from ISP-B carries 65000:200 together with local preference 200. Now, on the core router, we can understand which border a route came from without checking the next-hop address.

Both directions together:

Key points:

  • We need two of everything. If there is no second path, policy cannot change the path selection. The two providers are connected with each other, and two border routers with two core routers give an alternative path for every prefix. Local preference, AS path and MED only matter when this second path exists.
  • If Junos cannot resolve an iBGP next hop, it keeps the route as hidden. It does not remove the route completely because the next hop may become reachable later.
  • Cluster ID shows the cluster, not one router. When two reflectors are in the same cluster, each one drops the route reflected by the other one. This is the loop protection. It does not create a problem here because every client peers with both reflectors, so each reflector learns all routes directly.
  • Cluster list and originator ID are added only when reflection happens. For this reason, we check these values on the client receiving the reflected route, not on the reflector itself.
  • advertise-external keeps the border's own provider path visible to the cores. It continues to work even when the path from the other border becomes the best path.
  • Local preference is checked before AS path. Because of this, it can move traffic away from the router's own external session. In this case, traffic follows a longer path.
  • Local preference decides the exit, but AS path prepend only affects the entrance. Local preference is removed at the eBGP boundary, so the provider cannot see it. The prepend works because the neighbouring AS makes its own decision, and we change the path information used for this decision.
  • A term that has only a modifying action does not finish the policy check. Evaluation continues with the next term. NHS uses this behaviour, and ADV-ENT needs one final reject term to stop all other routes.
  • The final reject term is the part that prevents the route leak. Without this term, the default eBGP export sends every active BGP route to the providers. Then the enterprise AS becomes a transit between them. We use show route advertising-protocol to prove that this is not happening.
  • Inactive reason: shows us at which step a path was rejected. In this way, we can read the selection directly from output instead of calculating it by hand.