The first post in this series covered the design and why this lab took me over a month to get right.
I thought I had it working at the time. Turns out I only had partial connectivity and a lot of false confidence. Almost another month later I finally got full end-to-end connectivity.
This post goes through the configuration and the outputs that confirmed things were actually working.
Step 0: Know What "Working" Means
Before touching DCI, make sure these are true:
- EVPN VXLAN works inside DC1
- EVPN VXLAN works inside DC2
- Tenant subnets exist in a VRF
- Border leaf can send those tenant routes to the edge router
- Edge routers can exchange VRF routes through the WAN
- Remote DC subnets show up in the local DCI VRF
If any one of these is missing, DCI becomes "advanced imagination".
lab topology
Step 1: Build EVPN VXLAN Inside Each Data Center
This part worked fast, which was a nice change compared to the rest of the lab. Each data center runs its own EVPN VXLAN fabric. Leafs act as VTEPs and spines are EVPN route reflectors.
VLAN to VNI mapping on the DC1 border leaf:
set vlans v100 vlan-id 100set vlans v100 vxlan vni 100100set vlans v100 l3-interface irb.100set vlans v101 vlan-id 101set vlans v101 vxlan vni 100101set vlans v101 l3-interface irb.101IRB gateways:
set interfaces irb unit 100 family inet address 10.100.100.11/24 virtual-gateway-address 10.100.100.1set interfaces irb unit 101 family inet address 10.101.101.11/24 virtual-gateway-address 10.101.101.1DC2 uses the same idea with different subnets:
set interfaces irb unit 100 family inet address 10.200.100.25/24 virtual-gateway-address 10.200.100.1set interfaces irb unit 101 family inet address 10.201.101.25/24 virtual-gateway-address 10.201.101.1Step 2: L3VNI and EVPN Type-5 Routes
To move IP subnets between leafs and toward the DCI edge, I used EVPN Type-5 routes over an L3VNI.
On the border leaf:
set routing-instances TENANT-A protocols evpn ip-prefix-routes encapsulation vxlanset routing-instances TENANT-A protocols evpn ip-prefix-routes vni 999set routing-instances TENANT-A protocols evpn ip-prefix-routes advertise direct-nexthopset routing-instances TENANT-A protocols evpn ip-prefix-routes export EXPORT-TYPE5This is where I lost many hours. When Type-5 worked life was good. When it didn't work I blamed MPLS, BGP, Juniper, my laptop, and sometimes the universe. In the end it was usually just a missing policy.
Step 3: Send Tenant Routes From Border Leafs to the Edge Router
Each border leaf peers with the DC edge router inside the tenant VRF.
set routing-instances TENANT-A protocols bgp group TO-EDGE-MX type internalset routing-instances TENANT-A protocols bgp group TO-EDGE-MX local-address 10.0.113.1set routing-instances TENANT-A protocols bgp group TO-EDGE-MX export EXPORT-TO-EDGEset routing-instances TENANT-A protocols bgp group TO-EDGE-MX neighbor 10.0.113.0BGP sessions came up immediately but getting the routes to work took some trying. This is where I learned that "BGP is up" does not mean "BGP is doing anything useful".
Step 4: Exporting Tenant Routes to the WAN
After the border leafs send tenant routes to the edge MX routers, those routes need to be exported into the MPLS VPN toward the ISP core.
On MX3 this is done with a simple BGP group inside the DCI VRF:
set routing-instances DCI protocols bgp group TO-ISP type externalset routing-instances DCI protocols bgp group TO-ISP neighbor 203.0.113.2set routing-instances DCI protocols bgp group TO-ISP peer-as 65002set routing-instances DCI protocols bgp group TO-ISP export EXPORT-DC-ROUTESAnd the export policy:
set policy-options policy-statement EXPORT-DC-ROUTES term 1 from protocol bgpset policy-options policy-statement EXPORT-DC-ROUTES term 1 then acceptWithout this, everything looked up but nothing actually crossed the WAN.
Step 5: Carry Tenant Routes Across the WAN
Between DC1 and DC2 I built a small ISP-style MPLS core with:
- IS-IS underlay
- SR-MPLS transport
- BGP VPNv4
- MPLS L3VPN
At first I thought I would just stretch VXLAN across it. Then I searched and learned that it doesn't work like that in real life. I accepted reality and reconfigured everything to be routed DCI.
ISP PE example (MX21):
set routing-instances C1-DC2 instance-type vrfset routing-instances C1-DC2 route-distinguisher 10.255.255.21:200set routing-instances C1-DC2 vrf-target target:65002:100set routing-instances C1-DC2 vrf-table-labelset routing-instances C1-DC2 protocols bgp group CE type externalset routing-instances C1-DC2 protocols bgp group CE neighbor 203.0.113.5set routing-instances C1-DC2 protocols bgp group CE peer-as 65100set routing-instances C1-DC2 protocols bgp group CE as-overrideset routing-instances C1-DC2 protocols bgp group CE import FROM-CEset routing-instances C1-DC2 protocols bgp group CE export TO-CEVPNv4 toward other ISP routers:
set protocols bgp group INT type internalset protocols bgp group INT family inet-vpn unicastset protocols bgp group INT neighbor 10.255.255.20set protocols bgp group INT neighbor 10.255.255.19Step 5.5: How I Finally Debugged It
Before showing the working outputs, it's worth mentioning how I actually figured out what was wrong.
I went through all the usual troubleshooting first, checking BGP sessions, policies, route tables, and EVPN routes on every device, and most of the time everything looked correct even though traffic was clearly not flowing end to end.
What finally helped was using Wireshark and following each ping hop by hop while at the same time comparing advertised and received routes across the fabric, the edge routers, and the ISP core. Seeing the forwarding path together with the control plane made it obvious where routes were not being exported, imported, or preferred the way I expected.
Step 6: EVPN and DCI Were Actually Working
Now for the fun part. Seeing real routes move across fabrics.
On DC1 border leaf (TENANT-A VRF):
show route table TENANT-A.inet.0And suddenly I could see local EVPN subnets, remote DC2 subnets coming through BGP, and both EVPN and DCI paths visible:
10.100.100.0/24 Direct via irb.10010.101.101.0/24 Direct via irb.10110.200.100.0/24 BGP via 10.0.113.010.201.101.0/24 BGP via 10.0.113.0Which means DC1 fabric learned DC2 tenant networks. This was the moment I stopped trusting my eyes and checked it five more times.
Final Thoughts
This setup probably still has like a million flaws. But building it slowly helped me understand a lot of the technologies used in this lab much better than any polished lab guide. Seeing real routes flow between two EVPN VXLAN fabrics after months of trying was extremely satisfying.
