DMVPN | Phase 3 | IPsec | VRF | Per-Tunnel QoS
Covering the configuration, confirmation and troubleshooting of DMVPN Phase 3 with IPsec and Per-Tunnel QoS. The main reason behind this is to help out my friend who is implementing a production design similar to this.
Below is the high level topology used for this lab. Its a fairly simple diagram but gives you the idea, the devil is in the detail (config)
For the dynamic routing protocol, I have chosen EIGRP, you could use BGP if you want a hyper-scale design. Phase 3 DMVPN is chosen simply to enable spoke-to-spoke communication and maintain a default route to the spokes. Phase 3 allows this by using redirect messages / shortcut routing in NHRP.
The purpose of this blog is not to explain DMVPN, there are plenty of resources for that.
Here is the config for the HUB.
vrf definition internet
!
address-family ipv4
exit-address-family
ip route vrf internet 0.0.0.0 0.0.0.0 [internet next-hop IP]
!
interface Tunnel0
ip address 10.100.0.4 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication cisco
ip nhrp map multicast dynamic
ip nhrp map group Spoke-5 service-policy output DMVPN_5000kbps
ip nhrp map group Spoke-1 service-policy output DMVPN_1500kbps
ip nhrp map group Spoke-2 service-policy output DMVPN_10000kbps
ip nhrp network-id 123
ip nhrp redirect
ip tcp adjust-mss 1360
qos pre-classify
tunnel source Ethernet0/0
tunnel mode gre multipoint
tunnel key 123
tunnel vrf internet
tunnel protection ipsec profile DMVPN
!
router eigrp DMVPN
!
address-family ipv4 unicast autonomous-system 123
!
af-interface Tunnel0
summary-address 0.0.0.0 0.0.0.0
no split-horizon
exit-af-interface
!
topology base
exit-af-topology
network 10.0.0.0
network 10.100.0.0 0.0.0.255
exit-address-family
crypto keyring dmvpn vrf internet
pre-shared-key address 0.0.0.0 0.0.0.0 key topsecret!
!
crypto isakmp policy 100
encr aes 256
authentication pre-share
group 5
!
crypto ipsec transform-set DMVPN esp-aes esp-sha-hmac
!
crypto ipsec profile DMVPN
set transform-set DMVPN
QOS – On the hub tunnel interface you can set the QoS policies to map to a group name, then on the spoke you can set the command to have the tunnel subscribe to a particular group.
The QoS polices need to be setup in a specific way with parent and child policies, this way you can be specific in which spoke gets which shaper settings. Create a policy with your required settings see example below.
policy-map DMVPN
class icmp
police 8000
class telnet
set precedence 4
class www
police 80000
set precedence 6
Then create your shaping policy and attach the child policy like below.
policy-map DMVPN_5000kbps
class class-default
shape average 5000000
service-policy DMVPN
Front Door VRF – The reason for this is to allow a default route for internet access and also allow a default route for LAN traffic. The Internet facing interface is in its own VRF and routes from this are not part of the global routing table. We simply tell the tunnel to use the VRF for its NBMA routing with a simple command “tunnel vrf [name of VRF]“. Once this is enabled we can have a default route for LAN traffic via the tunnel and a default route for internet traffic.
A great write up by Denise Fishburn can be found here. http://www.networkingwithfish.com/tunnels-and-the-use-of-front-door-vrfs
IPsec – Using the front door VRF, we have to adjust the crypto keyring slightly to apply the VRF to the key and address.
Confirmation – From the HUB one command can tell us all we need to know from a DMVPN perspective. It shows the tunnel interface and number of peers, DMVPN peers and the QoS profiles they are subscribed to and shows the IPsec details.
HUB#show dmvpn detail
Interface Tunnel0 is up/up, Addr. is 10.100.0.4, VRF “”
Tunnel Src./Dest. addr: 172.16.3.10/MGRE, Tunnel VRF “internet”
Protocol/Transport: “multi-GRE/IP”, Protect “DMVPN”
Interface State Control: Disabled
nhrp event-publisher : Disabled
Type:Hub, Total NBMA Peers (v4/v6): 3
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb Target Network
—– ————— ————— —– ——– —– —————–
1 172.16.1.10 10.100.0.1 UP 02:54:32 D 10.100.0.1/32
NHRP group: Spoke-1
Output QoS service-policy applied: DMVPN_1500kbps
1 172.16.2.10 10.100.0.2 UP 02:54:32 D 10.100.0.2/32
NHRP group: Spoke-2
Output QoS service-policy applied: DMVPN_100kbps
1 172.16.5.10 10.100.0.5 UP 02:54:32 D 10.100.0.5/32
NHRP group: Spoke-5
Output QoS service-policy applied: DMVPN_5000kbps
Crypto Session Details:
——————————————————————————–
Interface: Tunnel0
Session: [0xF384C6B8]
IKEv1 SA: local 172.16.3.10/500 remote 172.16.1.10/500 Active
Capabilities:(none) connid:1005 lifetime:21:05:26
Crypto Session Status: UP-ACTIVE
fvrf: internet, Phase1_id: 172.16.1.10
IPSEC FLOW: permit 47 host 172.16.3.10 host 172.16.1.10
Active SAs: 2, origin: crypto map
Inbound: #pkts dec’ed 2635 drop 0 life (KB/Sec) 4150500/3474
Outbound: #pkts enc’ed 2655 drop 0 life (KB/Sec) 4150500/3474
Outbound SPI : 0x B4E148D, transform : esp-aes esp-sha-hmac
Socket State: Open
Here is the config for one of the spokes. The crypto config is identical to the HUB so omitted for brevity.
vrf definition internet
!
address-family ipv4
exit-address-family
ip route vrf internet 0.0.0.0 0.0.0.0 [internet next-hop ip]
interface Tunnel0
ip address 10.100.0.1 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication cisco
ip nhrp group Spoke-1
ip nhrp map 10.100.0.4 [NBMA – HUB]
ip nhrp map multicast [NBMA – HUB]
ip nhrp network-id 123
ip nhrp nhs 10.100.0.4
ip nhrp shortcut
ip tcp adjust-mss 1360
qos pre-classify
tunnel source Ethernet0/0
tunnel mode gre multipoint
tunnel key 123
tunnel vrf internet
tunnel protection ipsec profile DMVPN
router eigrp DMVPN
!
address-family ipv4 unicast autonomous-system 123
!
topology base
exit-af-topology
network 10.0.0.0
network 10.100.0.0 0.0.0.255
eigrp stub connected summary
exit-address-family
Confirmation on the spoke is similar to the hub using the “show dmvpn detail” command lists everything you need to know from a DMVPN and IPsec perspective.
Just to confirm everything is OK lets run some commands from the spoke
Spoke1# ping 10.0.2.1 sou lo 2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.2.1, timeout is 2 seconds:
Packet sent with a source address of 10.0.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 9/15/22 ms
!
Spoke1#traceroute 10.0.2.1 sou lo 2
Type escape sequence to abort.
Tracing the route to 10.0.2.1
VRF info: (vrf in name/id, vrf out name/id)
1 10.100.0.2 13 msec 9 msec 8 msec
!
Spoke1#sh ip nhrp shortcut
10.0.2.0/24 via 10.100.0.2
Tunnel0 created 00:00:26, expire 00:01:13
Type: dynamic, Flags: router used rib
NBMA address: 172.16.2.10
!
Spoke1#sh ip route nhrp
H 10.0.2.0/24 [250/1] via 10.100.0.2, 00:00:31, Tunnel0
!
Spoke1#sh ip route eigrp
Gateway of last resort is 10.100.0.4 to network 0.0.0.0
D* 0.0.0.0/0 [90/76800640] via 10.100.0.4, 03:07:27, Tunnel0
!
Spoke1#sh ip eigrp ne
EIGRP-IPv4 VR(DMVPN) Address-Family Neighbors for AS(123)
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
0 10.100.0.4 Tu0 13 03:08:04 23 1398 0 30
!
Spoke1#show ip cef 10.0.2.1
10.0.2.0/24
nexthop 10.100.0.2 Tunnel0
We can see spoke to spoke ping is ok, traceroute is direct to spoke, NHRP has a shortcut, the routing table has an NHRP route, and a default route (Global Routing Table) with one EIGRP neighbor. All looks good.
Troubleshoot – The best command to use is the “debug dmvpn detail all” this will debug crypto and nhrp. It can be a lot to look at so perhaps you might want to use the”debug crypto isakmp“or “debug crypto ipsec” individually, depends on which part is failing.
RH
all-things-networking 