One of the most interesting features i’ve noticed about LISP is the ability to encapsulate IPv6 packets and transport them between IPv4 only RLOC’s. This can be used for a transition strategy.

The best parts are you don’t have to configure tunnel endpoints, it works in a multi-homed environment and supports communications between LISP and non-LISP sites (with a PxTR). The document describes this as ‘IPv6 islands over an IPv4 core’.

I read about LISP online and watched the Cisco Live presentation from Berlin 2017 (BRKRST-3045). I wanted to see this for myself.

The lab is direct from If you want to lab this, go grab the PDF. The lab is pretty good and gives you the chance to see LISP in action, all the configs and step by step guides are provided.



Look at the packet from wireshark. This is a ping between R111 host and R117 Host. This was captured at the exit of  core R113 (before decapsulation). It has the IP header from R112 (ETR) to R116 (ITR), next is UDP dst.port 4341 (LISP), next LISP Data, then the IPv6 header and ICMP message. So its easy to see how this works.

packet Capture

For traceroute , the only hops that are visible are IPv6.


Before I could connect to anything in Site2, the ETR (R112) had to request mapping information from the MS/MR, once it was returned R112 then knew how to forward traffic towards 2001:DB8:B:1::3. From R112 perspective it now has a map for the prefix 2001:DB8:B::/48 and the RLOC is


If you want to know what LISP can be described as read this article by Ethan Banks



DMVPN | Phase 3 | IPsec | VRF | Per-Tunnel QoS

Covering the configuration, confirmation and troubleshooting of DMVPN Phase 3 with IPsec and Per-Tunnel QoS. The main reason behind this is to help out my friend who is implementing a production design similar to this.

Below is the high level topology used for this lab. Its a fairly simple diagram but gives you the idea, the devil is in the detail (config)


For the dynamic routing protocol, I have chosen EIGRP, you could use BGP if you want a hyper-scale design. Phase 3 DMVPN is chosen simply to enable spoke-to-spoke communication and maintain a default route to the spokes. Phase 3 allows this by using redirect messages / shortcut routing in NHRP.

The purpose of this blog is not to explain DMVPN, there are plenty of resources for that.

Here is the config for the HUB.

vrf definition internet
address-family ipv4
ip route vrf internet [internet next-hop IP]
interface Tunnel0
ip address
no ip redirects
ip mtu 1400
ip nhrp authentication cisco
ip nhrp map multicast dynamic
ip nhrp map group Spoke-5 service-policy output DMVPN_5000kbps
ip nhrp map group Spoke-1 service-policy output DMVPN_1500kbps
ip nhrp map group Spoke-2 service-policy output DMVPN_10000kbps
ip nhrp network-id 123
ip nhrp redirect
ip tcp adjust-mss 1360
qos pre-classify
tunnel source Ethernet0/0
tunnel mode gre multipoint
tunnel key 123
tunnel vrf internet
tunnel protection ipsec profile DMVPN
router eigrp DMVPN
address-family ipv4 unicast autonomous-system 123
af-interface Tunnel0
no split-horizon
topology base
crypto keyring dmvpn vrf internet
pre-shared-key address key topsecret!
crypto isakmp policy 100
encr aes 256
authentication pre-share
group 5
crypto ipsec transform-set DMVPN esp-aes esp-sha-hmac
crypto ipsec profile DMVPN
set transform-set DMVPN


QOS – On the hub tunnel interface you can set the QoS policies to map to a group name, then on the spoke you can set the command to have the tunnel subscribe to a particular group.

The QoS polices need to be setup in a specific way with parent and child policies, this way you can be specific in which spoke gets which shaper settings. Create a policy with your required settings see example below.

policy-map DMVPN
class icmp
police 8000
class telnet
set precedence 4
class www
police 80000
set precedence 6


Then create your shaping policy and attach the child policy like below.

policy-map DMVPN_5000kbps
class class-default
shape average 5000000
service-policy DMVPN


Front Door VRF – The reason for this is to allow a default route for internet access and also allow a default route for LAN traffic. The Internet facing interface is in its own VRF and routes from this are not part of the global routing table.  We simply tell the tunnel to use the VRF for its NBMA routing with a simple command “tunnel vrf [name of VRF]“. Once this is enabled we can have a default route for LAN traffic via the tunnel and a default route for internet traffic.

A great write up by Denise Fishburn can be found here.

IPsec – Using the front door VRF, we have to adjust the crypto keyring slightly to apply the VRF to the key and address.

Confirmation – From the HUB one command can tell us all we need to know from a DMVPN perspective. It shows the tunnel interface and number of peers,  DMVPN peers and the QoS profiles they are subscribed to and shows the IPsec details.

HUB#show dmvpn detail


Interface Tunnel0 is up/up, Addr. is, VRF “”
Tunnel Src./Dest. addr:, Tunnel VRF “internet”
Protocol/Transport: “multi-GRE/IP”, Protect “DMVPN”
Interface State Control: Disabled
nhrp event-publisher : Disabled
Type:Hub, Total NBMA Peers (v4/v6): 3
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb Target Network
—– ————— ————— —– ——– —– —————–
1 UP 02:54:32 D
NHRP group: Spoke-1
Output QoS service-policy applied: DMVPN_1500kbps
1 UP 02:54:32 D
NHRP group: Spoke-2
Output QoS service-policy applied: DMVPN_100kbps
1 UP 02:54:32 D
NHRP group: Spoke-5
Output QoS service-policy applied: DMVPN_5000kbps
Crypto Session Details:
Interface: Tunnel0
Session: [0xF384C6B8]
IKEv1 SA: local remote Active
Capabilities:(none) connid:1005 lifetime:21:05:26
Crypto Session Status: UP-ACTIVE
fvrf: internet, Phase1_id:
IPSEC FLOW: permit 47 host host
Active SAs: 2, origin: crypto map
Inbound: #pkts dec’ed 2635 drop 0 life (KB/Sec) 4150500/3474
Outbound: #pkts enc’ed 2655 drop 0 life (KB/Sec) 4150500/3474
Outbound SPI : 0x B4E148D, transform : esp-aes esp-sha-hmac
Socket State: Open


Here is the config for one of the spokes. The crypto config is identical to the HUB so omitted for brevity.

vrf definition internet
address-family ipv4
ip route vrf internet [internet next-hop ip]
interface Tunnel0
ip address
no ip redirects
ip mtu 1400
ip nhrp authentication cisco
ip nhrp group Spoke-1
ip nhrp map [NBMA – HUB]
ip nhrp map multicast [NBMA – HUB]
ip nhrp network-id 123
ip nhrp nhs
ip nhrp shortcut
ip tcp adjust-mss 1360
qos pre-classify
tunnel source Ethernet0/0
tunnel mode gre multipoint
tunnel key 123
tunnel vrf internet
tunnel protection ipsec profile DMVPN
router eigrp DMVPN
address-family ipv4 unicast autonomous-system 123
topology base
eigrp stub connected summary


Confirmation on the spoke is similar to the hub using the “show dmvpn detail” command lists everything you need to know from a DMVPN and IPsec perspective.

Just to confirm everything is OK lets run some commands from the spoke

Spoke1# ping sou lo 2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to, timeout is 2 seconds:
Packet sent with a source address of
Success rate is 100 percent (5/5), round-trip min/avg/max = 9/15/22 ms
Spoke1#traceroute sou lo 2
Type escape sequence to abort.
Tracing the route to
VRF info: (vrf in name/id, vrf out name/id)
1 13 msec 9 msec 8 msec
Spoke1#sh ip nhrp shortcut via
Tunnel0 created 00:00:26, expire 00:01:13
Type: dynamic, Flags: router used rib
NBMA address:
Spoke1#sh ip route nhrp
H [250/1] via, 00:00:31, Tunnel0
Spoke1#sh ip route eigrp
Gateway of last resort is to network
D* [90/76800640] via, 03:07:27, Tunnel0
Spoke1#sh ip eigrp ne
EIGRP-IPv4 VR(DMVPN) Address-Family Neighbors for AS(123)
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
0 Tu0 13 03:08:04 23 1398 0 30
Spoke1#show ip cef
nexthop Tunnel0


We can see spoke to spoke ping is ok, traceroute is direct to spoke, NHRP has a shortcut, the routing table has an NHRP route, and a default route (Global Routing Table) with one EIGRP neighbor. All looks good.

Troubleshoot – The best command to use is the “debug dmvpn detail all” this will debug crypto and nhrp.  It can be a lot to look at so perhaps you might want to use the”debug crypto isakmp“or “debug crypto ipsec” individually, depends on which part is failing.


IPv4 MTU issues can be hard to spot initially, there is a solution and its called Path MTU Discovery (RFC1191). The RFC describes it as the following “a technique for using the Don’t Fragment (DF) bit in the IP header to dynamically discover the PMTU of a path”

Further to that the RFC states “The basic idea is that a source host initially assumes that the PMTU of a path is the (known) MTU of its first hop, and sends all datagrams on that path with the DF bit set. If any of the datagrams are too large to be forwarded without fragmentation by some router along the path, that router will discard them and return ICMP Destination Unreachable messages with a code meaning “fragmentation needed and DF set” (Type 3, code 4)

The unfortunate issue is that the message that’s sent back doesn’t actually say what the MTU is.

A colleague of mines who is a Windows 7 expert, has reliably informed me that by default Windows 7 has PMTUD enabled.

The important point to focus on is the ICMP unreachable (Type 3, code 4). To put this quite simply, if you don’t receive an ICMP message back with the code for fragmentation needed then, your PC will assume that the MTU is fine and continue to send the packets even though somewhere in the path the packets are potentially being dropped.

There can be a number of reasons for this, including firewalls blocking the message, ICMP unreachable disabled on an interface, a transparent host between 2 endpoints (Often done in service provider networks) that has a lower MTU value.

I recently ran into an issue where IP connectivity between 2 sites looked to be fine, ping, traceroute and SSH were all working, but certain applications and protocols were not, most notably HTTPS.

Below I will explain how to spot this issue.

Take a look at the diagram below, i have deliberately used a transparent device as its most likely what you might see in a L3VPN (MPLS) network. The last mile provider provides a layer 2 path (perhaps a L2TPv3) from CE to PE and the underlying hops are hidden from us.  From the service provider perspective the routers are directly connected.

This is perhaps where an MTU issue has occurred. For this scenario I have reduced it quite significantly for effect.


Lets say for example you have a perfectly functioning network where MTU is fine along the path. Initially you can send a ping with 1460bytes and you will get a reply. Lets increase this to something we know is to big (1550bytes). This works great in a perfectly functioning network where you receive an ICMP type 3, you will get the “packet needs to be fragmented but DF set” message.


Now lets try that through our network where the MTU is set lower but the sending device doesn’t know about it.


At first you think its OK because you can ping along the path and get a reply, you try SSH and it works too. Now lets try to ping with different MTU sizes. Remember your PC doesn’t receive the ICMP message this time, so what happens is you get a “request timed out” message.


The reason for that is the packet is being dropped and the ICMP message isn’t being returned. If I ping with an MTU that is lower than the 1000 i get a reply.


Now the question, why would HTTPS not work? well in some cases web applications or your client might set the Do Not Fragement bit in the IP header SYN request. This means the packet should not be fragmented, so when we send this on our network with the bad MTU in the path, the packet is dropped and the sending device never receives the ICMP message. It never knows that it has to reduce the MTU value. The packet capture below shows where the DF bit is set.


I had a look through the RFC2246 for TLS1.0 and it doesn’t specify that the DF bit should be set. It’s most likely a vendor or O/S specific setting, so your observed results may differ from vendor to vendor.


Ethernet traffic is transported in frames, the maximum size of this frame is known as the Maximum Transmission Unit (MTU). By default an Ethernet packet has an MTU size of 1500 bytes. An Ethernet packet larger than 1500 bytes is known as a jumbo frame.

When a network endpoint receives a packet that is larger than its MTU, it can either fragment the packet into smaller parts or drop the packet.

Fragmentation is the process of breaking up a large packet into smaller chunks before sending, the receiving host then re-assembles the packets. This is not desirable when trying to optimize your network. The main reason I say this is because if one fragment is lost the entire datagram is lost. IP has no error correction and UDP has no re-transmission feature. TCP re-transmits the whole datagram if one fragment is lost.

An excellent document which explains fragmentation in detail can be found at the link below.

Increasing the MTU can improve network efficiency, this is achieved by transmitting larger payloads within the packets. Changing the MTU on a LAN should be planned out correctly, performance issues can be expected if the MTU of one device is different from the MTU of another device. What you might think should improve performance is likely to have the opposite effect.

The recommended approach is to increase the MTU on all of your network devices in the LAN, the network would then at that point benefit from less fragmentation and less overhead. Your network topology and traffic flows should be fully understood before making these changes.

To demonstrate we can use the ping utility on a windows 7 PC. This particular PC is connected to a switch with an MTU size of 1500 bytes.

A ping packet has a default value of 32 bytes, we will try to send a ping with the default MTU and then send a ping with the jumbo MTU value and see what happens.


Pinging with 32 bytes of data:
Reply from bytes=32 time=12ms TTL=254
Reply from bytes=32 time=12ms TTL=254
Reply from bytes=32 time=12ms TTL=254
Reply from bytes=32 time=12ms TTL=254

We can see the latency is 12 milliseconds.

Now send a packet with a jumbo frame, we will see the impact of fragmentation. The -l switch allows is to increase the packet size.

C:\Users\rhill>ping -l 9000

Pinging with 9000 bytes of data:
Reply from bytes=9000 time=19ms TTL=254
Reply from bytes=9000 time=24ms TTL=254
Reply from bytes=9000 time=20ms TTL=254
Reply from bytes=9000 time=20ms TTL=254

We can see that the latency has increased, this is due to the fragmentation that has taken place. You may not experience increased latency if you are on a local LAN, but the packets are still being fragmented.

A WireShark capture shows the extra work that needs to be done on the sending device to fragment the packet, the receiving device then has to re-assemble this at the opposite end.

The output below shows the packet being split into 6 different parts in order to send the jumbo frame.


As you can see, simply increasing the MTU on a sending device may not actually improve the efficiency of your network.

It’s worth noting that when using TCP the MSS is negotiated in the 3-way handshake and will default to the lowest value.


Did you know that OSPF neighbors do not move to the  FULL state with mismatched MTU?

I found this out at the weekend when I was working on some Data Centre switches, within the fabric these switches use jumbo MTU. So when I tried to peer a device that was not part of the fabric, I got stuck in EXSTART.

At first I was wondering if I had the OSPF configuration. I checked and double checked but all looked good.


run “debug ip ospf adj” you will get a message similar to this.

*Nov 16 21:30:45.551: OSPF-1 ADJ Gi0/0: Nbr has smaller interface MTU

Answer: match the MTU on both sides.
I had a read through TCP/IP Volume 1, It doesn’t mention MTU size anywhere. RFC 2328 does mention it.

If the Interface MTU field in the Database Description packet
indicates an IP datagram size that is larger than the router can
accept on the receiving interface without fragmentation, the
Database Description packet is rejected.

This is where wireshark comes in handy, I had to see this for myself.

The MTU isnt sent in the Hello packet its sent in the type 2 DBD packet, this is after the neighbors acknowledge each other (2WAY). See below.




Some time ago I setup SIP trunking , I had to configure voice translation rules on the CUBE. Here’s are a few I setup that might help you (numbers changed). Once the numbers were translated they would match dial-patterns and be routed respectively.

Using regular expression.

Number 1 translates an incoming call to a 4 digit extension.

voice translation-rule 1
rule 1 /^02081234567/ /1111/

R1#test voice translation-rule 1 02081234567
Matched with rule 1
Original number: 02081234567 Translated number: 1111

Number 2 takes any number and adds a 9 to the beginning.

voice translation-rule 2
rule 1 /^.*/ /9\0/

R1#test voice translation-rule 2 02081234567
Matched with rule 1
Original number: 02081234567 Translated number: 902081234567

Number  3 translates a dialed number beginning with +44 or + to 90 or 900.

voice translation-rule 3
rule 1 /^\+44/ /90/
rule 2 /^\+/ /900/

R1#test voice translation-rule 3 +442081234567
Matched with rule 1
Original number: +442081234567 Translated number: 902081234567

R1#test voice translation-rule 3 +1123456789
Matched with rule 2
Original number: +1123456789 Translated number: 9001123456789

Number 4 translates a 4 digit number beginning with 5, drops the 5 and appends a country code and an area code.

voice translation-rule 4
rule 1 /^5\(…$\)/ /00442081234\1/

R1#test voice translation-rule 4 5678
Matched with rule 1
Original number: 5678 Translated number: 00442081234678

Number 5 translates any number beginning with 901* to 0044 and any number beginning with 900* drops the 9.

R1#test voice translation-rule 5 9020812345678
Matched with rule 1
Original number: 9020812345678 Translated number: 004420812345678

R1#test voice translation-rule 5 90012345678910
Matched with rule 2
Original number: 90012345678910 Translated number: 0012345678910


I was asked question about MPLS labels within a VRF. Well the question was ” when I type the command ‘show mpls ldp bindings vrf xxx‘ ” I don’t see any labels for my remote site prefixes.

In this case these labels aren’t assigned by LDP they are assigned by BGP.

The command he was looking for was “show ip bgp vpnv4 vrf xxx labels” I think a light went on in his head when i explained this to him!

If your still confused , imagine the LSP (unidirectional) R1 -> R4 is basically a virtual connection if you like , where R4 is the egress LSR for the packet. The packet will travel via R1 -> R2 -> R3 -> R4. The next hop actually is listed as R4 even though it has to pass through R2 and R3. Which by the way has it own labels to add on top of bottom of stack label for R4.


R1 pushes on the label 1234 for R4 and another label on top for R2 which is 22, next R2 removes its own label (22) and adds the label for R3 which is 33, R3 removes its own label (33) and forwards the packet with the label 1234 to R4. Bottom of stack bit is set to 1, R4 removes the label and forwards.