Archives for category: Network

When troubleshooting traffic flows through the Palo Alto NGFW, it can be difficult to see what’s happening. Using the logs from the GUI can help.

An alternative is, to use the command line “show session all filter destination [IP]“. This shows you a filtered view of stateful sessions going through the firewall. It provides information like state, source/destination, translated IP/Port. An example is shown below. (Use the context sensitive help if you need more options)

admin@PA-FW> show session all filter destination

ID Application State Type Flag Src[Sport]/Zone/Proto (translated IP[Port])
Vsys Dst[Dport]/Zone (translated IP[Port])
37167 web-browsing ACTIVE FLOW *NS[55927]/trust/6 ([40416])
vsys1[443]/untrust ([443])

Once you have the session ID (in the case above the session ID is 37167), use the command “show session id 37167” to drill down further into the information within that session.

admin@PA-FW> show session id 37167
Session 37167

c2s flow:
 source: [trust]
 proto: 6
 sport: 55927 dport: 443
 state: ACTIVE type: FLOW
 src user: test\robert
 dst user: unknown
 qos node: ethernet1/1, qos member Qid 0
 match src interface: any
 match src address: ('any ',)

s2c flow:
 source: [untrust]
 proto: 6
 sport: 443 dport: 40416
 state: ACTIVE type: FLOW
 src user: unknown
 dst user: test\robert

start time : Fri Jan 26 10:03:53 2018
 timeout : 60 sec
 time to live : 48 sec
 total byte count(c2s) : 1924
 total byte count(s2c) : 5115
 layer7 packet count(c2s) : 14
 layer7 packet count(s2c) : 10
 vsys : vsys1
 application : web-browsing
 rule : Rule 2
 session to be logged at end : True
 session in session ager : True
 session synced from HA peer : False
 address/port translation : source + destination
 nat-rule : NAT-Outside(vsys1)
 layer7 processing : completed
 URL filtering enabled : True
 URL category : Rule 2
 session via syn-cookies : False
 session terminated on host : False
 session traverses tunnel : False
 captive portal session : False
 ingress interface : ethernet1/2
 egress interface : ethernet1/1
 session QoS rule : N/A (class 4)
 tracker stage l7proc : proxy timer expired

As you can see above, the amount of information can be very helpful for troubleshooting. If you suspect a particular session is causing a problem, you can clear with the command “clear session ID 37167



This feature is similar to Cisco IPSLA, in that it tracks the reachability of a destination and can remove static routes based on the ping response.

Simple topology with Palo-Alto connected to the internet and using path monitor on the default route. Internal interface peering OSPF with the core router and redistributing the static route but only when the ping responds.











First create the static route
Network -> Virtual Routers -> (router) -> Static Routes -> Add+

Virtual Router - Static Routes

In this scenario the path monitor will ping the opposite side of the link and Google DNS, both must fail for the condition to be met. Interval and count are default (5 pings 3 seconds apart). Once the pings fail, the route will be removed from the routing table. When the router is able to ping the destination after a failure it waits 2 minutes before re-installing the route, this is default preemptive behaviour and can be changed.

Next create a redistribution profile that redistributes your routes, what I found was that if you redistribute ‘’ that means all routes, if you have other routes you don’t want to redistribute just match them with a lower priority and choose ‘No redist

Network -> Virtual Routers -> (router) -> Redistribution Profile -> Add+

Redistribution Profile

Configure OSPF as you normally would with any other device no difference here the usual attributes must match. Area 0 is the same as Area

Next apply the redistribution profile to OSPF and check ‘Allow Redistribute Default Route‘. You have the option to set external type, metric and tag.

Network -> Virtual Routers -> (router) -> OSPF -> Export Rules -> Add+

Export Rules

The Palo-Alto should have formed neighbors with the core router and be redistributing the default route. This can be seen here. Network -> Virtual Routers -> More Runtime Stats. You can also view the routing table here and the forwarding table along with OSPF neighbors etc.

Run Time Stats

Currently the core router receives the route from the Palo-Alto


Next fail the routing on the internet router to see the impact on the path monitoring. The outcome is, the route is withdrawn (debug ip routing)

Route withdrawn

Path Monitor (down)

On the core device you may have a floating static or default route with a higher metric from a different IGP, waiting to take over in the event of a failure to the Palo-Alto.

When routing is restored you can view the preempted route counting down. After the 2 minutes the route is re-instated.

Preempt hold

That’s it, works great.


Dynamic Virtual Tunnel Interface.

It’s a similar concept to DMVPN but with a few differences

  • dVTI requires a smaller packet header only 4 bytes compared to DMVPN which is an additional 28 bytes
  • dVTI does not  use NHRP
  • dVTI has backwards compatibility with IPsec direct encapsulation
  • dVTI requires IP unumbered
  • dVTI requires the use of a dynamic routing protocol instead of keepalives
  • dVTI must be initiated by the remote branch to the head-end

I personally like the fact that the interface is unnumbered as it reduces the amount of IP address space that you need to manage. Each virtual template can be configured with different characteristics on the head-end device so that common branch offices share the same settings. This type of solution is ideal for a hub and spoke design.

I’m not saying dVTI is better or worse than DMVPN it’s just different.

The spoke forms an EIGRP neighbor with the HUB, you can then advertise a default route to the spoke. If you want to apply policies like QoS you can do this directly on the template interface.  The spoke will advertise the LAN network to the Hub and you can then summarise at the Hub into the data centre for multiple spokes.

Here is the configuration with a front VRF, this was tested in the lab on live equipment.

Hub Configuration

vrf definition internet
 rd 1:1
 address-family ipv4
 crypto keyring KEYRING vrf internet 
  pre-shared-key address [IP ADDRESS of SPOKE or match all] key cisco
 crypto isakmp policy 10
  encr aes
  hash sha256
  authentication pre-share
  group 14

crypto isakmp profile ISAKMPPROFILE
  keyring KEYRING
  match identity address [IP ADDRESS of SPOKE or match all] internet
  virtual-template 1

crypto ipsec transform-set TSET_SECURE esp-aes esp-sha256-hmac

interface Loopback0
  ip address
 interface gig0/0
  vrf forwarding internet
  description outside_interface
  ip address [WAN Interface IP and Subnet Mask] 
  no ip redirects
  no ip unreachables
  no ip proxy-arp

interface gig0/1
  description inside_interface
  ip address

interface Virtual-Template1 type tunnel
  ip unnumbered Loopback0
  ip mtu 1408
  ip summary-address eigrp 1
  tunnel mode ipsec ipv4
  tunnel vrf internet
  tunnel protection ipsec profile IPSECPROFILE_SECURE

router eigrp 1

ip route vrf internet [next-hop to internet]

Spoke Configuration

vrf definition internet
 rd 1:1
  address-family ipv4

crypto keyring 1 vrf internet 
  pre-shared-key address [IP address of HUB] cisco

crypto isakmp policy 10
  encr aes
  hash sha256
  authentication pre-share
  group 14

crypto ipsec transform-set TSET_SECURE esp-aes esp-sha256-hmac

crypto ipsec profile IPSECPROFILE_SECURE
  set transform-set TSET_SECURE

interface Loopback0
 ip address
 interface Tunnel0
  ip unnumbered Loopback0
  ip mtu 1408
  tunnel source gig0/1
  tunnel mode ipsec ipv4
  tunnel destination [IP address of HUB]
  tunnel vrf internet
  tunnel protection ipsec profile IPSECPROFILE_SECURE

interface gig0/1
  description outside_interface
  vrf forwarding internet
  ip address [WAN Interface IP and Subnet Mask or DHCP]

interface vlan 1 
  description inside_interface
  ip address
  no shut

router eigrp 1
  eigrp stub connected summary

To verify you can use commands like below.

“show ip int brief”
“show ip interface virtual-access1”
“show ip eigrp neighbors”
“show crypto isakmp sa”
“show crypto engine connections active”

Update: You might notice I have used classic EIGRP instead of EIGRP named mode, the reason was due to the fact that under the af-interface virtua- template I wasn’t able to set the summary route, the command was accepted but wasn’t being sent to the neighbor. Perhaps this is a bug in the version I was running.


IPv4 MTU issues can be hard to spot initially, there is a solution and its called Path MTU Discovery (RFC1191). The RFC describes it as the following “a technique for using the Don’t Fragment (DF) bit in the IP header to dynamically discover the PMTU of a path”

Further to that the RFC states “The basic idea is that a source host initially assumes that the PMTU of a path is the (known) MTU of its first hop, and sends all datagrams on that path with the DF bit set. If any of the datagrams are too large to be forwarded without fragmentation by some router along the path, that router will discard them and return ICMP Destination Unreachable messages with a code meaning “fragmentation needed and DF set” (Type 3, code 4)

The unfortunate issue is that the message that’s sent back doesn’t actually say what the MTU is.

A colleague of mines who is a Windows 7 expert, has reliably informed me that by default Windows 7 has PMTUD enabled.

The important point to focus on is the ICMP unreachable (Type 3, code 4). To put this quite simply, if you don’t receive an ICMP message back with the code for fragmentation needed then, your PC will assume that the MTU is fine and continue to send the packets even though somewhere in the path the packets are potentially being dropped.

There can be a number of reasons for this, including firewalls blocking the message, ICMP unreachable disabled on an interface, a transparent host between 2 endpoints (Often done in service provider networks) that has a lower MTU value.

I recently ran into an issue where IP connectivity between 2 sites looked to be fine, ping, traceroute and SSH were all working, but certain applications and protocols were not, most notably HTTPS.

Below I will explain how to spot this issue.

Take a look at the diagram below, i have deliberately used a transparent device as its most likely what you might see in a L3VPN (MPLS) network. The last mile provider provides a layer 2 path (perhaps a L2TPv3) from CE to PE and the underlying hops are hidden from us.  From the service provider perspective the routers are directly connected.

This is perhaps where an MTU issue has occurred. For this scenario I have reduced it quite significantly for effect.


Lets say for example you have a perfectly functioning network where MTU is fine along the path. Initially you can send a ping with 1460bytes and you will get a reply. Lets increase this to something we know is to big (1550bytes). This works great in a perfectly functioning network where you receive an ICMP type 3, you will get the “packet needs to be fragmented but DF set” message.


Now lets try that through our network where the MTU is set lower but the sending device doesn’t know about it.


At first you think its OK because you can ping along the path and get a reply, you try SSH and it works too. Now lets try to ping with different MTU sizes. Remember your PC doesn’t receive the ICMP message this time, so what happens is you get a “request timed out” message.


The reason for that is the packet is being dropped and the ICMP message isn’t being returned. If I ping with an MTU that is lower than the 1000 i get a reply.


Now the question, why would HTTPS not work? well in some cases web applications or your client might set the Do Not Fragement bit in the IP header SYN request. This means the packet should not be fragmented, so when we send this on our network with the bad MTU in the path, the packet is dropped and the sending device never receives the ICMP message. It never knows that it has to reduce the MTU value. The packet capture below shows where the DF bit is set.


I had a look through the RFC2246 for TLS1.0 and it doesn’t specify that the DF bit should be set. It’s most likely a vendor or O/S specific setting, so your observed results may differ from vendor to vendor.


Ran into this at the weekend when I was working on some Data Centre switches, within the fabric the switches use jumbo MTU. So when I tried to peer a device that was not part of the fabric, I got stuck in EXSTART.

run “debug ip ospf adj” you will get a message similar to this.

OSPF-1 ADJ Gi0/0: Nbr has smaller interface MTU

Answer: match the MTU on both sides.
I had a read through TCP/IP Volume 1, It doesn’t mention MTU size anywhere. RFC 2328 does mention it.

If the Interface MTU field in the Database Description packet
indicates an IP datagram size that is larger than the router can
accept on the receiving interface without fragmentation, the
Database Description packet is rejected.

This is where wireshark comes in handy.

The MTU isnt sent in the Hello packet its sent in the type 2 DBD packet, this is after the neighbors acknowledge each other (2WAY). See below.



Some time ago I setup SIP trunking , I had to configure voice translation rules on the CUBE. Here’s are a few I setup that might help you (numbers changed). Once the numbers were translated they would match dial-patterns and be routed respectively.

Using regular expression.

Number 1 translates an incoming call to a 4 digit extension.

voice translation-rule 1
rule 1 /^02081234567/ /1111/

R1#test voice translation-rule 1 02081234567
Matched with rule 1
Original number: 02081234567 Translated number: 1111

Number 2 takes any number and adds a 9 to the beginning.

voice translation-rule 2
rule 1 /^.*/ /9\0/

R1#test voice translation-rule 2 02081234567
Matched with rule 1
Original number: 02081234567 Translated number: 902081234567

Number  3 translates a dialed number beginning with +44 or + to 90 or 900.

voice translation-rule 3
rule 1 /^\+44/ /90/
rule 2 /^\+/ /900/

R1#test voice translation-rule 3 +442081234567
Matched with rule 1
Original number: +442081234567 Translated number: 902081234567

R1#test voice translation-rule 3 +1123456789
Matched with rule 2
Original number: +1123456789 Translated number: 9001123456789

Number 4 translates a 4 digit number beginning with 5, drops the 5 and appends a country code and an area code.

voice translation-rule 4
rule 1 /^5\(…$\)/ /00442081234\1/

R1#test voice translation-rule 4 5678
Matched with rule 1
Original number: 5678 Translated number: 00442081234678

Number 5 translates any number beginning with 901* to 0044 and any number beginning with 900* drops the 9.

R1#test voice translation-rule 5 9020812345678
Matched with rule 1
Original number: 9020812345678 Translated number: 004420812345678

R1#test voice translation-rule 5 90012345678910
Matched with rule 2
Original number: 90012345678910 Translated number: 0012345678910


I was asked question about MPLS labels within a VRF. Well the question was ” when I type the command ‘show mpls ldp bindings vrf xxx‘ ” I don’t see any labels for my remote site prefixes.

In this case these labels aren’t assigned by LDP they are assigned by BGP.

The command he was looking for was “show ip bgp vpnv4 vrf xxx labels” I think a light went on in his head when i explained this to him!

If your still confused , imagine the LSP (unidirectional) R1 -> R4 is basically a virtual connection if you like , where R4 is the egress LSR for the packet. The packet will travel via R1 -> R2 -> R3 -> R4. The next hop actually is listed as R4 even though it has to pass through R2 and R3. Which by the way has it own labels to add on top of bottom of stack label for R4.


R1 pushes on the label 1234 for R4 and another label on top for R2 which is 22, next R2 removes its own label (22) and adds the label for R3 which is 33, R3 removes its own label (33) and forwards the packet with the label 1234 to R4. Bottom of stack bit is set to 1, R4 removes the label and forwards.