Archives for category: Security

DMVPN | Phase 3 | IPsec | VRF | Per-Tunnel QoS

Covering the configuration, confirmation and troubleshooting of DMVPN Phase 3 with IPsec and Per-Tunnel QoS. The main reason behind this is to help out my friend who is implementing a production design similar to this.

Below is the high level topology used for this lab. Its a fairly simple diagram but gives you the idea, the devil is in the detail (config)

DMVPN

For the dynamic routing protocol, I have chosen EIGRP, you could use BGP if you want a hyper-scale design. Phase 3 DMVPN is chosen simply to enable spoke-to-spoke communication and maintain a default route to the spokes. Phase 3 allows this by using redirect messages / shortcut routing in NHRP.

The purpose of this blog is not to explain DMVPN, there are plenty of resources for that.

Here is the config for the HUB.

vrf definition internet
!
address-family ipv4
exit-address-family
ip route vrf internet 0.0.0.0 0.0.0.0 [internet next-hop IP]
!
interface Tunnel0
ip address 10.100.0.4 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication cisco
ip nhrp map multicast dynamic
ip nhrp map group Spoke-5 service-policy output DMVPN_5000kbps
ip nhrp map group Spoke-1 service-policy output DMVPN_1500kbps
ip nhrp map group Spoke-2 service-policy output DMVPN_10000kbps
ip nhrp network-id 123
ip nhrp redirect
ip tcp adjust-mss 1360
qos pre-classify
tunnel source Ethernet0/0
tunnel mode gre multipoint
tunnel key 123
tunnel vrf internet
tunnel protection ipsec profile DMVPN
!
router eigrp DMVPN
!
address-family ipv4 unicast autonomous-system 123
!
af-interface Tunnel0
summary-address 0.0.0.0 0.0.0.0
no split-horizon
exit-af-interface
!
topology base
exit-af-topology
network 10.0.0.0
network 10.100.0.0 0.0.0.255
exit-address-family
crypto keyring dmvpn vrf internet
pre-shared-key address 0.0.0.0 0.0.0.0 key topsecret!
!
crypto isakmp policy 100
encr aes 256
authentication pre-share
group 5
!
crypto ipsec transform-set DMVPN esp-aes esp-sha-hmac
!
crypto ipsec profile DMVPN
set transform-set DMVPN

 

QOS – On the hub tunnel interface you can set the QoS policies to map to a group name, then on the spoke you can set the command to have the tunnel subscribe to a particular group.

The QoS polices need to be setup in a specific way with parent and child policies, this way you can be specific in which spoke gets which shaper settings. Create a policy with your required settings see example below.

policy-map DMVPN
class icmp
police 8000
class telnet
set precedence 4
class www
police 80000
set precedence 6

 

Then create your shaping policy and attach the child policy like below.

policy-map DMVPN_5000kbps
class class-default
shape average 5000000
service-policy DMVPN

 

Front Door VRF – The reason for this is to allow a default route for internet access and also allow a default route for LAN traffic. The Internet facing interface is in its own VRF and routes from this are not part of the global routing table.  We simply tell the tunnel to use the VRF for its NBMA routing with a simple command “tunnel vrf [name of VRF]“. Once this is enabled we can have a default route for LAN traffic via the tunnel and a default route for internet traffic.

A great write up by Denise Fishburn can be found here. http://www.networkingwithfish.com/tunnels-and-the-use-of-front-door-vrfs

IPsec – Using the front door VRF, we have to adjust the crypto keyring slightly to apply the VRF to the key and address.

Confirmation – From the HUB one command can tell us all we need to know from a DMVPN perspective. It shows the tunnel interface and number of peers,  DMVPN peers and the QoS profiles they are subscribed to and shows the IPsec details.

HUB#show dmvpn detail

 

Interface Tunnel0 is up/up, Addr. is 10.100.0.4, VRF “”
Tunnel Src./Dest. addr: 172.16.3.10/MGRE, Tunnel VRF “internet”
Protocol/Transport: “multi-GRE/IP”, Protect “DMVPN”
Interface State Control: Disabled
nhrp event-publisher : Disabled
Type:Hub, Total NBMA Peers (v4/v6): 3
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb Target Network
—– ————— ————— —– ——– —– —————–
1 172.16.1.10 10.100.0.1 UP 02:54:32 D 10.100.0.1/32
NHRP group: Spoke-1
Output QoS service-policy applied: DMVPN_1500kbps
1 172.16.2.10 10.100.0.2 UP 02:54:32 D 10.100.0.2/32
NHRP group: Spoke-2
Output QoS service-policy applied: DMVPN_100kbps
1 172.16.5.10 10.100.0.5 UP 02:54:32 D 10.100.0.5/32
NHRP group: Spoke-5
Output QoS service-policy applied: DMVPN_5000kbps
Crypto Session Details:
——————————————————————————–
Interface: Tunnel0
Session: [0xF384C6B8]
IKEv1 SA: local 172.16.3.10/500 remote 172.16.1.10/500 Active
Capabilities:(none) connid:1005 lifetime:21:05:26
Crypto Session Status: UP-ACTIVE
fvrf: internet, Phase1_id: 172.16.1.10
IPSEC FLOW: permit 47 host 172.16.3.10 host 172.16.1.10
Active SAs: 2, origin: crypto map
Inbound: #pkts dec’ed 2635 drop 0 life (KB/Sec) 4150500/3474
Outbound: #pkts enc’ed 2655 drop 0 life (KB/Sec) 4150500/3474
Outbound SPI : 0x B4E148D, transform : esp-aes esp-sha-hmac
Socket State: Open

 

Here is the config for one of the spokes. The crypto config is identical to the HUB so omitted for brevity.

vrf definition internet
!
address-family ipv4
exit-address-family
ip route vrf internet 0.0.0.0 0.0.0.0 [internet next-hop ip]
interface Tunnel0
ip address 10.100.0.1 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication cisco
ip nhrp group Spoke-1
ip nhrp map 10.100.0.4 [NBMA – HUB]
ip nhrp map multicast [NBMA – HUB]
ip nhrp network-id 123
ip nhrp nhs 10.100.0.4
ip nhrp shortcut
ip tcp adjust-mss 1360
qos pre-classify
tunnel source Ethernet0/0
tunnel mode gre multipoint
tunnel key 123
tunnel vrf internet
tunnel protection ipsec profile DMVPN
router eigrp DMVPN
!
address-family ipv4 unicast autonomous-system 123
!
topology base
exit-af-topology
network 10.0.0.0
network 10.100.0.0 0.0.0.255
eigrp stub connected summary
exit-address-family

 

Confirmation on the spoke is similar to the hub using the “show dmvpn detail” command lists everything you need to know from a DMVPN and IPsec perspective.

Just to confirm everything is OK lets run some commands from the spoke

Spoke1# ping 10.0.2.1 sou lo 2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.2.1, timeout is 2 seconds:
Packet sent with a source address of 10.0.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 9/15/22 ms
!
Spoke1#traceroute 10.0.2.1 sou lo 2
Type escape sequence to abort.
Tracing the route to 10.0.2.1
VRF info: (vrf in name/id, vrf out name/id)
1 10.100.0.2 13 msec 9 msec 8 msec
!
Spoke1#sh ip nhrp shortcut
10.0.2.0/24 via 10.100.0.2
Tunnel0 created 00:00:26, expire 00:01:13
Type: dynamic, Flags: router used rib
NBMA address: 172.16.2.10
!
Spoke1#sh ip route nhrp
H 10.0.2.0/24 [250/1] via 10.100.0.2, 00:00:31, Tunnel0
!
Spoke1#sh ip route eigrp
Gateway of last resort is 10.100.0.4 to network 0.0.0.0
D* 0.0.0.0/0 [90/76800640] via 10.100.0.4, 03:07:27, Tunnel0
!
Spoke1#sh ip eigrp ne
EIGRP-IPv4 VR(DMVPN) Address-Family Neighbors for AS(123)
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
0 10.100.0.4 Tu0 13 03:08:04 23 1398 0 30
!
Spoke1#show ip cef 10.0.2.1

10.0.2.0/24
nexthop 10.100.0.2 Tunnel0

 

We can see spoke to spoke ping is ok, traceroute is direct to spoke, NHRP has a shortcut, the routing table has an NHRP route, and a default route (Global Routing Table) with one EIGRP neighbor. All looks good.

Troubleshoot – The best command to use is the “debug dmvpn detail all” this will debug crypto and nhrp.  It can be a lot to look at so perhaps you might want to use the”debug crypto isakmp“or “debug crypto ipsec” individually, depends on which part is failing.

RH

Advertisements

IPv4 MTU issues can be hard to spot initially, there is a solution and its called Path MTU Discovery (RFC1191). The RFC describes it as the following “a technique for using the Don’t Fragment (DF) bit in the IP header to dynamically discover the PMTU of a path”

Further to that the RFC states “The basic idea is that a source host initially assumes that the PMTU of a path is the (known) MTU of its first hop, and sends all datagrams on that path with the DF bit set. If any of the datagrams are too large to be forwarded without fragmentation by some router along the path, that router will discard them and return ICMP Destination Unreachable messages with a code meaning “fragmentation needed and DF set” (Type 3, code 4)

The unfortunate issue is that the message that’s sent back doesn’t actually say what the MTU is.

A colleague of mines who is a Windows 7 expert, has reliably informed me that by default Windows 7 has PMTUD enabled.

The important point to focus on is the ICMP unreachable (Type 3, code 4). To put this quite simply, if you don’t receive an ICMP message back with the code for fragmentation needed then, your PC will assume that the MTU is fine and continue to send the packets even though somewhere in the path the packets are potentially being dropped.

There can be a number of reasons for this, including firewalls blocking the message, ICMP unreachable disabled on an interface, a transparent host between 2 endpoints (Often done in service provider networks) that has a lower MTU value.

I recently ran into an issue where IP connectivity between 2 sites looked to be fine, ping, traceroute and SSH were all working, but certain applications and protocols were not, most notably HTTPS.

Below I will explain how to spot this issue.

Take a look at the diagram below, i have deliberately used a transparent device as its most likely what you might see in a L3VPN (MPLS) network. The last mile provider provides a layer 2 path (perhaps a L2TPv3) from CE to PE and the underlying hops are hidden from us.  From the service provider perspective the routers are directly connected.

This is perhaps where an MTU issue has occurred. For this scenario I have reduced it quite significantly for effect.

Capture3

Lets say for example you have a perfectly functioning network where MTU is fine along the path. Initially you can send a ping with 1460bytes and you will get a reply. Lets increase this to something we know is to big (1550bytes). This works great in a perfectly functioning network where you receive an ICMP type 3, you will get the “packet needs to be fragmented but DF set” message.

Capture2

Now lets try that through our network where the MTU is set lower but the sending device doesn’t know about it.

Capture4

At first you think its OK because you can ping along the path and get a reply, you try SSH and it works too. Now lets try to ping with different MTU sizes. Remember your PC doesn’t receive the ICMP message this time, so what happens is you get a “request timed out” message.

Capture5

The reason for that is the packet is being dropped and the ICMP message isn’t being returned. If I ping with an MTU that is lower than the 1000 i get a reply.

Capture6

Now the question, why would HTTPS not work? well in some cases web applications or your client might set the Do Not Fragement bit in the IP header SYN request. This means the packet should not be fragmented, so when we send this on our network with the bad MTU in the path, the packet is dropped and the sending device never receives the ICMP message. It never knows that it has to reduce the MTU value. The packet capture below shows where the DF bit is set.

Capture7

I had a look through the RFC2246 for TLS1.0 and it doesn’t specify that the DF bit should be set. It’s most likely a vendor or O/S specific setting, so your observed results may differ from vendor to vendor.

RH

This blog will cover setting up 2 Cisco ASA firewall’s Active / Standby, so if one of the firewalls has a power issue or hardware failure, the standby firewall will wait a set amount of time before taking over from the failed device and resuming the traffic as if nothing happened. You can use stateful failover to resume IPsec sessions, NAT xlate, TCP, UDP, ARP etc. From version 8.4 routing information is also replicated.

The first thing you should know is you only need to set up 1 individual firewall, so my recommendation would be to get a firewall fully functional with all the configuration necessary, then the secondary firewall only needs a few commands before taking an entire copy of the primary firewall’s config, and becoming a standby firewall.

Before we begin lets take a look at the diagram, I used GNS3 but if you require to do this in real-life you would use crossover cables between the failover interfaces.This diagram depicts a single point of failure as the LAN switch but this is not the point of the blog, You would have more than one switch within your core design.

Active-Standby

We will just use the basic config of the firewall, in my setup i have an IPsec tunnel between my ASA and R3, I used this to confirm that IPsec tunnel failover was working as expected.
You will need to decide beforehand which address space you will use for your failover interfaces, and your standby IP’s must be another useable IP within your primary IP subnet.

On ASA 1 use a console or SSH session and input the following commands. Be sure to use “no shutdown” on any interfaces you decide to use.

interface GigabitEthernet1
 nameif inside
 security-level 100
 ip address 192.168.1.1 255.255.255.0 standby 192.168.1.2
!
interface GigabitEthernet0
 nameif outside
 security-level 0
 ip address 172.16.1.1 255.255.255.0 standby 172.16.1.2
!
failover lan interface lan_fail1 gig3
failover interface ip lan_fail1 10.2.2.1 255.255.255.252 standby 10.2.2.2
!
failover link stateful_fail2 gig2
failover interface ip stateful_fail2 10.1.1.1 255.255.255.252 standby 10.1.1.2
!
failover key cisco123
!
failover lan unit primary
!
failover replication http
!
prompt hostname priority state
!
failover

Once that is complete, you can now configure the secondary firewall, you can use the command “clear config all” on the secondary firewall to clear all config before you input the commands below. This is the absolute minimum you require to get the secondary firewall up and running, again be sure to use “no shutdown” on any interfaces you need.

interface gig3
no shutdown
!
failover lan interface lan_fail1 gig3
failover interface ip lan_fail1 10.2.2.1 255.255.255.252 standby 10.2.2.2
!
failover key cisco123
!
failover lan unit secondary
!
failover

After you enter the commands you will see the message “detected and active mate” this will confirm the 2 firewalls can see each other. The firewalls will begin to synchronise their configurations, the primary ASA will send over the configuration to the secondary. You will notice the prompt changes to ASA1/sec/stby# we use the command “prompt hostname priority state” for this purpose, so we can be sure which firewall is which.
The primary firewall prompt will be ASA1/pri/act#.

After this point you only need to make changes to the primary unit and any changes to the configuration and “write memory” you do on the primary will also be replicated to the standby unit.

The command “show failover” will provide you with all the necessary information you require should you need to troubleshoot or confirm your configuration is working as expected.

To manually failover the devices you can use the command “no failover active” on the active firewall or from the standby you can use ” failover active” note that when the primary unit recovers from a failure it does not automatically assume the active role.

You can have the ASA alert you when a failover has taken place by setting up email alerts.

logging mail critical
logging from-address asa1@yourdomain.com
logging recipient-address rh@mydomain.com
smtp-server x.x.x.x

This was one of my favourite blogs to write because the with the Cisco ASA everything just “works”

RH.

A common scenario you might come across is having a device on your network that needs to be restricted. So for example a third party PC that doesn’t meet your security requirements but needs to access a production server.

If your network can support it then you would segment this part of your network with a VLAN then route between them on a layer 3 switch or at the router. So in my example its a layer 3 switch, write the access list then apply it at the VLAN interface either inbound or outbound.

Below is the basic config for VLAN 10, anything in this VLAN will only be able to access the production server at 10.50.5.10 which happens to be on VLAN 20, this would apply to devices or whole subnets on a different switch or even across the WAN. I’ll make this interesting and allow the clients only access on TCP port 80, and also allow the clients to lookup DNS on the same server this is UDP port 53.

ACL_pic

Create your ACL
————————————-
ip access-list extended 100
permit ip any 192.168.1.0 0.0.0.255
permit tcp any host 10.50.5.10 eq www
permit udp any host 10.50.5.10 eq domain
deny ip any any log 

Apply your ACL to the VLAN interface
————————————-
interface vlan 10
ip access-group 100 in

That’s all you have to do to restrict the traffic, its very simple the inbound and outbound commands are hard to understand at first but just imagine yourself standing on top of the switch and think of traffic direction, so in my case its inbound to the switch.

RH

When you receive your new Cisco switch, its inherently insecure out of the box so here’s a few things you might want to think about before you go ahead and use in a production environment.

1. Set an enable password “(config)#enable secret 0 cisco12345” using the secret will encrypt your password with a strong MD5 hash.

2. Encrypt any plain text passwords with “(config)#service password-encryption”

3. Use a range command on your interfaces, enable spanning-tree portfast and switchport mode access in one go with this handy command “(config-if)#switchport host” this also disables etherchannel capabilities.

4. Still inside the range command you could use “(config-if)#spanning-tree bpduguard enable” this will shut down the port should another bogus switch be plugged into any of the ports and start sending BPDU’s, you can also enable this globally with the command “(config)#spanning-tree bpduguard default”

5. A simple yet effective practice is to never use VLAN1 for anything unless you really need to!

6. Use a blackhole VLAN like some obscure unused VLAN 999, put all unused ports in there so should an attacker plug into an unused port they just aren’t going to get access to anything, or simply shut the unused port down until needed.

7. Use port-security to restrict the amount of MAC address’s you expect to see on a switch so for example on a normal data / voice port you’d expect to see 1 PC and 1 VoIP device. “(config)#switchport port-security maximum 2” next enable it on the port with “(config)#switchport port-security” So any more than 2 MAC address’s seen on the port and it will disable. There are other more granular options with this command so investigate them and use what suits you.

8. Create a local user with this command “(config)#username admin_cisco privilege 15 secret 0 cisco54321”

9. Secure your console and VTY lines with this command use under (line console 0 and vty 0 15) “(config)#login authentication local”

These steps are very simple and easy, and hopefully you will think about using them to secure your device. I will say that this list is not exhaustive so do some research.

RH