Cisco PIX 7.0 VPN Troubleshooting
Quick overview of IPSEC
It is important to understand how IPSEC works in order to understand how to troubleshoot a VPN connection. This is a quick overview of IPSEC and is by no means a complete detailed guide.
IPSEC is a suite of protocols, defined in RFC 2401, that is used to protect information as it travels from one private network to another private network over a public network.
IPSEC consists of Security Protocols (AH and ESP), Key Management (ISAKMP, IKE, and SKEME), and Algorithms (3DES, AES256, etc).
ISAKMP defines the procedures and packet formats used to establish, negotiate, and modify Security Associations. ISAKMP communicates over UDP 500.
Security Protocols consist of AH (Authentication Header) and ESP (Encapsulating Security Payload). AH communicates over IP 51 and provides data authentication, integrity, and replay protection (for man in the middle attacks), but does not provide confidentiality. It is important to understand that AH encapsulates the IP packet but does not encrypt it.
ESP communicates over IP 50 and provides the same service as AH in addition to providing data confidentiality by encrypting the original payload and encapsulating the packet.
SA’s (Security Associations):
In order to have an IPSEC conversation, you first need a security association. Each device must agree on the policies or rules of the conversation by negotiating these policies with their potential peers. The SA represents a unidirectional instance of a security policy for a given connection.
Main mode IPSEC packet exchange:
--Initiator--- ---Responder---
----------pk#1—Policy Proposal------>
<-------pk#2---Policy Accept/Reject-- ----------pk#3---DH Exchange-------->
<-------pk#4---DH Exchange---------- ----------pk#5---ID/Hash------------->
<------pk#6---ID/Hash--------------->
Packet handling order:
Step 1 Access lists applied to an interface and crypto map are used by Cisco IOS software to select interesting traffic to be encrypted.
Step 2 Cisco IOS software checks to see if IPSec SAs have been established.
Step 3 If the SA has already been established by manual configuration using the crypto ipsec transform-set and crypto map commands or has been previously set up by IKE, the packet is encrypted based on the policy specified in the crypto map and is transmitted out of the interface.
Step 4 If the SA has not been established, Cisco IOS software checks to see if an IKE SA has been configured and set up.
Step 5 If the IKE SA has been set up, the IKE SA governs negotiation of the IPSec SA as specified in the IKE policy configured by the crypto isakmp policy command, the packet is encrypted by IPSec, and it is transmitted.
Step 6 If the IKE SA has not been set up, Cisco IOS software checks to see if certification authority (CA) has been configured to establish an IKE policy.
Step 7 If CA authentication is configured with the various crypto ca commands, the router uses public and private keys previously configured, obtains the CA's public certificate, gets a certificate for its own public key, and then uses the key to negotiate an IKE SA, which in turn is used to establish an IPSec SA to encrypt and transmit the packet.
Configuring Phase 1:
The first 2 octets of IPs have been replaced with "y.y."
Phase I is not configured on a per connection basis. When a Phase I connection is being established, configured ISAKMP policies will be tried one at a time until a match is found.
Example of an ISAKMP policy:
#isakmp policy 20 authentication pre-share
#isakmp policy 20 encryption 3des
#isakmp policy 20 hash md5
#isakmp policy 20 group 2
#isakmp policy 20 lifetime 43200
Troubleshooting Phase I:
Check the syslogs
Show run isakmp
This will show the isakmp policies for all VPN connections. To view a specific ISAKMP policy type show run isakmp | grep
Show crypto isakmp sa detail – This command will display the state of Phase I of the IPSEC tunnel. A state of MM_Active indicates that Phase I was successfully completed. If Phase I does not complete, refer to the table below to find out exactly what state the Phase I connection is currently in. This will give you an indication of where the problem has occurred. More specific information can be found by running a debug(discussed later).
State Description
OAK_MM_No_STATE This is the initial state of Phase I. If you see Phase I
In this state for longer than a few seconds, this is an
indication that a failure of tunnel establishment for
Phase I has occurred.
OAK_MM_SA_SETUP The peers have agreed on parameters for the ISAKMP
SA. Phase I will be in this state after packet 1 and packet 2 exchange of the Main Mode negotiation (see above).
MM_WAIT_MSG The firewall is waiting on the remote end device to respond with DH and public key.
OAK_MM_KEY_EXCH The peers have exchanged DH public keys and have generated a shared secret.
OAK_MM_KEY_AUTH The ISAKMP SA has been authenticated.
The debug crypto isakmp 5 command will display real time information on every step of the Phase I connection. Debug level 5 should be sufficient for most troubleshooting however level 7 provides more detailed information if necessary.
Please note that you cannot limit the debug output to a specific tunnel.
IKMP_NO_ERROR_NO_TRANS indicates a matching transform set was not found
No Proposal Chosen=isakmp policy mismatch
syslog sample of a completed connection:
The following shows the initiation of the first packet for an IPSEC tunnel.
58534 02/27/2004 07:42:38.600 SEV=4 IKE/41 RPT=8619 y.y.11.49
IKE Initiator: New Phase 1, Intf 2, IKE Peer
The following indicates that the IKE Phase I policy was accepted by the remote gateway.
58534 02/27/2004 07:42:38.600 IP = y.y.11.49, Oakley proposal is acceptable
This indicates Phase I has completed.
58534 02/27/2004 07:42:38.600 Group= y.y.11.49, IP=y.y.11.49, Oakley begin quick mode
The following indicates that the remote gateway has indicated that none of the policies are acceptable.
5|Oct 02 2006 09:41:41|713904: IP = y.y.138.12, Received an un-encrypted NO_PROPOSAL_CHOSEN notify message, dropping
To clear the Security Associations related to Phase 1, use the clear crypto isakmp command. This will clear ALL of the SA’s currently built on this firewall.
To confirm that the IPSEC packets are reaching the firewall, a capture can be created for all UDP 500 traffic.
First create an access-list for the traffic you would like to capture.
Access-list capture1 permit udp any any eq 500
Next create a capture.
Capture cap1 access-list capture1 interface outside
Next display the results of the capture.
Show capture cap1 detail
ciscoasa#show capture cap1 detail
1: 13:04:06.284897 192.168.1.50 > 192.168.1.1: UDP:500
View capture on web
https://
View pre-shared keys:
more system:running-config
Configuring Phase 2:
A transform set combines encryption method and authentication method. During the IPSec security association negotiation with ISAKMP, the peers agree to use a particular transform set to protect a particular data flow. The transform set must be the same for both peers.
You can create multiple transform sets, and then specify one or more of these transform sets in a crypto map entry.
You can view previously created transform sets by typing the show crypto ipsec transform-set command. If the desired transform set has not been previously defined, the crypto ipsec transform-set command is used to create it.
Example:
#(config)crypto ipsec transform-set 3desmd5 esp-3des esp-md5-hmac
An access-list is used to define the “interesting traffic” or the traffic that should be encrypted and allowed through the VPN Tunnel. The access-list should always be defined from local to remote. The subnet sizes need to match on the remote gateway.
Example:
#(config) access-list tunnel1 extended permit ip y.y..191.0 255.255.255.0 host y.y..155.12
If port filtering is being used, and traffic is being initiated from the remote side, the destination port of the remote host must be the source port of the local matching acl.
A tunnel group is used to identify specific connection parameters and the definition of a group policy. The default tunnel groups are DefaultRAGroup (used for Remote Access tunnels) and DefaultL2Lgroup(used for IPSEc Lan-to-Lan tunnels).
Example:
#(config)tunnel-group y.y.155.1 type IPsec_l2l
#(config)tunnel-group y.y.155.1 ipsec-attributes
#(config-attributes) pre-shared-key abc123
The crypto map ties together several components that define the VPN tunnel. This is where the peer defined in the tunnel-group command is tied to the access-list and transform-set. The crypto map must be assigned a unique map id #. To view the previously used crypto map id numbers run the show ru crypto command.
Example:
#(config)crypto map mymap 10 match address tunnel1
#(config)crypto map mymap 10 set peer y.y,155.1
#(config)crypto map mymap 10 set transform-set 3desmd5
Nat considerations:
If a local address is going to be natted outbound, the crypto acl should use the outside ip address.
Troubleshooting Phase II:
Check syslogs
Show crypto ipsec sa- This command shows the output of the IPSEC SA’s. The SA will include the ip address of the local and remote endpoints, encryption domains (interesting traffic), transform set (what encryption and hash is being used), key lifetime, and # of packet encrypt/decrypts.
This shows the crypto map used for this connection.
Crypto map tag: vpn_map, seq num: 130, local addr: x.x.160.45
The following line shows the crypto acl that includes the traffic to be protected.
access-list VPN-CIDS704976 permit ip x.x.190.0 255.255.254.0 host 10.2 5.4.80
local ident (addr/mask/prot/port): (x.x.190.0/255.255.254.0/0/0)
remote ident (addr/mask/prot/port): (10.25.4.80/255.255.255.255/0/0)
current_peer: y.y.227.136
Encrypts indicate that this side is encrypting and sending traffic. Decrypts indicates that the other side is sending traffic.
#pkts encaps: 5, #pkts encrypt: 5, #pkts digest: 5
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 5, #pkts comp failed: 0, #pkts decomp failed: 0
#send errors: 0, #recv errors: 0
This lists the local and remote endpoints.
local crypto endpt.: x.x.160.45, remote crypto endpt.: y.y.227.136
path mtu 1500, ipsec overhead 74, media mtu 1500
current outbound spi: 2AFEA5C7
There is a separate sa for inbound and outbound.
inbound esp sas:
spi: 0x9D111D2A (2635144490)
transform: esp-aes-256 esp-sha-hmac none
in use settings ={L2L, Tunnel, PFS Group 5, }
slot: 0, conn_id: 317225, crypto-map: vpn_map
sa timing: remaining key lifetime (kB/sec): (4275000/28789)
IV size: 16 bytes
replay detection support: Y
outbound esp sas:
spi: 0x2AFEA5C7 (721331655)
transform: esp-aes-256 esp-sha-hmac none
in use settings ={L2L, Tunnel, PFS Group 5, }
slot: 0, conn_id: 317225, crypto-map: vpn_map
sa timing: remaining key lifetime (kB/sec): (4274999/28789)
IV size: 16 bytes
replay detection support: Y
Clear crypto ipsec sa peer
debug crypto ipsec—Displays the IPSec negotiations of phase 2.
No Valid SA/ Identity mismatch – Transform set or crypto acl
Sample Debug output:
The following shows that the tunnel group configuration was found.
Oct 26 15:42:43 [IKEv1]: IP =y.y.205.92, Connection landed on tunnel_group y.y,.205.92
Sample syslog errors:
This shows interesting traffic ACL getting exchanged.
1754 11/29/2001 16:20:18.500 SEV=7 IKEDBG/0 RPT=546 y.y.205.92
Transmitting Proxy Id:
Remote host: 192.168.1.1 Protocol 0 Port 0
Local host: 10.64.10.9 Protocol 0 Port 0
Completion of Phase II.
1949 11/29/2001 16:20:18.540 SEV=4 IKE/49 RPT=3 y.y.205.92
Security negotiation complete
Responder, Inbound SPI = 0x11a56495, Outbound SPI = 0xb17718a5
Mar 10 2008 18:47:05: %PIX-5-713120: Group = y.y.41.250, IP = y.y.41.250, PHASE 2 COMPLETED (msgid=0f78e513)
1754 11/29/2001 16:20:18.500 Group = 172.16.172.63, IP = 172.16.172.63, Received an un-encrypted PAYLOAD_MALFORMED notify message, dropping.
Pre-shared key mismatch reported by the report peer(receiving peer):
1754 11/29/2001 16:20:18.500 Group = 172.16.172.63, IP = 172.16.172.63, Received non-routine Notify message: No Proposal Chosen
Transform-set mismatch on remote peer(receiving peer):
713904” IP = 10.51.16.1, Received encrypted packet with no matching SA, dropping
713048: IP = 10.51.16.1 Error processing payload. Payload ID 1
The following indicates that the remote gateway is not finding matching interesting traffic.
1754 11/29/2001 16:20:18.500 Group = y.y.172.63, IP = y.y.172.63, Received non-routing Notify message: Invalid ID info (18)
The following indicates that the local gateway is not finding matching interesting traffic.
1754 11/29/2001 16:20:18.500 Group =y.y.172.63, IP = y.y.172.63, Static Crypto Map check, map = mymap, seq = 10, ACL does not match proxy IDs src:192.168.1.0 dst:192.168.2.0
PFS mismatch:
This indicated that the remote peer is natting:
%ASA-4-402116: IPSEC: Received an ESP packet (SPI= 0x72DEC2AA, sequence number= 0x41) from y.y.28.178 (user= y.y.28.178) to y.y.83.194. The decapsulated inner packet doesn't match the negotiated policy in the SA. The packet specifies its destination as y.y.83.194, its source as y.y.28.178, and its protocol as 1. The SA specifies its local proxy as y.y.10.16/255.255.255.240/0/0 and its remote_proxy as y.y.63.0/255.255.255.0/0/0.
When reverse route is turned on:
GIVE PROPS FROM WHERE U STOLE THIS SHIT FROM YOU DUMBFUCK!
Thanks for stealing my Homie's article YOU JACKASS!!!!
Please remove all posts, like this one, stolen from netleets.com. You are welcome to post hyperlinks to netleets.com but are not allowed to steal entire articles.
I think he will remove them within the next 10 years.