In this part I will be discussing the following problem scenarios----
- IKE SA not established
- IPSec SA’s not established
- MTU/Fragmentation Issues
Problem Scenario 1: No IKE SAs
If we are unable to establish IPSec tunnel from Branch location to Hub location
Check for Routing. Ping the Branch (using HUB’s IKE endpoint)
HUB# ping ip 40.10.1.1 source 30.3.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 40.10.1.1, timeout is 2 seconds:
Packet sent with a source address of 30.3.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 40.10.1.1, timeout is 2 seconds:
Packet sent with a source address of 30.3.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms
Check for IKE SA
HUB# sh crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst src state conn-id slot status
30.3.1.1 40.10.1.1 MM_NO_STATE 0 0 ACTIVE (deleted)
IPv6 Crypto ISAKMP SA
IPv4 Crypto ISAKMP SA
dst src state conn-id slot status
30.3.1.1 40.10.1.1 MM_NO_STATE 0 0 ACTIVE (deleted)
IPv6 Crypto ISAKMP SA
Use IKE Debugs to troubleshoot [ debug crypto isakmp ]
Problem Scenario 1a:
No IKE SAs
No IKE SAs
ISAKMP:(0):Checking ISAKMP transform 1 against priority 10 policy
ISAKMP: encryption 3DES-CBC
ISAKMP: hash SHA
ISAKMP: default group 2
ISAKMP: auth pre-share
ISAKMP: life type in seconds
ISAKMP: life duration (basic) of 7200
ISAKMP:(0):Encryption algorithm offered does not match policy!
ISAKMP:(0):atts are not acceptable. Next payload is 0
ISAKMP: encryption 3DES-CBC
ISAKMP: hash SHA
ISAKMP: default group 2
ISAKMP: auth pre-share
ISAKMP: life type in seconds
ISAKMP: life duration (basic) of 7200
ISAKMP:(0):Encryption algorithm offered does not match policy!
ISAKMP:(0):atts are not acceptable. Next payload is 0
ISAKMP:(0):Checking ISAKMP transform 1 against priority 65535 policy
ISAKMP: encryption 3DES-CBC
ISAKMP: hash SHA
ISAKMP: default group 2
ISAKMP: auth pre-share
ISAKMP: life type in seconds
ISAKMP: life duration (basic) of 7200
ISAKMP:(0):Encryption algorithm offered does not match policy!ISAKMP:(0):atts are not acceptable. Next payload is 0
ISAKMP: encryption 3DES-CBC
ISAKMP: hash SHA
ISAKMP: default group 2
ISAKMP: auth pre-share
ISAKMP: life type in seconds
ISAKMP: life duration (basic) of 7200
ISAKMP:(0):Encryption algorithm offered does not match policy!ISAKMP:(0):atts are not acceptable. Next payload is 0
ISAKMP:(0):no offers accepted!ISAKMP:(0): phase 1 SA policy not acceptable! (local 30.3.1.1 remote 40.10.1.1)
Check the IKE Policies
HUB# sh crypto isakmp policy
Global IKE policy
Protection suite of priority 10
encryption algorithm: AES - Advanced Encryption Standard (128 bit keys).
hash algorithm: Secure Hash Standard
authentication method: Pre-Shared Key
Diffie-Hellman group: #2 (1024 bit)
lifetime: 7200 seconds, no volume limit
Default protection suite
encryption algorithm: DES - Data Encryption Standard (56 bit keys).
Global IKE policy
Protection suite of priority 10
encryption algorithm: AES - Advanced Encryption Standard (128 bit keys).
hash algorithm: Secure Hash Standard
authentication method: Pre-Shared Key
Diffie-Hellman group: #2 (1024 bit)
lifetime: 7200 seconds, no volume limit
Default protection suite
encryption algorithm: DES - Data Encryption Standard (56 bit keys).
So once we change the encryption algorithm at spoke side to aes, phase 1 will come up.
Problem Scenario 1b:
No IKE SAs
ISAKMP:(1017): sending packet to 40.10.1.1 my_port 500 peer_port 500 (R) MM_KEY_EXCH
ISAKMP:(1017):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE
ISAKMP:(1017):Old State = IKE_R_MM3 New State = IKE_R_MM4
ISAKMP:(1017):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE
ISAKMP:(1017):Old State = IKE_R_MM3 New State = IKE_R_MM4
ISAKMP (0:1017): received packet from 40.10.1.1 dport 500 sport 500 Global (R) MM_KEY_EXCH
ISAKMP: reserved not zero on ID payload!
%CRYPTO-4-IKMP_BAD_MESSAGE: IKE message from 40.10.1.1 failed its sanity check or is malformed
ISAKMP: reserved not zero on ID payload!
%CRYPTO-4-IKMP_BAD_MESSAGE: IKE message from 40.10.1.1 failed its sanity check or is malformed
It means we have a mismatch in pre-shared key, on correcting it our IKE SA should come up.
HUB# sh cry isa sa
IPv4 Crypto ISAKMP SA
dst src state conn-id slot status
30.3.1.1 40.10.1.1 QM_IDLE 1019 0 ACTIVE
IPv4 Crypto ISAKMP SA
dst src state conn-id slot status
30.3.1.1 40.10.1.1 QM_IDLE 1019 0 ACTIVE
Problem Scenario 2: No IPSec SAs
If you notice that there is no traffic is being received through the IPSec tunnel
IKE SAs exist, but no IPSec SAs
Check for IPSEC SA (look for inbound and outbound SPI’s)
HUB# sh crypto ipsec sa peer 40.10.1.1
interface: GigabitEthernet0/1
Crypto map tag: CMAP, local addr 30.3.1.1
protected vrf: (none)
local ident (addr/mask/prot/port): (3.1.1.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (4.1.1.0/255.255.255.0/0/0)
current_peer 40.10.1.1 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0
local crypto endpt.: 30.3.1.1, remote crypto endpt.: 40.10.1.1
path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/1
current outbound spi: 0x0(0)
inbound esp sas:
inbound ah sas:
interface: GigabitEthernet0/1
Crypto map tag: CMAP, local addr 30.3.1.1
protected vrf: (none)
local ident (addr/mask/prot/port): (3.1.1.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (4.1.1.0/255.255.255.0/0/0)
current_peer 40.10.1.1 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0
local crypto endpt.: 30.3.1.1, remote crypto endpt.: 40.10.1.1
path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/1
current outbound spi: 0x0(0)
inbound esp sas:
inbound ah sas:
outbound esp sas:
outbound ah sas:
HUB#
Use IPSec Debugs to troubleshoot [ debug crypto ipsec ]
Problem Scenario 2a: No IPSec SAs
ISAKMP (0:1022): received packet from 40.10.1.1 dport 500 sport 500 Global (R) QM_IDLE
ISAKMP:(1022): processing SA payload. message ID = -549695704
ISAKMP:(1022):Checking IPSec proposal 1
ISAKMP: transform 1, ESP_3DES
ISAKMP: attributes in transform:
ISAKMP: encaps is 1 (Tunnel)
ISAKMP: SA life type in seconds
ISAKMP: SA life duration (basic) of 1800
ISAKMP: SA life type in kilobytes
ISAKMP: SA life duration (VPI) of 0x0 0x46 0x50 0x0
ISAKMP: authenticator is HMAC-SHA
ISAKMP:(1022):atts are acceptable.
ISAKMP:(1022):Checking IPSec proposal 1
ISAKMP: transform 1, ESP_3DES
ISAKMP: attributes in transform:
ISAKMP: encaps is 1 (Tunnel)
ISAKMP: SA life type in seconds
ISAKMP: SA life duration (basic) of 1800
ISAKMP: SA life type in kilobytes
ISAKMP: SA life duration (VPI) of 0x0 0x46 0x50 0x0
ISAKMP: authenticator is HMAC-SHA
ISAKMP:(1022):atts are acceptable.
IPSEC(validate_proposal_request): proposal part #1,
(key eng. msg.) INBOUND local= 30.3.1.1, remote= 40.10.1.1,
local_proxy= 3.1.1.0/255.255.255.0/0/0 (type=4),
remote_proxy= 4.1.1.0/255.255.255.0/0/0 (type=4),
protocol= ESP, transform= NONE (Tunnel),
lifedur= 0s and 0kb,
spi= 0x0(0), conn_id= 0, keysize= 0, flags= 0x0
Crypto mapdb : proxy_match
src addr : 3.1.1.0
dst addr : 4.1.1.0
protocol : 0
src port : 0
dst port : 0
IPSEC(ipsec_process_proposal): transform proposal not supported for identity:
{esp-3des esp-sha-hmac }
ISAKMP:(1022): IPSec policy invalidated proposal with error 256ISAKMP:(1022): phase 2 SA policy not acceptable! (local 30.3.1.1 remote 40.10.1.1)
(key eng. msg.) INBOUND local= 30.3.1.1, remote= 40.10.1.1,
local_proxy= 3.1.1.0/255.255.255.0/0/0 (type=4),
remote_proxy= 4.1.1.0/255.255.255.0/0/0 (type=4),
protocol= ESP, transform= NONE (Tunnel),
lifedur= 0s and 0kb,
spi= 0x0(0), conn_id= 0, keysize= 0, flags= 0x0
Crypto mapdb : proxy_match
src addr : 3.1.1.0
dst addr : 4.1.1.0
protocol : 0
src port : 0
dst port : 0
IPSEC(ipsec_process_proposal): transform proposal not supported for identity:
{esp-3des esp-sha-hmac }
ISAKMP:(1022): IPSec policy invalidated proposal with error 256ISAKMP:(1022): phase 2 SA policy not acceptable! (local 30.3.1.1 remote 40.10.1.1)
Check the IPSec Transform Sets
HUB# sh cry ips transform-set
Transform set TS: { esp-aes esp-sha-hmac }
will negotiate = { Tunnel, },
Transform set TS: { esp-aes esp-sha-hmac }
will negotiate = { Tunnel, },
On Correcting encryption algorithm in tranform-set , tunnel should come up.
Problem Scenario 2b: No IPSec SAs
Check the Crypto ACLs
HUB# sh access-list SPOKE-10-ACL
Extended IP access list SPOKE10-ACL
10 permit ip 3.1.1.0 0.0.0.255 5.1.1.0 0.0.0.255
HUB#
Extended IP access list SPOKE10-ACL
10 permit ip 3.1.1.0 0.0.0.255 5.1.1.0 0.0.0.255
HUB#
On Correcting crypto access-list , tunnel should come up.
Problem Scenario 3: Anti-Replay Issues
If you notice that some of the applications are losing intermittent traffic, or that Voice quality through tunnel is bad.
Check if the IPSec SA is showing anti-replay drops
HUB# sh cry ips sa peer 40.10.1.1 detail
interface: GigabitEthernet0/1
Crypto map tag: CMAP, local addr 30.3.1.1
protected vrf: (none)
local ident (addr/mask/prot/port): (3.1.1.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (4.1.1.0/255.255.255.0/0/0)
current_peer 40.10.1.1 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 2900, #pkts encrypt: 2900, #pkts digest: 2900
#pkts decaps: 1909, #pkts decrypt: 1909, #pkts verify: 1909
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#pkts no sa (send) 0, #pkts invalid sa (rcv) 0
#pkts encaps failed (send) 0, #pkts decaps failed (rcv) 0
#pkts invalid prot (recv) 0, #pkts verify failed: 0
#pkts invalid identity (recv) 0, #pkts invalid len (rcv) 0
#pkts replay rollover (send): 0, #pkts replay rollover (rcv) 0
##pkts replay failed (rcv): 1000 #pkts internal err (send): 0, #pkts internal err (recv) 0
local crypto endpt.: 30.3.1.1, remote crypto endpt.: 40.10.1.1
path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/1
current outbound spi: 0xC37422AA(3279168170)
inbound esp sas:
spi: 0x135E76B1(324957873)
transform: esp-3des esp-sha-hmac ,
in use settings ={Tunnel, }
conn id: 41, flow_id: SW:41, crypto map: CMAP
sa timing: remaining key lifetime (k/sec): (4419198/860)
IV size: 8 bytes
replay detection support: Y
Status: ACTIVE
interface: GigabitEthernet0/1
Crypto map tag: CMAP, local addr 30.3.1.1
protected vrf: (none)
local ident (addr/mask/prot/port): (3.1.1.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (4.1.1.0/255.255.255.0/0/0)
current_peer 40.10.1.1 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 2900, #pkts encrypt: 2900, #pkts digest: 2900
#pkts decaps: 1909, #pkts decrypt: 1909, #pkts verify: 1909
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#pkts no sa (send) 0, #pkts invalid sa (rcv) 0
#pkts encaps failed (send) 0, #pkts decaps failed (rcv) 0
#pkts invalid prot (recv) 0, #pkts verify failed: 0
#pkts invalid identity (recv) 0, #pkts invalid len (rcv) 0
#pkts replay rollover (send): 0, #pkts replay rollover (rcv) 0
##pkts replay failed (rcv): 1000 #pkts internal err (send): 0, #pkts internal err (recv) 0
local crypto endpt.: 30.3.1.1, remote crypto endpt.: 40.10.1.1
path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/1
current outbound spi: 0xC37422AA(3279168170)
inbound esp sas:
spi: 0x135E76B1(324957873)
transform: esp-3des esp-sha-hmac ,
in use settings ={Tunnel, }
conn id: 41, flow_id: SW:41, crypto map: CMAP
sa timing: remaining key lifetime (k/sec): (4419198/860)
IV size: 8 bytes
replay detection support: Y
Status: ACTIVE
Default IPSec Anti-Replay window is 64
Packets received outside the window are dropped
Re-ordering of packets could happen due to QoS on the encrypting router (Spoke) or in the Transit Network
In current Cisco IOS versions, the Anti-Replay window can be increased up to 1024, or diabled altogether
crypto ipsec security-association window-size
crypto ipsec security-association replay disable
Not recommended to disable anti-replay; first try to fix the QoS issue in the network or encrypting router; give better QoS to Voice traffic, or use crypto LLQ; then try to increase the anti-replay window size.