[Openswan Users] no suitable connection for peer

Mon Oct 26 15:56:38 EDT 2009

Hello,
since some days iam trying to bring up a tunnel with certificates between a linux vpn gateway and a windows client. yet only psk works fine for me. i created an own ca with openssl and two certificates, signed by the ca. i exported one of them to windows format (PKCS12) and succesfully imported it to windows.

but the linux maschine still reject the certificate with following log:

Oct 26 15:27:53 swan ipsec__plutorun: Starting Pluto subsystem...
Oct 26 15:27:53 swan pluto[12731]: Starting Pluto (Openswan Version 2.6.14; Vendor ID OEoSJUweaqAX) pid:12731
Oct 26 15:27:53 swan pluto[12731]: Setting NAT-Traversal port-4500 floating to off
Oct 26 15:27:53 swan pluto[12731]: port floating activation criteria nat_t=0/port_float=1
Oct 26 15:27:53 swan pluto[12731]: including NAT-Traversal patch (Version 0.6c) [disabled]
Oct 26 15:27:53 swan pluto[12731]: using /dev/urandom as source of random entropy
Oct 26 15:27:53 swan pluto[12731]: ike_alg_register_enc(): Activating OAKLEY_TWOFISH_CBC_SSH: Ok (ret=0)
Oct 26 15:27:53 swan pluto[12731]: ike_alg_register_enc(): Activating OAKLEY_TWOFISH_CBC: Ok (ret=0)
Oct 26 15:27:53 swan pluto[12731]: ike_alg_register_enc(): Activating OAKLEY_SERPENT_CBC: Ok (ret=0)
Oct 26 15:27:53 swan pluto[12731]: ike_alg_register_enc(): Activating OAKLEY_AES_CBC: Ok (ret=0)
Oct 26 15:27:53 swan pluto[12731]: ike_alg_register_enc(): Activating OAKLEY_BLOWFISH_CBC: Ok (ret=0)
Oct 26 15:27:53 swan pluto[12731]: ike_alg_register_hash(): Activating OAKLEY_SHA2_512: Ok (ret=0)
Oct 26 15:27:53 swan pluto[12731]: ike_alg_register_hash(): Activating OAKLEY_SHA2_256: Ok (ret=0)
Oct 26 15:27:53 swan pluto[12731]: starting up 1 cryptographic helpers
Oct 26 15:27:53 swan pluto[12743]: using /dev/urandom as source of random entropy
Oct 26 15:27:53 swan pluto[12731]: started helper pid=12743 (fd:7)
Oct 26 15:27:54 swan pluto[12731]: Using Linux 2.6 IPsec interface code on 2.6.18-8.el5 (experimental code)
Oct 26 15:27:54 swan pluto[12731]: ike_alg_register_enc(): WARNING: enc alg=0 not found in constants.cakley_enc_names
Oct 26 15:27:54 swan pluto[12731]: ike_alg_register_enc(): Activating <NULL>: Ok (ret=0)
Oct 26 15:27:54 swan pluto[12731]: ike_alg_register_enc(): WARNING: enc alg=0 not found in constants.cakley_enc_names
Oct 26 15:27:54 swan pluto[12731]: ike_alg_add(): ERROR: Algorithm already exists
Oct 26 15:27:54 swan pluto[12731]: ike_alg_register_enc(): Activating <NULL>: FAILED (ret=-17)
Oct 26 15:27:54 swan pluto[12731]: ike_alg_register_enc(): WARNING: enc alg=0 not found in constants.cakley_enc_names
Oct 26 15:27:54 swan pluto[12731]: ike_alg_add(): ERROR: Algorithm already exists
Oct 26 15:27:54 swan pluto[12731]: ike_alg_register_enc(): Activating <NULL>: FAILED (ret=-17)
Oct 26 15:27:54 swan pluto[12731]: ike_alg_register_enc(): WARNING: enc alg=0 not found in constants.cakley_enc_names
Oct 26 15:27:54 swan pluto[12731]: ike_alg_add(): ERROR: Algorithm already exists
Oct 26 15:27:54 swan pluto[12731]: ike_alg_register_enc(): Activating <NULL>: FAILED (ret=-17)
Oct 26 15:27:55 swan pluto[12731]: ike_alg_register_enc(): WARNING: enc alg=0 not found in constants.cakley_enc_names
Oct 26 15:27:55 swan pluto[12731]: ike_alg_add(): ERROR: Algorithm already exists
Oct 26 15:27:55 swan pluto[12731]: ike_alg_register_enc(): Activating <NULL>: FAILED (ret=-17)
Oct 26 15:27:55 swan pluto[12731]: ike_alg_register_enc(): WARNING: enc alg=0 not found in constants.cakley_enc_names
Oct 26 15:27:55 swan pluto[12731]: ike_alg_add(): ERROR: Algorithm already exists
Oct 26 15:27:55 swan pluto[12731]: ike_alg_register_enc(): Activating <NULL>: FAILED (ret=-17)
Oct 26 15:27:55 swan pluto[12731]: Changed path to directory '/etc/ipsec.d/cacerts'
Oct 26 15:27:55 swan pluto[12731]: loaded CA cert file 'ca1Key.pem' (963 bytes)
Oct 26 15:27:55 swan pluto[12731]: no passphrase available
Oct 26 15:27:55 swan pluto[12731]: loaded CA cert file 'ca1Cert.pem' (1269 bytes)
Oct 26 15:27:55 swan pluto[12731]: Changed path to directory '/etc/ipsec.d/aacerts'
Oct 26 15:27:55 swan pluto[12731]: Could not change to directory '/etc/ipsec.d/ocspcerts': /
Oct 26 15:27:55 swan pluto[12731]: Changing to directory '/etc/ipsec.d/crls'
Oct 26 15:27:55 swan pluto[12731]: Warning: empty directory
Oct 26 15:27:55 swan pluto[12731]: Changing back to directory '/' failed - (2 No such file or directory)
Oct 26 15:27:55 swan pluto[12731]: Changing back to directory '/' failed - (2 No such file or directory)
Oct 26 15:27:56 swan pluto[12731]: loading certificate from /etc/ipsec.d/certs/linuxSign.pem
Oct 26 15:27:56 swan pluto[12731]: loaded host cert file '/etc/ipsec.d/certs/linuxSign.pem' (1318 bytes)
Oct 26 15:27:56 swan pluto[12731]: added connection description "l2tp-x509"
Oct 26 15:27:56 swan pluto[12731]: listening for IKE messages
Oct 26 15:27:56 swan pluto[12731]: adding interface eth0/eth0 10.245.23.198:500
Oct 26 15:27:56 swan pluto[12731]: adding interface eth1/eth1 172.17.240.1:500
Oct 26 15:27:56 swan pluto[12731]: adding interface lo/lo 127.0.0.1:500
Oct 26 15:27:56 swan pluto[12731]: adding interface lo/lo ::1:500
Oct 26 15:27:56 swan pluto[12731]: loading secrets from "/etc/ipsec.secrets"
Oct 26 15:27:56 swan pluto[12731]: loading secrets from "/etc/ipsec.d/ipsec.secrets"

Oct 26 15:28:10 swan pluto[12731]: packet from 172.18.240.34:500: ignoring Vendor ID payload [MS NT5 ISAKMPOAKLEY 00000003]
Oct 26 15:28:10 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: responding to Main Mode from unknown peer 172.18.240.34
Oct 26 15:28:10 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: transition from state STATE_MAIN_R0 to state STATE_MAIN_R1
Oct 26 15:28:10 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: STATE_MAIN_R1: sent MR1, expecting MI2
Oct 26 15:28:10 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: transition from state STATE_MAIN_R1 to state STATE_MAIN_R2
Oct 26 15:28:10 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: STATE_MAIN_R2: sent MR2, expecting MI3
Oct 26 15:28:10 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: Main mode peer ID is ID_DER_ASN1_DN: 'C=DE, ST=LandA, O=FirmaA, L=StadtA, OU=AbteilungA, CN=musterfrau, E=test at test.de'
Oct 26 15:28:10 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: no crl from issuer "C=DE, ST=LandA, L=StadtA, O=FirmaA, OU=AbteilungA, CN=mustermann, E=test at test.de" found (strict=no)
Oct 26 15:28:10 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: no suitable connection for peer 'C=DE, ST=LandA, O=FirmaA, L=StadtA, OU=AbteilungA, CN=musterfrau, E=test at test.de'
Oct 26 15:28:10 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: sending encrypted notification INVALID_ID_INFORMATION to 172.18.240.34:500
Oct 26 15:28:11 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: Main mode peer ID is ID_DER_ASN1_DN: 'C=DE, ST=LandA, O=FirmaA, L=StadtA, OU=AbteilungA, CN=musterfrau, E=test at test.de'
Oct 26 15:28:11 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: no crl from issuer "C=DE, ST=LandA, L=StadtA, O=FirmaA, OU=AbteilungA, CN=mustermann, E=test at test.de" found (strict=no)
Oct 26 15:28:11 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: no suitable connection for peer 'C=DE, ST=LandA, O=FirmaA, L=StadtA, OU=AbteilungA, CN=musterfrau, E=test at test.de'
Oct 26 15:28:11 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: sending encrypted notification INVALID_ID_INFORMATION to 172.18.240.34:500
Oct 26 15:28:13 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: Main mode peer ID is ID_DER_ASN1_DN: 'C=DE, ST=LandA, O=FirmaA, L=StadtA, OU=AbteilungA, CN=musterfrau, E=test at test.de'
Oct 26 15:28:13 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: no crl from issuer "C=DE, ST=LandA, L=StadtA, O=FirmaA, OU=AbteilungA, CN=mustermann, E=test at test.de" found (strict=no)
Oct 26 15:28:13 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: no suitable connection for peer 'C=DE, ST=LandA, O=FirmaA, L=StadtA, OU=AbteilungA, CN=musterfrau, E=test at test.de'
Oct 26 15:28:13 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: sending encrypted notification INVALID_ID_INFORMATION to 172.18.240.34:500
Oct 26 15:28:14 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: next payload type of ISAKMP Hash Payload has an unknown value: 167
Oct 26 15:28:14 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: malformed payload in packet
Oct 26 15:28:14 swan pluto[12731]: | payload malformed after IV
Oct 26 15:28:14 swan pluto[12731]: | 69 71 b7 38 42 1b 24 ce 9c 1b 9e 70 ad 29 4f 80
Oct 26 15:28:14 swan pluto[12731]: | c1 62 c4 b1
Oct 26 15:28:14 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: sending notification PAYLOAD_MALFORMED to 172.18.240.34:500
Oct 26 15:28:20 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: ignoring informational payload, type INVALID_COOKIE msgid=00000000
Oct 26 15:28:20 swan pluto[12731]: "l2tp-x509"[1] 172.18.240.34 #1: received and ignored informational message

my ipsec.conf:

config setup
#interfaces=%defaultroute
#protostack=netkey
#nat_traversal=yes
#virtual_private=%v4:192.168.2.0/24,%v4:0/24

include /etc/ipsec.d/no_oe.conf

conn l2tp-x509
left=10.245.23.198
leftprotoport=17/%any
leftcert=/etc/ipsec.d/certs/linuxSign.pem
rightprotoport=17/%any
right=%any
rightrsasigkey=%cert
auto=add
rightid="C=DE, ST=LandA, L=StadtA, O=FirmaA, OU=AbteilungA, CN=mustermann, E=test at test.de"
pfs=no

i played a lot with the settings, but nothing worked for me, always this error. i also tried things like rightid="C=DE, ST=*, L=*, O=*, OU=*, CN=*, E=*", with no success. 
maybe someone could help me, in the meantime i had to work with psk but my employer is not happy about that.

regards
_______________________________________________________________
Neu: WEB.DE DSL bis 50.000 kBit/s und 200,- Euro Startguthaben!
http://produkte.web.de/go/02/