Здравствуйте коллеги.
Столкнулся с проблемой настройке IPSec между FreeBSD и Linux.
Использую ESP в транспортном режиме для шифрования GRE-туннеля.
Без ESP всё работает без проблем.
После запуска IPSEC пробую ssh по туннелю. `racoon -F -d` пишет всё established? error-нет
Секунд через 30 соединение на ssh рвётся "Write error: operation not permitted". Ftp через тоже время тоже умирает.
Файрволами разрешил весь трафик между хостами. ipsec-tools собирал только с опцией enable-frag.
debug.log пишет
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: call pfkey_send_dump
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: pk_recv: retry[0] recv()
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: compute IV for phase2
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: phase1 last IV:
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG:  0834b0a8 b552f32a b17cd0d8
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: hash(sha1)
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: encryption(3des)
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: phase2 IV computed:
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG:  3b0c7a39 b7b0513e
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: HASH with:
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG:  b17cd0d8 0000001c 00000001 01100001 02855fd0 83160c2e 9ac4c168 1d9d7a32
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: hmac(hmac_sha1)
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: HASH computed:
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG:  6fa8d1ff ab404892 95177e54 4a333c64 95f67b9f
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: begin encryption.
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: encryption(3des)
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: pad length = 4
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG:  0c000018 6fa8d1ff ab404892 95177e54 4a333c64 95f67b9f 0000001c 00000001 01100001 02855fd0 83160c2e 9ac4c168 1d9d7a32 d9b7cf03
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: encryption(3des)
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: with key:
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG:  63017ae7 12718bcc 85e8969e cbffc9b6 9af56627 af20c383
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: encrypted payload by IV:
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG:  3b0c7a39 b7b0513e
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: save IV for next:
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG:  21d41bcf 58ef1df7
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: encrypted.
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: 84 bytes from 172.27.173.249[500] to 172.27.173.250[500]
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: sockname 172.27.173.249[500]
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: send packet from 172.27.173.249[500]
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: send packet to 172.27.173.250[500]
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: 1 times of 84 bytes message will be sent to 172.27.173.250[500]
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG:  02855fd0 83160c2e 9ac4c168 1d9d7a32 08100501 b17cd0d8 00000054 f24ce1e8 a3a2a968 cc0cb1b3 bdaa55b9 be0eeb1e d237ac85 ea1487ca c6d717b2 e89fe7fb 03de9cae 1e312693 594fd173 21d41bcf 58ef1df7
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: sendto Information delete.
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: IV freed
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: an undead schedule has been deleted.
Jun  2 08:34:31 vcshluz racoon: 2009-06-02 08:34:31: DEBUG: IV freed

Что ещё смущает это отсутствие (точнее отсутсвие после инициализации соединения) isakmp пакетов. tcpdump показывает что по сети бегают только ESP пакеты

FreeBSD - side

rc.conf

ipsec_enable="YES"
ipsec_file="/etc/ipsec.conf"
raccon="YES"

ipsec.conf

spdadd A B gre -P out ipsec
    esp/transport//require;
spdadd B A gre -P in  ipsec
    esp/transport//require;

raccon.conf

path pre_shared_key "/usr/local/etc/racoon/psk.txt";
path certificate "/usr/local/etc/racoon/cert";
listen
{
        isakmp A [500];
}
remote B
{
        exchange_mode main;
        proposal {
                encryption_algorithm 3des;
                hash_algorithm sha1;
                authentication_method pre_shared_key;
                dh_group modp1024;
                lifetime time 1 hour;
        }
}
sainfo address A 47 address B 47
{
        pfs_group modp1024;
        encryption_algorithm 3des;
        authentication_algorithm hmac_sha1;
        compression_algorithm deflate;
        lifetime time 1 hour;
}

pf.conf

pass out on $ext_if from A to B keep state
pass in on $ext_if from B to A keep state

Linux - side

rc.local

/./././setkey.conf
/./././racoon

setkey.conf

#!/usr/local/sbin/setkey -f
flush;
spdflush;
spdadd B A 47 -P out ipsec
    esp/transport//require;
spdadd A B 47 -P in  ipsec
    esp/transport//require;

raccon.conf

path pre_shared_key "/usr/local/etc/psk.txt";
path certificate "/usr/local/etc/cert";
listen
{
        isakmp  B [500];
}

remote A
{
        exchange_mode main;

        proposal {
                encryption_algorithm 3des;
                hash_algorithm sha1;
                authentication_method pre_shared_key;
                dh_group modp1024;
                lifetime time 1 hour;
        }
}

sainfo address B 47 address A 47
{
        pfs_group modp1024;
        encryption_algorithm 3des;
        authentication_algorithm hmac_sha1;
        compression_algorithm deflate;
        lifetime time 1 hour;
}

rc.firewall
iptables -A INPUT -i $ext_if -s B -d A -j ACCEPT
iptables -A OUTPUT -j $ext_if -s A -d B -j ACCEPT