ssh IPv6 connections die with "packet_read: Setup timeout expired, giving up" after sending publickey packet
Shadow Hawkins on Thursday, 19 June 2008 16:20:45
I am fighting an interesting issue which prevents my internal hosts from
talking SSH with external IPv6 hosts.
My setup is:
piper - an internal host (2001:41e0:ff12:0:211:2fff:fe6b:c869), connected via IPv6 to
wall - my gateway (2001:41e0:ff12::1), connected by sixxs heartbeat tunnel to a PoP (chzrh01)
pulse - my external host (2001:41e0:ff1a::1), connected by sixxs static tunnel to the same PoP
Both wall and pulse have a pMTU of 1500 to the PoP, so I set it all up to use
MTUs of 1480. To be exact:
- both tunnels have MTU==1480 on sixxs.net's webpage
- both machines have MTU==1500 on their external IPv4 ifaces
- both machines have tunnel ifaces with MTU==1480
I can (and you can) ping6 all machines. Furthermore, I can pull hundreds
of megabytes from pulse to piper (or wall) (using HTTP) and push them back
(using FTP).
There's a bit of a curious thing happening when I push: the FTP application
sends about 80k of data and then the transfer rate drops to 0 for 10 seconds
with an occasional ACK packet; after the 10 seconds, the transfer continues at
maximum rate through the end. tcpdump capture file follows:
http://scratch.madduck.net/.tmp__cdt.XoV25955__tcpdump.ftp
Possibly related is that I I cannot establish SSH IPv6 connections from piper
to pulse, while they work fine the other way, and also between wall to pulse
and wall and piper. However, IPv4 SSH connections between piper and pulse work
fine.
Here is what happens when I try to ssh -6 from piper to pulse:
piper:~|master|% ssh -6vSnone pulse
OpenSSH_4.7p1 Debian-12, OpenSSL 0.9.8g 19 Oct 2007
debug1: Reading configuration data /home/madduck/.ssh/config
debug1: Applying options for pulse
debug1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to pulse.madduck.net [2001:41e0:ff1a::1] port 22.
debug1: fd 3 clearing O_NONBLOCK
debug1: Connection established.
debug1: identity file /home/madduck/.ssh/id_rsa type 1
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.3p2 Debian-9etch2
debug1: match: OpenSSH_4.3p2 Debian-9etch2 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_4.7p1 Debian-12
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-cbc hmac-sha1 none
debug1: kex: client->server aes128-cbc hmac-sha1 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<2048<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host 'pulse.madduck.net' is known and matches the RSA host key.
debug1: Found key in /home/madduck/.ssh/known_hosts:73
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/madduck/.ssh/id_rsa
packet_read: Setup timeout expired, giving up
The tcpdump captures on both sides are identical:
http://scratch.madduck.net/.tmp__cdt.XoV25955__tcpdump.ssh.piper
I have tried lowering the MTUs to 1280, and I have also verified with
tracepath6 that connectivity is there. I have no idea what's causing this
problem, which was not present a few weeks ago. Until a few days ago it was
only one of the machines on my network that couldn't connect to pulse; now
none of them can.
Any help/tips appreciated!
ssh IPv6 connections die with "packet_read: Setup timeout expired, giving up" after sending publickey packet
Shadow Hawkins on Friday, 20 June 2008 15:05:39
Of course, today it all works. I guess the Internet is dynamic. :)
Not sure how to move on with this...
ssh IPv6 connections die with "packet_read: Setup timeout expired, giving up" after sending publickey packet
Shadow Hawkins on Friday, 27 June 2008 03:13:49
Hi Martin,
what about writing a regression test, and see if it happens again. If so, change
the source, e.g. put higher values into "Setup timeout". Also it would be interessting to know the underlying protocols, such as if you used IEEE 802.11 or
TCP over something obscure :o
It would also be good if you can set all starting conditions so, as they were
when the error occured.
remember the internet is dynamic :)
[QUOTE]
SSH-2.0-OpenSSH_4.7p1 Debian-12
[/QUOTE]
maybe also check Debian bugs
regards
Stefan
ssh IPv6 connections die with "packet_read: Setup timeout expired, giving up" after sending publickey packet
Shadow Hawkins on Friday, 27 June 2008 04:44:14
hmm, re-reading your post:
in the ftp session: one tcp window-full message (#114) and after that lots of duplicated acks and later TCP retransmissions..
i'd add for now:
hihi, I grabbed the banner from one of the packets, but it was already in the post :o
did you changed any firewall configuration and/or traffic shaping?
regards
Stefan
ssh IPv6 connections die with "packet_read: Setup timeout expired, giving up" after sending publickey packet
Shadow Hawkins on Friday, 27 June 2008 09:33:46
How would you suggest I write such a regression test? I suppose I don't understand what you are trying to do.
I can certainly do the SetupTimeout increase next time I see it.
The machine is connected by 100Mbps LAN to a Linux router where it arrives on a bridged interface and leaves by way of
52: wan: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:14:a4:eb:d3:e5 brd ff:ff:ff:ff:ff:ff
inet 84.75.148.163/22 brd 255.255.255.255 scope global wan
inet6 fe80::214:a4ff:feeb:d3e5/64 scope link
valid_lft forever preferred_lft forever
to a cable modem. I don't know what happens then, but I've never had problems, and I only have these problems with IPv6.
I can't find any Debian bugs on this.
Yes, the Internet is dynamic. I did not change any firewall rules or shaping policies, so the reason for the intermittent failures must be one of the possible routes the packets take, likely on the IPv4 path between the above IP and my PoP in Zurich.
ssh IPv6 connections die with "packet_read: Setup timeout expired, giving up" after sending publickey packet
Jeroen Massar on Wednesday, 09 July 2008 17:49:15
Did you check the MTU values with 'ip -6 ro sho cache'?
ssh IPv6 connections die with "packet_read: Setup timeout expired, giving up" after sending publickey packet
Shadow Hawkins on Thursday, 10 July 2008 09:06:04
Yes, everything seemed in order. I also tracepath'd the routes and always got the same MTU from both sides.
ssh IPv6 connections die with "packet_read: Setup timeout expired, giving up" after sending publickey packet
Jeroen Massar on Thursday, 10 July 2008 09:31:57
meh, if I had spare time then I would look into it. I don't have any issues though with chzrh01 and am probably one of the heavier users of that PoP ;)
But I am using AYIYA which changes the scheme a bit.
Posting is only allowed when you are logged in. |