SixXS::Sunset 2017-06-06

Ticket ID: SIXXS #1149258
Ticket Status: User

PoP: noosl01 - Availo (Oslo)

noosl01 upstream routing problems
[no] Shadow Hawkins on Thursday, 23 July 2009 16:52:18
noosl01s upstream seems to be having trouble Ping from client to POP endpoint address: --- 2001:16D8:EE00:28::2 ping statistics --- 122 packets transmitted, 122 received, 0% packet loss, time 121057ms rtt min/avg/max/mdev = 0.583/0.774/4.874/0.568 ms Ping to noc.sixxs.net: --- sixxs.net ping statistics --- 282 packets transmitted, 7 received, +58 errors, 97% packet loss, time 382450ms rtt min/avg/max/mdev = 56.738/58.614/62.780/2.028 ms Further info: danhusan@leda:~$ ping6 nona.me PING nona.me(nona.me) 56 data bytes From 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net icmp_seq=1 Destination unreachable: No route From 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net icmp_seq=2 Destination unreachable: No route danhusan@leda:~$ ping6 ipv6.google.com PING ipv6.google.com(ey-in-x68.google.com) 56 data bytes From 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net icmp_seq=19 Destination unreachable: No route From 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net icmp_seq=20 Destination unreachable: No route danhusan@leda:~$ traceroute6 sixxs.net traceroute to sixxs.net (2001:838:1:1:210:dcff:fe20:7c7c), 30 hops max, 40 byte packets 1 2001:16d8:ee19:1::1 (2001:16d8:ee19:1::1) 2.101 ms 2.324 ms 2.789 ms 2 gw-41.osl-01.no.sixxs.net (2001:16d8:ee00:28::1) 35.493 ms 35.891 ms 36.001 ms 3 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net (2001:16d8:aaaa:4::1) 36.689 ms * * 4 v1611-r23.cr0-r69.tc-sto.se.ip6.p80.net (2001:16d8:1:1611::69) 44.210 ms 44.313 ms 44.428 ms 5 * * * 6 * * * 7 * * * 8 * * * 9 * * * 10 * * * 11 * * * 12 * * * 13 * * * 14 * * * 15 * * * 16 * * * 17 * * * 18 * * * 19 * * *
State change: user Locked
[ch] Jeroen Massar SixXS Staff on Thursday, 23 July 2009 16:53:43
Message is Locked
The state of this ticket has been changed to user
noosl01 upstream routing problems
[ch] Jeroen Massar SixXS Staff on Thursday, 23 July 2009 16:57:28
danhusan@leda:~$ ping6 nona.me
PING nona.me(nona.me) 56 data bytes
From 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net icmp_seq=1 Destination
unreachable: No route
"-n" is a very useful option that I suggest you use. At the moment "nona.me" maps to 2002:3e94:2407::1 which is a 6to4 address. 6to4 can go wrong on a lot of levels. Next to that, you are not showing the source address and SixXS pops will reject packets which are wrongly sourced. That said: # traceroute6 nona.me traceroute to nona.me (2002:3e94:2407::1) from 2001:16d8:aaaa:4::2, 30 hops max, 16 byte packets 1 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net (2001:16d8:aaaa:4::1) 0.293 ms * 0.194 ms 2 2001:7f8:12:1::2603 (2001:7f8:12:1::2603) 8.712 ms 67.916 ms 8.727 ms 3 dk-uni.nordu.net (2001:948:0:f056::2) 32.212 ms nona.me (2002:3e94:2407::1) 18.21 ms 18.368 ms # traceroute6 ipv6.google.com traceroute to ipv6.l.google.com (2001:4860:a003::68) from 2001:16d8:aaaa:4::2, 30 hops max, 16 byte packets 1 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net (2001:16d8:aaaa:4::1) 0.28 ms * 0.44 ms 2 v1611-r23.cr0-r69.tc-sto.se.ip6.p80.net (2001:16d8:1:1611::69) 7.699 ms 7.698 ms 7.734 ms 3 v1315-r69.cr0-r84.kn1-sto.se.ip6.p80.net (2001:16d8:1:1315::84) 7.976 ms 7.961 ms 7.733 ms 4 v1317-r84.cr0-r73.gbl-mlm.se.ip6.p80.net (2001:16d8:1:1317::73) 14.722 ms 14.709 ms 14.726 ms 5 v1306-r73.cr0-r72.gbl-cph.dk.ip6.p80.net (2001:16d8:1:1306::72) 15.22 ms 15.213 ms 15.226 ms 6 v1308-r72.cr0-r70.tc2-ams.nl.ip6.p80.net (2001:16d8:1:1308::70) 26.461 ms 177.36 ms 261.325 ms 7 pr61.ams04.net.google.com (2001:7f8:1::a501:5169:1) 28.211 ms 28.704 ms 28.716 ms ...[don't expect google to allow traceroutes into their network]... "Works" here.
noosl01 upstream routing problems
[no] Shadow Hawkins on Thursday, 23 July 2009 19:10:46
Have another look at this one: Ping to noc.sixxs.net: --- sixxs.net ping statistics --- 282 packets transmitted, 7 received, +58 errors, 97% packet loss, time 382450ms rtt min/avg/max/mdev = 56.738/58.614/62.780/2.028 ms This shows im sourcing my packets correctly (as I am actually recieving responses sometimes) But to be 100% clear im currently sourcing from 2001:16d8:ee19:1:240:63ff:fef3:7d7a danhusan@leda:~$ traceroute6 -n sixxs.net traceroute to sixxs.net (2001:838:1:1:210:dcff:fe20:7c7c), 30 hops max, 40 byte packets 1 2001:16d8:ee19:1::1 2.128 ms 2.370 ms 3.006 ms 2 2001:16d8:ee00:28::1 56.404 ms 56.515 ms 56.617 ms 3 2001:16d8:aaaa:4::1 57.752 ms * * 4 2001:16d8:1:1611::69 64.742 ms 64.854 ms 64.955 ms 5 * * * 6 * * * 7 * * * 8 * * * This has been confirmed by a friend on the same pop.
noosl01 upstream routing problems
[no] Shadow Hawkins on Friday, 24 July 2009 11:19:43
I have the same problem on the same POP. aiccu autotest AICCU 2007.01.15-console-linux by Jeroen Massar. Debian 5.0.2 (lenny) with Linux 2.6.26-2-ixp4xx #1 Sun Jun 21 23:07:09 UTC 2009 armv5tel GNU/Linux.
noosl01 upstream routing problems
[no] Shadow Hawkins on Thursday, 23 July 2009 19:18:23
More information, this time from a Windows 7 client with IP: 2001:16d8:ee19:1:31b0:3712:6460:f04d When running this ping I also had a ping going to my remote tunnel endpoint (2001:16d8:ee00:28::1) in another window, that one did not loose any packets. Pinging noc.sixxs.net [2001:838:1:1:210:dcff:fe20:7c7c] with 32 bytes of data: Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable. Request timed out. Destination net unreachable. Destination net unreachable. Destination net unreachable. Request timed out. Request timed out. Request timed out. Request timed out. Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=55ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=55ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=57ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=55ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=56ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=58ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=54ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=60ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=55ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=56ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=55ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=56ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=56ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=55ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=56ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=56ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=59ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=57ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=55ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=65ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=57ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=61ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=62ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=57ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=57ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=56ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=80ms Reply from 2001:838:1:1:210:dcff:fe20:7c7c: time=55ms Request timed out. Request timed out. Request timed out. Request timed out. Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable. Request timed out. Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable. Request timed out. Destination net unreachable. Destination net unreachable. Request timed out. Request timed out. Destination net unreachable. Destination net unreachable. Destination net unreachable. Destination net unreachable.
noosl01 upstream routing problems
[ch] Jeroen Massar SixXS Staff on Friday, 24 July 2009 18:09:21
When running this ping I also had a ping going to my remote tunnel endpoint
(2001:16d8:ee00:28::1) in another window, that one did not loose any packets.
Thus the PoP is working fine.
Pinging noc.sixxs.net [2001:838:1:1:210:dcff:fe20:7c7c] with 32 bytes of data:
Destination net unreachable.
Destination net unreachable.
Destination net unreachable.
Destination net unreachable.
What host/router is reporting that "destination net unreachable"? What is the source of that packet? Did you check with wireshark? Where is the traceroute? etc etc etc
noosl01 upstream routing problems
[no] Shadow Hawkins on Friday, 24 July 2009 19:08:42
$ ip -6 addr list dev sixxs 10: sixxs: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 qlen 500 inet6 2001:16d8:ee00:4a::2/64 scope global valid_lft forever preferred_lft forever inet6 fe80::14d8:ee00:4a:2/64 scope link valid_lft forever preferred_lft forever $ ping6 -n -c10 noc.sixxs.net PING noc.sixxs.net(2001:838:1:1:210:dcff:fe20:7c7c) 56 data bytes From 2001:16d8:aaaa:4::1 icmp_seq=1 Destination unreachable: No route From 2001:16d8:aaaa:4::1 icmp_seq=2 Destination unreachable: No route From 2001:16d8:aaaa:4::1 icmp_seq=3 Destination unreachable: No route From 2001:16d8:aaaa:4::1 icmp_seq=4 Destination unreachable: No route From 2001:16d8:aaaa:4::1 icmp_seq=5 Destination unreachable: No route From 2001:16d8:aaaa:4::1 icmp_seq=6 Destination unreachable: No route From 2001:16d8:aaaa:4::1 icmp_seq=7 Destination unreachable: No route From 2001:16d8:aaaa:4::1 icmp_seq=8 Destination unreachable: No route From 2001:16d8:aaaa:4::1 icmp_seq=9 Destination unreachable: No route From 2001:16d8:aaaa:4::1 icmp_seq=10 Destination unreachable: No route --- noc.sixxs.net ping statistics --- 10 packets transmitted, 0 received, +10 errors, 100% packet loss, time 9103ms $ traceroute6 -n -m 15 noc.sixxs.net traceroute to noc.sixxs.net (2001:838:1:1:210:dcff:fe20:7c7c), 15 hops max, 40 byte packets 1 2001:16d8:ee00:4a::1 26.459 ms 28.565 ms 31.243 ms 2 2001:16d8:aaaa:4::1 30.345 ms * * 3 * * * 4 * * * 5 * * * 6 * * * 7 * * * 8 * * * 9 * * * 10 * * * 11 * * * 12 * * * 13 * * * 14 * * * 15 * * * And a packet capture from the sixxs interface (tcpdump -i sixxs -s0 -w ping_noc.sixxs.net_200907241837+02.pcap) while pinging noc.sixxs.net.
noosl01 upstream routing problems
[no] Shadow Hawkins on Friday, 24 July 2009 19:07:56
Forgot a /link.. :(
$ date; sudo /etc/init.d/aiccu restart; sleep 5; sudo tcpdump -i sixxs -s0 -w sixxs.pcap & ip -6 addr list sixxs && ping6 -c 10 -n noc.sixxs.net; traceroute6 -n -m 15 noc.sixxs.net; sudo /etc/init.d/aiccu stop Fri Jul 24 18:50:41 CEST 2009 Restarting SixXS Automatic IPv6 Connectivity Client Utility (aiccu).... [1] 7167 13: sixxs: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 qlen 500 inet6 2001:16d8:ee00:4a::2/64 scope global valid_lft forever preferred_lft forever inet6 fe80::14d8:ee00:4a:2/64 scope link valid_lft forever preferred_lft forever PING noc.sixxs.net(2001:838:1:1:210:dcff:fe20:7c7c) 56 data bytes From 2001:16d8:aaaa:4::1 icmp_seq=1 Destination unreachable: No route tcpdump: WARNING: arptype 65534 not supported by libpcap - falling back to cooked socket tcpdump: WARNING: sixxs: no IPv4 address assigned tcpdump: listening on sixxs, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes From 2001:16d8:aaaa:4::1 icmp_seq=2 Destination unreachable: No route From 2001:16d8:aaaa:4::1 icmp_seq=3 Destination unreachable: No route --- noc.sixxs.net ping statistics --- 10 packets transmitted, 0 received, +3 errors, 100% packet loss, time 9059ms traceroute to noc.sixxs.net (2001:838:1:1:210:dcff:fe20:7c7c), 15 hops max, 40 byte packets 1 2001:16d8:ee00:4a::1 26.620 ms 28.780 ms 30.003 ms 2 2001:16d8:aaaa:4::1 28.926 ms * * 3 2001:16d8:1:1611::69 39.252 ms 38.206 ms 37.094 ms 4 * * * 5 * * * 6 * * * 7 * * * 8 * * * 9 * * * 10 * * * 11 * * * 12 * * * 13 * * * 14 * * * 15 * * * Stopping SixXS Automatic IPv6 Connectivity Client Utility (aiccu)...tcpdump: pcap_loop: recvfrom: Network is down 64 packets captured 64 packets received by filter 0 packets dropped by kernel . [1]+ Exit 1 sudo tcpdump -i sixxs -s0 -w sixxs.pcap
pcap
noosl01 upstream routing problems
[no] Shadow Hawkins on Friday, 24 July 2009 18:57:14
Source of the route unreachable packets is 2001:16d8:aaaa:4::1 as seen in the traceroutes posted in another of my posts further down.
noosl01 upstream routing problems
[no] Shadow Hawkins on Friday, 24 July 2009 18:58:23
Sorry for my problems posting... What I wanted to post the packet capture
noosl01 upstream routing problems
[no] Shadow Hawkins on Sunday, 26 July 2009 18:06:54
When running this ping I also had a ping going to my remote tunnel endpoint
(2001:16d8:ee00:28::1) in another window, that one did not loose any packets.
Thus the PoP is working fine.
Not necessarily. It only confirms that the IPv6 encapsulated packets can reach the ipv4 tunnel endpoint, it doesn't mean the IPv6-paths from the PoP are working as they should. About time to escalate this ticket perhaps? And change the ticket state from "user" as this is obviously not an user error.
noosl01 upstream routing problems
[no] Shadow Hawkins on Friday, 24 July 2009 16:51:54
This PoP has given me headaches for a week as well. Basically the same symptoms as Daniel reports. Interestingly enough, it seems to be different to different networks -- at times, I can reach UNINETT (AS224) but not Google (AS15169), for instance.
noosl01 upstream routing problems
[ch] Jeroen Massar SixXS Staff on Friday, 24 July 2009 18:10:04
Interestingly enough, it seems to be different to different networks -- at
times, I can reach UNINETT (AS224) but not Google (AS15169), for instance.
Where is a traceroute? What source address? etc etc
noosl01 upstream routing problems
[no] Shadow Hawkins on Friday, 24 July 2009 18:40:10
Further traceroutes to show how different destinations are working at different times, as seen in the timestamps all of these were done within 2 minutes of eachother. Sourcing from: 2001:16d8:ee19:1:240:63ff:fef3:7d7a danhusan@leda:~$ date && traceroute6 -n www.sixxs.net Fri Jul 24 18:35:59 CEST 2009 traceroute to www.sixxs.net (2001:838:1:1:210:dcff:fe20:7c7c), 30 hops max, 40 byte packets 1 2001:16d8:ee19:1::1 2.130 ms 2.362 ms 2.980 ms 2 2001:16d8:ee00:28::1 25.452 ms 25.564 ms 28.604 ms 3 2001:16d8:aaaa:4::1 28.716 ms * * 4 2001:16d8:1:1611::69 36.387 ms 36.509 ms 36.611 ms 5 2001:16d8:1:1315::84 36.721 ms 36.811 ms 36.922 ms 6 2001:16d8:1:1317::73 43.147 ms 41.326 ms 41.217 ms 7 2001:16d8:1:1306::72 41.151 ms 39.138 ms 39.203 ms 8 2001:16d8:1:1308::70 50.817 ms 50.901 ms 51.615 ms 9 2001:7f8:1::a501:2871:2 66.585 ms 66.766 ms 66.510 ms 10 2001:838:5:2::1 52.146 ms 51.889 ms 58.163 ms 11 2001:838:5:a::2 61.982 ms 62.490 ms 61.534 ms 12 2001:838:1:1:210:dcff:fe20:7c7c 55.723 ms 55.725 ms 54.431 ms danhusan@leda:~$ date && traceroute6 -n www.sunet.se Fri Jul 24 18:36:33 CEST 2009 traceroute to www.sunet.se (2001:6b0:e:1::f:1), 30 hops max, 40 byte packets 1 2001:16d8:ee19:1::1 2.166 ms 2.405 ms 3.011 ms 2 2001:16d8:ee00:28::1 25.565 ms 28.563 ms 28.676 ms 3 * * * 4 2001:7f8:12:1::2603 37.315 ms 37.673 ms 38.044 ms 5 2001:948:0:f049::1 37.390 ms 37.862 ms 38.243 ms 6 2001:948:0:f051::2 37.450 ms 35.716 ms 35.858 ms 7 2001:6b0:dead:beef:2::de 35.519 ms 43.016 ms 43.517 ms 8 2001:6b0:dead:beef:2::2e6 55.879 ms 49.718 ms 49.991 ms 9 2001:6b0:e::205a 50.360 ms 50.370 ms 50.350 ms 10 2001:6b0:e:1::f:1 49.239 ms 49.228 ms 49.367 ms danhusan@leda:~$ date && traceroute6 -n www.sixxs.net Fri Jul 24 18:36:47 CEST 2009 traceroute to www.sixxs.net (2001:838:1:1:210:dcff:fe20:7c7c), 30 hops max, 40 byte packets 1 2001:16d8:ee19:1::1 2.149 ms 2.369 ms 2.971 ms 2 2001:16d8:ee00:28::1 28.644 ms 28.764 ms 28.867 ms 3 2001:16d8:aaaa:4::1 29.398 ms !N * * danhusan@leda:~$ date && traceroute6 -n www.sunet.se Fri Jul 24 18:37:00 CEST 2009 traceroute to www.sunet.se (2001:6b0:e:1::f:1), 30 hops max, 40 byte packets 1 2001:16d8:ee19:1::1 2.159 ms 2.902 ms 3.390 ms 2 2001:16d8:ee00:28::1 110.974 ms 111.080 ms 111.182 ms 3 2001:16d8:aaaa:4::1 111.295 ms * * 4 2001:7f8:12:1::2603 119.951 ms 120.063 ms 123.858 ms 5 2001:948:0:f049::1 123.455 ms 123.934 ms 124.175 ms 6 2001:948:0:f051::2 123.531 ms 122.013 ms 121.530 ms 7 2001:6b0:dead:beef:2::de 120.425 ms 108.597 ms 108.643 ms 8 2001:6b0:dead:beef:2::2e6 120.387 ms 120.254 ms 136.026 ms 9 2001:6b0:e::205a 139.517 ms 144.500 ms 146.956 ms 10 2001:6b0:e:1::f:1 146.986 ms 146.988 ms 146.997 ms
noosl01 upstream routing problems
[no] Shadow Hawkins on Friday, 24 July 2009 18:53:17
I'm having the same problems on the same pop: cisco#traceroute ipv6 Target IPv6 address: ipv6.google.com Source address: 2001:16D8:EE00:31::2 Insert source routing header? [no]: Numeric display? [no]: Timeout in seconds [3]: Probe count [3]: Minimum Time to Live [1]: Maximum Time to Live [30]: 10 Priority [0]: Port Number [33434]: Type escape sequence to abort. Tracing the route to ipv6.l.google.com (2001:4860:A005::68) 1 gw-50.osl-01.no.sixxs.net (2001:16D8:EE00:31::1) 4 msec 0 msec 4 msec 2 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net (2001:16D8:AAAA:4::1) !U * * cisco#sh run int tunn0 Building configuration... Current configuration : 266 bytes ! interface Tunnel0 description SixXS bandwidth 100000 no ip address ip mtu 1280 load-interval 30 ipv6 address 2001:16D8:EE00:31::2/64 ipv6 flow ingress ipv6 flow egress tunnel source FastEthernet0/0 tunnel destination 83.140.3.18 tunnel mode ipv6ip end cisco#sh run | i ipv6 route ipv6 route ::/0 2001:16D8:EE00:31::1 And it seems to be a recurring problem. Some times it's working great but now it's unstable.
noosl01 upstream routing problems
[no] Shadow Hawkins on Friday, 24 July 2009 19:20:48
From the sunet looking glass I can see the prefix flapped 1w2d ago which also seems to be when the problems first occured. Coincident? Router: c1sth.sunet.se Command: 'show route protocol bgp 2001:16D8::' inet6.0: 2230 destinations, 2466 routes (2230 active, 0 holddown, 11 hidden) + = Active Route, - = Last Active, * = Both 2001:16d8::/32 *[BGP/170] 1w2d 09:11:54, MED 9, localpref 100 AS path: 2603 16150 I > to 2001:948:0:f051::1 via so-6/0/0.0
Unstable connectivity to multiple destinations via noosl01
[no] Shadow Hawkins on Friday, 24 July 2009 19:03:25
I have read and followed the "Reporting Problems" section on the Contact page. Hello, I'm experiencing unstable connectivity via noosl01 (tunnel ID T19995, route ID R8490). When testing towards Google, I get "no route" ICMP messages from 2001:16d8:aaaa:4::1, a router in Phonera's network: traceroute to ipv6.google.com (2001:4860:a003::68), 30 hops max, 40 byte packets 1 gw.v6.fud.no (2001:16d8:ee47::1) 0.456 ms * * 2 gw-135.osl-01.no.sixxs.net (2001:16d8:ee00:86::1) 13.804 ms 15.422 ms 16.922 ms 3 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net (2001:16d8:aaaa:4::1) 23.417 ms !N * * When it works, it looks like this: traceroute to ipv6.google.com (2001:4860:a003::68), 30 hops max, 40 byte packets 1 gw.v6.fud.no (2001:16d8:ee47::1) 0.452 ms * * 2 gw-135.osl-01.no.sixxs.net (2001:16d8:ee00:86::1) 14.756 ms 15.686 ms 16.665 ms 3 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net (2001:16d8:aaaa:4::1) 18.089 ms * * 4 v1611-r23.cr0-r69.tc-sto.se.ip6.p80.net (2001:16d8:1:1611::69) 29.857 ms 30.782 ms 32.119 ms 5 v1315-r69.cr0-r84.kn1-sto.se.ip6.p80.net (2001:16d8:1:1315::84) 33.420 ms 34.706 ms 35.544 ms 6 v1317-r84.cr0-r73.gbl-mlm.se.ip6.p80.net (2001:16d8:1:1317::73) 43.922 ms 36.831 ms 31.941 ms 7 v1306-r73.cr0-r72.gbl-cph.dk.ip6.p80.net (2001:16d8:1:1306::72) 33.210 ms 33.182 ms 32.640 ms 8 v1308-r72.cr0-r70.tc2-ams.nl.ip6.p80.net (2001:16d8:1:1308::70) 39.129 ms 39.134 ms 39.510 ms 9 pr61.ams04.net.google.com (2001:7f8:1::a501:5169:1) 40.871 ms 40.523 ms 40.605 ms 10 * * * 11 * * * 12 fx-in-x68.google.com (2001:4860:a003::68) 47.810 ms 47.204 ms 46.714 ms Multiple destinations are affected, I tried pinging simultaneously to ipv6.google.com, www.sixxs.net, and www.ripe.net, and cr2.hmg9.no.v6.linpro.net (the last one is a router I administer, so I know for certain that it has good IPv6 connectivity at the time of testing) - most of the times I only get that ICMP error from 2001:16d8:aaaa:4::1, but when I start getting reply packets, they appear from all destinations at the same time (and they stop appearing at the same time as well). It seems to me, therefore, that this error is found somewhere inside Phonera's network. A link/route that's flapping, probably. Perhaps it also affects Phonera's other PoPs? A traceroute from RIPE's looking glass towards the PoP end of my tunnel also yields a "no route" error - that should rule out any mis-configuration on my end, I believe: traceroute to 2001:16d8:ee00:86::1 (2001:16d8:ee00:86::1) from 2001:610:240:3:ffff:0:4:28, 30 hops max, 16 byte packets 1 2001:610:240:3::1:1 (2001:610:240:3::1:1) 1.572 ms 89.031 ms 5.13 ms 2 ams-ix.he.net (2001:7f8:1::a500:6939:1) 0.708 ms 0.639 ms 0.573 ms 3 ams-ix.tc2-ams.nl.p80.net (2001:7f8:1::a501:6150:1) 85.521 ms !N 276.096 ms !N 1.64 ms !N The problem has been going on for a few days (at least - I just returned from a three-week long vacation). Best regards, Tore Anderson (TA1353-RIPE)
noosl01 upstream routing problems
[no] Shadow Hawkins on Saturday, 25 July 2009 12:02:26
I also experience this flapping. Pinging from Uninett towards the tunnel gives about 60 seconds working, 60 seconds broken .. rinse and repeat. I've been running an outward ping6 from behind the tunnel to work around any sudden NAT-problems. It doesn't seem to have any impact on the measurements. Lengthy ping6 paste here: http://pastebin.com/m5af6df2f lkarsten@jungel:~$ dig enjoy.hyse.org aaaa [..] ;; ANSWER SECTION: enjoy.hyse.org.85087INAAAA2001:16d8:ee26:1::1 From NTNU/Uninett: (working) lkarsten@jungel:~$ tracepath6 enjoy.hyse.org 1?: [LOCALHOST] pmtu 1500 1: 2001:700:300:22::1 0.892ms 1: 2001:700:300:22::1 0.763ms 2: ntnu-gsw2.nettel.ntnu.no 1.196ms 3: hovedbygget-gw4.uninett.no 1.026ms asymm 5 4: hovedbygget-gw1.uninett.no 1.037ms asymm 5 5: trd-gw1.uninett.no 1.942ms asymm 4 6: oslo-gw.uninett.no 8.880ms asymm 5 7: se-tug.nordu.net 16.596ms asymm 6 8: 2001:7f8:12:1::1:6150 25.175ms asymm 6 9: sixxs-oslo-demarc0.cust.ip6.p80.net 25.657ms asymm 7 10: sixxs-oslo-demarc0.cust.ip6.p80.net 25.587ms pmtu 1280 10: 2001:16d8:ee26:1::1 57.033ms reached 10: 2001:16d8:ee26:1::1 56.558ms reached Resume: pmtu 1280 hops 10 back 57 ~1-2 minutes later: lkarsten@jungel:~$ tracepath6 enjoy.hyse.org 1?: [LOCALHOST] pmtu 1500 1: 2001:700:300:22::1 1.524ms 1: 2001:700:300:22::1 0.774ms 2: ntnu-gsw2.nettel.ntnu.no 1.052ms 3: hovedbygget-gw4.uninett.no 1.029ms asymm 5 4: hovedbygget-gw1.uninett.no 1.052ms asymm 5 5: trd-gw1.uninett.no 1.169ms asymm 4 6: oslo-gw.uninett.no 8.941ms asymm 5 7: se-tug.nordu.net 16.697ms asymm 6 8: se-fre.nordu.net 17.126ms asymm 7 9: 2001:7f8:d:fb::143 16.894ms !N Resume: pmtu 1500 lkarsten@jungel:~$ LC_ALL=en_en date Sat Jul 25 11:55:50 CEST 2009 lkarsten@jungel:~$
noosl01 upstream routing problems
[no] Shadow Hawkins on Sunday, 26 July 2009 15:50:38
I have read and followed the "Reporting Problems" section on the Contact page. I am removing these four pre-filled lines to demonstrate that I really read it. I am providing the full details as asked on the above mentioned page. ------------------------------------------------------------------------------>8 I have the exact same problem. I'm pinging from my endpoint 2001:16d8:ee00:91::2/64. eirik@srv:~$ date Sun Jul 26 15:47:33 CEST 2009 eirik@srv:~$ traceroute6 ipv6.google.com traceroute to ipv6.l.google.com (2001:4860:a005::68) from 2001:16d8:ee00:91::2, 30 hops max, 24 byte packets 1 gw-146.osl-01.no.sixxs.net (2001:16d8:ee00:91::1) 0.538 ms 0.31 ms 0.493 ms 2 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net (2001:16d8:aaaa:4::1) 0.723 ms * 0.954 ms 3 v1611-r23.cr0-r69.tc-sto.se.ip6.p80.net (2001:16d8:1:1611::69) 8.3 ms 8.258 ms 7.987 ms 4 * * * 5 * * * 6 * * * 7 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net (2001:16d8:aaaa:4::1) 0.807 ms !N eirik@srv:~$ date Sun Jul 26 15:49:58 CEST 2009 eirik@srv:~$ mtr ipv6.google.com 1. gw-146.osl-01.no.sixxs.net 0.0% 11 0.5 0.5 0.3 1.2 0.2 2. 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net 30.0% 11 0.8 0.7 0.6 0.8 0.1 3. v1611-r23.cr0-r69.tc-sto.se.ip6.p80.net 10.0% 11 8.3 8.2 8.1 8.3 0.1 4. v1315-r69.cr0-r84.kn1-sto.se.ip6.p80.net 10.0% 11 8.7 8.3 8.1 8.7 0.2 5. v1317-r84.cr0-r73.gbl-mlm.se.ip6.p80.net 10.0% 11 15.1 15.2 15.1 15.3 0.1 6. v1306-r73.cr0-r72.gbl-cph.dk.ip6.p80.net 10.0% 11 15.9 15.7 15.6 15.9 0.1 7. v1308-r72.cr0-r70.tc2-ams.nl.ip6.p80.net 0.0% 10 26.5 29.3 26.5 49.6 7.6 8. pr61.ams04.net.google.com 0.0% 10 27.3 37.8 27.1 92.4 21.5 9. ??? 10. ??? 11. ey-in-x68.google.com 0.0% 10 31.6 34.3 31.6 45.6 5.1
POP flapping
[no] Shadow Hawkins on Monday, 27 July 2009 16:00:11
NIC-handle: PH2910-RIPE Tunnel: T15346 It appears to be an unreliable connection between noosl01 and the rest of the v6 network. Ping6 running against noc.sixxs.net alternates between ok, no response, and unreachables from 2002-sixxs.cr0-r23.ver-osl.no.ip6.p80.net. Outside connectivity only remain open for 10-30 sec at a time. Meanwhile, the AICCU-tunnel itself is rock stable when tested against the far end of the tunnel. Trace with v6 on the webtool from noc to noosl01 also fails (as expected) during the periods that the pop returns "no route" to the client. IPv6 traceroute from noc.sixxs.net @ SixXS NOC, AS12871 to noosl01.sixxs.net / Phonera, AS16150 : Hop Node Loss% Sent Last Avg Best Worst StDev ASN Organisation 1. 2001:838:1:1::1 0.0% 5 0.4 2.1 0.4 9.2 3.9 [.nl] Netherlands, The 12871 Concepts ICT ge-1-3-0.breda.ipv6.concepts-ict.net. 2. 2001:838:5:a::1 0.0% 5 4.0 4.0 4.0 4.1 0.1 [.nl] Netherlands, The 12871 Concepts ICT 3. 2001:7f8:1::a501:6150:1 0.0% 5 4.6 4.7 4.6 4.9 0.1 [.nl] Netherlands, The AMS-IX-IPV6 ams-ix.tc2-ams.nl.p80.net. 4. 2001:16d8:1:1308::72 0.0% 5 15.6 15.6 15.6 15.8 0.1 [.se] Sweden 16150 Port80 AB v1308-r70.cr0-r72.gbl-cph.dk.ip6.p80.net. 5. ??? 100.0 5 0.0 0.0 0.0 0.0 0.0 It varies how far this trace gets which indicate that the prefix-announcement from noosl01 may be withdrawn due to a failure in the pop and then re-propagated, but that's only speculation from my pov. The client software and config works fine against T9801 on sesto01 (no changes to the config other than tunnel id).
Seems fixed
[no] Shadow Hawkins on Tuesday, 04 August 2009 02:30:08
The issue seems to be resolved - I've pinged through my tunnel for 24 hours and the packet loss was insignificant: --- cr2.hmg9.no.v6.linpro.net ping statistics --- 86400 packets transmitted, 86173 received, 0% packet loss, time 86745053ms rtt min/avg/max/mdev = 26.905/41.496/256.920/23.071 ms I think this ticket can be closed. Tore

Please note Posting is only allowed when you are logged in.

Static Sunset Edition of SixXS
©2001-2017 SixXS - IPv6 Deployment & Tunnel Broker