|
Many Google sub-sites unresponsive over IPv6 as of Nov 7 (fonts.gstatic.com, others)
![[us]](/s/countries/us.gif) Shadow Hawkins on Friday, 07 November 2014 17:30:45
This is no doubt unrelated to the Akamai issue, but symptoms are similar: lots of hanging web sites that partially render with successful ping connectivity. Given the number of sites that rely on Google in one fashion or another, it breaks a *lot* of stuff. Anyone else seeing it here?
Users on Tunnel Broker are reporting the same: https://forums.he.net/index.php?topic=3281.0
As expected, no changes with MTU fiddling (as it shouldn't).
Many Google sub-sites unresponsive over IPv6 as of Nov 7 (fonts.gstatic.com, others)
![[de]](/s/countries/de.gif) Carmen Sandiego on Friday, 07 November 2014 17:55:24
Brian Conway wrote:
This is no doubt unrelated to the Akamai issue, but symptoms are similar: lots of hanging web sites that partially render with successful ping connectivity. Given the number of sites that rely on Google in one fashion or another, it breaks a *lot* of stuff. Anyone else seeing it here?
Users on Tunnel Broker are reporting the same: https://forums.he.net/index.php?topic=3281.0
As expected, no changes with MTU fiddling (as it shouldn't).
Yes i seen this too.
Many Google sub-sites unresponsive over IPv6 as of Nov 7 (fonts.gstatic.com, others)
Shadow Hawkins on Friday, 07 November 2014 18:03:22
Stefan Gebhardt wrote:
Yes i seen this too.
Cool, thanks. I saw a few mentions on Twitter as well.
No easy ways to contact Google network folks, to say the least.
Many Google sub-sites unresponsive over IPv6 as of Nov 7 (fonts.gstatic.com, others)
Carmen Sandiego on Friday, 07 November 2014 20:11:10
Brian Conway wrote:
Stefan Gebhardt wrote:
No Problem.
well ill work around it (i hope it fixes it self soon) with disableing ipv6 on my client (but not on my router, aiccu still running).
with only ipv4 all runs fine. :(
Yes i seen this too.
Cool, thanks. I saw a few mentions on Twitter as well.
No easy ways to contact Google network folks, to say the least.
Many Google sub-sites unresponsive over IPv6 as of Nov 7 (fonts.gstatic.com, others)
Shadow Hawkins on Friday, 07 November 2014 21:33:55
agree ... same problem here
Germany / Kln (Cologne)
TweetDesk & Google wont work
Many Google sub-sites unresponsive over IPv6 as of Nov 7 (fonts.gstatic.com, others)
Shadow Hawkins on Saturday, 08 November 2014 03:15:11
I'm seeing the same thing on the occaid POP at Ashburn VA. Traceroutes look OK to me but I'm getting timeouts on a lot of google stuff on IPV6 that work fine on V4. I've shut down my tunnel for now since google is probably half of my IPV6 (the other half being FB).
Joel
StrongBad> traceroute6 mail.google.com
traceroute to mail.google.com (2607:f8b0:4004:800::1016), 30 hops max, 16 byte packets
1 gw-1216.qas-01.us.sixxs.net (2001:4830:1600:4bf::1) 9.614 ms 9.415 ms 10.178 ms
2 sixxs-asbnva-gw.customer.occaid.net (2001:4830:e6:7::2) 9.712 ms 9.319 ms 9.696 ms
3 iad0-b0-ge2.hotnic.net (2001:4810:0:100::1) 13.084 ms 12.845 ms 9.685 ms
4 pr61.iad07.net.google.com (2001:504:0:2:0:1:5169:1) 12.370 ms 11.558 ms 13.087 ms
5 2001:4860::1:0:9ff (2001:4860::1:0:9ff) 11.834 ms 14.952 ms 24.821 ms
6 2001:4860:0:1::551 (2001:4860:0:1::551) 11.824 ms 15.088 ms 12.827 ms
7 2607:f8b0:8000:18::c (2607:f8b0:8000:18::c) 11.846 ms 12.413 ms 12.357 ms
StrongBad> /opt/sbin/aiccu stop
Many Google sub-sites unresponsive over IPv6 as of Nov 7 (fonts.gstatic.com, others)
Shadow Hawkins on Saturday, 08 November 2014 21:01:45
Brian Conway wrote:
This is no doubt unrelated to the Akamai issue, but symptoms are similar: lots of hanging web sites that partially render with successful ping connectivity. Given the number of sites that rely on Google in one fashion or another, it breaks a *lot* of stuff. Anyone else seeing it here?
Users on Tunnel Broker are reporting the same: https://forums.he.net/index.php?topic=3281.0
As expected, no changes with MTU fiddling (as it shouldn't).
Ping6 seems to work fine to these sites. Looks like a tcp thing. Tcpdump reports lots of "length 0" packets. Maybe their load balancers?
Many Google sub-sites unresponsive over IPv6 as of Nov 7 (fonts.gstatic.com, others)
As per a post to the outages list, these problems should now be resolved.
It seems to have been a ICMPv6 filter on a variety of their clusters.
And indeed it seems to be the case from my POV.
Thus it seems that public complaining in the relevant mailinglists (nanog, outages, ipv6-ops), helps, not doing so, means there is unfortunately no contact to Google (the email address ipv6@google.com is bouncing, and that used to be the direct contact for IPv6 related matters..).
Now for Akamai to step up and figure out and fix their problems and everything is fine again.
Many Google sub-sites unresponsive over IPv6 as of Nov 7 (fonts.gstatic.com, others)
Shadow Hawkins on Sunday, 09 November 2014 12:54:39
Jeroen Massar wrote:
As per a post to the outages list, these problems should now be resolved.
It seems to have been a ICMPv6 filter on a variety of their clusters.
And indeed it seems to be the case from my POV.
Yes, here, too.
Many Google sub-sites unresponsive over IPv6 as of Nov 7 (fonts.gstatic.com, others)
From an email by Colitti Lorenzo @ Google:
As to what happened: what happened here is that some Google servers
temporarily stopped doing MSS clamping. That was an outage, and AIUI it has
since been fixed. (Some parts of) Google infrastructure do not do PMTUD for
the latency reasons above and for reasons similar to those listed in
https://tools.ietf.org/html/draft-v6ops-jaeggli-pmtud-ecmp-problem-00 .
Or otherwise said: Google is on-purpose breaking PMTUD and favouring to 'fix' this using MSS clamping on *THEIR* side.
As such, even native 1500-MTUd networks will get smaller packets than they should.
It is so good nobody uses IPSEC AH...
One could only wish big networks like that would help towards fixing problems, instead of just giving them longer live.
And one only has to guess what happens when QUIC/HTTP2 becomes ubiquitous....
|
![[ch]](/s/countries/ch.gif)
on Sunday, 09 November 2014 09:20:53
Posting is only allowed when you are