|
Ticket ID: SIXXS #4370681 Ticket Status: User PoP: fihel01 - DNA Oy (Helsinki)
Tunnel broken. webUI & AICCU disagree.
![[fi]](/s/countries/fi.gif) Shadow Hawkins on Sunday, 24 April 2011 02:00:23
* Always include clear descriptive information about your problem or inquiry.
Sixxs webUI and aiccu both say my tunnel is up. I can't even ping the tunnel endpoint ipv6 address or get any kind of data moving through the tunnel. Tried it with 4 different computers under two different routers which I've configuref myself to pass through AYIYA.
* When on a UNIX-alike platform that supports AICCU, please run AICCU and run it using "aiccu test".
* When using AICCU, check that you are using the latest version, and of course please specify where the binary comes from.
AICCU says it's up and running. All software is updated.
* Always provide your NIC handle and if applicable the Tunnel or Route IDs you wish to discuss.
MAK4-SIXXS and it is my only tunnel ATM
* Use the email address you have provided in your handle as that is what we use as a contact handle.
* Provide details of the setup, type of connections, where NATs are located.
I have a Ubuntu 10.04 LTS working as a tunnel & subnet endpoint with AYIYA and giving out ipv6 addresses through RADVD. It is behind a pfsense box with a NAT, but the port forwards are done.
* Provide information of your OS type, version and release (ie. uname -a), noting also the distribution name.
uname -a:
Linux modafiniili 2.6.32-30-generic #59-Ubuntu SMP Tue Mar 1 21:30:21 UTC 2011 i686 GNU/Linux
* Include full interface, routing and firewall tables.
ip link:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 00:13:8f:e8:0f:67 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:07:e9:10:24:40 brd ff:ff:ff:ff:ff:ff
4: sit0: <NOARP> mtu 1480 qdisc noop state DOWN
link/sit 0.0.0.0 brd 0.0.0.0
22: sixxs: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 qdisc pfifo_fast state UNKNOWN qlen 500
link/[65534]
ip -6 route:
2001:14b8:100:2c1::/64 dev sixxs proto kernel metric 256 mtu 1280 advmss 1220 hoplimit 0
unreachable fe80::/64 dev lo proto kernel metric 256 error -101 mtu 16436 advmss 16376 hoplimit 0
fe80::/64 dev eth1 proto kernel metric 256 mtu 1500 advmss 1440 hoplimit 0
fe80::/64 dev sixxs proto kernel metric 256 mtu 1280 advmss 1220 hoplimit 0
default via 2001:14b8:100:2c1::1 dev sixxs metric 1024 mtu 1280 advmss 1220 hoplimit 0
sudo ip6tables -L -v -n
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 48 packets, 3840 bytes)
pkts bytes target prot opt in out source destination
* Include a IPv4 and IPv6 traceroute to the PoP in question.
traceroute to fihel01.sixxs.net (62.78.96.38), 30 hops max, 60 byte packets
1 nat.metropolis (192.168.1.1) 0.083 ms 0.064 ms 0.077 ms
2 gw-620.tontut.fi (94.237.72.1) 0.528 ms 0.576 ms 0.628 ms
3 toas-gw1-x0101-her.tontut.fi (94.237.0.250) 0.493 ms 0.478 ms 0.452 ms
4 funet-tut6-x1130.tontut.fi (94.237.0.241) 0.764 ms 0.747 ms 0.729 ms
5 csc6-x0100-tut6.funet.fi (193.166.255.77) 3.478 ms 3.407 ms 3.432 ms
6 dna.ficix1.ficix.fi (193.110.226.20) 3.604 ms 3.603 ms 3.589 ms
7 esp2-tr1.dnaip.fi (62.78.107.44) 4.057 ms 4.048 ms 4.015 ms
8 van1-er2.dnaip.fi (62.78.111.87) 4.455 ms 4.513 ms 4.637 ms
9 fihel01.sixxs.net (62.78.96.38) 4.366 ms 4.350 ms 4.342 ms
The ipv6 traceroute to fihel01.sixxs.net doesn't even finish.
* With heartbeat tunnels, first check clock synchronization or better: use AICCU.
Checked & using.
* Check with Wireshark or tcpdumps of the interface over which the tunnel runs.
Use -n (numeric) as an option and don't filter returning ICMP which could also come from routers between your endpoint and the PoP and also use -s 1500 so that one gets the full packet.
tcpdump doesn't show any traffic between sixxs and my server.
* The status of the PoP is listed on the PoP Status page, if it is marked down there we are aware of the issue and we will try to resolve it as soon as possible. Additionally other issues are listed in the Ticket Tracker Ticket Tracker.
It is listed as working.
P.S. Thanks to Joonas Kortesalmi for the base of this ticket. :)
State change: user
The state of this ticket has been changed to user
Tunnel broken. webUI & AICCU disagree.
Tried it with 4 different computers under two different routers which I've configuref myself to pass through AYIYA.
What exactly did you configure? The way you write it you state that you configured them to pass through AYIYA, thus all of them through that protocol, which does not make sense.
AICCU says it's up and running. All software is updated.
That is because it only configures the tunnel, it does not do any connectivity test.
It is behind a pfsense box with a NAT, but the port forwards are done.
Port forwards? The whole point of AYIYA is that it does not need any port forwards as it can cross normal NATs anyway (unless you filter ports in a firewall that is).
Also, some people claim pfsense has IPv6 support, thus you might want to check if that is causing you issues and/or terminate the tunnel there.
The ipv6 traceroute to fihel01.sixxs.net doesn't even finish.
But where does it do go?
tcpdump doesn't show any traffic between sixxs and my server.
Does that not quite well tell you that something is wrong on your side?
Please check your local configuration and use the forums for resolving your issues, as there is nothing we can do here to help you.
Tunnel broken. webUI & AICCU disagree.
Shadow Hawkins on Monday, 25 April 2011 03:02:45Tried it with 4 different computers under two different routers which I've configuref myself to pass through AYIYA. What exactly did you configure? The way you write it you state that you >configured them to pass through AYIYA, thus all of them through that protocol, >which does not make sense.
Let's try it again:
The tunnel was working fine until 15.4.2011 when the traffic just died unprovoked.
It was tested in the same network with another computer (Debian Squeeze). It was also tested at a different site with two additional computers (another Ubuntu 10.04 & Windows 7). There was no data or even a successful ping to the POP IPV6 address. The tunnel had worked at both sites previously, hence the "configured" remark.
Testing method: I opened the tunnel with aiccu and tried to ping the ipv6 endpoint and ipv6.google.com.
The pings do not produce any kind of response after the dns resolve. Pings to the tunnel endpoint (ipv6) from outside the tunnel were also unsuccessful with the "Destination unreachable: No route" message. The IPV4 endpoint is reachable via ping.
It is behind a pfsense box with a NAT, but the port forwards are done. Port forwards? The whole point of AYIYA is that it does not need any port >forwards as it can cross normal NATs anyway (unless you filter ports in a >firewall that is).
Also, some people claim pfsense has IPv6 support, thus you might want to check >if that is causing you issues and/or terminate the tunnel there.
Yes, the ipv6 support for pfsense is pretty experimental at the moment.
And yes, I meant to say the ports are cleared in the firewall.
The ipv6 traceroute to fihel01.sixxs.net doesn't even finish. But where does it do go?
traceroute to fihel01.sixxs.net (2001:14b8:0:3007::6), 30 hops max, 80 byte packets
1 * * *
2 * * *
3 * * *
4 * * *
5 * * *
etc.
tcpdump doesn't show any traffic between sixxs and my server. Does that not quite well tell you that something is wrong on your side?
That was my first idea, but since anything hadn't changed in my setup and I think I've checked thoroughly the common errors mentioned in your faq, I deicided to turn to here.
Tunnel broken. webUI & AICCU disagree.
Shadow Hawkins on Wednesday, 21 September 2011 12:17:59
Well this is weird...
I stopped using the tunnel in april, since no one fixed the problem.
I decided to try it again in the beginning of september and it worked fine until this morning (21.09.2011) when it died again. No one did any modifications to any configurations at the time, so it really happened unprovoked. All the tests reveal the same data as last april. There's no ping to the ipv6 pop nor does the traceroute6 say anything. POP IPV4 responds correctly and so does the traceroute for it.
All the settings are also the same: I'm using latest AICCU, timesync is fine. It just DIED.
Tunnel broken. webUI & AICCU disagree.
As stated before, you'll have to provide quite a lot more details than "it just died".
Note also that AICCU only configures tunnels, it has a 'test' mode for checking if things works. It has no provision for verifying that everything is set up correctly.
Tunnel broken. webUI & AICCU disagree.
Shadow Hawkins on Wednesday, 21 September 2011 12:57:19
Well the situation is exactly the same as in the first two posts I posted here. All the tests give the same results. IPV4 POP endpoint responds to ping and traceroute completes as it should. IPV6 POP endpoint doesn't respond to ping from anywhere and the traceroute doesn't provide any steps. The actual POP (fihel01)is marked as functional and responds to ping and traceroute.
P.S. The problem seems to be fixed now. I didn't do anything so the problem has to be somewhere else than in my end.
Tunnel broken. webUI & AICCU disagree.
Shadow Hawkins on Wednesday, 12 October 2011 13:47:45
And again: all the conditions in the first post applies, the pop says it's up, but data isn't going anywhere. I've pinned the program down to the pop since everything is ok in my setup and aiccu is working correctly so it's not in your setup (well tic at least)
Tunnel broken. webUI & AICCU disagree.
I've pinned the program down to the pop
What do you mean with 'pinned the program'?
since everything is ok in my setup
As you are not providing those details that are currently present, we cannot help you with this.
and aiccu is working correctly
How do you determine that it is correctly working, what part 'works'?
so it's not in your setup (well tic at least)
If it is not in our setup, then why are you posting problem tickets to us?
As mentioned before, please peruse the forums for resolving your problem.
Tunnel broken. webUI & AICCU disagree.
Shadow Hawkins on Wednesday, 12 October 2011 15:49:19I've pinned the program down to the pop What do you mean with 'pinned the program'?
Sorry, meant to write "pinned the problem"
since everything is ok in my setup As you are not providing those details that are currently present, we cannot help you with this.
I've provided the details in my earlier message(s), please read it/them.
I've also stated many times that the conditions remain the same, so please read my replies.
and aiccu is working correctly How do you determine that it is correctly working, what part 'works'?
Well it starts without any error messages and your WebUI says the tunnel is up.
so it's not in your setup (well tic at least) If it is not in our setup, then why are you posting problem tickets to us?
Well where am I supposed to post the tickets concerning the pops? The link issues tracker puts me right here where I posted my problem.
As mentioned before, please peruse the forums for resolving your problem.
As I mentioned before: The forums and faq are useless on this particular problem.
Tunnel broken. webUI & AICCU disagree.
Well it starts without any error messages and your WebUI says the tunnel is up.
Which part of the WebUI do you mean that it claims that it is 'up'? The only portion that can state anything related to this is the ping graph which indicates that the tunnel pings properly. But there is no other notion of 'up' or 'configured' as the WebUI in it's current form does not ask the PoP.
As you mention 'without any error messages', what kind of error messages do you get when it does not work?
Well where am I supposed to post the tickets concerning the pops? The link issues tracker puts me right here where I posted my problem.
You state that it is the PoP, but we cannot find anything wrong with it.
As I mentioned before: The forums and faq are useless on this particular problem.
If you know what the problem is, then why don't you describe that?
Tunnel broken. webUI & AICCU disagree.
Shadow Hawkins on Wednesday, 12 October 2011 16:13:42Well it starts without any error messages and your WebUI says the tunnel is up. Which part of the WebUI do you mean that it claims that it is 'up'? The only >portion that can state anything related to this is the ping graph which >indicates that the tunnel pings properly. But there is no other notion of 'up' >or 'configured' as the WebUI in it's current form does not ask the PoP.
Well it reads enabled after my tunnel. Ping graph is clear after this morning when the problems started.
As you mention 'without any error messages', what kind of error messages do >you get when it does not work?
I've never had any error messages since aiccu has always worked. But I'd guess it would tell me if it couldn't open up the tunnel. I have tried this with verbose.
Well where am I supposed to post the tickets concerning the pops? The link >>issues tracker puts me right here where I posted my problem. You state that it is the PoP, but we cannot find anything wrong with it.
Well this is the third time the same problem has occured. It has always popped up totally unprovoked, there hasn't been any recent changes to hardware or routing or whatever in my setup. WebUI says the tunnel is enabled, I can ping and traceroute the tunnel endpoint ipv4-address, but ping to the pop endpoint ipv6 doesn't say anything nor does the traceroute to the pop ipv6. So let's call it and educated guess.
As I mentioned before: The forums and faq are useless on this particular problem. If you know what the problem is, then why don't you describe that?
I've described earlier, please read my messages.
Tunnel broken. webUI & AICCU disagree.
Well it reads enabled after my tunnel. Ping graph is clear after this morning when the problems started.
The 'enabled' there is because one can enable and disable tunnels. It thus solely show the current state.
The ping graph being 'clear' (I assume you mean no measurement points) indicates that your endpoint is not responding.
I've never had any error messages since aiccu has always worked. But I'd guess it would tell me if it couldn't open up the tunnel. I have tried this with verbose.
AICCU primarily only configures the tunnel, for that it can show a couple of errors, but not many.
For the actual operation of the tunnel there is little it can state, as it just sends a packet and receives it (for AYIYA that is). There the only problem that can be reported is if the OS tells AICCU that a packet could not be sent (eg when a local firewall rule does a REJECT).
Unfortunately for the rest there is not much it can diagnose.
It has always popped up totally unprovoked,
Well, that is the problem with problems, they don't come when one asks for them ;)
there hasn't been any recent changes to hardware or routing or whatever in my setup.
Please do realize that it might quite well be any of the myriads of hops in the network between your endpoint and the PoP.
WebUI says the tunnel is enabled,
As stated above, that is because one can enable/disable tunnels (and they can be in request and delete state etc). It further has no real effect, except that when disabled it won't get configured on the PoP either.
I can ping and traceroute the tunnel endpoint ipv4-address,
Thus we can assume that that portion of IPv4 works.
but ping to the pop endpoint ipv6 doesn't say anything
Which could mean that a hop between your endpoint and the PoP is dropping or mangling packets causing the PoP to never see them or accept them as valid.
This could be caused by a local or in-between firewall-alike rule and plenty of other problems.
Unfortunately there is nothing we can do about this as we cannot find anything wrong on the PoP and thus it is outside of our control.
Tunnel broken. webUI & AICCU disagree.
Shadow Hawkins on Thursday, 13 October 2011 08:42:39
Well now this is getting very interesting.
I've tested the problem at two different sites and the traceroute to the pop ipv4 tells me the first hop they have in common is the pop providers link to the finnish backbone, which I believe has to work so that other peoples tunnels at the same pop will work.
site 1 (this is where the tunnel is normally):
traceroute to 62.78.96.38 (62.78.96.38), 30 hops max, 60 byte packets
1 pfsense.metropolis (192.168.1.1) 0.179 ms 1.674 ms 1.670 ms
2 gw-620.tontut.fi (94.237.72.1) 2.221 ms 2.233 ms 2.224 ms
3 toas-gw1-x0101-her.tontut.fi (94.237.0.250) 2.210 ms 2.189 ms 2.194 ms
4 funet-tut6-x1130.tontut.fi (94.237.0.241) 77.588 ms 77.730 ms 77.716 ms
5 csc6-x0100-tut6.funet.fi (193.166.255.77) 5.025 ms 4.894 ms 5.010 ms
--> 6 dna.ficix1.ficix.fi (193.110.226.20) 5.050 ms 3.796 ms 3.761 ms
7 van1-er2.dnaip.fi (62.78.111.87) 4.633 ms 4.630 ms 4.765 ms
8 fihel01.sixxs.net (62.78.96.38) 4.369 ms 4.594 ms 4.590 ms
site 2 (my connection)
traceroute to 62.78.96.38 (62.78.96.38), 30 hops max, 60 byte packets
1 kammio.krypta.ath.cx (192.168.1.1) 0.236 ms 0.389 ms 0.378 ms
2 dsl-trebrasgw1-fe40fa00-1.dhcp.inet.fi (84.250.64.1) 18.585 ms 18.665 ms 18.656 ms
3 trecredger01-e-2-2.datanet.tele.fi (141.208.151.149) 18.892 ms 18.886 ms 18.881 ms
4 trecore3-o-0-1-0-0.datanet.tele.fi (141.208.204.25) 18.869 ms 19.253 ms 19.244 ms
5 141.208.25.125 (141.208.25.125) 30.650 ms 30.646 ms 30.636 ms
6 hkiasbr1-o-4.datanet.tele.fi (141.208.25.86) 56.615 ms 55.356 ms 55.314 ms
--> 7 dna.ficix1.ficix.fi (193.110.226.20) 22.795 ms 22.562 ms dna.ficix2.ficix.fi (193.110.224.20) 22.164 ms
8 van1-er2.dnaip.fi (62.78.111.87) 22.632 ms 22.618 ms esp2-tr1.dnaip.fi (62.78.107.99) 23.084 ms
9 fihel01.sixxs.net (62.78.96.38) 23.316 ms van1-er2.dnaip.fi (62.78.111.87) 22.797 ms fihel01.sixxs.net (62.78.96.38) 23.241 ms
Tunnel broken. webUI & AICCU disagree.
--> 6 dna.ficix1.ficix.fi (193.110.226.20) 5.050 ms 3.796 ms 3.761 ms
Your packet went over that hop.
--> 7 dna.ficix1.ficix.fi (193.110.226.20) 22.795 ms 22.562 ms dna.ficix2.ficix.fi (193.110.224.20) 22.164 ms
One of the packets went over the first one, another over the next. That happens sometimes when there are multiple paths to the same destination (thus from fihel01 to 84.250.64.1 and behind).
Nothing 'interesting' there, just happens, it is how the Internet works.
Obviously your packets in that test are arriving, thus nothing wrong there,
Tunnel broken. webUI & AICCU disagree.
Shadow Hawkins on Thursday, 13 October 2011 08:53:39
Umm... What I meant was that since the same problem applies to both sites and we've concluded that some hop along the way is causing the problem, It's quite interesting that the DNA's link to Ficix is the first hop common for both sites. Wouldn't this mean that the problem has to be in that hop or the few after it before the actual pop?
Tunnel broken. webUI & AICCU disagree.
You are concluding things.
Quite a number of ISPs connect multiple times, for redundancy, to Internet Exchanges, nothing odd there, that is how the Internet works.
Please a reminder that the Ticket system is a not a discussion forum. If you have an actual proper valid problem report, then please provide details. If you are trying to make assumptions, then please use the forum.
|
![[ch]](/s/countries/ch.gif)
on Sunday, 24 April 2011 15:48:07
Posting is only allowed when you are