|
Ticket ID: SIXXS #14247149 Ticket Status: Resolved PoP: ausyd01 - OCCAID Inc. (Sydney)
ausyd01 routing loop.
![[au]](/s/countries/au.gif) Shadow Hawkins on Friday, 11 September 2015 01:26:59
It appears that there are IPv4 issues with ausyd01. When doing a traceroute to the broker, it loops inside overthewire.net.au:
My tunnel is T169021.
root@core:/home/vyos# traceroute 66.186.206.66
traceroute to 66.186.206.66 (66.186.206.66), 30 hops max, 60 byte packets
1 10.100.21.1 (10.100.21.1) 0.243 ms 0.185 ms 0.276 ms
2 10.8.1.1 (10.8.1.1) 1.498 ms 1.355 ms 1.681 ms
3 203.170.8.17 (203.170.8.17) 9.646 ms 9.615 ms 9.627 ms
4 203-174-183-105.bri.static-ipl.aapt.com.au (203.174.183.105) 11.496 ms 11.438 ms 10.813 ms
5 59-100-210-198.bri.static-ipl.aapt.com.au (59.100.210.198) 26.010 ms 25.691 ms 25.631 ms
6 te4-2.pe1.100wic.bne.core.overthewire.net.au (180.214.88.136) 41.450 ms 32.132 ms 32.107 ms
7 ibr01.bris01.occaid.net (203.55.174.73) 23.104 ms 23.103 ms 23.035 ms
8 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 193.307 ms 193.615 ms 193.591 ms
9 ibr01.bris01.occaid.net (203.55.174.73) 23.417 ms 23.476 ms 23.451 ms
10 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 194.352 ms 214.665 ms 214.598 ms
11 ibr01.bris01.occaid.net (203.55.174.73) 24.085 ms 24.064 ms 24.034 ms
12 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 214.643 ms 214.596 ms 25.814 ms
13 ibr01.bris01.occaid.net (203.55.174.73) 23.917 ms 23.900 ms 23.871 ms
14 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 25.847 ms 25.813 ms 25.900 ms
15 ibr01.bris01.occaid.net (203.55.174.73) 23.988 ms 23.964 ms 23.742 ms
16 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 33.282 ms 33.752 ms 33.722 ms
17 ibr01.bris01.occaid.net (203.55.174.73) 24.481 ms 24.751 ms 24.419 ms
18 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 24.182 ms 24.618 ms 24.561 ms
19 ibr01.bris01.occaid.net (203.55.174.73) 24.533 ms 24.495 ms 24.674 ms
.. and so on. Oddly enough, the tunnel broker sees my tunnel as up, even though I'm not able to contact it.
ausyd01 routing loop.
 Shadow Hawkins on Friday, 11 September 2015 01:33:41
Rob Thomas wrote:
It appears that there are IPv4 issues with ausyd01. When doing a traceroute to the broker, it loops inside overthewire.net.au:
My tunnel is T169021.
Same issue for me, got an e-mail saying my tunnel was down... Started to look into why and seems I'm suffering the same routing loop.
1 10.20.22.149 (10.20.22.149) 16.787 ms 17.703 ms 25.242 ms
2 203-219-5-237static.tpgi.com.au (203.219.5.237) 41.186 ms 40.533 ms 39.633 ms
3 nme-apt-bur-crt2-te-gi-0-7-0-3.tpgi.com.au (203.219.106.17) 40.649 ms 42.904 ms 44.074 ms
4 203-219-106-205.tpgi.com.au (203.219.106.205) 48.509 ms 41.855 ms 47.852 ms
5 203-219-107-114.static.tpgi.com.au (203.219.107.114) 60.403 ms 40.074 ms 80.348 ms
6 bri-pow-que-wgw1-be-10.tpgi.com.au (203.219.166.144) 42.247 ms 41.287 ms 65.566 ms
7 203-219-107-210.static.tpgi.com.au (203.219.107.210) 40.173 ms 48.403 ms 42.108 ms
8 po9.bqueedist02.aapt.net.au (202.10.14.40) 42.510 ms
po8.bqueedist01.aapt.net.au (202.10.14.38) 40.699 ms 41.024 ms
9 1-1-1.bqueeinte01.aapt.net.au (203.131.62.23) 48.928 ms 42.494 ms
5-1-1.bqueeinte01.aapt.net.au (203.131.62.25) 39.778 ms
10 59-100-210-198.bri.static-ipl.aapt.com.au (59.100.210.198) 45.368 ms 44.525 ms 45.090 ms
11 te4-2.pe1.100wic.bne.core.overthewire.net.au (180.214.88.136) 42.834 ms 40.521 ms 41.450 ms
12 ibr01.bris01.occaid.net (203.55.174.73) 41.600 ms 50.198 ms 42.619 ms
13 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 41.570 ms 42.709 ms 42.292 ms
14 ibr01.bris01.occaid.net (203.55.174.73) 46.040 ms 41.549 ms 79.108 ms
15 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 44.999 ms 43.604 ms 42.855 ms
16 ibr01.bris01.occaid.net (203.55.174.73) 42.822 ms 41.738 ms 43.949 ms
17 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 43.462 ms 54.281 ms 43.255 ms
18 ibr01.bris01.occaid.net (203.55.174.73) 42.094 ms 44.328 ms 43.290 ms
19 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 43.968 ms 43.301 ms 43.329 ms
20 ibr01.bris01.occaid.net (203.55.174.73) 45.675 ms 42.110 ms 42.884 ms
21 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 44.591 ms 43.290 ms 42.900 ms
22 ibr01.bris01.occaid.net (203.55.174.73) 43.652 ms 43.135 ms 42.283 ms
23 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 43.951 ms 43.653 ms 43.610 ms
24 ibr01.bris01.occaid.net (203.55.174.73) 42.931 ms 43.582 ms 43.497 ms
25 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 45.699 ms 45.996 ms 82.917 ms
26 ibr01.bris01.occaid.net (203.55.174.73) 58.118 ms 83.835 ms 69.215 ms
27 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 104.643 ms 83.397 ms 97.390 ms
28 ibr01.bris01.occaid.net (203.55.174.73) 82.767 ms 84.158 ms 68.973 ms
29 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 56.439 ms 56.485 ms 61.052 ms
30 ibr01.bris01.occaid.net (203.55.174.73) 44.929 ms 43.791 ms 45.545 ms
root@core:/home/vyos# traceroute 66.186.206.66
traceroute to 66.186.206.66 (66.186.206.66), 30 hops max, 60 byte packets
1 10.100.21.1 (10.100.21.1) 0.243 ms 0.185 ms 0.276 ms
2 10.8.1.1 (10.8.1.1) 1.498 ms 1.355 ms 1.681 ms
3 203.170.8.17 (203.170.8.17) 9.646 ms 9.615 ms 9.627 ms
4 203-174-183-105.bri.static-ipl.aapt.com.au (203.174.183.105) 11.496 ms 11.438 ms 10.813 ms
5 59-100-210-198.bri.static-ipl.aapt.com.au (59.100.210.198) 26.010 ms 25.691 ms 25.631 ms
6 te4-2.pe1.100wic.bne.core.overthewire.net.au (180.214.88.136) 41.450 ms 32.132 ms 32.107 ms
7 ibr01.bris01.occaid.net (203.55.174.73) 23.104 ms 23.103 ms 23.035 ms
8 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 193.307 ms 193.615 ms 193.591 ms
9 ibr01.bris01.occaid.net (203.55.174.73) 23.417 ms 23.476 ms 23.451 ms
10 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 194.352 ms 214.665 ms 214.598 ms
11 ibr01.bris01.occaid.net (203.55.174.73) 24.085 ms 24.064 ms 24.034 ms
12 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 214.643 ms 214.596 ms 25.814 ms
13 ibr01.bris01.occaid.net (203.55.174.73) 23.917 ms 23.900 ms 23.871 ms
14 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 25.847 ms 25.813 ms 25.900 ms
15 ibr01.bris01.occaid.net (203.55.174.73) 23.988 ms 23.964 ms 23.742 ms
16 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 33.282 ms 33.752 ms 33.722 ms
17 ibr01.bris01.occaid.net (203.55.174.73) 24.481 ms 24.751 ms 24.419 ms
18 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 24.182 ms 24.618 ms 24.561 ms
19 ibr01.bris01.occaid.net (203.55.174.73) 24.533 ms 24.495 ms 24.674 ms
.. and so on. Oddly enough, the tunnel broker sees my tunnel as up, even though I'm not able to contact it.
ausyd01 routing loop.
Shadow Hawkins on Friday, 11 September 2015 02:52:17
Jonathan Morcom wrote:
Same issue for me, got an e-mail saying my tunnel was down... Started to look into why and seems I'm suffering the same routing loop.
Phew. Glad it's not just me. I was just starting to build up a nice chunk of ISK, too!
ausyd01 routing loop.
Shadow Hawkins on Friday, 11 September 2015 03:06:22
Same here for me. I also got the email saying my tunnel was down.
Here is my Traceroute
traceroute to 66.186.206.66 (66.186.206.66), 30 hops max, 60 byte packets
1 * * *
2 * * *
3 10.1.71.33 (10.1.71.33) 28.810 ms 29.953 ms 32.215 ms
4 as30071.qld.ix.asn.au (218.100.76.9) 33.915 ms 34.403 ms 37.302 ms
5 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 39.548 ms 39.430 ms 41.543 ms
6 ibr01.bris01.occaid.net (203.55.174.73) 41.634 ms 29.207 ms 28.716 ms
7 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 31.865 ms 29.919 ms 31.916 ms
8 ibr01.bris01.occaid.net (203.55.174.73) 30.039 ms 29.316 ms 31.520 ms
9 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 30.778 ms 34.287 ms 31.720 ms
10 ibr01.bris01.occaid.net (203.55.174.73) 30.594 ms 30.090 ms 31.385 ms
11 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 35.590 ms 34.234 ms 32.783 ms
12 ibr01.bris01.occaid.net (203.55.174.73) 30.840 ms 30.808 ms 29.326 ms
13 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 33.776 ms 33.441 ms 44.972 ms
14 ibr01.bris01.occaid.net (203.55.174.73) 31.494 ms 31.642 ms 32.798 ms
15 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 36.112 ms 36.290 ms 35.323 ms
16 ibr01.bris01.occaid.net (203.55.174.73) 35.321 ms 31.768 ms 31.927 ms
17 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 33.772 ms 33.357 ms 32.318 ms
18 ibr01.bris01.occaid.net (203.55.174.73) 30.724 ms 33.532 ms 33.130 ms
19 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 35.089 ms * 32.253 ms
20 ibr01.bris01.occaid.net (203.55.174.73) 33.291 ms 32.378 ms 31.290 ms
21 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 34.145 ms * 31.891 ms
22 ibr01.bris01.occaid.net (203.55.174.73) 33.606 ms 33.118 ms 32.447 ms
23 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 32.896 ms 31.971 ms *
24 ibr01.bris01.occaid.net (203.55.174.73) 34.443 ms 31.496 ms 32.455 ms
25 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 32.575 ms 32.896 ms *
26 ibr01.bris01.occaid.net (203.55.174.73) 31.927 ms 33.728 ms 33.570 ms
27 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 32.761 ms * 33.528 ms
28 ibr01.bris01.occaid.net (203.55.174.73) 33.230 ms 33.257 ms 33.485 ms
29 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 33.762 ms 33.419 ms 32.929 ms
30 ibr01.bris01.occaid.net (203.55.174.73) 32.398 ms 34.165 ms 33.369 ms
Rob Thomas wrote:
It appears that there are IPv4 issues with ausyd01. When doing a traceroute to the broker, it loops inside overthewire.net.au:
My tunnel is T169021.
root@core:/home/vyos# traceroute 66.186.206.66
traceroute to 66.186.206.66 (66.186.206.66), 30 hops max, 60 byte packets
1 10.100.21.1 (10.100.21.1) 0.243 ms 0.185 ms 0.276 ms
2 10.8.1.1 (10.8.1.1) 1.498 ms 1.355 ms 1.681 ms
3 203.170.8.17 (203.170.8.17) 9.646 ms 9.615 ms 9.627 ms
4 203-174-183-105.bri.static-ipl.aapt.com.au (203.174.183.105) 11.496 ms 11.438 ms 10.813 ms
5 59-100-210-198.bri.static-ipl.aapt.com.au (59.100.210.198) 26.010 ms 25.691 ms 25.631 ms
6 te4-2.pe1.100wic.bne.core.overthewire.net.au (180.214.88.136) 41.450 ms 32.132 ms 32.107 ms
7 ibr01.bris01.occaid.net (203.55.174.73) 23.104 ms 23.103 ms 23.035 ms
8 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 193.307 ms 193.615 ms 193.591 ms
9 ibr01.bris01.occaid.net (203.55.174.73) 23.417 ms 23.476 ms 23.451 ms
10 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 194.352 ms 214.665 ms 214.598 ms
11 ibr01.bris01.occaid.net (203.55.174.73) 24.085 ms 24.064 ms 24.034 ms
12 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 214.643 ms 214.596 ms 25.814 ms
13 ibr01.bris01.occaid.net (203.55.174.73) 23.917 ms 23.900 ms 23.871 ms
14 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 25.847 ms 25.813 ms 25.900 ms
15 ibr01.bris01.occaid.net (203.55.174.73) 23.988 ms 23.964 ms 23.742 ms
16 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 33.282 ms 33.752 ms 33.722 ms
17 ibr01.bris01.occaid.net (203.55.174.73) 24.481 ms 24.751 ms 24.419 ms
18 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 24.182 ms 24.618 ms 24.561 ms
19 ibr01.bris01.occaid.net (203.55.174.73) 24.533 ms 24.495 ms 24.674 ms
.. and so on. Oddly enough, the tunnel broker sees my tunnel as up, even though I'm not able to contact it.
State change: resolved
The state of this ticket has been changed to resolved
ausyd01 routing loop.
Oddly enough, the tunnel broker sees my tunnel as up, even though I'm not able to contact it.
That is because our monitoring comes in from a different path, that seemed to work.
Our monitoring also primarily checks that IPv6 works and that there are good percentage of tunnels online to determine if a PoP is 'up' or 'down.
It does not look anymore to have a routing loop at the moment, thus marking as resolved for the time being.
Credits lost have been refunded.
ausyd01 routing loop.
Shadow Hawkins on Friday, 11 September 2015 07:13:26
Jeroen Massar wrote:
It does not look anymore to have a routing loop at the moment, thus marking as resolved for the time being.
Sadly, it is still down:
root@ipv6 firewall]# traceroute 66.186.206.66
traceroute to 66.186.206.66 (66.186.206.66), 30 hops max, 60 byte packets
1 203.25.118.1 (203.25.118.1) 0.366 ms 0.280 ms 0.271 ms
2 192.168.5.248 (192.168.5.248) 1.724 ms 1.918 ms 3.464 ms
3 10.100.21.1 (10.100.21.1) 4.134 ms 3.965 ms 4.031 ms
4 10.8.1.1 (10.8.1.1) 4.535 ms 4.663 ms 4.637 ms
5 203.170.8.17 (203.170.8.17) 12.541 ms 12.846 ms 13.014 ms
6 203-174-183-105.bri.static-ipl.aapt.com.au (203.174.183.105) 21.604 ms 14.050 ms 14.499 ms
7 59-100-210-198.bri.static-ipl.aapt.com.au (59.100.210.198) 31.401 ms 28.447 ms 28.468 ms
8 te4-2.pe1.100wic.bne.core.overthewire.net.au (180.214.88.136) 27.192 ms 24.291 ms 27.198 ms
9 ibr01.bris01.occaid.net (203.55.174.73) 27.522 ms 27.306 ms 27.634 ms
10 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 27.812 ms 27.728 ms 27.839 ms
11 ibr01.bris01.occaid.net (203.55.174.73) 27.395 ms 27.413 ms 25.584 ms
12 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 28.505 ms 28.486 ms 26.254 ms
13 ibr01.bris01.occaid.net (203.55.174.73) 26.038 ms 26.439 ms 26.351 ms
14 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 28.944 ms 29.195 ms 29.235 ms
15 ibr01.bris01.occaid.net (203.55.174.73) 28.845 ms 29.100 ms 28.674 ms
16 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 29.108 ms 29.128 ms 29.097 ms
17 ibr01.bris01.occaid.net (203.55.174.73) 26.467 ms 26.485 ms 26.514 ms
18 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 36.630 ms 36.395 ms 36.437 ms
19 ibr01.bris01.occaid.net (203.55.174.73) 29.218 ms 29.051 ms 28.904 ms
20 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 36.123 ms 36.300 ms 36.270 ms
21 ibr01.bris01.occaid.net (203.55.174.73) 28.739 ms 28.840 ms 28.936 ms
22 ge8-28.232.pe1.100wic.bne.core.overthewire.net.au (203.55.174.78) 36.076 ms 29.717 ms 29.117 ms
^C
ausyd01 routing loop.
Then it looks like that is an issue on that specific interface. I'll kick it forward for resolution.
PING ausyd01.sixxs.net (66.186.206.66) 56(84) bytes of data.
64 bytes from 66.186.206.66: icmp_seq=1 ttl=50 time=314 ms
12 bundle-101.cor02.lax01.ca.VOCUS.net (114.31.199.50) 312.076 ms 311.996 ms 311.865 ms
13 bundle-100.cor01.lax01.ca.VOCUS.net (114.31.199.48) 312.397 ms 312.210 ms 312.311 ms
14 bundle-102.cor01.sjc01.ca.VOCUS.net (114.31.199.63) 314.125 ms 316.898 ms 316.888 ms
15 bundle-100.cor02.sjc01.ca.VOCUS.net (114.31.199.59) 314.275 ms 314.243 ms 317.127 ms
16 bundle-100.cor01.sjc01.ca.VOCUS.net (114.31.199.58) 316.862 ms 316.860 ms ten-0-2-0-2.cor01.syd04.nsw.VOCUS.net.au (114.31.199.45) 311.828 ms
17 ten-0-2-0-3.cor01.syd04.nsw.VOCUS.net.au (114.31.199.28) 311.999 ms ten-0-1-0-5.cor01.syd04.nsw.VOCUS.net.au (114.31.199.84) 314.499 ms ten-2-2-0.bdr03.syd03.nsw.VOCUS.net.au (114.31.192.55) 316.867 ms
18 ten-0-1-0.bdr03.syd03.nsw.VOCUS.net.au (114.31.192.35) 314.901 ms as30071.cust.bdr02.syd03.nsw.VOCUS.net.au (114.31.200.93) 314.104 ms ten-0-1-0.bdr03.syd03.nsw.VOCUS.net.au (114.31.192.35) 311.786 ms
19 66.186.206.66 (66.186.206.66) 314.056 ms as30071.cust.bdr02.syd03.nsw.VOCUS.net.au (114.31.200.93) 314.019 ms 314.245 ms
State change: confirmed
The state of this ticket has been changed to confirmed
State change: resolved
The state of this ticket has been changed to resolved
ausyd01 routing loop.
The transport between Sydney and Brisbane was down, and it is now back up again.
|
![[ch]](/s/countries/ch.gif)
on Friday, 11 September 2015 06:39:14
Posting is only allowed when you are