|
I'm confused, is this a good place to ask?
![[nz]](/s/countries/nz.gif) Shadow Hawkins on Saturday, 02 March 2013 23:07:15
I've recently set up ipv6 at home, after setting it up last month at the community center. AYIYA / AICCU and RADVD on a firewall machine at both ends,
and everything seems to work. I have an ipv6-only webserver 'pi.fairfield.org.nz' (it's a raspberry pi) at the center, and now I can reach it from all the machines at home no problems. And I can ssh directly from the centre to machines at home too.
But a few days ago my wife wanted to look un www.health.govt.nz and couldn't. Firefox just times out eventually waiting for a response. The same site came up right away on my machine.
So I did a bit more digging. The site is dual-stack, ipv4 works, but traceroute6 to the same site falls down a deep hole about five hops from me. I expected something like that.
What I do not understand; it works on my machine but not hers. I'm presuming mine just falls back to ipv4 or something. But I can't figure out where or why there's a difference.
Both are using Firefox 19. Both are debian 6.0.6. Both use the same nameserver. both get the same results from a 'host' or a 'dig'. Both have the same /etc/gai.conf (default, everything comment out)
About the only difference; my machine is the gateway.
How do I get to the bottom of this mystery?
I'm confused, is this a good place to ask?
![[it]](/s/countries/it.gif) Shadow Hawkins on Sunday, 03 March 2013 12:35:11
is it possible that you drop icmpv6 packets (especially packet too big)?
If you play with firewall. If you do not know exactly what you are doing: never block icmpv6 in any direction
I'm confused, is this a good place to ask?
Shadow Hawkins on Sunday, 03 March 2013 22:52:22
I don't think I'm blocking anything I shouldn't. I did add firewall rules so that we're not sharing all of nfs, mysql and so on with the whole world but I think these apply equally to the tunnel endpoint and the rest of the network.
Copypasta from an example on the net plus I added interfaces and ports of my own. I'm pretty sure I'm allowing everything ICMPv6.
I just noticed I've got rules for br0 that we don't use (shouldn't matter) and eth1, eth2, ports for gaming servers open at home which only need to be open at the center. Also I don't have subnets for eth1 and eth2, waiting for some more ISK to get those.
Perhaps someone can look through these rules and advise me if any make no sense. I sort-of-know what most of them should do but I'm bit new to all this.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:Firewall - [0:0]
-A INPUT -j Firewall
-A FORWARD -j Firewall
-A Firewall -i lo -j ACCEPT
-A Firewall -i eth0 -j ACCEPT
-A Firewall -i eth1 -j ACCEPT
-A Firewall -i eth2 -j ACCEPT
-A Firewall -i br0 -j ACCEPT
-A Firewall -p icmpv6 -j ACCEPT
-A Firewall -p 50 -j ACCEPT
-A Firewall -p tcp -m tcp --dport 22 -j ACCEPT
-A Firewall -p tcp -m tcp --dport 80 -j ACCEPT
-A Firewall -p tcp -m tcp --dport 8081 -j ACCEPT
-A Firewall -p udp -m udp --dport 27960 -j ACCEPT
-A Firewall -p udp -m udp --dport 30730 -j ACCEPT
-A Firewall -p udp -m udp --dport 5154:5200 -j ACCEPT
-A Firewall -p udp -m udp --dport 5353 -d ff02::fb -j ACCEPT
-A Firewall -p udp -m udp --dport 631 -j ACCEPT
-A Firewall -p tcp -m tcp --dport 631 -j ACCEPT
-A Firewall -p udp -m udp --dport 32768:61000 -j ACCEPT
-A Firewall -p tcp -m tcp --dport 32768:61000 ! --syn -j ACCEPT
-A Firewall -j REJECT --reject-with icmp6-adm-prohibited
COMMIT
I'm confused, is this a good place to ask?
Your firewall seems to just opens random ports and will magically break all kinds of other things.
Note that copying and pasting a firewall ruleset does not work.
For most generic usage you likely want to do stateful firewalling so that a source from the inside of your network can contact an outside server but not the other way around.
See the wiki for a couple of examples, but as above, verbatim copying typically is not what you want.
I'm confused, is this a good place to ask?
Shadow Hawkins on Monday, 04 March 2013 18:02:59
I agree with Jeroen.
I would drop this two rules
-A Firewall -p udp -m udp --dport 32768:61000 -j ACCEPT
-A Firewall -p tcp -m tcp --dport 32768:61000 ! --syn -j ACCEPT
And insert
-A Firewall -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
somewhere at the beginning
also note that -p 50 probably does not work correctly. consider using -m esp instead.
Before the last rule you could also put the following two lines (I needed them for ike because somehow connection tracking did not work if already the first udp packet was too big, or more precisely, it dropped all fragments except the first. But if you do not need something like this, ignore this two lines)
-A Firewall -m frag --fragid 0:4294967295 --fragfirst -j DROP
-A Firewall -m frag --fragid 0:4294967295 -j ACCEPT
I'm confused, is this a good place to ask?
Shadow Hawkins on Monday, 04 March 2013 18:27:23
For the case you consider using the rules of my previous post:
I mostly do it like this:
first I put something like
-A Firewall -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
which more or less says, every already established connection (or related) may pass.
after that I put the "trusted" interfaces which can establish connections to every destination for example
-A Firewall -i eth0 -j ACCEPT
Afther that I put some services I allow passing thru
-A Firewall -m tcp -p tcp --dport 80 -j ACCEPT
which says EVERYONE can create a connection to ANY webserver on that host and bebind (if that is not what you want, -d 2001:.... is what you want, but if unsure read the manual ;) )
At the end I close with
-A Firewall -m frag --fragid 0:4294967295 --fragfirst -j DROP
-A Firewall -m frag --fragid 0:4294967295 -j ACCEPT
-A Firewall -j REJECT --reject-with icmp6-adm-prohibited
But again, if you don't know what a rule does, it may hurt you. So never copy rules from some source if you do not know what it does
Also Note: Using
-A INPUT -j Firewall
-A FORWARD -j Firewall
you state that the firewall rules for the host itself and the rule for the networks this host routes are the same.
Also note that in your template (and also when you apply my additions) you do not firewall outgoing packets originating form you pc (this is not a bad thing).
I'm confused, is this a good place to ask?
Please note that the best firewall ruleset is the one which is not there.
If you do not want services exposed or do not want things reachable then do not expose those service by not letting them listen, putting them in a VLAN and not routing that prefix, or filter off that explicit IP/port.
Coming up with a generic firewall will not help your case. If a process is able to listen randomly on a port, your firewall is already defeated, if a user is able to connect to a proxy that goes through your firewall you lost too.
Which also points to the state tracking. The moment you do state tracking you open for a resource exhaustion as that state can easily be filled up. As such, if you really need state tracking, do use -j CT --notrack" to make things that you often use and do not need that state non-tracked.
It of course always depends on what you are protecting against.
I'm confused, is this a good place to ask?
Shadow Hawkins on Monday, 04 March 2013 20:40:15
well, most of the time the "inner" site is trusted.
If not skype or sip would not work, since they do some tricks to connection tracking.
Securing a proxy server from outside (not exposing it) should be quite easy (do not forget ssh, it can also be a proxy).
As for the resource exhaustion. It's true that with something like syn flooding or also random udp flooding could exhause connection tracking. But to be honest: its a residental network, not a high volume network with thousands of pc's in it.
There is also a high probability that your slow mips powered router does connection tracking for ipv4 (since its implicitly needed for nat). At the end, that device, if someone wants to hurt you, is more interresting
Listening on a random port however should no be enough since your last rule says, everything not explicitly allowed from wan to lan is not allowed to pass thru your firewall.
I'm confused, is this a good place to ask?
Shadow Hawkins on Monday, 04 March 2013 23:21:40
The ports are not all random.
22 -- ssh .. I need this, all hosts run sshd
80 -- http .. I run apache2 on the server (also mysql-server which I don't want open to the world)
8081 -- motion (although turns out it does not support ipv6)
27960 -- we run an urban terror 4 game server
30730 -- tremulous gaming server
5154:5200 -- bzflag server
5353 and 631 I can't recall, but there was some reason for them being open in the howto I read.
32768:61000 I thought was for user apps that need to listen for incoming connections. I'm not sure about this.
Perhaps I should filter most of those to the one host that actually needs them, but I don't think it's a major risk anyhow.
It's all well and good saying that other processes should not open ports to the world, but reality is different. I am happy for user processes to open listening ports if they need to. I'm not happy to expose things like nfs, mysql and cups to the world. They may already only listen locally but I prefer to firewall as well. That's what firewalls are for?
And in answer to the original question apparently the site in question is up, but for some reason not reachable from my wife's machine even though it can be reached from mine. Also the following ipv6 test shows that her machine and mine can both reach ipv6 sites and are both preferring ipv6 as they should. http://ds.testmyipv6.com/
This is maddening.
I'm confused, is this a good place to ask?
8081 -- motion (although turns out it does not support ipv6)
Some (IPv) 6 years ago I submitted a patch for that already and from what I recall it should be in the released code too, as it states there "SVN r344" which was in May 2008, thus almost 5 years ago.
You are likely better off, because of authentication and other security concerns better off running it behind a HTTP proxy anyway.
27960 -- we run an urban terror 4 game server
Does that support IPv6? (It might, as that is the "Quake port number" and that has IPv6 support in at least the open source versions)
30730 -- tremulous gaming server 5154:5200 -- bzflag server
Does that support IPv6? If not, why open them?
5353
Multicast DNS sometimes uses that, but that is not something you share with the world.
631
Internet Print Protocol (IPP) or "CUPS" in the Linux world, thus if you want the rest of the world to print... then well share away ;)
32768:61000 I thought was for user apps that need to listen for incoming connections. I'm not sure about this.
And that is around HALF of the port space. Hence why I said "random", as you do not know what they are for.
If you want to have things "open", better just have everything open and close what you need closed instead of accidentally breaking something. Or at minimum use "LOG" to log what goes wrong so that you are aware (if you get to read/check the logs)
but I don't think it's a major risk anyhow.
All depends on the reason why you are firewalling in the first place.
but reality is different.
If you do not trust the code you run then run a host-local firewall, eg on Mac there is this great tool called Little Snitch, similar host-based firewalls exist for other operating systems.
I am happy for user processes to open listening ports if they need to.
Are you sure about that?
With all the 'updater' software out there that is showing a fingerprint and exposing my details to some organisation that does not need to know every x minutes where I am located and what versions of all my software I am running. Hence the "Little Snitch", processes only get to do what they need to and not much more. Next to them possibly downloading new 'updates' and installing random code of course.
Listening ports are of course worse, but when the code already runs one is too late to avoid any problems there.
I'm not happy to expose things like nfs, mysql and cups to the world.
See above for what the unknown-for-you port 631 does ;)
That's what firewalls are for?
Depends on your purpose, if it is a last-line defence for configuration mistakes then your purpose might be fulfilled.
This is maddening.
Add a LOG rule where your REJECT/DROP, next to that Wireshark is a great debugging tool.
I'm confused, is this a good place to ask?
Shadow Hawkins on Tuesday, 05 March 2013 01:37:32
OK, so I have cleaned up my rules a bit. No more cups (I noticed that after I sent my last message) and no more 8081. I don't know which of the same support ipv6, I think urban terror does. I'll experiment more on that.
The mystery still remains about this website though (which was my original request)
I can ping6 www.health.govt.nz
I can connect to http://www.health.govt.nz.ipv4.sixxs.org/ so clearly it can be reached by sixxs through ipv6
I can't view this one site directly from anything but the tunnel endpoint machine, even though I can view every other site (ipv6 and ipv4) that I try. Firefox just sits for a long time waiting, before telling me "The connection was reset"
And this is the same at the community center as at home.
I'm confused, is this a good place to ask?
I can ping6 www.health.govt.nz
try a 'tracepath6' that typically will show if there is an MTU hole somewhere.
And then the next step is to use Wireshark to see what packets fly and do not fly over the wire
I'm confused, is this a good place to ask?
Shadow Hawkins on Monday, 04 March 2013 23:23:47
Hmm, also I am trying to understand and take on board the suggestions in this thread. Sorry if the previous reply sounded ungrateful ;-)
I'm confused, is this a good place to ask?
Shadow Hawkins on Tuesday, 05 March 2013 17:34:21
To rule out some things I would try to disable the firewall completly
(=loading this ruleset, so you do not have to rmmod)
filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT
if it works correcty afterwards, its defenitly something in your firewall ruleset
if it does not work, well, i guess you will need wireshark...
I'm confused, is this a good place to ask?
Shadow Hawkins on Wednesday, 06 March 2013 10:10:56
Did that, same issue.
Figured it must be an MTU problem, changed the host MTU to 1280 and could reach the website no problem.
I've added 'AdvLinkMTU 1280;' to my radvd.conf and now it seems to work. Tested at both sites.
I'm still confused though. Why does only this site break? I'm guessing something to do with path mtu discovery and some host along the way filtering icmpv6?
I'm confused, is this a good place to ask?
I've added 'AdvLinkMTU 1280;' to my radvd.conf and now it seems to work. Tested at both sites.
As the gateway machine has an upstream of 1280, you should not have to do this.
When you use "AdvLinkMTU 1280" you are basically telling your local network that the MTU on your local network is 1280. And that is likely not true at all. As such you are impeding the local network.
Why does only this site break? I'm guessing something to do with path mtu discovery and some host along the way filtering icmpv6?
If they are filtering ICMPv6 then most people will have issues with connecting to their site.
$ tracepath6 www.health.govt.nz
1?: [LOCALHOST] 0.016ms pmtu 1500
1: ge-1-3-0.breda.ipv6.concepts-ict.net 0.627ms
1: ge-1-3-0.breda.ipv6.concepts-ict.net 0.667ms
2: 2001:838:5:11::2 0.770ms asymm 1
3: gi2-11.nikhef.ipv6.concepts-ict.net 3.171ms asymm 2
4: 20gigabitethernet1-3.core1.ams1.ipv6.he.net 2.197ms asymm 3
5: 10gigabitethernet1-4.core1.lon1.he.net 9.157ms asymm 4
6: 10gigabitethernet7-4.core1.nyc4.he.net 88.725ms asymm 5
7: 10gigabitethernet5-3.core1.lax1.he.net 141.775ms asymm 6
8: 2001:504:0:3:0:1:7746:1 137.387ms
9: ae1.cre1.nct.odyssey.net.nz 286.069ms asymm 8
10: 2400:4800:4003:8004::2 313.557ms asymm 8
11: TenGigabitEthernet0-1-0-536.wnmur-rt1.fx.net.nz 303.417ms asymm 9
12: TenGigabitEthernet0-1-0-537.wnwnr-rt1.fx.net.nz 303.947ms asymm 9
13: catalyst-it2.wix.net.nz 300.707ms asymm 10
14: no reply
15: no reply
They are definitely filtering ICMP there, thus very likely also ICMPv6 PTB and thus they are breaking things.
It is also breaking from other connections.
I'll forward this to some people who might know the right person to kick so that this gets resolved.
I'm confused, is this a good place to ask?
Shadow Hawkins on Wednesday, 06 March 2013 15:34:47
Can it be that you changed your mtu on your sixxs interface. it should be 1280 in your case. If that's the case, your machine should send a packet too bit to your machines in your network.
check with "ip l sh sixxs", where sixxs is the name of your sixxs tunnel device
something like this (in my case my mtu is 1480, but yours should be 1280)
1?: [LOCALHOST] 0.605ms pmtu 1500 <- this is the mtu of ethernet
1: cl-920.trn-01.it.sixxs.net 2.218ms
1: cl-920.trn-01.it.sixxs.net 2.140ms
2: cl-920.trn-01.it.sixxs.net 2.249ms pmtu 1480 < sent by MY router
2: gw-920.trn-01.it.sixxs.net 84.364ms
3: frejus.ITgate.net 74.095ms asymm 2
4: if-0-2-12.samba.edge.TRN.ipv6.ITgate.net 74.964ms asymm 3
5: if-9-4.scooby-monster.core.TRN.ipv6.ITgate.net 74.831ms asymm 4
...
if it doesn't, something goes really wrong. (now if you advertise 1280, already at the beginning there may be 1280)
I'm confused, is this a good place to ask?
If that's the case, your machine should send a packet too bit to your machines in your network.
The problem in this case really is that the website in question (www.health.govt.nz) is filtering ICMP.
Though the mismatch between the MTU and the MSS likely can cause other issues.
I'm confused, is this a good place to ask?
Shadow Hawkins on Thursday, 07 March 2013 05:43:46
I'm using an AYIYA tunnel so my MTU is 1280, I think.
What I understand is happening (although I might be totally wrong, but I ran into something similar with ipv4 about 15 years ago) is that the other end sends 1500 byte packets at me with the don't fragment bit sent, to probe the path MTU. sixxs can't send those packets through my tunnel without fragmenting them, so it sends back 'host unreachable' icmp replies.
This should tell health.govt.nz to send smaller packets until it's got the largest size that can pass through everything without needing to be fragmented, but in this case somewhere around catalyst-it the icmp replies are being dropped so health.govt.nz never finds out that it's packets aren't getting through.
What I don't understand is why changing the MTU at my end apparently solves this ;)
I'm confused, is this a good place to ask?
I'm using an AYIYA tunnel so my MTU is 1280, I think.
Per default it is 1280, but it can be changed to a higher value, see the FAQ for the details.
is that the other end sends 1500 byte packets at me with the don't fragment bit sent, to probe the path MTU.
There is no "DF" bit in IPv6. End-Hosts do fragmentation, routers just send "this packet is to big" ICMPv6 messages.
Thus what happens is that the webserver sends a 1500 byte packet, one of the intermediate routers will be "that does not fit on my next link", it then sends a ICMPv6 PTB with the size that will fit on that next link (minimum = 1280) to the sending host. The sending host then minimizes the packets to that size and resends the packet again.
sixxs can't send those packets through my tunnel without fragmenting them, so it sends back 'host unreachable' icmp replies.
Correct. And it can't fragment them as it is no endhost, it is a router.
What I don't understand is why changing the MTU at my end apparently solves this ;)
You changed the MTU of what where?
If you mean lowering the MTU on your local network, the reason why things start to "work" is because TCP includes the MSS option and that is set to the MTU (minus some size) of the link. As such in TCP it is negotiated what the size of the packets can be and then ICMPv6 is not needed.
While that causes TCP to 'work' that is not the right solution though, it does not solve UDP, SCTP or anything else that goes over that link.
I'm confused, is this a good place to ask?
Shadow Hawkins on Thursday, 07 March 2013 12:03:18
I lowered the MTU of all hosts on my LAN via changing the radvd config.
If you mean lowering the MTU on your local network, the reason why things start to "work" is because TCP includes the MSS option and that is set to the MTU (minus some size) of the link. As such in TCP it is negotiated what the size of the packets can be and then ICMPv6 is not needed.
While that causes TCP to 'work' that is not the right solution though, it does not solve UDP, SCTP or anything else that goes over that link.
Yes. I had a feeling that was the case, and only intended it to be a temporary 'hack' to further pinpoint exactly what's going wrong.
So are we sure that the dropping icmpv6 is happening at www.health.govt.nz or somewhere upstream? I'm presuming it's "whatever site is after catalyst" because that's where the tracepath6 vanishes, but there may be more hops after catalyst that we just can't see?
I'm confused, is this a good place to ask?
So are we sure that the dropping icmpv6 is happening at www.health.govt.nz or somewhere upstream?
One can never be sure, but it is very likely at least.
I'm presuming it's "whatever site is after catalyst" because that's where the tracepath6 vanishes, but there may be more hops after catalyst that we just can't see?
Most very likely some admin type who thinks that filtering ICMP is a good thing while not actually understanding what he/she is filtering... happens all the time.
|
![[ch]](/s/countries/ch.gif)
on Monday, 04 March 2013 08:11:01
Posting is only allowed when you are