|
AICCU integration removed from OpenWRT
![[de]](/s/countries/de.gif) Shadow Hawkins on Saturday, 14 July 2012 23:33:46
There has been some controversy between SixXS and OpenWRT development, see:
--> https://dev.openwrt.org/ticket/11592
As a consequence AICCU integration has been removed from OpenWRT which means that AICCU for OpenWRT now is available as binary only but without UCI integration. Starting with OpenWRT release r32666 the start/stop script /etc/init.d/aiccu has been removed from the AICCU package. See:
--> https://dev.openwrt.org/changeset/32666
As a consequence during OpenWRT boot AICCU cannot be started automatically any more and the UCI config file /etc/config/aiccu is useless.
Instead after any reboot (wanted or accidentally e. g. after a power break down) you have to login to the OpenWRT router and start AICCU by typing 'aiccu start'. The configuration now has to come from /etc/aiccu.conf as described here.
This is not a nice situation for the OpenWRT users of AICCU. The issue should be resolved by cooperation between SixXS and OpenWRT.
AICCU integration removed from OpenWRT
As you can read from the ticket we have attempted that, but they do not want to either understand the problem and/or do not want to spend their time on it, though they did spend a lot of time on adding code to AICCU which automatically restarted it even though this is something we explicitly state it should not do. This is gone though and thus avoids people from hammering the TIC server and being locked out from it.
AICCU integration removed from OpenWRT
![[nl]](/s/countries/nl.gif) Shadow Hawkins on Saturday, 21 July 2012 20:27:47
Sorry to read this, and sorry to read about the argument damaging a very fine platform to run AICCU on: cheap, low-power, ependable(!).
FWIW, just a few weeks ago I ran AICCU in an environment that lost power several times per day, every day (as well as the rest as the buildings a few blocks in the neighbourhood), and manually starting simply was not feasible (typically, grid power was restored after midnight and removed again at 8AM or thereabouts with several blackouts during daytime as well).
There is merit to automatic restarts. However, I do recognize the damage frequent re-connects do as well.
The NTP protocol knows the concept of a Kiss of Death. Once the server sends a KoD, the client isn't supposed to contact the server and can have a notion of 'do not re-contact, please fix!'
Even for an embedded environment, this surely seems to be a solution that can do the best of both worlds. Perhaps the newer windows code can be backported and some of it used in embedded environments?
As to the interpersonal things, there's nothing any of the forum members can say or do: both sides are vulunteers, both sides have strong opinions, things have escalated. However, for OpenWRT, for Sixxs, for aiccu, I think the current status is not optimal.
I'm hoping that as emotions cool down, things will get a second look.
Geert Jan
AICCU integration removed from OpenWRT
Once the server sends a KoD, the client isn't supposed to contact the server and can have a notion of 'do not re-contact, please fix!'
In SMTP and HTTP and various other protocols amongst which TIC, there is the concept of a permanent (5xx) and a transient (4xx) failure. Your KoD packet equals a 5xx. We are sending the TIC-equivalent of a 5xx's - permanent error.
Thus this is exactly what is being done already.
People apparently do not read those log files and thus never notice it though and thus try to override it by breaking the protocol and thus effectively ignoring the KoD packet.
Perhaps the newer windows code can be backported and some of it used in embedded environments?
There is no need for backports as the code has always been the same source tree with #ifdefs to take different paths for the various platforms. And this is the problem, the platforms, we need to test it on many platforms properly before we can safely release it to thousands of users. Broken software causes more email and we have enough of that already.
things have escalated.
If people would ask (info@sixxs.net email works, no openwrt person has contacted us there) and honor statements made, or heck the documentation delivered (AICCU's doc/README) then there would not have been any need for the discussion in the first place.
However, for OpenWRT, for Sixxs, for aiccu, I think the current status is not optimal.
The time problem on OpenWRT is a problem they can easily resolve. They want to resolve it differently and wrongly and thus as they cannot have it the way they want it they just drop it. Their choice, their userbase.
They have been given alternatives (setup clock before starting) but they apparently do not care about having a proper time for anything in their system.
I'm hoping that as emotions cool down, things will get a second look.
There is no 'emotion' from this side, that is also why we are not trying to come up with all kinds of unrelated comments in those threads like some of the commenters try to do.
We want it to work too, but if the 'developers' are uninformed and are not willing to read what is written, there is little we can do about it.
Users who cannot use SixXS on OpenWRT will just use the myriad of other WRT platforms out there that do do it properly or likely easier: they will fix their clock and then start AICCU as they are supposed to, search for rdate in this thread and you'll find an example of that.
AICCU integration removed from OpenWRT
Shadow Hawkins on Tuesday, 21 August 2012 11:59:56
Using Aiccu on OpenWRT vor several years, i never run into this issue. I'm very suprised using latest trunk on a new router these days.
Like Geert jan I'm hoping that emotions will cool down and all together have a deeper look into this issue without shifting the blame on others. As bugreports for debian and ubuntu shows, people create workarounds because they are not ever satisfied with the function of aiccu. so everyone should review his own opinion and look forward to how users could be satisfied with minimal cost for everyone. Even if all participants are volunteers IMHO none of you (and us) wants dissatisfied users which did not using the services.
As we can read there are restrictions on embedded hardware (like missing rtc) which can cause trouble using aiccu in dialup environment. Why is reconnection every 120s is a problem for TIC-Servers (heavy load?) ? E.g. PPP or wpa_supplicant always reconnects on failure and did not exit themself - IMHO better example than apache or bind.
AICCU integration removed from OpenWRT
Using Aiccu on OpenWRT vor several years, i never run into this issue.
Some people do, some do not. Not all environments are the same.
because they are not ever satisfied with the function of aiccu.
Because they expect it to do something it is not made for.
As we can read there are restrictions on embedded hardware (like missing rtc) which can cause trouble using aiccu in dialup environment.
If you know your time is an issue, fix your time and then start using it, just put a:
rdate ntp.openwrt.org
before starting AICCU and voila problem resolved. This is what I have had on my own openwrt box for years already and works like a charm.
Indeed, the dailup (or non-working connection) could mean you can't fetch time directly. Then in that case loop that construct and hit your own servers. But do not loop TIC, as then you are hitting somebody else who is not involved in that brokenness.
Note that if the host can't keep proper time at one point or another it will fail anyway and anything related to crypto that requires some form of synchronised time will break (read: amongst others SSL, but also DNSSEC etc and heartbeat and AYIYA as they use timestamps too).
Why is reconnection every 120s is a problem for TIC-Servers (heavy load?)
Because you are not the only user and because there are too many people who have over time misconfigured it and then hammered them.
And I refer to Macports bug 22872 which was opened there 3 years ago as they have an item that causes AICCU to be automatically and continuously restarted, like every second.
As the users do not read their log files, they never see what the problem is and thus never fix it, but the TIC server will keep on getting a constant flow of connection attempts.
Because of those kind of events the TIC server rate limits, to avoid it overloading the TIC server with useless requests so that it can serve real users who do behave.
E.g. PPP or wpa_supplicant always reconnects on failure and did not exit themself - IMHO better example than apache or bind.
Broken Time and no network connectivity at starts are cases of broken configuration. AICCU states that and exits.
PPP + wpa_supplicant are local things and if something is broken with the local configuration will only affect the local link and maybe the next hop.
AICCU uses a protocol called TIC that goes over the far route of the internet to fetch it's configuration details, it is not local. Rate limitting is thus a default protection against misconfigurations.
AICCU integration removed from OpenWRT
Shadow Hawkins on Tuesday, 21 August 2012 14:02:22And I refer to Macports bug 22872 which was opened there 3 years ago as they have an item that causes AICCU to be automatically and continuously restarted, like every second.
Maybe, if aiccu supports auto-reconnect (instead of exit) no one will write scripts which restarts aiccu every second...?
As the users do not read their log files, they never see what the problem is and thus never fix it, but the TIC server will keep on getting a constant flow of connection attempts.
Because of those kind of events the TIC server rate limits, to avoid it overloading the TIC server with useless requests so that it can serve real users who do behave.
If aiccu supports reconnect-by-rate-limit (e.g. mailservers with greylisting does) IMHO no one will restart it because waiting for auto-reconnect is a
working way.
Broken Time and no network connectivity at starts are cases of broken configuration. AICCU states that and exits.
Nope, that are cases of a adverse environment - aiccu should state this an try again later. (e.g. for the people who will have old-time-behavior just add "exit on any failure")
PPP + wpa_supplicant are local things and if something is broken with the local configuration will only affect the local link and maybe the next hop.
Ok, better: any other VPN-Software (e.g. OpenVPN)? We'll find an example that is equivalent to Aiccu
AICCU uses a protocol called TIC that goes over the far route of the internet to fetch it's configuration details, it is not local. Rate limitting is thus a default protection against misconfigurations.
But quitting aiccu is also not a good practice.
AICCU integration removed from OpenWRT
And I refer to Macports bug 22872 which was opened there 3 years ago as they have an item that causes AICCU to be automatically and continuously restarted, like every second. Maybe, if aiccu supports auto-reconnect (instead of exit) no one will write scripts which restarts aiccu every second...?
The auto-reconnect does not solve the problem that AICCU logs: broken time or broken connectivity.
> As the users do not read their log files, they never see what the problem is and thus never fix it, but the TIC server will keep on getting a constant flow of connection attempts. > Because of those kind of events the TIC server rate limits, to avoid it overloading the TIC server with useless requests so that it can serve real users who do behave. > If aiccu supports reconnect-by-rate-limit (e.g. mailservers with greylisting does) IMHO no one will restart it because waiting for auto-reconnect is a working way.
What exactly do you define as "reconnect-by-rate-limit" ?
Lots of mail server setups do exactly the same as what the TIC server does: if you hammer it, you are banned.
Broken Time and no network connectivity at starts are cases of broken configuration. AICCU states that and exits. Nope, that are cases of a adverse environment - aiccu should state this an try again later. (e.g. for the people who will have old-time-behavior just add "exit on any failure")
AICCU states it, and as I state in a lot of places people do not read the logs and do not resolve the problem. AICCU cannot resolve these problems for you, other tools or or the person has to do this.
Please realize: The major problem is that people do not read... or well, they are not aware of the issue as the logs are hidden somewhere.
One 'advantage' of the TIC server blocking repeat offenders is that they receive an email and sometimes people do read those and then they actually fix things and are happy that stuff works again.
PPP + wpa_supplicant are local things and if something is broken with the local configuration will only affect the local link and maybe the next hop. Ok, better: any other VPN-Software (e.g. OpenVPN)? We'll find an example that is equivalent to Aiccu
OpenVPN with broken configuration will also exit (eg if it's certifcates are broken or the config is broken).
Please realize that TIC is the configuration for AICCU, and thus no TIC means it can't work.
And the whole point of TIC is that people do not have to manually type in that configuration details to avoid them from filling in the wrong details.
AICCU uses a protocol called TIC that goes over the far route of the internet to fetch it's configuration details, it is not local. Rate limitting is thus a default protection against misconfigurations. But quitting aiccu is also not a good practice.
You state that, but no reasoning. You think it is bad practice, while every service that has a broken configuration exits with a nice log message telling what is wrong.
AICCU integration removed from OpenWRT
Shadow Hawkins on Tuesday, 21 August 2012 15:15:35Please realize: The major problem is that people do not read... or well, they are not aware of the issue as the logs are hidden somewhere.
For users, aiccu has a config (e.g. username/password, servername, interface, tunnel-ID) which will be setup by user. Afterwards aiccu does something (connecting TIC-Servers, getting extended config) which is not config-related for the user! If this did not work, aiccu should log that and retry it later - ipv6-tunnel is a service which should stay online as long as it is possible.
With "reconnect-by-rate-limit" i mean, maybe the TIC-Server responds: "Please come back in 15min" and aiccu-client recognizes it (i don't know if the protocol allows such a thing). People restarting aiccu themself to fast will get banned - that's ok then.
So there can be no other possibility of changing tunnel broker or building ugly workarounds? But is this the goal you'll achieve?
Outside OpenWRT/SIXXS-staff it seems so, that everyone is wrong and you're right. That's a real shame...
AICCU integration removed from OpenWRT
If this did not work, aiccu should log that and retry it later
Why should it retry if it is a known fact documented in a lot of places that people do not fix the problems?
- ipv6-tunnel is a service which should stay online as long as it is possible.
I fully agree. And you should realize that the moment that the system is configured correctly it will fetch the config from the TIC server once and keep on running and then should never exit.
With "reconnect-by-rate-limit" i mean, maybe the TIC-Server responds: "Please come back in 15min" and aiccu-client recognizes it
The moment you can talk to the TIC server it will give you the config and you never have to talk to it again.
Exceptions: broken client time, broken user/password.
As such the moment that the TIC server would be able to state something like the above, you would have your configuration and AICCU will not abort and keep running; or it will reject you and it will stop.
(i don't know if the protocol allows such a thing).
Well, it is quite obvious that you and a lot of other people who comment on this are not aware of all possibile scenarios and failure modes. Let alone that AICCU is not the only TIC client out there.
I've tried to provide these detailed answer already several times, am not going to keep on repeating them over and over and over again. Instead I'd rather spend time on actually pushing the next AICCU version out.
I also suggest that you read: this whitepaper on greylisting as then you'll realize how different these situations are.
In short: TIC errors are final (equivalent to SMTP 5xx) and greylisting errors are temporary (4xx).
People restarting aiccu themself to fast will get banned - that's ok then.
No, it is not okay, as they are affecting resources they should not be affecting.
So there can be no other possibility of changing tunnel broker or building ugly workarounds? But is this the goal you'll achieve?
I don't completely understand what you are trying to state here, but the solution for the generic problem are simple:
- Time issues: make sure you have synchronized time ("rdate ntp.openwrt.org" does this)
- Network issues: do not start aiccu (or heck ntpclient/rdate) before you actually have connectivity
Also note that another case is people who start/stop aiccu at every network event. This is also not needed.
Outside OpenWRT/SIXXS-staff it seems so, that everyone is wrong and you're right.
If you would completely understand what was happening, which obviously from the above you are not, you would understand where our statements are coming from and why we configure systems in certain ways.
That's a real shame...
The shame here is that people who do not understand what is going on try to force their opinion through even though they have no idea about the problem.
I'll btw add one really funny detail: AICCU asks for the time from the TIC server and then actually exits as the local time is wrong (thus the TIC server does not care about it that much), then again what else could it do, it is not like it could start sending hearttbeat or AYIYA packets as these will have the wrong time too, and likely you will be sending a lot more packets (like all those folks running pings all the time) than the few, in comparison, TIC server queries.
Now, if people would let this subject rest for a bit, then maybe I'll have enough time to actually do the new AICCU release and put it all out on github (as I noted in the Debian thread...) so that we can move past all this nonsense discussions and useless accusations...
AICCU integration removed from OpenWRT
Shadow Hawkins on Wednesday, 22 August 2012 13:43:16AICCU asks for the time from the TIC server and then actually exits as the local time is wrong (thus the TIC server does not care about it that much), then again what else could it do, it is not like it could start sending hearttbeat or AYIYA packets as these will have the wrong time too ...
It could "think" more positive that local time mismatch is not caused by a broken configuration but by a race condition as is the case with OpenWRT during boot time, where ntpd is started asynchronously and simply time had not been synchronized yet when AICCU starts. So the solution best implemented in AICCU could be to start a TIC session, do some "get unixtime" commands repeatedly with increasing time periods between the repetitions (10, 20, 40 sec. ...) until local time is okay (and normal AICCU starting procedure continues) or a limit of a maximum number of repetitions is reached (where AICCU exits because in this case the configuration seems to be broken).
From your expert view: Could this be an appropriate solution?
That's what the OpenWRT developers were doing (or trying do do) within their start script as a workaround because present AICCU version doesn't do it.
From an old log of my OpenWRT device:
... OpenWrt local7.err syslog: The clock is off by 1340833460 seconds, use NTP to sync it!
... OpenWrt local7.err syslog: Retrying TIC login in 10 seconds...
and 10 seconds later:
... OpenWrt local7.info syslog: Succesfully retrieved tunnel information for TXXXXX
... OpenWrt local7.info syslog: AICCU running as PID 1232
In my case, it was always only the second time TIC was successful.
Of course, OpenWRT developers could try to create a smart start script for AICCU, testing repeatedly (up to some time limit) the conditions that must be met so that AICCU is happy when it gets started finally or isn't started at all. I cant't tell how complicated this is, but I guess in case of OpenWRT it might be more complicated than just doing an 'rdate'. I guess it's better if you implement the proposed exponential backoff procedure within AICCU, instead others building workarounds for the problem.
Now, if people would let this subject rest for a bit, then maybe I'll have enough time to actually do the new AICCU release and put it all out on github...
Thank you very much in advance!
AICCU integration removed from OpenWRT
It could "think" more positive that local time mismatch is not caused by a broken configuration but by a race condition as is the case with OpenWRT during boot time, where ntpd is started asynchronously and simply time had not been synchronized yet when AICCU starts
AICCU does not know if the user is actually running NTP after you tried to use it.
It will also not make any assumption towards this.
Simple solution, as mentioned many times before: sync your time before attempting to do TIC.
So the solution best implemented ...
You think that is a good solution but it is far from.
A proper solution though could be to use the time the TIC server provides to bootstrap the time.
This is something that a lot of people do not want to do though. It will be an option in the next AICCU release. It does not solve the problem of clock-skew that might happen on a client computer though, thus NTP is still required for it to work long term. And AICCU is not a timesync daemon.
That's what the OpenWRT developers were doing (or trying do do) within their start script as a workaround because present AICCU version doesn't do it.
What you describe above and what they did is not the same at all, they where restarting connections in the mean time. Note also that TIC has already given you a proper answer, thus there is no reason to ask again and again and again and again.
In my case, it was always only the second time TIC was successful.
If you had synced your time before starting AICCU then it would have been a one shot.
I guess in case of OpenWRT it might be more complicated than just doing an 'rdate'
As I personally run OpenWRT box in a remote location that has power outages every once in a while due to the rural environment, I know that that resolves the problem perfectly fine. it did so also on DD-WRT that I ran on it.
I guess it's better if you implement the proposed exponential backoff procedure within AICCU,
It is not better as it does not resolve any problem at all. It just causes more connections to the TIC servers with the same result.
AICCU integration removed from OpenWRT
Shadow Hawkins on Wednesday, 22 August 2012 15:54:14
Thanks for your fast reply!
A proper solution though could be to use the time the TIC server provides to bootstrap the time. ... It will be an option in the next AICCU release. It does not solve the problem of clock-skew that might happen on a client computer though, thus NTP is still required for it to work long term.
That would be okay for me either. Main thing is that AICCU gets started automatically no matter how.
If you had synced your time before starting AICCU then it would have been a one shot.
Of course!
As I personally run OpenWRT box in a remote location that has power outages every once in a while due to the rural environment, I know that that (meaning: using 'rdate') resolves the problem perfectly fine.
Believe me, that's what I would really like to do! But how?
OpenWRT give me no solution for that (delivering only the naked AICCU binary) and I'm no expert in creating start script's for AICCU of my own. All I know is that it must be some script file <name> lying in /etc/init.d/, doing a chmod 755 for it to get it executable and finally doing a '/etc/init.d/<name> enable' to register it for getting started automatically at boot time (Am I right so far?).
So, if you could give me or others (and even the OpenWRT developers) the content of your perfectly working start script (here or in the Wiki) this would be of great use!
I still have to find out, which package to install in order to get 'rdate' command which is not contained in my OpenWRT version. Also the possibility to install additional packages is very limited for my device (http://wiki.openwrt.org/toh/tp-link/tl-mr3020) due to only about 660 KB space left for that purpose. So I hope this will not be a KO criterion.
AICCU integration removed from OpenWRT
As I personally run OpenWRT box in a remote location that has power outages every once in a while due to the rural environment, I know that that (meaning: using 'rdate') resolves the problem perfectly fine. Believe me, that's what I would really like to do! But how? OpenWRT give me no solution for that (delivering only the naked AICCU binary) and I'm no expert in creating start script's for AICCU of my own.
You mean: creating start scripts for OpenWRT.
Please note that the time issue is an issue on the OpenWRT platform because of missing RTC on the hardware they claim to support.
Thus first fix what is broken: the fact that your box does not set it's time properly. There are many many articles about this, amongst others Time Synchronisation on OpenWrt which is the first hit on google for this.
As for doing it just before AICCU, stuff it in the script that launches it, just before you call the AICCU binary to start it.
A start script can be as simple as:
#!/bin/sh
rdate ntp.openwrt.org
aiccu start
Of course, that is very simple you might need more details, ask your favourite platform about it and you will need to have the rdate binary on your host which might involve installing it.
So, if you could give me or others (and even the OpenWRT developers) the content of your perfectly working start script (here or in the Wiki) this would be of great use!
They know perfectly fine what to do, they like other people want to do it their way though.
But if you are not a developer or don't want to mess around with things there is a MUCH simpler way around all of this: get a AVM Fritz!Box. They work perfectly fine and don't require fiddling. (and no, that is not a way to say that one should not be able to run aiccu on OpenWRT, just that that is a way to get it without having to mess with anything; note that Fritz!Box use their own TIC client, not AICCU, but also that that platform has a working RTC unlike most platforms that OpenWRT runs on)
AICCU integration removed from OpenWRT
Shadow Hawkins on Wednesday, 22 August 2012 16:54:19
Thanks for your advice! I'll do some more reading about solving timesync problems (thanks for the link) and will test it.
But if you are not a developer or don't want to mess around with things there is a MUCH simpler way around all of this: get a AVM Fritz!Box. ...
Yes, I know and a second SixXS-tunnel I use in business works perfectly and stable for 1.5 years with the 7390 model I'm operating there. Unfortunately at home I only have a weak 2000 DSL access where a FRITZ!Box 7390 refuses to synchronize, hence the workaround solution with this small TP-Link box running OpenWRT, which so far works fine, too. The idea using this originated from here: http://www.heise.de/netze/artikel/Taschenrouter-als-IPv6-Verteiler-1440851.html (sorry German only), maybe you know this article. Hence, it was not too difficult to get it to work taking this article as a cooking recipe and reading some OpenWRT basics in addition.
AICCU integration removed from OpenWRT
Shadow Hawkins on Wednesday, 22 August 2012 20:32:30
Okay, here is what I did, and at a first glance it seems to work fine:
1. 'rdate' exists by default within OpenWRT, but it is called 'ntpd' now and is part of busybox, see http://wiki.openwrt.org/doc/uci/system.
2. I created a script file '/etc/init.d/aiccu' with the following contents:
#!/bin/sh /etc/rc.common
START=90
STOP=90
start () {
ntpd -n -q -p <ntp1> -p <ntp2> ...
/usr/sbin/aiccu start
}
stop () {
/usr/sbin/aiccu stop
}
where '-p <ntpX>' denotes a list of your favorite NTP servers to use.
3. chmod 755 /etc/init.d/aiccu
4. /etc/init.d/aiccu enable
5. reboot
Voila`: While booting AICCU comes up automatically without any problems. And doing '/etc/init.d/aiccu stop' or '/etc/init.d/aiccu start' manually works as expected, except that for '/etc/init.d/aiccu start' you have to take into account another ntpd call that is not needed, when system is up, because time is already synchronized.
6. Add a line '/etc/init.d/aiccu' to '/etc/sysupgrade.conf' so that the script file will survive the next firmware update.
Maybe the START and STOP values should be modified, but that's only fine tuning.
AICCU integration removed from OpenWRT
1. 'rdate' exists by default within OpenWRT, but it is called 'ntpd' now and is part of busybox, see http://wiki.openwrt.org/doc/uci/system.
Busybox does multi-function binaries for space-savings, which is a good thing.
Voila`: While booting AICCU comes up automatically without any problems.
Yes, looks good. As I have been stating in various places, fixing the time issue by actually asking for the time is the right way to go ;) And a simple 'rdate' or similar resolves this quite fine.
No changes needed inside AICCU to fix something that is broken on the system, which also affects the rest of the system.
Nevertheless, I have also made some changes this morning to AICCU which should avoid this situation a bit and handle things differently. Still without doing reconnects to the TIC server btw. Unfortunately replacing noc.sixxs.net due to hardware failures will destroy my available time a bit but I hope to be able to do more tests and release the code as a new version and on github somewhere next week.
AICCU integration removed from OpenWRT
Shadow Hawkins on Wednesday, 22 August 2012 22:25:12Yes, looks good. As I have been stating in various places, fixing the time issue by actually asking for the time is the right way to go ;) And a simple 'rdate' or similar resolves this quite fine.
But there's another race-condition if aiccu starts while pppoe-session is not established - especially after power lost and dsl have to resync. so rdate fails and aiccu starts (with wrong clock ;)) So the solution is not final imho.
Nevertheless, I have also made some changes this morning to AICCU which should avoid this situation a bit and handle things differently. Still without doing reconnects to the TIC server btw.
Checking Min-time is a nice trick to prevent connecting to TIC-Server with non-synced clock. *thumbs up* and thanks
AICCU integration removed from OpenWRT
But there's another race-condition if aiccu starts while pppoe-session is not established - especially after power lost and dsl have to resync. so rdate fails and aiccu starts (with wrong clock ;)) So the solution is not final imho.
If you do not have connectivity yet, then do not start things, as well, you have no connectivity yet.
Checking Min-time is a nice trick to prevent connecting to TIC-Server with non-synced clock. *thumbs up* and thanks
And it possibly also solves the no-connectivity issue you mention above as it waits for the time to be right.
It retries max a 100 times before giving up, first 10 tries it waits 10 seconds and then for 11-100 it waits that 5 times the amount of seconds thus from the code:
* We wait max:
* 1 - 10: (10 * 10)
* 11 - 100: (5 * ((100 * (100+1) / 2) - (10 * (10+1) / 2)))
* = 25075 seconds which is almost 7 minutes
* this should give enough time to connect to the Internet
Note that the original I did this morning did not wait if fixtime was set, the updated version now waits for 5x the retry count in seconds, otherwise it would only in total wait for max 2 minutes, the 7 is better.
AICCU integration removed from OpenWRT
![[ch]](/s/countries/ch.gif) Shadow Hawkins on Thursday, 23 August 2012 18:21:39
I had a similar problem with an ALIX-Box (without HW clock) and Debian.
The main problem is that the boot scripts that fetch the time with ntpdate
are written to do this asynchronously. This is usually a good thing
because you do not want to delay booting just because the time is not
correct yet, but is a bad thing of course for applications like aiccu
which depend on the correct time.
My solution was:
. move /etc/network/if-up.d/ntpdate to /etc/ppp/ip-up.d
. modify the ntpdate script so that the call to ntpdate is NOT
backgrounded any more
. modify the ntpdate script so that return code is evaluated and
retry to get the time after 5 seconds of sleep. If ntpdate
cannot find the correct time, this will loop forever.
==> when the ntpdate script finishes, I KNOW that time is set correctly
. move the aiccu start script to /etc/ppp/ip-up.d/z00_aiccu
so that it is called after the ntpdate script is done
The only thing you have to be careful that NOTHING is started after
aiccu that you depend on and could run without IPv6...
AICCU integration removed from OpenWRT
 ![[cz]](/s/countries/cz.gif) Shadow Hawkins on Saturday, 06 October 2012 17:56:40
I have been trying to look for the source code (here http://www.sixxs.net/tools/aiccu/), but found only version called "current" (http://www.sixxs.net/archive/sixxs/aiccu/unix/aiccu_current.tar.gz), which seems to correspond to version 2007.01.15.
It seems the source has evolved since then as you mention in your previous post (https://www.sixxs.net/tools/aiccu/changelog). Is it possible to get the current source?
AICCU integration removed from OpenWRT
![[pt]](/s/countries/pt.gif) Shadow Hawkins on Monday, 25 March 2013 01:10:20
Marcel Pennewiss wrote:
With "reconnect-by-rate-limit" i mean, maybe the TIC-Server responds: "Please come back in 15min"
I have the solution for this:
the tic picks the client time, make the difference to the current time+15 min (or something like that) and tells the client to try again when that time is reached.
So if a router is years in the past, it will not try again until is fixed... if its a small difference, after 15min maybe ntp have already fixed it.
After some attempts (maybe 3) with the same difference (more or less), do the existent process, shutdown the daemon with the warning about the time (use a special error code for that, so that scripts can warn the user or fix the time)
AICCU integration removed from OpenWRT
Daniel Mota Leite wrote:
Marcel Pennewiss wrote:
There is absolutely no reason for doing any of this.
When the TIC server indicates an error there is nothing that the client can do to resolve this.
The user/administrator of the client has to resolve the problem.
With "reconnect-by-rate-limit" i mean, maybe the TIC-Server responds: "Please come back in 15min"
I have the solution for this:
the tic picks the client time, make the difference to the current time+15 min (or something like that) and tells the client to try again when that time is reached.
AICCU integration removed from OpenWRT
Shadow Hawkins on Tuesday, 23 October 2012 20:27:10
Maybee I'm completely wrong. But with i.e start priority 51 for aiccu (see System startup initsscript) everything should work fine.
See my wiki http://wiki.openwrt.org/doc/uci/aiccu?s[]=aiccu
I'm on OpenWRT Beta with several devices (OpenWRT) DIR-600 and RT-16N for a long time and I've never seen that issues before.
(i.e. RT-16N is on ATTITUDE ADJUSTMENT (bleeding edge, r29617) )
Unfortunately for OpenWRT I'm currently on OpenBSD (see my current configuration on the wiki page) with a dedciated WRAP device in my LAN.
Maybee the difference is that I ordered a 6in4 instead of an AYIYA tunnel.
And honestly the main reason why I'm now on OpenBSD is the I enjoy pf instead of any other packetfilter.
cu
Thomas
AICCU integration removed from OpenWRT
Shadow Hawkins on Wednesday, 24 October 2012 14:09:57But with i.e start priority 51 for aiccu (see System startup initsscript) everything should work fine.
See my wiki http://wiki.openwrt.org/doc/uci/aiccu?s[]=aiccu
I am using ATTITUDE ADJUSTMENT (Bleeding Edge, r33482) on a TP-Link TL-MR3020 which has no real time clock. So following your wiki and (due to changeset 32666) starting aiccu via /etc/rc.local I can't see how you can influence any start priority except by manipulating the start priorities of the relevant start scripts such like /etc/init.d/done (which calls /etc/rc.local and has start priority 95) and /etc/init.d/sysntpd (which starts ntpd and has start priority 98). Without doing so an aiccu start via rc.local results in the expected failure:
logread:
...
Jan 1 01:00:26 OpenWrt local7.err syslog: The clock is off by 1351082047 seconds, use NTP to sync it!
Jan 1 01:00:26 OpenWrt local7.err syslog: Couldn't retrieve first tunnel for the above reason, aborting
...
So my start script as shown above in this thread circumvents this problem by calling "ntpd -n -q -p ..." before starting aiccu.
Andreas
AICCU integration removed from OpenWRT
Shadow Hawkins on Tuesday, 30 October 2012 10:59:04
Could you please check if ntpd is enabled.
If so, please disable ntpd.
Time is set via ntpclient in /etc/hotplug.d
root@OpenWrt:/etc/hotplug.d/iface# ls
00-netstate 05-radvd 10-routes 20-ntpclient
05-3g 10-qos 20-firewall 25-ddns
That support for aiccu is disappear from OpenWRT seems to be an misunderstanding.
Do we know how many accounts are locked by this TIC issue?
Bevore I do some upgrades with OpenWRT I do revers engineering on my ALIX to get the scripts and configurations back to my local needs.
Thomas
Good idea to play with FreeBSD and OpenBSD. But main reason for me is that I like the simple charm of pf (OpenBSD packet filter).
AICCU integration removed from OpenWRT
If so, please disable ntpd.
ntpclient is one-shot, you call it at start of a machine to sync the time directly.
ntpd is required as it keeps updating the time continuously, thus do not disable it, it needs to run after it, this as a lot of computers do not keep a proper local time, hence why ntpd exists in the first place.
That support for aiccu is disappear from OpenWRT seems to be an misunderstanding.
It 'disappeared' because the OpenWRT folks did not want to resolve the problem, as if they would have resolved it the way that we indicated to them they would have to admit that they where wrong, and some people just do not want to do that.
Do we know how many accounts are locked by this TIC issue?
Only very few (<10 I think) have been locked as they do not respond or act at all. Most people fix their problems when they get notified, but by then they have been hammering on the TIC servers a lot which causes them to be automatically be blocked from the TIC server, their accounts remain okay though.
AICCU integration removed from OpenWRT
Shadow Hawkins on Sunday, 11 November 2012 19:18:56
See my message regarding issues I have with my internet supplier.
I just change my router on the fly (no internet for approx. 15 minutes).
My ipv6 box is a WRAP running OpenBSD 5.1 changed the lan ip adress during my maintenance work and the tunnel was back immediately.
Ok, with Cable if have the same ipv4 internet adress for ages.
Tom
AICCU integration removed from OpenWRT
![[br]](/s/countries/br.gif) Shadow Hawkins on Thursday, 15 November 2012 10:55:43
Hello guys,
As a constant user of aiccu with openwrt, this issue become a real problem for me. So I'm working on a middle solution that might be OK for both parts.
I might be mistaken but I see some issues:
1) aiccu cannot start before internet connection or time sync (that happens on openwrt)
2) openwrt patches aiccu to reconnect frequently to solve issue #1
3) tic servers got too much connections attempt from misconfigured clients
4) the suggestion of forcing a synchronization before starting aiccu does not solve the problem as internet connection might be not available at that time
Thinking on all these issues, I wrote a scheme that might solve the problem for everyone. I would like to get the feedback if this provide a good solution for both parts.
First, I did not modified the aiccu binary. This seems to be a good start.
So, if any connection/configuration problem happens, aiccu just quits and logs the problem.
The startup script was modified to differ two offline states:
1) aiccu is stopped because nobody started it or someone stopped it
2) aiccu is stopped because it quit some some unknown reason
The identification of each state is done checking if there is an aiccu configuration file (that is regenerated on each start). When it is explicitly stopped, the startup script removes the configuration file. If the file exists
but aiccu is not running, it stopped by some unknown reason.
I also added a new startup script command: retry. It is equals to "start" but only runs if if identifies the configuration file (state #2). So, if called on state #1, nothing happens, while when called on state #2, it starts again aiccu. If "retry" is called when aiccu is already running, even though, it calls "aiccu start". However, this results in nothing but a log message "Already running instance HUP'ed, exiting". Actually, it does not "HUP" the running aiccu (kill -1) as it sends a "kill -0" to the running process which, according to "man 2 kill", is silently ignored. "kill -HUP" would really kill the aiccu as it is an untreated signal.
Now the question is when to reconnect the aiccu. As time-based is out of question, I configure to call this aiccu "retry" when a interface gets up or when ntp updates the clock. These events occurs only a couple of times and they might not lead to any TIC DOS attack.
So, a normal running situation would be:
1) openwrt starts booting
2) openwrt brings up wan
2.1) aiccu is not started as it was still stopped
3) openwrt starts ntp
4) ntp updates clock
4.1) aiccu is not started as it was still stopped
5) openwrt starts aiccu
5.1) aiccu start and connects without problem
If wan gets some time to get up:
1) openwrt starts booting
2) openwrt starts ntp
3) openwrt starts aiccu
4.1) aiccu tries to connect but quits as there is no internet connection
4) openwrt brings up wan
4.1) aiccu tries to connect but quits as clock is not updated
5) ntp updates clock
5.1) aiccu start and connects without problem
The only situation that TIC might suffer from a connection storm would be if aiccu have some configuration problem and wan or ntp has also some configuration problem that results in constantly events. I think that this situation might be rare and can be an accepted problem.
I have already posted a patch to add needed modification on ntp scripts that is useful even without aiccu.
link
I'll wait for this feedback before posting the aiccu init script patch.
AICCU integration removed from OpenWRT
Shadow Hawkins on Thursday, 15 November 2012 10:52:00
Hi Luiz,
I don't know WRT, but I have solved the same problem on debian (running on ALIX).
I don't understand your statement:
4) the suggestion of forcing a synchronization before starting aiccu does not solve the problem as internet connection might be not available at that time
I fully disagree. Exactly this should be the solution. When you don't have internet connectivity, you can neither synchronize the time nor can you establish the aiccu tunnel.
When you actually have internet connectivity, you can both synchronize the time and aiccu.
Your solution sounds too complicated to me (you are introducing more dependencies, heuristics) and it is not perfect (i.e. there are still situations that end up with hammering the TIC servers).
Why not simply run ntpdate and check the return code?
Nicolas
AICCU integration removed from OpenWRT
Why not simply run ntpdate and check the return code?
And that is indeed what I have been trying to explain to a lot of people over time already, that if they simply run a ntp single shot script that they then know that they have connectivity and the right time.
Unfortunately, there are people who do not want to understand that there are simple solutions to some problems :(
That said though, I have made some changes to AICCU which will just keep trying without doing any connects till the time it has looks sane, when that has happened it can then still take the time from the TIC service when it connects to force the local time to it. Now only to find some time to do a full test run on all platforms that AICCU supports to see if it works properly everywhere.
AICCU integration removed from OpenWRT
Shadow Hawkins on Thursday, 15 November 2012 11:57:56
I was also once thinking about
... till the time it has looks sane ...
But I was unable to come up with more than "time is later than I am coding this
right now" and "time is before 31.12.2030" but that sounded so arbitrary to
me that I preferred to investigate on the return code of ntpdate ;-)
AICCU integration removed from OpenWRT
But I was unable to come up with more than "time is later than I am coding this right now" and "time is before 31.12.2030" but that sounded so arbitrary to me that I preferred to investigate on the return code of ntpdate ;-)
And that is indeed why the problem cannot be fully solved inside AICCU unless it becomes an NTP client...
The bigger point there is that a lot of SSL/TLS and other crypto also depend on the right time, especially for certificate expiry verification (which is a date) and one reason why AICCU also needs the time: replay attacks.
Other tools have the same issue, thus solving it only in one tool is not the right location.
AICCU integration removed from OpenWRT
Instead of coming up with a convoluted way to keep on starting AICCU, why don't you resolve the REAL problem: no connectivity and unsynchronized time?
The only situation that TIC might suffer from a connection storm would be if aiccu have some configuration problem and wan or ntp has also some configuration problem that results in constantly events. I think that this situation might be rare and can be an accepted problem.
Hammering on an external service is NEVER "acceptable".
Please actually resolve your real problems instead.
AICCU integration removed from OpenWRT
Shadow Hawkins on Saturday, 17 November 2012 16:02:42
Hello Nicolas and Jeroen, thanks for your feedback.
The 4th issue is still a problem because at the time you run the "single shot ntp", you might still be offline. So, what should the system do? Give up aiccu completely and wait for some router admin to restore it? Or should I loop ntp single shot until it correctly synchronize? Wouldn't it possibly harm the ntp server the same way that TIC got harmed with reconnections?
What I did was to start again aiccu when ntp got a correct synchronization. Not that much complicated. If the aiccu connection was already up, it does not bother the running aiccu session. The same happens when an interface goes up or down. Possibly, the new link or even the disconnection of an existing link might fix the connectivity problem. Again, if the aiccu connection was already up, this process does not bother the running session.
I guess a real solution must consider many scenarios. OpenWRT might run in machines with RTC and where NTP is not desired. NTP server might be accessible before internet is good to go. Consider that NTP might be in LAN or aiccu traffic might only be authorized over an VPN connection. So, "successful NTP synchronization" is not equivalent to "capable of connecting a TIC server" and neither "not successful NTP synchronization" means the opposite.
I agree that this is an heuristic approach (isn't the NTP single shoot too?). However, no complete solution would work as the amount of possible problem is not treatable. Network protocol since the beginning works very well until today with many retries/heuristic.
Jeroen, the only completely secure way to not "hammer on an external service" is to not have an external service. It is beyond our powers to control every variable in this process. If a router connects to a TIC but, because of a HW problem, reboots every minute, it would be equals to the impact of retrying to reconnect every minute. The number of possibilities is not treatable. If this case of HW problem is rare and can be accepted, so it is the case I wrote when it would need multiples problems together in order to reconnect constantly the TIC server. The best solution for this case would be a server side protection like blocking the IP address/user that have a harming behavior. Even the biggest service providers use this approach. Try to use "google search cache" a thousands of time from a single IP to see what it would do.
AICCU integration removed from OpenWRT
Shadow Hawkins on Sunday, 18 November 2012 11:32:04
Hi Luiz,
The 4th issue is still a problem because at the time you run the "single shot ntp", you might still be offline. So, what should the system do? Give up aiccu completely and wait for some router admin to restore it? Or should I loop ntp single shot until it correctly synchronize?
The 2nd. As aiccu absolutely needs correct time, simply loop (maybe every 5 seconds or so) with the "single shot ntp" until you have a successful synchronisation. Only then start aiccu once. Of course, you have to take care that you do not block other startup jobs while waiting for the correct time.
Wouldn't it possibly harm the ntp server the same way that TIC got harmed with reconnections?
In the worst case (i.e. no sync possible), you will send out an ntp package every 5 seconds, but this package does never hit the ntp server (because if it would, you could synchronize).
You argue that systems can have the correct time even without an NTP connection. But this would only be the case for machines connected directly to a Stratum 0 source like an atomic clock or a DCF77 receiver (a rather rare case). Here a simple config parameter HAS_HARDWARE_TIME (or similar) could be used to disable the looping with the ntp client.
You also argue that successful ntp lookup is not equivalent to have connectivity to the TIC server. To be very prudent, you could implement a ping loop (e.g. every 5 seconds) to the TIC server that only terminates after a successful ping. Replying to a ping uses much less resources on the server side.
When both conditions are met (correct time AND connectivity to TIC server), try to start aiccu once. If this still does not work, you need an administrator to look into it.
Jeroen, correct me if I'm wrong...
AICCU integration removed from OpenWRT
Shadow Hawkins on Wednesday, 21 November 2012 03:54:12
Hello Nicolas,
I think that "wait until ntp gets a correct synchronization" is better than "loop until ntp gets a correct synchronization". That's why a ntp loop is not desired. Also, if not implemented as a background service, it will block the startup jobs. So, it would need more complex code.
About system without NTP, it is just the case of any device that has RTC and the admin adjusted the clock manually. aiccu is a little flexible about time synchronization and even manual configured clocks can satisfy its needs. If the device has some cellphone interface, it can also use the telecom time. The same for GPS. Strictly using ntp command is not ideal.
Also, (un)successful ping is not equals to (not) connectivity to aiccu service on that TIC. The only way for sure is to really try to connect. Imagine the situation where there is a wan external firewall would block all connections, including aiccu and only allow a VPN connection and icmp. ping would give the wrong status that connectivity is ok. It is just a hypothetical case but many other situation could result in the need of a special "IGNORE_XXX" attribute or even a brand-new startup script. Imagine multiwan, pppoe, vpn, etc...
I have an new alternative solution that might satisfy all needs. This will result in a "single-shoot aiccu start" and only after the needed WAN interface is up and NTP was synchronized (or explicitly ignored). Here is some kind-of pseudo-code:
--
aiccu startup script:
start():
AICCU_START_WANTED=true
if not NTP_SYNCHRONIZED and not AICCU_IGNORE_NTP; then
waiting for ntp to get sync
exit
end
if AICCU_WAN_INTERFACE != up; then
waiting for wan xxx to get up
exit
end
/usr/bin/aiccu start
AICCU_START_WANTED=false
And on ntp synchronization or ifup events:
ntp():
if AICCU_START_WANTED
start aiccu
end
ifup():
if INTERFACE == AICCU_WAN_INTERFACE and AICCU_START_WANTED
start aiccu
end
--
If both "NTP_SYNCHRONIZED" and "AICCU_WAN_INTERFACE == up" gets satisfied, and "AICCU_START_WANTED", the aiccu launch is tried. If it fails, it would never try again until the user restart manually the aiccu service.
I still think that the previous solution would also do no harm to the TIC but I would not disagree with the TOS. The good thing of retrying to connect on interface/ntp events is that it would be a more general solution for complex situation when ntp is not mandatory or it is provided by a different interface than aiccu would use.
As a suggestion, I think that aiccu could retry to connect indefinitely until it gets a TIC response (with increasing generous sleep time between them). Only after a negative TIC response it would die. Currently, aiccu just dies if there is no TIC connection on startup. So, if I have not connection to TIC before startup and launch, I have no aiccu. This could happen if TIC is down or wan is still not completely operational. If wan goes down and up after TIC connection was established of TIC service restart, I have no problem. Retrying until TIC responds the first time is more network-service-like (just like ntpd). This modification would simplify much of the startup code I need and improve aiccu robustness, while keeping the single-shot connection to TIC, at least the one it answered.
|
on Sunday, 15 July 2012 09:28:14
Posting is only allowed when you are