Tuesday, August 2, 2011

Some thoughts

Since the network is evolving very slowly, I'd write in a post some thoughts.

I noticed a problem with IGS. The mechanism of dissemination of the available gateways should, with a use as low as possible of traffic, make every node aware at any moment of the address of the n closest gateways. What happens instead is that when a node which was a gateway gets removed, this information is often lost. The result is that the nodes that use those gateways will have soon a not reliable link to the Internet.

A workaround could be to set the number of used gateways to 1, and then make sure that the closest gateway is always on.

It is just a ugly workaround. The bug is critical and should be immediately dealt with.
Nevertheless I'm not going to dedicate time to that immediately. At the moment I am very busy with the porting of the code to Vala, and AFAIK nobody (except me) is using the current version.

Another thought. Based on the first tests conducted, it seems that reaching the Internet with different IP addresses is not a problem for most applications. Obviously except that a TCP connection that has been initiated throughout a certain gateway has to be carried on on that gateway. This is guaranteed by netsukuku.

Nevertheless in theory some applications might be bothered that they reach the Internet with an address that changes very often. We cannot do much for them. The nodes that want to use those applications will have to set the number of used gateways to 1, and live with it.

Thursday, July 28, 2011

The network grows

A friend of mine and I have at last made a not-so-trivial wireless link.
Our houses are within about 200 meters, and from the roofs there is optimal line of sight.
First, each of us passed a network cable from the apartment to the roof. Then we made the wireless link by means of two routers of Ubiquity.

As you know, at the moment the netsukuku daemon cannot run on such a router.
Nevertheless, in order to connect to a netsukuku network a node must run the daemon and be directly connected to at least another node on which the daemon runs.
This gives to me an occasion to describe a possible solution to that situation.

We elected two computers to be always on (or at least when we need the network to function).
Each one of these is directly connected to the router on the roof.
These two computers have an IP address in the class The same goes for the routers.
Some static routes have been saved to the computers and routers to make the two computers able to communicate.

At this point, we need an application of VPN able to emulate a physical link at the layer 2 level.
I chose tinc. It's easy to install on Ubuntu. The configurations that one needs to do in order to realize a "virtual hub" between two computers is very simple.
When tinc is in action, the two computers see a new NIC, named for instance tinc0.
Via that NIC it's as if the two computer were directly connected.

After these operations, the computer of my friend is a node of the network in its own right.
The addition of the new node is not problematic for the routing. After all the topology of the network deployed til now is not complex at all.
On the other hand an interesting aspect is that now there are 2 nodes with a direct access to the Internet. A situation where the IGS feature hasn't been well tested yet.

As I said previously on this blog, the current IGS implementation makes a node able to share its own link to the Internet OR to use the links that are shared by other nodes. But not both.
So, I made some tests from a node that uses the shared links (IGS_MODE='USE').

luca@luca-laptop:~$ ip route list
       nexthop via  dev ntk-to-inet-0 weight 3 onlink
       nexthop via  dev ntk-to-inet-1 weight 1 onlink

The browser (firefox) seems to behave normally.
The download manager DownThemAll gave some satisfactions. Putting few files in the queue I reached from time to time a rate of 800 KBs, about twice the rate I normally get with my ISP.
I used Skype for some time, without noticing any anomaly.
I haven't had enough time to test aMule's performance, yet.

Stay tuned.

Saturday, February 19, 2011

IGS: first steps of the baby

The code responsible for the Internet sharing is committed to the repository. It is working just fine with my use cases.

The dislocation and settings of nodes in my house slightly changed.
Firstly, I replaced the Dell mini 9 with the EeePC. This machine (luca-eeepc) is now providing the ad-hoc wireless network (essid luca-ntk) which legacy DHCP clients can connect to.
Secondly, the server in the garage hasn't got anymore a direct route towards the Internet router over the roof. It will use the IGS mechanism to reach the Internet. As such it will be able, I hope, in the future to exploit more than one ISP connections. As I said before, it is problematic to exploit the IGS as a user and have a direct connection at the same time.

Configuration changes

The nodes luca-desktop and luca-laptop now have the file
with a line that says:
whilst the node luca-eeepc has in the same file the line:

The commands that I issue on each node before launching the netsukuku daemon change a bit.
I note them down, for the sake of completeness. Feel free to skip this section.
Just note a little change to the route of the EeePC towards its DHCP clients. Previously I forgot to add the parameter 'src' and this was confusing the DNS resolver of the DHCP clients.

on the EeePC

# disable network management
sudo stop network-manager
sudo killall dhclient
sudo ip route flush table main
sudo ip address flush dev eth0
sudo ip address flush dev wlan0
# connection to the WISP via eth0
sudo ip link set eth0 up
sudo ip address add dev eth0
sudo ip route add default via dev eth0 src
# activate adhoc wifi
sudo ip link set wlan0 down
sudo iw dev wlan0 del
sleep 1
sudo iw phy phy0 interface add adhoc0 type ibss
sleep 1
sudo ip link set adhoc0 up
sleep 1
# join network luca-ntk
sudo iw dev adhoc0 ibss join luca-ntk 2462 fixed-freq

# prepare to use ANDNA
sudo tee /etc/resolv.conf <<EOF >/dev/null
# launch the daemon
cd ~/netsukuku/pyntk
sudo /opt/stackless/bin/python2.6 ntkd -i eth0 adhoc0 -vvvv 2>&1 | sudo tee -a /var/log/netsukuku.all.log >/dev/null
) &

# prepare and launch the dhcp server
sudo ip addr add dev adhoc0
sudo ip route add dev adhoc0 src
sudo iptables -t nat -A POSTROUTING -s \! -d -j MASQUERADE
sudo dhcpd3 adhoc0

on the server

# stop networkManager
sudo stop network-manager
sudo killall dhclient
sudo ip route flush table main
sudo ip address flush dev eth0
sleep 1

# prepare to use ANDNA
sudo tee /etc/resolv.conf <<EOF >/dev/null
# launch the daemon
cd ~/netsukuku/pyntk
sudo /opt/stackless/bin/python2.6 ntkd -i eth0 -vvvv 2>&1 | sudo tee -a /var/log/netsukuku.all.log >/dev/null
) &

on the laptop

# disable network management
sudo stop network-manager
sudo killall dhclient
sudo ip route flush table main
sudo ip address flush dev eth0
sudo ip address flush dev wlan0
# activate adhoc wifi
sudo ip link set wlan0 down
sudo iw dev wlan0 del
sleep 1
sudo iw phy phy0 interface add adhoc0 type ibss
sleep 1
sudo ip link set adhoc0 up
sleep 1
# join network luca-ntk
sudo iw dev adhoc0 ibss join luca-ntk 2462 fixed-freq

# prepare to use ANDNA
sudo tee /etc/resolv.conf <<EOF >/dev/null
# launch the daemon
cd ~/netsukuku/pyntk
sudo /opt/stackless/bin/python2.6 ntkd -i eth0 adhoc0 -vvvv 2>&1 | sudo tee -a /var/log/netsukuku.all.log >/dev/null &


What I achieve is that every node in the network can reach the Internet. Obviously the sweet thing is that most nodes don't need to have a direct connection to an ISP for that, and don't need even to be configured to have a knowledge, before-hand, of which node will provide to them a path to reach the Internet.
And of course, each node can reach all the nodes in netsukuku with all the benefits of its dynamic routing protocol. E.g. today I was in front of my laptop in the kitchen, connected via wireless. I started a copy of a file from the server in the garage and the "copy dialog" was telling me that at a rate of 1.7 MB/s it would have finished in about 10 minutes. Without canceling the copy I moved the laptop to a position where I have a plug, and I plugged the laptop into the wired LAN. In about 5 seconds I saw the rate going up to 70 MB/s. That's what I call plug'n'play! When the copy finished I unplugged the laptop again, I tried to reload an Internet web page and it was already up and running through the wireless link.

Now let me show you some technical details of what rules have been injected in the nodes.

on the EeePC

This is the node that has IGS_MODE='SHARE'.

luca@luca-eeepc:~$ sudo iptables -n -t nat -L POSTROUTING
target     prot opt source               destination        
MASQUERADE  all  --      !     
MASQUERADE  all  --          !         

This command shows that the packets with source in and destination elsewhere have to be masqueraded. This is the same that has been done manually to NAT the legacy DHCP clients, which obtain an address in

luca@luca-eeepc:~$ ip link show tunl0
5: tunl0: <NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN 
    link/ipip brd

This command shows that the node is accepting to tunnel traffic via IPIP.

on the laptop

This is a node that has IGS_MODE='USE'.

luca@luca-laptop:~$ ip link show ntk-to-inet-0
17: ntk-to-inet-0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN 
    link/ipip peer

This command shows that the node has created a virtual interface through which it will tunnel traffic via IPIP to the node (the current address of luca-eeepc)

luca@luca-laptop:~$ ip route list table main
default via dev ntk-to-inet-0 

This command shows that the default route in the table main (which is examined only for addresses not found in the table ntk, that is for Internet addresses) is through the IPIP tunnel cited above.
In the current situation this would suffice. But we'd love to be able to exploit more than one tunnel. In this case we want to make sure that TCP connections initiated with one tunnel will always use that one. How is this achieved?

luca@luca-laptop:~$ sudo iptables -t mangle -n -v -L POSTROUTING
Chain POSTROUTING (policy ACCEPT 728K packets, 89M bytes)
 pkts bytes target     prot opt in     out     source               destination        
 2477  157K CONNMARK   all  --  *      ntk-to-inet-0             ctstate NEW CONNMARK xset 0x63/0xffffffff 

This command shows that every new TCP connection going out through the tunneled interface ntk-to-inet-0 has to be marked 99 (0x63).

luca@luca-laptop:~$ sudo iptables -t mangle -n -v -L OUTPUT
Chain OUTPUT (policy ACCEPT 665K packets, 72M bytes)
 pkts bytes target     prot opt in     out     source               destination        
31801 4678K CONNMARK   all  --  *      *             ctstate RELATED,ESTABLISHED CONNMARK restore 

This command shows that every TCP connection has to restore the mark it was created with (if any).

luca@luca-laptop:~$ ip rule
0:    from all lookup local 
32762:    from all fwmark 0xc6 lookup 198 
32763:    from all fwmark 0xc7 lookup 199 
32764:    from all fwmark 0x63 lookup 99 
32765:    from all lookup ntk 
32766:    from all lookup main 
32767:    from all lookup default 

This command shows that for packets marked with 99 we have to examine table 99.

luca@luca-laptop:~$ ip route list table 99
default via dev ntk-to-inet-0  src onlink 

This command shows that these packets will be sent through the tunnel ntk-to-inet-0.

Thus, when the default route in the table main will present many tunneled interfaces, we will rest assured that TCP connections initiated with one tunnel will always use that same tunnel for future packets.

on the server

The same configuration as the laptop.


Every now and then I am experiencing wrong behavior from the ANDNA system. This comes not as a surprise: I was aware of some undiscovered bug in it. I am going to do some research on this side.

Stay tuned!

Sunday, February 13, 2011

Internet Gateway Sharing, soon

A first implementation of an Internet Gateway Sharing (IGS for short) mechanism will land in the repository very soon.
A node that is connected directly to the Internet will then be able to share its connection with other nodes. The nodes that haven't got a direct connection to the Internet receive informations on the 10 nearest nodes that allow to use their connection. Thus, they are able to use any number of them as gateways.
The current implementation (once again only linux is supported) allows to configure a node as a gateway or as a user. Theoretically it should be possible to be at the same time a gateway for other nodes and a user of other gateways but I had some troubles with this configuration in my tests, so I disabled this possibility.
The choice has to be made in the configuration file (/etc/netsukuku/settings.conf) by setting the variable IGS_MODE.
Set it to 'SHARE' to be a gateway. Set it to 'USE' to be a user. By default it is set to 'NONE' and the node will neither share its connection (if it has one) nor use the connection of others.
Further, if you set it to 'SHARE' you can indicate a Internet IP address to be periodically ping-ed in order to check if the Internet is at the moment reachable or not. This address can be set in the variable IGS_ANNOUNCE_INTERNET, which is by default set to 'ALWAYS', meaning that this node is to be considered always connected to the Internet.
Finally, if you set IGS_MODE to 'USE' you can indicate how many gateways you want to use at the same time. This can be set in the variable IGS_TUNNEL, by default 4. Min value is 1 and max value is 10.
With real hardware and real Internet I can only test one gateway. I hope to be able in a near future to test the real benefits one can get by using more than one ISP connection.

I want to test it a little bit more before doing a commit.

At the same time I am having some troubles with my netbook (Dell) wireless chipset. Its functioning is not reliable at all. I think it is a problem with the driver or with the hardware.
Since I have got another netbook, one of the first Eeepc shipped by Asus, I thought to use it. Its wireless chipset is much more reliable. The driver supports the new mac802 framework, so I can use the 'iw' commands instead of 'iwconfig' et al.
Thus, in the end, I will dedicate it to be running the DHCP daemon that I talked about in my previous post, for non-netsukuku clients.

While I am deploying this network I will also try to make use as much as possible of the methods we have got to try and mitigate the bufferbloat problem, that we so much experience these days on the Internet. I began to be aware of this problem by chance, on the blog of Jim Gettys.
So, if you feel brave enough when you build and deploy your very own TCP/IP mesh network, then look and ask for advices at www.bufferbloat.net.
You'll never regret!

Tuesday, January 18, 2011

Simple entry point for "legacy systems"

A bit of new code has been committed that should make netsukuku work better as a wrapper for DNS resolution on the Internet side. See my previous post, the part about asynchronous resolution.

I am working out things for a wireless link with a house nearby. Two friends of mine live there and we want to try and see what useful benefits we can get from such a connection.
We are not ready yet but I think that soon this will be a subject for a new post on this netsukuku deploy.

In the mean time, I want to spend a post to give a humble tip on what we can do, at this moment, to make happy also the people with Windows or other OS's.
I say at the moment because I hope in a future release we'll be able to run the netsukuku daemon also in other OS's, whilst now only linux is supported. (btw, Ubuntu and Fedora have both been quite successfully tested)
Anyway, also at this stage, there can be situations in which we'd want to use the network with devices that are not so easy to hack. A smart-phone for example. Or a net-top-box or entertainment device.

A device where the netsukuku daemon cannot be run, cannot by design be a first-class citizen of the network. Its problem is that it cannot have an address that is guaranteed to be unique and reachable by any other node.
Anyway this does not preclude the possibility for us to give a NATted address to it. All in all, it's not much different from what the average user of the Internet can expect, these days.
All that is needed is a direct neighbor that is able to run netsukuku and serve as a NAT. Any linux distro can act as such.

As I said, this post wants to be a humble tip for the reader, which I suppose is not a network illiterate. But nevertheless this is a valid post in this blog because I will effectively document what I actually do in this real network deploy.

The scenario

I recall briefly the situation in my house. There is a wired LAN that connects a server (luca-desktop) in the garage, a netbook (luca-dell) in the apartment and a wireless router (owned by the WISP) in the roof. Then, the netbook maintains an ad-hoc wifi network to which a laptop (luca-laptop) is connected. (btw, a wireless card in AP mode would have worked fine, as well)

The server and the netbook are able to access directly the Internet, while the laptop is not.

Now, I have a Nokia N900 smart-phone. It can connect to wifi networks, both managed and ad-hoc ones. I want it to be able to use the net.
The ideal candidate for this job is luca-dell, since it is connected to the Internet too. It is possible (and easy) to use, at the same time, the same ad-hoc network that is used to provide access to other netsukuku nodes.

The solution

I show you again the current state of addresses and routes in the netbook. This is the machine that will do the NAT.

luca@luca-dell:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:21:70:c8:0d:c0 brd ff:ff:ff:ff:ff:ff
    inet scope global eth0
    inet scope global eth0
    inet6 fe80::221:70ff:fec8:dc0/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:23:08:1f:90:ba brd ff:ff:ff:ff:ff:ff
    inet scope global eth1
    inet6 fe80::223:8ff:fe1f:90ba/64 scope link 
       valid_lft forever preferred_lft forever
luca@luca-dell:~$ ip r dev eth0  scope link 
default via dev eth0 
luca@luca-dell:~$ ip r list table ntk dev eth0  scope link  src via dev eth0  src 

"eth1" is the wireless network interface, in ad-hoc mode.

First, I give to the netbook a new address in a private subnet. The addresses in this subnet will be reached via the wireless interface.

luca@luca-dell:~$ sudo ip addr add dev eth1
luca@luca-dell:~$ sudo ip route add dev eth1

Now, the magic of NAT is done by iptables. The following command instructs the kernel to use masquerading for traffic originated in the subnet and destined to the outside.

luca@luca-dell:~$ sudo iptables -t nat -A POSTROUTING -s \! -d -j MASQUERADE

Last, I install a DHCP server and configure it to listen to requests in interface eth1. This is not strictly needed, but allows a client to auto-configure itself. This part is dependent on the distribution in use. The following commands work on Ubuntu.

luca@luca-dell:~$ sudo apt-get install dhcp3-server
luca@luca-dell:~$ sudo tee -a /etc/dhcp3/dhcpd.conf <<EOF >/dev/null
subnet netmask {
  option routers;
  option broadcast-address;
  option domain-name-servers;
luca@luca-dell:~$ sudo dhcpd3 eth1

Now, a client that is not managed by the netsukuku daemon, when it ties up to the wireless network luca-ntk, it will look for a DHCP server. It will get an address in the range, it will have a default gateway and will use it also as a DNS. In particular, this last bit means that the hostname resolution is delegated to our "DNS wrapper". Hence, the client will be able to translate Internet hostnames as well as netsukuku ones.

Picture worth a thousand words

In this screenshot from my phone (good ol' maemo) you can see the status notification for my account on Gtalk, the green circle near the battery indicator; that means that the phone reaches the Google servers.
Further, you see that the browser has loaded a page from the web server at luca-desktop.ntk; that means that the phone reaches any node in netsukuku.

That's it. Stay tuned for more updates!

Saturday, January 1, 2011

Sharing stats

As a post for the first day of the new year I want to share with you some statistics about the readers of the blog.
This is for me a hint on the usefulness (or lack of) of this effort. Whether you check the blog just for curiosity, or you want to start a similar testbed of your own, or you're waiting for someone else to just deploy the new internet for you, the simple fact that you read these lines pushes me to keep on testing and documenting.
So, since these figures are visible in real time only to me, I thought I could share them with you today.

The first post is dated Nov 25. Up to now the number of pageviews (my own visits are not included) is 360.

Pageviews by Countries (All time)

Italy 148
United States 52
Russia 44
Australia 31
Germany 21
France 21
United Kingdom 7
Ukraine 5
Ireland 3
Albania 2

A big portion of these pageviews have been recorded in the first few days, when I shared the info of this new blog with some mailing lists.
To make a comparison, the following are figures for just the month of December, when the pageviews are well distributed in all days.

Pageviews by Countries (December)

United States 29
Russia 28
Italy 27
Australia 19
France 13
Germany 9
Ukraine 5
United Kingdom 4
Argentina 2
Croatia 2

To finish, a graph of the pageviews of December:

 Happy new year!