Decentralized/P2P virtual private network (VPN)

Question

I've been using OpenVPN to manage a network between a few computers, but it needs a central server and all data need to go through it. Additionally, changing the central server potentially means updating the whole network.

I'm looking for an open-source decentralized (P2P) VPN network that doesn't require a central server. Requirements:

• No central server (all peers are equal).
• Secure - all data must be encrypted. Peers that are allowed to connect should be managed by giving them a signed certificate that others can verify.
• Some peers don't have a public IP address (and this can change dynamically as some of them could be mobile); the software should be able to determine this and let peers forward traffic as needed.
• It should be possible to give peers fixed IP addresses in the internal network.
• The software needs to be actively maintained (to fix security issues).
• Low memory and CPU impact, if possible.
• I'm not looking for a Social VPN, in particular, I don't want the system to rely on any social network.
• Linux and Windows support.
• The system should run as a system service, without requiring any GUI for administration.
• Optional IPv6 support would be beneficial (some peers can have a public IPv6 address, but not an IPv4 one).
• Optionally also internal DNS so all computers can be addressed by local names.

Answer 1

Tor

At first I thought Tor was the solution for you, but it is an open network: anyone can join, so your "secure" requirement doesn't work.

Tinc

So I would suggest taking a look at tinc. It's a p2p VPN where peers are authenticated through public keys and supports some NAT transversal, although it may not deal so well with hosts changing addresses because those are hardcoded in the config. It also has full IPv6 support. Short primer:

apt-get install tinc
mkdir -p /etc/tinc/net0/hosts
cat >> /etc/tinc/net0/tinc.conf <<EOF
Name = host
Mode = switch
Address = host.example.com
EOF
tinc -n net0 generate-keys
cat >> /etc/tinc/net0/tinc-up <<EOF
ifconfig $INTERFACE 172.16.242.43 netmask 255.255.255.255 up
EOF
chmod +x /etc/tinc/net0/tinc-up
tincd -d4 -D -n net0

Do this on each host, changing tinc.conf and tinc-up as appropriate. The hosts directory contains copies of each host's tinc.conf and sharing that configuration is out of band (e.g. I used git).

Also note that there were some serious security issues with Tinc, and it's not clear to me that the security design is sound.

Wireguard

If I would start from scratch now, I would look at Wireguard which has better roaming support and sound crypto. Their quickstart is excellent and basically amounts to:

apt install wireguard-dkms wireguard-tools
ip link add dev wg0 type wireguard
ip address add dev wg0 10.0.0.1/8
umask 077
wg genkey | tee /etc/wireguard/private.key | wg pubkey > /etc/wireguard/public.key
wg set wg0 listen-port 51820 private-key /etc/wireguard/private.key peer $(cat /etc/wireguard/peers/peer0.pub) allowed-ips 10.0.0.2/32 endpoint 192.0.2.1:51820
ip link set up dev wg0

This assumes the other has its public key already generated and stored in /etc/wireguard/peers/peer0.pub. That peer should be reachable at 192.0.2.1 and have a similar configuration with 10.0.0.2 assigned on the wg0 interface.

Have fun!

Answer 2

It is not possible without coordinator.

For example you have find some code on github like wireguard-p2p. After searching a bit you will find its dht bootstrap node: bootstrap.ring.cx:4222.

Another thing is hyprspace. It is using IpfsDHT with a list of bootstrap nodes. Of course, this code won't work in China, because GFW is just dropping IPFS traffic. Same thing about Tor and others.

Personally, I am using 2 approaches:

1. Git repository as coordinator. Each machine updates its own config when it receives RTM_NEWLINK, RTM_DELLINK, RTM_GETLINK. Other machines are watching git repository updates and reading other machine configs. I am using this approach for development purposes to keep machines in the same network.
2. Your own cloud coordinator. It uses a special tunnel that imitates regular web browser traffic, uses cloud fronting, and is reaching clients in China. I am using this approach for production purposes.

I am not using UDP, because firewalls can just block it or reduce its performance. Of course, I am not recommending using UDP hole punching.

Answer 3

Well, I've received some experience and found that it is impossible to build a truly p2p network using wg. I want to clearly describe why and how it happened.

There are 3 types of peers in wireguard:

1. Peer that we can connect to directly:

[Peer]
PublicKey = %
Endpoint = 127.0.0.1:2001
AllowedIPs = 20.0.0.4

2. Peer that will connect to us directly:

[Peer]
PublicKey = %
AllowedIPs = 20.0.0.5

3. Peer that is not connectable from any side (for example behind the NAT):

[Peer]
PublicKey = %

Now let's imagine that we are on the 3-rd peer type and we are reviewing its configuration. We can allow this peer to route to another 3-rd peer type by using an additional hop.

[Interface]
Address = 20.0.0.6/16

[Peer]
PublicKey = %
Endpoint = 127.0.0.1:2001
AllowedIPs = 20.0.0.4, 20.0.0.3

[Peer]
PublicKey = %
Endpoint = 127.0.0.1:2001
AllowedIPs = 20.0.0.5, 20.0.0.3

We want wg to use any of the multiple hops for routing traffic from the current peer (behind the NAT) to 20.0.0.3 peer (behind the NAT). However, wg will use just the last peer 20.0.0.5 as an additional hop in this configuration.

It means 20.0.0.6 will be available from 20.0.0.3 using the additional hop 20.0.0.5. From the opposite side, 20.0.0.3 will be available from 20.0.0.6 using the additional hop 20.0.0.5.

The problem is that wg won't ever try to use 20.0.0.4 as an additional hop when 20.0.0.5 is not available. It means when 20.0.0.5 is dead, than 20.0.0.3 becomes unavailable <-> 20.0.0.6.

The problem is the wg itself: it supports only flat routing and has no internal method for choosing the right route. For example, the pseudo-routing table for 20.0.0.6:

20.0.0.3 -> 20.0.0.5
20.0.0.4
20.0.0.5

What routing table is intended:

20.0.0.3 -> [20.0.0.4, 20.0.0.5]
20.0.0.4
20.0.0.5

The second type of routing tale is not supported by wg. Moreover, wg has no internal method of choosing the right route from such an internal routing table (round robin, etc).

My conclusions are the following:

1. Truly p2p network can be build using wg only when you have no peers behind the NAT (or just one single peer behind the NAT).
2. If you have at least two peers behind the NAT, than you won't be able to build truly p2p network using wg.