Monday, October 15, 2012

DD-wrt, openvpn and selectively routing for multiple tunnels


So, as an expat living abroad, I have times when I want to have my traffic appearing as though I am from the US. Most recently, Newegg rejected some purchases I made b/c I am coming from an IP abroad. Dd-wrt has the facility to use an openvpn client but I don’t really want to send all my traffic overseas just for the few instances I need to redirect my traffic.

As an American living in the UK, most things are pretty similar but one thing that grates on me is the ‘big brother’ government. Recently, the gov’t tried to push through a snooping bill that would track all internet use. Australia has a similar one in the works.

Now, I am pretty boring but, in principle, I completely disagree with this. So in order to add more encrypted static to the world, I am also planning to route all of my non time critical internet traffic out a local UK vpn.

In some things, like RDP, latency really matters, so I will route those out my local ISP.

Mostly though, this is a just b/c I wanted to see if I could do it. B)

Below are some proof of concept steps that I hope will help someone else get up and running. This should work w/ any number of VPNs (or other interfaces) and w/ any linux w/ the right routing in the kernel. (it was there in dd-wrt so I didn't go too much into it, I saw some mention on several of the pages I used that you need certain kernel options).

I got some help on the forums, and these three pages really brought it all together for me.

To start, I have a Netgear WNR3500v2/U/L. Based on the peacock thread, I put in DD-WRT v24-sp2 (08/12/10) mega - build 14929. This has the necessary routing options compiled into the linux kernel to do what I need.

The basic idea is this:
  1. Bring up two VPN tunnels, one to the UK and one to US
  2. Create routing tables to send traffic in that table out the specific gateway
  3. Us iptables to mark packets for specific tables/routing
I am going to do this through a start up script. Getting all of this set up required me to get the ssh daemon running and I recommend the same if you try to do this. You can do this in DD-WRT under Services –> Services –> Secure Shell. 

Once that is set you can ssh to your dd-wrt box. Login with root and your password you use to get into the web console.

For a vpn provider, I am using Private Internet Access. Anybody that gives you an OpenVPN client and allows multiple connections should be fine but you may need to tweak your openVPN.conf files.

The script (in several parts):

First, since we are not configuring the openvpn client through the GUI (I leave Services –> VPN –> OpenVPN client disabled), we need to create our own config files.

Begin by creating the directory to hold your configuration:
                mkdir /tmp/openvpncl

Create a file w/ the cert for your vpn provider:
echo "-----BEGIN CERTIFICATE-----
--snip--
-----END CERTIFICATE-----
" >> /tmp/openvpncl/ca.crt

Next, we want to create our configuration files. We do this by echoing a bunch of lines to create a config file. We do this twice, once for each VPN.
#Setup US Tunnel Config
echo client > /tmp/openvpncl/openvpn-US.conf
echo dev tun >> /tmp/openvpncl/openvpn-US.conf
echo proto udp >> /tmp/openvpncl/openvpn-US.conf
echo remote _YourVpnServer_ 1194 >> /tmp/openvpncl/openvpn-US.conf
echo resolv-retry infinite >> /tmp/openvpncl/openvpn-US.conf
echo nobind >> /tmp/openvpncl/openvpn-US.conf
echo persist-key >> /tmp/openvpncl/openvpn-US.conf
echo persist-tun >> /tmp/openvpncl/openvpn-US.conf
echo ca /tmp/openvpncl/ca.crt >> /tmp/openvpncl/openvpn-US.conf
echo tls-client >> /tmp/openvpncl/openvpn-US.conf
echo remote-cert-tls server >> /tmp/openvpncl/openvpn-US.conf
echo auth-user-pass /tmp/password.txt >> /tmp/openvpncl/openvpn-US.conf
echo comp-lzo >> /tmp/openvpncl/openvpn-US.conf
echo verb 1 >> /tmp/openvpncl/openvpn-US.conf
echo reneg-sec 0 >> /tmp/openvpncl/openvpn-US.conf


#Setup UK Tunnel Config
echo client > /tmp/openvpncl/openvpn-UK.conf
echo dev tun >> /tmp/openvpncl/openvpn-UK.conf
echo proto udp >> /tmp/openvpncl/openvpn-UK.conf
echo remote _YourOtherVpnServer_ 1194 >> /tmp/openvpncl/openvpn-UK.conf
echo resolv-retry infinite >> /tmp/openvpncl/openvpn-UK.conf
echo nobind >> /tmp/openvpncl/openvpn-UK.conf
echo persist-key >> /tmp/openvpncl/openvpn-UK.conf
echo persist-tun >> /tmp/openvpncl/openvpn-UK.conf
echo ca /tmp/openvpncl/ca.crt >> /tmp/openvpncl/openvpn-UK.conf
echo tls-client >> /tmp/openvpncl/openvpn-UK.conf
echo remote-cert-tls server >> /tmp/openvpncl/openvpn-UK.conf
echo auth-user-pass /tmp/password.txt >> /tmp/openvpncl/openvpn-UK.conf
echo comp-lzo >> /tmp/openvpncl/openvpn-UK.conf
echo verb 1 >> /tmp/openvpncl/openvpn-UK.conf
echo reneg-sec 0 >> /tmp/openvpncl/openvpn-UK.conf

Note that you will need to replace YourVpnServer and YourOtherVpnServer w/ the details appropriate to your service.  There may be other options to change as well, though, the above is fairly basic and should work w/ most services.

When we bring our tunnels up and down, we will need to tell iptables to MASQUERADE for this connection.  These will be run as scripts so we need to set the execute bit for them.  We create our up and down scripts here:
#Tun0 route up script
echo iptables -A POSTROUTING -t nat -o tun0 -j MASQUERADE > /tmp/openvpncl/route-up-UK.sh
chmod 700 /tmp/openvpncl/route-up-UK.sh
#Tun0 route down script
echo iptables -D POSTROUTING -t nat -o tun0 -j MASQUERADE > /tmp/openvpncl/route-down-UK.sh
chmod 700 /tmp/openvpncl/route-down-UK.sh

#Tun1 route up script
echo iptables -A POSTROUTING -t nat -o tun1 -j MASQUERADE > /tmp/openvpncl/route-up-US.sh
chmod 700 /tmp/openvpncl/route-up-US.sh
#Tun1 route down script
echo iptables -D POSTROUTING -t nat -o tun1 -j MASQUERADE > /tmp/openvpncl/route-down-US.sh
chmod 700 /tmp/openvpncl/route-down-US.sh

of course, we need a username and password:
#General Config
echo USER > /tmp/password.txt
echo PASSWORD >> /tmp/password.txt

Now we bring up the tunnels.  Note the –route-nopull.  This ignores routing info from the openvpn server.  We want to specify our own routing.  Without that, openvpn seems to set our default traffic to go out the last tunnel brought up.  If you are having any trouble on this step, run the line without –daemon.
#Setup tunnels. 
/usr/bin/killall openvpn
/usr/sbin/openvpn --config /tmp/openvpncl/openvpn-UK.conf --route-nopull --route-up /tmp/openvpncl/route-up-UK.sh --down-pre /tmp/openvpncl/route-down-UK.sh --daemon
/usr/sbin/openvpn --config /tmp/openvpncl/openvpn-US.conf --route-nopull --route-up /tmp/openvpncl/route-up-US.sh --down-pre /tmp/openvpncl/route-down-US.sh --daemon

In order to setup routes, we need to get our default gateways for each interface.  First, though, we hang out for a couple seconds to allow the tunnels to establish:
#The tunnels can take a couple seconds to establish.  Hold for 5 seconds to allow for this
sleep 5

# get gateway addresses
IspGateway=$(ip route list table main | awk '/default/ { print $3}')
Tun0Gateway=$(ip route list table main | awk '/tun0/ { print $1}')
Tun1Gateway=$(ip route list table main | awk '/tun1/ { print $1}')


Now, we have our two tunnels established as tun0 (UK) and tun1 (US).  We also have our default (ppp0) route to our ISP.  Now we start getting fancy.  B) 

We are going to use iptables to MARK our packets.  Here we tell iproute route any packets marked with a 2 based on the routing table, 2.
# Create fwmark to table bindings
ip rule add fwmark 1 table main # ISP
ip rule add fwmark 2 table 2 # Tunnel 0 UK
ip rule add fwmark 3 table 3 # Tunnel 1 US

You can use ‘ip route show” to see current ip rules.

Now we setup the routing for each table.  For my setup, I am just giving the default route.
# Create table to tunnel bindings
ip route add default via $Tun0Gateway dev tun0 table 2 #Send out UK Tunnel
ip route add default via $Tun1Gateway dev tun1 table 3 #Send out US Tunnel

You can see the ip routes per table with:
ip route show table 2

Now we are ready to create specific rules.  You will probably have a lot more rules than the below. 
#UK tunnel rules
iptables -A PREROUTING -t mangle -s 192.168.1.0/24 -j MARK --set-mark 2

#US Tunnel rules
ip rule add to 174.129.0.77/32 table 3

#ISP rules
iptables -A PREROUTING -t mangle -p tcp --destination-port 3389 -j MARK --set-mark 1

In order, we first say that anything coming from this specific IP range (192.168.1.0/24) should route out the UK tunnel.
Next we use a different way of layer 3 routing w/ ip rule.  I am actually just going to stick to the iptables MARKing for simplicity but I wanted to show this was possible as well
Finally, we mark a packet based on destination port.  So we are saying any RDP traffic should go out the ISP interface.

That is all there is to it.  Iptables is pretty full featured, you can get pretty crazy w/ it.  Good luck and if you do something fun, please tell me about it.

Notes:
  •  If one of your tunnels goes down, your routing for that tunnel goes away.  Your traffic will start to flow out your default interface.  This may or may not be ideal depending on the sensitivity of what you are passing.  You can manage that with the route down commands if you are concerned.
  • the order your rules apply in matters.  see ip rule output to see the order.  If you want to 'override' the UK routing, in my example, you will need your rule to to have a lower ip rule id than the UK line.

The full script is below:
mkdir /tmp/openvpncl
echo "-----BEGIN CERTIFICATE-----
--snip--
-----END CERTIFICATE-----
" >> /tmp/openvpncl/ca.crt
#Setup US Tunnel Config
echo client > /tmp/openvpncl/openvpn-US.conf
echo dev tun >> /tmp/openvpncl/openvpn-US.conf
echo proto udp >> /tmp/openvpncl/openvpn-US.conf
echo remote _YourVpnServer_ 1194 >> /tmp/openvpncl/openvpn-US.conf
echo resolv-retry infinite >> /tmp/openvpncl/openvpn-US.conf
echo nobind >> /tmp/openvpncl/openvpn-US.conf
echo persist-key >> /tmp/openvpncl/openvpn-US.conf
echo persist-tun >> /tmp/openvpncl/openvpn-US.conf
echo ca /tmp/openvpncl/ca.crt >> /tmp/openvpncl/openvpn-US.conf
echo tls-client >> /tmp/openvpncl/openvpn-US.conf
echo remote-cert-tls server >> /tmp/openvpncl/openvpn-US.conf
echo auth-user-pass /tmp/password.txt >> /tmp/openvpncl/openvpn-US.conf
echo comp-lzo >> /tmp/openvpncl/openvpn-US.conf
echo verb 1 >> /tmp/openvpncl/openvpn-US.conf
echo reneg-sec 0 >> /tmp/openvpncl/openvpn-US.conf


#Setup UK Tunnel Config
echo client > /tmp/openvpncl/openvpn-UK.conf
echo dev tun >> /tmp/openvpncl/openvpn-UK.conf
echo proto udp >> /tmp/openvpncl/openvpn-UK.conf
echo remote _YourOtherVpnServer_ 1194 >> /tmp/openvpncl/openvpn-UK.conf
echo resolv-retry infinite >> /tmp/openvpncl/openvpn-UK.conf
echo nobind >> /tmp/openvpncl/openvpn-UK.conf
echo persist-key >> /tmp/openvpncl/openvpn-UK.conf
echo persist-tun >> /tmp/openvpncl/openvpn-UK.conf
echo ca /tmp/openvpncl/ca.crt >> /tmp/openvpncl/openvpn-UK.conf
echo tls-client >> /tmp/openvpncl/openvpn-UK.conf
echo remote-cert-tls server >> /tmp/openvpncl/openvpn-UK.conf
echo auth-user-pass /tmp/password.txt >> /tmp/openvpncl/openvpn-UK.conf
echo comp-lzo >> /tmp/openvpncl/openvpn-UK.conf
echo verb 1 >> /tmp/openvpncl/openvpn-UK.conf
echo reneg-sec 0 >> /tmp/openvpncl/openvpn-UK.conf

#Tun0 route up script
echo iptables -A POSTROUTING -t nat -o tun0 -j MASQUERADE > /tmp/openvpncl/route-up-UK.sh
chmod 700 /tmp/openvpncl/route-up-UK.sh
#Tun0 route down script
echo iptables -D POSTROUTING -t nat -o tun0 -j MASQUERADE > /tmp/openvpncl/route-down-UK.sh
chmod 700 /tmp/openvpncl/route-down-UK.sh

#Tun1 route up script
echo iptables -A POSTROUTING -t nat -o tun1 -j MASQUERADE > /tmp/openvpncl/route-up-US.sh
chmod 700 /tmp/openvpncl/route-up-US.sh
#Tun1 route down script
echo iptables -D POSTROUTING -t nat -o tun1 -j MASQUERADE > /tmp/openvpncl/route-down-US.sh
chmod 700 /tmp/openvpncl/route-down-US.sh


#General Config
echo USER > /tmp/password.txt
echo PASSWORD >> /tmp/password.txt

#Setup tunnels. 
/usr/bin/killall openvpn
/usr/sbin/openvpn --config /tmp/openvpncl/openvpn-UK.conf --route-nopull --route-up /tmp/openvpncl/route-up-UK.sh --down-pre /tmp/openvpncl/route-down-UK.sh --daemon
/usr/sbin/openvpn --config /tmp/openvpncl/openvpn-US.conf --route-nopull --route-up /tmp/openvpncl/route-up-US.sh --down-pre /tmp/openvpncl/route-down-US.sh --daemon

#The tunnels can take a couple seconds to establish.  Hold for 5 seconds to allow for this
sleep 5

# get gateway addresses
IspGateway=$(ip route list table main | awk '/default/ { print $3}')
Tun0Gateway=$(ip route list table main | awk '/tun0/ { print $1}')
Tun1Gateway=$(ip route list table main | awk '/tun1/ { print $1}')



# Create fwmark to table bindings
ip rule add fwmark 1 table main # ISP
ip rule add fwmark 2 table 2 # Tunnel 0 UK
ip rule add fwmark 3 table 3 # Tunnel 1 US

# Create table to tunnel bindings
ip route add default via $Tun0Gateway dev tun0 table 2 #Send out UK Tunnel
ip route add default via $Tun1Gateway dev tun1 table 3 #Send out US Tunnel


#UK tunnel rules
iptables -A PREROUTING -t mangle -s 192.168.1.0/24 -j MARK --set-mark 2

#US Tunnel rules
ip rule add to 174.129.0.77/32 table 3

#ISP rules
iptables -A PREROUTING -t mangle -p tcp --destination-port 3389 -j MARK --set-mark 1

11 comments:

  1. Wow !!! Great post and excellent guide.
    Just a quick question...
    Would it be possible to get a specific ip address or range to go through a specific vpn and if that tunnel fails or is down. The system with that specific address would not go out until the tunnel is back up again.

    ReplyDelete
    Replies
    1. probably the easiest way to accomplish that would be to add commands to your route down command to send that specific range to a phantom gateway or block it entirely.

      Delete
  2. I'm trying yo get this to work but it doesn't seem to work anymore? Is it because its 2 years old? If someone can help that would be great

    ReplyDelete
  3. Just tried this script and it still rocks, thank you!!

    If you ever get a chance, can you put up commands for the first question? ie. Route a specific range through a specific VPN and kill it if the tunnel is down.

    ReplyDelete
    Replies
    1. Did this get answered? I am also looking for this config.

      Delete
  4. This comment has been removed by the author.

    ReplyDelete
  5. Thank you.Good solution for VPN client and tunnels.
    It works excellent.Good work.
    10webhostingservice.com

    ReplyDelete
  6. Can we set multiple openvpn clients with PBR without marking the packets? My scenario is the same like you it's just that my normal traffic would be through the ISP and I would like to use 2 tunnels corresponding to 2 different locations on 2 SSIDs.

    ReplyDelete
  7. Good to know about this Openvpn server and multiple tunneling techniques but for now I simply want to go for good but free vpn server software. I have tried searching online for it but first I need some expert advice. Can you help me?

    ReplyDelete
  8. Informative article on OpenVPN DD-WRT solution. I am using FastestVPN and configured on my router which is securing our more than 10 devices..

    ReplyDelete

analytics