Off-site backups for Synology NAS – using two raspberry pi’s, behind dynamic NAT IP’s

I recently bought a 4 bay synology NAS (DS416 Play). The main issue I had was off-site backups. It’s ok having 4 disks for resilience, but if my house burns down or gets burgled, I still lose everything.

So I started to think up ways of doing an offsite backup, without having to remember to do it or drive around with hard disks. I came up with the idea of putting a Raspberry Pi at a family members house with an external drive attached, and rsync’ing to it. If I do this in the middle of the night, then it won’t noticeably interfere with internet connections.

The main issue is that family members don’t have static IP’s (they are behind typical ISP routers with dynamic IP and NAT), and the synology makes an outbound connection to do the rsync. So I decided to use an intermediate server which I already have in the internet, and tunnel the rsync over a reverse SSH tunnel. Another stumbling block was synology trying to be too clever – initially I tried to set up a reverse tunnel on the NAS, but the HyperBackup software won’t let you back up to a local IP. For this reason I ended up with another Raspberry Pi next to the NAS, but I suppose you could use any device that’s always on. I could have cut out this second Raspberry Pi by going straight via the external box too, but I didn’t want to open the forwarded ports to the internet – using the second Pi, the forwarded port is only open to my LAN.

So the topology will end up looking like this:

NAS — LocalPi — InternetServer — RemotePi

Initially though, it’s best to set up this:

NAS — RemotePi

This lets you test that the rsync works, and also lets you do the initial sync (which might be a big one) on the LAN rather than uploading it to the internet over ADSL!

So, lets get started.

Install Raspbian on two Pi’s. Configure them to boot to console, not GUI (these will be headless).

Setting up the external storage

1. Set up the remote pi with Raspbian, and connect it to the same LAN as the NAS. I’ve left it on DHCP, so that wherever I go plug it in, as long as it has a wired connection it will just work (we’ll set up autossh to establish a tunnel as soon as the network comes up later).
2. Follow the message log (tail -f /var/log/messages) and plug in the USB hard disk. You should see a bunch of messages about it being connected, and somewhere in there should be the name of the device – usually sda.
3. Run this:ls -ltra /dev/disk/by-uuid/
4. You should see the UUID of the device you just plugged in – it’s probably the bottom one, but should have the name “sda” in it, something like this:lrwxrwxrwx 1 root root  10 Nov  8 13:00 0b4e2012-c5dd-492c-bd3f-ce3b7038c108 -> ../../sda1
5. Make a directory where you want to mount the hard disk. In my case I used /hdd. (mkdir /hdd)
6. At this point you might want to mount and format the disk, if you haven’t already formatted it. Some decent instructions for that can be found here: http://okomestudio.net/biboroku/?p=291
7. Edit fstab, and add the following line, obviously replacing the UUID and mount point with yours:UUID=0b4e2012-c5dd-492c-bd3f-ce3b7038c108     /hdd    ext4    defaults    0    2
8. Edit /boot/cmdline.txt, and add rootdelay=10 to the end of the line. This allows the USB disk to mount on boot without failing. It should read something like this:

    dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p7 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait rootdelay=10

1. Reboot the pi. Make sure your disk is writable (touch /hdd/test).

Setting up rsync

1. Make sure its installed (apt-get rsync)
2. Create a /etc/rsyncd.conf file. The contents of mine are (and yes, I probably shouldn’t be using root for the uid/gid):

    pid file = /var/run/rsyncd.pid
    lock file = /var/run/rsync.lock
    max connections = 100
    log file = /var/log/rsync.log
    timeout = 300
    [backup]
path = /hdd
read only = no     
list = yes     
uid = root     
gid = root     
hosts allow = 192.168.1.0/24 127.0.0.0/8 ::1     
secrets file = /etc/rsyncd.secrets 

1. Create the secrets file, with a username:password in it (separated by a colon)
2. Edit /etc/default/rsync and set RSYNC_ENABLE=true
3. I think you might need to reboot here, but I can’t remember.
4. Set rsync to start automatically:`update-rc.d rsync enable`
5. Either reboot, or manually start rsync: /etc/init.d/rsync start

Configure Hyper Backup

1. Log in to the NAS web interface and install Hyper Backup if it isn’t already.
2. Open Hyper Backup
3. Add a new Data backup task
4. Select Remote rsync Server
5. Select the following options:
   1. Server type: rsync-compatible server
   2. Server name or IP address: The IP of the Remote Pi (we’ll come back and edit this later)
   3. Transfer encryption: Off (we don’t need it, as it will be tunneled over SSH through the internet)
   4. Port: 873
   5. Username: The username you put in the rsyncd.secrets file
   6. Password: The password you put in the rsyncd.secrets file
   7. Backup module – this should be selectable in the drop down list, and give the name of whatever you had in square brackets (if you used my sample, it will be Backup)
   8. Directory: whatever you like. It will likely prepopulate to your host name, but if you have multiple backup tasks you might want to name them something sensible.
6. Follow the rest of the wizard through and select what folders etc you want to back up.
7. Perform the initial backup when prompted.
8. Add any other backups you might want, and run the initial backups for those too.

Set up the tunnel on the remote Pi

1. On the remote pi, which is the one we’ve done all the config on so far, we need to set it up to automatically connect to our intermediate server in the internet.
2. Configure your private keys – make sure you can connect to your intermediate server using key auth with no passwords.
3. Install autossh (apt-get install autossh)
4. Create a new script in /etc/network/if-up.d/ (vi /etc/network/if-up.d/reversersync), with the following:

    #!/bin/sh
    su -c "autossh -f -R 12345:localhost:873 username@intermediate.alwaysnetworks.co.uk -i /home/pi/.ssh/id_rsa -N -M 0 -o \"ServerAliveInterval 30\" -o \"ServerAliveCountMax 3\"" pi

1. Make sure you swap out any bits needed – specifically, the username, the hostname, the path to the private key, and the username at the end (I’m using the default “pi” user). This will ensure that autossh starts every time the interface comes up.

Set up the tunnel on the local Pi

1. Configure raspbian on the local pi. I gave this one a DHCP reservation, as the IP needs to remain fixed – a static IP works too.
2. Configure your private keys – make sure you can connect to your intermediate server using key auth with no passwords.
3. Set up sshd to allow forwarding from remote connections. Edit /etc/ssh/sshd_config, and add “GatewayPorts yes” to the end of the file.
4. Restart sshd – systemctl restart ssh
5. Install autossh (apt-get install autossh)
6. Create a new script in /etc/network/if-up.d/ (vi /etc/network/if-up.d/reversersync) with the following:

    #!/bin/sh
    su -c "autossh -f -L 192.168.1.110:23456:localhost:12345 username@intermediate.alwaysnetworks.co.uk -i /home/pi/.ssh/id_rsa -N -M 0 -o \"ServerAliveInterval 30\" -o \"ServerAliveCountMax 3\"" pi

1. Make sure you swap out any bits needed – specifically, the IP (replace 192.168.1.110 with your IP – this is the interface the tunnel port will bind to), the username, the hostname, the path to the private key, and the username at the end (I’m using the default “pi” user). This will ensure that autossh starts every time the interface comes up.

Reconfigure the NAS

1. Back on the NAS, edit the backup policies you already made, and change the IP to the local pi IP. Change the port to 23456. If all is working, the NAS should see the backup server online.

Troubleshooting

1. Check that the SSH established on both boxes (ps aux | grep ssh) (netstat -anp)
2. Test rsync on the remote Pi – you should be able to do “rsync rsync://localhost” and it should come back with the word “backup” (or whatever was in square brackets in the rsync config file)
3. Test rsync on the intermediate server – you should be able to do “rsync rsync://localhost:12345” and it should come back with “backup”
4. Test rsync on the local Pi – this time you can’t use localhost, because the SSH tunnel is bound to the IP – so you need to do “rsync rsync://192.168.1.110:23456” and it should return backup.

That should be it. Test a backup – it shouldn’t be loads of data, as you already did the initial sync on the LAN and now it’s just doing diffs. Reboot both pi’s a couple of times and make sure that the NAS eventually (it takes a few minutes) realises that the backup servers are available again. If it doesn’t, figure out if there is a service which isn’t starting automatically or something on the pi’s – you need rsync and autossh to run on boot / if-up, and you need the disk to mount on boot too.

If they do, you should be able to take the remote Pi and the external disk (I’d use a powered one), to a friend / family members house and ask them nicely if you can leave it there. You just need two plug sockets and a router/switch port. It should auto-reestablish the connection back to the intermediate server and you’re away.

The nice thing about using Hyper Backup is that it will email you on successes or failures (though I turned successes off – it was getting annoying at 3am!)