Redundant Cloud Storage With a Friend and a Raspberry Pi

Do you enjoy the many benefits of cloud services but do you also like to be in full control of your own data? Maybe you are worried about privacy or maybe you just want a little more flexibility. In my case I wanted to have a redundant storage drive for 100's of gigabytes of footage but I did not wan't to pay a premium to Google, Dropbox, Amazon or another provider for it. In this instruct able I will show you how you can use Bittorrent Sync and Raspberry Pi to set up your own redundant cloud storage with a friend.

The basic idea is to have two devices in two different locations, one at your own place and one at your friends place. It should be a computer you don't have to take with you so it can be more or less permanently connected to the internet. Each device has an external USB drive of equal size. Then you can fill up to 50% of your own drive with your own files and the the other 50% is reserved for a read only copy of your friends data. For this guide we will be using a Raspberry Pi, but it can basically be any Mac, Windows or Linux device that can run Bittorrent Sync and read a USB drive. My friend e.g. uses an old Macbook in his end. 

What you need (for one end)
- Router or switch to put the pi on the same network as your computer
- Raspberry Pi
- SD card
- Keyboard
- Monitor
- USB Drive

I chose this 3 TB Seagate Expansion Desktop Drive
http://www.seagate.com/external-hard-drives/desktop-hard-drives/expansion-hard-drive/

I am going to format my drive as EXT4 for use with the Raspberry Pi, but you can basically choose anything you like, and it does not need to be the same in both ends.

Step 1: Configure Your Pi

Setup your Raspberry Pi with Raspbian, you can use the new NOOBS installer, download and follow the instructions at http://www.raspberrypi.org/downloads.

Enter the raspberry pi configuration tool using

$ sudo raspi-config

Then enter the advanced menu. From here set the hostname to the name you would like your Pi to turn up as on the network, for this tutorial we will stick with the default name 'raspberrypi'. And enable the SSH server.

$ sudo apt-get update

$ sudo apt-get upgrade

Install your favourite editor I will be using vim

$ sudo apt-get install vim

install zeroconf

$ sudo apt-get install libnss-mdns

At this point you can open up your laptop and connect to the pi using ssh.

$ ssh [e-mail]

Step 2: Format Your Drive

Format as ext4

Mount the drive
$ sudo mount /dev/sda1 /mnt/usbdrive

Edit fstab to mount the drive on startup. Add following to beginning of /etc/fstab

$ sudo vim /etc/fstab
/dev/sda1       /mnt/usbdrive   ext4    defaults          0       0

The drive I choose has the problem that it does not automatically spin down when idle. You can install a utility called hdparm to fix this.

$ sudo apt-get install hdparm

This will make the drive spin down after 10 seconds of inactivity.

$ sudo hdparm -k1 -K1 -S2 /dev/sda1

Step 3: Setup BitTorrent Sync

Follow the guide here to install Bittorrent sync and add your folders.
http://blog.bittorrent.com/2013/08/20/sync-hacks-how-to-sync-without-data-loss-using-btsync-raspberry-pi/
Remember to add the /etc/init.d/btsync file and register with update-rc.d so it will continue running after a reboot.

Create two folders or images on the drive one for you and one for your friend. If you want encryption you can use an encrypted sparse bundle for your entire part of the backup drive.

Create a bittorrent hash for each folder and share the red only key with your friend. This way you make sure you do not accidentally override each others files.

Step 4: Access Your Files From a Mac Via AFP

Netatalk

I use Lightroom to organise pictures and footage on my Laptop and I wanted a way to keep track of the files on the redundant storage from the application. Lightroom has a neat way of saving thumbnails of any media that you move to an externally mounted folder. To make this work you can install a little application called Netatalk on the Raspberry Pi. It enables AFP over TCP access to selected folders on the pi, or the connected USB drive.

$ sudo apt-get install Netatalk

By default netatalk only shares your users home directory. Add the external drive in /etc/netatalk/AppleVolumes.default

/mnt/usbdrive/yourfolder  VolumeName allow:@groupname options:usedots,upriv

yourfolder should be a folder on the drive with your content, your friend has a read only hash for this folder and will make sure it gets backed up.