New Raspberry Pi 4 USB 3.0 Personal Cloud With RAID Backup

Hello and welcome. Today we will be looking at how to build your very own

Cloud software system that will allow you to store your personal information in a cloud that you Control and maintain. This cloud will also employ a RAID 1 mirror so that you can have backup Hard Disk drives so that you data will not be lost if a drive should fail.

Supplies

Links:

How to set up raspberry pi nextcloud server

https://pimylifeup.com/raspberry-pi-nextcloud-server/

How to set up software RAID 1 Mirror https://www.ricmedia.com/build-raspberry-pi3-raid-nas-server/amp/

MDADM RAID cheat sheet http://www.ducea.com/2009/03/08/mdadm-cheat-sheet/

Bill of Materials:

Raspberry pi 4 https://amzn.to/2GPc8jP

Micro SD card 16GB https://amzn.to/31ux1J1

External HDD 1TB external powered https://amzn.to/2YGiR5H

Step 1: The Setup of the PI

First you will need to install some software to get NextCloud going. Once you have flashed a good image onto a microSD card and inserted that into the pi you will need to connect both of the external hard disk drives to your pi on the USB 3.0 plugs. Then connect a keyboard to ti so you can configure things.

Then you will power up the pi and go to the top right of the screen and set up your wireless or wired internet to get an ip address.

Once that is done you will need to open up a terminal for the next step.

Step 2: Install Componants for Next Cloud

Next you will need to install some packages to get NextCloud up and running

Following commands in order

sudo apt-get update 
sudo apt-get upgrade (if you do not have latest version of OS)

sudo apt-get install apache2

Once that is finished you will need to install all the PHP with the following command:

sudo apt-get install php7.3 php7.3-gd sqlite php7.3-sqlite3 php7.3-curl php7.3-zip php7.3-xml php7.3-mbstring

After that you will need to restart the apache2 web service to make the php stuff take effect

sudo service apache2 restart

Step 3: Install the Next Cloud Software

Next we will need to install the next cloud software so you will first need to change directories to the html directory

cd /var/www/html/

Now we need to go get the software and download and extract it to this location:

curl https://download.nextcloud.com/server/releases/nextcloud-10.0.3.tar.bz2 | sudo tar -jxv

We need create a folder to store the data in temporarily on the main memory card and also need to set permissions and owner/group

sudo mkdir -p /var/www/html/nextcloud/data 

sudo chown -R www-data:www-data /var/www/html/nextcloud

sudo chmod 750 /var/www/html/nextcloud/data

Step 4: Finalize Your NextCloud Setup

Now you will need to finalize by creating a new admin user and password.

To do this you will need to go to your pi's ip address. If you do not know the address you can type the following in the terminal:

ipconfig

After gathering the IP address you will enter something like the following into either the chromium browser on the pi or on another web browser from a computer on the same network

http://192.168.1.28/nextcloud

Step 5: Setup RAID 1 Mirror

Now you will need to build your RAID volume from your two hard disk drives.

You should already have both HDD inserted into the USB3.0 ports of the Raspberry PI 4

First you will need to install the RAID software.

sudo apt-get install mdadm

Now we need to find out where the drive devices are so to do so you need to run the following command:

pi@raspberrypi:~ $ sudo blkid
/dev/mmcblk0p1: LABEL_FATBOOT="boot" LABEL="boot" UUID="3FFE-CDCA" TYPE="vfat" PARTUUID="80da4694-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="3122c401-b3c6-4d27-8e0d-6708a7613aed" TYPE="ext4" PARTUUID="80da4694-02"
/dev/sda1: UUID="76c0abd6-d547-3882-a94f-1bc4c78addf3" UUID_SUB="755aead2-13e8-04ed-d5f5-7f9805ae72b5" LABEL="raspberrypi:0" TYPE="linux_raid_member" PARTUUID="9b3ff72d-01"
/dev/sdb1: UUID="76c0abd6-d547-3882-a94f-1bc4c78addf3" UUID_SUB="057c766c-556d-9c96-cb6c-b55d3721c4bf" LABEL="raspberrypi:0" TYPE="linux_raid_member" PARTUUID="cc00f35e-52e9-43b9-b955-33f4d54b203d"
/dev/mmcblk0: PTUUID="80da4694" PTTYPE="dos"
/dev/md0: UUID="94103a0c-0985-4d75-957f-042f4d9f3bd0" TYPE="ext4"

After this command is ran we see that the two drives are reccognized as the two devices:

/dev/sda1

/dev/sdb1

Next we will need to create the RAID volume (this will be a mirror where whatever is written to one drive is automatically copied to the other drive)

sudo mdadm --create --verbose /dev/md0 --level=mirror --raid-devices=2 /dev/sda1 /dev/sdb1

Now that the RAID array is created you can verify that with the following command:

cat /proc/mdstat/Personalities : [raid10] 
md0 : active raid10 sdd1[3] sdc1[2] sdb1[1] sda1[0] 
15319040 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU] 
[>....................] resync = 0.4% (61504/15319040) finish=28.9min speed=8786K/sec 
unused devices: 

Now you will need to save your array creation by writing it to the mdadm configuration file with the following command:

sudo -i

mdadm --detail --scan >> /etc/mdadm/mdadm.conf

Then exit the root user by typing "exit" in the command window.

Now you will create the file system on your new RAID volume:

mkfs.ext4 -v -m .1 -b 4096 -E stride=32,stripe-width=64 /dev/md0

If successful you should see something like the following output:

1605632, 2654208 
Allocating group tables: done 
Writing inode tables: done 
Creating journal (16384 blocks): done 
Writing superblocks and filesystem accounting information: done

Next we need to mount the newly formatted file system on the RAID volume with the following command:

sudo mount /dev/md0 /mnt

If you get no errors things are working properly now.

Finally in order to make sure your RAID volume is allways mounted at start up you will need to edit the fstab file with the following command:

cd /etc/

sudo vi fstab

Add the following line to the bottom:

/dev/md0	      /mnt	      ext4    defaults	        0	0

Save this with ":wq" and now when you reboot your pi it will automatically mount the /dev/md0 file system to the /mnt folder.

Step 6: Moving NextCloud Data Repo to New RAID Volume

Now we will need to move the data folder we created to the new RAID volume so it is backed up between the two hard drives.

We first need to make the directory structure on our RAID volume with the following command:

sudo mkdir -p /mnt/nextcloud

after that we need to move the data folder from the html folder over to the new nextcloud folder we just created:

sudo mv -v /var/www/html/nextcloud/data /mnt/nextcloud/data

Now we will need to tell NextCloud in it's config file where to find the new location.

To do this we need to edit with the following command:

cd /var/www/html/nextcloud/config/

sudo vi config.php

Now search within this file and find the following line:

'datadirectory' => '/var/www/html/nextcloud/data',

Change that to the following:

'datadirectory' => '/mnt/nextcloud/data',

Save your changes with " :wq "

Step 7: Increase the Max Upload

Now we need to increase the maximum upload since that is very small by default.

To do this we need to edit the php.ini file with the following commands:

sudo vi /etc/php/7.3/apache2/php.ini

find the two lines:

post_max_size = 8M

upload_max_filesize = 2M

And change them both to the following:

post_max_size = 2048M

upload_max_filesize = 2048M

This sets them both to 2GB. If you need more feel free to set them higher.

Finally restart the apache2 webservice to make changes take effect with the following command:

sudo service apache2 restart

Step 8: Allow .htaccess Override in Prep for SSL SEC

To allow .htaccess to have override abilities to begin the process for securing your site you will need to edit the config file for apache2 with the following:

sudo vi /etc/apache2/apache2.conf

Find the section that looks like the following:

<Directory /var/www/>
     Options Indexes FollowSymLinks
     AllowOverride None
     Require all granted
</Directory>

Change the AllowOverride to the following:

<Directory /var/www/>
     Options Indexes FollowSymLinks
     AllowOverride All
     Require all granted
</Directory>

Now you will need to restart apache2 web service to take effect:

sudo service apache2 restart

Step 9: Setting Up SSL

Next we need to set up SSL so that we can have a self signed certificate for the https instead of http.

To begin we need the following commands to create the keys:

sudo mkdir -p /etc/apache2/ssl

sudo openssl req -x509 -nodes -days 365 -newkey rsa:4096 -keyout /etc/apache2/ssl/apache.key -out /etc/apache2/ssl/apache.crt

When the key builds there will be some information that you may want to fill out like the country and city but you do not have to fill out things like organizational unit, common name, or email address.

Once finished creating the keys you will need to enable the SSL module for apache wiht the following command:

sudo a2enmod ssl

Now we need to edit the config to use our keys we just created with the following commands:

sudo vi /etc/apache2/sites-available/default-ssl.conf

Within this file you will need to find the following two lines

SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key

Change these to the following:

SSLCertificateFile /etc/apache2/ssl/apache.crt

SSLCertificateKeyFile /etc/apache2/ssl/apache.key

Now that is chagned we need to enable the defatult-ssl config as well as restart the apache2 web service to make all these changes take effect:

sudo a2ensite default-ssl.conf

sudo service apache2 restart

Now you should be able to replace your http:// with https:// and get a secure connection using ssl certificates.

**************************

Bonus if you want to have the webpage redirect anyone that enters http instead of https you can do the following:

Open the 000 config file and add some lines as follows:

sudo vi /etc/apache2/sites-available/000-default.conf

Comment out everything in this file with a # sign and enter the following at the bottom of the file

<VirtualHost *:80>

     ServerAdmin example@example

     RewriteEngine On

     RewriteCond %{HTTPS} off

     RewriteRule ^(.*)$  https://%{HTTP_HOST}   [R=301,L]

</VirtualHost>

After adding this issue the two following commands to save it:

sudo a2enmod rewrite

sudo service apache2 restart

Step 10: For Any Additional Inforamtion

If you would like additional information here are some great links and my video from my youtbue channel to help you out. Please consider coming by my channel and dropping a sub. I know you won't regret it.

Links:

How to set up a Raspberry Pi Nextcloud Server

https://pimylifeup.com/raspberry-pi-nextcloud-serv...

How to set up RAID on Raspberry pi

https://www.ricmedia.com/build-raspberry-pi3-raid-...

-----------------------------------------------------------------------------------------------------------

Try Amazon Prime 30-Days https://www.amazon.com/tryprimefree?ref_=assoc_tag_ph_1427739975520&_encoding=UTF8&camp=1789&creative=9325&linkCode=pf4&tag=misperry-20&linkId=3266ea72a7bf7fef009fd902c4fc81f8

Support the Channel and Become a Patron http://www.patreon.com/misperry

Check out the Channel Store https://www.tindie.com/stores/misperry/

Checkout the Forum!! https://groups.google.com/forum/#!forum/misperry

Support the channel tip with bitcoins Address: 1MvcZHRbDm9czS8s776iutBBPJ39K4PEHh

Follow me on Instructables https://www.instructables.com/member/misperry

Follow me on Facebook https://www.facebook.com/misperryee/

Follow me on Twitter http://www.twitter.com/misperryee

T-Shirts https://teespring.com/stores/misperry-channel-store

Comments

[jeffbbarnett]

I followed this video and have successfully setup nextcloud with two externals drives as my raid devices. Everything has worked as per the tutorial. Sadly I upgraded my pi to bullseye and that created some problems for the pi -this has not effected the nextcloud cloud I am using. My question is, can I unlug the two raid drives, rebuild the pi (4) itself, reinstall nextcloud and plug the two raid volumes back in without losing the existing data on the two raid drives?

[Hitmepls]

If you get an error on step 2 try using 7.4 instead of 7.3

[8bitvic20]

what if you just have one external usb 3.0 SSD ?
should you go throught the raid process ? or is there another way ?

[gmaniackyle1]

If anyone has issues when following this guide about PHP being the incorrect version, i.e:
"This version of Nextcloud is not compatible with PHP 7.1.
You are currently running 7.3.14-1~deb10u1."

Just make sure you are installing the latest version of Nextcloud. Do so by dissecting the code for the install media url. Under step 3 you will find the following line of code:
curl https://download.nextcloud.com/server/releases/nextcloud-10.0.3.tar.bz2 | sudo tar -jxv

Just extract the url below and see which version they have that is the latest:

https://download.nextcloud.com/server/releases/

Now modify the code to point to the latest version, i.e at the time of me posting this the latest version is 18.0,2, so the code would be:

curl https://download.nextcloud.com/server/releases/nextcloud-18.0.2.tar.bz2 | sudo tar -jxv

to fix this issue just delete the older nextcloud by using the following commands:
cd /var/www/html
sudo rm -R nextcloud

now confirm that the nextcloud folder has been removed with the following command:
ls

You should not see nextcloud. Now install nextcloud again with the latest version i.e in my case it is:
curl https://download.nextcloud.com/server/releases/nextcloud-18.0.2.tar.bz2 | sudo tar -jxv

Hope this helps, now you can continue following the guide

[SanjayaRukshan]

Worked for me. Thank You so much! Cheers!

[borneo1910]

Every time I get to the step to change the data destination to the usb drive (mine is not a raid in this case), I get an internal server error and I can't figure out why. I've tried eveything. reinstall apache2, php, change config.php ports etc.

[nickafs3]

I am attempting to build this but with 3 2.5 in HDD with a powered usb hub, the problem I am running into is that the drives will randomly eject out of nowhere and the system freezes? I have lowered the drive count thinking it was a power issues (2 and then to 1) and am still having this issue? Any thoughts?

[ILHAM ALQADHRI]

I have some question from your video on youtube "New Raspberry Pi 4 USB 3.0 Personal Cloud with Raid Back Up"

How do you see the storage capacity of our nextcloud that has been raided? Directly on the web nextcloud

[thepk]

I'm stuck on step five it keeps telling me that it cannot open/ use do to /dev/sda1 is busy.

[barricada5]

Hi i admire your work!
How if nextcloud says “this version is not compatible with php 7.1”

[JoshH274]

There may be a more efficient method, but I fixed this by connecting my raspberry pi to my monitor, going to nextcloud.com/install/#, and downloading the "For Server" package. Then, when it finished downloading, I right-clicked the zip file and selected "Extract here". Then, I opened the terminal, entered "sudo move /home/pi/Downloads/nextcloud /var/www/html". After that, it works perfectly. Just needed to update to the most recent version of nextcloud. Make sure to do this before the step where you set the permissions.

[JoshH274]

If anyone is getting an error message that says "This version is not compatible with PHP7.1", it is because the instructions use an old version of Nextcloud. There may be a more efficient method, but I fixed this by connecting my raspberry pi to my monitor, going to nextcloud.com/install/#, and downloading the "For Server" package. Then, when it finished downloading, I right-clicked the zip file and selected "Extract here". Then, I opened the terminal, entered "sudo mv /home/pi/Downloads/nextcloud /var/www/html". After that, it works perfectly. Just needed to update to the most recent version of nextcloud. Make sure to do this before the step where you set the permissions.

[booknerdcarp]

When I get to this -
sudo mdadm --create --verbose /dev/md0 --level=mirror --raid-devices=2 /dev/sda1 /dev/sdb1

It just tells device or resource busy - I'm stuck at this point.

[howard.michael]

I have 2 powered external hard drives. However 1 of them has about 3tb of files on it. The other is a blank 4tb drive. I therefore do not have the ability to temporarily store all that is on the 3tb drive.
I like what you have put together and thank you for your hard work.
Can I follow your instructions and not lose my existing files on the 3tbndrive?

[afontoura]

Does anyone else failed to get HTTPS going?
After enabling SSL, creating certificate and key, changing apache configuration file and restarting the service, I got Page not found (404) on port 443 (HTTPS).

Edit: It worked eventually. I've done a few things, but the command I think did the trick was the following:
a2enconf ssl-params

[aircooledcafe]

You need to change the version of NextCloud your installing to the latest version. Run the following in section 3 instead or to upgrade if your getting the PHP version error.

curl https://download.nextcloud.com/server/releases/nextcloud-16.0.4.tar.bz2 | sudo tar -jxv

[AronS25]

Same as Carlos.
I get the error, this version of nextcloud is not compatible with php7.1
You are currently running 7.3.4-2

[carlos4computers]

I get the error, this version of nextcloud is not compatible with php7.1
You are currently running 7.3.4-2
Help please