loading

By itself the Raspberry Pi doesn't boast impressive specs. But with the dirt cheap price, buying several of these and connecting them to use they're combined processing power could potentially make a decent low cost computer. There have been several impressive rigs built connecting dozen's of Pi's together. So in this instructable, let's explore how the technology behind cluster computing and make our own Bramble Pi!

Step 1: Watch the Video Version

I've also made a two part video version of this exact same project. So if you're not one for reading, kick back and watch the vids! Please note that the video version uses Raspbian Wheezy. However, the instructions below are for the current Raspbian Distro, which is Jessie at the time I am writing this.

Step 2: What You Will Need

To follow along with this project, here are the parts you will need to acquire

Here’s what you will need:

  1. 2 or more Raspberry Pi’s
  2. SD cards for each Pi
  3. Power Cables for each Pi
  4. Powered USB Hub (optional)
  5. Networking Cables
  6. A Hub or a Router

TOTAL COST: ~$100.00

Step 3: Installing and Configuring Raspbian

Once you have all the parts, the next step is to download and configure Raspbian OS on one of the Raspberry Pi's. This will be your master Pi. Here's the steps:

  1. Download the Raspbian Image from here.
  2. Burn the Raspbian image to each SD Card you have for each Raspberry Pi.
    1. If you have Windows, you can follow these instructions.
    2. If you have a Mac, you can follow these instructions.
  3. Once the image is burned to your SD Card, put it into each of the Raspberry Pi and boot it up.
  4. Upon first boot, you should see the Rasbperry Pi Desktop. Click on the menu icon in the upper lefthand corner and go to Preferences > Raspberry Pi Configuration. Here’s the options we’ll need to configure
    1. Expand the File SystemIf needed.
    2. Change the hostname to Pi01
    3. Change the boot option to CLI (Command Line Interface), since we won't really be using the Desktop Interface.
    4. Clock on the "Interfaces" tab and make sure SSH is enabled.
    5. Click on the "Overclock" tab and choose "Turbo".
    6. Change the GPU memory to 16mb.
    7. Click on the "Localization" tab and set your keyboard layout to match that of your countries.
    8. Finish out of the configuration, and reboot your Pi.

Step 4: Installing MPICH

Still using only the one Pi as the master, we now need to install the primary software that is going to allow us to use the processing power of all the Pi's on our network. That software is called MPICH, which is a Message Passing Interface. Here's what you need to do to install it:

sudo apt-get update

mkdir mpich2

cd ~/mpich2

wget http://www.mpich.org/static/downloads/3.1/mpich-3.1.tar.gz

tar xfz mpich-3.1.tar.gz

sudo mkdir /home/rpimpi/

sudo mkdir /home/rpimpi/mpi-install

mkdir /home/pi/mpi-build

cd /home/pi/mpi-build

sudo apt-get install gfortran

sudo /home/pi/mpich2/mpich-3.1/configure -prefix=/home/rpimpi/mpi-install

sudo make

sudo make install

nano .bashrc

PATH=$PATH:/home/rpimpi/mpi-install/bin

sudo reboot

mpiexec -n 1 hostname

These commands will download and install MPICH, as well as add it as a path to your BASHRC boot file. The last command runs a test to see if it works. If the last command returns “Pi01”, then you did everything successfully.

Step 5: Installing MPI4PY

As it is, MPICH can run C and Fortran programs. But since the Raspberry Pi has the Python coding environment pre-installed, it would be easiest to install a Python to MPI interpreter. Here’s the commands to do that:

sudo aptitude install python-dev

wget https://mpi4py.googlecode.com/files/mpi4py-1.3.1.tar.gz

tar -zxf mpi4py-1.3.1

cd mpi4py-1.3.1

python setup.py build

python setup.py install

export PYTHONPATH=/home/pi/mpi4py-1.3.1

mpiexec -n 5 python demo/helloworld.py

That last command should return five responses. Each one is a different process on Pi01 running the python program "Hello World" that we just made.

Step 6: Copying the Image

Now that we've successfully configured our master Pi, we need to copy that Pi's SD card image to all the other Pi's. Here's how you can do that on Windows:

  1. Take the master SD Card out of the Pi and insert it into your computer.
  2. Using Win32DiskImager, use the "Read" button to save the contents of the SD card to your computer.
  3. Eject the master SD Card and insert an SD card for one of the other Pi's. Then use the Win32DiskImager "Write" option to write the image we saved to the new SD Card.
  4. Repeat step 3 until you have the master image written to all of the SD cards.

Step 7: Configuring the Remaining Raspberry Pi's

Now that we have all of the SD Cards prepped, insert the Master SD Card back into the Master Pi, connect it to a router and boot it back up. Then for the remaining Raspberry Pi's, insert SD Cards into all of them, connect them to the same router as your Master Pi, and then boot them all up. None of the secondary Pi's need to have keyboards, mice, or monitors.

Once all of the Pi's are powered on, using our Master Pi, we should be able to get the IP addresses of each Pi on the network. Here's how:

  1. First Install NMAP
    sudo apt-get update
    sudo apt-get install nmap
  2. Then get the current IP for the master Pi
    ifconfig
  3. Now you can scan your routers subnet for the other Pi IP addresses
    sudo nmap -sn 192.168.1.*

Copy down all of the IP addresses that pertain to the other Raspberry Pi's on the network. We will then be able to use those IP's to connect into each of the other Pi's using SSH. What we need to do first is rename each of the secondary Pi's to a unique network name. Right now they're all set to Pi01. Assuming that one of the secondary Pi's IP addresses is 192.168.0.3, here's how you can connect to it and change it's name:

  1. Establish an SSH connection
    ssh pi@192.168.0.3
  2. Run raspi-config
    sudo raspi-config
  3. In the interface, scroll down to the Advanced option, and then choose Hostname.
  4. For the hostname, change Pi01 to the next sequential number, which is Pi02.
  5. Then exit out of the SSH session
    exit

You want to repeat those steps for each of the other Pi's on the network renaming them to Pi03, Pi04, etc.

On your master Pi, you want to create a new textfile called "machinefile"

nano machinefile

And in it, you want to type in each of the Pi's IP addresses (including the Master IP address) on a new line and then save the file.

At this point, we could run a test file using mpiexec -f machinefile -n 4 hostname, but it will error out saying that there was a "host key verification failure". So in the next step, let's fix that.

Step 8: Verifying Host Keys

To fix it so that communicating with each Pi doesn't result in a host key verification failure, we need to create and swap keys for each of our Raspberry Pi's. This part may get slightly complicated, but hopefully you can stay with me.

  1. On the Master Pi, in the default home folder, create a new key.
    cd~
    ssh-keygen
  2. Navigate to the ssh folder and copy the key file to a new file called "pi01"
    cd .ssh
    cp id_rsa.pub pi01
  3. Next you want to connect via SSH into Pi02 and repeat those same steps to create a Pi02 keyfile
    ssh pi@192.168.1.3
    ssh-keygen
    cd .ssh
    cp id_rsa.pub pi02
  4. Before exiting out of Pi02, we need to copy the Pi01 keyfile over to it and authorize it.
    scp 192.168.1.2:/home/pi/.ssh/pi01 .
    cat pi01 >> authorized_keys
    exit
  5. With Pi02 done, repeat all of those steps for Pi03
    ssh pi@192.168.1.4
    ssh-keygen
    cd .ssh
    cp id_rsa.pub pi03
    scp 192.168.1.2:/home/pi/.ssh/pi01 .
    cat pi01 >> authorized_keys
    exit
  6. Repeat the last step for the remaining Pi's that you have on your network.
  7. After generating keys for each of the Pi's, exit back to your Master Pi and copy over all of the keys generated on each of the Pi's.
    cp 192.168.1.3:/home/pi/.ssh/pi02
    cat pi02 >> authorized_keys
    cp 192.168.1.4:/home/pi/.ssh/pi03
    cat pi03 >> authorized_keys
    cp 192.168.1.5:/home/pi/.ssh/pi02
    cat pi04 >> authorized_keys
    (repeat for as many Pi's are on your network)

Step 9: Running a Program on Your Supercomputer

Now everything should be set up. While still on your Master Pi, try running this machinefile again:

cd ~
mpiexec -f machinefile -n 4 hostname

If everything was done correctly, it should return the IP addresses of all your Raspberry Pi's. Now that we've successfully tested out our supercomputer, lets run a python program on it:

  1. Download and unzip my test Python password cracking script.
    wget http://www.tinkernut.com/demos/364_cluster_comp/python_test.tar.gz
    tar -zxf python_test.tar.gz
  2. Edit the password hash to one that you'd like to crack.
    nano python_test/md5_attack.py
  3. Copy the Python file to all of your Pi's.
    scp -r python_test 192.168.1.3:/home/pi
    scp -r python_test 192.168.1.4:/home/pi
    scp -r python_test 192.168.1.5:/home/pi
    (repeat for all remaining Pi's)
  4. Run the python script.
    mpiexec -f machinefile -n 5 python python_test/md5_attack.py

The script will run using the processing power of all of the Pi's on your network! Feel free to test it out using your own python script!

<p>ok i have done this so many times now everything works until i try the test command after the reboot but all i get is that theres no such command, so how do i use this &uml;mpiexec -n 1 hostname&uml; to test that it worked because i have gotten everything up to that point done code for code command for command.</p>
<p>ok found my problem</p><p>make: *** No targets specified and no makefile found. Stop.</p><p>so how do i fix it?</p>
<p>hello great instructable im currently stuck at this part though im not sure if its because its a pi 3 please help</p><p>everything was going fine until</p><p>nano .bashrc</p><p>PATH=$PATH:/home/rpimpi/mpi-install/bin</p><p>sudo reboot</p><p>mpiexec -n 1 hostname</p><p>every time i type the nano .bashrc it starts something like ther terminal but works more like a text document, yes im a bit new to the rpi community so sorry if its a lame question lol</p>
<p>Hi! Im planning to do this for my school project, but I have a question; I know that the instructions say that you should set your pi to boot on command line, but could I skip that step? Would the supercomputer work properly with the desktop?</p>
<p>I had built a raspberry pi mini supercomputer with 70 nodes thanks to my college for funding it. I had followed the documentation from SouthHampton University but because of the upgrade in hardware and software, there were any issues that we faced. I wrote a list of most common errors that occur and how to solve them: <a href="http://sourcedexter.com/6-common-errors-when-building-a-raspberry-pi-supercomputer/" rel="nofollow">http://sourcedexter.com/6-common-errors-when-build...</a> . This can be very helpful to people who are starting out to build this kind of a cluster. We also built a small application that summarises multiple technical blog posts by making use of the distributed computing network.</p>
<p>when i try to copy across the keys to the master i just get this error:</p><p>cp: missing destination file operand after &lsquo;192.168.1.4:/home/pi/.ssh/jexpi1&rsquo;</p><p>Try 'cp --help' for more information.</p>
<p>never mind. they should be scp instead of cp</p>
<p>i also have 2 questions:</p><p>1. can you use mixed types of Pi in the cluster?</p><p>2. can you do the same thing with the raspberry pi compute module?</p>
<p>yes you can mix them but it is a different install for each one so you can not pxe install the system. as for the other part i would never say anything is not able to do with time and being able to work hard at something i think you could but i would not know how to. </p>
<p>I have two questions for the community to answer:</p><p>1. Could Pi Zeros be used.</p><p>2. Does the hub need to be connected to the internet.</p>
<p>this will let you use a rj45 in a zero ENC28J60 based Ethernet board from DX.com and as for the hub thing depending on what you are doing with it is it needed. that is a more up to you think you migh want it for the frist time install but after ten you might be good </p>
Like maewert said. The Pi Zero wouldn't really be up for the task. However, if you feel like getting a fairly thorough knowledge with computer engineering, I have recently thought about engineering a new IO board for the RPi Compute Module 3 with stackable headers specifically for clustering units.
<p>1. Pi Zero does not have built-in Ethernet, so no, this method is not suited to Pi Zeros. (There may be other methods tho that are suitable to Pi Zeros)</p><p>2. No, once the super computer is setup you can run the programs on the cluster with no internet connection.</p>
<p>If all are tied together via ssh, all net traffic loses bandwidth for encryption, and it's the net traffic that provides the distributed processing access. Why not have the cluster on a subnet and have them proxy into a firewalled gateway (PC or another Pi) if internet access is needed. That would be more efficient and safe. Maybe I'm short sighted somewhere here, since I'm not a networking expert by any measure.</p>
<p>I think you're right. This could be done with Telnet instead of SSH, maybe one Raspberry Pi functioning as a kind of secure controller/proxy, and a firewall.</p>
Yea, then you'll be vulnerable to Firewall and NAT evasion techniques (such as Sami Kamkar's NAT pinning) and by extension Arbitrary Code Execution and Traffic Sniffing (in case you authenticate with Telnet which is unencrypted). Telnet needed to stay dead in the 70's when it was created. Go with SSH or better yet create a VPN on a controller box with decent quality encryption algorithm and key length and a thorough and restrictive firewall. Don't ever install a service that doesn't care about your security or you will have people farming out your resources for password cracking/mining or a botnet.
<p>Firewalls are the crunchy part of the Tootsie Pop, with the network behind it the qooy chewy center.</p><p>VLANs help on sniffing &amp; such. But this then becomes a nightmare for the typical hobbyist who would be building something like this. SSH is your friend.</p>
<p>Use a standard PC running Linux as the master unit as well as a router to the rest of your home/office LAN. Make all of the RPi slaves version 2 B+ units, with all their quad-core 1GB RAM goodness. Enable SSH compression. You can SSH into the master unit to start processes, and with 4 cores on each RPi compressed SSH encryption will be no trouble. Network traffic usage due to SSH encryption without compression won't be appreciably greater than the same traffic over unencrypted protocols, with compression it shouldn't be greater at all, and either way the encryption/compression load won't be very big on the individual cores.</p><p>SSH is very easy to set up and use, even considering the need to generate and distribute system keys.</p>
<p>Trouble is that a standard PC would probably vastly outperform the cluster it controls :-)</p><p>Roughly speaking an 8 node Pi 2B cluster costs &pound;280 and has about the processing power of a 3GHz Pentium 4 system (but split 8 ways so a single process that takes 10 seconds on the PC will take 80 seconds running on a node) </p><p>Pi clusters are awesome toy systems great to play with and learn on but have limited real world use (the only edge they have is the power consumption would be about half the PC)</p>
<p>Use a standard PC running Linux as the master unit as well as a router to the rest of your home/office LAN. Make all of the RPi slaves version 2 B+ units, with all their quad-core 1GB RAM goodness. Enable SSH compression. You can SSH into the master unit to start processes, and with 4 cores on each RPi compressed SSH encryption will be no trouble. Network traffic usage due to SSH encryption without compression won't be appreciably greater than the same traffic over unencrypted protocols, with compression it shouldn't be greater at all, and either way the encryption/compression load won't be very big on the individual cores.</p><p>SSH is very easy to set up and use, even considering the need to generate and distribute system keys.</p>
<p>How many Pi's could one network take, if i were using a ethernet hub?</p>
<p>A good managed switch would probably allow up to about 128 nodes. Or so.</p>
<p>if i used 6 raspberry pi with master, how many ram can get ? just ram of master or ram's of six raspberry pi?</p>
<p>Each node has its own unique memory. There's no combining the memory as each node is potentially running at a different spot in the program, assuming every node is identically tasked. Clusters (unlike &quot;super computers&quot;) do not all have to be running the same code.</p>
<p>Damn it I want a pie now. </p>
<p>Do you know of any way to take a bunch of old 396 and 496 computers and hook them all together to make a much more powerful computer, not just networking them, but sharing processing power, or is that even practical?</p>
<p>That would be the original beowulf cluster http://yclept.ucdavis.edu/Beowulf/aboutbeowulf.html</p>
<p>As a novel I am, I would think that a tetrahedral connection of four RPis plus one fifth linked straight to the other four would obviate ssh, hub, and some. And this fifth device (Pi or non-Pi) could act as a master to the rest or in a smp way, even using my favorite GUI with user/external interface, everything else distributed by the grace of linux+mpi.</p>
<p>I definitely wanna build this!!!!!!!!!</p>
<p>Amazing!!! Very Cool XD</p>
<p>What kind of tablet is he using to install rasbian ?</p>
<p>Just a simple question: How about using a wifi dongle for each pi instead of using LAN cable connection? Does it effect the performance? Thanks.</p>
Yes, it will effect in moisture like in rainy days.
<p>Just wondering, are not the CP commands in step 7 missing a destination</p>
<p>They should be 'scp' commands. Looks like a typo. </p>
To avoid all the file copying nonsense, you could mount all the home directories with NFS.
<p>How it this done?</p>
<p>thank you</p>
Is there a way to set this up so one of the nodes runs in GUI mode so it's easier to use the console and run applications that share the processor load?<br><br>I have little experience with Linux command line usage, much more with a GUI that every moron like me can use.
<p>This can run aircrack?<br></p>
It can be use in any purpose? For example in editing videos..as computer
<p>For step six I keep getting a &quot;bad termination of one of your application processes&quot; can anyone help with this problem? I tried the latest version of 2.0.0 and 1.3.1 both gave me the same problem. Thanks</p>
<p>Any real life practical applications (render servers or similar)? </p>
<p>When you first started running this stuff, did you ever get the following errors (ignore the references to bramble-1, I'm running from bramble-0)? It only happens when I point it to my other systems. I made sure the machinefile has the right information.</p><p>After compiling, I ran this command:</p><p>mpirun -np 3 -machinefile machines ./cpi</p><p>[proxy:0:1@bramble-1] HYDU_create_process (/home/pi/mpich2/mpich-3.1/src/pm/hydra/utils/launch/launch.c:75): execvp error on file ./cpi (No such file or directory)</p><p>===================================================================================</p><p>= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES</p><p>= PID 1355 RUNNING AT 192.168.1.12</p><p>= EXIT CODE: 255</p><p>= CLEANING UP REMAINING PROCESSES</p><p>= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES</p><p>===================================================================================</p><p>[proxy:0:0@bramble-0] HYD_pmcd_pmip_control_cmd_cb (/home/pi/mpich2/mpich-3.1/src/pm/hydra/pm/pmiserv/pmip_cb.c:886): assert (!closed) failed</p>
Well played on scripting the commands within SSH. not many people know how to do it. you could take it a step further by making IP address a variable (can't remember how though) and have a file preloaded with all values you want to be used. I am actually working on automating the whole process as an exercise to get back to my roots as an old SA.
<p>Hi! I'm a newbie in this rpi world.<br>Does this works with C++ programs or only with python scripts. I want to try this with openframeworks especifically to get a better performance.</p>
<p>Hi! Checkout the following links on setting up a parallel compute cluster with openmpi.</p><p>Compiling openMPI in general.</p><p><a href="https://www.open-mpi.org/faq/?category=building#easy-build" rel="nofollow">https://www.open-mpi.org/faq/?category=building#ea...</a></p><p>Building and setting up openMPI on ARM platforms specifically RPi<br></p><p><a href="https://rhinohide.wordpress.com/2012/02/26/openmpi-on-raspberry-pi/" rel="nofollow">https://rhinohide.wordpress.com/2012/02/26/openmpi...</a></p><p>Following this setup will get you parallelizing C++ applications via Message Passing.</p><p>If you need some literature on parallel coding with openMPI checkout this book. I used it in my parallel programming class and it's a great reference and a classic!</p><p><a href="http://www.amazon.com/gp/product/0071232656/ref=as_li_tf_il?ie=UTF8&tag=softengiintet-20&linkCode=as2&camp=217145&creative=399377&creativeASIN=0071232656" rel="nofollow">http://www.amazon.com/gp/product/0071232656/ref=as...</a></p><p>Hope this helps. Cheers!</p>
<p>It works with C++, if your program was written for mpi and compiled with mpich, the mpi implementation used by the author. Language itself is not so important as it is to have mpi support </p>
<p>such a nice project but personally have no clue what you talking about</p>
<p>true that</p>

About This Instructable

177,926views

1,181favorites

License:

More by gigafide:DIY Vintage Streaming Radio With a Raspberry Pi Turn an Old Cell Phone Into a Smartwatch Alexa Ruxpin - Arduino & Raspberry Pi Powered Voice Assistant 
Add instructable to: