A Raspberry Pi Multispectral Camera

17,625

110

42

A multispectral camera can be a handy tool to detect stress in plants, or recognise different species in lieu of the differences in the reflectance signatures of plants in general. If combined with a drone, the camera can provide the data for quick NDVIs (Normalised Difference Vegetation Index), create mosaics of farms, forests or woodlands, understand nitrogen consumption, create yield maps and so on. But multispectral cameras can be costly, and their price is directly proportional to the sort of technology they implement. A traditional approach to spectrometry is to use several cameras with long or short bandpass filters that allow the required spectrum to pass trough while blocking the others. There are two challenges to that approach; first, you need to trigger the cameras at the same time, or as close as possible; and second, you need to register (merge images layer after layer) the images so they can form one final composite with the desire bands in it. This means that a big deal of post-processing needs to be done, consuming time and resources (using expensive software such as arcmap, but not necessarily). Other approaches have dealt with this in different ways; recent technology developments at processor level have allowed the creation of scan CMOS sensors with band filters integrated in the sensor's layout. Another approach is to use a beam splitter (prism) that would direct the different beams of light to a different sensor. All these technologies are extremely expensive and therefore out of the reach for explorers and makers. The Raspberry pi compute module and its development board offer a cheap answer to few of these questions (not all though).

Step 1: Enabling the Cameras

Make sure you follow the steps for setting up the cameras in the CM as indicated in the following tutorials:

https://www.raspberrypi.org/documentation/hardware...

Trigger both cameras at the same time using:

sudo raspistill -cs 0 -o test1.jpg & raspistill -cs 1 -o test2.jpg

Use the following topic if for any reason it didn't work:

https://www.raspberrypi.org/forums/viewtopic.php?f...

Further instructions in case you are starting from scratch with the CM here:

https://www.raspberrypi.org/documentation/hardware...

Step 2: Wireless Serial Communication

Buy a set of telemetry radios like these:

http://hobbyking.co.uk/hobbyking/store/__55559__HK...

These radios have four wires: Ground (black), TX, RX, VCC (red). Peal off one extreme of the cables and use female connectors that fit the GPIO pins. Connect the black connector to ground, red to 5V, TX to pin 15, and RX to pin 14 of the J5 GPIO header of the compute module development board.

Make sure you set the baud rate to 57600, and that your host computer has recognised and added the radio as COM (in Windows use the device manager for that). If using Putty, chose serial, the COM port (3, 4 or whatever it is in your computer), and set the baud rate to 57600. Switch your CM on and after it finishes loading, click enter in your computer if you don't see any text coming through the connection. If you notice any garbled text, go and check /boot/cmdline.txt. The baud rate should be 57600. if any further problems arise, please check the following tutorial:

http://www.hobbytronics.co.uk/raspberry-pi-serial-...

Step 3: The Cameras...

You can actually use the cameras in their original configuration, but if not, you will need to modify them in order to accommodate the M12 lenses. Bear in mind that the raspberry pi cameras V1 and V2 are slightly different, so, old M12 holders won't work on new cameras. Also, there was some problems when triggering the new cameras in parallel, if you experience any of this problems please check this topic in the raspberry pi forum:

https://www.raspberrypi.org/forums/viewtopic.php?t...

In any case, a sudo rpi-update should fix the issue.

The M12 lens holder can be 'grind' with a Dremel in order to fit the connector of the CMOS sensor with the camera board. Unscrew the original lens, and place the new lens over the M12 holder. For better results you can actually get rid altogether of the original lens adapter, but it might not be worth the work in light of the risk that entails to damage the sensor. I destroyed at least six camera boards before managing to get rid of the plastic holder that sits above the CMOS sensor.

Step 4: Wifi Connection and Extra Storage

The CM development board has just one USB port; as a result of that you have to use it very wisely, e.g. wifi connection. If you want to go around that, you will have to use your soldering iron skills and attach a dual USB connector under the development board, where the USB is soldered. If you are using the same I have

https://www.amazon.co.uk/gp/product/B00B4GGW5Q/ref...

https://www.amazon.co.uk/gp/product/B005HKIDF2/ref...

Just follow the cable order in the picture.

Once done, attach your wifi module to the dual port, power on the CM and see if the wifi module is working correctly.

It is easier to attach an SD card than a USB drive, so buy something like this:

https://www.amazon.co.uk/gp/product/B00KX4TORI/ref...

To mount the new external storage, follow this tutorial carefully:

http://www.htpcguides.com/properly-mount-usb-stora...

Now you have 2 USB ports, extra storage and wifi connection.

Step 5: Print the Case

Use ABS

Step 6: Put the Pieces Together

Before you assemble the camera, connect a monitor and keyboard to the CM, and focus the lenses. The best way to do that is to use the following command:

raspistill -cs 0 -t 0 -k -o my_pics%02d.jpg

That runs the camera forever, so observing your screen, tight the lens until is focused. Remember to do that with the other camera by changing the -cs command from 0 to 1.

Once your lenses are focused put a small drop of glue between the lens and the M12 lens holder to prevent any movement of the lens. Do the same while attaching the lenses to the case. Make sure that both lenses are aligned as much as possible.

Use a drill to open a hole on the side of the case and put through the radio antenna. Place the radio securely by using double face tape and connect it to the GPIO.

Place the CM development board inside the case and secure it with 4 10mm metal hexagonal extenders. Secure the camera connector adapters so they don't bounce freely inside.

Step 7: Configure Dropbox-Uploader, Install the Camera Script.

Install dropbox_uploader following the instructions provided here

https://github.com/andreafabrizi/Dropbox-Uploader

Use a script similar to that in the picture.

Step 8: Final Product

The final camera can be placed under a medium size (650 mm ⌀) drone or even smaller. It all depends on the configuration. The camera is no more than 350-400 grams.

To power the camera, you will have to provide a separate battery, or connect the camera to the power board of your drone. Be careful not to exceed the power requirements of the CM board. You can use the following items to power your camera:

https://www.adafruit.com/products/353

https://www.amazon.co.uk/USB-Solar-Lithium-Polymer...

You can also build the support, and the anti-vibration dampers according to your drone specifications.

Once you have taken the first pictures, use a GIS program such as Qgis or Arcgis Map to register your images. You can also use matlab.

Happy flight!

Share

    Recommendations

    • Planter Challenge

      Planter Challenge
    • Sensors Contest

      Sensors Contest
    • Classroom Science Contest

      Classroom Science Contest

    42 Discussions

    0
    None
    tim.26.nardo

    11 days ago

    Hi, this one is amazing and this is exactly what we need for our thesis. We intend to do an advanced controlled
    environment that can adopt to changes regarding abiotic factors that affects
    plants and even react regarding the condition of the plant itself using machine learning. However, we need to detect the heat stress and water stress using camera and based on our research and the description you've provided, this one exactly fits.
    Hopefully, you can give us some advise and help us doing this "multispectral camera project".
    It would be a great help sir. :)

    5 replies
    0
    None
    MaykeS1tim.26.nardo

    Reply 10 days ago

    Hi,



    There are fundamental differences between what you get from a multispectral camera and from a thermal camera. Furthermore, water and heat stress will affect the plant differently and therefore will show up different symptoms. So, my first recommendation, particularly if this is academic related, you need to narrow down this further.

    Cost reduction. How are you planning to use it? A fixed rig, as in no moving around, will substantially reduce your need for multiple cameras, or even render the CM unnecessary. You could buy a raspberry pi and a multiplexer, and as many cameras as wavelengths you may need. Filters are not cheap though, and the more precision you are after, the more expensive they’ll be. So, again, set a realistic baseline with regards to what quality your data must have.

    Lastly, the CM allows you to connect two cameras, as well as buy/design an IO board for your specific needs. The latter is far, far away from amateur/school projects, as you need to invest money and longer time scales to achieve things. A raspberry pi = 1 camera. As I have said above, you can always use a multiplexer to connect more cameras; but bear in mind that they will trigger in a successive order. If you are not moving around, that should not be a big problem.

    Hope this helps

    0
    None
    tim.26.nardoMaykeS1

    Reply 8 days ago

    Wow. Thanks a lot sir! 
    However, I have some other questions. 
    1. If I will limit my detection into stress or not stress classification only, will it be possible if I only use an rpi and a NoIR camera? 

    2. I found some projects similar to yours that uses two cameras and also some projects using only one NoIR camera. What is the purpose or advantage of using two cameras for spectral analysis? 

    3. I am planning to buy a NoIR rpi camera which it includes a blue filter. What is the best filter to use - blue filter or red filter? 
    :)

    0
    None
    MaykeS1tim.26.nardo

    Reply 8 days ago

    Your questions in order:



    1. No necessarily. The biggest problem is to discern or differentiate between the stress and daily changes in reflectance, the type of stress (hydric, heat, disease), etc. Plants are living organisms; they will not behave exactly the same way all the time. When you feel cold, you put a jacket on; if you are hot, you drink a cool lemonade; plants will close stomata in severe cold and open it in hot conditions; chemical quenching will occur to dissipate heat and evotranspiration will speed up. Do you think these phenomena will affect the sort of data you are aiming to get? YES!
    2. There is also a problem of scale; how granular needs your data to be?
    3. More cameras mean more wavelengths in one go, the difficulty of the project is proportional to the amount of cameras you use.
    4. Blue or red filter means nothing to me. Please familiarise yourself with Photosynthetic Active Radiation (PAR), and the type of wavelengths used by plants in photosynthesis. What you want is to measure reflectance and then derive from it some stress scale. For that you need to look at infrared light so, you need an IR or NIR filter.



    Good luck and keep asking if needed

    0
    None
    MaykeS1tim.26.nardo

    Reply 11 days ago

    Hi,
    Thanks for your comment. Feel free to ask any questions

    0
    None
    tim.26.nardoMaykeS1

    Reply 10 days ago

    Hi sir, this are my questions. 
    1. In determining the water stress and heat stress of plants, which one is the best to use and why - thermal camera or multispectral camera? 

    2. Is there a way to reduce the cost of this project if I will make this a wired one instead of doing wireless? 

    3. I am new regarding this rpi compute module. What is the differences of this module compared to rpi 3B+? 

    :)

    0
    None
    sakaic

    13 days ago

    Hi,

    I was intending to build a camera based on the same principles but using usb camera modules and I have some questions bugging me.

    First, do you have different exposures for the different cameras? Or are they set to 0ev at calibration?

    Second, is it critical that the cameras are absolute calibrated or just relative calibrated is fine?

    Can I email you for further questions?

    3 replies
    1
    None
    MaykeS1sakaic

    Reply 13 days ago

    Hi @sakaic,

    Happy to answer your questions. Just email them through.

    0
    None
    sakaicMaykeS1

    Reply 13 days ago

    Can I have your email address?

    0
    None
    MaykeS1sakaic

    Reply 13 days ago

    maykef at gmail dot com

    0
    None
    MJB267

    7 months ago

    Hi, it sounds like a very interesting project! Could you tell me what your budget was?

    3 replies
    0
    None
    MJB267MaykeS1

    Reply 7 months ago

    Cool! I have some more questions that I would like to discuss with you. Can I send you an email?

    0
    None
    texacoon

    Question 9 months ago

    I was wondering if you added four cameras for a better multispectrum.. OR is there any advantage adding addtional cameras. I would like to build this for my drone for agriculture instead of paying for some overpriced camera that does the samething as this project. Any advice would be appricatated.
    Thanks
    Doug

    1 answer
    0
    None
    MaykeS1texacoon

    Reply 9 months ago

    Hi texacoon,
    Adding extra cameras would be advantageous if you want to have more control over specific wavelengths. For example, using 4 cameras would enable you to filter the red, green, blue and NIR light down to a specific wavelength and bandwidth. The question is, do you really need that level of control over the spectrum? Maybe not. Visible light, that is, red, green and blue, might not need that level of control if you simply want to produce NDVIs to detect general stress in plants. In that case, all you need to do is to add an extra band such as NIR (840nm with a bandwidth of 20-40nm) to detect stress. That means that you can have just 2 cameras, one getting RGB, the other one getting the NIR, for you to produce NDVIs.
    If you give me a little bit more of information I might be able to be more specific in the requirements you need. What drone do you use? Are you a farmer yourself? If so, what crops do you grow? What is it that you want to detect? Water or nutrient deficiencies? Maybe all you want to see is if your crop is growing uniformly across your field?
    When you say overpriced camera, what sort of product have you checked/used? Most available products such as the Micasense Rededge/Sequoia cameras are not overpriced, they are simply targeting a specific niche with specific requirements.
    Having said all that, I'm about to finish a second iteration of the multispectral camera so, it would be very interesting for me to add your feedback on the new specifications. The final product would be considerable cheaper than the products available on the market now.

    0
    None
    AndriyD2

    Question 1 year ago on Step 8

    Hi!

    Soorry fo rmy English...=)

    I'm very interesed it's project

    Where you can buy red and green lens or dilters 12mm for your cameras?

    1 answer
    0
    None
    Kankamuso

    1 year ago

    Hi. Congrats for this fantastic work. Just a couple of questions before I purchase some pieces for my own project.

    a) Regarding focus... have you focused them at the infinite or will I have problems with different flight altitudes?

    b) These are progressive shutter cameras... did you experience any blurring effect or aliasing? what speed did you test it for?

    c) As it is virtually impossible that both cameras get the same FOV you should align and, probably, crop both of them to make them fit.. did you use any software for this?. In such case... on board or after downloading photos to the PC?

    I am planning to build a global shutter camera like yours and those questions are still pending to me.

    Thanks in advance!

    Jose

    1 reply
    0
    None
    MaykeS1Kankamuso

    Reply 1 year ago

    Hi @Kankamuso,


    a) You can definitely focus it at infinite.
    b) Yes, I did experience blurring. It's not suitable for taking pictures while the drone is moving. You could potentially stop over your AOI and take some photographs. The more serious problem is that the cameras would not trigger at the same time; there is a lag between of around 8 miliseconds, which is disastrous if you are moving too fast.
    c) FOV should not be a problem, at least if you are flying over 50m. You don't need to crop the images, but you'll have to align them either using a commercial software like Arcmap, or following few tutorials to use Opencv with C or Python for that purpose. Either way, you should get good results. You'll also have to take in consideration things like vignette effect, row gradient and irradiance calibration if you DO want to make something serious out of it.
    d) Remember that you get what you pay for, so do not get too excited about excellent results with this system. A commercial multispectral camera costs around $5K, at least 10 times more than this simple camera, but on the other hand, you get monochrome sensors, perfect sync, and technical assistance. I still believe that thus system should be enough to intro hobbyists into multispectral imaging, learn the basics, and then move on.

    I hope this helps.