Introduction: Using a LED Matrix As a Scanner
Ordinary digital cameras work by using a large array of light sensors to capture light as it is reflected from an object. In this experiment, I wanted to see whether I could build a backwards camera: instead of having an array of light sensors, I have just a single sensor; but I control each of 1,024 individual light sources in a 32 x 32 LED matrix.
The way it works is that the Arduino illuminates one LED at a time, while using the analog input to monitor changes in the light sensor. This allows the Arduino to test whether the sensor can "see" a particular LED. This process is repeated for each of the 1,024 individual LEDs rapidly to generate a map of visible pixels.
If an object is placed between the LED matrix and the sensor, the Arduino is able to capture the silhouette of that object, which is lit up as a "shadow" once the capture is complete.
BONUS: With minor tweaks, the same code can be used to implement a "digital stylus" for painting on the LED matrix.
Step 1: Parts Used in This Build
For this project, I used the following components:
- An Arduino Uno with Breadboard
- 32x32 RGB LED matrix (either from AdaFruit or Tindie)
- 5V 4A Power Adapter (from AdaFruit)
- Female DC Power Adapter 2.1mm jack to Screw Terminal block (from AdaFruit)
- A clear, 3mm TIL78 phototransistor
- Jumper wires
AdaFruit also sells an Arduino shield which can be used instead of jumper wires.
As I had some Tindie credits, I got my matrix from Tindie, but the matrix from AdaFruit seems to be identical, so either one should work.
The phototransistor came from my decades old collections of parts. It was a clear 3mm part labeled as a TIL78. As far as I can tell, that part is meant for IR and comes either a clear case or a dark casing that blocks visible light. Since the RGB LED matrix puts out visible light, the clear version must be used.
This TIL78 appears to have been discontinued, but I imagine this project could be made using contemporary phototransistors. If you find something that works, let me know and I will update this Instructable!
Step 2: Wiring Up and Testing the Phototransistor
Normally, you would need a resistor in series with the phototransistor across power, but I knew that the Arduino had the ability to enable an internal pull-up resistor on any of the pins. I suspected that I could take advantage of that to hook up the phototransistor to the Arduino without any additional components. It turned out my hunch was correct!
I used wires to connect the phototransistor to the GND and A5 pins on the Arduino. I then created a sketch that set the A5 pin as an INPUT_PULLUP. This is normally done for switches, but in this case it provides power to the phototransistor!
#define SENSOR A5 void setup() { Serial.begin(9600); pinMode(SENSOR, INPUT_PULLUP); } void loop() { // Read analog value continuously and print it Serial.println(analogRead(SENSOR)); }
This sketch prints values to the serial port corresponding to the ambient brightness. By using the handy "Serial Plotter" from the "Tools" menu of the Arduino IDE, I can get a moving plot of ambient light! As I cover and uncover the phototransistor with my hands, the plot moves up and down. Nice!
This sketch is a nice way to check whether the phototransistor is wired up with the right polarity: the phototransistor will be more sensitive when hooked up one direction versus the other.
Step 3: Wiring Up the Matrix Ribbon Cable to the Arduino
To wire up the matrix to the Arduino, I went through this handy guide from Adafruit. For convenience, I pasted the diagram and pinouts into a document and printed a quick reference page to use while wiring everything up.
Take care to ensure the tab on the connector matches the one in the diagram.
Alternatively, for a cleaner circuit, you could use the RGB matrix shield that AdaFruit sells for these panels. If you use the shield, you will need to solder in a header or wires for the phototransistor.
Attachments
Step 4: Connecting the Matrix
I screwed down the fork terminals on the matrix power leads to the jack adapter, making sure the polarity was correct. Since part of the terminals were left exposed, I wrapped the whole thing up with electrical tape for safety.
Then, I plugged in the power connector and ribbon cable, being careful not to disturb the jumper wires in the process.
Step 5: Install the AdaFruit Matrix Library and Test the Matrix
You will need to install the "RGB matrix Panel" and the AdaFruit "Adafruit GFX Library" in your Arduino IDE. If you need help doing this, the tutorial is the best way to go.
I suggest you run some of the examples to make sure your RGB panel works before proceeding. I recommend the "plasma_32x32" example as it is quite awesome!
Important note: I found that if I powered up the Arduino before I plugged in the 5V supply to the matrix, the matrix would light up dimly. It appears that the matrix tries to draw power from the Arduino and that's definitely not good for it! So to avoid overloading the Arduino, always power up the matrix before you power up the Arduino!
Step 6: Load the Matrix Scanning Code
Now, load the Arduino sketch for scanning the matrix. I have provided the Arduino source code. Once you have loaded it, you should see a flash of light sweeping the matrix every few seconds.
If you bring the phototransistor close to the matrix, you should see that LEDs near the phototransisor are illuminated.
Try moving the phototransistor further and closer to the matrix to see how it behaves like a flashlight. If you place an object on the matrix and hold the phototransistor above it, you should be able to capture a "shadow" of the object!
There are a few parameters you can tweek in the code:
static constexpr uint8_t READINGS_PER_PIXELS = 1; static constexpr uint8_t THRESHOLD = 15; static constexpr bool INVERT = false; static constexpr bool CLEAR = true;
If the detection isn't working well, you can try increasing READING_PER_PIXEL and THRESHOLD. Set CLEAR to false to get a paintbrush like effect where each new scan adds more "paint" to the canvas. Set INVERT to true for a negative effect, where the matrix is normally lit, but the phototransistor darkens what it can "see".
Attachments
Step 7: How the Code Works: Detecting a Visible Versus an Obscured LED
After some experimentation, I found that the best way to detect whether a pixel is visible from the sensor was to take pairs of analog readings from the sensor: one reading when the LED is off and another reading when the LED is on. If the sensor cannot see the LED, then the readings will be practically the same; if the sensor can see the LED, the readings will be different.
To increase the sensitivity, I take multiple readings while using the OE (output enable) line to turn the LED on and off. I use two accumulators to sum the readings taken when the LED is off vs when the LED is on. I then compare difference between these two sums against a threshold to determine whether the value exceeds some threshold, giving me a decision on whether the LED is within line of sight from the sensor.
Step 8: How the Code Works: Scanning the Matrix
For scanning the matrix, I manipulate the data lines directly, as this is more efficient than using the AdaFruit library to paint pixels. To disable the AdaFruit library, I shut off interrupts prior to beginning a scan. Since the AdaFruit library works by using interrupts, this allows me to take control of the matrix temporarily.
The matrix has six shift registers, corresponding to the R, G and B colors of an entire rows of pixels in the upper and lower halves of the matrix. There is also a latch control line (LAT), which copies the values from the shift register to the LED drivers. Four address lines (A, B, C and D) select which rows of the upper and lower half are active and an OE (output enable) turns the LED drivers on and off.
In normal operation, the latch keeps a row illuminated with previous values while a new replacement row is shifted into the shift registers. Once the entire row has been loaded, the new values are latched and the process repeats for the next row. This manner of updating requires an entire row to be written, even if only one pixel needs to be written.
For scanning, I chose to take advantage of the shift register, but not the latching functionality. I set the latch control line (LAT) to HIGH, making it so the contents of the shift register are loaded into the LED drivers immediately on each CLK pulse. Once I illuminated the first pixel, each pulse on the clock line (CLK) causes the lit pixel to get pushed down the row to the next pixel. In so doing, I can push the illuminated pixel down the row to scan the entire row.
For each position in the row, I use the OE (output enable) line to strobe the pixel on and off while taking the readings from the light sensor. Once I have enough readings to make a determination of whether that pixel is visible or not, I write a color to the AdaFruit library's framebuffer (while scanning, I can still write pixel values to the library's framebuffer, even though the library is temporarily suspended).
After all the rows have been scanned. I re-enable interrupts, which causes the AdaFruit library to refresh the matrix with the data in the buffer until I am ready for the next scan.

Second Prize in the
Arduino Contest 2019
31 Comments
3 years ago
Has anyone tried this with a normal display? (e.g: lcd)
4 years ago
Thank you so much. It gave me some ideas.
Question 4 years ago
O.K. maybe I'm missing something obvious here, but where's the code? I've looked top to bottom several times and there doesn't seem to be any link to the final code. Maybe I'm going nuts. I dunno....
Answer 4 years ago
That's funny, you're the first one to notice! Anyhow, I fixed it now. It's on step 6.
Reply 4 years ago
Well I've given it a go but something is wrong. All that I get is a single pixel flashing down at one corner of the matrix panel. There is no light sweep occurring at all. I've been careful to use all of the same parts that you used. I ordered the 32x32 matrix panel through Adafruit. I'm using an authentic Arduino Uno. I'm even using TIL78 phototransistors just like you did that I ordered through eBay and they are working fine. The serial plotter sketch show that my phototransistors are good. All of the RGB matrix panel example sketches are working perfectly as well (the plasma sketch is wild!). But, when I load the BasicMatrixScanner code......nothing. Is there maybe a bug that got into your code? Perhaps you could double check. Thanks!
Reply 4 years ago
It's time to get your troubleshooting hat on and write simple sketches to understand where the problem is. The fact that the Adafruit library works is a good first step. Troubleshooting further will require you to take one step at a time and try a few sample sketches. I'll add a few additional steps to this Instructable with ideas.
Reply 4 years ago
Unfortunately, the Instructables website is broken today and I can't add new steps. But here is some code for you to try (tabs are lost in comments, so this will look messy):
#define CLK 8
#define OE 9
#define LAT 10
#define A A0
#define B A1
#define C A2
#define D A3
#define R1_PIN 2
#define G1_PIN 3
#define B1_PIN 4
#define R2_PIN 5
#define G2_PIN 6
#define B2_PIN 7
void setup() {
// Configure all the matrix control lines as outputs
pinMode(CLK, OUTPUT);
pinMode(LAT, OUTPUT);
pinMode(OE, OUTPUT);
pinMode(A, OUTPUT);
pinMode(B, OUTPUT);
pinMode(C, OUTPUT);
pinMode(D, OUTPUT);
pinMode(R1_PIN, OUTPUT);
pinMode(G1_PIN, OUTPUT);
pinMode(B1_PIN, OUTPUT);
pinMode(R2_PIN, OUTPUT);
pinMode(G2_PIN, OUTPUT);
pinMode(B2_PIN, OUTPUT);
// Enable the output
digitalWrite(OE, LOW);
// Put a RED pixel in the top row, and a green pixel in the BOTTOM row:
digitalWrite(R1_PIN, HIGH);
digitalWrite(G1_PIN, LOW);
digitalWrite(B1_PIN, LOW);
digitalWrite(R2_PIN, LOW);
digitalWrite(G2_PIN, HIGH);
digitalWrite(B2_PIN, LOW);
// Select row zero
digitalWrite(A, LOW);
digitalWrite(B, LOW);
digitalWrite(C, LOW);
digitalWrite(D, LOW);
// Clock and latch a row of pixels
for(uint8_t pixels = 0; pixels < 32; pixels++) {
digitalWrite(CLK, HIGH);
delay(10);
digitalWrite(CLK, LOW);
delay(10);
}
// Latch the row to the LED output
digitalWrite(LAT, LOW);
delay(1000);
digitalWrite(LAT, HIGH);
}
void loop() {
}
If this doesn't work, try changing "digitalWrite(OE, LOW)" to "digitalWrite(OE, HIGH)". If that still doesn't help, try exchanging the LOW and HIGH in the "digitalWrite(LAT, LOW)" and "digitalWrite(LAT, HIGH)" lines. Let me know if and whether you get some rows to light up.
Reply 4 years ago
I'm not sure what outcome I would be expecting but the unmodified code as written results in two solid lines: Red at row zero and Green at row 16 (if that's the way they're numbered). If I change OE from LOW to HIGH then I get a single very brief flash on the same two corresponding rows. I'm guessing that's the intended result? I also tried the other two possible combinations, one of which again gives me the solid two lines and the other which gives the two briefly flashing rows. Finally I tried swapping in the alternate LED row output latch block and that didn't change the outcomes so it's probably not needed.
Reply 4 years ago
eburman, a red and green line is what I was expecting. This confirms that at least the basics are working. Here is another slightly modified sketch for you to try out. It should cause a single red and green pixel to march across the rows:
#define CLK 8
#define OE 9
#define LAT 10
#define A A0
#define B A1
#define C A2
#define D A3
#define R1_PIN 2
#define G1_PIN 3
#define B1_PIN 4
#define R2_PIN 5
#define G2_PIN 6
#define B2_PIN 7
void setup() {
// Configure all the matrix control lines as outputs
pinMode(CLK, OUTPUT);
pinMode(LAT, OUTPUT);
pinMode(OE, OUTPUT);
pinMode(A, OUTPUT);
pinMode(B, OUTPUT);
pinMode(C, OUTPUT);
pinMode(D, OUTPUT);
pinMode(R1_PIN, OUTPUT);
pinMode(G1_PIN, OUTPUT);
pinMode(B1_PIN, OUTPUT);
pinMode(R2_PIN, OUTPUT);
pinMode(G2_PIN, OUTPUT);
pinMode(B2_PIN, OUTPUT);
// Enable the output
digitalWrite(OE, LOW);
// Put a RED pixel in the top row, and a green pixel in the BOTTOM row:
digitalWrite(R1_PIN, HIGH);
digitalWrite(G1_PIN, LOW);
digitalWrite(B1_PIN, LOW);
digitalWrite(R2_PIN, LOW);
digitalWrite(G2_PIN, HIGH);
digitalWrite(B2_PIN, LOW);
// Select row zero
digitalWrite(A, LOW);
digitalWrite(B, LOW);
digitalWrite(C, LOW);
digitalWrite(D, LOW);
// Clock and latch a row of pixels
for(uint8_t pixels = 0; pixels < 32; pixels++) {
digitalWrite(CLK, HIGH);
delay(10);
digitalWrite(CLK, LOW);
delay(10);
// Latch the row to the LED output
digitalWrite(LAT, LOW);
delay(1000);
digitalWrite(LAT, HIGH);
// Turn off the LED input
digitalWrite(R1_PIN, LOW);
digitalWrite(G2_PIN, LOW);
delay(500);
}
// Latch the row to the LED output
digitalWrite(LAT, LOW);
delay(1000);
digitalWrite(LAT, HIGH);
}
void loop() {
}
Reply 4 years ago
Bazinga! Yes that is working! I did have to correct a minor error in your code. The final latch block was outside the loop and it wasn't working right so I put it back in. I assume that's what you had intended because the red and green pixels are marching across the rows from one side to the other now. It's interesting to note that if I don't make sure to completely unplug the matrix board from all power sources before loading new code it will behave very erratically. I'm guessing that the shift registers have to be reset to their initial state or they will still be set to whatever state they were in before the new code was run. By the way, thanks for taking time to work through this with me. I'm learning a lot about how shift registers work by going through these exercises.
Reply 4 years ago
That's great! I think I understand why my original code wasn't working now. It has to do with the LAT (latch) input. The LAT signal causes the panel to copy whatever data is in the shift register into the output register that drives the LEDs, but I made an incorrect assumption about how it works. In my code, I leave the latch HIGH while pushing the pixels down the rows -- my panel copies continuously while LAT is held HIGH, but in yours it appears to only happens when LAT *changes* from either LOW to HIGH, or when it changes from HIGH to LOW. I can't tell which direction, so here is some additional code to answer the question:
#define CLK 8
#define OE 9
#define LAT 10
#define A A0
#define B A1
#define C A2
#define D A3
#define R1_PIN 2
#define G1_PIN 3
#define B1_PIN 4
#define R2_PIN 5
#define G2_PIN 6
#define B2_PIN 7
void setup() {
// Configure all the matrix control lines as outputs
pinMode(CLK, OUTPUT);
pinMode(LAT, OUTPUT);
pinMode(OE, OUTPUT);
pinMode(A, OUTPUT);
pinMode(B, OUTPUT);
pinMode(C, OUTPUT);
pinMode(D, OUTPUT);
pinMode(R1_PIN, OUTPUT);
pinMode(G1_PIN, OUTPUT);
pinMode(B1_PIN, OUTPUT);
pinMode(R2_PIN, OUTPUT);
pinMode(G2_PIN, OUTPUT);
pinMode(B2_PIN, OUTPUT);
// Enable the output
digitalWrite(OE, LOW);
// Start all colors off
digitalWrite(R1_PIN, LOW);
digitalWrite(G1_PIN, LOW);
digitalWrite(B1_PIN, LOW);
digitalWrite(R2_PIN, LOW);
digitalWrite(G2_PIN, LOW);
digitalWrite(B2_PIN, LOW);
// Select row zero
digitalWrite(A, LOW);
digitalWrite(B, LOW);
digitalWrite(C, LOW);
digitalWrite(D, LOW);
// Start with LAT low
digitalWrite(LAT, LOW);
// Load and shift in red pixels
digitalWrite(R1_PIN, HIGH);
digitalWrite(G1_PIN, LOW);
digitalWrite(B1_PIN, LOW);
digitalWrite(R2_PIN, HIGH);
digitalWrite(G2_PIN, LOW);
digitalWrite(B2_PIN, LOW);
digitalWrite(CLK, HIGH);
digitalWrite(CLK, LOW);
// Trigger LOW to HIGH transition on LAT
digitalWrite(LAT, HIGH);
// Load and shift in green pixels
digitalWrite(R1_PIN, LOW);
digitalWrite(G1_PIN, GREEN);
digitalWrite(B1_PIN, LOW);
digitalWrite(R2_PIN, LOW);
digitalWrite(G2_PIN, GREEN);
digitalWrite(B2_PIN, LOW);
digitalWrite(CLK, HIGH);
digitalWrite(CLK, LOW);
// Trigger HIGH to LOW transition on LAT
digitalWrite(LAT, LOW);
}
void loop() {
}
You should either see red pixels, or red *and* green pixels, after this code runs. Once I know the exact behavior of the LAT I should be able to modify my original scanning code so it works with your panel.
Incidentally, I provided a link to the Adafruit panel because I figured most people would want to buy everything from one place, but my panel was originally from Tindie. My guess is that these panels are built slightly different and my code just made assumptions that weren't valid for the Adafruit panel. It's a good thing you and I are able to work this out and hopefully get the code to works no matter what panel is used! I think we are close!
Reply 4 years ago
And the result is: green pixel and red pixel (column 31 green, column 30 red) on row zero. Same again on row 16. Also, corrected GREEN to HIGH. I usually order inexpensive Chinese parts through eBay or Banggood or Aliexpress. This time I ordered the panel through Adafruit. Much more expensive that way but I know that they offer consistantly high quality parts. Since this was a more complex project I wanted to eliminate any questions about the part being defective or incompatible. I guess it didn't work out that way though. But I I'm learning more this way.
Reply 4 years ago
Ha, ha. Sorry, I keep making mistakes in the code I send you. Good thing you're picking up on it enough to correct those :) Anyhow, the results show that your matrix copies the values when LAT goes from HIGH to LOW, which actually is fairly typical for logic circuits, so it does not surprise me that much. I went ahead and modified BasicMatrixScanner.ino with that knowledge. Download the new one and see if it works now!
Anyhow, I'm glad we ended up with different matrices, as otherwise I never would have learned the "proper" way to drive these! Thanks for your patience in sorting this through!
Reply 4 years ago
Ta Da! You did it! It seems to be working now just as originally intended. Strong work!
Reply 4 years ago
Glad to hear! Thanks for the help in making the code better!
4 years ago
the idea is brilliant, but now someone needs to put this concept into something that blows everyone's mind!
Reply 4 years ago
These panels are tillable and meant for large video walls. If someone had a few grand to burn, they could make something large enough to fill a stage and capture an entire band as they played. That would certainly be impressive!
Reply 4 years ago
OK now you blew my mind.
4 years ago on Step 3
This looks very cool! Anyone want to try making a simulated Chladni Plate with this?
Reply 4 years ago
The plasma demo is pretty cool, but the patterns are not symmetrical as they would be in a Chladni plate (BTW, I had to look that up!)