How to interface digital displays with your PC

This instructable shows you how to read information from your temperature gauge, water heater, barbecue temperature, coke machine, laundry timer, etc. from your PC. While most modern sensors today have the ability to interface to PC based systems at lot of older devices do not. They do, however, offer a digital display that can be read by humans. You can use RoboRealm to read these digits from those devices and use that information in new ways on your PC.

The setup requires a camera focused on the device to read with the image being passed back to your desktop/laptop computer for processing. Below is a DLink DCS900 that provides a wireless link to access the image from the temperature panel seen attached to the wall. This instructable is shows how to convert the image into a meaningful number that you can use.

You can download RoboRealm for FREE by going to RoboRealm.

Step 1 Camera View

Camera View

(image#1)

The first task is to segment the digits from the rest of the scene. Note that there are many ways
to accomplish this task and the specific modules you may use may differ from this approach.

To begin our segmentation we need to remove the lighting issue. To do this we use the Adaptive_Threshold module. Adding this module converts the above image to (image#2)

The parameters of the adaptive thresholding should be set to approximately the width of one of the digits in order to extract out the digit. We found 20 to be an appropriate window size.

One can easily see that the adaptive threshold does a great job of eliminating any lighting issues and highlights the digits from the background.

But you can notice we've still got some cleanup to do ....

Comments

[Tinker83]

if your equipment has a 7segment display, why not just tap into that, and make a 232 or usb dongle? you could get /real/ complicated and use a PIC and put it on your home network too.

[davidprosser]

I wouldn't be too suprised if it was all integrated onto one chip now, so that might not be possible in some cases i guess.

[Alathald]

I was thinking the same thing, though I'd plug into the binary output (beginning of the circuit that converts binary to 7segment or the even more complicated digital display) rather than the 7segment display. It would make it much easier to read out to the computer. If you want to get real elaborate, you could bypass the switches and buttons to control you thermostat, security system, microwave, refrigerator, etc. and use a serial port to easily control all of those things with your computer. Just be sure that if the bypass circuit is removed (say if the serial plug is removed or computer is off), the circuit goes back to how it was.

[josh92176]

I think that the idea was to demonstrate what their new software was capable of. -josh

[duncant20196]

I like! I'm going to try. You don't have any way to output the data at certain intervals? I'd like to use this to chart my electricity meter on an hourly bases...

[DeusXMachina]

At first I though, oh he just took a webcam and pointed at a display, but that program is really cool! It's amazing that they've pretty much open source recognition software. A few things: - Just a thought, if anyone ran into light contrast issues, you could make a black box for it, and supply your lighting, that way it'll be 100% constant (a lot of cams i know have built in "I am on" LEDs it wouldn't be much of a mod to put in a bright white) - The next step to go with this is integrate it into a system a la SmartHouse and make your house do things for you. (Kitchen! Sudo make me a sandwich!)

[MadMechanicMike]

(removed by author or community request)

[mr_step]

(removed by author or community request)

[MadMechanicMike]

(removed by author or community request)

[fungus amungus]

Mike - 1) opinions are fine, but are better when backed up with some kind of reason. That's kind of a clause of the "be nice" policy 2) so why can't you ignore this instructable if it offends you so? 3) makes sense since you offer no explanations of your own 4) He never said he was sick of people, just people who make rude comments. Others have found constructive things to say in the comments so if you have nothing to offer this may not be the place for you.

[mr_step]

(removed by author or community request)

[gormly]

you are fostering these kind of comments by replying to them.

[thearchitect]

This is sweet, thanks for sharing!.. K.

[lilpunk1302]

I was hoping for this to be more of a what to do with the numbers AFTER you've captured them, future instructable, I presume( btw, first person to combine this into some form of a Konfabulator(I still call it that, you yanks call it Yahoo! Widget engine) widget, that shows room temp, gets huge props from me.

But yeah, also, wouldn't it be easier to just crop the image to get rid of the other stuff, or move the camera closer?

Also, I disagree with your statement,

"You might be able to but you'd then have a hard time putting the different pieces together since you'd be matching two parts of each digit at a time. In addition to that the matches would be less accurate since matching the bottom and top part of the shape would be a more generic shape and thus less distinguishable from the each other. Filling the gaps makes each shape much more distinct and thus easier to match."

Since the software is matching the image it sees with a preset list of images, it doesn't matter if the image is in 5000 pieces(but at that point it'd be a little too intricate and the adjustments would mess it up, NOT THE POINT) 5000 pieces or 2, the soft. is still doing the same thing:

10 If X = Y, Print Y
20 else
30 if X = Z, Print Z

You get the point.

[frollard]

Thats a really neat process... If it's just looking for certain shapes in the end, would it not be okay to just leave the 'gaps' in the 7-segment display digits?

[RoboRealm]

You might be able to but you'd then have a hard time putting the different pieces together since you'd be matching two parts of each digit at a time. In addition to that the matches would be less accurate since matching the bottom and top part of the shape would be a more generic shape and thus less distinguishable from the each other. Filling the gaps makes each shape much more distinct and thus easier to match.

[richelton]

So are you saying that your software isn't just taking a 'known region' from the camera image and pattern-matching that area to a bitmap database--which would be useless if the 'known' camera placement moved even slightly such as being bumped--but instead it is leaning to 'read' those digits in the incoming image regardless of size or angle of incidence? If so, THAT is WAY cool. Thanks for the instructable.

[creatordave]

i agree with MadmechanicMike There is not much use for this

[josh92176]

This is sooo cool!

[joejoerowley]

Very Cool! Great instructable!