Imprint Invisible Sound and Radio Waves Onto Your Retina: Augmented Reality With Perfect Alignment

40,032

332

32

Published

Introduction: Imprint Invisible Sound and Radio Waves Onto Your Retina: Augmented Reality With Perfect Alignment

About: I grew up at a time when technologies were transparent and easy to understand, but now society is evolving toward insanity and incomprehensibility. So I wanted to make technology human. At the age of 12, I...

With the S.W.I.M. (Sequential Wave Imprinting Machine) you can see otherwise invisible sound waves and radio waves, imprinted onto your retina, onto photographic media, or eyeglass/camera.

This is due to something I call Phenomenological or Phenomenal Augmented Reality, i.e. the AR (Augmented Reality) of physical phenomena.

A unique feature of Phenomenal Augmented Reality is that the alignment (registration) between the direct view and the overlaid information is near-perfect, because the alignment happens naturally, in the feedback loop of the process. In this sense SWIM is a Natural User Interface.

SWIM is a super-simple-to-build form of augmented reality == so simple to build, in fact, that I built the first one 42 years ago, back when I was 12 years old, out of a bunch of old Christmas tree lights that were thrown in the garbage, which I mounted to some scrap wood, driven by a home-made wearable computer I built from surplus parts.

You can build one in less than one hour, for less than $10.

Step 1: Understanding the Historical Context and Background

Above pictures: SWIM, 1974; Visualization of circularly-polarized radio waves, as a form of visual art, Impulse Magazine 12(2), 1985; SWIM on display at the National Gallery of Canada in Ottawa.

In my childhood I noticed a transition from transparent easy-to-understand vacuum tube technologies, where manufacturers printed schematic diagrams inviting end users to understand their technologies, to a more secretive closed-source mentality where manufactures started using ICs (Integrated Circuits), and no longer including schematic diagrams. Not only where the schematics absent but many manufacturers took the extra time to grind numbers of the chips to make things harder to understand. So I witnessed the change from manufacturers providing "maps" (schematics = deliberate openness), to manufacturers providing gouges and scratches (deliberate obfuscation).

This was in the early 1970s, and I wanted to be able to see the otherwise invisible radio waves coming from all these new incomprehensible gadgets.

I had an oscilloscope, but it lacked the bandwidth to view radio waves directly. Moreover, its sweep generator was broken: the dot on the screen would only go up-and-down, not across. So I had only a one-dimensional vertically-oriented display. I discovered that if I connected it to a radio receiver, and placed the receive antenna on top of the oscilloscope, while moving the oscilloscope along, that the radio wave from a stationary transmitter would be "painted" out in space. In this sense, I discovered a concept I call "spacebase" rather than "timebase". The result was a display device that:

  1. makes otherwise invisible sound waves or radio waves visible;
  2. makes them appear in exactly the same place as they actually are -- perfectly aligned with their actual location in space.

Instead of the oscilloscope, I discovered that I could use a linear array of light sources, electrically controlled, to make a giant augmented reality oscilloscope that, when waved through space, made the radio waves visible in perfect alignment with their actual physicality. I built a wearable computer system to control the lights and display a variety of physical quantities such as sound, video, and radio signals.

I completed this project in 1974 and named it the Sequential Wave Imprinting Machine because it made waves visible by "imprinting" them on the retina of the human eye, or upon photographic film, through PoE (Persistence of Exposure), i.e. the time-integrating property of photographic exposure to light.

Step 2: Manifesting the "Mann-Effect": Understanding the Principle of Phenomenal Augmented Reality

Above illustrations: Top illustration, a transmitter at a fixed location generates an electromagnetic wave which is received by a moving antenna. The moving antenna is affixed to a linear display medium that displays the voltage of a received demodulated signal. As the receive antenna moves through space, it displays each point along the electromagnetic wave. Below are shown two alternative embodiments of this effect.

SWIM (Sequential Wave Imprinting Machine) is a device that imprints waves onto your retina or photographic media as you wave it around in space [Mann, IEEE CE 2015 4(4), Cover+pp92-97] [Mann, Wavelets and chirplets, Cover + pp99-128, World Scientific 1992].

This is due to an effect that I discovered in my childhood, when moving a broken oscilloscope (no sweep) back and forth to simulate a timebase by moving through space. What I discovered in spacetime was that the base can be spatial rather than temporal, and thus the spacebase shows waves perfectly aligned with where they actually are in physical reality, when a sensor (such as a receive antenna) is moved back and forth together with the oscilloscope.

A traveling wave may be represented by cos(ωt-kx). A superheterodyne receiver picks up this signal and, let's say it is in tune with the transmitted signal, thus the local oscillator (chalk drawing on left) is cos(ωt). Thus the result, coming down to baseband is cos(-kx) = cos(kx).

Thus what we see traced out by the oscilloscope is a function cos(kx) only of space, not time. The wave is "sitting still" now, and we can see it in exact alignment with where it is in space (no longer traveling at the speed of light).

More generally, I discovered that this concept can generalize to overlay any physical quantity on top of reality, and works especially well when the alignment with reality occurs in the feedback loop of a process -- something my colleagues call the "Mann effect".

I want to now describe a very simple way that you can reproduce this effect.

One of the simplest ways to reproduce this effect is to wave a dotgraph back and forth in front of a Doppler radar while the dotgraph displays the "zero IF" (zero Intermediate Frequency) of the radar.

You can demonstrate this effect for a low cost (under 10 dollars) with something you can build in less than an hour.

Step 3: A Cheap and Easy Phenomenal Augmented Reality Example You Can Prepare in a Few Minutes

The cheapest and simplest way to reproduce the "Mann effect", in visualization of electromagnetic radiation is a low cost radar. For example the HB100, selling for $3.99, can be prepared in a couple of minutes: connect red and black wires to the ground and +5 volt terminals of the HB100 module. Connect yellow and black wires to the ground and "IF" (Intermediate Frequency) terminals.

I usually like to twist the wires together to reduce common-mode interference.

The yellow and black wires are then connected to an amplifier of adjustable gain (op amp) that feeds into the dotgraph display.

In the original system in the 1970s I used a long chain of comparators driving silicon controlled rectifiers to control light bulbs.

Then in 1980 the LM3914 chip was introduced which made this a whole lot easier.

Today you can get LM3914 chips in lots of 50 for only $11.18, i.e. a price of less than 23 cents per chip.

So for very little cost you can drive 10 LEDs and reproduce the effect.

And you can easily cascade the chips together, for example, with 50 chips you can drive 500 lights which gives you a 500 by infinity pixel AR display for $11.18 plus cost of LEDs and a few other small miscellaneous components.

Let's start with just 10 LEDs, and then do 3 chips in a row (30 LEDs) and then 10 chips in a row (100 LEDs).

Step 4: Building the SWIM (Sequential Wave Imprinting Machine)

Begin with just one LM3914 chip as shown in the chalk drawing above. You might want to refer to the LM3914 datasheet.

The HB100 radar is fixed stationary in the environment, and the linear array of LEDs is waved back and forth. It functions as a "target" for the radar, thus returning a Doppler shifted radio signal that is displayed on the dotgraph.

An amplifier of adjustable gain can be used to set the sensitivity of the SWIM depending on how far the dotgraph is from the radar. If you are finding that the gain is insufficient, you can increase the gain of the amplifier, or you can also increase the radar cross section of the target by affixing a piece of metal or other radar reflective material to the dotgraph.

If you want a higher-resolution SWIM, you can cascade the LM3914 chips as shown in the second chalk drawing. Here, rather than using an internally generated reference, we use an external 5 volt reference provided by a 78L05 voltage regulator, which gives us more flexibility.

The picture shown at lower center shows my assembly of three LM3914 chips to drive 30 LEDs. The HB100 radar is shown at the top of the picture, and is what is generating the radio waves that we see.

The picture shown in the lower right is a picture I took of a breadboard setup put together by a student, with 10 LM3914 chips driving 100 LEDs. In my critique of this lesson, I noted that the LEDs should be more carefully arranged in a straight line, because slight flaws in their alignment result in massive disruption of the shape of the waveform. The use of a circuit board with surface mount LEDs resulted in improved performance regarding alignment. Nevertheless, you can quickly and easily implement this "Persistence of Exposure Phenomenal Augmented Reality Effect".

As a teaching tool, this also illustrates the concept of "sitting waves" as compared with standing waves.

Step 5: "Sitting Waves" and Learning With SWIM

I'll conclude with a new way of thinking about waves.

Standing waves result from the superposition (addition) of two waves traveling in opposite directions, as with a skipping rope or a violin string.

The effect that I have discovered and presented is something different. I like to call it "sitting waves" because the waves appear to "sit still" [Mann, IEEE Consumer Electronics 5(1), 2015dec02, pp 33-143].

Sitting waves are formed by the product of two waves traveling in the same direction.

The image at left is one that I generated from the example shown in Wikipedia (Wikimedia Commons). The image on the right is one that I scanned from a roll of film captured by my SWIM (1974).

Over the past 42 years SWIM has evolved from 35 incandescent light bulbs totaling up to 2500 watts, to 1100 LEDs (i.e. to exceed 1080p by infinity HDTV resolution) on a 7 foot long stick, and finally, to an eyeglass based device that let's us see radio waves scanned onto the retina of the eye, thus giving rise to the new concept of Phenomenal Augmented Reality -- allowing us to see physical phenomena overlaid onto the real world.

For more information on Metasensing (the sensing of sensors and sensing their capacity to sense), see Janzen and Mann 2015.

For more information on Metaglasses, see our company website, http://getameta.com

For more information on some of the things you can visualize with SWIM, including sound waves, audio, video, radio, water, and any other kind of field, see http://wearcam.org/fieldary.pdf

Have fun and if you get as far as just building the simplest version with 10 LEDs, which you should be able to do in less than 1 hour, for less than $10, please reply with pictures and I'll give you some constructive criticism and useful feedback.

11 People Made This Project!

Recommendations

  • Water Contest

    Water Contest
  • Creative Misuse Contest

    Creative Misuse Contest
  • Oil Contest

    Oil Contest

32 Discussions

I am trying to build this with an arduino, and I am having a hard time with the pinout. Can you post the pinout for arduino, or maybe edit the instructable and put it on.

4 replies

You just need to write an Arduino "sketch" that reads from one of the analog inputs, and drives a linear array of lights on the output, where the illuminated light varies in proportion the the voltage input.

Do you have already some lights connected to an Arduino?

Alternatively you could just use one multicolor LED and vary the color based on the input voltage.

Lots of fun things are possible with an Atmel AVR for example!

I am getting confused with arduino, is there any circuit diagram, How can i use arduino and analog sensors like ultrasonic sensors for input, Could some provide me more details about this.

A simple bargraph in fact. Oh, cos(a)=cos(-a), for me there's no "negative" angle, only opposite, but maybe I'm wrong.

Yes, cos is a symmetric function, so the cosine of a negative angle is the same as the cosine of the absolute value of that angle.

Sine is antisymmetric so the opposite is true, i.e. sin(-a) = -sin(a).

So there are 2 wires in the IF one of them goes through an amplifier then to input in lm3914 in pin number 5 right? ( is it for every number 5 pin we should connect the IF or only one) and where do the other wires in the hb100 go (is it to a battery source and which wires exactly?) and for the led should the short ends be in front or the long positive ends? and the amplifier it is a piece right like the lm741 but how to connect to it? and at what speed should we move the SWIM so it is visible to the naked eye?

Thank You Sooooo Much ( If you can send in a message your number that will be great )

We have a physics competition after 2 days and we need to finish the circuit we already made the circuit but the hb100 was our only problem or barrier

I made the whole circuit but the problem is where should I connect the HB100 and to what exactly ( In the instructable it is mentioned only about the yellow and the black wires where does the other 2 go) and is the amplifier the machine or the piece( what is connected to it?) and does the led blink one after another because of the hb100 or is it an Arduino used ?

Please we need this information as soon as possible it is very urgent as we have a competition after 2 days exactly please help us..........

Anyone can help please.....

Instructables 1.jpgInstructables 2.jpgInstructables 3.jpgInstructables 4.jpg
1 more answer

Yes, you need another wire connected to the terminal that I labeled "IF". That wire goes through an amplifier to the LM3914 input. The choice of amplifier gain is up to you, depending on how sensitive you want to make it (e.g. how far away you want to pick up objects, etc.).

By the way, let me know why the rush, e.g. what is the deadline (usually I do these things just for fun, on my own clock, whenever I feel like it ... to be done).

Hi!

What do you think is better -
following other people's instructions and building things they invented
or coming up with something by yourself ? In the second case how should I start ?

Thank you.

0
user
mknorr

1 year ago

I'm assuming this is actually just a persistence of vision thing, rather than the weird title?

1 reply

There's a lot of controversy with many people believing there's no such thing as "persistence of vision", disproved in 1912 by Wertheimer[2] https://en.wikipedia.org/wiki/Persistence_of_vision

Since there's no such thing as persistence of vision, let me instead claim that there's at least a concept of persistence of exposure.

PoE (Persistence of Exposure) when combined with perfect alignment, as a form of real augmented reality, gives us the ability to see otherwise invisible real world phenomena, as well as complex-valued signals from the physical world around us.

See Figure 7 in http://wearcam.org/kineveillance.pdf

If you can think of a better title, please let me know.

I want to use Osaic x 3d holoaudio by Jason rigs with your mixed reality head gear, can we make this happen?

Steve Mann may be emailed at epilab@jeyetap.org

Also see; holoaudio. www. rexreseach.com

1 reply

Why "mixed reality"?

Mixed reality only gives you the virtuality axis, but you might also want the mediality axis.

Compare figures 1 and 2 in https://en.wikipedia.org/wiki/Reality%E2%80%93virtuality_continuum
and see also https://en.wikipedia.org/wiki/Computer-mediated_reality

I feel like I'm close. One LED will illuminate, but not the others unless I shake the HB100 and then the others will begin to flicker. Seems like a great project, but a few holes in how it's presented which make it hard to debug. My hypothesis is I need to add the amplifier, but it's not clear what type of amplifier and I can't see any on the pictures.

This instructable has the opportunity to be clearer by:

1. A photo of the entire set up, not just the bread board. How is the HB100 wired to the breadboard?

2. What amplifier should be used?

3. A detailed materials list would be ideal.

4. Pictures and Schematics that aligned. Ie a schematic for 10 leds and a picture of the actual 10 led circuit with HB100.

Thanks! Looking forward to clicking that 'i made it' button once I have it up and running, but could definitely use help.

Here's my current setup as well.

IMG_8246.JPG
1 reply

Yes, an amplifier of adjustable gain helps greatly, e.g. to adjust for the range or distance away. One or 2 stages of op amp should work ok for close range.

hi,

im a little confused about how to set up the HB100 with the LM3914.

Does the IF (on the HB100) go to the SIG (on second picture under: Step 4: Building the SWIM).

If so, IF first goes through and amp right?

thanks

2 replies

Yes, you need an amplifier and you need to select the gain of the amplifier appropriate for how far away you want to pick up the signal.

In radar, the signal falls off as 1/(r^4) where r is the radius away. So there's a wide range of gain adjustment needed. You also need to be able to adjust the bias (offset) so the signal will fall between 0 and 5 volts on the output.

A good starting point is to connect the HB100 to an oscilloscope or AC voltmeter and see what the signal level is like for your operating distance and operating parameters. Then you can design your amplifier to suit these conditions.

If your breadboard has a lot of metal in it, you'll get a stronger signal.

If your breadboard is backed with metal (like some breadboards are) you'll get a stronger signal. You can also add metal backing to strengthen the signal. If the signal is stronger you won't need as much gain in your amplifier which might simplify its design.

May I ask how you made your cover image animated? I tried using an animated gif for one of my cover images and it only displays the first frame. Does it need to be a specific size?