Tablet/Phone As Arduino Screen, and a $2 Oscilloscope

Intro: Tablet/Phone As Arduino Screen, and a $2 Oscilloscope

While one can buy a cheap 320x240 LCD touch screen for an Arduino-based project, it can be more convenient--especially for prototyping and testing a sketch--to use a tablet or phone as both a touch screen and a power source for a project. You can have a much higher resolution and better looking display on your Android device (e.g., all your lines will be anti-aliased).

The Android-based screen could be connected via USB Serial, Bluetooth or WiFi (e.g., ESP8266).

To that end I wrote VectorDisplay (source here), an Android app that pairs with an Arduino library that implements a large subset of the Adafruit GFX interface. You can write code that can then be easily ported to use a standalone screen, or keep using the sketch with an Android-based display. And you can send commands from the Android app to control the Arduino sketch. The Arduino library is largely board-independent: it should work with any board that provides a USB serial port object named Serial or with an ESP8266 over WiFi or with Bluetooth (pair your board first).

As a proof of concept application, I ported the bare-bones STM32-O-Scope project to use VectorDisplay in place of the ILI9341 display. The result is a (rough around the edges) portable, battery-powered 1.7MS/s oscilloscope that requires nothing more than a $2 STM32F103C board (using the libmaple-based Arduino core), two wires, a USB OTG cable and an Android device. Of course, all you get with this is a range from 0 to about 3.3V.

STEP 1: Install Software

I assume you have an Arduino IDE set up for your favorite board and that your favorite board has a USB serial interface.

Go to Sketch | Include library | Manage libraries. Put "VectorDisplay" in the search area and click on "Install" once it's found.

Download the library zip from here.

Unzip into a folder inside your Arduino/libraries folder.

Download VectorDisplay from Google Play and install it on your Android device. You may need to enable installation from unknown sources on your Android device. The Android app uses the UsbSerial library and the starting point was one of the example apps for the library.

STEP 2: Demo Sketch

Connect your board (in upload mode if need be) to your computer and go to File | Examples | VectorDisplay | circles in your Arduino IDE. Click on the upload button (right arrow).

Start the VectorDisplay app on your Android device. Plug your board into the Android device via USB OTG cable. (If your board has a USB micro port, make sure your USB OTG host side goes to the Android device). You should now get a permission query for VectorDisplay. Press OK.

If all goes well, VectorDisplay will now show two buttons on the left side of the screen: Circle and Color. Pressing Circle draws a random circle on the screen and Color changes the color to a random color before the next circle.

If you look at the circles sketch in the IDE, you will see that the serial vector display is declared with:

SerialDisplayClass Display;

and then initialized in setup() with:

Display.begin();

Then command buttons are requested with Display.addButton(). Then loop() calls Display.readMessage() to look for commands being sent via the command buttons.

By default, the coordinate system for the display is 240x320. However, lines and text are all drawn using the full resolution of your Android device screen, with antialiasing for good appearance. That's why the app is called VectorDisplay.

STEP 3: API

The API in the library is in the VectorDisplay.h file. You need to first initialize a Display object. For USB use, do that with:

SerialDisplayClass Display;

Initialize the connection with Display.begin().

There are two sets of methods available in the SerialDisplayClass object: one set uses 32-bit color (including alpha) and commands that are pretty close to the USB serial protocol that my VectorDisplay app uses, and the other set is a subset of the standard Adafruit GFX library methods, using 16-bit color. For the most part you can freely mix the two sets of commands, with the exception that if you use the Adafruit compatible methods, you should use the 16-bit color commands whose names end with 565 instead of the 32-bit ones.

You can set the coordinate system with Display.coordinates(width,height). The default is width=240 and height=320. If you want to emulate a display with non-square pixels, you can use Display.pixelAspectRatio(ratio).

A few of the methods, including pixelAspectRatio(), take a FixedPoint32 argument. This is a 32-bit integer representing a floating point number, where 65536 represents 1.0. To convert a floating point number x to FixedPoint32, do: (FixedPoint32)(65536. * x) (or just TO_FP32(x)).

In addition to being able to send commands from Android buttons, screen touch events are also sent to the MCU.

For WiFi use, see the circles_esp8266 example. You'll need to press the USB button in the app to switch to WiFi mode.

For Bluetooth, you should be able to do:

SerialDisplayClass Display(MyBluetoothSerial);

...
MyBluetoothSerial.begin(115200);
Display.begin();

and then proceed just as in the USB serial case, where MyBluetoothSerial is whatever Stream object (e.g., Serial2) is connected to your Bluetooth adapter.

STEP 4: A $2 Oscilloscope

For the quick and dirty oscilloscope, you will need a blue or black (easier to deal with) pill STM32F103C8 board, which you can get on Aliexpress for under $2. I describe how to prepare the board for use with the Arduino environment for it and install sketches here.

Download this sketch on the board, which is a modified version of Pingumacpenguin's STM32-O-Scope sketch. Edit the #define BOARD_LED line to match your board. I am using a black pill whose LED is PB12. The blue pills (and some black pills that has the same pinout as the blue pill) have the LED at PC13.

Connect one wire--ground probe--to the board's ground and another wire to the board's B0 pin. Plug the board into an Android device with VectorDisplay running, and you have a portable, battery powered oscilloscope.

In the photo I have the oscilloscope hooked up to a phototransistor. The trace on the screen is from a TV infrared remote control.

Comments

[chipstein1]

The simple demo for 8266 is itself a versatile little program! It can be easily modified to draw other types of displays, and is blazingly fast on ESP32. It's running at 750X1000 resolution on an old LG cell phone that had been lying in my drawer for 4-5 years, and is about 100 times easier to hook up than a comparable LCD display. A nice bonus is that with a wired microUSB connection the ESP32 circuit runs directly from the cell phone battery, removing one more cost and complication from portable circuit designs.

[andy-techy6]

This is a marvelous contribution i was playing with pigoscope and almost any mcu-oscpoe and the idea of pickling my brain more with adding faster adc convertors but when I finnaly started getting some where with the scope projects and realised that my eyes are really 60 years old my searches led me here and it turned out to be a two in one although I have only managed to test the example as opposed to get an ideal scope up and running but I feel this opened up the possibility of moving in the correct direction-ish. Well done arpruss I would love to get advice from you on writing a similar apk to VectorDisplay, its all I can find that proves how important it is to learn better coding! But God speed to you anyway!

[enverkcabuk]

Hi.If I want to work on keil or CubeMX, And If we add the following lib to this IDE and we download your app t our android devices, Can android devices display like your examples.

[Alberto_92]

hi @arpruss, when trying to replicate your project I got:
- successfully installed vectorDisplay app
- successfully connected arduino to smartphone and got permission question on app
- but nothing happens, circles do not appear

I believe that might be because I'm getting an alert on IDE:
In file included from C:\Users\AppData\Local\Temp\arduino_modified_sketch_542254\circles.ino:1:0:
C:\Users\Arduino\arduino-1.8.13\portable\sketchbook\libraries\VectorDisplay\src/VectorDisplay.h: In member function 'uint32_t VectorDisplayClass::color565To8888(uint16_t)':
C:\Users\Desktop\Arduino\arduino-1.8.13\portable\sketchbook\libraries\VectorDisplay\src/VectorDisplay.h:725:65: warning: left shift count >= width of type [-Wshift-count-overflow]
return 0xFF000000 | ((((c>>11) & 0x1F) * 255 / 0x1F) << 16) | ((((c>>5) & 0x3F) * 255 / 0x3F) << 8) | ((c & 0x1F) * 255 / 0x1F);
Do you have a hint about the issue?
Thanks

[vaitfi]

Hi, vectordisplay is the great solution, I've been looking for. It realizes all the needs, which where spoken about.
For this, I have a question, but couldn't find a solution in the discussions yet. At using vectordisplay, is it possible to place the "command"-buttons also on the bottomline of the display? Or is it maybe possible to return a pressed touch-position (x, y) back to the Arduino to work with?
Thx very much for an answer and help.
vaitfi

[arpruss]

The current code doesn't do these things. You're free to add it to the code.

[codebeat]

Hai, very nice idea, I have similar ideas (only with NodeJS). Love the effort, great start, very clever! Download it all and got it working with use of a Samsung J8 and an Arduino (ATMega328p) Nano however not everything is working as should.

The verctordisplaytest sketch works however get a weird blue (turned red when taking a screen capture, maybe Android draw issue) triangle on the screen and pressing a button doesn't work. The second circle sketch shows only a black screen, no buttons. I can see there is some (heavy) interaction when pressing the screen but nothing happen.

I think the serial bus is very busy, maybe you have to do some optimizations to reduce the traffic. Think in events instead of polling all of the time and add debouncing so MCU got time to do other things.

The shapes are very simple so interaction could be very simple. I don't know if you using some sort of error detection at data because it is possible something get lost or received with errors. I don't either if you implement a binary format (not ASCII) to send information between both devices, doing so reduces the amount of bytes required.

Overall a great idea and first implementation, I think when you solve the bugs (and some speed problems) it can be very usable. Maybe my posted comments will help and also provided screenshots.

Kind regards,
codebeat.

[dancopy]

Excelent idea!
Do you intend to expand this project to work directly with esp8266 or esp32 (not with the STM32 board)? It was one of the things I was looking for long ago. I would like to use these Tablet screens with Arduino to do, for example, weather stations and also as presentation of images and scrolling texts with ESP.
If you have anything or any idea on this subject, I thank you very much.

[arpruss]

In theory the library should work with esp8266, but some other comments indicate it doesn't.

[elcomed]

1.I have downloaded the VectorDisplay application on Samsung A410F phone and Notes tablet. The application does not communicate with vector sketch (circle is compiled and uploaded to arduino uno). The buttons appear only for a while, but even if I touch the button, no circle etc. is displayed. Only the menu Hlep, USB etc. is displayed, the rest of the screen is black.
2. compilation of the Picoscope sketch has an error with libdma library.
3. Compilation of the circles_esp8266 has several errors!

Please be so very kind and help. Thank you.

[arpruss]

What microcontroller are you using this with?

[trmac]

Arduino UNO
ESP32 DEVKIT V12 DOIT

[arpruss]

What are the errors with circles_esp8266?

[Blackhock2]

Hello! The Library VectorDisplay does not work on NODE MCU V3.

[Blackhock2]

What is the host address you need to specify?

[arpruss]

The ip address of the phone, I think.

[arpruss]

Tell me more.

[Blackhock2]

After downloading the sketch to the board, wifi is not included, therefore I cannot connect to the board from the phone.

[arpruss]

In the sketch, have you set the ssid and password to match your WiFi network, and then the host IP to match your phone?

[Blackhock2]

node MCU does not distribute wifi. Probably the library does not work correctly.