# Measuring Finger Positions on a Violin With ESP32

283

1

As a violin player I always wanted an app or tool that could show me the position of my fingers on the violin very precise. With this project I tried to build this. Although this is a prototype and you could still add many features.

I also tried to separate the ESP32 and the rPI and thus I made the ESP32 send data wireless to the rPi. Which is probably the most difficult thing about this project.

It is also very important that at the end of this project nothing is stored on your computer but it is either on the rPI or ESP32.

### Teacher Notes

Teachers! Did you use this instructable in your classroom?
Add a Teacher Note to share how you incorporated it into your lesson.

## Step 1: Materials and Tools

Before getting into the specifics of building this project we need a few things.

1. 4x Linear Softpot: Linear Potentiometers to measure the position of a finger (a violin has 4 strings)
2. ESP32: An ESP32 module to read the data from the linear softpots.
3. a 4/4 violin: a violin to place the linear softpots on top.
4. a Raspberry Pi with an SD card: a raspberry pi which will store our database and website.
5. 10k potentiometer: a potentiometer for the brightness of the LCD
6. LCD-screen: an LCD screen to show to the ip addres of the rPi
7. Soldering Kit: For soldering all the elements togheter
8. Male-to-male wires and male-to-female wires: Cables for connecting all the elements
9. Micro USB cable: For powering the ESP32

## Step 2: Connecting the Softpots to the ESP32

First of all we need to connect our softpots to the esp32. We connect the left and right pins to the 5V and GND respectively. We connect the middle pin to an analog pin on the ESP32. We also need to connect the middle pin with a resistance of 10k ohm and connect this to the GND. This is so that our output of the softpots doesn't return random value.

Then we connect the ESP32 with the micro usb cable to our pc so that we can upload code to it. We will use the Arduino IDE for programming the ESP32. But first we need to install the Arduino core for the ESP32 so that we can upload to it. This can be done here.

Then we can begin to write code.

First we need to assign our pins to which we connected our middle pin of the softpots.

const int SOFT_POT_PIN1 = 34;

const int SOFT_POT_PIN2 = 35;

const int SOFT_POT_PIN3 = 32;

const int SOFT_POT_PIN4 = 33;

unsigned long onTime;

unsigned long softPotTime;

Then we can set up our pins. And we need to start our serial monitor and our time.

void setup() {

onTime = millis();

Serial.begin(115200);

Serial.println("Program start");

pinMode(SOFT_POT_PIN1, INPUT);

pinMode(SOFT_POT_PIN2, INPUT);

pinMode(SOFT_POT_PIN3, INPUT);

pinMode(SOFT_POT_PIN4, INPUT); }

void getdata(byte pdata[]) {

Then we need to read our pins so we can receive our data.

Then we put the values in a list so we can easily output it later.

for (int i=0; i < 4; i++){

int softpot = Names[i];

if (softpot > 10){

pdata[0] = i;

pdata[1] = softpot;

pdata[2] = millis();

} } }

}

## Step 3: Connecting the ESP32 and the RPI Wirelessly

For connecting the ESP32 and RPI wirelessly, we will use a library called websocket. To install this library, we can get the files here. We will need to change some code in the files itself in order to use this library for the ESP32.

We will need to change the MD5.c and MD5.h.

• MD5Init to MD5InitXXX
• MD5Update to MD5UpdateXXX
• MD5Final to MD5FinalXXX

We will also need to delete the avr/io.h lines in the sha1 files.

Then we can add the library to our Arduino IDE by sketch > include library > add .ZIP library and then we can select your library in a zip file.

After that we can start to write our code.

First for the ESP32:

Including our library

#include
#include

Assigning our pins again.

const int SOFT_POT_PIN1 = 34;

const int SOFT_POT_PIN2 = 35;

const int SOFT_POT_PIN3 = 32;

const int SOFT_POT_PIN4 = 33;

Assigning our wifi server

WiFiServer server(80);

Starting our websocket server

WebSocketServer webSocketServer;

const char* ssid = "your wifi SSID";

void setup() {

Serial.begin(115200);

pinMode(SOFT_POT_PIN1, INPUT);

pinMode(SOFT_POT_PIN2, INPUT);

pinMode(SOFT_POT_PIN3, INPUT);

pinMode(SOFT_POT_PIN4, INPUT);

Starting our wifi and connecting to it

while (WiFi.status() != WL_CONNECTED) {

delay(1000);

Serial.println("Connecting to WiFi.."); }

Serial.println("Connected to the WiFi network");

Serial.println(WiFi.localIP());

server.begin(); delay(100); }

void getdata(char *pdata) {

Placing the data in a list and converting it to hexadecimal.

}

void loop() {

WiFiClient client = server.available();

if (client.connected()) {

delay(10);

if (webSocketServer.handshake(client)) {

Serial.println("Client connected");

Sending and receiving data.

while (client.connected()) {

char data[30];

getdata(data);

Serial.println(data);

webSocketServer.sendData(data);

delay(10); // Delay needed for receiving the data correctly }

Serial.println("The client disconnected");

delay(100); }

else {

Serial.println("shitsfuckedyo");

} } }

Then for the rPI in python:

Importing our libraries

import websocket
import time

Assasigning a globale variabel i

i = 0

Setting a maximum of 200 messages that we can receive

nrOfMessages = 200

class Websocket():

def __init__(self):

Initialising our websocket and connecting it to our ESP32

self.ws = websocket.WebSocket()

self.ws.connect("ws://172.30.248.48/")

Receiving our data

def work(self):

self.ws.send("message nr: 0" )

result = self.ws.recv() time.sleep(0.5) return result

Closing the websocket after receiving everything

def close(self):

self.ws.close()

## Step 4: Connecting Your Website and Database

As for connecting our Database and website, you will first of all need to create your database on the pi by installing mariadb: sudo apt install mariadb.

Then you can acces it by doing: sudo mariadb.

Then you will also need to create your website. You can do this however you like, but you have to use Flask and you need to have a form in your HTML for stopping and starting your data.

Then you can insert this code to connect your database and your website (both your website and database have to both be on your pi, this can be done by using the deployment tab in settings of pycharm)

app.config["MYSQL_DATABASE_HOST"] = "localhost"

Function for getting data out of our database.

def get_data(sql, params=None):

conn = mysql.connect()

cursor = conn.cursor()

print("getting data")

try:

print(sql)

cursor.execute(sql, params)

except Exception as e:

print(e)

return False

result = cursor.fetchall()

data = []

for row in result:

data.append(list(row))

cursor.close()

conn.close()

return data

Function for inserting data into our database

def set_data(sql, params=None):

conn = mysql.connect()

cursor = conn.cursor()

try:

log.debug(sql)

cursor.execute(sql, params) conn.commit()

log.debug("SQL uitgevoerd")

except Exception as e:

log.exception("Fout bij uitvoeren van sql: {0})".format(e))

return False

cursor.close()

conn.close()

return True

We will also need to thread our application so that you can do other things while you are recording.

def __init__(self, ):

self.data_all =[]

def run(self):

time.sleep(5)

for i in range(0, 200):

self.data_all.append(w.work().split(","))

print(self.data_all)

## Step 5: Connecting Everything Together

For running your website from your Pi you have to use a service:

[Unit]
Description=uWSGI instance to serve project1 web interface

After=network.target

BindsTo=mysqld.service

After=mysqld.service

[Service]

User=pi

Group=www-data

WorkingDirectory=/home/pi/project1/web

Directory of your ini file that can be found later.

[Install]

WantedBy=multi-user.target

uwsgi-flask.ini that you need to place in the directory you specified in ExecStart above

[uwsgi]
module = web:app virtualenv = /home/pi/project1/env

master = true processes = 5

plugins = python3

socket = project1.sock chmod-socket = 660 vacuum = true

die-on-term = true

## Step 6: Extra: Connecting LCD Screen

We can connect an LCD screen so that we can show the ip-adress of our Pi for our website.

import RPi.GPIO as GPIO
import time

import commands

GPIO.cleanup()

D0 = 22

D1 = 5

D2 = 6

D3 = 13

D4 = 19

D5 = 26

D6 = 20

D7 = 21

list = [22,5,6,13,19,26,20,21]

E = 24

RS = 23

class Screen:

def __init__(self):

GPIO.setmode(GPIO.BCM)

self.setup()

#Function set self.stuur_instructie(0x3f) #Display self.stuur_instructie(0x0c) #On + cursor self.stuur_instructie(0x01) @staticmethod def setup(): GPIO.setup(list, GPIO.OUT) GPIO.setup([E,RS],GPIO.OUT)

def stuur_instructie(self,byte):

GPIO.output(E,GPIO.HIGH)

GPIO.output(RS,GPIO.LOW)

self.set_GPIO_bits(byte)

time.sleep(0.005)

GPIO.output(E,GPIO.LOW)

def stuur_teken(self,char):

temp = ord(char)

GPIO.output(E,GPIO.HIGH)

GPIO.output(RS,GPIO.HIGH)

self.set_GPIO_bits(temp)

time.sleep(0.005)

GPIO.output(E,GPIO.LOW)

def set_GPIO_bits(self,byte):

for i in range(0,8):

if (byte & (2**i)) == 0:

GPIO.output(list[i],GPIO.LOW)

else:

GPIO.output(list[i], GPIO.HIGH)

def main():

s = Screen()

for letter in teken:

s.stuur_teken(letter)

teken2 = commands.getoutput("ip addr show wlan0 | grep -Po 'inet \K[\d.]+'")

print(teken2)

s.stuur_instructie(0xc0)

for letter2 in teken2:

s.stuur_teken(letter2)

if __name__ == '__main__': #Program starting from here

try:

main()

except KeyboardInterrupt:

pass

Then we can create a service to start the LCD on start-up.

## Recommendations

• ### Internet of Things Class

22,724 Enrolled

## Discussions

This looks very handy for anyone learning to play. Great job!