COVID19 is really a historic pandemic affecting the whole world very badly and people are building a lot of new devices to fight with it. We have also built an automatic sanitization machine and Thermal Gun for Contactless temperature screening. Today we will build one more device to help fighting with Coronavirus. It is a cough detection system, which can distinguish between noise and cough sound and can help finding Corona suspect. It will use machine learning techniques for that.

In this tutorial, we are going to build a Cough Detection system using Arduino 33 BLE Sense and Edge Impulse Studio. It can differentiate between normal background noise and coughing in real-time audio. We used Edge Impulse Studio to train a dataset of coughing and background noise samples and build a highly optimized TInyML model, that can detect a Cough sound in real-time.

Supplies

Hardware

Arduino 33 BLE Sense
LEDJumper
Wires

Software

Edge Impulse Studio
Arduino IDE

Step 1: Circuit Diagram

Circuit Diagram for Cough Detection Using Arduino 33 BLE Sense is given above. Fritzing part for Arduino 33 BLE was not available, so I used Arduino Nano as both have the same pin-out.

The Positive lead of LED is connected to digital pin 4 of Arduino 33 BLE sense and Negative lead is connected to the GND pin of Arduino.

Step 2: Creating the Dataset for Cough Detection Machine

As mentioned earlier, we are using Edge Impulse Studio to train our cough detection model. For that, we have to collect a dataset that has the samples of data that we would like to be able to recognize on our Arduino. Since the goal is to detect the cough, you'll need to collect some samples of that and some other samples for noise, so it can distinguish between Cough and other Noises. We will create a dataset with two classes “cough” and “noise”. To create a dataset, create an Edge Impulse account, verify your account and then start a new project. You can load the samples by using your mobile, your Arduino board or you can import a dataset into your edge impulse account. The easiest way to load the samples into your account is by using your mobile phone. For that, you have to connect your mobile with Edge Impulse. To connect your Mobile phone, click on ‘Devices’ and then click on ‘Connect a New Device’.

Step 3: Connect to Mobile Phone

Now in the next window, click on ‘Use your Mobile Phone’, and a QR code will appear. Scan the QR code with your Mobile Phone using Google Lens or other QR code scanner app.

This will connect your phone with Edge Impulse studio.

With your phone connected with Edge Impulse Studio, you can now load your samples. To load the samples, click on ‘Data acquisition’. Now on the Data acquisition page, enter the label name, select the microphone as a sensor, and enter the sample length. Click on ‘Start sampling’, to start sampling a 40 Sec sample. Instead of forcing yourself to cough, you can use online cough samples of different lengths. Record a total of 10 to 12 cough samples of different lengths.

Step 4:

After uploading the cough samples, now set the label to ‘noise’ and collect another 10 to 12 noise samples.

These samples are for Training the module, in the next steps, we will collect the Test Data. Test data should be at least 30% of training data, so collect the 3 samples of ‘noise’ and 4 to 5 samples of ‘cough’.
Instead of collecting your data, you can import our dataset into your Edge Impulse account using the Edge Impulse CLI Uploader. To install the CLI Uploader, first, download and install Node.js on your laptop. After that open the command prompt and enter the below command:

npm install -g edge-impulse-cli

Now download the dataset (Dataset Link) and extract the file in your project folder. Open the command prompt and navigate to dataset location and run the below commands:

edge-impulse-uploader --clean
edge-impulse-uploader --category training training/*.json

edge-impulse-uploader --category training training/*.cbor

edge-impulse-uploader --category testing testing/*.json edge-impulse-uploader --category testing testing/*.cbor

Step 5: Training the Model and Tweaking the Code

As the dataset is ready, now we will create an impulse for data. For that go to the ‘Create impulse’ page.

Now on the ‘Create impulse’ page, click on ‘Add a processing block’. In the next window, select the Audio (MFCC) block. After that click on ‘Add a learning block’ and select the Neural Network (Keras) block. Then click on ‘Save Impulse’.

In the next step, go to the MFCC page and then click on ‘Generate Features’. It will generate MFCC blocks for all of our windows of audio.

After that go to the ‘NN Classifier’ page and click on the three dots on the upper right corner of the ‘Neural Network settings’ and select ‘Switch to Keras (expert) mode’.

Replace the original with the following code and change the ‘Minimum confidence rating’ to ‘0.70’. Then click on the ‘Start training’ button. It will start training your model.

import tensorflow as tf
from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense, InputLayer, Dropout, Flatten, Reshape, BatchNormalization, Conv2D, MaxPooling2D, AveragePooling2D from tensorflow.keras.optimizers import Adam from tensorflow.keras.constraints import MaxNorm # model architecture model = Sequential() model.add(InputLayer(input_shape=(X_train.shape[1], ), name='x_input')) model.add(Reshape((int(X_train.shape[1] / 13), 13, 1), input_shape=(X_train.shape[1], ))) model.add(Conv2D(10, kernel_size=5, activation='relu', padding='same', kernel_constraint=MaxNorm(3))) model.add(AveragePooling2D(pool_size=2, padding='same')) model.add(Conv2D(5, kernel_size=5, activation='relu', padding='same', kernel_constraint=MaxNorm(3))) model.add(AveragePooling2D(pool_size=2, padding='same')) model.add(Flatten()) model.add(Dense(classes, activation='softmax', name='y_pred', kernel_constraint=MaxNorm(3))) # this controls the learning rate opt = Adam(lr=0.005, beta_1=0.9, beta_2=0.999) # train the neural network model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy']) model.fit(X_train, Y_train, batch_size=32, epochs=9, validation_data=(X_test, Y_test), verbose=2)

Step 6:

After training the model, it will show the training performance. For me, the accuracy was 96.5% and loss was 0.10 that is good to proceed.

Now as our cough detection model is ready, we will deploy this model as Arduino library. Before downloading the model as a library, you can test the performance by going to the ‘Live Classification’ page. Go to the ‘Deployment’ page and select ‘Arduino Library’. Now scroll down and click on ‘Build’ to start the process. This will build an Arduino library for your project.

Now add the library in your Arduino IDE. For that open the Arduino IDE and then click on Sketch > Include Library > Add.ZIP library. Then, load an example by going to File > Examples > Your project name - Edge Impulse > nano_ble33_sense_microphone. We will make some changes in the code so that we can make an alert sound when the Arduino detects cough. For that, a buzzer is interfaced with Arduino and whenever it detects cough, LED will blink three times. The changes are made in void loop() functions where it is printing the noise and cough values. In the original code, it is printing both the labels and their values together. for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) { ei_printf(" %s: %.5f\n", result.classification[ix].label, result.classification[ix].value); }We are going to save both the noise and cough values in different variables and compare the noise values. If the noise value goes below 0.50 that means cough value is more than 0.50 and it will make the sound. Replace the original for loop() code with this: for (size_t ix = 1; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) { Serial.print( result.classification[ix].value); float Data = result.classification[ix].value; if (Data < 0.50){ Serial.print("Cough Detected"); alarm(); } }After making the changes, upload the code into your Arduino. Open the serial monitor at 115200 baud.

So this is how a cough detection machine can be built, it's not a very effective method to find any COVID19 suspect but it can work nicely in some crowded area.