This project uses Embedded AI frameworks to build an ML model that can classify various keywords from speech. We then deploy it to an Arduino BLE through TinyML, to perform a related home automation task‑ switching on fan/light, etc.
Overview
The system contains 3 primary parts:
Keyword Detection Model
Embedded System Component to Perform Actions
BLE based Authentication Feature
Keyword Detection Model
We have tried two approaches for the keyword audio detection:
First is through an artificial neural network -
It first starts by resizing the sample to 32 by 32. Then it is normalized, follows 2 2D convolutions, a 2D maxpooling and dropout. It is then flattened, and run through a Dense ReLu convolution, dropped out and then convoluted one last time.
The second is a traditional classifier -
The methods tried were SVM, Random Forest and Decision Trees. We found the Random Forest implementation to give the best results while being lightweight.
The keywords we trained our model to detect were:
Light: To create a pattern of colors with the LEDs on the Arduino Nano 33 BLE Sense and flash the pattern.
Sound: To ring a buzzer connected to the Arduino Nano 33 BLE Sense.
Fan: To switch on the Servo Motor connected to the Arduino Nano 33 BLE Sense
Embedded System Component to Perform Actions
In order to capture and analyze sound in real time, we used the omnidirectional digital microphone (MP34DT05) on the Arduino Nano 33 BLE Sense. The PDM library is used to implement this functionality.
The microphone is used to capture samples for the training data to build the dataset as well as to detect the keyword
The peripherals are connected to the Arduino using a breadboard and jumper cables. The peripherals used are:
SG90 9G Micro Servo Motor
PT-1306T Piezo Buzzer
On-Board Arduino LEDs
BLE based Authentication Feature
The Arduino Nano 33 BLE Sense has a powerful 2.4 GHz Bluetooth® 5 Low Energy module from u-blox, with an internal antenna. We use two Arduino devices, one acting as the central server and one acting as a peripheral to implement the proximity based security feature.
The peripheral device contains the model file. The microphone of the peripheral can be accessed only the BLE authentication between the peripheral and central server has been completed. For this example, we have hardcoded the central server to accept the connection request for this specific peripheral. This can be expanded further in the future to control more devices.
Final notes:
This project would not have been possible without the support of my teammates Rahul Shanbhag and Vignesh Gopal. Rahul has structured the code base and implementation on GitHub, and it can be accessed through this link!
Comentarios