Unlocking my apartment with my face
Unlocking my apartment with my face
As part of a previous project, I installed an IoT lock on my front door, which allowed me to control it using my smartphone. However, having to take out my smartphone every time I want to unlock the door quickly becomes annoying. To improve on the situation, I installed an RFID reader on the lock, which I used in combination with an RFID ring which now serves as my key. To follow up on this trend of finding new ways to unlock my door, I decided to explore face recognition technologies.
The first step in this endeavor was to get my hands on a camera that I could use. Luckily, the ESP32 microcontroller provides support for OV2640 cameras and including those can be purchased for less than USD 10$ online. Moreover, the ESP32 can handle MQTT, which my lock can use to receive the unlock command. Here I got myself a TTGO T-camera plus as it also includes a display, useful to inform the user of what is happening.
There are various Arduino sketch available online that can interact with an OV2640 camera from the ESP32. One such sketch in particular performs face detection and even recognition. For this project, I created a firmware for the T-camera plus based on this face detection system. However, I left the face recognition aside as I didn't trust it so much and wanted to have more control on how to achieve it.
For face recognition, I used the popular face_recognition pip package. Here, I designed a simple FastAPI application that exposes the face recognition abilities of the package via HTTP requests, which the ESP32 can handle. This application also manages user registration, storing user information in a MongoDB database.
I deployed the application in K3s on an Nvidia Jetson Nano in order to benefit from GPU acceleration.
To facilitate user management, I built a simple front-end for the application using Vue.js and Vuetify.
With this setup, the ESP32 constantly scans for faces. When a face is detected, a frame is captured and sent to the face recognition service running on the Jetson Nano. The latter then checks the face against those stored in the user database. If a match is found, the application respond with an HTTP 200, after which the ESP32 sends an MQTT message to the lock so as to unlock the door.
The code for this project is available on GitHub:
Moreover, the web application can be deployed as docker containers, for which images can be found on Docker Hub: