My first commit to
on-air shows March 3, 2020. I know that the weeks leading up to that commit I spent some time reading through the TF Lite documentation, playing with Cloudflare Workers K/V and getting my first setup of
esp-idf squared away. After that it was off to the races. I outlined my original goal in the planning post. I didn't quite get to that goal. The project currently doesn't have a VAD to handle the scenario where I forget to activate the display before starting a call or hangout. Additionally I wasn't able to train a custom keyword as highlighted in the custom model post. I was however able to get a functional implementation of the concept. I am able to hang the display up, and then in my lab with the
ESP-EYE plugged in I can use the wake word
visual followed by
on/off to toggle the display status.
While it's not quite what I had planned it's a foundation. I've got a lot more tools and knowledge under my belt. Round 2 will probably involved Skainet just due to the limitations in voice data that's readily available. Keep an eye out for a couple more post highlighting some bumps along the way and final take aways.
The code, docs, images etc for the project can be found here and I'll be posting any further updates to HackadayIO. For anybody that might be interested in building this the instructions below provide a brief outline. Updated versions will be hosted in the repo. If you have any questions or ideas reach out.
- Optional ESP-EYE case
- Optional PyPortal case
- Two 3.3v usb to outler adapters and two usb to usb mini cables
- Two 3.3v micro usb wall outlet chargers
- Clone the on-air repo.
- Setup Cloudflare DNS records for your domain and endpoint, or setup a new domain with Cloudflare if you don't have one to resolve the endpoint.
- Setup a Cloudflare workers account with worker K/V.
- Setup the Wrangler CLI tool.
[Makefile](https://git.sr.ht/~n0mn0m/on-air/tree/master/sighandler/Makefile)with your domain and test calling.
Setup CircuitPython 5.x on the PyPortal.
If you're new to CircuitPython you should read this first.
Go to the directory where you cloned on-air.
[secrets.py](https://git.sr.ht/~n0mn0m/on-air/tree/master/display/secrets.py) with your wifi information and status URL endpoint.
secrets.pyand the bitmap files in
screens/to the root of the PyPortal.
The display is now good to go.
[esp-idf](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/)using the 4.1 release branch.
Setup a Python 3.7 virtual environment and install Tensorflow 1.15.
chmod +x orchestrate.shand
Once training completes
esp-idftooling so that
$IDF_PATHis set correctly and all requirements are met.
idf.py menuconfigand set your wifi settings.
Update the URL in
This should match the host and endpoint you deployed the Cloudflare worker to above
idf.py --port \<device port\> flash monitor
You should see the device start, attach to WiFi and begin listening for the wake word "visual" followed by "on" or "off".