Speech-to-text pipeline for facial recognition wearable system.
- Previously, adding new faces to the Memento system was done by annotating pictures in the companion app. In order to provide a more accessible approach to users, we focused on incorporating speech-enabled input methods for recognizing new faces. The final output of our project was building out the pipeline for a visually-impaired user to be able to input new faces into the Memento system.
- Previously, we created an app that would let users “swipe” on new faces they have encountered and input their names. In this iteration, we created a mock front end to mimic Google Glass: an Android app with a user interface that allowed users to record audio such as “Nice to meet you, [insert name here]” as a auditory cue to input a new face.. Then, we used Google Speech-To-Text to transcribe the audio into text. We built a Python server that used spaCy’s name recognition functions to parse the name from the text and return it to the Android frontend.