CogniSpeak
Speech Analysis Demo
HackHarvard 2024, team of three
- Python
- FastAPI
- React
What it is
A weekend hackathon build that records your voice in the browser, runs it through a machine learning pipeline on the server, and returns a result. The most useful thing about it is that it tells you the result means nothing.
Who it helped
The idea was early dementia screening from speech, which is a real research area. What three of us actually built in a weekend was the whole pipeline: the browser records you answering ten questions, the audio goes to our server, the server converts it, pulls out the sound features and runs a classifier, and an answer comes back. The recording is deleted before the response is even sent, so nobody's voice is left sitting on a server.
The model behind it means nothing, and that is the honest part. It is trained on ten public voice clips that we split into two groups at random, so its output is closer to a coin flip than a screening. We could have hidden that. Plenty of hackathon projects do, and a confident looking percentage on screen would have demoed better.
We put the limitations at the top of the README instead, and made that section the longest one. If you are old enough to be worried about dementia, a number on a screen that looks like a diagnosis is not a harmless thing to show you. Someone might believe it and be frightened for no reason, or be reassured and not see a doctor when they should. For the person on the other side of the screen, the honest version is worth more than the impressive one.
What went well
The pipeline works end to end and it is private by default. The recording exists on the server only as long as it takes to process, and is deleted before the answer goes back, so trying the demo does not leave your voice on somebody's laptop.
We also wrote down what a real version would need: a properly labelled clinical dataset, features based on what a person says rather than how their microphone sounds, and testing across hundreds of speakers instead of ten clips. Understanding why it does not work is worth more than the demo itself.
What's next
Put the warning where the user is, not only in the code and the README. Anyone reading the source knows the result is meaningless, but a person clicking through the demo sees a clean, confident looking answer. If the honesty only lives where engineers look, it is not protecting the people it is actually for.