← All work
Case study 08 · Maternal health · AI safety
Moma.
An AI companion that knows where its authority ends.
Case study 08 · Maternal health · AI safety
An AI companion that knows where its authority ends.
Pregnancy apps are good at telling you your baby is the size of a peach. They're weaker when you have a real question at 11pm, and the stakes are unusual: most questions are harmless, but a few symptoms (reduced movements, a severe headache with vision changes, heavy bleeding) need a phone call within the hour, not a reassuring paragraph.
Health data is also about as sensitive as data gets. An app that wants people to tell it everything has to earn that by collecting as little as possible.
So the brief: make everyday questions easy, make the dangerous ones impossible to miss, and keep health data on the phone.
Every message goes through a fixed red-flag check first. A match gets a call-now card with the country's numbers, whatever the AI says next. The model can add context but can never downgrade an escalation.
Check-ins, moods and AI chats stay on the phone. The server sees a random ID, pregnancy stage, country and active days, and users can switch that off or delete it.
UK, Canada, US and Nigeria each get their own emergency and advice numbers, so "who do I call" is never generic.
Ask Moma knows the week, the stage and recent check-ins, so "is this normal?" gets an answer about week 31, not pregnancy in general.
A "talk to the Moma team" chat feeds a support inbox. Red-flag messages jump to the top as urgent.
Onboarding asks two things: due date and country. Every extra question before the first useful moment loses people.
The red-flag layer is plain pattern rules, deliberately boring and fully testable, written for how people actually type: "baby hasn't kicked since this morning", "soaking a pad an hour", "my face is puffy and I'm seeing flashing lights".
A set of 24 test messages, a mix of real emergencies and everyday questions that only look scary, runs on every change. A missed emergency is never acceptable; an occasional over-cautious card is.
The same set runs in the admin portal's safety lab, so anyone on the team can test a phrase and see exactly which rule caught it.
Moma hasn't launched yet, so these are targets, not results.
I designed and built Moma, and made the product calls: what the AI may and may not do, which data never leaves the phone, what the safety tests are, and when something was ready to go live. Nothing shipped without passing through a preview I approved.
It runs on Vercel with a Postgres database, and the AI is called from the server, so no keys sit on anyone's phone.
"Being right most of the time isn't enough for an AI product. It has to know where its authority ends."