Case 02 / AI Learning Platform
Eeva is a Korean-first English refinement platform for people who already speak the language well. I am the sole engineer on this contract build. A learner records or uploads a spoken answer; the system transcribes it, analyses it in two stages, and returns the specific expressions worth refining — each one anchored to the exact words they said. It is in active development and not yet publicly launched.
01 — Where it started
These learners are already fluent. They do not need a grammar course; they need to see which expression in the answer they just gave would sound different coming from a native speaker. Every annotation is anchored to the relevant span of their transcript. “Your prepositions need work” is not enough. A person needs to be able to check the claim against their own words and act on it.
02 — The audit that changed the pipeline
A native-English tutor audited the system against his own evaluations of three Korean learners. The transcription came back 98–100% identical to his. The analysis did not: he found roughly twice as many issues, and on one sample he flagged eight where the system flagged one. The misses clustered around articles, prepositions, and plurals.
The prompt was doing what I had asked: omit anything borderline and silently discard weakly supported findings. That instruction was meant to prevent false alarms. It also suppressed the subtle issues these learners needed help identifying.
I split the analysis into two stages. The first looks broadly for possible issues, with explicit examples of the three categories the audit exposed. The second verifies the candidates, narrowing overlapping findings to the smallest accurate span instead of dropping them. Detection and verification can now be evaluated and tuned separately, instead of forcing one instruction to balance both.
03 — Decisions that mattered
Analysis could take 30–60 seconds, while the gateway closed a connection after roughly 30 seconds without data. Keep-alive writes were buffered, so they did not keep it active. Production measured 28.6 seconds; one preview took 29.8 seconds and another returned a 504 timeout. I moved the analysis into a background job: the upload request returns a run ID, and the client checks for the result. Processing can continue after the request ends.
Raw provider payloads, transcripts, and model outputs are never logged. Analytics never receives transcript text, run IDs, filenames, or user identifiers. Uploaded audio is deleted once processing finishes, and a learner can delete a run's audio, export their data, or delete the account outright.
The Korean prompt and copy assets stay marked as candidates until a native-speaker review is complete. Until then, generation uses the English assets as a complete set, keeping unreviewed Korean material out of the feedback. For a product built around language precision, that review is a release condition enforced in the system.
Start with the work you want the system to do and the mistake it cannot make. We can decide whether AI belongs in that workflow before deciding what to build.