Case 02 / AI Learning Platform

Feedback a fluent speaker can actually check.

Eeva is a Korean-first English refinement platform for people who already speak the language well. I am the engineer on the build. A learner records or uploads a spoken answer; the system transcribes it, analyses it in two stages, and returns the specific expressions worth refining — each one anchored to the exact words they said. It is in active development and not yet publicly launched.

My role: Sole engineer
Decision records: 29
Status: Pre-launch

Visit live site

01 — Where it started

Not another beginner course.

The learners this is built for are already fluent. They do not need to be told they did well, and they do not need a grammar course. They need to know which specific expressions in the answer they just gave would sound different coming from a native speaker — and they need to be able to check the claim against their own words.

That constraint shaped everything. Feedback has to start from the English the learner actually used, which means every annotation is anchored to an exact span of the transcript. A model that says "your prepositions need work" is useless here. A model that points at the third word of the second sentence and says what to say instead is evidence a person can act on.

02 — The audit that changed the pipeline

Built for precision. Which made it useless.

A native-English tutor audited the system against his own evaluations of three Korean learners. The transcription came back 98–100% identical to his. The analysis did not: he found roughly twice as many issues, and on one sample he flagged eight where the system flagged one. The misses clustered in three patterns that Korean speakers hit constantly — articles, prepositions, and plurals.

The cause was in the prompt, and it was deliberate. It instructed the model to omit anything borderline and to silently discard weakly supported findings. The system had been tuned for precision, and it had overpaid: a tool that only reports what it is certain of tells an advanced learner almost nothing.

The fix was to stop asking one pass to do two jobs. A first pass now sweeps for recall, with explicit examples of the three categories that matter. A second verifies the candidates and restores precision, narrowing overlapping findings to the smallest accurate span instead of dropping them. Precision became a stage that could be tuned without suppressing recall — and the gap between them became something we could measure rather than argue about.

03 — Decisions that mattered

Four that were not obvious up front.

The pipeline sat one second inside a hard timeout

Full analysis ran 30–60 seconds inside the HTTP request, against a gateway that closes after roughly 30 seconds of inactivity. Production measured 28.6s; a preview returned 29.8s on one run and a 504 on the next. Success was sampling luck, not headroom. Rebuilt as an async job — the request returns a run ID immediately, a background function does the work, the client polls. The run store uses strong consistency, because polling that cannot see its own writes is worse than not polling.

An optimisation that could never have worked

Two passes open with a byte-identical 1,580-token block, so prompt caching looked like free money. It was implemented, measured live, and rescinded the same day: structured outputs key the cache per output schema, and each pass uses a different schema, so the two could never share a cache key. It was caught in hours rather than shipped because the per-pass tracing built one decision earlier was already there to measure it.

Learner audio is a liability, not an asset

Raw provider payloads, transcripts, and model outputs are never logged. Analytics never receives transcript text, run IDs, filenames, or user identifiers. Uploaded audio is deleted once processing finishes, and a learner can delete a run's audio, export their data, or delete the account outright. None of that was a compliance requirement — it was cheaper to decide once than to retrofit.

Korean falls back to English until a native speaker signs off

The Korean prompt and copy assets stay marked as candidates until a native-speaker review is complete, and production generation falls back atomically to English while they are unreviewed. Shipping machine-translated coaching to the audience the product is named for would have undone the whole premise, so the system is built so that cannot happen by accident.

04 — What it runs on

Ordinary parts, carefully wired.

A speech provider transcribes, a language model analyses, a background function does the slow work so no request is waiting on it, and Postgres holds the accounts and run history. The interesting engineering is not in any one of those choices — it is in the contracts between them, which is where a pipeline like this actually breaks.

Nuxt 3AssemblyAI transcriptionAnthropic Claude analysisTwo-stage detect + verifyNetlify background functionsNetlify Blobs run storePostgres + DrizzleSelf-hosted Better AuthKorean / English i18nVitest + eval harness

05 — Common questions

Before you ask.

Is this shipped?

It is in active development and deliberately pre-launch — the Netlify subdomain is intentional, not an oversight. Recording, upload, transcription, two-stage analysis, accounts with email verification, session history, data export and deletion all work today. The curriculum layer — programs, syllabus personalisation, scheduling — is designed in the decision records but not built.

Why two model passes instead of one?

Because one pass has to choose between finding everything and being right, and it will quietly choose being right. Tuned for precision, the system found one issue where a human tutor found eight. Splitting recall and verification into separate stages let each be tuned on its own, and made the trade-off measurable instead of a matter of opinion.

What happens to a learner's recording?

It is deleted once processing finishes. Raw provider payloads, transcripts, and model outputs are never written to logs, and analytics never receives transcript text or identifiers. A learner can delete an individual run's audio, export everything held about them, or delete the account outright.

How do you know the analysis is any good?

Ground-truth fixtures drawn from human evaluations, run by an evaluation harness that executes as a build gate rather than a thing someone remembers to check. A native-English tutor audited detection quality, and a native-Korean reviewer gates the Korean assets. Where those reviews are incomplete, the system falls back rather than guessing.

What does a language app have to do with my business?

The transferable part is not the subject matter. It is scoping AI around one job with a measurable outcome, deciding what the output is worth before it reaches a person, putting a verification step where the cost of being wrong is real, and measuring quality rather than asserting it. That is the same discipline whether the output is coaching notes or an invoice summary.

Have an idea like this?

The hard part of this build was never getting a model to produce text. It was deciding what the output had to be worth before it was allowed in front of a person. That is the same question behind every AI project that turns out well.

Start a project See the AI integration service