Building an AI Grocery App After the Kids' Bedtime

Over a decade ago, I wrote the copy for a marketing campaign that opened with a question: what if you could cut your family's shopping / planning / cooking / food-related-kid-wrangling time in half — while still getting quick, healthy, delicious meals on the table?

That was for Relay Foods, an online grocer where a small team of us spent years building ahead-of-the-curve grocery experiences. We coaxed a natural language processing tool into turning recipes into shoppable ingredient lists, bent a search engine into a grocery taxonomy engine, and even ran conversational-UI experiments with IBM Watson. None of it came cheap or easy: each of those took a team, real money, and a lot of creative duct tape. We were early. The company dissolved via an acquisition, but the problem never stopped nagging at me.

Since then, AI-based tooling has exploded. So I got curious: how much of that work could one person rebuild now, in the margins between the kids' bedtime and mine? The answer turned out to be a new app, called Relay in homage to the original, that in some ways does more than the original could. This post is what building it taught me: where AI actually belongs in a product, how much the tooling has changed, and why the hardest part of planning a family's dinners was never about finding recipes.

Two Relay app screens: a weekly meal plan with dinners listed by day, and a chat where the user asks what to make and the app suggests dishes.

Three ways to make software better with AI

First, a quick bit of vocabulary, because every company must now say "we use AI" to make their investors happy, regardless of what they're actually doing. I think AI can make products better in 3 core ways:

  1. Smarter — the app learns. This is really just classic machine learning wrapped in a new buzzword. The Relay app notices which meals actually get eaten and how quickly you burn through different types of staples, so each week takes a little less thought than the last.
  2. More powerful — it can do things that simply weren't possible before. This is the generative stuff: Relay turns a half-formed request — "something quick the kids will actually eat" — into a real meal plan and shopping list.
  3. Faster to build — AI coding tools are the reason this app exists at all. A person with a full-time job and a house full of kids does not ship this complex of a native iOS app in his spare time without them.

The art of "using AI" is knowing which tools to use for which purposes, and how to get the most out of them working together.

Why I didn't just build a chatbot

When people picture "an AI app" today, they picture a chat box: a blinking cursor, a cheerful "How can I help?", and you type what you want. Relay has one — you can ask it, typed or out loud to Siri, what to do with the odds and ends left in the fridge. But it's one tool among several, and figuring out when to reach for it was half the design work.

Chat has a real superpower: it's the fastest way to say what you want. You can speak at around 150 words a minute and thumb-type at maybe 40. And as Don Norman argued in The Design of Everyday Things, every interface is a bridge between what the maker intended and what the user is trying to do — most make you cross on the designer's terms, hunting for the menu that maps to your intent. Conversation lets you skip that and just say it: "I've got half a rotisserie chicken and twenty minutes."

But the same open-ended text that's great for saying what you want is lousy for what comes next — choosing between options, holding a week of plans in your head, making fiddly edits. So Relay matches the medium to the moment:

  • Talk to explain something open-ended, without wrestling a UI.
  • Images and a few options when the app should guide you — X, Y, or Z? — so you recognize an answer instead of composing one.
  • Taps for structured tweaks, like toggling a set of ingredients in or out of a recipe.
  • Persistent visuals for anything you'd otherwise hold in your head — a day-by-day plan you can see at a glance, not reconstruct from a transcript.

None of that is anti-chatbot — just the opposite. The best AI interfaces aren't a chat box or a set of screens; they're a deliberate blend, each part doing what it's best at.

Two Relay screens: a dish-picker offering three suggestions to choose from, and an in-store shopping list grouped by aisle with checkboxes to tick off items.
Two of the four mediums - see screenshots above for the other two

The architecture is mostly just: the app remembers, so you don't have to

Here's the reframe the whole app is built on: planning dinner isn't a search problem, it's a memory problem. The hard part was never finding a recipe — it's that nothing ever remembered my household from one week to the next. So the architecture is mostly: whatever it takes to remember what the user already did, instead of a pile of features that each start from scratch.

Under the hood, it's a Swift app (for native, responsive UI) and a Python web service that orchestrates both AI and non-AI tools. But to the user, it looks like this: what you cook shapes next week's suggestions, what you buy updates a virtual pantry, what you upload via receipt becomes the budget. Plan, shop, buy, cook: each stage is both a sensor and a consumer, folded into one loop that gets a little tighter every week. That way, what feels magical is not just the AI understanding a free-form request, it's that it seems to understand how any given family operates.

Diagram of Relay's Plan, Shop, Buy, and Cook stages in a clockwise loop, each handing off to the next — this week's dishes, an aisle-sorted list, a stocked kitchen, what everyone loved — around a center labeled "it remembers: favorites, pantry, prices, staples."

The hardest part of the app is reading a receipt

A great example of this at play centers on the ratty piece of paper that you usually toss as soon as you get home from the store: the lowly grocery receipt.

A crumpled Costco receipt with personal details blocked out, showing grocery lines in cryptic abbreviations like BUTER CROISS and KS ULTRA, with prices, subtotal, tax, and total.

That's a real one from my last Costco run. Quick — what did I buy? A couple are guessable. Most aren't. BUTER CROISS is butter croissants. NUGODRKCHOC is a NuGo dark chocolate something-or-other. And my favorite: KS TOWEL is Kirkland paper towels while KS ULTRA is Kirkland toilet paper.

Getting a machine to read that reliably is hard, and it takes more than pointing an AI at it. A vision model (Claude Sonnet, here) does the first pass, but the trick that makes it accurate is handing it your shopping list — so the job isn't "extract this text," it's "here are the eighteen things this person meant to buy; which line is which?" From there it's mostly unglamorous engineering: a little logistic regression model that learns which abbreviations I can trust so I can skip ones I've seen before; a quick human validation on iffy matches; and a set of guardrails so a bad read can't silently turn one avocado into ten. The AI is maybe a fifth of it. The rest is the boring software that makes it trustworthy.

But reading the receipt was never really the point — it's what reading it unlocks. A grocery receipt is worthless the moment you have it: a wad of cryptic abbreviations you can't do anything with. Turn it into structured, remembered data, though, and that same scrap starts answering questions you could never answer before. What did I spend this week? What does ground beef cost lately, and how much has it crept up over the year? How much did I throw in the cart on impulse versus what was on the list? When did I last buy dishwasher pods? None of that is a feature I sat down and designed — it's a byproduct of the app finally remembering a piece of paper that was always destined for the trash.

Two Relay insight screens: a weekly budget showing $264 spent of $300 with spending broken down by aisle, and a list of frequently bought items showing how often each is purchased.

The staff I don't have

Back at the start I called the third kind of AI "faster to build." But building was never the hard part. The features are the easy part; keeping them working is the job. Under every app that runs reliably is a constant, invisible swirl of upkeep — curating data, catching what broke, retuning models, noticing when a result quietly went wrong. At a real company, that swirl is scores of people's jobs, and it's why most solo side projects rot the moment their maker gets busy.

That upkeep is what I handed off. I run a small staff of background agents, each with a narrow job I'd otherwise have to hire for:

  • catalog curator tends the grocery taxonomy, the master list of items and aisles that everything else matches against. Left alone, a taxonomy like that slowly degrades: duplicates creep in, categories drift. The curator works through it over time, tidying and reorganizing, so the foundation the receipt matcher depends on stays clean without me editing a spreadsheet by hand.
  • dish inspector hunts for recipes and ingredients that look broken — a dish missing a step, an ingredient that didn't parse cleanly — and flags or fixes them before I'd ever notice.
  • receipt scorer retrains the receipt scoring models from the last section, re-fitting them on recent confirmations so the matcher keeps getting sharper on its own.
  • chat evaluator runs the conversational part of the app against a bank of test cases, so when I tweak a prompt I can tell whether I improved the chat or quietly broke it.

None of these is the flashy, autonomous kind of agent that makes headlines. They're more of a night janitorial crew: dull, essential, never-done upkeep that a working product quietly demands. And honestly, that's the part of this AI moment that grabs me. Everyone's dazzled that a model can write code; I'm more struck that the endless work of keeping a real product alive can now fall to one person who's careful about what to hand off and what to check.

A status feed titled "Background Crew" showing four automated agents and what each did: a catalog curator merging duplicate items, a dish inspector fixing broken recipes, a receipt scorer retraining a model, and a chat evaluator running test prompts.

Dinner, finally figured out

Years ago, long before any of this, I wrote about grocery shopping as a balance of four things: price, time, health, and the one nobody accounts for — mental energy. I called it, with great originality, The Grocery Formula. The argument was that the grocery business badly underestimates how much of feeding a family happens in your head.

This new Relay is that same argument more than a decade later, with better tools. Planning dinner is a memory problem because the expensive part is the mental energy of remembering, deciding, forgetting, and then re-deciding at 6:14 in front of an open fridge. AI didn't change that problem; it changed who can build software to carry some of the load — now one person, in the margins of a busy life, can make something useful.

I want to be careful not to oversell it, though. A working app is a long way from a working company — the old Relay was warehouses, trucks, payroll, and years of effort from a lot of people, and I know exactly how much of that I'm not rebuilding. This new one has a landing page at relayredux.com, but it lives in a private TestFlight beta, not the App Store — I built it for my family and a handful of others, and I'm not ready to support it as a real product.

What I learned in doing so is that almost none of the interesting work was the AI itself — it was deciding where it belonged, where it didn't, and how to pull it together into something that made weeknight dinner easier.

And that's the thing I keep coming back to, well past this app. I spend a lot of time thinking about where AI is taking us, and I land somewhere split: excited that one person can now build what used to take a team, and wary of the reflex to hand it the wheel just because it sounds sure of itself. The discipline is the same whether you're building a grocery app, leading a company, or making sense of AI's place in society: use it deliberately, and keep a person in charge.

Father wears his Sunday best. Mother's tired, she needs a rest, the kids are playing up downstairs...

Tags

Close You've successfully subscribed to Night Train.
Close Great! You've successfully signed up.
Close Welcome back! You've successfully signed in.
Close Success! Your account is fully activated, you now have access to all content.
Close Success! Your billing info is updated.
Close Billing info update failed.