Arya
Software Engineer / AI & ML
Most multi-agent write-ups assume a server. A router calls an orchestrator, the orchestrator calls out to GPT or Claude, agents talk to each other over HTTP, and the whole thing lives behind an API key. That architecture doesn't fit an iOS app that has to work offline.
Apple's Foundation Models framework, new this year, runs a language model on-device. No API key, no per-token bill, but also a much smaller context window and a much smaller model to reason with it.
This post walks through a real app I built, a moka-pot coffee assistant, to show what multi-agent design looks like when the constraint is a token budget instead of a network call.
Everything here targets iOS 26, where a session's tools and instructions are frozen the moment you construct it. iOS 27, currently in developer beta, adds APIs that remove that constraint entirely. That's a different enough shape that it gets its own post once iOS 27 ships. See the note near the end.
The constraint that shapes everything
SystemLanguageModel.default gives you an on-device model with a context window of about 4096 tokens. Every tool you hand a LanguageModelSession costs tokens just to describe, its name, its description, and the schema for its arguments. In this app, giving one session all seven tools (bean corpus search, owned-bean lookup, brew advice, a taste quiz, web search, nearby cafes, and a choice-offering tool) cost 1785 tokens before the conversation even started. That's nearly half the budget gone to tool schemas the model wasn't going to use on most turns.Router plus specialists
The app defines a
Specialty enum with four cases, beans, cupboard, shopping, and general as the fallback.Each specialty maps to a subset of the tools.
Before the first message of a conversation, a separate, tiny session classifies the question.
The router session has no tools at all. It exists to output one enum case, which costs on the order of thirty tokens. If the router isn't confident (
unsure), it falls through to general, which holds every tool. A wrong guess that lands on beans when the user actually wanted shopping costs a missing tool and a bad answer. A guess that lands on unsure costs some tokens but keeps the app working, falling back to the full generalist rather than confidently answering with the wrong specialist.Once a specialty is chosen for a conversation,
ChatManager rebuilds the model session with just that specialty's tools, and every subsequent tool-calling turn only pays for what it actually needs.Instructions scale the same way tools do
Tool schemas aren't the only thing eating the budget. Instructions do too, so they're split the same way.
coreInstructions covers what every specialist needs (the app is moka-pot only, cite real beans, say when unsure). Each specialty then adds only the paragraphs about the tools it actually holds.The
general specialist gets the full persona, every briefing concatenated. A specialist gets the core plus one briefing. The bigger saving is that a specialist is never told how to use a tool it doesn't hold.Where tools deliberately overlap
Comparing a bag in someone's cupboard to its reference profile needs both the owned-bean data and the corpus, so the domain boundary isn't clean here. Rather than force a hard split, the
cupboard specialist gets its own compareToReference path and the beans specialist keeps corpus search. The overlap is a design decision, not an oversight. A misrouted comparison would cost a wrong answer, which is worse than a few extra tokens of tool description.What a tool actually costs
A
Tool in this framework is four things, a name, a description, an Arguments type, and a call(arguments:) method. The name and description are the only information the model has when it decides whether a tool is relevant to a prompt, and a vague description is the reason i found that make a model skips a tool it should have used.Arguments is itself @Generable, so the model doesn't hand you a string to parse, it fills in a typed struct, and @Guide narrows what it's allowed to produce. BeanCorpusTool's arguments look like this..range(1...5) isn't documentation, it's a constraint the model can't violate. The framework enforces it during generation, not after.None of this is free. A small tool, once its name, description, and argument schema are serialized into the prompt, costs on the order of 68 tokens, measurable directly with
tokenCount(for:) against a tool array before you ever call it.Structured output keeps tools honest
Every tool in this app returns a
@Generable struct, Apple's macro for typed, schema-constrained generation, rather than a free-text blob. BeanCorpusTool is a representative example.The status enum matters more than it looks like it should.
noMatchesInIndex means the corpus was searched and came up empty, a real answer. indexUnavailable means the search never ran. Those are different facts, and the app's instructions tell the model to treat them differently. "Any status ending in Unavailable, plus indexStale, means the search did not run. Say it is unavailable, never that nothing was found." Without a typed status, a small on-device model is far more likely to blur "I found nothing" and "I couldn't check" into the same sentence.What this costs you
Every conversation now pays for a routing call before it pays for the real one, and a wrong route means a worse answer instead of just a slower one. The
general fallback exists precisely to bound that risk, but it means the generalist path (full tool set, full instructions) still has to be correct and still has to fit, so specialization doesn't remove the original budget problem, it just makes the common case cheaper.What is on iOS 27
iOS 27, currently in developer beta, adds
DynamicProfile, DynamicInstructions, historyTransform, and @SessionProperty. A profile's tools and instructions re-evaluate before every request, so a specialty switch stops being a session rebuild and becomes a property change, with no KV-cache reset and no lost context.