Foundation Models in iOS 27: What a 4K Context Window Means for Your Architecture

A 4,096-token context window sounds like a number you can ignore until you meet it. Then a user pastes a long email thread, or a chat session runs for twenty turns, and the session throws instead of answering. The limit is not a tuning detail. It decides what your feature can be, and it should be settled in the architecture, not discovered in a crash report.

Apple’s Foundation Models framework in iOS 27 gives you two models behind one API. The on-device model has a 4K-token context. Private Cloud Compute (PCC) offers, in Apple’s words, “a larger 32K-token context size and stronger reasoning for handling long documents or extended multiturn conversations.” This post is about the decisions that follow from those two numbers, from the point of view of someone who owns a mobile platform rather than a single screen.

What I have and have not verified. The 4K and 32K figures, the offline behaviour and the daily quota below come from Apple’s own developer documentation. I have not run any of this myself, and the docs leave a few things open that matter for design, which I call out where they come up. The code in this post is limited to what Apple publishes.

The two limits that shape everything

The context window is the obvious one. The less obvious one is that PCC is not a bigger version of the same thing. It differs in three ways that change how you build around it.

Two rows in that table do most of the damage. PCC does not work offline, and PCC has a quota. A feature that quietly depends on PCC inherits both. It will fail on a plane and it will fail on the heavy user’s fortieth request of the day, which is exactly the user you least want to fail.

What stays on-device

The default should be on-device, and the reason is not purity. It is that on-device is the only path with no network dependency and no quota, so it is the only one you can promise to every user all the time.

Keep on-device anything that is short, latency-sensitive or must work offline: rewriting a sentence, classifying a single item, extracting fields from a short input, generating a title or a suggestion. If the whole prompt plus the expected answer fits comfortably inside 4K tokens, there is no reason to leave the device.

The part that is easy to get wrong is that the window is shared. Instructions, tool definitions, the running transcript, the user input and the model’s own output all spend from the same 4,096. A system prompt that grows to a page and a few tools with long descriptions can take a large share before the user types anything. Apple documents that a session which exceeds the available context throws LanguageModelError.contextSizeExceeded, and recommends Instruments to analyse token consumption while the app runs. Measure your own fixed overhead first. I would treat that number as a budget line in the design doc.

What goes to PCC

Send to PCC what does not fit in 4K and is worth a network round trip: long documents, long multi-turn conversations, and requests that need deeper reasoning. The reasoning setting is a real lever here. Deeper reasoning trades latency for analysis, so it belongs on the one request where the user is waiting for an answer they care about, not on every call.

Apple’s routing is a one-line change at session creation, and its documentation shows availability gating for older OS versions:

let session = LanguageModelSession(model: PrivateCloudComputeLanguageModel())

Source: Apple, “Adding server-side intelligence with Private Cloud Compute.”

That simplicity is the trap. Because switching models is one line, it is tempting to scatter the choice through view code. Do not. Put the decision in one place that every AI feature calls, so the policy (which requests go to PCC, what happens at quota, what happens offline) is one reviewable file rather than a convention spread across a codebase. This is the same reason an SDK exposes one entry point instead of five.

Designing the handoff

Apple’s guidance for network failure is to retry the request using the on-device model. That is the right instinct, with one catch: a request that needed 32K cannot simply be replayed into 4K. A real fallback needs a smaller version of the same task.

The usual answer is to truncate and hope. The better answer is to decide ahead of time what the degraded behaviour is, per feature. Three strategies cover most cases:

• Summarise, then ask. Compress older turns or a long document into a short summary on-device, then send the summary plus the current question. This costs an extra call but keeps a long conversation inside 4K.
• Chunk and combine. Split a long document into pieces that each fit, process each on-device, then combine the partial results. It works for extraction and classification. It works poorly for anything that needs the whole document in view at once.
• Fail visibly. For features where a degraded answer is worse than none, tell the user the feature needs a connection, rather than returning a confident answer built from a fraction of the input.

The third one is not a failure of design. It is a decision, and it should be made on purpose.

Failure modes to plan for

Each of these is a state your UI and your tests need to cover, not an edge case.

• Context exceeded. Thrown as contextSizeExceeded. Handle it by shrinking the input, not by retrying the same call.
• Quota reached. quotaLimitReached with a resetDate. Show the user when it resets, and fall back to on-device behaviour where the feature allows it.
• No network. PCC is unavailable offline. Check availability before routing, and decide the on-device behaviour in advance.
• Model not ready. The on-device model can be unavailable because it is still downloading, or because the device or region does not support Apple Intelligence. This one is easy to miss because it is a device state, not an error you caused.
• Older OS. PCC is gated to the 27 releases, so anything supporting earlier versions needs the on-device-only path anyway.

Privacy is a design input, not a footnote

Apple states that PCC preserves privacy. The documentation I read is thinner on what that means for your design. It does not say how much of a session’s history is sent with each request, or whether tool calls run on the device or on the server. I would not guess.

Until that is confirmed, the safe working assumption is that anything in the prompt, the instructions or the transcript can leave the device on a PCC request. That points to a concrete rule: filter what goes into a PCC-bound session the same way you would filter what goes into any server call. Strip identifiers the task does not need. Keep sensitive fields out of the transcript. Decide per feature whether the content is allowed to leave the device at all, and make that a property of the feature, not of the model it happens to use.

Where AI fits

For a developer building with these models, the context limit changes how you design the prompt more than how you write the code. A 4K budget forces the discipline that long-context models let teams skip: short instructions, few tools, retrieved context instead of pasted context, and summaries instead of full history. Those habits make the 32K path cheaper and faster too.

The real-world impact is on reliability. An AI feature that works for the median input and throws on the long one is a feature whose worst failures land on the users with the most content, the most engaged ones. The same applies to the quota. Treat both as capacity limits in a platform, the way you would treat a rate limit on any backend, with a defined behaviour at the limit. The on-device model also has adapters, trained with Apple’s toolkit as LoRA layers, but those specialise the model for a task and do not change the window size. They are a separate decision, and they must be retrained per OS version.

Open questions

Some things I would check before committing a roadmap to this:

• What Apple does with PCC request data beyond “preserves privacy,” and what is sent per request.
• Whether tool calls execute on-device when the session uses PCC.
• What the daily quota actually is for a given user, and how it behaves for apps with many users.
• How the mid-session switch from on-device to PCC treats the existing transcript.

None of these are reasons to avoid the framework. They are reasons to put the model choice behind one boundary, so that the answers, whatever they turn out to be, change one file.