On-device or cloud? Where the AI in earbuds and glasses actually runs

Header photo: CCadio / Wikimedia Commons, CC BY 4.0

In almost every product shipping today, the honest answer is: a very small amount runs on the device, most of it runs on the paired phone, and the expensive part runs in a data centre. The question "on-device or cloud" is a bad question, because there are three tiers, not two. The number that settles most arguments is 10.6 MB — the amount of RAM on Qualcomm's flagship audio chip for earbuds, the S7 platform. A phone sold in 2026 carries 8 GB. Sentence-level machine translation, compressed to 4-bit and stripped of unused languages, still runs to hundreds of megabytes. So when a supplier says "on-device AI translation" about a pair of earbuds, one of three things is true: it is a phrasebook of a few hundred commands, the phone is doing the work, or the bud is carrying flash and RAM nobody put on the spec sheet. Ask which.

This matters more than it did two years ago, for a reason that has nothing to do with AI. Memory prices tripled through the first half of 2026, and every megabyte you move onto the device is now a line item you will argue about in a meeting. The architecture is no longer a software decision. It is a purchasing decision, and it belongs in the RFQ.

Three tiers, not two

The marketing binary is device versus cloud. The engineering reality has a middle tier that does most of the work: the phone. A modern translation pipeline is four stages — capture, speech-to-text, machine translation, speech synthesis — and different products split those four stages in different places. The split is the product.

Read that table as a buyer and the third row should make you nervous. A cloud tier is not a feature. It is a permanent operating cost attached to a product you have already sold, and someone has to fund it — either the end customer through a subscription, or you through a per-unit allowance baked into the price. Products that quietly assumed the second option are the ones that get a subscription button added eighteen months after launch.

The 10.6 MB that decides the argument

Qualcomm's S7 and S7 Pro Gen 1 sound platforms are the reference point for what an earbud can do locally. The headline is a 64 GOPS micro NPU, a dedicated AI engine on the audio die, alongside two 500 MHz DSP cores and one 250 MHz core. Qualcomm quotes almost 100 times the AI throughput of the previous generation, six times the general compute, and 10.6 MB of RAM — up from 2.64 MB on the S5 Gen 2. Those are vendor figures, and they are directionally right.

Now do the division. A keyword spotter for a wake word occupies a few hundred kilobytes. A small voice activity detector is smaller still. Adaptive noise cancelling lives in the DSP, not the NPU. All of that fits in 10.6 MB with room to spare, which is exactly why an earbud can hear you say a wake word for 16 hours on a 50 mAh cell without killing the battery.

A translation model does not fit. Not a stripped one, not a 4-bit one, not a single language pair. Even after quantisation, distillation and language pruning, a usable speech-to-speech pair lands in the hundreds of megabytes, because the model has to hold two vocabularies, two acoustic models and enough context to not translate "we need 3,000 pieces" as "we need 3,000 poems".

Which is why the honest architecture for a 2026 earbud is: wake word and voice activity on the bud, everything linguistic on the phone, and the hard cases in the cloud. We wrote about the power arithmetic behind that split in our piece on the five technology shifts reshaping earbuds in 2027; the conclusion has not moved. What has moved is the price of the alternative.

Glasses are the exception, and only just

Smart glasses get away with more local compute because they have more room, more thermal mass and a bigger battery — and because a frame can hide a second chip. Qualcomm's AR1 Gen 1, the part behind most of the category, is a 4 nm design with four Kryo cores running up to 1.9 GHz, an Adreno 621 GPU, a third-generation Hexagon NPU, dual image signal processors for 12 MP stills and 6 MP video, support for up to eight microphones, and Wi-Fi 7. In the glasses it is the main brain.

But look at the board and you find the second chip, and the second chip is where the interesting decision lives. Teardowns of Meta's display glasses describe an NXP RT700-class co-processor sitting alongside the Qualcomm part: a Cortex-M33 core at 325 MHz plus a separate always-on sensing domain at 250 MHz with its own small NPU. That co-processor handles wake word detection and audio routing so the 4 nm chip can go to sleep. Two chips, two power domains, one product.

That is the signature of a design that has actually been engineered rather than assembled. If a glasses supplier shows you one chip doing everything, one of two things is true: the always-on function is expensive enough to cost you battery life, or the always-on function is not really always on. Ask which, and ask what the standby current is with the display off and the assistant armed.

Even with that headroom, glasses stay on the same three-tier ladder. Meta's AI answers are widely reported to be processed in the cloud; the local NPU handles wake word, audio enhancement and image pre-processing, not the answer. A 69 g frame with a 6-hour battery has roughly 2.5 watt-hours to spend across cameras, display, radios and compute. There is no version of that budget where a 2 GB model runs continuously on the frame.

What the latency budget actually looks like

Battery is the constraint people talk about. Latency is the one that decides whether a product feels like a translator or a walkie-talkie with a delay.

The human number that matters is a conversational turn gap of roughly 200 ms. Above that, people start talking over the machine. A cloud-only loop lands at 1.5 to 3 seconds in the field, which is the difference between following a conversation and reading a transcript of the last one.

This is why the good implementations cheat. Apple's Live Translation, the one that runs with AirPods, does not wait for a sentence to end: it starts speaking as soon as it has one coherent thought, and it speeds up delivery when it senses the speaker is running ahead. It also runs the models on the iPhone, not on the bud and not on a server, which is why Apple lists an iPhone 15 Pro or newer as a requirement. That requirement is a polite way of saying the phone is the computer.

There is a commercial consequence hidden in that, and it is the least discussed risk in the category. When the phone does the work, the product's performance depends on hardware you do not control: the customer's phone model, its RAM, its operating system version, its region. A feature that works beautifully on a flagship and stutters on a three-year-old mid-range Android is not a bad feature. It is a feature whose spec sheet was written without its execution environment.

The 2026 plot twist: memory became the expensive part

For a decade, the engineering rule was simple — put more on the device, because compute keeps getting cheaper. That rule broke this year, and it broke in the least glamorous component in the product.

TrendForce data reported in September 2026 shows general-purpose DRAM contract prices rising 93% to 98% quarter on quarter in the first quarter, then another 58% to 63% in the second. NAND went up 55% to 60% in both quarters. A mainstream DDR4 8 Gb part that cost US$2.90 in June 2016 was trading at about US$21 in June 2026. The cause is not demand for earbuds. It is AI data centres absorbing the supply, and consumer electronics queuing for what is left.

IDC already names memory tightness as a factor pushing wearable prices up in 2026. That lands directly on the architecture question. If you want a language pack on the wearable instead of the phone, you are buying flash at the worst moment in a decade, and you are buying it per unit, in a product whose average selling price is being pushed down at the same time: IDC puts smart glasses at roughly US$376 in 2026, compressing to about US$229 by 2030.

Put those two curves together and the conclusion is uncomfortable but clear. Local AI has to be cheap or it has to be justified. A buyer who asks "can it work offline" without asking "how much flash does that sentence cost me per unit" will get a quote they cannot defend three months later when the memory market moves again.

What the market is actually shipping

IDC puts 2026 smart glasses shipments at 13.6 million units, up 41% year on year, and hearables at 407.6 million, up 4.0%. Meta held 69.2% of the glasses market in the first quarter. Those are the numbers everyone quotes, and they say nothing about architecture.

The architectural number comes from China, where the catalogue is deepest. IDC's China tracker reported that in the third quarter of 2026, 35.7% of smart glasses shipped with on-device AI support, and 53.9% connected to a large model. Read those two figures together and the picture is: about a third of the market can do something with the network switched off, and about half is explicitly a thin client for somebody's server.

That is not a criticism. It is a description of where the cost sits today. But it means the phrase "AI glasses" covers products with completely different economics, and a buyer comparing quotations from three factories may be comparing three architectures without knowing it.

Regulation moved into the architecture

There is now a legal reason to care where inference happens, and it arrived in August. Article 50 of the EU AI Act applies from 2 August 2026. It requires that a system designed to interact directly with people tells those people they are dealing with an AI, unless it is obvious. It requires that synthetic audio, image, video and text be marked in a machine-readable format so it can be detected as generated. Systems already on the market before that date have until 2 December 2026 to meet the marking requirement. Infringements carry fines up to EUR 15 million or 3% of worldwide turnover, whichever is higher.

For a translation earbud or a pair of AI glasses sold into Europe, that changes the paperwork in a specific way. A product whose voice never leaves the device has a much shorter conversation with a data protection authority than one whose audio is streamed to a processor in a third country. The second is not illegal — it is just more work: a lawful basis, a retention rule, a processor agreement with the model vendor, a description of where the audio goes, and a way to answer a customer who asks. One of those designs needs a lawyer in the first week. The other needs one in the third month.

There is also the disclosure itself. If your glasses speak a translation into the wearer's ear, that is not an AI interacting with the other person — the other person is talking to a human. But if the product answers questions aloud, or if a voice assistant speaks to a bystander, you are in scope for the disclosure rule. This is the sort of thing that gets discovered during a retail listing review, three weeks before a container ships.

A programme that got this wrong

A distributor in southern Europe came to us in late 2025 with a specification he had written himself: AI translation glasses, on-device processing, no app required. Three lines. The demo he had seen at a trade fair did all three, in the sense that the glasses were on a face and translation came out of them.

The demo was running the app on the salesman's phone, which was in his pocket, paired to the glasses, on the venue Wi-Fi. Nobody had lied. The specification had simply described the user experience rather than the architecture, and the factory had quoted what was asked.

We rewrote the requirement into three lines and the project changed shape. Wake word and audio capture on the frame, with a stated standby current. Speech recognition and translation on the phone, with a named minimum device and a stated language-pack size. Cloud fallback for six rare language pairs, with a named data region. The bill of materials went down, because the flash came off the frame. The launch slipped by five weeks, because the app had to be rebuilt around downloaded packs. And the distributor now opens every demo by asking the salesman to put the phone in airplane mode, which is the single most useful test in this category.

What to write into the RFQ

Eight lines. They fit on one page, and they separate a product from a demo.

1. Where does inference run, per feature. Not per product — per feature. Wake word, speech recognition, translation, synthesis, vision query. A supplier who answers this in one sentence has thought about it.

2. Declare the RAM and flash on the wearable and in the app. If the number is 10.6 MB on the device and 8 GB on the phone, that is a fine answer. It is the answer that tells you the design is real.

3. Airplane-mode demonstration. How many language pairs work with no network, and how many seconds until the first translated word. Time it yourself, with a stopwatch, in the room where you will sign.

4. Latency, measured how. On which phone, at what distance, in what background noise. A datasheet number measured in a quiet office at 30 cm is not the number your customer will experience in a market.

5. Model version and update path. Who pushes the model, how often, and over the air or only through an app store release. A model frozen at the factory is a product that ages at the speed of the language it translates.

6. The data path. What leaves the device, where it is stored, for how long, and which sub-processors touch it. This one is not optional if you sell into the EU.

7. Degraded behaviour. What still works with no phone and no network, stated as a list rather than a promise. "Basic translation" is not a specification; "14 language pairs, 2.5 seconds, 300 phrases" is.

8. Who pays for the cloud after year two. Per-unit allowance, subscription, or nothing because the feature does not exist. Get this in writing before the tooling is cut, because it is the clause that decides whether the product is still profitable in year three.

Our stance

Do not buy "on-device AI" as a feature label. Buy the split, in writing, with a number attached to each tier. If a supplier cannot state the split in one sentence, assume the work is happening somewhere else, and price the product as cloud-dependent — because it is.

And do not assume on-device is automatically better. A phone in a pocket is still somebody else's computer. The device you control is the one whose behaviour you can promise in a contract, and in 2026 that is usually the smallest tier of the three.