In short
Most business AI today runs in the cloud: your app sends a request to a provider’s data centre, a large model processes it, and the answer comes back. Edge AI moves that processing, called inference, onto local hardware. Training a model still usually happens in large data centres. Running it, once trained, can often happen on much smaller machines.
Why run AI locally
| Speed | No round trip over the internet, which matters for robots, voice and anything reacting in real time. |
|---|---|
| Reliability | Keeps working when the connection is slow or drops: in a hall with poor wifi, on a rural site, in a vehicle. |
| Privacy | Images, audio and documents can stay on the device or premises, which can simplify data protection. |
| Predictable cost | You pay for the hardware once and avoid per-request charges, which helps at high volume. |
| Control | You choose the model and when it changes, so behaviour does not shift when a provider updates theirs. |
The trade-offs
Local hardware limits model size. The largest frontier models need data-centre hardware, so local systems usually run smaller models. Those smaller open-weight models have improved quickly and handle many focused tasks well, but for the hardest reasoning or writing tasks a large cloud model is usually still stronger.
You also take on the work the cloud provider used to do: buying suitable hardware, often with a capable graphics processor; installing and updating models; monitoring performance; and securing the device. A model on a laptop that walks out of the door is a data risk of its own.
Techniques such as quantisation, which stores a model’s numbers at lower precision, shrink models so they fit on smaller hardware with some loss of quality. Choosing the right balance is part of the engineering.
Worked example: Jimmy
Jimmy, New Era AI’s Unitree R1 Air humanoid, is a working example. His brain runs on our own hardware rather than a cloud service: microphone to local speech-to-text, to a 26-billion-parameter language model run locally, to text-to-speech and his speaker. Object and person detection from his camera also runs locally, at roughly 120 milliseconds per frame.
We built it that way because he works at trade shows, conferences and colleges, where wifi is often poor and a pause of several seconds kills a conversation. By streaming the first sentence of each reply as soon as it is ready, and choosing the voice engine by measurement, the time from text to his first spoken sentence came down to about half a second.
The same design keeps conversations and camera images on our own equipment, and his behaviour does not change unless we change it.
Where edge AI fits in a business
Robots, drones and vehicles that must react instantly. Cameras that count, inspect or detect locally and send only the result. Sites with poor connectivity. Organisations handling sensitive information, such as health, legal or financial records, that prefer it never leaves their premises. And high-volume, repetitive tasks where per-request cloud charges add up.
Many real systems are hybrid: a small local model handles the fast, routine or private work, and harder requests go to a cloud model under clear rules.
Before buying hardware, test the candidate model on a sample of your real tasks and time it. Accuracy and speed on your own work matter more than any published benchmark.
When this matters
Relevant if latency, connectivity or data sensitivity rule out a cloud-only approach. Jimmy is New Era AI’s showcase for local AI, and our AI strategy work can assess whether a local, cloud or hybrid design suits your use case.
Questions
Is edge AI the same as on-device AI?
On-device AI is one form of edge AI. Edge AI also covers servers on your own premises or near the data source.
Can we run a ChatGPT-style model on our own computers?
Yes. Open-weight language models can run on suitable local hardware. They are usually smaller than the leading cloud models, so test them on your real tasks before committing.
Is local AI automatically GDPR compliant?
No. Keeping data on your premises can reduce some risks, but you still need a lawful basis, security, retention rules and the other UK GDPR obligations.
Does Jimmy need the internet to talk?
No. His speech recognition, language model and voice run on our own hardware, which is why he keeps working in halls with poor wifi.