BlogIoT and edge AI

AIoT explained: what changes when connected devices get AI

AIoT is what you get when connected devices stop only reporting and start deciding. Models on the device, on a local gateway and in the cloud turn sensor readings into events and actions, so the value no longer depends on someone watching a dashboard. The hard parts are the old IoT ones (connectivity, power, security, data quality) plus a new one: keeping models correct on hardware you cannot reach.

From dashboards to decisions

The first generation of IoT projects followed one pattern. Fit sensors, connect them, store the readings, draw charts. The technology worked. The value often did not arrive, because a chart only helps if the right person looks at it at the right moment and knows what it means. Many deployments ended as a wall screen nobody read and an alarm list everybody ignored.

AIoT, short for artificial intelligence of things, changes who does the looking. A model watches every signal all the time and produces something a person or another system can act on: this compressor is behaving unlike itself, this bay has been blocked for twenty minutes, this building can drop its cooling load for the next hour.

It is a label, not a product category. Nobody sells a box of AIoT. In practice three things change:

  • Devices decide. A sensor node that used to send raw readings now sends conclusions, and sometimes acts on them.
  • Fleets learn. Data from a thousand units trains models that no single unit could, and one unit’s odd behaviour stands out against the rest.
  • People ask. Operators question the fleet in plain language and delegate routine responses to software.

What does not change is the physics. Radio links drop, batteries run down, and a sensor glued to the wrong pipe reports the wrong temperature with great confidence.

Where should the intelligence sit?

There are three places a model can run, and choosing between them is most of the architecture.

Where What belongs there Typical work Main constraint
On the device Decisions that cannot wait, must work offline, or use data that should not leave Wake-word and keyword detection, vibration or current anomaly scoring, simple vision, turning raw signals into events Memory, power and the difficulty of updating
On a gateway or site server Anything that needs several devices’ data at once, or a model too big for a sensor Multi-camera vision, site-level anomaly detection, a small language model, buffering and protocol translation One more machine to secure and maintain at every site
In the cloud Anything that benefits from the whole fleet or from large models Training, fleet comparison, forecasting, natural-language queries, agents, long-term storage Latency, bandwidth cost and dependence on the link

The device tier is a subject of its own, with its own hardware classes and ways of shrinking models. We cover it in edge AI explained and will not repeat it here.

The gateway tier is the one teams underrate. A small industrial computer on site can run models a sensor never could, keep working when the uplink fails and hold raw data locally. The platform vendors have moved the same way. Microsoft describes Azure IoT Operations as a data plane for the edge with a local MQTT broker, able to operate offline for up to 72 hours.

The rule we use: place each decision, not each product. For every decision the system makes, write down how fast it must be, whether it must work without a connection, how sensitive the data is and what a wrong answer costs. A door-forced alarm and a monthly efficiency report can come from the same sensor and belong at opposite ends of the table.

Which use cases hold up?

Six do, in our experience. It helps to separate them from the ones that only demo well.

Anomaly detection. A model learns what normal looks like for each machine and flags departures. It holds up because it needs no history of failures, which most sites do not have. It is the foundation of predictive maintenance, which has its own guide.

Vision. A camera plus a model becomes a general-purpose sensor for counting, presence, queue length, fill level and inspection. Processing on site and sending events instead of video keeps bandwidth and privacy manageable. See computer vision at the edge.

Voice on the device. Wake words and a fixed set of spoken commands run on a microcontroller. Espressif’s ESP-SR library, for example, provides wake-word detection, command recognition and an audio front end for its ESP32-S3 chips. That suits gloves-on, hands-busy work: “start cycle”, “next step”, “call supervisor”. Open conversation needs a larger model on a gateway or in the cloud, and our article on small language models covers how small that model can be.

Energy optimisation. Heating, cooling, compressed air and refrigeration all have slack: a setpoint can move within limits without anyone noticing. A model forecasts load and shifts consumption inside those limits. Two conditions keep it honest. The limits are enforced by the controller, not by the model. And savings are measured against a baseline period adjusted for weather and output, not against a guess.

Questions over fleet data. “Which freezers ran above minus 15 for more than an hour last night?” A language model turns the question into a query against the time-series store and the asset register, runs it, and reports the result with the query attached. AWS documents this pattern in its IoT SiteWise Assistant, which draws on IoT data, asset models and manuals to summarise events and answer follow-up questions. It works when assets are named and modelled consistently, which makes it as much a data engineering job as an AI one. The model must query, never recall.

Agents that act on alerts. An alert fires. Software gathers the last 48 hours of data, checks the maintenance history, looks for the same pattern on sister units, drafts a work order with the evidence attached and routes it to the right person. This removes the dull half of an engineer’s response. These are ordinary agentic workflows with one extra rule: anything that changes a physical setpoint or dispatches a technician waits for a person’s approval until the record justifies more trust.

What does not hold up: a neural network where a threshold would do, and generative models placed inside the control loop of equipment that can hurt someone. Control stays deterministic. AI advises it.

What is hard?

Connectivity. Design for the link being down. Devices buffer and forward, gateways keep deciding, and the cloud tolerates data that arrives late and out of order. Cellular data priced per megabyte will shape what you can send more than any architecture diagram.

Power. On a battery, inference is a budget line. The usual answer is a cascade: a cheap trigger, such as motion, a threshold or a tiny always-on model, wakes a larger model only when there is something to look at. Decide the duty cycle before choosing the chip.

Updating models in the field. A model is software that goes stale as the world changes. It needs signed updates, rolled out in stages, with a way back, all of which the edge AI article covers. The AIoT twist is version skew. A large fleet runs several model versions at once, and every event must carry the version that produced it, or your analytics will compare unlike with unlike.

Device security. Every device is a computer on someone’s network. Unique credentials per device, signed firmware and a way to report and fix vulnerabilities are the baseline, and increasingly the law. The UK’s product security regime has banned shared default passwords on consumer connectable products since April 2024. Under the EU’s Cyber Resilience Act, reporting obligations for manufacturers have applied since 11 September 2026 and the main obligations apply from 11 December 2027. AI adds to the attack surface. Model files are intellectual property sitting on a device someone can open, and an agent with write access to a fleet is a target worth attacking.

Data quality. Models inherit every flaw in the data: drifting sensors, clocks that disagree, units that changed after a firmware update, assets renamed in one system and not another. Most AIoT schedules slip here. The pipeline that prevents it is the subject of from sensor to model.

A realistic first project

Take a grocery chain with 60 small shops, each with a dozen refrigerated cabinets and a cold room. Every unit already has a temperature probe and a door sensor reporting to a dashboard. Alarms are fixed thresholds, so they fire on every defrost cycle and every delivery, and staff have learned to silence them. Now and then a compressor fails overnight and a shop loses its stock.

A first project that fits in a quarter looks like this.

  1. One decision. “Is this unit recovering from defrosts and door openings the way it normally does?” Nothing else yet.
  2. Use the data already there. A few months of history per unit is usually enough to learn each one’s cycle. Before modelling, fix what the audit finds: probes mapped to the wrong cabinet, gateway clocks that drift, gaps.
  3. Put the model where the data is. Here that is the shop gateway, so it keeps working when the broadband fails. A small statistical or tree-based model per unit will do. Nothing here needs a neural accelerator.
  4. Replace the alarm, do not add to it. The learned model takes over from the threshold for everything except a hard food-safety limit, which stays as a fixed rule.
  5. Send alerts where work happens. Into the maintenance contractor’s ticket system, with a chart of the last three days attached.
  6. Close the loop. Each ticket gets an outcome: real fault, false alarm or nothing found. That label is the only way the model improves.

Once alerts are trusted, add the second layer. A fleet view in the cloud compares each unit with others of its model and age. A question interface lets the operations lead ask about last night without learning a query language. An agent drafts the contractor ticket and proposes a priority, and a person approves it.

The order is deliberate. Teams that begin with the chat interface, because it demos best, find it answering questions over data nobody has cleaned.

How do you tell it is working?

Not by model accuracy alone. Watch the operation:

  • Alert precision. Of the alerts raised, how many led to real work. This is the number that keeps staff listening.
  • Time from fault to action. From the first abnormal reading to a ticket in someone’s queue.
  • Missed events. Failures the system did not flag, found afterwards. Review every one.
  • Coverage. The share of devices online, reporting and running the current model version.
  • Cost per device per month. Connectivity, cloud and support together, set against the loss the system is meant to prevent.

If the first two are not improving after a couple of months, the cause is usually the data or the alert routing, and rarely the algorithm.

How to start

Spend the first week on an audit, not a model. Choose one asset type and list the decisions people make about it today. Pull three months of its data and check four things: are the timestamps right, are the units consistent, is each sensor attached to the asset the system says it is, and how much is missing.

Then make the first decision, which is about scope and ownership: which single decision will software make, where must it be made, and who is accountable when it is wrong. Hardware, model type and platform follow from that answer.

This is the work our IoT and edge AI team does, from the first data audit through to fleet updates.

Questions

Asked often.

What does AIoT mean?

AIoT stands for artificial intelligence of things. It describes connected devices and fleets where machine learning models, running on the device, on a local gateway or in the cloud, turn sensor data into decisions and actions. Plain IoT collects data and shows it to people. AIoT has the system interpret the data and act, with people supervising.

Is AIoT the same as edge AI?

No. Edge AI is one part of it: running a model on or near the device. AIoT covers the whole system, including models that run in the cloud over data from many devices, language interfaces for asking questions about a fleet, and agents that respond to alerts. Most AIoT projects use a mix of all three locations.

Do we need new hardware to add AI to an existing IoT deployment?

Often not at first. If devices already send data to a gateway or the cloud, anomaly detection, fleet analytics, natural-language queries and alert-handling agents can all run there with no change to the devices. New hardware is needed when a decision must be made on the device itself, for speed, privacy or offline operation.

What is a good first AIoT project?

One decision, on one type of asset, using data you already collect. Replacing a noisy threshold alarm with a learned model of normal behaviour is the usual candidate, because it needs no failure history and its success is easy to measure: fewer false alarms, and real faults raised earlier. Add language interfaces and agents after that works.

Not on the list? Ask us directly

Keep reading

More notes from the build.

All blogs

Let’s build your stack.

Send a brief and we’ll reply with next steps.