IoT Power Monitoring

Commercial submetering hardware is priced per circuit and usually locks the data behind a vendor dashboard. The client wanted per-circuit visibility across a building, cost estimates in pesos rather than kilowatt-hours, and the ability to cut power to a load remotely, without paying per point of measurement.

So the whole thing had to be built end to end: firmware on the metering node, an ingest and analytics backend, and a dashboard. The real constraint was not the electronics. It was that the entire platform had to run inside free hosting tiers, on a network that drops out. Both of those shaped the architecture more than any feature on the list.

Each node is an ESP32 driving four PZEM-004T energy meters and four solid-state relays. The four meters share a single hardware serial line and are addressed individually over Modbus (0x01 through 0x04), which is what makes one microcontroller able to meter four circuits instead of one.

Wi-Fi credentials are not compiled in. On first boot, or whenever it cannot associate, the node opens a WiFiManager captive portal that never times out, so the device is commissioned by connecting to its access point rather than by reflashing it. Time comes from NTP, which matters because the backend trusts the device's timestamps when replaying buffered data.

A 30-second hardware watchdog sits underneath all of it. A node deployed in an electrical cabinet is not somewhere you want to visit to press reset.

Relay control is pull-based: the node polls its own relay state every 10 seconds and actuates on change. Toggling a load from the dashboard therefore takes effect within 10 seconds rather than instantly, which was an acceptable trade for never having to expose the device to inbound connections.

The site's Wi-Fi was unreliable, and a metering system with holes in its history is not a metering system. So the node treats the network as optional.

Every reading is appended to an SD card. When the backend is reachable, readings post normally. When it is not, they queue to pending.csv and a flag is set. The node keeps checking backend health once a minute, and on the next successful check it replays the entire pending file, then deletes it. If the delete fails, the flag stays set and it retries rather than assuming success.

This is why the ingest path had to be idempotent, and it is the reason the daily rollup upserts on a unique (deviceId, date) index instead of inserting. A replayed batch spanning a day boundary recomputes that day rather than duplicating it.

Hot and cold storage split

InfluxDB holds raw readings for real-time charting and recent history. A daily job aggregates each device's readings into a single summary row in MongoDB. Monthly and historical analytics read only those summaries, which is why they still work after the raw data ages out of the free-tier retention window.

The cost: two databases to operate, two client libraries, and a dual write path. Worse, the rollup runs as an HTTP endpoint (/internal/aggregate/daily) triggered by an external scheduler rather than an in-process cron. If the scheduler misses a day, that day is missing from cold storage and nothing notices. The upsert makes a manual re-run safe, but "safe to re-run" is not the same as "runs".

Batching writes to fit the free tier

Four sensors reporting every 10 seconds is 120 readings per 5 minutes per node, against a free-tier budget of roughly 300 writes per 5 minutes. Writing each reading individually would have consumed most of the allowance with a single node deployed.

The batcher buffers readings and writes when it has 10 of them, or every 30 seconds, whichever comes first, with retries on failure.

The cost: up to 30 seconds of readings live only in process memory. A backend restart at the wrong moment loses them, and unlike a network outage, the node has no idea this happened, so its SD buffer will not replay them. Fixing that properly means acknowledging writes only after the flush, which the current ingest endpoint does not do.

Push over polling

Rather than having dashboards poll for updates, the ingest path broadcasts each reading to sockets that have subscribed to that specific device.

const broadcast = ({ deviceId, type, data }) => {
  const message = JSON.stringify({ type, data });
 
  for (const client of clients) {
    if (
      client.readyState === client.OPEN &&
      client.subscriptions?.has(deviceId)
    ) {
      try {
        client.send(message);
      } catch {
        client.terminate();
        clients.delete(client);
      }
    }
  }
};

Subscriptions are per-connection, so a dashboard watching one circuit does not receive traffic for the other three. A 30-second ping/pong heartbeat terminates connections that stop answering, because a dropped browser tab otherwise stays in the client set forever and the send only fails much later.

Aggregate in the query, not in Node

Budget alerts need 30 days of consumption per device. Pulling raw readings to compute that would mean moving roughly a million rows per device into application memory.

SELECT
  device,
  DATE_BIN('1 hour', time) AS hour_bucket,
  AVG(power)  AS avg_power,
  COUNT(*)    AS reading_count
FROM "power_reading"
WHERE time >= now() - interval '30 days'
GROUP BY device, DATE_BIN('1 hour', time)
ORDER BY hour_bucket ASC

Bucketing to the hour in the database returns about 720 rows per device instead, and energy is summed from the bucket averages.

The cost: this approximates. Each bucket contributes avg_power × 1 hour, so a circuit that spikes hard for two minutes inside an otherwise idle hour is averaged into something gentler than reality. For budget alerts, where the question is "roughly how much has this cost this month", that is the right trade. For anything billing-grade it would not be.

The offline buffering on the device is the part of this system I am most confident in, and it makes the gap on the server side more obvious by comparison. The node will not lose a reading to a network outage, but the backend will lose up to 30 seconds of readings to a restart, and there is no acknowledgement protocol that would let the node know to resend them. The device is more careful with the data than the server is.

I would also make the daily rollup own its own schedule. Putting it behind an HTTP endpoint made it easy to trigger during development and easy to forget in production, and a missing summary row is invisible until someone opens a monthly report and finds a gap.

The last one is a measurement point I never added: nothing records how long a node was offline, or how many readings were replayed. That information existed on the device and was thrown away at flush time, and it is exactly what you want when a client asks whether the data can be trusted.