Mastodon Politics, Power, and Science: The local IoT Logging Architecture Addition

Saturday, May 30, 2026

The local IoT Logging Architecture Addition

J. Rogers, SE Ohio

Pico W — Real-time data source only: - Sensors read → registry updated → FIFO sync → served as JSON - No ring buffers, no SD cards, no flash wear, no graph rendering - Just the same /api/data?idx=0,3,5 endpoint, polled as always

Home Automation Bridge (desktop, Raspberry Pi, NAS) — All persistence and visualization: - Polls the Pico’s /api/manifest once → knows every item - Registers with api to be a graph provider - Polls /api/data every N seconds → timestamps and stores - Builds a time-series database (SQLite, InfluxDB, even flat files) - Renders graphs, dashboards, alerts - Can aggregate multiple Pico W devices - Still bridges to home automation software

What This Means For Your Framework

The Pico side changes very little. That’s the beauty — it already exposes everything the bridge needs:

Bridge                          Pico W
─────                           ──────
GET /api/manifest         →     Full item list with IDs, names, units, types
GET /api/data?idx=all     →     Current values
GET /api/identity         →     Device name, location, unique ID
need to add a register graph agent call

The bridge can discover devices via mDNS, pull their manifests, and start logging automatically. A new Pico W device joins the network, the bridge sees picow-iot-device.local, queries its identity and manifest, and starts polling.

The framework’s job was to make the device discoverable and interrogable. We already did that. The logging, graphing, alerting — that’s a separate system that treats the framework as a first-class data source.

And the bridge can be smarter than the Pico ever could be: - Downsampling old data (keep 5s resolution for 24h, 1min for a week, 1hr for a year) - Anomaly detection - Multi-device correlation (living room temp vs. outdoor temp) - Push notifications - Integration with Home Assistant, MQTT, etc.

The Pico doesn’t need to know any of that exists. It just needs to serve clean, structured data on demand. Which it already does.

// Registry — what data exists
SENSOR_AUTO("temp_a", "Temperature A", 6004, "°F", readAM2302a_temp);

// Layout — where to put it on the page
{"w_temp_a", "sensor_card", "Temperature A", "temp_a", W_TEXT, ""},

// Help — what the tooltip says
{"help_temp", "<b>Temperature A</b><br>AM2302 sensor on GPIO 2."},

What We Add

A graph table — defines what graphs exist and how they’re built:

const GraphNode graph_table[] = {
    // id              title            series[]           hours   interval
    {"temp_24h",       "Temperature A — 24 Hours",  {"temp_a"},          24,     300},
    {"temp_humidity",  "Temp & Humidity",           {"temp_a", "humidity_a"}, 12, 120},
    {"system_health",  "System Overview",           {"cpu_temp_f", "free_ram"}, 6, 60},
};

Where It Goes In The Layout

const LayoutNode layout_table[] = {
    // ... existing nodes ...
    
    // A new card with graphs
    {"history_card",   "main_page",    "History",           "",         W_CARD,     ""},
    {"graph_temp_24h", "history_card", "Temperature 24h",   "temp_24h", W_GRAPH,    ""},
    {"graph_sys",      "history_card", "System Trends",     "system_health", W_GRAPH, ""},
    
    // Or inline on the system page
    {"graph_cpu",      "status_card",  "CPU History",       "system_health", W_GRAPH, ""},
};

What The Framework Does With It

At page render time, it sees W_GRAPH, looks up the graph id in graph_table, and injects:

<div class="card">
  <h3>Temperature A — 24 Hours</h3>
  <canvas id="graph_temp_24h" 
          data-series="temp_a" 
          data-hours="24" 
          data-interval="300"
          data-title="Temperature A">
  </canvas>
</div>

And the JavaScript that’s already injected into the page gets a new section:

// Graph initialization — runs once at page load
document.querySelectorAll("canvas[id^=graph_]").forEach(canvas => {
    const series = canvas.dataset.series.split(",");
    const hours = parseInt(canvas.dataset.hours);
    // Build a Chart.js / lightweight SVG graph
    // Initial data fetch from bridge API
    // Set interval to refresh
});

The Bridge API

The graphs don’t poll /api/data — they hit the bridge:

fetch(`http://bridge.local:8080/series?device=weather_a&series=temp_a&hours=24`)

The bridge already has the time-series database. It returns JSON the graph renderer can consume directly.

The Developer’s Job

// 1. Define the data (already done)
SENSOR_AUTO("temp_a", "Temperature A", 6004, "°F", readAM2302a_temp);

// 2. Define the graph
{"temp_24h", "Temperature A — 24 Hours", {"temp_a"}, 24, 300},

// 3. Place it on a page
{"graph_temp_24h", "history_card", "Temperature 24h", "temp_24h", W_GRAPH, ""},

They never write a SQL query, a Chart.js config, an API endpoint, or a canvas element. They declare what data they want graphed, over what time window, at what resolution, and where on the page it goes if there is a graph provider registered.

The framework handles the rest — just like it already does for text widgets, dials, bars, sliders, and buttons. Same pattern. New widget type. The architecture doesn’t change at all.

It transforms the bridge from a passive logger into a zero-configuration device aggregator. Let me trace what becomes possible.


The Bridge Discovers Devices Automatically

mDNS scan on startup:

Bridge starts → scans for _iot-framework._tcp.local
Finds:
  picow-iot-device.local  (Weather Station — weather_a)
  picow-iot-device.local  (Greenhouse Controller — greenhouse_01)
  picow-iot-device.local  (Garage Monitor — garage_sensor)

Each device is already advertising itself via mDNS. The bridge doesn’t need IP addresses, hostnames, or manual configuration.


The Bridge Interrogates Each Device

Bridge → GET /api/identity → {
    "project_name": "Weather Station",
    "device_id": "weather_a",
    "device_name": "Backyard Weather",
    "api_version": "1.0"
}

Now the bridge knows: - What kind of device this is (project_name) - Its unique ID (device_id) - Its human-readable name (device_name) - What API version it speaks

Bridge → GET /api/manifest → [
    {"id":"temp_a",       "name":"Temperature A",   "type":0, "unit":"°F",  ...},
    {"id":"humidity_a",   "name":"Humidity A",       "type":0, "unit":"%",   ...},
    {"id":"cpu_temp_f",   "name":"CPU Temperature",  "type":0, "unit":"°F",  ...},
    {"id":"free_ram",     "name":"Free RAM",         "type":0, "unit":"%",   ...},
    {"id":"heat_index",   "name":"Heat Index",       "type":0, "unit":"°F",  ...},
    {"id":"moisture_target","name":"Target Moisture", "type":2, "unit":"%",  ...},
    {"id":"water_now",    "name":"Manual Water",     "type":4, "unit":"",    ...},
]

Now the bridge knows every data point and control this device exposes — names, units, types, ranges, everything.


The Bridge Auto-Configures Its Capabilities

Without any user configuration, the bridge now knows:

Device Capability Details
weather_a Temperature sensor °F, updates every 6s
weather_a Humidity sensor %, updates every 7s
weather_a Heat index (derived) °F
weather_a Dew point (derived) °F
weather_a CPU temperature °F, updates every 8s
weather_a Free RAM monitor %, updates every 9s
weather_a Moisture target slider Control, range 20-80%
weather_a Manual water button Control, momentary
greenhouse_01 Temperature B °F, updates every 6s
greenhouse_01 Soil moisture %, updates every 5min
greenhouse_01 Irrigation valve Control, toggle
garage_sensor Door state Open/Closed
garage_sensor Motion detected Binary

The bridge can now:

1. Auto-Build Dashboards

Backyard Weather Dashboard:
  - Temperature A graph (last 24h)
  - Humidity A graph (last 24h)
  - Current conditions card
  - System health card

Greenhouse Dashboard:
  - Temperature B graph
  - Soil moisture trend
  - Irrigation controls
  - Valve state history

Whole-House Overview:
  - All temperatures on one graph
  - All humidity readings
  - System health across all devices

No dashboard YAML. No config files. The bridge generates dashboards from the manifests it discovered.

2. Auto-Start Logging

Discovered weather_a with items: [temp_a, humidity_a, cpu_temp_f, free_ram]
→ Create time-series tables for each
→ Start polling /api/data every 5s
→ Begin accumulating history immediately

3. Cross-Device Intelligence

weather_a.temp_a vs greenhouse_01.temp_b:
  "Greenhouse is consistently 8°F warmer than outside"

weather_a.humidity_a vs greenhouse_01.soil_moisture:
  "Soil moisture dropping despite high outdoor humidity — check irrigation"

4. Unified Control Interface

Bridge presents all controls from all devices:
  - weather_a: moisture_target slider
  - weather_a: water_now button
  - greenhouse_01: irrigation_valve toggle

User adjusts any slider → Bridge POSTs to /api/update on the right device

5. Alerting Without Configuration

Bridge knows:
  - weather_a.free_ram is a percentage
  - It updates every 9 seconds
  → Auto-alert if free_ram drops below 20% for more than 5 minutes

Bridge knows:
  - greenhouse_01.soil_moisture is a percentage, range 0-100
  → Auto-alert if soil moisture drops below moisture_target for more than 30 minutes

6. Device Topology Awareness

Three devices discovered on mDNS:
  - weather_a (Backyard Weather)
  - greenhouse_01 (Greenhouse Controller)
  - garage_sensor (Garage Monitor)

Bridge builds a network map:
  - All devices online
  - All speaking API v1.0
  - All pollable at 5s intervals
  - If one goes silent, alert

The Key Insight

The framework’s design — mDNS advertisement, /api/identity, /api/manifest, /api/data — makes each device fully self-describing.

The bridge doesn’t need: - A device registry - A sensor catalog - A unit conversion table - A control range database - A polling interval configuration

It asks the device. The device tells it everything.

New device joins the network: 1. mDNS broadcast 2. Bridge discovers it 3. Bridge queries identity → “I’m a Weather Station named Backyard” 4. Bridge queries manifest → “I have temp, humidity, heat index, dew point, moisture control” 5. Bridge starts logging, builds dashboards, enables alerting 6. Zero user interaction

This is plug-and-play at the network level. The framework didn’t just make the Pico W easier to program — it made the Pico W discoverable, interrogable, and automatically integratable into a larger system. The bridge isn’t just a logger anymore. It’s an orchestration platform that grows its capabilities every time a new device appears on the network.

No comments:

Post a Comment

The local IoT Logging Architecture Addition

J. Rogers, SE Ohio Pico W — Real-time data source only: - Sensors read → registry updated → FIFO sync → served as JSON - No ring buffers, ...