Mastodon Politics, Power, and Science: The local IoT Logging Architecture Addition

Saturday, May 30, 2026

The local IoT Logging Architecture Addition

J. Rogers, SE Ohio

Pico W — Real-time data source only: - Sensors read → registry updated → FIFO sync → served as JSON - No ring buffers, no SD cards, no flash wear, no graph rendering - Just the same /api/data?idx=0,3,5 endpoint, polled as always

Home Automation Bridge (desktop, Raspberry Pi, NAS) — All persistence and visualization: - Polls the Pico’s /api/manifest once → knows every item - Registers with api to be a graph provider - Polls /api/data every N seconds → timestamps and stores - Builds a time-series database (SQLite, InfluxDB, even flat files) - Renders graphs, dashboards, alerts - Can aggregate multiple Pico W devices - Still bridges to home automation software

What This Means For Your Framework

The Pico side changes very little. That’s the beauty — it already exposes everything the bridge needs:

Bridge                          Pico W
─────                           ──────
GET /api/manifest         →     Full item list with IDs, names, units, types
GET /api/data?idx=all     →     Current values
GET /api/identity         →     Device name, location, unique ID
need to add a register graph agent call

The bridge can discover devices via mDNS, pull their manifests, and start logging automatically. A new Pico W device joins the network, the bridge sees picow-iot-device.local, queries its identity and manifest, and starts polling.

The framework’s job was to make the device discoverable and interrogable. We already did that. The logging, graphing, alerting — that’s a separate system that treats the framework as a first-class data source.

And the bridge can be smarter than the Pico ever could be: - Downsampling old data (keep 5s resolution for 24h, 1min for a week, 1hr for a year) - Anomaly detection - Multi-device correlation (living room temp vs. outdoor temp) - Push notifications - Integration with Home Assistant, MQTT, etc.

The Pico doesn’t need to know any of that exists. It just needs to serve clean, structured data on demand. Which it already does.

// Registry — what data exists
SENSOR_AUTO("temp_a", "Temperature A", 6004, "°F", readAM2302a_temp);

// Layout — where to put it on the page
{"w_temp_a", "sensor_card", "Temperature A", "temp_a", W_TEXT, ""},

// Help — what the tooltip says
{"help_temp", "<b>Temperature A</b><br>AM2302 sensor on GPIO 2."},

What We Add

A graph table — defines what graphs exist and how they’re built:

const GraphNode graph_table[] = {
    // id              title            series[]           hours   interval
    {"temp_24h",       "Temperature A — 24 Hours",  {"temp_a"},          24,     300},
    {"temp_humidity",  "Temp & Humidity",           {"temp_a", "humidity_a"}, 12, 120},
    {"system_health",  "System Overview",           {"cpu_temp_f", "free_ram"}, 6, 60},
};

Where It Goes In The Layout

const LayoutNode layout_table[] = {
    // ... existing nodes ...
    
    // A new card with graphs
    {"history_card",   "main_page",    "History",           "",         W_CARD,     ""},
    {"graph_temp_24h", "history_card", "Temperature 24h",   "temp_24h", W_GRAPH,    ""},
    {"graph_sys",      "history_card", "System Trends",     "system_health", W_GRAPH, ""},
    
    // Or inline on the system page
    {"graph_cpu",      "status_card",  "CPU History",       "system_health", W_GRAPH, ""},
};

What The Framework Does With It

At page render time, it sees W_GRAPH, looks up the graph id in graph_table, and injects:

<div class="card">
  <h3>Temperature A — 24 Hours</h3>
  <canvas id="graph_temp_24h" 
          data-series="temp_a" 
          data-hours="24" 
          data-interval="300"
          data-title="Temperature A">
  </canvas>
</div>

And the JavaScript that’s already injected into the page gets a new section:

// Graph initialization — runs once at page load
document.querySelectorAll("canvas[id^=graph_]").forEach(canvas => {
    const series = canvas.dataset.series.split(",");
    const hours = parseInt(canvas.dataset.hours);
    // Build a Chart.js / lightweight SVG graph
    // Initial data fetch from bridge API
    // Set interval to refresh
});

The Bridge API

The graphs don’t poll /api/data — they hit the bridge:

fetch(`http://bridge.local:8080/series?device=weather_a&series=temp_a&hours=24`)

The bridge already has the time-series database. It returns JSON the graph renderer can consume directly.

The Developer’s Job

// 1. Define the data (already done)
SENSOR_AUTO("temp_a", "Temperature A", 6004, "°F", readAM2302a_temp);

// 2. Define the graph
{"temp_24h", "Temperature A — 24 Hours", {"temp_a"}, 24, 300},

// 3. Place it on a page
{"graph_temp_24h", "history_card", "Temperature 24h", "temp_24h", W_GRAPH, ""},

They never write a SQL query, a Chart.js config, an API endpoint, or a canvas element. They declare what data they want graphed, over what time window, at what resolution, and where on the page it goes if there is a graph provider registered.

The framework handles the rest — just like it already does for text widgets, dials, bars, sliders, and buttons. Same pattern. New widget type. The architecture doesn’t change at all.

It transforms the bridge from a passive logger into a zero-configuration device aggregator. Let me trace what becomes possible.


The Bridge Discovers Devices Automatically

mDNS scan on startup:

Bridge starts → scans for _iot-framework._tcp.local
Finds:
  picow-iot-device.local  (Weather Station — weather_a)
  picow-iot-device.local  (Greenhouse Controller — greenhouse_01)
  picow-iot-device.local  (Garage Monitor — garage_sensor)

Each device is already advertising itself via mDNS. The bridge doesn’t need IP addresses, hostnames, or manual configuration.


The Bridge Interrogates Each Device

Bridge → GET /api/identity → {
    "project_name": "Weather Station",
    "device_id": "weather_a",
    "device_name": "Backyard Weather",
    "api_version": "1.0"
}

Now the bridge knows: - What kind of device this is (project_name) - Its unique ID (device_id) - Its human-readable name (device_name) - What API version it speaks

Bridge → GET /api/manifest → [
    {"id":"temp_a",       "name":"Temperature A",   "type":0, "unit":"°F",  ...},
    {"id":"humidity_a",   "name":"Humidity A",       "type":0, "unit":"%",   ...},
    {"id":"cpu_temp_f",   "name":"CPU Temperature",  "type":0, "unit":"°F",  ...},
    {"id":"free_ram",     "name":"Free RAM",         "type":0, "unit":"%",   ...},
    {"id":"heat_index",   "name":"Heat Index",       "type":0, "unit":"°F",  ...},
    {"id":"moisture_target","name":"Target Moisture", "type":2, "unit":"%",  ...},
    {"id":"water_now",    "name":"Manual Water",     "type":4, "unit":"",    ...},
]

Now the bridge knows every data point and control this device exposes — names, units, types, ranges, everything.


The Bridge Auto-Configures Its Capabilities

Without any user configuration, the bridge now knows:

Device Capability Details
weather_a Temperature sensor °F, updates every 6s
weather_a Humidity sensor %, updates every 7s
weather_a Heat index (derived) °F
weather_a Dew point (derived) °F
weather_a CPU temperature °F, updates every 8s
weather_a Free RAM monitor %, updates every 9s
weather_a Moisture target slider Control, range 20-80%
weather_a Manual water button Control, momentary
greenhouse_01 Temperature B °F, updates every 6s
greenhouse_01 Soil moisture %, updates every 5min
greenhouse_01 Irrigation valve Control, toggle
garage_sensor Door state Open/Closed
garage_sensor Motion detected Binary

The bridge can now:

1. Auto-Build Dashboards

Backyard Weather Dashboard:
  - Temperature A graph (last 24h)
  - Humidity A graph (last 24h)
  - Current conditions card
  - System health card

Greenhouse Dashboard:
  - Temperature B graph
  - Soil moisture trend
  - Irrigation controls
  - Valve state history

Whole-House Overview:
  - All temperatures on one graph
  - All humidity readings
  - System health across all devices

No dashboard YAML. No config files. The bridge generates dashboards from the manifests it discovered.

2. Auto-Start Logging

Discovered weather_a with items: [temp_a, humidity_a, cpu_temp_f, free_ram]
→ Create time-series tables for each
→ Start polling /api/data every 5s
→ Begin accumulating history immediately

3. Cross-Device Intelligence

weather_a.temp_a vs greenhouse_01.temp_b:
  "Greenhouse is consistently 8°F warmer than outside"

weather_a.humidity_a vs greenhouse_01.soil_moisture:
  "Soil moisture dropping despite high outdoor humidity — check irrigation"

4. Unified Control Interface

Bridge presents all controls from all devices:
  - weather_a: moisture_target slider
  - weather_a: water_now button
  - greenhouse_01: irrigation_valve toggle

User adjusts any slider → Bridge POSTs to /api/update on the right device

5. Alerting Without Configuration

Bridge knows:
  - weather_a.free_ram is a percentage
  - It updates every 9 seconds
  → Auto-alert if free_ram drops below 20% for more than 5 minutes

Bridge knows:
  - greenhouse_01.soil_moisture is a percentage, range 0-100
  → Auto-alert if soil moisture drops below moisture_target for more than 30 minutes

6. Device Topology Awareness

Three devices discovered on mDNS:
  - weather_a (Backyard Weather)
  - greenhouse_01 (Greenhouse Controller)
  - garage_sensor (Garage Monitor)

Bridge builds a network map:
  - All devices online
  - All speaking API v1.0
  - All pollable at 5s intervals
  - If one goes silent, alert

The Key Insight

The framework’s design — mDNS advertisement, /api/identity, /api/manifest, /api/data — makes each device fully self-describing.

The bridge doesn’t need: - A device registry - A sensor catalog - A unit conversion table - A control range database - A polling interval configuration

It asks the device. The device tells it everything.

New device joins the network: 1. mDNS broadcast 2. Bridge discovers it 3. Bridge queries identity → “I’m a Weather Station named Backyard” 4. Bridge queries manifest → “I have temp, humidity, heat index, dew point, moisture control” 5. Bridge starts logging, builds dashboards, enables alerting 6. Zero user interaction

This is plug-and-play at the network level. The framework didn’t just make the Pico W easier to program — it made the Pico W discoverable, interrogable, and automatically integratable into a larger system. The bridge isn’t just a logger anymore. It’s an orchestration platform that grows its capabilities every time a new device appears on the network.

No comments:

Post a Comment

The universe held up a mirror and we did not recognize our own arbitrary scales.

  Because the reflection was so perfect we mistook our own face for the face of God. We built the axes. We invented length, duration, mass —...