The Second Executable in the Data Object Framework

J. Rogers, SE Ohio

Building a unit-test host on top of the framew
ork library, and what five simultaneous builds of it found

---

The framework has always carried its own tests. Every core module has a self-test compiled into it — DataTest() in data.c, NodeTest() in node.c, SchedTest() in sched.c, BuffTest() in dyn/, NameSpaceTest() in namespace.c — and ./framework -t runs them in sequence. They ship inside libframework.so alongside the code they exercise, which is exactly where they belong.

The problem isn’t the tests. It’s who runs them. main.c runs them, and main.c is the application. A program grading its own homework will always hand back an A.

unit_test.c

The fix started as a copy: cp main.c unit_test.c, delete the default app, then bring each module’s test across under a renamed symbol — UT_DataTest, UT_NodeTest, UT_BuffTest, and so on. Both sets now exist. ./unit_test -t calls the library’s copies through the same path framework -t uses; ./unit_test with no arguments calls the UT_ copies compiled into the executable. If those two ever disagree, one of them has drifted, and you find out immediately instead of six months later.

The build was restructured to match: sources moved to src/, every artifact lands in build/, and unit_test is built whenever framework is — same rules, same directory, no special case. A test binary that is a chore to build is a test binary that doesn’t get built.

Three of the copied tests are currently #if 0’d with the reason written where the code was. They reach into module-private state — a TaskPtr here, a static class list there — that a second host legitimately cannot see. That’s not a defeat; it’s the first thing the exercise told us.

What a second host sees that the first one can’t

Here is the part I didn’t expect to be the main event.

The library is supposed to be an embeddable core. The whole hosting contract is five calls: set the registry list, set the task list, install objects, then pump TimeUpdate() and ExecTasks() from whatever event loop you already have. Anything that can dlopen a shared object and call five functions gets the entire object system.

Except it didn’t, quite. Writing a second host that wanted the fabric and none of the furniture surfaced a list of things that had quietly become requirements of the library rather than of the app: building the palette, building the chrome, a known-classes list, the object scan path, the settings home, the reserved view names. GUI concerns, sitting in the core, invisible for as long as the only program linking the core was the GUI’s own host.

You cannot find that by reading main.c. main.c is the app, so everything in it looks like it belongs. You find it by writing a second program that links the same library and wants a different subset — the moment it needs a palette it will never draw, the boundary you thought you had is drawn in the wrong place. The unit-test host is a boundary detector that happens to also run tests.

That’s the microkernel argument made empirical instead of aspirational.

Five builds, one wall clock

Once tests run from a host that isn’t the app, the same source can be compiled several ways and tested identically. The harness now builds five variants — debug, release, ASAN, UBSAN, gcov — each into its own directory, each with its own ports above the production instance, each with its own hardlinked copy of the objects and the web client, each running the full suite list against its own engine.

They run in parallel. There are sixteen cores; five builds sitting on five of them cost the wall-clock time of the slowest one. Coverage instrumentation and sanitizer instrumentation are effectively free, because the thing you were waiting for anyway is the debug run.

Each variant writes its logs under its own log/, keeps its saved flows for forensics, and the only thing at the base of the tree is one report:

20:51:46  [release] framework quit when asked - exit-time leak check ran
20:51:46  [ubsan] framework quit when asked - exit-time leak check ran
20:51:47  all variants finished, writing the report
suite               debug    release  asan     ubsan    gcov     
---------------------------- -------- -------- -------- -------- 
unit_test           0        0        0        0        0        
rawtest             0        0        0        0        0        
connectiontest      0        0        0        0        0        
leaktest            0        0        0        0        0        
flowtest            2        2        2        2        2        
viewclonetest       0        0        0        0        0        
jstest              0        0        0        0        0        
scriptboxtest       1        1        CRASH    1        1        
scriptedwidgettest  0        0        CRASH    0        0        
widgettest          0        0        CRASH    0        0        
tcpporttest         6        10       CRASH    8        8        
guitest             6        6        CRASH    6        6        
---------------------------- -------- -------- -------- -------- 
failures            15       19       2        17       17       
crashed             0        0        5        0        0        
leaked              0        0        0        0        0        
sanitizer           0        0        1        0        0        

debug: build return code=0  web=8084 raw=8092 cdp=9224  logs=testharness/tests/20260805_203113/debug/log/
release: build return code=0  web=8085 raw=8093 cdp=9225  logs=testharness/tests/20260805_203113/release/log/
asan: build return code=0  web=8086 raw=8094 cdp=9226  logs=testharness/tests/20260805_203113/asan/log/
ubsan: build return code=0  web=8087 raw=8095 cdp=9227  logs=testharness/tests/20260805_203113/ubsan/log/
gcov: build return code=0  web=8088 raw=8096 cdp=9228  logs=testharness/tests/20260805_203113/gcov/log/
coverage: testharness/tests/20260805_203113/gcov/coverage.txt
logs: testharness/tests/20260805_203113/log/ and testharness/tests/20260805_203113/<variant>/log/   summary: testharness/tests/20260805_203113/report.txt
20:51:48  70 failures, 5 crashes and 0 leaks across all variants

Builds across the top, suites down the side. Getting the vocabulary right mattered more than it sounds: a dead engine and a single failed assertion both used to print 1, so one death read as one failure and eleven cascading connection errors read as eleven more. Now a suite that never measured anything says CRASH, a suite that ran but leaked says LEAK (or 2+LEAK, keeping its own count), and a crash ends that variant on the spot instead of running ten more suites against a corpse. Zero everywhere says success. The return code is failures plus crashes plus leaks.

The bugs

An overlapping memcpy, running for years. ASAN stopped on buffResize, on a line whose own comment predicted it would be fine:

/* we shouldn't need to use move mem here, there shouldn't be a memory overlap */
memcpy((char *)obj->buffer, (char *)obj->buffer+obj->tail, obj->head-obj->tail);

The ranges overlap whenever the live span is longer than the distance it slides down — in the observed case, dest and src 0x800 apart with a 0x9ad0 span. Undefined behaviour that happens to work, because glibc copies forward and dest < src. Two different callers reach it: the line reader, and the TCP drain. It is now a memmove.

Worth noting that the near-identical slide inside buffAdd is not the bug — it’s guarded by tail > size/2, so it genuinely cannot overlap. Two copies of almost the same code, one correct by accident and one incorrect by accident, is its own lesson.

A test that leaked 585 bytes. With the overlap fixed, the ASAN build stopped aborting, and for the first time LeakSanitizer’s exit-time check actually ran. It reported 14 allocations: five DataObjs created by the data test and never freed, plus the nine strings they allocated while walking the type-conversion matrix. The test, not the library — which is its own kind of finding, since it means every framework -t has leaked that much forever, and nobody could see it.

A use-after-free in a widget swap. The reward for keeping the sanitizer alive past its first find. Changing a scripted widget’s language deletes the old host and creates a new one, and the swap holds a node belonging to the deleted instance across its own DeleteInstance — freed at line 152, read at line 180 through Connect → ResolvePort → GetPropNode. Every build has this bug; only the instrumented one says so, because freed memory usually still reads plausibly. The fix is architectural rather than a patch: the widget should own its engine as a private opaque handle the way the network widgets do, so there is no internal node for it to hold in the first place.

Getting the leak check to run at all

LeakSanitizer only reports when a process exits normally, and the harness was killing every engine with a signal. The framework can be asked to leave — and it turned out no new mechanism was needed to ask. The main loop runs while Main’s State property is non-zero, /Main is an ordinary registered path, and set-property is an ordinary command. So:

{"cmd":"set-property","instance":"/Main","prop":"State","value":"0"}

The engine shuts down through the same cleanup path it uses when the flow drains, in about a second, and the leak check runs. A “quit verb” that already existed, spelled out of parts that were already there — which is the point of the design, and a nice reminder to look before adding.

What it cost, and what it’s worth

Most of a day was spent on the un-glamorous half: making the harness lie less. Hard linking the web client into each variant directory, because the HTTP object serves relative to the working directory and twenty GUI tests were cascading off a blank page. Separating “crashed” from “failed”. Not claiming a leak check ran on a process that died an hour earlier. Refusing to interpret a missing pid without asking what it means first.

That work isn’t decoration. A grid that reads 1 in twelve cells sends you looking for twelve bugs. The same grid reading CRASH in twelve cells sends you to one log, which is where the one bug is.

The core tests still don’t cover the objects — fifty modules with no self-tests reachable from the harness, though the registry already has the mechanism to register them the same way classes register. And a matrix makes flakiness visible without making it go away: one suite reported 7, 6, 1, 10 and 9 failures across five builds of identical source, which is a timing bug wearing five costumes.

But the instrument exists now, and it’s pointed at the right thing. The tests aren’t run by the program under test anymore. They’re run by a second program that wants the same library for different reasons — and that difference is what makes them tests.

Test Harness Plan for libframework.so

 J. Rogers, SE Ohio

1. Overview

We will extract all unit tests from the core source files (data.c, node.c, namespace.c, sched.c, object.c, widget.c, etc.) into a separate test harness that links against libframework.so. The harness will run every test against multiple builds of the library, each compiled with different instrumentation (debug, ASAN, UBSAN, coverage, release), and report only on failures. Each core module will have its own dedicated test function (e.g., test_data(), test_node(), …) that verifies correct behavior, checks memory integrity, and asserts expected outcomes.

The goal is to achieve 100% branch coverage and zero undefined behavior across all build variants, with the test suite serving as both regression protection and living documentation.

---

2. Architecture

2.1 Core Library (libframework.so)

• Production build – -O2 -DNDEBUG, stripped, no test code.

• Debug build – -O0 -g, full symbols, asserts enabled.

• ASAN build – -O1 -fsanitize=address for memory error detection.

• UBSAN build – -O1 -fsanitize=undefined for UB detection.

• GCOV build – -O0 -fprofile-arcs -ftest-coverage for coverage analysis.

All builds share the same source code; no #ifdef TESTBUILD remains in production files. All test logic lives outside the library.

2.2 Test Harness (testharness.c)

A standalone executable that:

• Links against libframework.so (or a variant via LD_LIBRARY_PATH).

• Calls each module’s test function in sequence.

• Each test function exercises the public API (DataNew, NSCreate, CreateObject, SetProp, Connect, DeliverMsg, etc.).

• Each test function returns 0 on success, 1 on failure.

• The harness aggregates results and exits with 0 only if all tests pass.

2.3 Per‑Module Test Functions

Each core module exposes a test function in its public header (or via a separate test_*.h):

c

// data.h
int test_data(void);

// node.h
int test_node(void);

// namespace.h
int test_namespace(void);

// sched.h
int test_sched(void);

// object.h
int test_object(void);

// widget.h
int test_widget(void);

These functions:

• Perform systematic checks on all API entry points.

• Use assert() or custom check macros that print specific error messages.

• Exercise edge cases (null pointers, out‑of‑range values, memory exhaustion simulation).

• (For memory testing) can be run under Valgrind or ASAN; the harness itself will be compiled with the same sanitizer flags.

---

3. Build System (Makefile)

3.1 Build Targets

make

# Shared library variants
libframework.so:         ... $(CC) -shared -O2 ...
libframework-debug.so:   ... $(CC) -shared -O0 -g ...
libframework-asan.so:    ... $(CC) -shared -O1 -fsanitize=address ...
libframework-ubsan.so:   ... $(CC) -shared -O1 -fsanitize=undefined ...
libframework-gcov.so:    ... $(CC) -shared -O0 -fprofile-arcs -ftest-coverage ...

# Test harness (links against the default production library)
testharness: testharness.c
	$(CC) -o $@ testharness.c -L. -lframework

# Variant‑specific harnesses (optional, or use LD_LIBRARY_PATH)
testharness-debug: testharness.c
	$(CC) -o $@ testharness.c -L. -lframework-debug

# ... similarly for asan, ubsan, gcov

Alternatively, use one harness binary and set LD_LIBRARY_PATH to point to the desired .so at runtime.

3.2 Test Execution

A top‑level make test target runs all variants:

make

test: all
	@echo "Running tests against all library variants..."
	@$(MAKE) -s run-tests VARIANT=release
	@$(MAKE) -s run-tests VARIANT=debug
	@$(MAKE) -s run-tests VARIANT=asan
	@$(MAKE) -s run-tests VARIANT=ubsan
	@$(MAKE) -s run-tests VARIANT=gcov
	@echo "All variants passed."

run-tests:
	@LD_LIBRARY_PATH=. ./testharness-$(VARIANT) --quiet

The --quiet flag (or -q) suppresses all output except errors. The harness will print a brief summary (e.g., PASS / FAIL) per module, but only details on failure.

---

4. Test Harness Implementation (testharness.c)

4.1 Structure

c

#include <stdio.h>
#include <stdlib.h>
#include "data.h"
#include "node.h"
#include "namespace.h"
#include "sched.h"
#include "object.h"
#include "widget.h"

/* Each test function returns 0 on success, 1 on failure */
extern int test_data(void);
extern int test_node(void);
extern int test_namespace(void);
extern int test_sched(void);
extern int test_object(void);
extern int test_widget(void);

typedef struct { const char *name; int (*fn)(void); } TestEntry;

static TestEntry all_tests[] = {
    {"data",       test_data},
    {"node",       test_node},
    {"namespace",  test_namespace},
    {"sched",      test_sched},
    {"object",     test_object},
    {"widget",     test_widget},
    {NULL, NULL}
};

int main(int argc, char **argv) {
    int quiet = 0;
    if (argc > 1 && strcmp(argv[1], "--quiet") == 0)
        quiet = 1;

    int failures = 0;
    for (TestEntry *t = all_tests; t->name; t++) {
        int ret = t->fn();
        if (ret != 0) {
            if (!quiet)
                fprintf(stderr, "FAIL: %s\n", t->name);
            failures++;
        } else {
            if (!quiet)
                printf("PASS: %s\n", t->name);
        }
    }
    if (failures == 0) {
        if (!quiet) printf("All tests passed.\n");
        return 0;
    } else {
        if (!quiet) fprintf(stderr, "Total failures: %d\n", failures);
        return 1;
    }
}

4.2 Per‑Module Test Skeleton (example for data.c)

c

// test_data.c (included in testharness, or #included from testharness.c)
#include "data.h"
#include <assert.h>
#include <string.h>

int test_data(void) {
    DataNode *d = DataNew();
    DataSetInt(d, 42);
    // Check basic retrieval
    if (DataGetInt(d) != 42) return 1;
    // Check lazy conversion to string
    const char *s = DataGetString(d);
    if (strcmp(s, "42") != 0) return 1;
    // Check caching – second call should return same pointer
    const char *s2 = DataGetString(d);
    if (s2 != s) return 1;
    // Check float conversion
    double f = DataGetFloat(d);
    if (f != 42.0) return 1;
    // Clean up
    DataRelease(d);
    return 0;
}

Each test function must be self‑contained, creating its own objects and cleaning up after itself. The harness can optionally run each test under a memory checker; ASAN will catch leaks automatically.

---

5. Brutality and Output Control

• Only output on error: The harness uses --quiet to suppress pass messages. CI logs only failures, keeping noise low.

• Memory checks: ASAN/UBSAN builds will abort on error; the harness exits with non‑zero, making failure obvious.

• Coverage: After running the GCOV variant, we run gcov on all source files to produce a coverage report. The harness itself does not output coverage; it’s a separate step.

• Repeated runs: The Makefile can loop the tests multiple times (e.g., for i in $(seq 1 1000); do ...) to catch intermittent issues.

---

6. Integration with CI

• Each commit triggers a build of all variants.

• The test harness runs against each variant; if any fails, the CI build fails.

• The GCOV build is used to generate a coverage report and optionally enforce a threshold (e.g., 100% branch coverage for the core).

• The entire suite (including all variants) completes in under a minute on a modern machine.

---

7. Benefits

• No test code in production – libframework.so is pure, minimal, and fast.

• Cross‑variant validation – catches compiler‑dependent bugs, sanitizer hits, and coverage gaps.

• Silent on success – developers only see failures, encouraging frequent test runs.

• Modular – each module’s tests live separately, making it easy to add new tests.

• Memory‑safe – ASAN and Valgrind run as part of the standard test regime.

---

8. Next Steps

1. Remove all #ifdef TESTBUILD blocks from core .c files.

2. Extract each module’s test code into standalone test functions (e.g., test_data()).

3. Create testharness.c that calls all test functions.

4. Update the Makefile to build all variants and the harness.

5. Integrate the new test regime into the daily workflow and CI.

With this plan, the 200 KB core will be mathematically verified across all builds, and the test harness will serve as the ultimate guardian of correctness