Summary
LeClaire, Iowa
I build the harness around the model: what it may touch, what it remembers, what it retrieves, and what it costs. At Clarus I work on the AI integration surface of four live B2B products in one monorepo of 10 backend services and 1,019 HTTP endpoints, routing GPT-4o, Anthropic Claude, and a self-hosted olmOCR model served on vLLM behind per-tenant key routing, token quotas, and Presidio-based PII anonymization. Outside the platform I have published two architecture specifications for long-running agent systems: RAG-OS, the compiled architecture of a system I run on my own hardware, and org-memory-os, which takes that design apart to name which single-operator invariants stop holding once an organization is using it. Both are organized around one question, which is where authority lives when a model is in the loop and the model is the least reliable component in it. The public code carries 1,177 automated tests green in CI across MIT-licensed repositories.SoughtThe AI platform itself. The harness, the retrieval, the memory, the controls, and the budget, owned end to end.
Selected systems
Four entries. Every number below is measured, not estimated.
01Production2026
Clarus platform: CLM, CFO, HR, Nexus
Four live B2B SaaS products in one production monorepo. The model layer is a governed surface with routing, quotas, and anonymization in front of it.
- The four productsClarus CLM, contract intelligence that thinks at the clause level. Clarus CFO, the autonomous CFO for software. Clarus HR, the HR engine that runs itself. Clarus Nexus, the organizational reality engine.Three model paths behind one integration layer: GPT-4o, Anthropic Claude, and a self-hosted olmOCR model served on vLLM for document extraction and classification. Per-tenant API key routing, per-tenant token quotas, and Presidio-based PII anonymization apply to all of it.
- Family capabilitiesClauses as versioned searchable entities; amendments that create versions instead of overwriting; search that comprehends meaning; autonomous spend and contract operation; intercepting auto-renewals; three-way matching across purchase orders, invoices and contracts; policy rollouts and acknowledgment campaigns; payroll exceptions; contractor classification risk; multi-state compliance; continuous modeling of company operations across contracts, spend, approvals, vendors, obligations and workflows; detecting contradictions between stated intent and actual operation; preserving decision reasoning; simulating changes before they are made.Contract intelligence parses raw OOXML and PDF into a ProseMirror representation, types every clause against a 42-type taxonomy across 11 categories, scores per-clause risk, and overlays amendments as new versions instead of overwriting the original. Retrieval is hybrid: semantic search over pgvector embeddings fused with keyword search. Editing is real-time and collaborative over Yjs.
- Per-request tenant isolation is enforced in the database by PostgreSQL row-level security across all 263 tables, under 203 Alembic migrations and 10 backend services.
- ClarusStream, an in-house Kafka-wire-compatible message broker written in Go, with consumer groups, SASL, ACLs, and S3 segment tiering, published in vendor-neutral form as kafka-wire. Producers publish through a transactional outbox drained by a relay service with retries and exponential backoff. The nine OAuth platform integrations and Stripe billing are verified by an adversarial chaos-testing harness running nightly and weekly burn-ins in CI.
- Stack
- Python, FastAPI, async SQLAlchemy 2.0, Pydantic v2, Alembic, Go, PostgreSQL 16, row-level security, pgvector, Redis, AWS S3, Next.js App Router, React 19, TipTap/ProseMirror, Yjs, Zustand, vLLM, Presidio, Stripe, Docker, Helm
clarusclm.comclarusclm.com/products
01aInfrastructure2026MIT
kafka-wire
The broker above, rebuilt as a public project: a Kafka wire protocol implementation in one Go binary, with every assumption about my own stack replaced by a choice.
- Why rebuild rather than open the originalThe internal version was correct for exactly one deployment: one object store, one hosting platform, one consumer per group, one client library. None of those are safe assumptions for a stranger, and each one was hiding a defect.Generalizing it surfaced three protocol defects that a single-client deployment could never have exercised. The request header's tagged-fields section was passed through to the body decoder, so every flexible-version request from a modern client was misparsed and the connection dropped. A Fetch response field left at Go's zero value told consumers to read from broker 0, which does not exist, so they silently re-fetched the same offset forever while the broker logged that it was serving the records correctly. Unimplemented APIs were answered with the wrong response type, desynchronizing the connection.
- The one that matteredThe fix was a real join barrier: hold every JoinGroup open until the generation's membership settles, then answer all members at once from a single decided generation, with the leader alone receiving the member list.The consumer-group coordinator settled each rebalance on the first member to arrive. Every member of a group was therefore assigned every partition and received every record twice. Nothing errored and nothing hung, which is why it survived. The regression test asserts an exact record count and zero duplicates, because a test written as "at least twenty" passes against the broken behavior.
- Not assuming AWSThe S3 driver deliberately avoids the AWS SDK, whose default integrity checksums switch uploads to a streaming chunked trailer that Hetzner, OVH, Garage and older MinIO, R2 and B2 releases reject outright.Cold storage sits behind one interface with three implementations, none, a filesystem, and any S3-compatible store, and all of them are held to a single shared conformance suite that runs against a live server in CI. That suite immediately earned its keep by catching a store that returns part checksums quoted from one call and unquoted from another, which would have silently degraded every resumed upload into a full re-transfer.
- Configuration is one surface with mechanically derived environment overrides, file indirection for secrets, and validation that refuses several configurations that would lose data rather than accepting them: an upload part count above the limit every object store enforces, an archive retention shorter than the archive age, half-configured TLS that would silently serve plaintext, and a network-facing listener with authentication disabled.
- Stack
- Go, Kafka wire protocol, segmented commit log, SASL/SCRAM, TLS, ACLs, Prometheus, S3-compatible object storage, Docker, Kubernetes, systemd, Nomad, MIT licensed
github.com/csnyder256/kafka-wire
02Two specifications2026
Agent harness architecture
Two specifications for long-running agent systems, one for a single operator and one for an organization. The second was written by taking the first apart and asking which of its invariants stop being true once there are many people, many permissions, and an erasure request.
Models change quarterly. The harness around them is the durable asset, which is the entire argument for writing these down.
02aSpecification2026MIT
RAG-OS
A vendor-neutral blueprint plus a runnable stdlib-only starter for a self-hosted, always-on personal AI operating system. It is the compiled architecture of a system I actually run.
- Compiled knowledgeA librarian-maintained Markdown wiki, read index-first. The hybrid index (BM25 keyword plus vector, fused with reciprocal rank fusion) is treated strictly as a rebuildable cache over the Markdown, never as a source of fact. Embeddings run locally on CPU through ONNX, so retrieval costs nothing.A zero-context supervisor kernel: one always-on process that holds no model context and is the only writer of a SQLite database. All intelligence runs in short-lived spawned sessions, so a crash, a restart, or a model downgrade loses no durable state. Above it, an orchestrator and worker split: one durable conversational session decides and delegates, and ephemeral headless workers do the real repository edits and never talk to the human.
- Memory disciplineSuperseded facts are invalidated into a history section excluded from retrieval instead of being deleted, so the knowledge base does not rot and old reasoning stays reconstructable.Four persistence layers ordered by authority: a transactional operational database, git-tracked Markdown as compiled knowledge, mission ledgers carrying proofs, and disposable model transcripts. The transcript is the least authoritative artifact in the system, which is what makes context compaction safe to do at all.
- Disk verifyA deterministic disk-verify stamp fingerprints a repository before and after a worker runs, so a model that fabricates an edit that never landed is caught by the fingerprint instead of believed.Security is enforced in code instead of in prompts, re-checked on every single tool call, failing closed, and audited. Path fences normalize against traversal, symlinks, directory junctions, home aliases, UNC and extended-length prefixes, and fail closed on any path that will not resolve. Dangerous actions sit behind an out-of-band single-use approval code with a short expiry.
- Nightly evalsA nightly deterministic zero-token evaluation suite runs the whole thing. Its most important check is a safety canary re-proving that the protected paths still deny.Cost and vendor independence are policy, not habit. Code requests capabilities such as classify, workhorse, or deep reasoning; model names appear in exactly one config file, enforced by a check that fails if a model name appears anywhere else, so switching vendors is a config edit. A budget governor meters spend in plain code before a session spawns: degrade to cheaper models, then park background work, then park everything.
Persistence layers
Authority increases downward.
Model transcriptsDisposable
Mission ledgersProofs
Git-tracked MarkdownCompiled knowledge
Operational databaseTransactional
Writers to the database1
- Stack
- Python (standard library only in the starter), SQLite, ONNX runtime, BM25, vector search, reciprocal rank fusion, git-tracked Markdown, MIT licensed
github.com/csnyder256/RAG-OS
02bSpecification2026MIT
org-memory-os
One shared, permission-aware, auditable AI memory that any number of employees use through their own agents, under one compaction and degradation-avoidance discipline. Published 2026.
- The 70 forksEach fork carries the options considered, a recommended default, and the reasoning, including the forks where carrying the single-user rule forward was the wrong answer.The organizing idea is naming exactly which single-user invariant breaks at organizational scale and what replaces it. Single-writer state becomes one writer per aggregate with a fencing token. Identity as a boolean becomes an identity graph with fresh permission checks on the candidate set. One conversation window becomes windows scoped per surface and per thread. Invalidate-never-delete stays right for institutional knowledge and becomes wrong for personal data the moment erasure applies.
- The 16 milestonesEvery milestone ends in a definition of done proven by a real command and its real output, so the build order is falsifiable rather than aspirational.The data model is authority-ordered. An append-only bi-temporal claim ledger is truth, git-tracked Markdown is a projection of it, and the hybrid retrieval index is an explicitly rebuildable cache. Nothing downstream of the ledger is ever allowed to be the source of a fact.
- ProvenanceWritten out of a large adversarial research process. Every figure in it was independently fact-checked before publication. A Principal Cloud Architect at AWS starred the platform write-up that led to this being written.Retrieval is permission-aware at the candidate set, filtered as the asking human through relationship-based access control with forced-fresh consistency. The generated answer is never filtered after the fact, and no shared service account holds the union of everyone's permissions. A cross-tenant isolation canary is a hard release gate: nightly it plants a unique marker in each team's memory and attempts to read another team's marker back through the real retrieval path, failing the build if that ever succeeds.
- The coordination layer owns the conversation window in its own database and treats the model transcript as disposable, so each turn is reassembled from recent exchanges, fresh permission-filtered retrieval, and externalized knowledge. Prompt zoning keeps a stable cached prefix and confines volatile retrieved content to an uncached tail. Governance sits in the schema: four retention classes, per-subject crypto-shredding for personal data, a legal-hold table consulted before any deletion, and an immutable audit tier with citation-level logging, so the blast radius of a bad fact can be reconstructed.
- Stack
- Architecture specification: bi-temporal claim ledger, relationship-based access control, hybrid retrieval with reciprocal rank fusion, prompt zoning, crypto-shredding, immutable audit tier, MIT licensed
github.com/csnyder256/org-memory-os
02cTool2026MIT
harness-tuner
The two specifications above argue that the harness is the durable asset. This measures one. It reads any harness through an adapter, reports where the scaffolding wastes the model, proposes a change, and then proves whether the change helped. Published 2026.
- Why zero published measurementsA leaderboard between products goes stale within weeks, invites methodology fights, and is almost always run by someone with a stake. Every number the tool produces belongs to whoever ran it.The rule the whole design rests on is that a value the adapter could not observe is null, and null reports as unavailable with a reason rather than as zero. Without that, a harness whose adapter cannot see prompt caching looks identical to one that genuinely caches nothing, and the tool has invented a finding out of a gap in its own instrumentation. Two conformance fixtures exist only to pin the distinction: one where an adapter that sees nothing but tool names must report nineteen of twenty-four metrics unavailable without producing a single zero, and one where an explicitly reported zero must come out measured.
- The check that found itEvery setting is read through a dotted path, so a consumer necessarily contains the setting's literal name and the check is mechanical rather than a review.A check that every configuration setting is actually read by something found, on its first run, that twelve of twenty-nine were accepted, type-checked, validated against their legal values, documented, written into the run artifacts, and read by absolutely nothing. The cause was that the function building the artifact stamp lived inside the configuration module, so every setting looked consumed because the module echoed it back to itself. The fix was to wire all twelve rather than to loosen the check, and a test now proves the check can still fail.
- Measured, not assertedOver a thousand simulated null sequences with the threshold checked at every prefix, the false positive rate came out at three percent against a nominal five. A real effect was detected in 198 of 200 runs at a median of 18 trials.Whether a change helped is decided by an anytime-valid e-process rather than a p-value, because trials cost real money and the person running it is going to watch the number regardless. Peeking after every trial is safe by construction instead of by discipline, and the run stops as soon as the evidence is conclusive. It reports an e-value, never a p-value, and returns one of three answers: confirmed, refuted, or unproven, where unproven means add more trials rather than the change failed.
- Detection is arithmetic rather than judgement. Every finding is produced by counting things in traces and carries the exact steps it came from, and the diagnostician model is explicitly instructed not to add findings of its own, because a model asked to find problems in a trace will find them in any trace and none of them will come with a step number. Proposals carry a bound computed from the observed run, at most nine of these reads were avoidable because each had already been performed earlier in the same task, rather than a forecast nobody can check. The tool never edits the harness it measures.
- Stack
- Python standard library only, enforced by an import check in CI across three operating systems and three versions, plus a container image, declarative TOML evaluation packs, anytime-valid sequential testing, MIT licensed
github.com/csnyder256/harness-tuner
The eval that matters most
- Metric
- A protected path denies, and the denial is readable back out of the audit table.
- Test set
- The real tool path rather than a mock, exercised nightly by a deterministic zero-token suite, so it costs nothing to run and cannot drift when a model changes.
- Why it exists
- The agent SDK's own permission callback was silently shadowed, fired zero times, and let a fenced read through. Enforcement moved to a hook that fires under every permission mode, and a live canary now re-proves the denial every night.
Full evidence ledger, margin notes and the harness breakdown at https://csnyder256.github.io/
- Agent loop
- Orchestrator and worker split. One durable conversational session decides and delegates; ephemeral headless workers do the repository edits and never talk to the human.
- Tool interface
- Permission hooks that fire on every tool call, path fences normalized against traversal, symlinks, junctions, home aliases and UNC prefixes, out-of-band single-use approval codes on dangerous actions, and a disk-verify fingerprint on whatever a worker claims it changed.
- Context management
- Four persistence layers ordered by authority, transcripts treated as disposable, compaction that reassembles each turn from recent exchanges plus fresh permission-filtered retrieval plus externalized knowledge, and prompt zoning that keeps the cached prefix stable.
- Control mechanisms
- A capability-based model registry with a single-source check on model names, a budget governor with three degradation stages enforced before a session spawns, and a nightly zero-token eval suite whose most important check is a safety canary.
03SaaS2026
ux-struggle-detector
Multi-tenant SaaS that reads a customer's own repository to understand their app, then detects user struggle server-side and answers inside the same request.
- Maps a customer web app by Babel AST parsing of its GitHub repository, producing the structure the detection rules run against.
- Runs 40 struggle-detection rules server-side over hydrated cross-request session history and returns interventions in the same HTTP response the events arrived in, so detection adds no extra round trip.
- Dependency-free browser SDK, client-side PII scrubbing, AES-GCM encrypted key storage.
- Stack
- TypeScript, Next.js 15, React, Prisma, PostgreSQL, Playwright
github.com/csnyder256/ux-struggle-detector
04Research2026
shadow-options-trading-laboption-contract-grader
Twenty strategies evaluated against live market data with zero orders placed, and a 6,000-name universe priced from first principles instead of from a vendor field.
- Runs 20 options strategies against live market data in shadow mode, placing zero orders. A subprocess-isolated test proves no order-placement code is reachable from the live path.
- Grades results with anytime-valid e-processes (test martingales, Ville's inequality), so nightly evaluation stays statistically valid under continuous monitoring.
- The grader sweeps a 6,000-name optionable universe, solving implied volatility and the Greeks from Black-Scholes-Merton locally instead of trusting vendor values, and scores every contract into A to F grades.
- Stack
- Python, numpy, pandas, FastAPI, SQLite, pytest, Hypothesis property-based fuzzing, GitHub Actions CI
shadow-options-trading-laboption-contract-grader
05
Also public: gba-rom-hack-ide, which exposes 71 tools through a Model Context Protocol server so a coding agent can edit a game directly; grain-bids-to-excel, 97 tests green in CI; and openrouter-model-picker, a client-side tool that joins live OpenRouter pricing and benchmark data and asks a free model to recommend one, 18 tests green in CI. Together with the four systems above, harness-tuner's 122 (three operating systems, three Python versions, per push), and openrouter-model-picker's 18, they account for the 1,177 tests.