Skip to main content

Posts

High-Throughput Systems for Blockchain, Web3, and AI

Onchain execution and settlement, indexers, market data ingestion, AI agent infrastructure. Systems that move fast and cannot be wrong.

Founder of Scalytics. co-creator and PMC of Apache Wayang. Author of KafScale. Allianz, Cloudera and E.ON before that.

Consulting

Recent articles:

AI Agent Consensus Is Not Truth. It Is a Stale Cache Read.

Summary Agent networks are pitched as a replacement for web retrieval: models already hold the knowledge, so you query a crowd of agents instead of a site. This article argues that consensus between agents produces agreement, not freshness. It covers the split between knowledge and state, correlated error in multi-agent debate, why low rank adapters do not override base model dispositions, provenance decay in agent memory graphs, and what an arbitration layer has to do before agent output can be trusted. Bottom Line Polling a network of agents is a quorum read across replicas that may never have seen a recent write. It returns what the replicas hold, which is not the same as what is true now. Somewhere in the loop a node still has to read the origin, and the moment it does you have reinvented the HTTP request with layers of hearsay in front of it. The practical takeaway: design the arbitration layer first. Disagreement between agents is only useful if something resolves ...

Green in Cloudera Manager is not the same as ready

Summary Cloudera clusters can look finished after installation and still fail operationally under real workload shape. This article argues that Cloudera management is a production control surface, not an admin checklist. It covers monitoring gaps, topology assumptions, API-driven metrics, alert design, database dependencies, and runbook discipline. Installation proves the cluster starts, not that it can be run Cloudera Manager fails when teams treat it as the part that happens after installation instead of the control surface for production. The cluster can pass a setup guide and still be hard to operate under load if alerts, metrics, role placement, storage, service ownership, and rollback paths were never proven. The practical takeaway is simple: start with the incident you cannot afford, then design Cloudera Manager, external observability, and runbooks around the signals that would shorten that incident. The business depends on clusters nobody ...

Production CDC Architecture: Debezium Scaling Lessons

Production CDC architecture breaks under load long before most teams expect it. With Debezium, Kafka Connect, and Postgres, the failure patterns are consistent: WAL pressure builds up, connector lag drifts unnoticed, and snapshot phases exhaust memory under bursty traffic. This is based on running these pipelines across high throughput systems, including workloads above 10k TPS. The difference between a system that works and one that holds under pressure comes down to observability, WAL discipline, and how connector scaling is handled. Production Debezium CDC Architecture Operational reality vs. tutorial defaults under real load (10k+ TPS) The Default "Tutorial" Setup Assumes low throughput and stable networks. Fails under pressure. Source: Postgres Single WAL Slot Shared slot coupling multiple connectors Default WAL retention settin...

BacNet => MQTT in Production: The Real Cost of Bridging BACnet to MQTT at Scale

bacnet2mqtt looks simple in a README and expensive in production. Once BACnet polling, reconnection behavior, stale state, and MQTT publishing collide, teams discover they are not deploying a lightweight adapter but operating infrastructure. This article breaks down where bacnet2mqtt works, where it becomes a bottleneck, and which production patterns reduce the operational damage before incidents, backlogs, and silent data loss turn a building integration into a long-running engineering problem. I inherited a building controls integration problem 18 months ago. Three office floors. 217 BACnet sensors covering temperature, occupancy, and HVAC actuators. The data was trapped inside the building automation network while the business wanted analytics, reporting, and compliance visibility in the data platform. The obvious answer looked easy enough: deploy bacnet2mqtt, bridge BACnet into MQTT, and push the stream into the lakehouse stack. The repository made it sound like a w...

Agent Observability for Multi-Agent Systems: How to Trace Agent Workflows in Production

Agent observability breaks down when teams try to force long-lived, stateful workflows into dashboards built for stateless microservices. In production, the real challenge is not collecting more logs. It is reconstructing what the agent saw, what state changed, which tool response altered the workflow, and why the system kept going. This article explains why replayable event streams are a better foundation for multi-agent tracing, how a Kafka-first design makes session replay practical, and where conventional tracing still helps but falls short on its own. The production failure that changed how we instrument agents I still remember the first time one of our production agent systems failed without actually crashing. An invoice-processing agent entered a recursive reasoning loop and burned through hundreds of dollars in API credits over a weekend because it kept insisting a validation error existed when it did not. We had logs. We had metrics. We had distributed traces. None of...