Engineering notes
I run a long-lived autonomous agent — a few dozen scheduled jobs, a forecasting bot, a handful of paid API routes, and an unusually detailed ledger of what all of it actually earns. These are write-ups of things that broke, with the measurements attached.
-
My Actors passed every test I ran. Every agent call still crashed.
Four Apify Actors passed every test I ran and crashed for every agent that called them — the run died in the charging manager before my code executed, billed nothing, and looked like a fast, healthy run. The one-line fix, the CI test that comes in through the agent's door, and the four counters that cheerfully reported everything was fine.
-
A green test is a claim about a sample
Three suites that passed for weeks. Each had quietly chosen a sample nobody had looked at — the first eight items, the wrong entry path, the top two files — and none of them said so in its own output.
-
Your cron expression is not your schedule
An hourly job delivered 11.3 runs a day, and the delivery ratio moved by 36 points when the cadence changed. Every cost estimate and coverage guarantee derived from the schedule was wrong, in both directions.
-
Count the silence: three green monitors that were lying
Three scheduled jobs reported success for weeks while doing nothing. In none of the three cases was the bug in the job — it was in what the monitor counted. Includes the twenty-line max-gap watchdog that replaced them.
-
Measure it to the payout address
Every claim we had about demand for the agent's services came from a request log. Then we counted the money that actually moved: 566 transfers, 124 buyers where the ledger said one, and 59% of the revenue paid by ourselves.
Elsewhere
Open-source work lives on GitHub — including awesome-AI-driven-development, a map of where AI-agent money actually flows, a stdlib-only forecasting bot for the Metaculus AI benchmark tournaments, and a static security scanner for MCP servers and agent skills calibrated to a 1% false-positive rate over 196 public servers.