KOM CLOUDSERVICE
All entries

Immediate Attention

Monday: all 21 checks in 69.0 seconds, our fastest day of the event and the next team home at 153.8. Nothing went wrong, so the rest of the day went to the thing that could: a challenge on Day 31 with nobody on it. Now two spare lanes start warm every morning.

Shaded passages were held back from the public edition while the event ran.

Monday was seven challenges and 21 checks, and the last one went CORRECT at 69.0 seconds after the release. That is our fastest full day of the event, more than 30 seconds better than Sunday, and the next team home took 153.8. Still 1st overall.

Nothing went wrong, and the day close agreed: no robot mistakes. The slow lanes were slow because they were new. So I spent the rest of Monday on the only thing left to worry about, which was the next new thing.

09:00

Seven challenges dropped. In the order they finished:

  • Mixed Signals (script): all three modes by 12.9 seconds.
  • Food for Thought (warmups): the whole blob was one esoteric program. 24.5.
  • Before You Go (metadata): a Windows prefetch file. 31.0.
  • Crash Course (binary): a Java crash trace. 46.5.
  • Serialized Offender (web): a session state the server trusts too much. 51.7.
  • Split Decision (mobile): an app split across several packages. 58.9.
  • Trust the Webhook (web): a webhook that believes what it’s told. 69.0.

The last four were shapes the robot had never seen, so each needed a hand agent at least once. A hand agent is an AI agent that works the question directly, the way a person would, and answers that one attempt instead of writing code.

Trust the Webhook took about 40 seconds. All five agents on the lane went after the request headers first, and between them sent about 120 forged events that came back webhook_rejected. The answer was in the body: the route trusted role and verification flags set in the event itself. One Sonnet agent found it. The second hand agent is Sonnet too and also missed, so this wasn’t one model beating another. Five agents probing the same door at once is what found the way in.

Split Decision cost 31.4 seconds on the Time Trial. The Single Solve was the familiar codec, but the Time Trial switched to the “analysis” codec, whose puzzle reads its pieces out of the split packages, and the robot’s dex interpreter couldn’t follow it there. The hand agent took about 31 seconds.

Crash Course asked for two fields the binary lane had never read, and the hand answer came 24.9 seconds into the Single Solve. A correct patch, the small fix an agent writes and the running lane picks up, was ready for the Time Trial. The judge, the automatic test every patch must pass, refused it as “no gain” 0.28 seconds before the Time Trial’s own ticket existed (a ticket is the short report the robot writes when it can’t answer, and it is what a patch is judged against). That cost about 5 to 6 seconds. The Mastery was bound by its download: ten files of about 35 MB each, around 8.4 seconds just to arrive.

Serialized Offender was mostly the server. The Game Master, the event’s API, spent 7.1, 5.5 and 10.1 seconds building the live targets for the three modes, and the hand answer took 14 seconds.

The record

From the public scoreboard, Day 26, all 21 checks:

  • Wizdumb 69.0 seconds, 1st on the day. OscarsWorld 153.8, crash 157.8.
  • First blood on 13 of 21 boards, first blood being the first team to solve a board: every mode of Crash Course, Split Decision, Trust the Webhook and Serialized Offender, plus the Mixed Signals Mastery.
  • 1st overall on 1,520 points. OscarsWorld and crash are on 1,520 too, so the order goes to whoever banks their last flag first, and that’s still us.

On the speed boards, as of 18:35, we’re 1st on 9 of Monday’s 14: both boards for Food for Thought, Mixed Signals and Split Decision, the Time Trials of both web challenges, and the Before You Go Mastery. 2nd on two more: Before You Go’s Time Trial (0.428 to c4rd0~‘s 0.350) and Crash Course’s Mastery (8.761 to team.dev’s 7.597).

Across all 26 days we hold 21 firsts, 32 seconds and 26 thirds. That’s three fewer firsts than Sunday night, even with nine new ones, because other teams keep rerunning the older boards. Sunday’s six are down to three. Everybody is sharpening their knives this week.

Immediate attention

What bothered me on Monday wasn’t anything that happened. It was what could.

The robot has a lane for each kind of challenge it has seen. If the organizers drop a sixth category on Day 31, or more challenges than the robot has places for, something has to pick it up the second it appears. Build that on the day and you are already late. So that morning I asked for the robot to account for every challenge released at the day’s start, however weird, and by late morning I had decided how: two additional spare lanes, spun up every day, specialty agnostic, able to take whatever gets presented. By the afternoon I had settled the rest: each one with the full roster of agents a real lane has. No shortcomings.

The rule behind all of it was simple: “It needs to be ensured that there are no challenges released that don’t get immediate attention and a subsequent solve.”

Two spares now start warm every morning. A spare takes any challenge the robot can’t place: a new category, a web challenge past the seventh slot (web slots went from 5 to 7 the same afternoon), or a second challenge in a lane that already has one, which the spare plays with that lane’s own solver. That last case also fixed a quiet bug: the release poller, the part of the robot that watches for the 09:00 drop, would have overwritten the first challenge in the lane with the second. A third spare and more start on demand, up to eight. And a route that ends the race with no player now fails the run outright, so a miss can’t hide.

We drilled it on the mock, our local stand-in for the event server. The drill threw a duplicate metadata challenge, a crypto challenge and a forensics challenge at the robot, plus one that needed a third spare from cold, and everything came back CORRECT in 6.75 seconds. The same drill with spares off ended in a failed run naming both uncovered routes, so the alarm works. It isn’t proven live yet; that needs a day that brings one.

By 11:40 the day’s eight live patches were folded into the solvers for good. The mock replay of Monday’s file lanes went 15 of 15 on the first try with no tickets, and a free live run of fresh web attempts with agents off went 4 of 4.

The evening

By 16:08 the spares were in, and before anything else I wanted every open item from the day closed out. One rule for the speed runs that followed: they must not end up in the corpus, the record of every file and grader verdict that each change is tested against. And they had to count for more than board places. Real Game Master calls against the current code are the best test there is.

They passed it. 33 ranked runs, every one CORRECT, on the evening’s code with agents off, and that is where most of Monday’s nine firsts came from.

What closed: speed runs now record to a separate folder the patch judge never reads (23,903 files went there, and the race corpus stayed at exactly 150,088). A carried patch, one reused from the previous mode of the same challenge, sends its answers one at a time until the grader accepts one, so a wrong patch can’t sink a whole batch. Web slots 2 to 7 are judged against the web lane’s own records. A challenge listed late is still picked up for up to 600 seconds after the release. And if a lane list shows two challenges in one lane, the robot waits about a second for the full list before choosing which is home, so the two machines can’t pick differently.

Two adversarial reviews went over it. The first found a blocker: a wide carried patch, say a script patch that answers every question, would have sent about 1,600 single answers and tripped the rate limit for the whole run. Singles are now capped at 20 per attempt, and past that the answers go back to the batch. It also found the new per-patch time cap would have cut off file-lane patches that cold-start their parser, so file lanes run uncapped again and the cap stays on web and spares at 10 seconds. The second review found the cap was counted per call, which let a Mastery send 20 singles per delivery. It’s per attempt now.

Then a blind before and after: the mock replayed Monday with the robot knowing nothing past Sunday, agents on, the code that raced against the code with every fix since. Last check at 108.4, 109.4 and 89.8 seconds before, 98.7 and 97.9 after, with the same tickets in the same lanes and modes and nothing new firing. No measurable change, which is what we expected: the fixes cover ways to fail that Monday didn’t have, and the spread between runs is the agents.

Crash Course’s Time Trial stayed 6th: its best run of the evening, 28.9 seconds, was 26.3 seconds of the server sending the download.

By the end of the evening nothing was outstanding, and I called the day done.

Five days

Monday is the shape we want for every day left: the known lanes done in under a minute, and anything new picked up the second it lands. If Day 31 brings a challenge nobody has seen, it’ll have a full team on it before the page finishes loading.

Five days to go.

How we built it: the Day 26 dev log

Why the duplicate probing stays. Five agents attacking Trust the Webhook’s headers at once looks like waste: about 120 rejected events. A shared probe log, so each agent sees what the others tried, was on the table. I turned it down, because “the duplicate agents on the same probe is what got us our solution, so I don’t want to break that.” It’s parked, not dropped. We would rather spend probes than seconds.

What the four refused patches were. The day close counted four refused patches on Split Decision, and I wanted to understand what had actually happened before weighing in on any fix. It was two patches, each judged twice, both shaped for the Single Solve rather than the Time Trial they were offered to. The judge was right, and there was nothing to fix.

Carry, and why it didn’t fire. Crash Course had a correct patch refused for “no gain” a fraction of a second before the next mode needed it. The fix: a patch that reproduces the answer the grader already accepted from the hand agent goes live for the next mode instead of being refused. I asked the plain question, whether patches and hand answers carry from each stage of a challenge to the next, and now they can. But the blind before and after never saw it fire, because the next mode’s ticket opens about a second after the Single Solve is CORRECT, before the patch arrives. Carry only helps when the judge finishes first. A drill that held the Time Trial back 30 seconds proved the mechanism: the patch carried in 882 milliseconds, and the Time Trial went CORRECT with no ticket.

The hollow check. A review found a patch could pass the carry test by reading the file and returning a constant: the right answer for the wrong reason. The commit’s own test fixture had exactly that shape. So a carried patch also runs on a hollow copy of the files, same names, empty contents. If it still gives the same answer, it never read anything, and it doesn’t carry.

Why no alert for coverage. A phone alert for an unclaimed challenge was offered. I’m in the watcher session anyway (the watcher is the AI session that watches the race live), so there was no reason to spend effort on better ntfy alerts. The watcher’s feed now prints which spare took which challenge, and a route with no player fails the run in plain text.

Why the timed runs were the test. Mock drills prove timing and wiring. They can’t prove the code talks to the real Game Master the way it thinks it does. So the 33 ranked runs did double duty, and they ran with the new recording folder set, which is how we know the corpus stayed untouched.