Anti-cheat event ledger

Status: Production
Source: 9 files · shared 2 · server 4 · web 3
shared
utils/ledger.ts
schemas/websockets.ts
server
lib/managers/ledger-manager.ts
lib/http/handlers/game-results.ts
lib/constants/env.ts
lib/mcp/create-mcp-server.ts
web
lib/api/flags-api.ts
lib/components/game/game-runtime.svelte.ts
lib/env/public.ts
Verified: 2026-09-27 against master

1. Foundational mental model

Solo games run entirely in the browser. The server never sees a question until the run is over, so it cannot hold the answers the way multiplayer does. Instead the client keeps an append-only list of events: a q event when a flag appears, an a event when the player answers or the timer runs out. At the end, the client signs the list and POSTs it to /api/game-results. The game server re-derives the signature, checks that every question has exactly one answer, and recomputes score and reaction times from the timestamps. Nothing the client claims about its own score is trusted.

The signature is a SHA-256 over the events, a salt, and the session token. The salt comes from PUBLIC_SOLO_GAME_SALT, a public env value that the web app ships to every browser. Anyone who reads the bundle can sign any ledger they like. The signature therefore catches accidental corruption and lazy edits to a captured request body. It does not stop a player who writes their own signer. The timing rules in §4 do the real filtering, and they only look at plausibility.

One solo run from first flag to server verdict.

The simulator below runs the same three timing rules as scoreAnswers. A parity test (apps/docs/src/lib/ledger-reaction-rules.test.ts) replays each preset and 300 random ledgers through the real ledgerManager.dryRunReplay and asserts the same verdict.

Interactive

Solo ledger reaction-time validator

Build a run's per-question reaction times and check them against the same three humanity rules the game server applies in ledger-manager.ts.

Preset

100 ms hard floor 350 ms minimum mean Run mean Reaction time (ms)

Reaction time per question (ms)

Statistics

Answers: 10
Mean μ: 1098.0 ms
Std dev σ: 443.2 ms

Every answer ≥ 100 ms Fastest: 610 ms Pass
Mean ≥ 350 ms Mean: 1098.0 ms Pass
Std dev ≥ 50 ms (only when > 5 answers) σ: 443.2 ms Pass

Verdict

Accepted: the run is scored

Response the client sees

HTTP 200 { "status": "success", "xpToken": "<token or null>" }

On acceptance the server recomputes the score from the ledger. The status is success, opted_out or no_country; xpToken is only minted for signed-in players on runs long enough to earn XP.

Runs a mirror of scoreAnswers() from apps/server/src/lib/managers/ledger-manager.ts; a bun parity test replays the same lists through the real ledgerManager.dryRunReplay(). Signature and event-structure checks run before these rules and are not simulated here.

2. Implementation status & known gaps

Active today: every solo submission that carries a session_token cookie is replayed by ledgerManager.validateAndReplay before stats or XP are recorded. A request with no cookie gets 200 { "status": "skipped" } and is not replayed.
Active today: score is recomputed server-side as correct answers × CORRECT_POINT_COST (10). The client’s own totals are ignored for scoring.
Active today: the 403 body is the fixed string Game result could not be verified. The rejection code and human-readable reason go only to the server log (logRejectedSoloGameResult) and the stats counter (recordSoloGameRejected).
Active today: GameEventSchema keeps options on question events. Schema key order is type, code, ts, index, options, matching the object the client signs. game-ledger-schema.test.ts checks that a parsed ledger with options still verifies, and that older events without options still verify. Reordering either side changes JSON.stringify output and fails the signature. See §5.
Known limit: the salt is public (PUBLIC_SOLO_GAME_SALT, read through $env/dynamic/public). The signature is an integrity check against casual tampering. It is not a secret MAC.
Known limit: timestamps come from the client’s Date.now(). A forged ledger can pick any timestamps that pass the rules.

3. Concrete implementation

Signing

The signer sorts the events by timestamp in place, serializes them, appends salt and session token, and hashes:

export const DEFAULT_SOLO_GAME_SALT = "flags-default-salt-2026";

/**
 * Generates a deterministic signature for a sequence of game events.
 */
export async function signLedger(
  events: GameEvent[],
  salt: string,
  sessionId: string
): Promise<string> {
  const data = JSON.stringify(events.sort((a, b) => a.ts - b.ts)) + salt + sessionId;
  const encoder = new TextEncoder();
  const buffer = encoder.encode(data);
  const hashBuffer = await crypto.subtle.digest("SHA-256", buffer);
  const hashArray = Array.from(new Uint8Array(hashBuffer));
  return hashArray.map((b) => b.toString(16).padStart(2, "0")).join("");
}

export async function verifyLedgerSignature(
  ledger: GameLedger,
  salt: string,
  sessionId: string
): Promise<boolean> {
  const expected = await signLedger(ledger.events, salt, sessionId);
  return ledger.signature === expected;
}

This is a plain hash of message ‖ key, not an HMAC. Because the key is public, the difference does not matter here. JSON.stringify output depends on key order, so the server’s parsed objects must list keys in the same order the client created them.

The server refuses to boot in production with the default salt:

if (
  env.NODE_ENV === "production" &&
  env.PUBLIC_SOLO_GAME_SALT === DEFAULT_SOLO_GAME_SALT
) {
  throw new Error("PUBLIC_SOLO_GAME_SALT must be set in production");
}

Recording events

game-runtime.svelte.ts pushes a q event when a question is shown and an a event on answer or timeout. A timeout records the literal code timeout; a wrong pick with no selected code records wrong.

    this.#ledgerEvents.push({
      type: "q",
      code: questionData.currentCountry.code,
      ts: Date.now(),
      index: questionIndex,
      options: questionData.options.map((option) => option.code),
    });

The server schema that parses those events:

export const GameEventSchema = z.object({
  type: z.enum(["q", "a"]),
  code: z.string(),
  ts: z.number(),
  index: z.number(),
});

export const GameLedgerSchema = z.object({
  events: z.array(GameEventSchema),
  signature: z.string(),
});

Replay order

replayLedger checks the signature first, then structure, then timing. The first failure wins.

Replay pipeline. Each stage returns a rejection code on failure.

The timing thresholds are three private constants:

class LedgerManager {
  private readonly MIN_HARD_FLOOR_MS = 100; // Physically impossible per question
  private readonly MIN_BIOLOGICAL_AVG_MS = 350; // Human recognition + motor limit
  private readonly MIN_JITTER_STD_DEV = 50; // Bots are too consistent
  private readonly invalidReasonCounts = new Map<string, number>();

  private markInvalid(reason: string): void {
    const nextCount = (this.invalidReasonCounts.get(reason) ?? 0) + 1;
    this.invalidReasonCounts.set(reason, nextCount);
    if (nextCount % 25 === 0) {
      logger.warn("Ledger validation rejections summary", {
        reason,
        count: nextCount,
      });
    }
  }

And the mean and standard deviation checks, run after every answer has passed the floor:

    const avgTime = reactionTimes.reduce((a, b) => a + b, 0) / reactionTimes.length;

    if (avgTime < this.MIN_BIOLOGICAL_AVG_MS) {
      return this.reject(
        "avg_below_human_limit",
        `Average time (${Math.round(avgTime)}ms) below human limit`,
        totalQuestions,
        totalAnswers,
        recordRejections
      );
    }

    const variance =
      reactionTimes.reduce((acc, b) => acc + (b - avgTime) ** 2, 0) /
      reactionTimes.length;
    const stdDev = Math.sqrt(variance);

    if (stdDev < this.MIN_JITTER_STD_DEV && reactionTimes.length > 5) {
      return this.reject(
        "insufficient_variance",
        `Insufficient response variance (${Math.round(stdDev)}ms) - potential bot`,
        totalQuestions,
        totalAnswers,
        recordRejections
      );
    }

The withheld reason

postSoloGameResults logs the full replay result and returns a fixed body:

    if (!replay.isValid) {
      logRejectedSoloGameResult({
        sessionLogRef,
        reason: replay.reason ?? "unknown",
        countryCode,
        origin,
        body: validation.data,
        replay,
      });
      recordSoloGameRejected({
        startedAt: getSoloLedgerStartTimestamp(validation.data),
        questionsCount: replay.totalQuestions || validation.data.totalQuestions,
        rejectionCode: replay.code ?? "unknown",
        sourceIp: getClientIPAddress(req),
        trustedCountryCode: countryCode,
      });
      return createValidatedJsonResponse(
        GameResultsVerificationFailedResponseSchema,
        { error: "Game result could not be verified" },
        403,
        origin
      );
    }

A script that probes the endpoint learns only pass or fail, not which rule it tripped.

The client treats a 403 as a possibly stale session. submitSoloGameResults refreshes the token, re-signs, and retries once:

      if (!result.ok && result.status === 403) {
        const refreshedToken = await flagsApiSubmitDeps.refreshSessionToken();
        if (!refreshedToken) {
          return { error: result.error };
        }
        activeSignature = await flagsApiSubmitDeps.signLedger(
          params.ledgerEvents,
          salt,
          refreshedToken
        );
        result = await postSoloGameResults(
          url,
          buildSoloGameResultsBody(params, activeSignature)
        );

Debugging with MCP

When the optional agent MCP server is enabled (docs/reference/mcp-server.md), replay_solo_ledger runs the same replay without side effects. It calls dryRunReplay, which skips the rejection counters, and returns the full result including code and reason:

    "replay_solo_ledger",
    {
      description:
        "Dry-run solo ledger validation and scoring. Read-only — does not persist stats, issue XP tokens, or update rejection counters.",
      inputSchema: {
        sessionToken: z
          .string()
          .min(1)
          .describe("Anonymous session token used when the ledger was signed."),
        ledger: GameLedgerSchema.describe("Signed solo game ledger from the client."),
      },
    },
    async ({ sessionToken, ledger }) => {
      const replay = await ledgerManager.dryRunReplay(ledger, sessionToken);

      return jsonToolResult({
        persisted: false,
        replay,
      });
    }

The tool input goes through the same GameLedgerSchema, so options is kept and the dry-run hash matches the signature the client produced.

4. Internal mechanics & mathematics

Reaction time

For question index $i$ with question timestamp $t^{q}_i$ and answer timestamp $t^{a}_i$ (both client Date.now() in milliseconds):

r_i = t^{a}_i - t^{q}_i

Timeouts count. A player who lets the timer expire contributes a large $r_i$ , which raises the mean and the spread.

Rules

With $n$ answered questions, the server computes the arithmetic mean and the population standard deviation (it divides by $n$ , not $n - 1$ ):

\mu = \frac{1}{n}\sum_{i=1}^{n} r_i \qquad \sigma = \sqrt{\frac{1}{n}\sum_{i=1}^{n} \left(r_i - \mu\right)^2}

A ledger is rejected by the first rule that holds, checked in this order:

\begin{aligned} &\text{floor:} && \exists\, i : r_i < 100 \\ &\text{mean:} && \mu < 350 \\ &\text{jitter:} && \sigma < 50 \;\land\; n > 5 \end{aligned}

All comparisons are strict. $r_i = 100$ passes the floor, $\mu = 350$ passes the mean rule, and $\sigma = 50$ passes the jitter rule. The floor rule reports the first offending answer in timestamp order, not the smallest.

Rule	Code	Threshold	Applies when
Hard floor	`inhuman_reaction_time`	100 ms	every answer
Biological mean	`avg_below_human_limit`	350 ms	$n \ge 1$
Jitter	`insufficient_variance`	50 ms	$n > 5$

What it takes to evade

A script that knows the rules only has to pick timestamps inside the accepted region. For reaction times drawn uniformly from $[m - w/2,\ m + w/2]$ , the standard deviation is $w / \sqrt{12}$ . Passing the jitter rule needs

\frac{w}{\sqrt{12}} \ge 50 \;\Longrightarrow\; w \ge 50\sqrt{12} \approx 173\ \text{ms}

and passing the floor needs $m - w/2 \ge 100$ . So uniform(450, 650) passes every rule. The rules raise the bar from “replay a fixed delay” to “add noise”, which is a few characters of code.

False positives

The jitter rule can also fire on a real player. A fast, consistent run such as the simulator’s Fast and steady preset (mean 419.5 ms, $\sigma \approx 46.9$ ms over 10 answers) is rejected. The rule only bites when the mean is already near the floor: at a 1-second mean, $\sigma < 50$ means a spread under 5 % of the mean, which is rare for real players. The Fast but human preset (mean 453.5 ms, $\sigma \approx 81.4$ ms) passes.

Rejection log sampling

markInvalid keeps a per-code counter for the process lifetime and emits one warn line every 25 rejections of the same code. Individual rejections are still logged by logRejectedSoloGameResult in the handler.

5. Threat model, failure modes & edge cases

Who the ledger stops and who it does not.

Key order is part of the signature. signLedger hashes JSON.stringify of the events. The client writes question events as type, code, ts, index, options. Zod emits keys in schema order, and GameEventSchema uses that same order, with options optional so events that omit it still verify. Reordering the schema or the client object changes the string and every submission fails invalid_signature. packages/shared/src/schemas/game-ledger-schema.test.ts locks both shapes.
Public salt. A player who extracts PUBLIC_SOLO_GAME_SALT from the bundle and their session_token from the session-token endpoint can sign any ledger. The signature catches casual edits only.
Client clock. All timestamps come from Date.now() in the browser. Adjusting the system clock mid-run can produce negative or huge $r_i$ . Negative values fail the floor; huge values inflate the mean and pass.
Equal timestamps. Events are sorted by ts with Array.prototype.sort, which is stable. A q and a with the same ts keep their insertion order, and $r_i = 0$ fails the floor anyway.
In-place sort. signLedger and replayLedger both sort ledger.events in place. Callers that reuse the array see it reordered.
Session requirement. Without a session_token cookie the server returns skipped and records nothing. Replays are not deduplicated by the ledger itself; the Convex forward uses a sourceKey built from session, start time and totals.
Rate limit before replay. /api/game-results is limited to 12 submissions per session and 40 per IP per minute before any hashing happens. See Rate limiting and moderation.

6. Architectural tradeoffs & non-goals

The ledger is for keeping honest leaderboards and country stats clean of edited request bodies and naive bots, at zero latency cost during play.

It is not for proving a run happened. A determined player can forge a passing ledger. Stakes in solo are XP and aggregate country stats, and the fingerprint and cohort checks described in Device fingerprint engine add a second, independent signal on the stats path.

Alternative	Why it was not chosen
Server-issued questions per round	Needs a round trip per flag and breaks offline and instant-start solo play. Multiplayer already works this way.
Secret HMAC key on the server only	The client must produce the signature, so any key it holds is public. A real MAC would need the server to sign each event as it happens.
Report the rejection reason in the 403	Tells a script author exactly which threshold to tune past. Operators get the reason from logs and `replay_solo_ledger` instead.
Sample standard deviation ( $n - 1$ )	Slightly more lenient for short runs. The server uses population $\sigma$ , and the jitter rule is skipped below 6 answers, which covers the small-sample bias.
Per-player adaptive thresholds	Requires history per player and opens a slow-poisoning attack. Fixed thresholds are simple to reason about and test.