Edge asset pipeline

1. Foundational mental model

Everything the browser downloads that is not HTML or an API response lives in one Cloudflare R2 bucket, flags. Three pipelines write to it, and three hostnames read from it.

Writer	R2 prefix	Public host	Who reads it
bun run upload:flags (manual)	images/flags/…	cdn.flags.games	FlagImage, service worker, OG renderer
scripts/build.ts on Vercel production	assets/_app/…	assets.flags.games via the assets worker	SvelteKit client runtime
GET /api/revalidate cron	static/og/daily/…	cdn.flags.games	Social crawlers reading og:image

The fourth host, static.flags.games, never touches R2. It is the static worker, and it only proxies analytics (PostHog, Vemetric, Sentry). See Client observability.

Build outputs land in one bucket; each host serves one prefix.

The client chunks moved off Vercel to keep edge requests inside the Vercel budget. HTML still comes from flags.games; only /_app/* points at assets.flags.games, and only when PUBLIC_SVELTEKIT_ASSETS_ORIGIN is set at build time.

2. Implementation status & known gaps

Active invariants & limitations

• Active today: every flag object and every hashed client chunk is written with Cache-Control: public, max-age=31536000, immutable. Freshness for flags comes from explicit CDN purges; freshness for chunks comes from content hashes in the filename.
• Active today: preview and dev Vercel builds never upload or prune client chunks. shouldUploadClientBuild and shouldPruneClientBuild both return false when VERCEL=1 and VERCEL_ENV is not production.
• Active today: vercel.json defines one cron, GET /api/revalidate at 0 0 * * *. It revalidates /daily and renders that UTC day’s share images.
• Known gap: the service worker is not built when PUBLIC_SVELTEKIT_ASSETS_ORIGIN is set. SvelteKit cannot compile src/service-worker.ts alongside kit.paths.assets, so scripts/build.ts stashes the file for the build. Production therefore runs without SW caching.
• Known gap: flag uploads are manual. Nothing in CI runs upload:flags; a new SVG in static/images/flags/ is invisible on the CDN until someone runs it with R2 keys.
• Known gap: the flag CDN purge only recognises two-letter codes (/^([a-z]{2})\.(?:svg|webp)$/i). Re-uploading gb-eng.svg or ge-ab.svg does not purge their cached copies.

AGENTS.md names the account variable CLOUDFLARE_R2_ACCOUNT_ID. Every script reads CLOUDFLARE_ACCOUNT_ID. AGENTS.md also mentions a second ?mode=prewarm cron at 0 12 * * *; that cron is not in vercel.json, and /api/og/daily/generate has no mode parameter. It accepts an optional ?date=YYYY-MM-DD.

3. Concrete implementation

Generating flag rasters

Full-aspect SVGs under apps/web/static/images/flags/ are committed. Two scripts rasterize derived WebPs with sharp:

Script	Input	Output	Size
gen:flag-icons-webp	icons/*.svg	icons/webp/*.webp	192×144 px (2× the 96×72 chip), fit: "fill", quality 92, density 288
gen:flag-preview-webp	challenge preview *.svg	webp/preview/*.webp	preview height, quality 88, density 200

The preview script rasterizes the codes returned by listChallengeCompactPreviewFlagCodes() and takes its height from getFullFlagPreviewWebpImgDimensions.

Icon colors are corrected with hex-for-hex remaps (flag-icon:audit, then flag-icon:apply against data/flag-icon-color-remaps.json), never by swapping in a different SVG. At the time of writing the tree holds 1,075 SVG and WebP files: 259 full SVGs, 259 full WebPs, 5 previews, 276 icon SVGs, 276 icon WebPs.

Uploading flags

upload-to-r2.ts picks a transport from the environment. With all three S3 credentials it uses signed PUTs through aws4fetch; without them it shells out to wrangler once per file.

apps/web/scripts/r2/upload-to-r2.ts

const accountId = process.env.CLOUDFLARE_ACCOUNT_ID;
const accessKeyId = process.env.CLOUDFLARE_R2_ACCESS_KEY_ID;
const secretAccessKey = process.env.CLOUDFLARE_R2_SECRET_ACCESS_KEY;

if (accountId && accessKeyId && secretAccessKey) {
  await uploadFlagsS3FastPath(stats);
} else {
  // ... logs the wrangler mode ...
  await uploadDir(BASE_DIR, stats);
}

await purgeUploadedFlagCdn();

Each readable key also gets an opaque twin (webp/fr.webp and webp/o/<token>.webp); see Opaque flag tokens. A full run therefore writes 2,150 objects. Opaque SVG copies go through stripFlagSvgIdentity first so the body does not name the country.

Each S3 upload is one whole-object PUT with no multipart:

apps/web/scripts/r2/lib/r2-flags-s3-upload.ts

const response = await client.fetch(url, {
  method: "PUT",
  body,
  headers: {
    "Content-Type": contentType,
    "Cache-Control": cacheControl,
  },
});

After the upload, purgeFlagCdnForCountryCodes posts to Cloudflare’s purge_cache for every code it touched. Each code expands to 10 URLs (5 readable, 5 opaque), sent 30 per request. Pass --no-purge or set R2_UPLOAD_NO_PURGE=1 to skip.

Checkpoint and fresh runs

The checkpoint file is plain text with one R2 key per line, despite the .jsonl name in the docs. Each successful PUT appends a line through a promise chain so concurrent workers never interleave writes:

apps/web/scripts/r2/lib/r2-flags-s3-upload.ts

let checkpointChain = Promise.resolve();
const noteCheckpoint = (r2Key: string) => {
  if (!checkpointPath) {
    return;
  }
  checkpointChain = checkpointChain.then(() =>
    fs.appendFile(checkpointPath, `${r2Key}\n`, "utf8")
  );
};

On the next run, keys already in the file are filtered out before upload. R2_UPLOAD_FRESH=1 deletes the file first. The checkpoint only records that a key was written once; it does not detect that the local file changed since. After editing an SVG, run with R2_UPLOAD_FRESH=1 or without a checkpoint.

Client chunks to assets.flags.games

scripts/build.ts runs vite build, then upload-client-build.ts when the gate allows it. The upload walks .svelte-kit/output/client/_app, maps each file to assets/_app/…, writes env.js (which SvelteKit omits when paths.assets is set), and PUTs everything not already in .r2-client-build-checkpoint.jsonl. env.js is always re-uploaded. Cache headers differ by path:

apps/web/scripts/r2/upload-client-build.ts

const IMMUTABLE_CACHE_CONTROL = "public, max-age=31536000, immutable";
const MUTABLE_CACHE_CONTROL = "public, max-age=0, must-revalidate";

function cacheControlForR2Key(r2Key: string): string {
  return r2Key.includes("/immutable/") ? IMMUTABLE_CACHE_CONTROL : MUTABLE_CACHE_CONTROL;
}

After the upload, pruneStaleClientBuildObjects deletes keys under assets/_app/immutable/ that are not in this build and whose LastModified is older than R2_CLIENT_BUILD_PRUNE_GRACE_HOURS (default 48). Legacy root _app/** keys are deleted immediately. Deletes go out as S3 DeleteObjects batches of up to 1,000 keys.

The assets worker maps /_app/… to assets/_app/…, reads R2 through the WEB_ASSETS binding, and sets the same cache split. Non-immutable paths also get Cloudflare-CDN-Cache-Control: no-store so env.js is never pinned at the edge.

workers/assets/src/serve-immutable.js

const headers = new Headers();
const contentType =
  object.httpMetadata?.contentType ?? contentTypeForPathname(pathname);
headers.set("Content-Type", contentType);
headers.set("Cache-Control", cacheControlForPath(pathname));
if (!pathname.includes("/immutable/")) {
  headers.set("Cloudflare-CDN-Cache-Control", "no-store");
}

Request handling in the assets worker.

Daily share images and the midnight cron

Vercel calls /api/revalidate with Authorization: Bearer ${CRON_SECRET}. cronAuthOk compares the header in constant time and fails closed when the secret is unset. The handler revalidates /daily (with a 60 s fetch timeout), then renders the day’s images regardless of whether ISR succeeded.

Midnight cron: ISR first, then share images, then CDN purge.

generateDailyOgImages renders one page image and one result image per win/loss pattern. Dated files are immutable; today.png is a short-lived alias:

apps/web/scripts/og/daily/daily-og-constants.ts

export const DAILY_OG_TODAY_PNG_CACHE_CONTROL =
  "public, max-age=300, s-maxage=300, must-revalidate";

purgeDailyOgDateCdn skips quietly when CLOUDFLARE_CACHE_PURGE_API_TOKEN is missing and reports skipped: true in the response body.

Environment variables

Variable	Read by	Effect
CLOUDFLARE_ACCOUNT_ID	all R2 scripts, $lib/daily-og/r2.ts	R2 S3 endpoint host
CLOUDFLARE_R2_ACCESS_KEY_ID	same	S3 credentials; absence selects the wrangler path for flags
CLOUDFLARE_R2_SECRET_ACCESS_KEY	same	S3 credentials
CLOUDFLARE_R2_BUCKET	client build, daily OG	Bucket name, default flags
CLOUDFLARE_CACHE_PURGE_API_TOKEN	flag upload, daily OG	Zone Cache Purge; absence skips purges
R2_S3_UPLOAD_CONCURRENCY	flag + client build uploads	Worker count override, capped at 64
R2_UPLOAD_CHECKPOINT / R2_UPLOAD_FRESH	flag upload	Resume file / clear it first
R2_UPLOAD_VERBOSE / R2_UPLOAD_NO_PURGE	flag upload	Per-file logs / skip purge
WRANGLER_UPLOAD_CONCURRENCY / WRANGLER_UPLOAD_PAUSE_MS	flag upload, wrangler path	Parallel width (max 6) / sequential gap (default 1000 ms)
PUBLIC_SVELTEKIT_ASSETS_ORIGIN	svelte.config.js, client build	Sets kit.paths.assets; empty disables the chunk CDN
R2_UPLOAD_CLIENT_BUILD_REQUIRED	build gate	Fail the build if upload would be skipped (production only)
R2_UPLOAD_CLIENT_BUILD_SKIP	build gate	Never upload
R2_CLIENT_BUILD_CHECKPOINT / R2_CLIENT_BUILD_FRESH	client build	Resume file / clear it once after a prefix change
R2_CLIENT_BUILD_PRUNE_SKIP / _GRACE_HOURS / _BATCH_SIZE / _CONCURRENCY	prune	Skip, grace window (48), batch (1000), workers (16)
CRON_SECRET, ISR_REVALIDATE_SECRET, VERCEL_AUTOMATION_BYPASS_SECRET	/api/revalidate	Cron auth, ISR bypass token, protection bypass

4. Internal mechanics & mathematics

S3 worker count

With $n$ objects to upload and no override, both uploaders use

$$c(n) = \min\!\big(32,\ \max(16,\ \lceil n/50 \rceil)\big)$$

The floor of 16 applies for $n \le 800$; the ceiling of 32 applies for $n \ge 1{,}551$. A full flag run ($n = 2{,}150$) and a cold client build both run 32 workers. Setting R2_S3_UPLOAD_CONCURRENCY replaces the formula with $\min(64, \lfloor v \rfloor)$.

Workers share one index counter. Each worker takes the next index, uploads, and loops, so a slow object blocks only its own worker.

Retry backoff

Both transports classify failures with isThrottleOrOverloadError (429, "code":971, 502/503/504, SlowDown, socket resets, timeouts). For attempt $a \ge 1$:

$$b_{\text{throttle}}(a) = \min\!\big(90{,}000,\ 8{,}000 \cdot 2^{a-1}\big)\ \text{ms} + U[0, 1{,}500)$$ $$b_{\text{other}}(a) = \min\!\big(25{,}000,\ \lfloor 900 \cdot a^{1.25} \rfloor\big)\ \text{ms} + U[0, 400)$$

The throttle curve hits its 90 s cap at $a = 5$. The S3 path allows 18 attempts, the wrangler path 14. Wrangler jitter is $U[0, 1{,}200)$ and it has no jitter on the non-throttle path.

The wrangler path also self-throttles between files. Each throttle adds 3.5 s to a shared pause (capped at 45 s), and each success halves it:

$$p_{k+1} = \begin{cases} \min(45{,}000,\ p_k + 3{,}500) & \text{throttled} \\ \lfloor p_k / 2 \rfloor & \text{success} \end{cases}$$

Daily OG object count

DAILY_ROUND_COUNT is 5, so each day has $2^5 = 32$ result patterns (p0 … p31, one bit per round), plus default.png, plus today.png for the current day: $1 + 32 + 1 = 34$ objects and 34 purge URLs, sent as $\lceil 34 / 30 \rceil = 2$ requests.

5. Threat model, failure modes & edge cases

• Preview builds pruning production. Two outages came from this bucket. On 2026-08-02 a scheduled GitHub Action pruned against a keep-set built on Actions, deleting live chunk hashes. On 2026-08-12 a PR deploy with R2_UPLOAD_CLIENT_BUILD_REQUIRED=1 uploaded preview hashes, then pruned production’s. The gate now checks the Vercel preview condition before the REQUIRED flag, and .github/workflows/prune-r2-client-build.yml is manual and dry-run by default. Recovery: bun apps/web/scripts/r2/restore-client-build-from-origin.ts or a production redeploy.
• Stale tabs requesting old hashes. An open tab loaded before a deploy imports chunk hashes from the previous build. The 48-hour prune grace keeps those objects alive. If a chunk still 404s, installClientCrashRecovery reloads the page once per session (flags.assets-chunk-reload in sessionStorage).
• Wrong worker on the domain. If assets.flags.games is bound to the static worker, every /_app/* request 404s. The two workers send different Access-Control-Allow-Methods (GET, HEAD, OPTIONS vs GET, POST, OPTIONS), which is the quickest way to tell them apart with curl -sI.
• Checkpoint masking edits. A key in the checkpoint file is skipped even if the local bytes changed. Clear the file when re-uploading edited flags.
• Immutable flags without a purge. Flag keys are not content-addressed. If the purge token is missing, browsers and the edge keep the old image for up to a year. The upload logs CDN purge skipped and exits 0 in that case.
• Hyphenated codes skip the purge. See §2. bun run purge:flags -- gb-eng works, because the manual script takes codes from argv without the regex.
• Midnight cron partial failure. An ISR failure is logged and the cron continues to the images. A purge failure is logged and returned in dailyOg.cdnPurge; the route still returns 200. Only an image generation or R2 failure returns 500.
• env.js caching. It is mutable and carries public env. The upload always re-PUTs it, and the worker sends Cloudflare-CDN-Cache-Control: no-store so the edge never holds a stale copy.

6. Architectural tradeoffs & non-goals

Choice	Alternative	Why
Chunks on R2 behind a worker	Serve /_app/* from Vercel	The go-live checklist tracks /_app/immutable/* dropping off the top of Vercel’s route list. Moving chunks cuts Vercel edge requests; the cost is losing the service worker.
Whole-object PUT	S3 multipart	Flags and chunks are small. Multipart adds three round trips per object for no benefit.
Worker reads R2 binding	R2 public bucket on assets.	The worker controls Cache-Control, CORS, and no-store for env.js; a public bucket would serve whatever metadata was uploaded.
Line-per-key checkpoint	Compare ETags with HEAD	One HEAD per object would double request count on a 2,150-object run. The checkpoint is local and free.
URL-list purges per code	Purge everything	Purge-everything invalidates OG images and chunk caches too.
One cron doing ISR and OG	Two crons	The OG images depend on the same UTC day rollover as /daily; one entry point keeps them in step.

Rollback: unset PUBLIC_SVELTEKIT_ASSETS_ORIGIN on Vercel and redeploy. Chunks return to flags.games and the service worker builds again.

Non-goals: automatic flag uploads from CI, content-hashed flag URLs, and signed or private asset URLs.