Field study / Version 1 · Archive
Version 1 · Archive

ChronoBench

Measuring how far different language models can progress inChrono Trigger using an autonomous vision-based game agent.

Last updated: 2026-04-18 · Frozen at 17 runs.

These runs were produced by the v1 harness. For current runs on the v3 architecture, seeVersion 3, or browse the intermediatev2 archive.

How It WorksMethodology

Leaderboard

ModelProviderLast Checkpoint123456CyclesStuckTokens Inms/cycleEst. CostDate
google/gemma-4-26b-a4bLocalmarle_met182448180——200*492,018,37224,398Local · $02026-04-11
google/gemini-3-flash-previewOpenRoutermarle_met24307290——200*523,726,27713,687$2.052026-04-16
google/gemini-3-flash-previewOpenRoutermarle_met142172137——200*534,465,03112,932$2.422026-04-17
google/gemini-3-flash-previewOpenRoutermarle_met15183369——200*684,446,61914,076$2.412026-04-17
google/gemini-3-flash-previewOpenRoutermarle_met14425174——200*334,576,34414,267$2.472026-04-17
google/gemini-3-flash-previewLocalmarle_met14639299——200*683,158,12413,429Local · $02026-04-18
google/gemma-4-26b-a4bLocalfair_entered184866———100*311,055,47025,174Local · $02026-04-09
google/gemma-4-26b-a4bLocalfair_entered182436———100*421,062,86225,482Local · $02026-04-09
google/gemma-4-26b-a4bLocalfair_entered244860———200*502,077,87624,953Local · $02026-04-11
google/gemma-4-26b-a4bLocalfair_entered243084———200*492,391,20923,392Local · $02026-04-12

* = cycle budget exhausted

Cycles per checkpoint

04080119159199CyclesLeft Bedroomgoogle/gemma-4-26b-a4b: 1818google/gemini-3-flash-preview: 2424google/gemma-4-31b: 2424Exited Housegoogle/gemma-4-26b-a4b: 2424google/gemini-3-flash-preview: 3030google/gemma-4-31b: 4848Reached the Fairgoogle/gemma-4-26b-a4b: 4848google/gemini-3-flash-preview: 7272google/gemma-4-31b: 6666Met Marlegoogle/gemma-4-26b-a4b: 180180google/gemini-3-flash-preview: 9090Reached TelepodTime Traveledgoogle/gemma-4-26b-a4bgoogle/gemini-3-flash-previewgoogle/gemma-4-31b

Estimated cost per checkpoint

$0.00$0.40$0.80$1.20$1.60$2.00Cost ($)Left Bedroomgoogle/gemma-4-26b-a4b: $0.00google/gemini-3-flash-preview: $0.25$0.25google/gemma-4-31b: $0.00Exited Housegoogle/gemma-4-26b-a4b: $0.00google/gemini-3-flash-preview: $0.31$0.31google/gemma-4-31b: $0.00Reached the Fairgoogle/gemma-4-26b-a4b: $0.00google/gemini-3-flash-preview: $0.74$0.74google/gemma-4-31b: $0.00Met Marlegoogle/gemma-4-26b-a4b: $0.00google/gemini-3-flash-preview: $0.92$0.92Reached TelepodTime Traveledgoogle/gemma-4-26b-a4bgoogle/gemini-3-flash-previewgoogle/gemma-4-31b

All runs

ModelLast checkpointCyclesStuckDate
google/gemma-4-26b-a4bhouse_exit100*432026-04-09
google/gemma-4-26b-a4bfair_entered100*312026-04-09
google/gemma-4-26b-a4bfair_entered100*422026-04-09
claude-haiku-4-5-20251001—100*122026-04-09
google/gemma-4-26b-a4bfair_entered200*502026-04-11
google/gemma-4-26b-a4bmarle_met200*492026-04-11
google/gemma-4-26b-a4bfair_entered200*492026-04-12
google/gemma-4-26b-a4bfair_entered200*542026-04-12
google/gemma-4-31bfair_entered200*602026-04-12
google/gemini-3-flash-preview—102026-04-16
google/gemini-3-flash-previewmarle_met200*522026-04-16
google/gemini-3-flash-previewmarle_met200*532026-04-17
google/gemini-3-flash-previewmarle_met200*682026-04-17
google/gemini-3-flash-previewmarle_met200*332026-04-17
google/gemini-3-flash-preview—91372026-04-17
google/gemini-3-flash-previewmarle_met200*682026-04-18
google/gemma-4-26b-a4bfair_entered200*312026-04-18