Design System
Benchmark the performance of agents with different design system tasks.
Runs
Capability performance
Benchmark results are shown first, followed by the percent change from Control in parentheses. Capability rows aggregate all of their scenario and model results. Latest results: .
| Capability | Scenario | Model | Reasoning effort | Tests | Output tokens | Premium requests | Session time | API time |
|---|---|---|---|---|---|---|---|---|
| Uses appropriate components by default | All scenarios | All models | 45/55 (+275.0%) | 45,668 (-3.9%) | 64 (0%) | 19m 28.0s (+41.3%) | 13m 9.2s (+46.8%) | |
| 001-agent-uses-button-from-primer | All models | 12/20 (+300.0%) | 14,970 (+170.6%) | 32 (0%) | 7m 33.4s (+166.0%) | 5m 5.8s (+146.3%) | ||
| gemini-3.5-flash | medium | 4/4 (N/A) | 1,074 (+44.0%) | 14 (0%) | 1m 21.5s (+59.6%) | 1m 7.7s (+63.9%) | ||
| claude-sonnet-5 | medium | 4/4 (N/A) | 2,619 (+102.7%) | 1 (0%) | 1m 3.4s (+164.3%) | 42.3s (+152.3%) | ||
| claude-opus-5 | medium | 4/4 (+33.3%) | 3,889 (+180.2%) | 15 (0%) | 1m 54.5s (+211.4%) | 1m 0.5s (+169.5%) | ||
| gpt-5.6-terra | medium | 0/4 (0%) | 1,455 (+19.1%) | 1 (0%) | 32.2s (+17.7%) | 22.4s (+11.3%) | ||
| gpt-5.6-sol | medium | 0/4 (0%) | 5,933 (+571.2%) | 1 (0%) | 2m 41.8s (+416.6%) | 1m 53.1s (+378.7%) | ||
| 003-agent-uses-form-from-primer | All models | 33/35 (+266.7%) | 30,698 (-26.9%) | 32 (0%) | 11m 54.5s (+8.9%) | 8m 3.4s (+16.9%) | ||
| claude-sonnet-5 | medium | 7/7 (+16.7%) | 5,126 (-28.5%) | 1 (0%) | 1m 48.5s (-18.9%) | 1m 6.5s (-16.8%) | ||
| gpt-5.6-sol | medium | 7/7 (+600.0%) | 5,605 (+30.3%) | 1 (0%) | 1m 57.7s (+15.8%) | 1m 16.3s (+96.3%) | ||
| gpt-5.6-terra | medium | 7/7 (+600.0%) | 5,831 (+67.7%) | 1 (0%) | 2m 22.4s (+84.3%) | 1m 25.6s (+178.5%) | ||
| gemini-3.5-flash | medium | 6/7 (+500.0%) | 4,021 (-77.6%) | 14 (0%) | 2m 10.2s (-21.9%) | 1m 50.7s (-26.1%) | ||
| claude-opus-5 | medium | 6/7 (N/A) | 10,115 (+11.6%) | 15 (0%) | 3m 35.8s (+21.9%) | 2m 24.4s (+26.4%) | ||
| Uses Octicons by default for iconography | All scenarios | All models | 10/10 (0%) | 3,901 (-9.2%) | 32 (0%) | 2m 40.2s (-8.3%) | 1m 39.0s (-19.9%) | |
| 002-agent-uses-octicon-from-primer | All models | 10/10 (0%) | 3,901 (-9.2%) | 32 (0%) | 2m 40.2s (-8.3%) | 1m 39.0s (-19.9%) | ||
| gpt-5.6-terra | medium | 2/2 (0%) | 544 (+2.3%) | 1 (0%) | 17.9s (+5.2%) | 11.6s (+24.8%) | ||
| gemini-3.5-flash | medium | 2/2 (0%) | 610 (-35.8%) | 14 (0%) | 46.5s (-35.6%) | 35.2s (-40.0%) | ||
| gpt-5.6-sol | medium | 2/2 (0%) | 738 (+5.3%) | 1 (0%) | 35.4s (+21.7%) | 16.4s (-16.3%) | ||
| claude-sonnet-5 | medium | 2/2 (0%) | 927 (-17.1%) | 1 (0%) | 24.1s (-7.4%) | 15.1s (-16.1%) | ||
| claude-opus-5 | medium | 2/2 (0%) | 1,082 (+8.5%) | 15 (0%) | 36.3s (+19.7%) | 20.8s (+14.7%) | ||
| Sets up new projects with Primer | All scenarios | All models | 62/130 (+195.2%) | 100,420 (+50.1%) | 64 (0%) | 38m 7.1s (+33.7%) | 24m 19.6s (+60.4%) | |
| 004-agent-setup-nextjs | All models | 22/40 (+4.8%) | 36,560 (+24.9%) | 32 (0%) | 15m 6.9s (-6.7%) | 9m 41.2s (+21.4%) | ||
| claude-opus-5 | medium | 6/8 (+100.0%) | 6,127 (-17.4%) | 15 (0%) | 2m 44.1s (+2.7%) | 1m 36.5s (-8.2%) | ||
| gpt-5.6-terra | medium | 5/8 (+66.7%) | 8,514 (+88.9%) | 1 (0%) | 3m 30.1s (+83.8%) | 2m 0.6s (+100.0%) | ||
| claude-sonnet-5 | medium | 4/8 (-42.9%) | 4,661 (-42.7%) | 1 (0%) | 1m 49.6s (-27.4%) | 58.3s (-39.0%) | ||
| gpt-5.6-sol | medium | 4/8 (-20.0%) | 10,896 (+229.4%) | 1 (0%) | 4m 11.4s (+174.0%) | 2m 36.4s (+160.6%) | ||
| gemini-3.5-flash | medium | 3/8 (0%) | 6,362 (+7.9%) | 14 (0%) | 2m 51.7s (-62.3%) | 2m 29.4s (-5.4%) | ||
| 005-agent-enables-theme-switching | All models | 40/90 (N/A) | 63,860 (+69.7%) | 32 (0%) | 23m 0.2s (+86.9%) | 14m 38.4s (+103.8%) | ||
| gemini-3.5-flash | medium | 15/18 (N/A) | 5,975 (-28.7%) | 14 (0%) | 4m 10.0s (+65.9%) | 2m 38.8s (+38.4%) | ||
| gpt-5.6-sol | medium | 15/18 (N/A) | 12,901 (+82.1%) | 1 (0%) | 4m 18.4s (+67.3%) | 2m 43.7s (+153.3%) | ||
| claude-sonnet-5 | medium | 9/18 (N/A) | 11,706 (+40.5%) | 1 (0%) | 3m 34.0s (+38.8%) | 2m 29.3s (+48.3%) | ||
| claude-opus-5 | medium | 1/18 (N/A) | 25,654 (+204.4%) | 15 (0%) | 8m 34.2s (+224.8%) | 5m 20.2s (+215.2%) | ||
| gpt-5.6-terra | medium | 0/18 (0%) | 7,624 (+41.1%) | 1 (0%) | 2m 23.6s (+18.8%) | 1m 26.4s (+75.3%) |