BaşlarkenLiderlik tablosuDecode calculatorModellerReportsDonanımKarşılaştırmalı DeğerlendirmelerPazar yeriKiralıklarProAPI Dokümanlar
Dil
Actual Computer — Every computer, one endpoint

HellaSwag

Official shard eval

HellaSwag deterministic disjoint eval shards from Rowan/hellaswag:validation. Running all 13 shards covers the full 10,042-question dataset; Postgres stores only shard metadata and pass/fail results.

Source
Category: ReasoningEval type: Shard evalQuestions: 10,042Shards: 13Runs: 201

Dry-run first:

lmx eval shard hellaswag --base-url http://localhost:8000 --questions 371 --dry-run

Then submit with a real model and hardware profile:

lmx eval shard hellaswag --base-url http://localhost:8000 --questions 371 --model <hfId> --hardware hardware.json --submit

Scores are pooled by unique question_id; the leaderboard is ranked by Wilson 95% lower bound. HellaSwag uses canonical continuation loglikelihood scoring. Use a logprob-capable /v1/completions endpoint, or pass --model-path model.gguf to use the bundled local llama.cpp scorer.

Leaderboard

Qwen3.6-27B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
unsloth/Qwen3.6-27B-GGUF · 1 run · 1/13 shards · harness-scoped
93.3%
95% CI 91.394.8%
721/773 correct · 7.7% coverage
2
Qwopus3.6-27B-Coder-MTP-GGUFQ5_K_M · ggufunknown protocol · unknown agent
Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF · 1 run · 1/13 shards · harness-scoped
92.9%
95% CI 90.894.5%
717/772 correct · 7.7% coverage
3
Qwen3.6-35B-A3B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
unsloth/Qwen3.6-35B-A3B-GGUF · 26 runs · 13/13 shards · harness-scoped
90.4%
95% CI 89.891.0%
9079/10042 correct · 100.0% coverage
4
Qwen3.5-27B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
unsloth/Qwen3.5-27B-GGUF · 1 run · 1/13 shards · harness-scoped
91.8%
95% CI 89.793.6%
709/772 correct · 7.7% coverage
5
Qwen3.6-35B-A3B-GGUFQ2_K_XL · GGUFunknown protocol · unknown agent
unsloth/Qwen3.6-35B-A3B-GGUF · 30 runs · 13/13 shards · harness-scoped
89.3%
95% CI 88.789.9%
8971/10042 correct · 100.0% coverage
6
Qwen3.5-27B-Uncensored-HauhauCS-AggressiveQ4_K_M · GGUFunknown protocol · unknown agent
HauhauCS/Qwen3.5-27B-Uncensored-HauhauCS-Aggressive · 1 run · 1/13 shards · harness-scoped
90.7%
95% CI 88.492.5%
700/772 correct · 7.7% coverage
7
Qwen3.8-27B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
unsloth/Qwen3.8-27B-GGUF · 1 run · 1/13 shards · harness-scoped
89.8%
95% CI 87.491.7%
694/773 correct · 7.7% coverage
8
Qwen3.8-27B-GGUFQ6_K_XL · GGUFunknown protocol · unknown agent
unsloth/Qwen3.8-27B-GGUF · 1 run · 1/13 shards · harness-scoped
89.1%
95% CI 86.791.1%
688/772 correct · 7.7% coverage
9
Qwen3.6-35B-A3B-Uncensored-HauhauCS-AggressiveQ2_K_P · GGUFunknown protocol · unknown agent
HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive · 13 runs · 13/13 shards · harness-scoped
85.4%
95% CI 84.786.1%
8579/10042 correct · 100.0% coverage
10
Qwen3.6-35B-A3B-Uncensored-HauhauCS-AggressiveQ4_K_M · GGUFunknown protocol · unknown agent
HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive · 13 runs · 13/13 shards · harness-scoped
85.0%
95% CI 84.385.7%
8537/10042 correct · 100.0% coverage
11
DeepSeek-V4-Flash-0731FP8unknown protocol · unknown agent
deepseek-ai/DeepSeek-V4-Flash-0731 · 1 run · 1/13 shards · harness-scoped
86.7%
95% CI 84.188.9%
670/773 correct · 7.7% coverage
12
gemma-4-31B-it-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
unsloth/gemma-4-31B-it-GGUF · 1 run · 1/13 shards · harness-scoped
86.0%
95% CI 83.488.3%
665/773 correct · 7.7% coverage
13
GLM-5.3-Flashexl3unknown protocol · unknown agent
zai-org/GLM-5.3-Flash · 1 run · 1/13 shards · harness-scoped
86.0%
95% CI 83.488.3%
664/772 correct · 7.7% coverage
14
GLM-5.3-Flash-tr3-4bpwexl3unknown protocol · unknown agent
brandonmusic/GLM-5.3-Flash-tr3-4bpw · 1 run · 1/13 shards · harness-scoped
84.7%
95% CI 82.087.1%
655/773 correct · 7.7% coverage
15
Qwen3.8-Flash-Next-NVFP4NVFP4unknown protocol · unknown agent
nvidia/Qwen3.8-Flash-Next-NVFP4 · 1 run · 1/13 shards · harness-scoped
84.7%
95% CI 82.087.1%
654/772 correct · 7.7% coverage
16
Qwen3.8-27B-W4A16-AWQunspecified quantunknown protocol · unknown agent
philbert440/Qwen3.8-27B-W4A16-AWQ · 1 run · 1/13 shards · harness-scoped
85.2%
95% CI 81.288.4%
316/371 correct · 3.7% coverage
17
Qwen3.8-27B-GPTQ-W4A16unspecified quantunknown protocol · unknown agent
pearsonkyle/Qwen3.8-27B-GPTQ-W4A16 · 13 runs · 13/13 shards · harness-scoped
81.9%
95% CI 80.883.0%
3951/4823 correct · 48.0% coverage
18
DeepSeek-V4-Flash-0731unspecified quantunknown protocol · unknown agent
deepseek-ai/DeepSeek-V4-Flash-0731 · 1 run · 1/13 shards · harness-scoped
86.0%
95% CI 80.590.1%
172/200 correct · 2.0% coverage
19
Gemma-4-31B-IT-NVFP4unspecified quantunknown protocol · unknown agent
nvidia/Gemma-4-31B-IT-NVFP4 · 12 runs · 12/13 shards · harness-scoped
80.9%
95% CI 79.782.0%
3600/4452 correct · 44.3% coverage
20
KAT-Coder-V2.5-Dev-AWQ-W4A16-ASYMunspecified quantunknown protocol · unknown agent
Ar4ikov/KAT-Coder-V2.5-Dev-AWQ-W4A16-ASYM · 1 run · 1/13 shards · harness-scoped
83.6%
95% CI 79.487.0%
310/371 correct · 3.7% coverage
21
Qwen3.8-Flash-Next-NVFP4NVFP4unknown protocol · unknown agent
local-inference-lab/Qwen3.8-Flash-Next-NVFP4 · 1 run · 1/13 shards · harness-scoped
82.1%
95% CI 79.384.7%
635/773 correct · 7.7% coverage
22
Qwen3.8-27B-MTP-NVFP4unspecified quantunknown protocol · unknown agent
sakamakismile/Qwen3.8-27B-MTP-NVFP4 · 1 run · 1/13 shards · harness-scoped
83.3%
95% CI 79.286.7%
309/371 correct · 3.7% coverage
23
Qwen3.6-35B-A3B-GPTQ-Int4unspecified quantunknown protocol · unknown agent
palmfuture/Qwen3.6-35B-A3B-GPTQ-Int4 · 7 runs · 6/13 shards · harness-scoped
80.6%
95% CI 78.982.2%
1795/2226 correct · 22.2% coverage
24
Qwen3.6-27B-NVFP4NVFP4unknown protocol · unknown agent
nvidia/Qwen3.6-27B-NVFP4 · 13 runs · 13/13 shards · harness-scoped
79.8%
95% CI 78.680.9%
3848/4823 correct · 48.0% coverage
25
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4unspecified quantunknown protocol · unknown agent
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 · 1 run · 1/13 shards · harness-scoped
82.7%
95% CI 78.686.3%
307/371 correct · 3.7% coverage
26
Muse-Glimmer-30B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
unsloth/Muse-Glimmer-30B-GGUF · 1 run · 1/13 shards · harness-scoped
80.8%
95% CI 77.983.5%
624/772 correct · 7.7% coverage
27
Muse-Glimmer-30B-GGUFQ5_K_XL · GGUFunknown protocol · unknown agent
unsloth/Muse-Glimmer-30B-GGUF · 1 run · 1/13 shards · harness-scoped
80.3%
95% CI 77.382.9%
619/771 correct · 7.7% coverage
28
Ornith-1.0-35B-PrismaAURA-4.75bit-vllm-MTPunspecified quantunknown protocol · unknown agent
rdtand/Ornith-1.0-35B-PrismaAURA-4.75bit-vllm-MTP · 2 runs · 2/13 shards · harness-scoped
80.2%
95% CI 77.282.9%
595/742 correct · 7.4% coverage
29
gemma-4-26B-A4B-it-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
unsloth/gemma-4-26B-A4B-it-GGUF · 1 run · 1/13 shards · harness-scoped
79.5%
95% CI 76.582.2%
614/772 correct · 7.7% coverage
30
Ornith-1.0-35B-AWQ-INT4unspecified quantunknown protocol · unknown agent
cyankiwi/Ornith-1.0-35B-AWQ-INT4 · 1 run · 1/13 shards · harness-scoped
80.6%
95% CI 76.384.3%
299/371 correct · 3.7% coverage
31
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressiveunspecified quantunknown protocol · unknown agent
HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive · 1 run · 1/13 shards · harness-scoped
79.1%
95% CI 76.181.9%
611/772 correct · 7.7% coverage
32
Muse-Glimmer-30B-NVFP4unspecified quantunknown protocol · unknown agent
RedHatAI/Muse-Glimmer-30B-NVFP4 · 1 run · 1/13 shards · harness-scoped
79.5%
95% CI 75.183.3%
295/371 correct · 3.7% coverage
33
Qwen3.6-27B-AEON-Ultimate-Uncensored-Text-NVFP4-MTP-XSunspecified quantunknown protocol · unknown agent
AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-Text-NVFP4-MTP-XS · 1 run · 1/13 shards · harness-scoped
79.0%
95% CI 74.582.8%
293/371 correct · 3.7% coverage
34
Ornith-1.5-35B-A3B-NVFP4unspecified quantunknown protocol · unknown agent
ornith-ai/Ornith-1.5-35B-A3B-NVFP4 · 1 run · 1/13 shards · harness-scoped
79.0%
95% CI 74.582.8%
293/371 correct · 3.7% coverage
35
Ornith-1.0-9Bunspecified quantunknown protocol · unknown agent
ornith-ai/Ornith-1.0-9B · 2 runs · 2/13 shards · harness-scoped
84.1%
95% CI 70.692.1%
37/44 correct · 0.4% coverage
36
Laguna-XS-2.1-INT4INT4unknown protocol · unknown agent
poolside/Laguna-XS-2.1-INT4 · 1 run · 1/13 shards · harness-scoped
73.6%
95% CI 68.977.8%
273/371 correct · 3.7% coverage
37
Hy-MT2-7Bunspecified quantunknown protocol · unknown agent
tencent/Hy-MT2-7B · 4 runs · 4/13 shards · harness-scoped
65.0%
95% CI 60.269.5%
260/400 correct · 4.0% coverage
38
gemma-4-E4B-itunspecified quantunknown protocol · unknown agent
google/gemma-4-E4B-it · 3 runs · 3/13 shards · harness-scoped
65.3%
95% CI 59.770.5%
194/297 correct · 3.0% coverage
39
granite-4.0-h-microunspecified quantunknown protocol · unknown agent
ibm-granite/granite-4.0-h-micro · 1 run · 1/13 shards · harness-scoped
76.7%
95% CI 59.188.2%
23/30 correct · 0.3% coverage
40
EXAONE-3.5-7.8B-Instruct-GGUFunspecified quantunknown protocol · unknown agent
LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct-GGUF · 1 run · 1/13 shards · harness-scoped
73.3%
95% CI 55.685.8%
22/30 correct · 0.3% coverage
41
gpt-oss-20b-GGUFMXFP4 · GGUFunknown protocol · unknown agent
ggml-org/gpt-oss-20b-GGUF · 8 runs · 8/13 shards · harness-scoped
51.4%
95% CI 49.753.2%
1527/2968 correct · 29.6% coverage
42
RWKV7-Goose-World3-2.9B-HFunspecified quantunknown protocol · unknown agent
RWKV/RWKV7-Goose-World3-2.9B-HF · 2 runs · 2/13 shards · harness-scoped
61.7%
95% CI 49.072.9%
37/60 correct · 0.6% coverage
43
Mellum2-12B-A2.5B-Instruct-GGUF-Q4_K_Munspecified quantunknown protocol · unknown agent
JetBrains/Mellum2-12B-A2.5B-Instruct-GGUF-Q4_K_M · 3 runs · 3/13 shards · harness-scoped
73.3%
95% CI 48.089.1%
11/15 correct · 0.1% coverage
44
MiniCPM3-4Bunspecified quantunknown protocol · unknown agent
openbmb/MiniCPM3-4B · 4 runs · 4/13 shards · harness-scoped
52.1%
95% CI 45.558.6%
114/219 correct · 2.2% coverage
45
gemma-4-12b-it-GGUFQ8_K_XL · gguf
unsloth/gemma-4-12b-it-GGUF · 1 run · 1/13 shards · harness-scoped
54.0%
95% CI 44.363.4%
54/100 correct · 1.0% coverage
46
gemma-4-E4B-it-qat-q4_0-ggufunspecified quantunknown protocol · unknown agent
google/gemma-4-E4B-it-qat-q4_0-gguf · 1 run · 1/13 shards · harness-scoped
83.3%
95% CI 43.697.0%
5/6 correct · 0.1% coverage
47
unsloth/gemma-4-26B-A4B-it-qat-GGUF · 1 run · 1/13 shards · harness-scoped
52.0%
95% CI 42.361.5%
52/100 correct · 1.0% coverage
48
Qwen3.8-27B-GGUFIQ1_M · GGUFunknown protocol · unknown agent
unsloth/Qwen3.8-27B-GGUF · 1 run · 1/13 shards · harness-scoped
47.5%
95% CI 40.754.4%
95/200 correct · 2.0% coverage
49
gemma-4-E4B-it-OBLITERATEDunspecified quantunknown protocol · unknown agent
OBLITERATUS/gemma-4-E4B-it-OBLITERATED · 1 run · 1/13 shards · harness-scoped
100.0%
95% CI 20.7100.0%
1/1 correct · 0.0% coverage
50
Laguna-XS-2.1-NVFP4NVFP4unknown protocol · unknown agent
poolside/Laguna-XS-2.1-NVFP4 · 1 run · 1/13 shards · harness-scoped
100.0%
95% CI 20.7100.0%
1/1 correct · 0.0% coverage
51
MiniCPM5-1B-GGUFunspecified quantunknown protocol · unknown agent
openbmb/MiniCPM5-1B-GGUF · 3 runs · 2/13 shards · harness-scoped
30.0%
95% CI 19.942.5%
18/60 correct · 0.6% coverage
52
LFM2.5-1.2B-Instruct-GGUFunspecified quantunknown protocol · unknown agent
LiquidAI/LFM2.5-1.2B-Instruct-GGUF · 3 runs · 3/13 shards · harness-scoped
26.7%
95% CI 18.636.6%
24/90 correct · 0.9% coverage
53
MiniCPM5-1Bunspecified quantunknown protocol · unknown agent
openbmb/MiniCPM5-1B · 3 runs · 3/13 shards · harness-scoped
18.6%
95% CI 11.828.1%
16/86 correct · 0.9% coverage
54
LFM2.5-8B-A1B-GGUFunspecified quantunknown protocol · unknown agent
unsloth/LFM2.5-8B-A1B-GGUF · 3 runs · 3/13 shards · harness-scoped
28.6%
95% CI 8.264.1%
2/7 correct · 0.1% coverage

Stability— historical rerun transparency

Leaderboard rank uses the canonical latest approved answer per question_id. These metrics include historical submissions too, so reruns and changed answers are visible but do not drive rank.

Qwen3.6-27B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
1 historical run · 773 unique questions
Canonical
93.3%
Row avg
93.3%
Run avg
93.3%
Repeated
0
Changed
0
Qwopus3.6-27B-Coder-MTP-GGUFQ5_K_M · ggufunknown protocol · unknown agent
1 historical run · 772 unique questions
Canonical
92.9%
Row avg
92.9%
Run avg
92.9%
Repeated
0
Changed
0
Qwen3.6-35B-A3B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
26 historical runs · 10,042 unique questions
Canonical
90.4%
Row avg
90.2%
Run avg
90.2%
Repeated
10,042
Changed
273
Qwen3.5-27B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
1 historical run · 772 unique questions
Canonical
91.8%
Row avg
91.8%
Run avg
91.8%
Repeated
0
Changed
0
Qwen3.6-35B-A3B-GGUFQ2_K_XL · GGUFunknown protocol · unknown agent
30 historical runs · 10,042 unique questions
Canonical
89.3%
Row avg
89.3%
Run avg
89.0%
Repeated
12,326
Changed
152
Qwen3.5-27B-Uncensored-HauhauCS-AggressiveQ4_K_M · GGUFunknown protocol · unknown agent
1 historical run · 772 unique questions
Canonical
90.7%
Row avg
90.7%
Run avg
90.7%
Repeated
0
Changed
0
Qwen3.8-27B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
1 historical run · 773 unique questions
Canonical
89.8%
Row avg
89.8%
Run avg
89.8%
Repeated
0
Changed
0
Qwen3.8-27B-GGUFQ6_K_XL · GGUFunknown protocol · unknown agent
1 historical run · 772 unique questions
Canonical
89.1%
Row avg
89.1%
Run avg
89.1%
Repeated
0
Changed
0
Qwen3.6-35B-A3B-Uncensored-HauhauCS-AggressiveQ2_K_P · GGUFunknown protocol · unknown agent
13 historical runs · 10,042 unique questions
Canonical
85.4%
Row avg
85.4%
Run avg
85.4%
Repeated
0
Changed
0
Qwen3.6-35B-A3B-Uncensored-HauhauCS-AggressiveQ4_K_M · GGUFunknown protocol · unknown agent
13 historical runs · 10,042 unique questions
Canonical
85.0%
Row avg
85.0%
Run avg
85.0%
Repeated
0
Changed
0
DeepSeek-V4-Flash-0731FP8unknown protocol · unknown agent
1 historical run · 773 unique questions
Canonical
86.7%
Row avg
86.7%
Run avg
86.7%
Repeated
0
Changed
0
gemma-4-31B-it-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
1 historical run · 773 unique questions
Canonical
86.0%
Row avg
86.0%
Run avg
86.0%
Repeated
0
Changed
0
GLM-5.3-Flashexl3unknown protocol · unknown agent
1 historical run · 772 unique questions
Canonical
86.0%
Row avg
86.0%
Run avg
86.0%
Repeated
0
Changed
0
GLM-5.3-Flash-tr3-4bpwexl3unknown protocol · unknown agent
1 historical run · 773 unique questions
Canonical
84.7%
Row avg
84.7%
Run avg
84.7%
Repeated
0
Changed
0
Qwen3.8-Flash-Next-NVFP4NVFP4unknown protocol · unknown agent
1 historical run · 772 unique questions
Canonical
84.7%
Row avg
84.7%
Run avg
84.7%
Repeated
0
Changed
0
Qwen3.8-27B-W4A16-AWQunspecified quantunknown protocol · unknown agent
1 historical run · 371 unique questions
Canonical
85.2%
Row avg
85.2%
Run avg
85.2%
Repeated
0
Changed
0
Qwen3.8-27B-GPTQ-W4A16unspecified quantunknown protocol · unknown agent
13 historical runs · 4,823 unique questions
Canonical
81.9%
Row avg
81.9%
Run avg
81.9%
Repeated
0
Changed
0
DeepSeek-V4-Flash-0731unspecified quantunknown protocol · unknown agent
1 historical run · 200 unique questions
Canonical
86.0%
Row avg
86.0%
Run avg
86.0%
Repeated
0
Changed
0
Gemma-4-31B-IT-NVFP4unspecified quantunknown protocol · unknown agent
12 historical runs · 4,452 unique questions
Canonical
80.9%
Row avg
80.9%
Run avg
80.9%
Repeated
0
Changed
0
KAT-Coder-V2.5-Dev-AWQ-W4A16-ASYMunspecified quantunknown protocol · unknown agent
1 historical run · 371 unique questions
Canonical
83.6%
Row avg
83.6%
Run avg
83.6%
Repeated
0
Changed
0
Qwen3.8-Flash-Next-NVFP4NVFP4unknown protocol · unknown agent
1 historical run · 773 unique questions
Canonical
82.1%
Row avg
82.1%
Run avg
82.1%
Repeated
0
Changed
0
Qwen3.8-27B-MTP-NVFP4unspecified quantunknown protocol · unknown agent
1 historical run · 371 unique questions
Canonical
83.3%
Row avg
83.3%
Run avg
83.3%
Repeated
0
Changed
0
Qwen3.6-35B-A3B-GPTQ-Int4unspecified quantunknown protocol · unknown agent
7 historical runs · 2,226 unique questions
Canonical
80.6%
Row avg
80.9%
Run avg
80.9%
Repeated
371
Changed
6
Qwen3.6-27B-NVFP4NVFP4unknown protocol · unknown agent
13 historical runs · 4,823 unique questions
Canonical
79.8%
Row avg
79.8%
Run avg
79.8%
Repeated
0
Changed
0
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4unspecified quantunknown protocol · unknown agent
1 historical run · 371 unique questions
Canonical
82.7%
Row avg
82.7%
Run avg
82.7%
Repeated
0
Changed
0
Muse-Glimmer-30B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
1 historical run · 772 unique questions
Canonical
80.8%
Row avg
80.8%
Run avg
80.8%
Repeated
0
Changed
0
Muse-Glimmer-30B-GGUFQ5_K_XL · GGUFunknown protocol · unknown agent
1 historical run · 771 unique questions
Canonical
80.3%
Row avg
80.3%
Run avg
80.3%
Repeated
0
Changed
0
Ornith-1.0-35B-PrismaAURA-4.75bit-vllm-MTPunspecified quantunknown protocol · unknown agent
2 historical runs · 742 unique questions
Canonical
80.2%
Row avg
80.2%
Run avg
80.2%
Repeated
0
Changed
0
gemma-4-26B-A4B-it-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
1 historical run · 772 unique questions
Canonical
79.5%
Row avg
79.5%
Run avg
79.5%
Repeated
0
Changed
0
Ornith-1.0-35B-AWQ-INT4unspecified quantunknown protocol · unknown agent
1 historical run · 371 unique questions
Canonical
80.6%
Row avg
80.6%
Run avg
80.6%
Repeated
0
Changed
0
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressiveunspecified quantunknown protocol · unknown agent
1 historical run · 772 unique questions
Canonical
79.1%
Row avg
79.1%
Run avg
79.1%
Repeated
0
Changed
0
Muse-Glimmer-30B-NVFP4unspecified quantunknown protocol · unknown agent
1 historical run · 371 unique questions
Canonical
79.5%
Row avg
79.5%
Run avg
79.5%
Repeated
0
Changed
0
Qwen3.6-27B-AEON-Ultimate-Uncensored-Text-NVFP4-MTP-XSunspecified quantunknown protocol · unknown agent
1 historical run · 371 unique questions
Canonical
79.0%
Row avg
79.0%
Run avg
79.0%
Repeated
0
Changed
0
Ornith-1.5-35B-A3B-NVFP4unspecified quantunknown protocol · unknown agent
1 historical run · 371 unique questions
Canonical
79.0%
Row avg
79.0%
Run avg
79.0%
Repeated
0
Changed
0
Ornith-1.0-9Bunspecified quantunknown protocol · unknown agent
2 historical runs · 44 unique questions
Canonical
84.1%
Row avg
84.1%
Run avg
86.4%
Repeated
0
Changed
0
Laguna-XS-2.1-INT4INT4unknown protocol · unknown agent
1 historical run · 371 unique questions
Canonical
73.6%
Row avg
73.6%
Run avg
73.6%
Repeated
0
Changed
0
Hy-MT2-7Bunspecified quantunknown protocol · unknown agent
4 historical runs · 400 unique questions
Canonical
65.0%
Row avg
65.0%
Run avg
58.7%
Repeated
0
Changed
0
gemma-4-E4B-itunspecified quantunknown protocol · unknown agent
3 historical runs · 297 unique questions
Canonical
65.3%
Row avg
65.3%
Run avg
65.1%
Repeated
0
Changed
0
granite-4.0-h-microunspecified quantunknown protocol · unknown agent
1 historical run · 30 unique questions
Canonical
76.7%
Row avg
76.7%
Run avg
76.7%
Repeated
0
Changed
0
EXAONE-3.5-7.8B-Instruct-GGUFunspecified quantunknown protocol · unknown agent
1 historical run · 30 unique questions
Canonical
73.3%
Row avg
73.3%
Run avg
73.3%
Repeated
0
Changed
0
gpt-oss-20b-GGUFMXFP4 · GGUFunknown protocol · unknown agent
8 historical runs · 2,968 unique questions
Canonical
51.4%
Row avg
51.4%
Run avg
51.4%
Repeated
0
Changed
0
RWKV7-Goose-World3-2.9B-HFunspecified quantunknown protocol · unknown agent
2 historical runs · 60 unique questions
Canonical
61.7%
Row avg
61.7%
Run avg
61.7%
Repeated
0
Changed
0
Mellum2-12B-A2.5B-Instruct-GGUF-Q4_K_Munspecified quantunknown protocol · unknown agent
3 historical runs · 15 unique questions
Canonical
73.3%
Row avg
73.3%
Run avg
77.4%
Repeated
0
Changed
0
MiniCPM3-4Bunspecified quantunknown protocol · unknown agent
4 historical runs · 219 unique questions
Canonical
52.1%
Row avg
52.1%
Run avg
75.3%
Repeated
0
Changed
0
gemma-4-12b-it-GGUFQ8_K_XL · gguf
0 historical runs · 100 unique questions
Canonical
54.0%
Row avg
Run avg
Repeated
0
Changed
0
gemma-4-E4B-it-qat-q4_0-ggufunspecified quantunknown protocol · unknown agent
1 historical run · 6 unique questions
Canonical
83.3%
Row avg
83.3%
Run avg
83.3%
Repeated
0
Changed
0
0 historical runs · 100 unique questions
Canonical
52.0%
Row avg
Run avg
Repeated
0
Changed
0
Qwen3.8-27B-GGUFIQ1_M · GGUFunknown protocol · unknown agent
1 historical run · 200 unique questions
Canonical
47.5%
Row avg
47.5%
Run avg
47.5%
Repeated
0
Changed
0
gemma-4-E4B-it-OBLITERATEDunspecified quantunknown protocol · unknown agent
1 historical run · 1 unique questions
Canonical
100.0%
Row avg
100.0%
Run avg
100.0%
Repeated
0
Changed
0
Laguna-XS-2.1-NVFP4NVFP4unknown protocol · unknown agent
1 historical run · 1 unique questions
Canonical
100.0%
Row avg
100.0%
Run avg
100.0%
Repeated
0
Changed
0
MiniCPM5-1B-GGUFunspecified quantunknown protocol · unknown agent
3 historical runs · 60 unique questions
Canonical
30.0%
Row avg
30.0%
Run avg
30.0%
Repeated
30
Changed
0
LFM2.5-1.2B-Instruct-GGUFunspecified quantunknown protocol · unknown agent
3 historical runs · 90 unique questions
Canonical
26.7%
Row avg
26.7%
Run avg
26.7%
Repeated
0
Changed
0
MiniCPM5-1Bunspecified quantunknown protocol · unknown agent
3 historical runs · 86 unique questions
Canonical
18.6%
Row avg
18.6%
Run avg
22.2%
Repeated
0
Changed
0
LFM2.5-8B-A1B-GGUFunspecified quantunknown protocol · unknown agent
3 historical runs · 7 unique questions
Canonical
28.6%
Row avg
28.6%
Run avg
22.2%
Repeated
0
Changed
0

Runs— sample traces per run

Qwen3.8-Flash-Next-NVFP4NVFP4unknown protocol · unknown agent
by Per0 · shard 12 · 9/5/2026, 10:44:40 PM · cmtoz069001lwn701989021z6
84.7%
654/772 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 1 · hellaswag:00007797pass · 100.0% · 353ms · ca10ca786b1b
Question
[header] How to save money renting a car in the united states [title] Check online travel sites. [step] You can compare car rental prices online through most travel aggregators, such as orbitz, priceline, expedia, travelocity, and kayak. Car rental prices can fluctuate wildly, so check multiple sites regularly, starting at least several weeks before you need the car.

A. [substeps] If you combine your car rental with a flight and/or hotel booking, you can often save significantly more than by reserving them separately. [title] Compare prices directly from the major company websites.
B. [substeps] Paying online usually involves submitting a bill to the online car rental service. Check car websites and discuss customizable payment plans before finalizing the transaction.
C. [title] Sign up with a car sharing company. [step] Certain companies such as airtrain, lyft, lambordia, or banal are also carriers for car rental companies.
D. [title] Add up your driving and parking costs. [step] Be prepared to get a loan or borrow money from a friend or family member.
Rendered prompt
[header] How to save money renting a car in the united states [title] Check online travel sites. [step] You can compare car rental prices online through most travel aggregators, such as orbitz, priceline, expedia, travelocity, and kayak. Car rental prices can fluctuate wildly, so check multiple sites regularly, starting at least several weeks before you need the car.
Agent trace
A
Extracted: A · Gold: A
sample 2 · hellaswag:00002368pass · 100.0% · 1441ms · e64052a5f5d3
Question
A woman is shown speaking to the camera while holding a dart. pictures of a man

A. 's arms are shown as well as this dart.
B. with a bowflex in one ear and a dart in her other ear next to a dart board in a room.
C. playing darts appear and he is shaking in distress before returning to talking to the camera.
D. are shown and leads into her throwing darts.
Rendered prompt
A woman is shown speaking to the camera while holding a dart. pictures of a man
Agent trace
D
Extracted: D · Gold: D
sample 3 · hellaswag:00003525pass · 100.0% · 682ms · 2637f8600bc9
Question
[header] How to wear a bomber jacket [title] Consider fit. [step] Because they started as a utilitarian piece for servicemen strapped with equipment, they're naturally oversized. However, designers haven't been afraid to play around with the bomber jacket design.

A. You've got tons of options-from thick american blazers to ¾-1-come-out bomber jackets. You just need to try them on and see if one fits you perfectly.
B. In fact, they've made many custom bomber jackets fit for military situations in the past. [substeps] Bomber jackets are wide-brimmed, functional, and tend to be less oversized.
C. You can wear them as an oversized piece, but you can also find them in slim fitting versions, cropped versions, and so on. [substeps] Getting the proper fit is key to rocking this look.
D. Try a simple button-down and it can work great. You can use a plain button-down instead if you want to wear some seasonal " steampunk " materials.
Rendered prompt
[header] How to wear a bomber jacket [title] Consider fit. [step] Because they started as a utilitarian piece for servicemen strapped with equipment, they're naturally oversized. However, designers haven't been afraid to play around with the bomber jacket design.
Agent trace
C
Extracted: C · Gold: C
sample 5 · hellaswag:00003679pass · 100.0% · 1442ms · 1d6c41ac9865
Question
[header] How to clean hairbrushes [title] Remove the hair from the brush. [step] Taking out the old hair is always a good idea, since dirt and skin particles tend to stick to it as it builds up in your brush. Reach in and pinch the hair with your fingers, then pull it out from around the bristles.

A. [substeps] Brushes that are attached to anything (such as the brush) will be clean and free of dirt and bacteria, but brushes with metal can damage them and produce residue. [title] Choose one that is easy to clean.
B. [substeps] These small pinches are usually easy to remove or you can simply take them out. Removing used hair from an old hairbrush, just may take some effort to get it off of the handle.
C. Keep doing this until you've extracted all of the hair from the brush, then throw the hair away. [substeps] If your brush's bristles are made from natural fibers, take care not to yank them out with the hair.
D. [substeps] Most brushes have a teeth crown screw or other attachment. If not, put the brush in to remove the crown.
Rendered prompt
[header] How to clean hairbrushes [title] Remove the hair from the brush. [step] Taking out the old hair is always a good idea, since dirt and skin particles tend to stick to it as it builds up in your brush. Reach in and pinch the hair with your fingers, then pull it out from around the bristles.
Agent trace
C
Extracted: C · Gold: C
sample 6 · hellaswag:00002729pass · 100.0% · 682ms · 25a7613dfe9e
Question
A clip begins and it features people running and the words on the screen say they are novices who are training to run a marathon in tel aviv, israel. a man

A. is seen speaking in a temple.
B. is then shown running and the words say can be seen on the screen again saying he has some good news.
C. appears training them and different marathoners in training take turns being interviewed while they continue to switch back and forth to everyone going through the training.
D. is seen standing in front of a street and he begins to perform an act while holding the statue and flipping back and fourth.
Rendered prompt
A clip begins and it features people running and the words on the screen say they are novices who are training to run a marathon in tel aviv, israel. a man
Agent trace
C
Extracted: C · Gold: C
Incorrect samples
sample 4 · hellaswag:00001285fail · 0.0% · 1103ms · 30d7069fbc34
Question
The players stand around talking to one another and one shoots a bow and arrow at a target. more people

A. watch while the owners wait.
B. are seen shooting the bows followed by interviewers speaking and team mates reacting.
C. walk by and the players talk to each other in the background.
D. come and go behind the referee who has just come on stage to take picture and record the game.
Rendered prompt
The players stand around talking to one another and one shoots a bow and arrow at a target. more people
Agent trace
C
Extracted: C · Gold: B
sample 9 · hellaswag:00000102fail · 0.0% · 1440ms · 5a11413033d2
Question
A man is in the gym in tight he bends over picks up a weight over his head and drops it back down. he

A. lifts his legs and pulls the bar up on his body and lifts the weight above his head.
B. lifts it again lifts it over his head and drops it this time and he loosens his way to the top and grabs it again.
C. raises the bar over his head followed by another.
D. walks back and loosens up before walking back up and doing it again adding more weight.
Rendered prompt
A man is in the gym in tight he bends over picks up a weight over his head and drops it back down. he
Agent trace
A
Extracted: A · Gold: D
sample 11 · hellaswag:00001467fail · 0.0% · 1382ms · 5b7790382de9
Question
A girl runs down a track in slow motion. she

A. is throwing a javelin over the bar.
B. smashes into a sand pit several times.
C. takes a huge jump, hurling herself over a bar.
D. jumps into a sand box and begins to play a game of hopscotch.
Rendered prompt
A girl runs down a track in slow motion. she
Agent trace
D
Extracted: D · Gold: C
sample 15 · hellaswag:00001181fail · 0.0% · 1365ms · b54501e55364
Question
This woman rides the white/brown horse under the shady trees for a little bit. the horse

A. is nodding its head and walks in circles a few times.
B. follows the woman under the oaks.
C. walks beside the woman letting her go for a little while.
D. comes to a stop and then the woman with the red tack begins to brush the horse's mane.
Rendered prompt
This woman rides the white/brown horse under the shady trees for a little bit. the horse
Agent trace
D
Extracted: D · Gold: A
sample 22 · hellaswag:00007915fail · 0.0% · 1345ms · 342258f10ed1
Question
[header] How to install chain link fence [title] Obtain any necessary permits. [step] Your local government may have building and zoning regulations that regulate fence setbacks, type, and height. If you install a fence without a permit, they may tear down the structure.

A. [title] Establish where your property lines are located. [step] This information can be obtained from city records, a realtor's line plot map, or by hiring a surveyor.
B. [substeps] If you get a construction permit, then you cannot attempt to install a fence until the building, zoning, and development process is underway. Even if there are no zoning restrictions, this can result from fines and/or foreclosure.
C. [title] Visit a website that will allow you to buy the type of fence you want to install. [step] You will need to find a particularly attractive and well-maintained site to start your chain link fence.
D. The required permits will differ from state to state. [substeps] Visit the national park service website to find out which permits are required in your state.
Rendered prompt
[header] How to install chain link fence [title] Obtain any necessary permits. [step] Your local government may have building and zoning regulations that regulate fence setbacks, type, and height. If you install a fence without a permit, they may tear down the structure.
Agent trace
D
Extracted: D · Gold: A
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4unspecified quantunknown protocol · unknown agent
by SgtPooki · shard 8 · 9/5/2026, 3:47:56 AM · cmtnuebfb027lmm014gn9n4ue
82.7%
307/371 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 1 · hellaswag:00008131pass · 100.0% · 494ms · a4ac87a8acc9
Question
[header] How to flex lats [title] Learn how to isolate your lats when flexing. [step] Your lats are a pair of wide muscles that run from your armpits to your lower spine, wrapping around the back of your ribcage just beneath your shoulder blades. To figure out how to isolate your lats, first stand with your feet together.

A. Your toes should be together, with your butt facing downward. Close your abdominal muscles in the groin to get into this position.
B. Hold your ankles firmly in your right hand and your ankles together with your left hand with your hands overlapped, in a criss cross pattern of parallel and vertical lines. [title] Use a ruler to identify your lats.
C. Squat forward so that your back and front ribs are aligned and your toes are parallel with the floor. Raise your elbows at an angle and bend your elbow at 90 degrees.
D. Arch your lower back so that your chest is pushed forward and your butt is pushed out. Spread your upper back by lifting your arms slightly away from your sides and holding your hands about 6 inches (15 cm) in front of your hips.
Rendered prompt
[header] How to flex lats [title] Learn how to isolate your lats when flexing. [step] Your lats are a pair of wide muscles that run from your armpits to your lower spine, wrapping around the back of your ribcage just beneath your shoulder blades. To figure out how to isolate your lats, first stand with your feet together.
Agent trace
D
Extracted: D · Gold: D
sample 3 · hellaswag:00009876pass · 100.0% · 150ms · 019b1852fb0c
Question
[header] How to show cleavage [title] Wear tops that show off your cleavage. [step] While this may seem like a given, the right top is absolutely essential when showing cleavage. You can't show something that's covered up.

A. Also, make sure your tops are not g-rated, meaning those aren't appropriate for a particular time of day, or if you have an event coming up soon. [substeps] If you love showing cleavage, you want to show off that.
B. Make sure you're comfortable with your body in the top so you can show it up. [substeps] Studies show that flat strapless tops are considered the sexiest choice for showing cleavage.
C. While a turtleneck sweater may make your bust look fantastic, it will hide your cleavage. Make sure what you're wearing is cut for visible cleavage.
D. Ask a friend if she'd be okay with a top showing their cleavage, and if she says yes! [title] Go for a sleek cut. [step] The summer and winter months are the ideal time to show your cleavage.
Rendered prompt
[header] How to show cleavage [title] Wear tops that show off your cleavage. [step] While this may seem like a given, the right top is absolutely essential when showing cleavage. You can't show something that's covered up.
Agent trace
C
Extracted: C · Gold: C
sample 4 · hellaswag:00003669pass · 100.0% · 141ms · a1faa682bdfb
Question
[header] How to enjoy your birthday [title] If you like planning, then plan how you'll spend the day at least a week ahead. [step] Check that your friends and family are free for the day and make sure you choose things to do that can involve the people you'd like to spend time with. [title] Get plenty of sleep.

A. [step] A really good time to get a full night's sleep is about 7 am; the rest of the night should be uninterrupted. It is also good to unwind a bit in the evening and can be done so that you can think about each day.
B. [step] And make sure to get 7-9 hours of sleep a night-1 1/2 hours for adults and 4-9 hours for children. You can even make it up to four hours of sleep on weekends.
C. [step] The day before your birthday, go to bed at a reasonable hour so that you can get the sleep needed to guarantee that you'll have lots of energy on your birthday. [title] Treat yourself to an awesome birthday outfit.
D. [step] The best things you can do for a birthday is to get a good quality sleep. Your body clock is only going to be able to tell you the exact amount of sleep you need.
Rendered prompt
[header] How to enjoy your birthday [title] If you like planning, then plan how you'll spend the day at least a week ahead. [step] Check that your friends and family are free for the day and make sure you choose things to do that can involve the people you'd like to spend time with. [title] Get plenty of sleep.
Agent trace
C
Extracted: C · Gold: C
sample 5 · hellaswag:00005144pass · 100.0% · 112ms · 73c837dec8a2
Question
[header] How to succeed in distance learning [title] Familiarize yourself with the syllabus. [step] Think of your syllabus as the blueprint for your class. It should contain all of the class guidelines and rules, as well as what your instructor expects from you over the course of the semester.

A. If you're having trouble understanding all of the syllabus instructions, ask your instructor for advice. [substeps] To become a successful athlete, you have to have a strong understanding of the game and field, and then, with that knowledge, bring out your best on a single outing.
B. Include these with your notes, papers, assignments and applications so you'll be up to date and able to fully experience your classroom. [substeps] Do a quick background check on your syllabus for your instructor.
C. Bring this information to the syllabus, stating all required daily requirements, as well as any tests you may have scheduled or deadlines you must meet for your session. This way, you can easily adjust any assignments you may finish before the next session starts.
D. Make sure you read through this document in its entirety and keep track of it all semester. [substeps] Ask your instructor any questions you might have about the syllabus.
Rendered prompt
[header] How to succeed in distance learning [title] Familiarize yourself with the syllabus. [step] Think of your syllabus as the blueprint for your class. It should contain all of the class guidelines and rules, as well as what your instructor expects from you over the course of the semester.
Agent trace
D
Extracted: D · Gold: D
sample 6 · hellaswag:00005768pass · 100.0% · 182ms · a8fb349bdd6f
Question
[header] How to make spicy chicken bites [title] Preheat the oven to 450º fahrenheit (230º celsius). [title] Line a large baking sheet with parchment paper. [title] Combine the chicken bites with the garlic and paprika.

A. [title] Season the chicken bites with salt and freshly ground black pepper. [title] Stir together the chickpeas, seasoning with nutmeg, currants, cumin and dried ginger in a mixing bowl.
B. [step] Toss the ingredients together in a large bowl until fully coated. [title] Toss in the flour and fold together until well-coated.
C. [title] Stir in ground beef and chicken bouillon and stir. [title] Add the oil and cook it for 2 minutes.
D. [title] Spread the chicken bites over the coated baking sheet. [title] Bake until they've cooked.
Rendered prompt
[header] How to make spicy chicken bites [title] Preheat the oven to 450º fahrenheit (230º celsius). [title] Line a large baking sheet with parchment paper. [title] Combine the chicken bites with the garlic and paprika.
Agent trace
B
Extracted: B · Gold: B
Incorrect samples
sample 2 · hellaswag:00009130fail · 0.0% · 98ms · 493709792d71
Question
[header] How to apply makeup for a glamour photography shoot [title] Clean your face. [step] Always start with a clean face. Leave no traces of your previous makeup.

A. If you will use a pore-cleaning strip, do it the day before to avoid makeup get caught in your newly cleansed pores. Exfoliate to feel that silky smooth texture of your skin.
B. Do not blush, define, or dry dry your face before applying eyeliner in this photoshoot. [title] Apply foundation and concealer to your face.
C. Inspect your face for blemishes with a makeup primer or something like that. Use a gentle cleanser and carefully pat your face dry with a soft soft cloth.
D. A healthy glow has been built up in your face over the past day. In order to achieve the glow, begin by exfoliating your face with a gentle face cleanser.
Rendered prompt
[header] How to apply makeup for a glamour photography shoot [title] Clean your face. [step] Always start with a clean face. Leave no traces of your previous makeup.
Agent trace
C
Extracted: C · Gold: A
sample 10 · hellaswag:00004482fail · 0.0% · 94ms · 37232b0902b6
Question
[header] How to annoy your roommate [title] Leave trash around the room. [step] First of all, this will make your room smell bad. Secondly, it'll attract flies, ants, and other bugs, which will gross your roommate out.

A. She will now be forced to clean up your mess or will leave it there, and it will annoy her. [substeps] Avoid leaving anything you might also find gross.
B. Leave trash around the room until it's gone! Remind your roommate to check their room frequently for trash. Finally, move a few boxes that are not worth moving around, from your room all the way to another part of the house to make room for trash.
C. If you're really into it, leave garbage around your walls and furniture and knock around some boxes. [substeps] Alternatively, if you're in a particularly bad mood, you could hang out with a friend who's unlikely to find trash, like a teddy bear.
D. No one likes to be around these things anyway, so if you act like an annoyance right from the start, people will start to go around and pick up a lot of clutter. [substeps] Take a small trash can and fill it full of personal trash, such as toilet paper, empty paint cans, and trash bags.
Rendered prompt
[header] How to annoy your roommate [title] Leave trash around the room. [step] First of all, this will make your room smell bad. Secondly, it'll attract flies, ants, and other bugs, which will gross your roommate out.
Agent trace
B
Extracted: B · Gold: A
sample 13 · hellaswag:00008048fail · 0.0% · 177ms · c074de6e1eea
Question
[header] How to be a strict mom [title] Make it clear what the rules are. [step] Do not expect your children to read your mind. When you set rules, explain them to the entire household.

A. Enforce your limits and rules in certain ways. For example, if your child always goes to school without adult supervision, make it clear that this behavior happens every day and is to be expected.
B. For example, if there are four children in the house, explain that children should only have access to the least amount of supervision. If children are sick or injured, they should be allowed in a daycare.
C. You can even post certain rules on the fridge. This will help your children understand how to act in the house and why you are disciplining them when they get in trouble.
D. Ask your children to follow the rules one way and then follow them the other way. Make sure that everyone understands that there are consequences when the rules are ignored.
Rendered prompt
[header] How to be a strict mom [title] Make it clear what the rules are. [step] Do not expect your children to read your mind. When you set rules, explain them to the entire household.
Agent trace
D
Extracted: D · Gold: C
sample 29 · hellaswag:00006369fail · 0.0% · 92ms · 0ec2a9504534
Question
[header] How to cook chuck steak [title] Preheat the oven and season the steak. [step] Turn the oven on to 325 degrees f (162 c). Place 2 tablespoons of vegetable or canola oil in a large pot or dutch oven.

A. Place the pot on the stove and turn the heat to medium high. Stir the ingredients constantly to ensure that they're not clumped together in the pot and that they cook evenly.
B. Turn on the burner to the highest setting possible and toss in the potatoes and onions. [substeps] To save time, put the pan upside down over a medium heat.
C. Let the oil heat up for about 15 minutes so that it only coats the steak. [substeps] The oil should be hot, but not overly hot.
D. Heat the oil over medium heat and sprinkle the chuck steak with salt and pepper. [substeps] You could use a big skillet if your chuck steaks are thin.
Rendered prompt
[header] How to cook chuck steak [title] Preheat the oven and season the steak. [step] Turn the oven on to 325 degrees f (162 c). Place 2 tablespoons of vegetable or canola oil in a large pot or dutch oven.
Agent trace
A
Extracted: A · Gold: D
sample 41 · hellaswag:00009889fail · 0.0% · 132ms · 5cba7c64cdb3
Question
[header] How to detangle hair without pain [title] Check for dryness. [step] Before you attempt to take out any tangles, make sure your hair is not severely dry. This will make it less painful when you are actually taking the knots out.

A. Run your fingers through your hair to see if it's slightly sticky to the touch. If it is, try rubbing some old hair conditioner into it.
B. Run your fingers through your hair to ensure that it is not too damp and that it is looking sleek. [substeps] Normal hair can feel rough or lifeless if it is over-dry.
C. [title] Separate the hair completely if you have it tied up and leave it free. [step] Attempt to free any loose hairs from a tangled clump.
D. Wet hair tends to dry very quickly and can cause less hair damage. [substeps] If the oil in your hair helps to detangle it while you're detangling it, hold it over a sink or tub and then gently pull the roots and tips of your hair downward to remove as much oil as possible.
Rendered prompt
[header] How to detangle hair without pain [title] Check for dryness. [step] Before you attempt to take out any tangles, make sure your hair is not severely dry. This will make it less painful when you are actually taking the knots out.
Agent trace
A
Extracted: A · Gold: C
Muse-Glimmer-30B-NVFP4unspecified quantunknown protocol · unknown agent
by SgtPooki · shard 8 · 9/5/2026, 3:38:19 AM · cmtnu1yms01temm01f8f4gwph
79.5%
295/371 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 1 · hellaswag:00008131pass · 100.0% · 390ms · a4ac87a8acc9
Question
[header] How to flex lats [title] Learn how to isolate your lats when flexing. [step] Your lats are a pair of wide muscles that run from your armpits to your lower spine, wrapping around the back of your ribcage just beneath your shoulder blades. To figure out how to isolate your lats, first stand with your feet together.

A. Your toes should be together, with your butt facing downward. Close your abdominal muscles in the groin to get into this position.
B. Hold your ankles firmly in your right hand and your ankles together with your left hand with your hands overlapped, in a criss cross pattern of parallel and vertical lines. [title] Use a ruler to identify your lats.
C. Squat forward so that your back and front ribs are aligned and your toes are parallel with the floor. Raise your elbows at an angle and bend your elbow at 90 degrees.
D. Arch your lower back so that your chest is pushed forward and your butt is pushed out. Spread your upper back by lifting your arms slightly away from your sides and holding your hands about 6 inches (15 cm) in front of your hips.
Rendered prompt
[header] How to flex lats [title] Learn how to isolate your lats when flexing. [step] Your lats are a pair of wide muscles that run from your armpits to your lower spine, wrapping around the back of your ribcage just beneath your shoulder blades. To figure out how to isolate your lats, first stand with your feet together.
Agent trace
D
Extracted: D · Gold: D
sample 3 · hellaswag:00009876pass · 100.0% · 50ms · 019b1852fb0c
Question
[header] How to show cleavage [title] Wear tops that show off your cleavage. [step] While this may seem like a given, the right top is absolutely essential when showing cleavage. You can't show something that's covered up.

A. Also, make sure your tops are not g-rated, meaning those aren't appropriate for a particular time of day, or if you have an event coming up soon. [substeps] If you love showing cleavage, you want to show off that.
B. Make sure you're comfortable with your body in the top so you can show it up. [substeps] Studies show that flat strapless tops are considered the sexiest choice for showing cleavage.
C. While a turtleneck sweater may make your bust look fantastic, it will hide your cleavage. Make sure what you're wearing is cut for visible cleavage.
D. Ask a friend if she'd be okay with a top showing their cleavage, and if she says yes! [title] Go for a sleek cut. [step] The summer and winter months are the ideal time to show your cleavage.
Rendered prompt
[header] How to show cleavage [title] Wear tops that show off your cleavage. [step] While this may seem like a given, the right top is absolutely essential when showing cleavage. You can't show something that's covered up.
Agent trace
C
Extracted: C · Gold: C
sample 4 · hellaswag:00003669pass · 100.0% · 55ms · a1faa682bdfb
Question
[header] How to enjoy your birthday [title] If you like planning, then plan how you'll spend the day at least a week ahead. [step] Check that your friends and family are free for the day and make sure you choose things to do that can involve the people you'd like to spend time with. [title] Get plenty of sleep.

A. [step] A really good time to get a full night's sleep is about 7 am; the rest of the night should be uninterrupted. It is also good to unwind a bit in the evening and can be done so that you can think about each day.
B. [step] And make sure to get 7-9 hours of sleep a night-1 1/2 hours for adults and 4-9 hours for children. You can even make it up to four hours of sleep on weekends.
C. [step] The day before your birthday, go to bed at a reasonable hour so that you can get the sleep needed to guarantee that you'll have lots of energy on your birthday. [title] Treat yourself to an awesome birthday outfit.
D. [step] The best things you can do for a birthday is to get a good quality sleep. Your body clock is only going to be able to tell you the exact amount of sleep you need.
Rendered prompt
[header] How to enjoy your birthday [title] If you like planning, then plan how you'll spend the day at least a week ahead. [step] Check that your friends and family are free for the day and make sure you choose things to do that can involve the people you'd like to spend time with. [title] Get plenty of sleep.
Agent trace
C
Extracted: C · Gold: C
sample 5 · hellaswag:00005144pass · 100.0% · 53ms · 73c837dec8a2
Question
[header] How to succeed in distance learning [title] Familiarize yourself with the syllabus. [step] Think of your syllabus as the blueprint for your class. It should contain all of the class guidelines and rules, as well as what your instructor expects from you over the course of the semester.

A. If you're having trouble understanding all of the syllabus instructions, ask your instructor for advice. [substeps] To become a successful athlete, you have to have a strong understanding of the game and field, and then, with that knowledge, bring out your best on a single outing.
B. Include these with your notes, papers, assignments and applications so you'll be up to date and able to fully experience your classroom. [substeps] Do a quick background check on your syllabus for your instructor.
C. Bring this information to the syllabus, stating all required daily requirements, as well as any tests you may have scheduled or deadlines you must meet for your session. This way, you can easily adjust any assignments you may finish before the next session starts.
D. Make sure you read through this document in its entirety and keep track of it all semester. [substeps] Ask your instructor any questions you might have about the syllabus.
Rendered prompt
[header] How to succeed in distance learning [title] Familiarize yourself with the syllabus. [step] Think of your syllabus as the blueprint for your class. It should contain all of the class guidelines and rules, as well as what your instructor expects from you over the course of the semester.
Agent trace
D
Extracted: D · Gold: D
sample 7 · hellaswag:00006318pass · 100.0% · 54ms · 8562f92e036c
Question
[header] How to get other guys to stop staring at your pretty wife [title] Get your wife's opinion. [step] Value how your wife feels about the situation and get an understanding of how if affects her. If she is bothered by the stares, discuss whether or not you should intervene on her behalf.

A. If you feel the stares are a direct threat, try to think about your wife's feelings. Most of all, give her the benefit of the doubt-no guy wants to get a first-degree stare.
B. If she says " i don't feel like i need your attention right now, " then you can bring it up with her and let her ask her opinions. [substeps] Try to use humor to break the ice.
C. [substeps] If you start to think her boyfriend is doing this, report it back to him. If he is only doing this to steal her attention, admit that you feel sorry for him.
D. [substeps] If you have addressed this issue in the past, then you may already know how your wife feels about the attention that she receives. If this is the case, make a decision in advance on how you will respond so that you do not overreact.
Rendered prompt
[header] How to get other guys to stop staring at your pretty wife [title] Get your wife's opinion. [step] Value how your wife feels about the situation and get an understanding of how if affects her. If she is bothered by the stares, discuss whether or not you should intervene on her behalf.
Agent trace
D
Extracted: D · Gold: D
Incorrect samples
sample 2 · hellaswag:00009130fail · 0.0% · 80ms · 493709792d71
Question
[header] How to apply makeup for a glamour photography shoot [title] Clean your face. [step] Always start with a clean face. Leave no traces of your previous makeup.

A. If you will use a pore-cleaning strip, do it the day before to avoid makeup get caught in your newly cleansed pores. Exfoliate to feel that silky smooth texture of your skin.
B. Do not blush, define, or dry dry your face before applying eyeliner in this photoshoot. [title] Apply foundation and concealer to your face.
C. Inspect your face for blemishes with a makeup primer or something like that. Use a gentle cleanser and carefully pat your face dry with a soft soft cloth.
D. A healthy glow has been built up in your face over the past day. In order to achieve the glow, begin by exfoliating your face with a gentle face cleanser.
Rendered prompt
[header] How to apply makeup for a glamour photography shoot [title] Clean your face. [step] Always start with a clean face. Leave no traces of your previous makeup.
Agent trace
C
Extracted: C · Gold: A
sample 6 · hellaswag:00005768fail · 0.0% · 47ms · a8fb349bdd6f
Question
[header] How to make spicy chicken bites [title] Preheat the oven to 450º fahrenheit (230º celsius). [title] Line a large baking sheet with parchment paper. [title] Combine the chicken bites with the garlic and paprika.

A. [title] Season the chicken bites with salt and freshly ground black pepper. [title] Stir together the chickpeas, seasoning with nutmeg, currants, cumin and dried ginger in a mixing bowl.
B. [step] Toss the ingredients together in a large bowl until fully coated. [title] Toss in the flour and fold together until well-coated.
C. [title] Stir in ground beef and chicken bouillon and stir. [title] Add the oil and cook it for 2 minutes.
D. [title] Spread the chicken bites over the coated baking sheet. [title] Bake until they've cooked.
Rendered prompt
[header] How to make spicy chicken bites [title] Preheat the oven to 450º fahrenheit (230º celsius). [title] Line a large baking sheet with parchment paper. [title] Combine the chicken bites with the garlic and paprika.
Agent trace
D
Extracted: D · Gold: B
sample 10 · hellaswag:00004482fail · 0.0% · 52ms · 37232b0902b6
Question
[header] How to annoy your roommate [title] Leave trash around the room. [step] First of all, this will make your room smell bad. Secondly, it'll attract flies, ants, and other bugs, which will gross your roommate out.

A. She will now be forced to clean up your mess or will leave it there, and it will annoy her. [substeps] Avoid leaving anything you might also find gross.
B. Leave trash around the room until it's gone! Remind your roommate to check their room frequently for trash. Finally, move a few boxes that are not worth moving around, from your room all the way to another part of the house to make room for trash.
C. If you're really into it, leave garbage around your walls and furniture and knock around some boxes. [substeps] Alternatively, if you're in a particularly bad mood, you could hang out with a friend who's unlikely to find trash, like a teddy bear.
D. No one likes to be around these things anyway, so if you act like an annoyance right from the start, people will start to go around and pick up a lot of clutter. [substeps] Take a small trash can and fill it full of personal trash, such as toilet paper, empty paint cans, and trash bags.
Rendered prompt
[header] How to annoy your roommate [title] Leave trash around the room. [step] First of all, this will make your room smell bad. Secondly, it'll attract flies, ants, and other bugs, which will gross your roommate out.
Agent trace
B
Extracted: B · Gold: A
sample 14 · hellaswag:00002838fail · 0.0% · 40ms · 3da7e04394f6
Question
This elderly man is blowing the leaves out of the way using a leaf blower. first the camera

A. zooms out slightly and we see he has a child wearing gloves.
B. is giving viewers a close up view only showing the man's face.
C. runs like a compass and then it cuts to a car and then the man blowing the leaves.
D. pans to a man sitting in the dirt close to the wind blowing the leaves in a breeze.
Rendered prompt
This elderly man is blowing the leaves out of the way using a leaf blower. first the camera
Agent trace
D
Extracted: D · Gold: B
sample 29 · hellaswag:00006369fail · 0.0% · 79ms · 0ec2a9504534
Question
[header] How to cook chuck steak [title] Preheat the oven and season the steak. [step] Turn the oven on to 325 degrees f (162 c). Place 2 tablespoons of vegetable or canola oil in a large pot or dutch oven.

A. Place the pot on the stove and turn the heat to medium high. Stir the ingredients constantly to ensure that they're not clumped together in the pot and that they cook evenly.
B. Turn on the burner to the highest setting possible and toss in the potatoes and onions. [substeps] To save time, put the pan upside down over a medium heat.
C. Let the oil heat up for about 15 minutes so that it only coats the steak. [substeps] The oil should be hot, but not overly hot.
D. Heat the oil over medium heat and sprinkle the chuck steak with salt and pepper. [substeps] You could use a big skillet if your chuck steaks are thin.
Rendered prompt
[header] How to cook chuck steak [title] Preheat the oven and season the steak. [step] Turn the oven on to 325 degrees f (162 c). Place 2 tablespoons of vegetable or canola oil in a large pot or dutch oven.
Agent trace
A
Extracted: A · Gold: D
DeepSeek-V4-Flash-0731FP8unknown protocol · unknown agent
by Per0 · shard 11 · 9/4/2026, 3:49:53 AM · cmtmf0yyz017wl601mbj4j6i2
86.7%
670/773 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 1 · hellaswag:00009544pass · 100.0% · 649ms · ed7d3a94464d
Question
[header] How to take action against capital punishment in your state [title] Focus your action on a specific case or cases. [step] It may help you to focus your efforts on a couple of current capital cases in your state. Focusing your work on a couple of specific cases may be more effective than tackling several cases or state laws as a whole.

A. However, you should also stay on top of general issues related to capital punishment. Combining this knowledge can help you become a very informed participant in the action against these cases, which in turn may help alter capital punishment laws.
B. Some of the cases in your state are relatively minor compared to other cases in your state. Still, you should aim to focus your efforts on a very specific case at all costs.
C. [substeps] You can look up places to locate a case. There is a wide variety of legal resources online that can help you locate the most common cases you may want to worry about.
D. [substeps] If you believe capital punishment in any of your cases is unjust, find a law that applies to those cases. Look at the organizations's forum links.
Rendered prompt
[header] How to take action against capital punishment in your state [title] Focus your action on a specific case or cases. [step] It may help you to focus your efforts on a couple of current capital cases in your state. Focusing your work on a couple of specific cases may be more effective than tackling several cases or state laws as a whole.
Agent trace
A
Extracted: A · Gold: A
sample 3 · hellaswag:00002205pass · 100.0% · 1905ms · 9fde6021f549
Question
We see a man and the intro scene. A man in a kitchen has knifes and a sharpening block. the man

A. grinds the knifes through a rod.
B. begins sharpening the arm of a woman and releases.
C. wets the block and runs a knife across two different blocks.
D. then gets to chopping vegetables mainly in front of a counter with other scenes working.
Rendered prompt
We see a man and the intro scene. A man in a kitchen has knifes and a sharpening block. the man
Agent trace
C
Extracted: C · Gold: C
sample 5 · hellaswag:00002478pass · 100.0% · 1259ms · 303275c321c6
Question
The ingredients are added to a tub and scoops of ice cream are taken from a tub after freezing. a bowl

A. is filled with ice cream.
B. of ice cream is added to the bowl and melted in four different pans.
C. of olive oil is added as a spoon and bowls of butter are added.
D. is used to rack ice cream.
Rendered prompt
The ingredients are added to a tub and scoops of ice cream are taken from a tub after freezing. a bowl
Agent trace
A
Extracted: A · Gold: A
sample 6 · hellaswag:00003152pass · 100.0% · 838ms · 4b76423bef3b
Question
A young man is outside attempting to do a highjump behind the london logo. A male man then stands in front of the mat and talks about the jump. another male

A. is then shown as he takes off and clears the bar of the high jump and there's an instant replay.
B. appears in before he is cased six times attempting to same doing the same jump in different locations and taking short breaks.
C. joins in and the two jump from different heights, falling hundreds of times when landing.
D. joins in and prepares to get his jump in acting steadier and hitting the mat.
Rendered prompt
A young man is outside attempting to do a highjump behind the london logo. A male man then stands in front of the mat and talks about the jump. another male
Agent trace
A
Extracted: A · Gold: A
sample 7 · hellaswag:00009865pass · 100.0% · 1071ms · 80bdeb7973ed
Question
[header] How to make natural shampoo for your hair type [title] Boil 1 cup of spring water. [title] Remove the pan from the heat, and add 2 tablespoons of dried chamomile. [title] Let steep for 15 minutes.

A. [title] Obtain suitable shampoo for your hair type. [step] These shampoos and conditioners are very absorbent and therefore should be dried separately for longer.
B. [title] Strain the herbs from the liquid, and throw away the herbs. [title] Let the liquid sit for about 20 minutes, allowing it to completely cool down.
C. [title] Let cool a little but leave the liquid in for 1-2 minutes. [title] Strain the liquid and store in a cool container to use again later.
D. [title] Test and see if your shampoo mixture will lather up. [step] Add more-combine half of the mixture with 3 tablespoons of water until it bubbles.
Rendered prompt
[header] How to make natural shampoo for your hair type [title] Boil 1 cup of spring water. [title] Remove the pan from the heat, and add 2 tablespoons of dried chamomile. [title] Let steep for 15 minutes.
Agent trace
B
Extracted: B · Gold: B
Incorrect samples
sample 2 · hellaswag:00000706fail · 0.0% · 1485ms · 92983ab87d7a
Question
The boy wipes down the sole of the tennis shoe using the wet wipe towel. The boy dries off the shoe using a towel. the boy

A. rinses the shoe inside the folded towel with water.
B. picks up a high top pair and wipes it off with the same wet towel.
C. tapes down the rear portion of the shoe.
D. shakes his foot with music and the camera follows the movements.
Rendered prompt
The boy wipes down the sole of the tennis shoe using the wet wipe towel. The boy dries off the shoe using a towel. the boy
Agent trace
A
Extracted: A · Gold: B
sample 4 · hellaswag:00006200fail · 0.0% · 413ms · 24dc8d273221
Question
[header] How to wear yoga pants to work [title] Invest in high-quality yoga pants. [step] If you're wearing yoga pants to work, you want them to be durable while also providing comfort. Try to find yoga pants made of a thicker material, and try them on before purchasing them to ensure they aren't see-through.

A. High-quality yoga pants will likely be designed for yoga or other types of work. [title] Stick to a thin, stretchy, short-waist.
B. Sheer yoga pants are definitely something to avoid in the workplace. [substeps] Yoga leggings offer the most outfit options and tend to look the best in a professional work environment.
C. [substeps] Make sure the pants are specifically made for yoga, preferably ones with natural fibers, such as cotton or silk. When in doubt, look for yoga pants in solid colors.
D. Yoga pants come in a variety of shapes and sizes, and none of them fit under normal circumstances. See the rough guide section for more specific instructions.
Rendered prompt
[header] How to wear yoga pants to work [title] Invest in high-quality yoga pants. [step] If you're wearing yoga pants to work, you want them to be durable while also providing comfort. Try to find yoga pants made of a thicker material, and try them on before purchasing them to ensure they aren't see-through.
Agent trace
C
Extracted: C · Gold: B
sample 9 · hellaswag:00002393fail · 0.0% · 1687ms · e6a396ef8b4f
Question
The lady taps the puck with her stick before pushing it hard and pushing the puck across the court. the camera

A. gets itself angled to keep the puck from hitting the boy playing while playground equipment sits nearby.
B. zooms in on the puck and two women are standing next to it talking to each other.
C. zooms in on cupcake detail while the lady takes a breath.
D. follows the pucks which stops short and teeters before resting.
Rendered prompt
The lady taps the puck with her stick before pushing it hard and pushing the puck across the court. the camera
Agent trace
B
Extracted: B · Gold: D
sample 14 · hellaswag:00009278fail · 0.0% · 1708ms · 14f1f1d8ee1d
Question
[header] How to restore dll files on your computer [title] Missing dll files that somehow disappear can cause your computer to act in strange ways, such as not shutting down properly. [step] Run a system disk scan in the norton antivirus suite. For instance norton has a tool called " windoctor " that makes repairs automatically.

A. [title] Burn dll files in a safe location if they disappear. [step] Dll files are damaged in the scanning process.
B. [title] Notice when your case communications is missing. [step] Sometimes, when your case communications on your computer become unreadable, as administrator you can notice that the case communications are missing.
C. [title] Do a complete reboot of your computer. [step] After reboot you are looking at dll files that are already in their original packaging.
D. It is suggested that if you remove norton for another free antivirus program, they retain norton utilities/windoctor and speed disk for defragging. [title] Always run a scan on anything you download before you open it.
Rendered prompt
[header] How to restore dll files on your computer [title] Missing dll files that somehow disappear can cause your computer to act in strange ways, such as not shutting down properly. [step] Run a system disk scan in the norton antivirus suite. For instance norton has a tool called " windoctor " that makes repairs automatically.
Agent trace
C
Extracted: C · Gold: D
sample 28 · hellaswag:00002266fail · 0.0% · 1708ms · 2220cd3a5599
Question
He is standing in a kitchen in front of the counter that has a variety of raw vegetables displayed. He places a kitchen towel on the counter and then places the knife on top of it. he

A. uses another knife to paint his finger tips.
B. holds a red container and then places sticks of bread on the counter.
C. rubs the towel against the block then scrubs it along the knife.
D. then demonstrates how to use the rada sharpening tool to sharpen the kitchen knife by running iit back and forth in gentle and steady motion.
Rendered prompt
He is standing in a kitchen in front of the counter that has a variety of raw vegetables displayed. He places a kitchen towel on the counter and then places the knife on top of it. he
Agent trace
B
Extracted: B · Gold: D
GLM-5.3-Flashexl3unknown protocol · unknown agent
by Per0 · shard 10 · 9/1/2026, 1:25:15 PM · cmtip9cfe05kmp401scai5k5b
86.0%
664/772 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 1 · hellaswag:00001464pass · 100.0% · 450ms · ba0eec183742
Question
Several athletes compete for the javelin world championships. The russia dimitri is 3rd bronze medal in the competition. The japanese jenki is silver medal. till

A. javelin 1990 the russian javelin champion anna rubens is strong.
B. from germany won the gold medal.
C. loesos facility there is no competition.
D. now the french have won each javelin every round.
Rendered prompt
Several athletes compete for the javelin world championships. The russia dimitri is 3rd bronze medal in the competition. The japanese jenki is silver medal. till
Agent trace
B
Extracted: B · Gold: B
sample 2 · hellaswag:00005973pass · 100.0% · 276ms · c98299ff5733
Question
[header] How to get japanese characters (kanji, hiragana, katakana) on firefox [title] In the firefox toolbar, go to view] character encoding] auto-detect. [step] While most browsers are already capable of reading international characters, you may simply need to help firefox detect them. [title] Select japanese.

A. [step] By default, most of the newer characters on a firefox list are japanese (or katakana). However, you may have to reach up and back to find ones you'd like to include or that make similarities between each other.
B. [title] Focus your attention on chikatakana. [title] Choose the gender and intent of the character, ren.
C. [step] Some of the same characters can be found on the scroll bar. Selected japanese is the most commonly used japanese toolbar.
D. [step] Your firefox should now be able to detect japanese characters. If not, you may need to install a language pack using one of the methods below.
Rendered prompt
[header] How to get japanese characters (kanji, hiragana, katakana) on firefox [title] In the firefox toolbar, go to view] character encoding] auto-detect. [step] While most browsers are already capable of reading international characters, you may simply need to help firefox detect them. [title] Select japanese.
Agent trace
D
Extracted: D · Gold: D
sample 3 · hellaswag:00006664pass · 100.0% · 711ms · e5e3b6eec41d
Question
[header] How to make cappuccino foam [title] Pour brewed espresso into your cappuccino cup. [step] Although specifics will depend on the type of cappuccino and the person making it, cappuccinos are roughly one quarter espresso and three quarters steamed milk. Pour your desired amount of brewed espresso into a large cappuccino cup before you steam the milk.

A. [substeps] Instead of cappuccino add straight milk. Espresso can be iced later, and most cappuccino companies encourage it to be iced after brewing to sweeten the milk.
B. That way, you'll have everything ready so you can pour out the steamed milk while it's still piping hot. [title] Fill a steam pitcher with cold milk.
C. This will preserve your milk-stirring capacity throughout the rest of the process. [substeps] Try the recommended amount of coffee to get the right amount.
D. Start off with hot to ensure the coffee in your cup is sufficiently cold to immerse the foam. [substeps] It's best to use skim milk.
Rendered prompt
[header] How to make cappuccino foam [title] Pour brewed espresso into your cappuccino cup. [step] Although specifics will depend on the type of cappuccino and the person making it, cappuccinos are roughly one quarter espresso and three quarters steamed milk. Pour your desired amount of brewed espresso into a large cappuccino cup before you steam the milk.
Agent trace
B
Extracted: B · Gold: B
sample 5 · hellaswag:00001931pass · 100.0% · 1167ms · fb41bcc882aa
Question
A boy holds up a bottle of mouth wash. he

A. bends down and throws it.
B. takes the mouthwash and blows his horn.
C. uses the mouthwash to wash himself and two other teens.
D. drinks a cap full of the mouth wash.
Rendered prompt
A boy holds up a bottle of mouth wash. he
Agent trace
D
Extracted: D · Gold: D
sample 6 · hellaswag:00007682pass · 100.0% · 961ms · f871412b315b
Question
[header] How to wash newborn hair [title] Rinse your newborn's hair for the first bath. [step] Because the hair and skin of a newborn are so delicate and sensitive, you don't actually need to use shampoo or cleansers. For your newborn's first bath, just pour a little plain water over their scalp.

A. Just enough so that the hair and skin are fully saturated. [substeps] This is also an optional step that you can do while breastfeeding.
B. You will make sure that they have nothing like a toothbrush or comb or baby oil in their shampoos. [title] Repeat the soap and water process with your baby's hair.
C. Pat the newborn's scalp dry with a soft towel. [substeps] You don't need to scrub, massage, or really wash the scalp until they're a little older.
D. For added comfort, you can place the newborn against a towel lined with skin-safe bath paper. [title] Combine the shampoo and water in a small bowl.
Rendered prompt
[header] How to wash newborn hair [title] Rinse your newborn's hair for the first bath. [step] Because the hair and skin of a newborn are so delicate and sensitive, you don't actually need to use shampoo or cleansers. For your newborn's first bath, just pour a little plain water over their scalp.
Agent trace
C
Extracted: C · Gold: C
Incorrect samples
sample 4 · hellaswag:00008041fail · 0.0% · 1421ms · c715921c871a
Question
[header] How to get an adult male to use his manners [title] Consider what direct and indirect demands you have been making. [step] Reflect on the balance of personal demands you are making to your partner, and evaluate whether that balance is just. [title] Be prepared to evaluate your viewpoint of the situation as objectively as possible.

A. [title] First, determine whether your partner was accurate in his assessment of his sexual capabilities and responses. [title] Discuss there are a number of different ways that men in modern society can show discretion, professionalism, and safety in the absence of sexual restraint.
B. [step] You have also made a choice that your partner must want and must follow. When you record and evaluate your demands, be sure you are or need to be able to speak with those who are making them.
C. [title] Consider your own perspective, not personal. [step] Communicating your viewpoint will provide you with invaluable resources.
D. [step] Don't go into the impending talk convinced that anything he might say is either wrong or not heartfelt. That won't help to solve the problem.
Rendered prompt
[header] How to get an adult male to use his manners [title] Consider what direct and indirect demands you have been making. [step] Reflect on the balance of personal demands you are making to your partner, and evaluate whether that balance is just. [title] Be prepared to evaluate your viewpoint of the situation as objectively as possible.
Agent trace
A
Extracted: A · Gold: D
sample 13 · hellaswag:00003095fail · 0.0% · 1365ms · 682abd21b892
Question
A man stands in the middle of a stadium. crowds

A. are gathered around him as he spins around.
B. watch and cheer the man on.
C. watch from behind a curtain.
D. flash around him.
Rendered prompt
A man stands in the middle of a stadium. crowds
Agent trace
A
Extracted: A · Gold: B
sample 21 · hellaswag:00008766fail · 0.0% · 1425ms · a32e88c9134b
Question
[header] How to get natural looking waves [title] Wash or mist your hair. [step] When braiding your hair to create waves, it is best to begin with damp hair. You may achieve this in one of two ways : [substeps] Wash your tresses prior to braiding your hair.

A. Tie off the ends of the braid and then let the loose hair dry slightly before washing it. Gradually do this for all hair types-curly, straight, with some parting.
B. When you begin braiding your hair, do not pat your locks dry. Instead, work a small amount of shampoo (coconut hair shampoo) into your hands and massage the shampoo into your hair.
C. In one, slightly wet your hair, then mist it with water. In the other, leave your hair damp and then mist your tresses gently with the palm of your hand.
D. Mist your hair with lukewarm water. [title] Combat frizz and snarls.
Rendered prompt
[header] How to get natural looking waves [title] Wash or mist your hair. [step] When braiding your hair to create waves, it is best to begin with damp hair. You may achieve this in one of two ways : [substeps] Wash your tresses prior to braiding your hair.
Agent trace
B
Extracted: B · Gold: D
sample 24 · hellaswag:00000385fail · 0.0% · 1349ms · df0d7fd9ff6f
Question
A small group of people are seen shaking hands and standing together followed by a man blowing a whistle and people running around. the people

A. then play a game of soccer with one another, running up and down the sandy field while coaches yell on the side and a goal is blocked.
B. continue to run around under the stage and close up of a man speaking and shows people playing a game.
C. move around with drums and hitting a tune in the middle of the gym and the man continues to blow a whistle and lead into more people walking past on the side.
D. continue riding around on skateboards as well as following behind one another as well as people speaking to the camera.
Rendered prompt
A small group of people are seen shaking hands and standing together followed by a man blowing a whistle and people running around. the people
Agent trace
D
Extracted: D · Gold: A
sample 28 · hellaswag:00006141fail · 0.0% · 1329ms · 33ca21fd29ba
Question
[header] How to give a baby a bath in the sink [title] Fill the bath a few inches deep. [step] Fill a safe, clean receptacle with roughly three inches of warm water. The water should be between between 90 ° f (32 ° c) and 100 ° f (38 ° c).

A. Once you have the appropriate amount of water, shut it off and swirl the water to ensure there are no especially hot spots. [substeps] Touch the water with the inside of your wrist.
B. If the baby is crying, they are likely to drown, so be sure to warm up. [substeps] Baby showers are often set up near the bath area.
C. [substeps] If the bath is not going to produce suds in the water, you may want to fill a bowl with clean tap water to create a cleaner float. [title] Submerge your baby in the water.
D. Add a few drops of essential oils to the water to add a relaxing feeling to the bath. [title] Submerge your newborn in the water.
Rendered prompt
[header] How to give a baby a bath in the sink [title] Fill the bath a few inches deep. [step] Fill a safe, clean receptacle with roughly three inches of warm water. The water should be between between 90 ° f (32 ° c) and 100 ° f (38 ° c).
Agent trace
D
Extracted: D · Gold: A
GLM-5.3-Flash-tr3-4bpwexl3unknown protocol · unknown agent
by Per0 · shard 9 · 9/1/2026, 7:44:19 AM · cmtid2wgt03r3p401wlqr940g
84.7%
655/773 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 1 · hellaswag:00001856pass · 100.0% · 194ms · 7dc0758352f8
Question
A group of men stand on top of logs, chopping away. The chopping legend speaks to the camera again. a couple of men

A. are in a group near a river, talking.
B. stand on the log top wearing protective gear.
C. chop some wood until one of them jumps off the log and raises his hands up in excitement.
D. are hanging up rocks above a river.
Rendered prompt
A group of men stand on top of logs, chopping away. The chopping legend speaks to the camera again. a couple of men
Agent trace
C
Extracted: C · Gold: C
sample 2 · hellaswag:00008393pass · 100.0% · 1195ms · 95202ba8c559
Question
[header] How to use movies to help kids build character [title] Pick movies by age group. [step] While some younger kids are more mature, generally, you don't want a story line that will go right over their head. Similarly, you don't want to lose older kids by showing them movies that are too simplistic.

A. [substeps] Often, the best movie lines are set in the last century. While you may have super teens, you can find lots of great movies for kids online.
B. If possible, choose movies by kids you love, not kids they already like. [substeps] For instance, in the movie audrey hepburn 2 2000, the tables at breakfast [[ (18.
C. Try to pick something that works for the age the kid is at. [substeps] For kids 2 to 7 years old, make sure the story is fairly simple, and that plot supports the idea you're trying to get across, as it needs to be glaringly obvious.
D. Most of the time, kids get what they want early on. Take your age group into consideration as you come up with a few choices for your movie series.
Rendered prompt
[header] How to use movies to help kids build character [title] Pick movies by age group. [step] While some younger kids are more mature, generally, you don't want a story line that will go right over their head. Similarly, you don't want to lose older kids by showing them movies that are too simplistic.
Agent trace
C
Extracted: C · Gold: C
sample 3 · hellaswag:00007942pass · 100.0% · 705ms · a4d6d883f5bb
Question
[header] How to choose an alternative to a flower girl [title] Invite other members of the family to join your wedding party. [step] Instead of having young flower girls, consider using older women in your family, such as your grandma or great aunt. They'll love to be included in a special way, and the sight of your grandma throwing flower petals into the air is to sure to bring a smile to everyone's face.

A. [substeps] Older women can even invite their significant other as well. Just be sure the couple makes time to discuss the details in advance.
B. [title] Let everyone know you're having a flower girl themed party. [step] In several ways, this will make your party more fun.
C. [title] Read opinions from other members of the family. [step] Reading magazines and articles on the flower girl instead of relying on your friends for opinions can help you understand what happened that night in the picture.
D. [title] Give the role to a friend who volunteers. [step] One of your adult friends may find the idea of being a flower girl enchanting.
Rendered prompt
[header] How to choose an alternative to a flower girl [title] Invite other members of the family to join your wedding party. [step] Instead of having young flower girls, consider using older women in your family, such as your grandma or great aunt. They'll love to be included in a special way, and the sight of your grandma throwing flower petals into the air is to sure to bring a smile to everyone's face.
Agent trace
D
Extracted: D · Gold: D
sample 4 · hellaswag:00007936pass · 100.0% · 938ms · 661a83c3c67a
Question
[header] How to care for your skin before and after a brazilian wax [title] Prepare 24 hours before wax. [step] The day before your wax, exfoliate the area. You don't need to use a special scrub.

A. You can exfoliate regularly with a loofah, facial bar, or regular washcloth. [substeps] Do not exfoliate right before you wax your brows.
B. All you need is a washcloth or bath pouffe with soap. Make sure to do this at least a day before your wax.
C. [substeps] As you rub on and dry your skin afterwards, avoid washing with soap or body wash. [title] Exfoliate with a facial cleanser.
D. [substeps] To get the most benefit, scrub with a pumice stone. If you are not exfoliating, begin by exfoliating with a pumice stone the day before wax.
Rendered prompt
[header] How to care for your skin before and after a brazilian wax [title] Prepare 24 hours before wax. [step] The day before your wax, exfoliate the area. You don't need to use a special scrub.
Agent trace
B
Extracted: B · Gold: B
sample 5 · hellaswag:00005966pass · 100.0% · 1441ms · 5bb3a5ea3513
Question
[header] How to make resurrection rolls [title] Pour the milk into a bowl. [step] To activate the yeast, you need to mix it with a warm liquid. Add ½ cup (118 ml) of warm milk to the bowl of a stand mixer.

A. [substeps] For a longer process, you might want to run the yeast in the sink before proceeding. [title] Gradually stir the milk and mixture with a whisk.
B. Mix until the milk is very light (around 110 ml). [substeps] If the milk is too soft for this recipe, add 1/2 cup (120 ml) of sour cream.
C. The milk should be 105 ° f (41 ° c). [substeps] You can use 1% or 2% milk, but whole milk usually results in the best rolls.
D. [substeps] If you have a hand mixer, you can make the dough for the roll yourself. You'll only need to make 2 cups (500 ml) of milk.
Rendered prompt
[header] How to make resurrection rolls [title] Pour the milk into a bowl. [step] To activate the yeast, you need to mix it with a warm liquid. Add ½ cup (118 ml) of warm milk to the bowl of a stand mixer.
Agent trace
C
Extracted: C · Gold: C
Incorrect samples
sample 15 · hellaswag:00006432fail · 0.0% · 1480ms · 3a1949a55cfa
Question
[header] How to make vegan mint chocolate cake [title] Start preheating the oven to 325 degrees f (160 c) as you coat your cooking pan with oil or cooking spray. [title] Place the beets in a saucepan and fill it with enough water to cover the beets. [title] Allow the water to come to a boil and cook them at a medium-high heat until they become soft when pierced with a fork.

A. [title] Add the fresh mint and let the beets simmer, stirring frequently, for a few minutes. [title] Cover with plastic wrap and allow the beets to simmer for up to five minutes.
B. [title] Remove the beets from the pan and place in a bowl. [step] The sieve will remove any dirt, sweat or bacteria clinging to the beets.
C. [title] Turn off the heat and place the beets in a large bowl. [step] Stir in the melted butter, chocolate syrup, and the vanilla extract.
D. [title] Allow the beets to cool off the heat as you place a fine mesh sieve over a medium bowl. [title] Pour the water and beats into the sieve.
Rendered prompt
[header] How to make vegan mint chocolate cake [title] Start preheating the oven to 325 degrees f (160 c) as you coat your cooking pan with oil or cooking spray. [title] Place the beets in a saucepan and fill it with enough water to cover the beets. [title] Allow the water to come to a boil and cook them at a medium-high heat until they become soft when pierced with a fork.
Agent trace
C
Extracted: C · Gold: D
sample 19 · hellaswag:00005749fail · 0.0% · 1526ms · f018acf43f5f
Question
[header] How to remove an application (game) off your facebook account [title] Log in to facebook. [step] Type in your username and password when prompted. [title] Locate the particular game/app that you intend to remove.

A. [step] This will be under the app category under " settings. " on the left sidebar, you should find " games " under " apps.
B. [step] Find the game/app, and download it to your computer. If you find it, navigate to the game/app download page and download the app : [substeps] Games for cfu game.
C. [step] Once you type in the name, you will be brought to the bottom of the list and must select it, otherwise the menu will close and you cannot see the game/app's icon yet. [substeps] The app icon is the middle portion of the app that is located to the right of the apple id card icon.
D. [step] Some activities are alike but for those activities which are more restricted, some can be blocked. Remember that what you're viewing/listening to and wanting to view on facebook is all unique.
Rendered prompt
[header] How to remove an application (game) off your facebook account [title] Log in to facebook. [step] Type in your username and password when prompted. [title] Locate the particular game/app that you intend to remove.
Agent trace
C
Extracted: C · Gold: A
sample 27 · hellaswag:00001112fail · 0.0% · 1341ms · 2be765b01326
Question
Man is siting in a bed playing an harmonica and an acordion. Man is sitting playing instruments in a room. man

A. is standing in the room with a harmonica in his hand.
B. is sitting in a room and instruments are hanging in the back wall.
C. is acorion talking to the camera.
D. is carrying an accordion in a room.
Rendered prompt
Man is siting in a bed playing an harmonica and an acordion. Man is sitting playing instruments in a room. man
Agent trace
A
Extracted: A · Gold: B
sample 39 · hellaswag:00001665fail · 0.0% · 1305ms · 5db69650a387
Question
A boy stands on a basketball court holding a ball. other players

A. make baskets on the court.
B. swarm the boy holding the ball.
C. are chasing after the ball.
D. approach in the background, then the boy in the court runs and gives a high jump.
Rendered prompt
A boy stands on a basketball court holding a ball. other players
Agent trace
C
Extracted: C · Gold: B
sample 42 · hellaswag:00001168fail · 0.0% · 1251ms · 9721472585d1
Question
A female weight lifter walks up to a barbell. she

A. squeezes it and then jumps over the bar backwards.
B. lifts it up to her neck.
C. lifts it onto her shoulder before lowering it to the ground.
D. lifts it up to her chest before seated.
Rendered prompt
A female weight lifter walks up to a barbell. she
Agent trace
C
Extracted: C · Gold: B
Qwen3.8-Flash-Next-NVFP4NVFP4unknown protocol · unknown agent
by Per0 · shard 7 · 8/31/2026, 1:47:41 PM · cmthamccx01gop4016p38wuvo
82.1%
635/773 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 1 · hellaswag:00007843pass · 100.0% · 418ms · 2b00dd2efd79
Question
[header] How to cite a book [title] Familiarize yourself with the accepted citation styles. [step] Use only one style when citing a book reference-each style has very specific rules about capitalization, punctuation and placement of the facts. That said, all styles are designed for the same purpose: to give proper credit where credit is due.

A. It is not necessary to include more than your attention span. You could include reading journal articles (if they were used before) or writing your own start point book reviews (if they are not never published by the author).
B. Listed below are the most common styles : [substeps] Modern language association (mla) style. This is a citation style most often used within the liberal arts and humanities departments of universities.
C. [substeps] Use a capital letter or a comma after the reference terms of a book reference. This will refer to the same format in a sentence-to-sentence format.
D. [substeps] There are several great writers' styles for reference: paul golding, glenwhelp, dell, k. paul wilson (1960s innovator), william investments investors, allan cannon (1938), bob n.
Rendered prompt
[header] How to cite a book [title] Familiarize yourself with the accepted citation styles. [step] Use only one style when citing a book reference-each style has very specific rules about capitalization, punctuation and placement of the facts. That said, all styles are designed for the same purpose: to give proper credit where credit is due.
Agent trace
B
Extracted: B · Gold: B
sample 2 · hellaswag:00006278pass · 100.0% · 1704ms · c6f497c808d7
Question
[header] How to be emo in middle school [title] Learn what it means to be emo. [step] Before you can become emo, you have to understand exactly what it means. " emo " is short for emotive, and people who consider themselves to be emo are usually in touch with their emotions.

A. They don't feel the need to hide how they feel, even when they are sad or angry. [substeps] While music, clothing, hair, and makeup can play a large role in being emo, being comfortable with your feelings and expressing them is the most important part.
B. This is an important step to emo, so have a good understanding of your gender. You don't have to have what everyone else has.
C. You don't need a dictionary dictionary and stick with the " emotive ". Instead, learn what emotive means, like " i am ", " i am skinny ", " i am strong and strong ", " i'm confident in my body ", etc.
D. You can't be emo without being emo. [substeps] If you're emo, you want an emo vibe that's unique to you.
Rendered prompt
[header] How to be emo in middle school [title] Learn what it means to be emo. [step] Before you can become emo, you have to understand exactly what it means. " emo " is short for emotive, and people who consider themselves to be emo are usually in touch with their emotions.
Agent trace
A
Extracted: A · Gold: A
sample 3 · hellaswag:00004307pass · 100.0% · 975ms · 0729af895014
Question
[header] How to make a smoke bomb [title] Gather three or four ping-pong balls. [step] For this method, you don't have to buy any extra ingredients other than basic household supplies, like aluminum foil, a pencil, screwdriver, and scissors. [title] Cut a hole in the top of one of the balls.

A. [step] You can poke a hole with a screwdriver or a knife. [title] Cut the other balls into small pieces and put them in the hole.
B. [step] Use a knick knack knife to create an x around one of the balls. [title] Punch a hole in the bottom of one of the balls.
C. [step] Align the hole in the top ball with the one that's down. [substeps] The hole needs to be about six-ten inches deep and about two inches wide.
D. [step] Grab a piece of foil about 15 inches from the edge of the ball. Get a zig-zag or intricate pattern.
Rendered prompt
[header] How to make a smoke bomb [title] Gather three or four ping-pong balls. [step] For this method, you don't have to buy any extra ingredients other than basic household supplies, like aluminum foil, a pencil, screwdriver, and scissors. [title] Cut a hole in the top of one of the balls.
Agent trace
A
Extracted: A · Gold: A
sample 4 · hellaswag:00003431pass · 100.0% · 1523ms · 727d315e3ed2
Question
[header] How to apply shadow on hooded eyes [title] Prime your eyelids. [step] While this is important for everyone, it's especially critical for those with hooded eyes. Primer creates a base for your makeup that helps it stay put and last all day.

A. [substeps] If you use liquid primer, try it first before you start applying it on your eyes. Otherwise, you'll try a different primer beforehand.
B. Primer also keeps your eyelids from getting heavy and swollen. To prime your eyelids : [substeps] Apply an eyeshadow to your lids using a dark gray eye liner or gel instead of a cream or gel liner.
C. Because hooded eyes are prone to smudging and smearing, primer can make all the difference. [substeps] Use your fingertip to apply the primer, and allow it to absorb into your skin for a minute or so before proceeding.
D. Apply a primer to your eyelids and eyelid over the primer you applied earlier. [substeps] Use your powder brush, primer, or a makeup sponge to carefully prime your eyelids.
Rendered prompt
[header] How to apply shadow on hooded eyes [title] Prime your eyelids. [step] While this is important for everyone, it's especially critical for those with hooded eyes. Primer creates a base for your makeup that helps it stay put and last all day.
Agent trace
C
Extracted: C · Gold: C
sample 6 · hellaswag:00002238pass · 100.0% · 780ms · fec39366eba5
Question
The man lays down and had a hard time breathing. After putting down sand he drinks a warm beverage. we

A. see the man underwater swimming.
B. see the ending credits screen.
C. then see two guys swimming nak * d and drinking.
D. see no other man and the other man is shown shirtless.
Rendered prompt
The man lays down and had a hard time breathing. After putting down sand he drinks a warm beverage. we
Agent trace
B
Extracted: B · Gold: B
Incorrect samples
sample 5 · hellaswag:00004697fail · 0.0% · 1159ms · 913f8d7c63a6
Question
[header] How to get favorites on your deviantart photo [title] Create your own username. [step] Have a name that can be easily recognized and defines you as a person. [substeps] Your username doesn't have to be your first name, but some well-known deviantart members remember their art style through their da names.

A. Avoid using names of famous celebrities or character from movies, video games, or animes. To change your username, you have to purchase premium membership.
B. If you don't want people to recognize you as a person, try to keep the name honest and easy to identify. You can do this by including your full name and what year it is, but stay away from androgynous names.
C. For instance, president christian founder of the white house had a username that " peter " (modern), and there are so many other names available today. In the wild, some names are simply variations of the last name.
D. [title] Create usernames for your favorite characters. [step] Depending on your theme and design, you may have many different options.
Rendered prompt
[header] How to get favorites on your deviantart photo [title] Create your own username. [step] Have a name that can be easily recognized and defines you as a person. [substeps] Your username doesn't have to be your first name, but some well-known deviantart members remember their art style through their da names.
Agent trace
D
Extracted: D · Gold: A
sample 8 · hellaswag:00002350fail · 0.0% · 1463ms · 2e760a84b9b4
Question
People are sitting on tubes in the snow. they

A. stand in a line and start going down a snowy hill.
B. are going over a large hill.
C. begin sledding down the hill on the tubes.
D. stand on the top of the tubes.
Rendered prompt
People are sitting on tubes in the snow. they
Agent trace
A
Extracted: A · Gold: C
sample 19 · hellaswag:00000609fail · 0.0% · 1635ms · 0df9993b1b59
Question
A man and a woman are in a ring wrestling while a referee is standing in the corner of the ring along with a woman holding a wrestling belt. man and woman

A. are doing wrestling movements in a ring in a dark room and a referee is waching and screaming at them.
B. are wrestling in punches and kicking in a ring of mats in front of judges.
C. are holding a wrestling pole as the referee is standing on the other side of the ring.
D. are standing on opposite ends of the ring holding each other's belts in their hands while lifting each other up in the air upside down.
Rendered prompt
A man and a woman are in a ring wrestling while a referee is standing in the corner of the ring along with a woman holding a wrestling belt. man and woman
Agent trace
D
Extracted: D · Gold: A
sample 26 · hellaswag:00007917fail · 0.0% · 1475ms · 3cede662d957
Question
[header] How to raise children successfullly [title] Help children learn about nature. [step] Teaching children to love and not fear nature is a vital part of teaching children that this a larger world than their bedroom and games station. It will give them the ability to feel a connection with something other than the house, school and local park.

A. [substeps] The elementary curriculum requires children to learn about nature from birth, and learn how to identify it and understand it. Even at a young age, children become proficient at discovering and appreciating nature and the way it feels to them.
B. It will also be a learning experience for young children. [substeps] Find new ways to enjoy outdoor sports.
C. Pack a couple sandwiches, take the kids into the woods and hike a deer trail through the woods. As you go : [substeps] Let the kids ask questions.
D. Many children like hiking, camping, wildlife photography, or nature watching, so they will enjoy the outdoors with you. [title] Show gratitude for your friends, family, and fellow caregivers.
Rendered prompt
[header] How to raise children successfullly [title] Help children learn about nature. [step] Teaching children to love and not fear nature is a vital part of teaching children that this a larger world than their bedroom and games station. It will give them the ability to feel a connection with something other than the house, school and local park.
Agent trace
A
Extracted: A · Gold: C
sample 38 · hellaswag:00002785fail · 0.0% · 1407ms · 6557f365ffed
Question
A man is shooting a basketball into a hoop. Another man starts shooting a basketball into the hoop. the first man

A. moves the hoop to the other side of the court.
B. is laying on the ground holding his chest.
C. watches the second man.
D. stands up and throws something onto the court.
Rendered prompt
A man is shooting a basketball into a hoop. Another man starts shooting a basketball into the hoop. the first man
Agent trace
A
Extracted: A · Gold: C
Qwen3.8-27B-GPTQ-W4A16unspecified quantunknown protocol · unknown agent
by SgtPooki · shard 13 · 8/22/2026, 2:02:08 PM · cmt4g690x02t3pu01o44yj4v6
79.8%
296/371 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 3 · hellaswag:00008485pass · 100.0% · 146ms · ef5ba648c703
Question
[header] How to burp a sleeping baby [title] Hold your baby and burp them. [step] This technique is good for babies who sleep on their stomach or who like to cuddle when they sleep. [substeps] Slowly move your baby next to your body so you do not wake them up.

A. Allow your baby's head or chin to rest on your shoulder and cup their bottom to support them so they don't slip as you hold them. Place your other hand on their back and gently pat it to help them burp.
B. You can also lower your baby's head and neck down into a deep belly position, which will help you burp your baby more easily. Gently nudge your baby at first and again until they are used to this position and begin to adjust to the feel of their neck and pillow.
C. Hold your baby close with your hand and record the amount of air they intake and how much you burp each time before you burp them. This way, you can do it for several days to see how comfortable they will be with your hand.
D. Then gently roll them over so they are lying on their back. Hold and burp your baby together until they become comfortable.
Rendered prompt
[header] How to burp a sleeping baby [title] Hold your baby and burp them. [step] This technique is good for babies who sleep on their stomach or who like to cuddle when they sleep. [substeps] Slowly move your baby next to your body so you do not wake them up.
Agent trace
A
Extracted: A · Gold: A
sample 4 · hellaswag:00005294pass · 100.0% · 173ms · 7f5b2215e320
Question
[header] How to use a highlighter stick [title] Choose an ivory or cream highlighter shade if you have fair skin. [step] If you have a pale complexion, cream and ivory highlighters with a pearlescent or icy-silver tinge work really well. While these shades tend to look ghostly and unnatural on other complexions, they add a natural-looking glow to fair skin.

A. [substeps] Look for highlighter shades that include words like " moonbeam, " " ice, " and " crystalline. " avoid shades that are darker than cream and ivory.
B. [title] Try a metallic highlighter shade to go with your outfit. [step] If you have a pale complexion, you may prefer something metallic.
C. [substeps] Ivory highlighters are quite pricey. They cost anywhere from $14-20.
D. They make blending difficult to maintain, so get creative and test out different color highlighters. [substeps] For darker skin colors, choose a similarly color highlighter.
Rendered prompt
[header] How to use a highlighter stick [title] Choose an ivory or cream highlighter shade if you have fair skin. [step] If you have a pale complexion, cream and ivory highlighters with a pearlescent or icy-silver tinge work really well. While these shades tend to look ghostly and unnatural on other complexions, they add a natural-looking glow to fair skin.
Agent trace
A
Extracted: A · Gold: A
sample 5 · hellaswag:00010031pass · 100.0% · 137ms · 34999dc38706
Question
[header] How to rent a limousine [title] Decide when you'll rent the limo. [step] Early summer is the peak of the season for hiring limousines. You will probably be able to secure a better price if you hire a limo outside the months of may and june.

A. You're also likely to get a better rate on weekdays. [substeps] Sunday through thursday are typically the least expensive days, while fridays are about 20% less than saturdays.
B. If you're looking for one in july or august, you might want to be the driver. [title] Decide whether to ask friends, relatives, or strangers.
C. [substeps] Many limousine offers large limousine packages for lower prices. However, the driver hiring a limousine for the funeral may not be the first limo you hire.
D. A limousine is usually on the streets or across city traffic and less hectic between the ages of september and july. Saturday taxis are typically used if you live in an area where it is hot.
Rendered prompt
[header] How to rent a limousine [title] Decide when you'll rent the limo. [step] Early summer is the peak of the season for hiring limousines. You will probably be able to secure a better price if you hire a limo outside the months of may and june.
Agent trace
A
Extracted: A · Gold: A
sample 6 · hellaswag:00005104pass · 100.0% · 145ms · ea6b5cd4a03e
Question
[header] How to dip dye dark hair [title] Protect yourself and work area from bleach stains. [step] Cover your counter with newspaper. Wrap a dyeing cape or an old towel around your shoulders and put on a pair of plastic gloves.

A. Lastly, make sure that you have all of your supplies laid out and ready to go. [title] Mix the bleach according to the instructions on the package.
B. [substeps] Deep dye dark hair requires heavy amounts of bleach. For deep dye, i recommend having a shirt with dark sleeves.
C. Wear a sturdy t-shirt and apron to protect yourself while you work. Put bleach bowls or a bucket into your sink so that you can come to an additional solution that's the same color as the dye.
D. Bleach is caustic and it affects your hair and eyes. [substeps] You can also wear a bandanna around your neck.
Rendered prompt
[header] How to dip dye dark hair [title] Protect yourself and work area from bleach stains. [step] Cover your counter with newspaper. Wrap a dyeing cape or an old towel around your shoulders and put on a pair of plastic gloves.
Agent trace
A
Extracted: A · Gold: A
sample 7 · hellaswag:00005596pass · 100.0% · 165ms · fb6cc22c3d0a
Question
[header] How to hire syrian refugees [title] Find your local resettlement office. [step] Refugees admitted into a new country are eligible to work immediately. Government programs assist refugees in finding employment as part of their resettlement programs.

A. If you identify yourself as a refugee, you must work as a refugee in the country. [title] Complete any required occupations (such as a position, career, or order) before applying for a job.
B. [substeps] Use the job search tools at the resource website: https: // www.resettlement-cepus.gov/workspace/guild-acanees. Cfm look on your city or town's website for a list of resettlement department locations.
C. [substeps] In the united states, you can find your local resettlement office by visiting https: // www.acf.hhs.gov/orr/state-programs-annual-overview. Click on your state on the map, or select your state from the drop-down menu below the map to get a list of local offices and contact information.
D. The national guard service reacts positively to refugees seeking work as part of their resettlement program, as they provide a much more productive working environment. [substeps] After visiting your local resettlement office, locate the website by typing " resettlement program " and your city and state into a search engine.
Rendered prompt
[header] How to hire syrian refugees [title] Find your local resettlement office. [step] Refugees admitted into a new country are eligible to work immediately. Government programs assist refugees in finding employment as part of their resettlement programs.
Agent trace
C
Extracted: C · Gold: C
Incorrect samples
sample 1 · hellaswag:00001623fail · 0.0% · 61ms · a061d3a6bb7b
Question
A man has climbed a large ladder outside. he

A. is using trimmers to cut and trim large trees.
B. ropes and lures a cow into the open.
C. is using it to pull himself up onto a platform where a car is parked.
D. is life saving at the bottom of the ladder.
Rendered prompt
A man has climbed a large ladder outside. he
Agent trace
C
Extracted: C · Gold: A
sample 2 · hellaswag:00006547fail · 0.0% · 184ms · e498ae6053dd
Question
[header] How to buy organic soil [title] Do your research. [step] Truly organic soil can be hard to come by since the market for organic potting media has only begun increasing in the past few years. Moreover, organic standards have also become stricter as the demand for organic goods has grown, so a soil that was once deemed organic may no longer fulfill those requirements legally.

A. To survive, the best option is to research the quality of organic soil you buy and see how they relate with your home or ecosystem. [substeps] Make sure that your soil is organic.
B. Determining what is organic will help you decide whether it would be a good idea to buy the soil you need. [substeps] There are two main kinds of organic soil: organic and non-organic.
C. Before shopping for an organic soil, inform yourself so that you know what to expect. Traces of chemicals can be common, especially in soils containing certain types of manure, but if the amount is low enough it could still qualify as organic.
D. [title] Buy organic potting soil instead of commercial ones. [step] Changing to commercial potting soil will reduce the amount of organic overgrowth in your field and reduce the amount of organic manure produced by your cultivars' buckers.
Rendered prompt
[header] How to buy organic soil [title] Do your research. [step] Truly organic soil can be hard to come by since the market for organic potting media has only begun increasing in the past few years. Moreover, organic standards have also become stricter as the demand for organic goods has grown, so a soil that was once deemed organic may no longer fulfill those requirements legally.
Agent trace
B
Extracted: B · Gold: C
sample 9 · hellaswag:00009657fail · 0.0% · 145ms · db239160640e
Question
[header] How to tell your parents you're flunking out [title] Help around the house. [step] Offer to do small things like cleaning the dishes, setting the table, washing the car...they'll be suspicious at first, but chill. If you flash a smile at them, they'll freak out even more.

A. [title] If your parents see you're kind or decent but have homework the night before, ask them if you could join them at their house around 11, or before their curfew. [step] This shows them that you can be organised and learn to be a good citizen.
B. When you let your parents know you can handle any all the chores, they'll get a sense that you've this all figured out. [substeps] Send them a little pep talk in the morning.
C. If you make a mess while doing dishes, congratulate yourself for being able to do it without them noticing. [substeps] Try putting a small potted plant, like a cranberry, right on the table.
D. Remember: known teenage trouble-makers are not supposed to smile. [title] Be nicer to them, so they can see that you're at least trying to be different.
Rendered prompt
[header] How to tell your parents you're flunking out [title] Help around the house. [step] Offer to do small things like cleaning the dishes, setting the table, washing the car...they'll be suspicious at first, but chill. If you flash a smile at them, they'll freak out even more.
Agent trace
C
Extracted: C · Gold: D
sample 13 · hellaswag:00000389fail · 0.0% · 59ms · 2802a406f49a
Question
A young man is seen playing drums in front of an audience. other people

A. are seen speaking to the camera as well as running into the audience.
B. are watching as he continues to play.
C. are walking by back and fourth.
D. are shown playing guitar while the man plays more drums.
Rendered prompt
A young man is seen playing drums in front of an audience. other people
Agent trace
B
Extracted: B · Gold: D
sample 15 · hellaswag:00008268fail · 0.0% · 119ms · d96daca04434
Question
[header] How to discreetly breastfeed your baby on the go [title] Wear nursing clothing. [step] Nursing tops can be purchased at maternity stores and most major clothing retailers. These tops have hidden openings that allow you to easily feed your baby.

A. The lining on these dresses is designed to allow you to access the breast once your baby has grown accustomed to you. To remove the lining, simply slide it off and then gently remove it.
B. [substeps] Wear one that covers the breasts, but does not cover them completely. Many maternity tops are available in mesh or sheer fabrics.
C. You can get a variety of items such as tops, bras, and dresses. [substeps] Nursing bras that are made like sports bras are usually better.
D. When your baby is traveling, these have tiny baby items that you can remove and show your baby. [substeps] You can also move to a place where you will not be bothered by baby items.
Rendered prompt
[header] How to discreetly breastfeed your baby on the go [title] Wear nursing clothing. [step] Nursing tops can be purchased at maternity stores and most major clothing retailers. These tops have hidden openings that allow you to easily feed your baby.
Agent trace
B
Extracted: B · Gold: C