Erste SchritteLeaderboardModelleHardwareBenchmarksMarktplatzVermietungenProAPI-Doku
Sprache
Atomic Chat DFlash campaign banner

HellaSwag

Official shard eval

HellaSwag deterministic disjoint eval shards from Rowan/hellaswag:validation. Running all 13 shards covers the full 10,042-question dataset; Postgres stores only shard metadata and pass/fail results.

Source
Category: ReasoningEval type: Shard evalQuestions: 10,042Shards: 13Runs: 158

Dry-run first:

lmx eval shard hellaswag --base-url http://localhost:8000 --questions 371 --dry-run

Then submit with a real model and hardware profile:

lmx eval shard hellaswag --base-url http://localhost:8000 --questions 371 --model <hfId> --hardware hardware.json --submit

Scores are pooled by unique question_id; the leaderboard is ranked by Wilson 95% lower bound. HellaSwag uses canonical continuation loglikelihood scoring. Use a logprob-capable /v1/completions endpoint, or pass --model-path model.gguf to use the bundled local llama.cpp scorer.

Leaderboard

Qwen3.6-27B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
unsloth/Qwen3.6-27B-GGUF · 1 run · 1/13 shards · harness-scoped
93.3%
95% CI 91.394.8%
721/773 correct · 7.7% coverage
2
Qwopus3.6-27B-Coder-MTP-GGUFQ5_K_M · ggufunknown protocol · unknown agent
Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF · 1 run · 1/13 shards · harness-scoped
92.9%
95% CI 90.894.5%
717/772 correct · 7.7% coverage
3
Qwen3.6-35B-A3B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
unsloth/Qwen3.6-35B-A3B-GGUF · 26 runs · 13/13 shards · harness-scoped
90.4%
95% CI 89.891.0%
9079/10042 correct · 100.0% coverage
4
Qwen3.5-27B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
unsloth/Qwen3.5-27B-GGUF · 1 run · 1/13 shards · harness-scoped
91.8%
95% CI 89.793.6%
709/772 correct · 7.7% coverage
5
Qwen3.6-35B-A3B-GGUFQ2_K_XL · GGUFunknown protocol · unknown agent
unsloth/Qwen3.6-35B-A3B-GGUF · 30 runs · 13/13 shards · harness-scoped
89.3%
95% CI 88.789.9%
8971/10042 correct · 100.0% coverage
6
Qwen3.5-27B-Uncensored-HauhauCS-AggressiveQ4_K_M · GGUFunknown protocol · unknown agent
HauhauCS/Qwen3.5-27B-Uncensored-HauhauCS-Aggressive · 1 run · 1/13 shards · harness-scoped
90.7%
95% CI 88.492.5%
700/772 correct · 7.7% coverage
7
Qwen3.6-35B-A3B-Uncensored-HauhauCS-AggressiveQ2_K_P · GGUFunknown protocol · unknown agent
HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive · 13 runs · 13/13 shards · harness-scoped
85.4%
95% CI 84.786.1%
8579/10042 correct · 100.0% coverage
8
Qwen3.6-35B-A3B-Uncensored-HauhauCS-AggressiveQ4_K_M · GGUFunknown protocol · unknown agent
HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive · 13 runs · 13/13 shards · harness-scoped
85.0%
95% CI 84.385.7%
8537/10042 correct · 100.0% coverage
9
gemma-4-31B-it-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
unsloth/gemma-4-31B-it-GGUF · 1 run · 1/13 shards · harness-scoped
86.0%
95% CI 83.488.3%
665/773 correct · 7.7% coverage
10
Gemma-4-31B-IT-NVFP4unspecified quantunknown protocol · unknown agent
nvidia/Gemma-4-31B-IT-NVFP4 · 12 runs · 12/13 shards · harness-scoped
80.9%
95% CI 79.782.0%
3600/4452 correct · 44.3% coverage
11
Qwen3.6-27B-NVFP4NVFP4unknown protocol · unknown agent
nvidia/Qwen3.6-27B-NVFP4 · 13 runs · 13/13 shards · harness-scoped
79.8%
95% CI 78.680.9%
3848/4823 correct · 48.0% coverage
12
Qwen3.6-35B-A3B-GPTQ-Int4unspecified quantunknown protocol · unknown agent
palmfuture/Qwen3.6-35B-A3B-GPTQ-Int4 · 2 runs · 1/13 shards · harness-scoped
82.2%
95% CI 78.085.8%
305/371 correct · 3.7% coverage
13
Ornith-1.0-35B-PrismaAURA-4.75bit-vllm-MTPunspecified quantunknown protocol · unknown agent
rdtand/Ornith-1.0-35B-PrismaAURA-4.75bit-vllm-MTP · 2 runs · 2/13 shards · harness-scoped
80.2%
95% CI 77.282.9%
595/742 correct · 7.4% coverage
14
gemma-4-26B-A4B-it-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
unsloth/gemma-4-26B-A4B-it-GGUF · 1 run · 1/13 shards · harness-scoped
79.5%
95% CI 76.582.2%
614/772 correct · 7.7% coverage
15
Ornith-1.0-35B-AWQ-INT4unspecified quantunknown protocol · unknown agent
cyankiwi/Ornith-1.0-35B-AWQ-INT4 · 1 run · 1/13 shards · harness-scoped
80.6%
95% CI 76.384.3%
299/371 correct · 3.7% coverage
16
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressiveunspecified quantunknown protocol · unknown agent
HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive · 1 run · 1/13 shards · harness-scoped
79.1%
95% CI 76.181.9%
611/772 correct · 7.7% coverage
17
Qwen3.6-27B-AEON-Ultimate-Uncensored-Text-NVFP4-MTP-XSunspecified quantunknown protocol · unknown agent
AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-Text-NVFP4-MTP-XS · 1 run · 1/13 shards · harness-scoped
79.0%
95% CI 74.582.8%
293/371 correct · 3.7% coverage
18
Ornith-1.0-9Bunspecified quantunknown protocol · unknown agent
deepreinforce-ai/Ornith-1.0-9B · 2 runs · 2/13 shards · harness-scoped
84.1%
95% CI 70.692.1%
37/44 correct · 0.4% coverage
19
Laguna-XS-2.1-INT4INT4unknown protocol · unknown agent
poolside/Laguna-XS-2.1-INT4 · 1 run · 1/13 shards · harness-scoped
73.6%
95% CI 68.977.8%
273/371 correct · 3.7% coverage
20
Hy-MT2-7Bunspecified quantunknown protocol · unknown agent
tencent/Hy-MT2-7B · 4 runs · 4/13 shards · harness-scoped
65.0%
95% CI 60.269.5%
260/400 correct · 4.0% coverage
21
gemma-4-E4B-itunspecified quantunknown protocol · unknown agent
google/gemma-4-E4B-it · 3 runs · 3/13 shards · harness-scoped
65.3%
95% CI 59.770.5%
194/297 correct · 3.0% coverage
22
granite-4.0-h-microunspecified quantunknown protocol · unknown agent
ibm-granite/granite-4.0-h-micro · 1 run · 1/13 shards · harness-scoped
76.7%
95% CI 59.188.2%
23/30 correct · 0.3% coverage
23
EXAONE-3.5-7.8B-Instruct-GGUFunspecified quantunknown protocol · unknown agent
LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct-GGUF · 1 run · 1/13 shards · harness-scoped
73.3%
95% CI 55.685.8%
22/30 correct · 0.3% coverage
24
RWKV7-Goose-World3-2.9B-HFunspecified quantunknown protocol · unknown agent
RWKV/RWKV7-Goose-World3-2.9B-HF · 2 runs · 2/13 shards · harness-scoped
61.7%
95% CI 49.072.9%
37/60 correct · 0.6% coverage
25
Mellum2-12B-A2.5B-Instruct-GGUF-Q4_K_Munspecified quantunknown protocol · unknown agent
JetBrains/Mellum2-12B-A2.5B-Instruct-GGUF-Q4_K_M · 3 runs · 3/13 shards · harness-scoped
73.3%
95% CI 48.089.1%
11/15 correct · 0.1% coverage
26
MiniCPM3-4Bunspecified quantunknown protocol · unknown agent
openbmb/MiniCPM3-4B · 4 runs · 4/13 shards · harness-scoped
52.1%
95% CI 45.558.6%
114/219 correct · 2.2% coverage
27
gemma-4-12b-it-GGUFQ8_K_XL · gguf
unsloth/gemma-4-12b-it-GGUF · 1 run · 1/13 shards · harness-scoped
54.0%
95% CI 44.363.4%
54/100 correct · 1.0% coverage
28
gemma-4-E4B-it-qat-q4_0-ggufunspecified quantunknown protocol · unknown agent
google/gemma-4-E4B-it-qat-q4_0-gguf · 1 run · 1/13 shards · harness-scoped
83.3%
95% CI 43.697.0%
5/6 correct · 0.1% coverage
29
unsloth/gemma-4-26B-A4B-it-qat-GGUF · 1 run · 1/13 shards · harness-scoped
52.0%
95% CI 42.361.5%
52/100 correct · 1.0% coverage
30
gemma-4-E4B-it-OBLITERATEDunspecified quantunknown protocol · unknown agent
OBLITERATUS/gemma-4-E4B-it-OBLITERATED · 1 run · 1/13 shards · harness-scoped
100.0%
95% CI 20.7100.0%
1/1 correct · 0.0% coverage
31
Laguna-XS-2.1-NVFP4NVFP4unknown protocol · unknown agent
poolside/Laguna-XS-2.1-NVFP4 · 1 run · 1/13 shards · harness-scoped
100.0%
95% CI 20.7100.0%
1/1 correct · 0.0% coverage
32
MiniCPM5-1B-GGUFunspecified quantunknown protocol · unknown agent
openbmb/MiniCPM5-1B-GGUF · 3 runs · 2/13 shards · harness-scoped
30.0%
95% CI 19.942.5%
18/60 correct · 0.6% coverage
33
LFM2.5-1.2B-Instruct-GGUFunspecified quantunknown protocol · unknown agent
LiquidAI/LFM2.5-1.2B-Instruct-GGUF · 3 runs · 3/13 shards · harness-scoped
26.7%
95% CI 18.636.6%
24/90 correct · 0.9% coverage
34
MiniCPM5-1Bunspecified quantunknown protocol · unknown agent
openbmb/MiniCPM5-1B · 3 runs · 3/13 shards · harness-scoped
18.6%
95% CI 11.828.1%
16/86 correct · 0.9% coverage
35
LFM2.5-8B-A1B-GGUFunspecified quantunknown protocol · unknown agent
unsloth/LFM2.5-8B-A1B-GGUF · 3 runs · 3/13 shards · harness-scoped
28.6%
95% CI 8.264.1%
2/7 correct · 0.1% coverage

Stability— historical rerun transparency

Leaderboard rank uses the canonical latest approved answer per question_id. These metrics include historical submissions too, so reruns and changed answers are visible but do not drive rank.

Qwen3.6-27B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
1 historical run · 773 unique questions
Canonical
93.3%
Row avg
93.3%
Run avg
93.3%
Repeated
0
Changed
0
Qwopus3.6-27B-Coder-MTP-GGUFQ5_K_M · ggufunknown protocol · unknown agent
1 historical run · 772 unique questions
Canonical
92.9%
Row avg
92.9%
Run avg
92.9%
Repeated
0
Changed
0
Qwen3.6-35B-A3B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
26 historical runs · 10,042 unique questions
Canonical
90.4%
Row avg
90.2%
Run avg
90.2%
Repeated
10,042
Changed
273
Qwen3.5-27B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
1 historical run · 772 unique questions
Canonical
91.8%
Row avg
91.8%
Run avg
91.8%
Repeated
0
Changed
0
Qwen3.6-35B-A3B-GGUFQ2_K_XL · GGUFunknown protocol · unknown agent
30 historical runs · 10,042 unique questions
Canonical
89.3%
Row avg
89.3%
Run avg
89.0%
Repeated
12,326
Changed
152
Qwen3.5-27B-Uncensored-HauhauCS-AggressiveQ4_K_M · GGUFunknown protocol · unknown agent
1 historical run · 772 unique questions
Canonical
90.7%
Row avg
90.7%
Run avg
90.7%
Repeated
0
Changed
0
Qwen3.6-35B-A3B-Uncensored-HauhauCS-AggressiveQ2_K_P · GGUFunknown protocol · unknown agent
13 historical runs · 10,042 unique questions
Canonical
85.4%
Row avg
85.4%
Run avg
85.4%
Repeated
0
Changed
0
Qwen3.6-35B-A3B-Uncensored-HauhauCS-AggressiveQ4_K_M · GGUFunknown protocol · unknown agent
13 historical runs · 10,042 unique questions
Canonical
85.0%
Row avg
85.0%
Run avg
85.0%
Repeated
0
Changed
0
gemma-4-31B-it-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
1 historical run · 773 unique questions
Canonical
86.0%
Row avg
86.0%
Run avg
86.0%
Repeated
0
Changed
0
Gemma-4-31B-IT-NVFP4unspecified quantunknown protocol · unknown agent
12 historical runs · 4,452 unique questions
Canonical
80.9%
Row avg
80.9%
Run avg
80.9%
Repeated
0
Changed
0
Qwen3.6-27B-NVFP4NVFP4unknown protocol · unknown agent
13 historical runs · 4,823 unique questions
Canonical
79.8%
Row avg
79.8%
Run avg
79.8%
Repeated
0
Changed
0
Qwen3.6-35B-A3B-GPTQ-Int4unspecified quantunknown protocol · unknown agent
2 historical runs · 371 unique questions
Canonical
82.2%
Row avg
82.5%
Run avg
82.5%
Repeated
371
Changed
6
Ornith-1.0-35B-PrismaAURA-4.75bit-vllm-MTPunspecified quantunknown protocol · unknown agent
2 historical runs · 742 unique questions
Canonical
80.2%
Row avg
80.2%
Run avg
80.2%
Repeated
0
Changed
0
gemma-4-26B-A4B-it-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
1 historical run · 772 unique questions
Canonical
79.5%
Row avg
79.5%
Run avg
79.5%
Repeated
0
Changed
0
Ornith-1.0-35B-AWQ-INT4unspecified quantunknown protocol · unknown agent
1 historical run · 371 unique questions
Canonical
80.6%
Row avg
80.6%
Run avg
80.6%
Repeated
0
Changed
0
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressiveunspecified quantunknown protocol · unknown agent
1 historical run · 772 unique questions
Canonical
79.1%
Row avg
79.1%
Run avg
79.1%
Repeated
0
Changed
0
Qwen3.6-27B-AEON-Ultimate-Uncensored-Text-NVFP4-MTP-XSunspecified quantunknown protocol · unknown agent
1 historical run · 371 unique questions
Canonical
79.0%
Row avg
79.0%
Run avg
79.0%
Repeated
0
Changed
0
Ornith-1.0-9Bunspecified quantunknown protocol · unknown agent
2 historical runs · 44 unique questions
Canonical
84.1%
Row avg
84.1%
Run avg
86.4%
Repeated
0
Changed
0
Laguna-XS-2.1-INT4INT4unknown protocol · unknown agent
1 historical run · 371 unique questions
Canonical
73.6%
Row avg
73.6%
Run avg
73.6%
Repeated
0
Changed
0
Hy-MT2-7Bunspecified quantunknown protocol · unknown agent
4 historical runs · 400 unique questions
Canonical
65.0%
Row avg
65.0%
Run avg
58.7%
Repeated
0
Changed
0
gemma-4-E4B-itunspecified quantunknown protocol · unknown agent
3 historical runs · 297 unique questions
Canonical
65.3%
Row avg
65.3%
Run avg
65.1%
Repeated
0
Changed
0
granite-4.0-h-microunspecified quantunknown protocol · unknown agent
1 historical run · 30 unique questions
Canonical
76.7%
Row avg
76.7%
Run avg
76.7%
Repeated
0
Changed
0
EXAONE-3.5-7.8B-Instruct-GGUFunspecified quantunknown protocol · unknown agent
1 historical run · 30 unique questions
Canonical
73.3%
Row avg
73.3%
Run avg
73.3%
Repeated
0
Changed
0
RWKV7-Goose-World3-2.9B-HFunspecified quantunknown protocol · unknown agent
2 historical runs · 60 unique questions
Canonical
61.7%
Row avg
61.7%
Run avg
61.7%
Repeated
0
Changed
0
Mellum2-12B-A2.5B-Instruct-GGUF-Q4_K_Munspecified quantunknown protocol · unknown agent
3 historical runs · 15 unique questions
Canonical
73.3%
Row avg
73.3%
Run avg
77.4%
Repeated
0
Changed
0
MiniCPM3-4Bunspecified quantunknown protocol · unknown agent
4 historical runs · 219 unique questions
Canonical
52.1%
Row avg
52.1%
Run avg
75.3%
Repeated
0
Changed
0
gemma-4-12b-it-GGUFQ8_K_XL · gguf
0 historical runs · 100 unique questions
Canonical
54.0%
Row avg
Run avg
Repeated
0
Changed
0
gemma-4-E4B-it-qat-q4_0-ggufunspecified quantunknown protocol · unknown agent
1 historical run · 6 unique questions
Canonical
83.3%
Row avg
83.3%
Run avg
83.3%
Repeated
0
Changed
0
0 historical runs · 100 unique questions
Canonical
52.0%
Row avg
Run avg
Repeated
0
Changed
0
gemma-4-E4B-it-OBLITERATEDunspecified quantunknown protocol · unknown agent
1 historical run · 1 unique questions
Canonical
100.0%
Row avg
100.0%
Run avg
100.0%
Repeated
0
Changed
0
Laguna-XS-2.1-NVFP4NVFP4unknown protocol · unknown agent
1 historical run · 1 unique questions
Canonical
100.0%
Row avg
100.0%
Run avg
100.0%
Repeated
0
Changed
0
MiniCPM5-1B-GGUFunspecified quantunknown protocol · unknown agent
3 historical runs · 60 unique questions
Canonical
30.0%
Row avg
30.0%
Run avg
30.0%
Repeated
30
Changed
0
LFM2.5-1.2B-Instruct-GGUFunspecified quantunknown protocol · unknown agent
3 historical runs · 90 unique questions
Canonical
26.7%
Row avg
26.7%
Run avg
26.7%
Repeated
0
Changed
0
MiniCPM5-1Bunspecified quantunknown protocol · unknown agent
3 historical runs · 86 unique questions
Canonical
18.6%
Row avg
18.6%
Run avg
22.2%
Repeated
0
Changed
0
LFM2.5-8B-A1B-GGUFunspecified quantunknown protocol · unknown agent
3 historical runs · 7 unique questions
Canonical
28.6%
Row avg
28.6%
Run avg
22.2%
Repeated
0
Changed
0

Runs— sample traces per run

Qwen3.6-27B-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
by messerli-wallace · shard 11 · 7/24/2026, 11:46:25 AM · cmryvk15e04dro4015zykmjpi
93.3%
721/773 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 1 · hellaswag:00009544pass · 100.0% · 902ms · 9299df090c57
Question
[header] How to take action against capital punishment in your state [title] Focus your action on a specific case or cases. [step] It may help you to focus your efforts on a couple of current capital cases in your state. Focusing your work on a couple of specific cases may be more effective than tackling several cases or state laws as a whole.

A. However, you should also stay on top of general issues related to capital punishment. Combining this knowledge can help you become a very informed participant in the action against these cases, which in turn may help alter capital punishment laws.
B. Some of the cases in your state are relatively minor compared to other cases in your state. Still, you should aim to focus your efforts on a very specific case at all costs.
C. [substeps] You can look up places to locate a case. There is a wide variety of legal resources online that can help you locate the most common cases you may want to worry about.
D. [substeps] If you believe capital punishment in any of your cases is unjust, find a law that applies to those cases. Look at the organizations's forum links.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

[header] How to take action against capital punishment in your state [title] Focus your action on a specific case or cases. [step] It may help you to focus your efforts on a couple of current capital cases in your state. Focusing your work on a couple of specific cases may be more effective than tackling several cases or state laws as a whole.

A. However, you should also stay on top of general issues related to capital punishment. Combining this knowledge can help you become a very informed participant in the action against these cases, which in turn may help alter capital punishment laws.
B. Some of the cases in your state are relatively minor compared to other cases in your state. Still, you should aim to focus your efforts on a very specific case at all costs.
C. [substeps] You can look up places to locate a case. There is a wide variety of legal resources online that can help you locate the most common cases you may want to worry about.
D. [substeps] If you believe capital punishment in any of your cases is unjust, find a law that applies to those cases. Look at the organizations's forum links.
Agent trace
A
Extracted: A · Gold: A
sample 2 · hellaswag:00000706pass · 100.0% · 653ms · e449ba510e79
Question
The boy wipes down the sole of the tennis shoe using the wet wipe towel. The boy dries off the shoe using a towel. the boy

A. rinses the shoe inside the folded towel with water.
B. picks up a high top pair and wipes it off with the same wet towel.
C. tapes down the rear portion of the shoe.
D. shakes his foot with music and the camera follows the movements.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

The boy wipes down the sole of the tennis shoe using the wet wipe towel. The boy dries off the shoe using a towel. the boy

A. rinses the shoe inside the folded towel with water.
B. picks up a high top pair and wipes it off with the same wet towel.
C. tapes down the rear portion of the shoe.
D. shakes his foot with music and the camera follows the movements.
Agent trace
B
Extracted: B · Gold: B
sample 3 · hellaswag:00002205pass · 100.0% · 643ms · 8a810c69d8d6
Question
We see a man and the intro scene. A man in a kitchen has knifes and a sharpening block. the man

A. grinds the knifes through a rod.
B. begins sharpening the arm of a woman and releases.
C. wets the block and runs a knife across two different blocks.
D. then gets to chopping vegetables mainly in front of a counter with other scenes working.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

We see a man and the intro scene. A man in a kitchen has knifes and a sharpening block. the man

A. grinds the knifes through a rod.
B. begins sharpening the arm of a woman and releases.
C. wets the block and runs a knife across two different blocks.
D. then gets to chopping vegetables mainly in front of a counter with other scenes working.
Agent trace
C
Extracted: C · Gold: C
sample 4 · hellaswag:00006200pass · 100.0% · 760ms · a6d1d00acadc
Question
[header] How to wear yoga pants to work [title] Invest in high-quality yoga pants. [step] If you're wearing yoga pants to work, you want them to be durable while also providing comfort. Try to find yoga pants made of a thicker material, and try them on before purchasing them to ensure they aren't see-through.

A. High-quality yoga pants will likely be designed for yoga or other types of work. [title] Stick to a thin, stretchy, short-waist.
B. Sheer yoga pants are definitely something to avoid in the workplace. [substeps] Yoga leggings offer the most outfit options and tend to look the best in a professional work environment.
C. [substeps] Make sure the pants are specifically made for yoga, preferably ones with natural fibers, such as cotton or silk. When in doubt, look for yoga pants in solid colors.
D. Yoga pants come in a variety of shapes and sizes, and none of them fit under normal circumstances. See the rough guide section for more specific instructions.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

[header] How to wear yoga pants to work [title] Invest in high-quality yoga pants. [step] If you're wearing yoga pants to work, you want them to be durable while also providing comfort. Try to find yoga pants made of a thicker material, and try them on before purchasing them to ensure they aren't see-through.

A. High-quality yoga pants will likely be designed for yoga or other types of work. [title] Stick to a thin, stretchy, short-waist.
B. Sheer yoga pants are definitely something to avoid in the workplace. [substeps] Yoga leggings offer the most outfit options and tend to look the best in a professional work environment.
C. [substeps] Make sure the pants are specifically made for yoga, preferably ones with natural fibers, such as cotton or silk. When in doubt, look for yoga pants in solid colors.
D. Yoga pants come in a variety of shapes and sizes, and none of them fit under normal circumstances. See the rough guide section for more specific instructions.
Agent trace
B
Extracted: B · Gold: B
sample 5 · hellaswag:00002478pass · 100.0% · 642ms · 7a4de1d8c01b
Question
The ingredients are added to a tub and scoops of ice cream are taken from a tub after freezing. a bowl

A. is filled with ice cream.
B. of ice cream is added to the bowl and melted in four different pans.
C. of olive oil is added as a spoon and bowls of butter are added.
D. is used to rack ice cream.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

The ingredients are added to a tub and scoops of ice cream are taken from a tub after freezing. a bowl

A. is filled with ice cream.
B. of ice cream is added to the bowl and melted in four different pans.
C. of olive oil is added as a spoon and bowls of butter are added.
D. is used to rack ice cream.
Agent trace
A
Extracted: A · Gold: A
Incorrect samples
sample 13 · hellaswag:00000193fail · 0.0% · 635ms · ad62609faa87
Question
We see the title introducing us to jesse. jesse the dog

A. owner shows us how he walks and how he breathes.
B. runs and catches frisbee.
C. runs in circles where the dog is his mate.
D. is shown from a shampoo advertisement one last time.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

We see the title introducing us to jesse. jesse the dog

A. owner shows us how he walks and how he breathes.
B. runs and catches frisbee.
C. runs in circles where the dog is his mate.
D. is shown from a shampoo advertisement one last time.
Agent trace
A
Extracted: A · Gold: B
sample 67 · hellaswag:00000194fail · 0.0% · 646ms · 74274c193506
Question
We see the title introducing us to jesse. Jesse the dog runs and catches frisbee. we

A. see an opening title screen.
B. see a guy hold a blue frisbee in the air and tell that the dog has a pink frisbee and we see its pink feet.
C. see jesse close up in the camera.
D. see a violinist doing a piano clip.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

We see the title introducing us to jesse. Jesse the dog runs and catches frisbee. we

A. see an opening title screen.
B. see a guy hold a blue frisbee in the air and tell that the dog has a pink frisbee and we see its pink feet.
C. see jesse close up in the camera.
D. see a violinist doing a piano clip.
Agent trace
A
Extracted: A · Gold: C
sample 73 · hellaswag:00005959fail · 0.0% · 773ms · f156f68f027f
Question
[header] How to prevent kidney stones from recurring [title] Determine which kind of kidney stone you have had. [step] Ask your doctor to identify your particular kind. It is important to know which particular type you have had, so you can work on specific ways to prevent it from recurring.

A. For example, there is a more specific type of kidney stone commonly known as chronic kidney stones. The most common type of kidney stones are a cystine kidney stone.
B. [title] Think about who will be doing the surgery. [step] You will have to decide what type of surgery is most appropriate for you.
C. [title] Ask your doctor to study your kidney stones. [step] Your doctor will dissect your kidney stones and perform a simple exam to determine what type of kidney stone you have.
D. Ensure your doctor checks your parathyroid to rule it out as a factor in the formation of kidney stones. [substeps] Calcium stones are caused by unused calcium not flushed out in urine and is collected in the kidneys.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

[header] How to prevent kidney stones from recurring [title] Determine which kind of kidney stone you have had. [step] Ask your doctor to identify your particular kind. It is important to know which particular type you have had, so you can work on specific ways to prevent it from recurring.

A. For example, there is a more specific type of kidney stone commonly known as chronic kidney stones. The most common type of kidney stones are a cystine kidney stone.
B. [title] Think about who will be doing the surgery. [step] You will have to decide what type of surgery is most appropriate for you.
C. [title] Ask your doctor to study your kidney stones. [step] Your doctor will dissect your kidney stones and perform a simple exam to determine what type of kidney stone you have.
D. Ensure your doctor checks your parathyroid to rule it out as a factor in the formation of kidney stones. [substeps] Calcium stones are caused by unused calcium not flushed out in urine and is collected in the kidneys.
Agent trace
C
Extracted: C · Gold: D
sample 76 · hellaswag:00001952fail · 0.0% · 652ms · 47598edfccf2
Question
She points at the next set of cards. She pulls a card out and sets that on top of the cards. she

A. pulls another card out to add to hers.
B. pulls the yellow card out and puts both cards onto the white.
C. then takes a third one to take a turn making sure the cards are all lined up.
D. repeats the move in the next set of cards.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

She points at the next set of cards. She pulls a card out and sets that on top of the cards. she

A. pulls another card out to add to hers.
B. pulls the yellow card out and puts both cards onto the white.
C. then takes a third one to take a turn making sure the cards are all lined up.
D. repeats the move in the next set of cards.
Agent trace
D
Extracted: D · Gold: A
sample 81 · hellaswag:00001323fail · 0.0% · 650ms · 52f7d70516d1
Question
A person is seen climbing up on top of a piece of exercise equipment and performing a gymnastics routine. the person

A. continues spinning around on the equipment and ends by jumping down on the side.
B. spins and gets off and performs several moves on the machine.
C. follows the instructor all along the beam while spinning and doing acrobatics on the beam.
D. spins his body around and around on the bar and ends by jumping off and appearing back on the side of the machine.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

A person is seen climbing up on top of a piece of exercise equipment and performing a gymnastics routine. the person

A. continues spinning around on the equipment and ends by jumping down on the side.
B. spins and gets off and performs several moves on the machine.
C. follows the instructor all along the beam while spinning and doing acrobatics on the beam.
D. spins his body around and around on the bar and ends by jumping off and appearing back on the side of the machine.
Agent trace
D
Extracted: D · Gold: A
Qwen3.5-27B-Uncensored-HauhauCS-AggressiveQ4_K_M · GGUFunknown protocol · unknown agent
by messerli-wallace · shard 10 · 7/24/2026, 4:12:35 AM · cmryfcef8034uo4014illklwv
90.7%
700/772 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 1 · hellaswag:00001464pass · 100.0% · 625ms · ed75d45530df
Question
Several athletes compete for the javelin world championships. The russia dimitri is 3rd bronze medal in the competition. The japanese jenki is silver medal. till

A. javelin 1990 the russian javelin champion anna rubens is strong.
B. from germany won the gold medal.
C. loesos facility there is no competition.
D. now the french have won each javelin every round.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

Several athletes compete for the javelin world championships. The russia dimitri is 3rd bronze medal in the competition. The japanese jenki is silver medal. till

A. javelin 1990 the russian javelin champion anna rubens is strong.
B. from germany won the gold medal.
C. loesos facility there is no competition.
D. now the french have won each javelin every round.
Agent trace
B
Extracted: B · Gold: B
sample 2 · hellaswag:00005973pass · 100.0% · 703ms · 74355aa1fcb7
Question
[header] How to get japanese characters (kanji, hiragana, katakana) on firefox [title] In the firefox toolbar, go to view] character encoding] auto-detect. [step] While most browsers are already capable of reading international characters, you may simply need to help firefox detect them. [title] Select japanese.

A. [step] By default, most of the newer characters on a firefox list are japanese (or katakana). However, you may have to reach up and back to find ones you'd like to include or that make similarities between each other.
B. [title] Focus your attention on chikatakana. [title] Choose the gender and intent of the character, ren.
C. [step] Some of the same characters can be found on the scroll bar. Selected japanese is the most commonly used japanese toolbar.
D. [step] Your firefox should now be able to detect japanese characters. If not, you may need to install a language pack using one of the methods below.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

[header] How to get japanese characters (kanji, hiragana, katakana) on firefox [title] In the firefox toolbar, go to view] character encoding] auto-detect. [step] While most browsers are already capable of reading international characters, you may simply need to help firefox detect them. [title] Select japanese.

A. [step] By default, most of the newer characters on a firefox list are japanese (or katakana). However, you may have to reach up and back to find ones you'd like to include or that make similarities between each other.
B. [title] Focus your attention on chikatakana. [title] Choose the gender and intent of the character, ren.
C. [step] Some of the same characters can be found on the scroll bar. Selected japanese is the most commonly used japanese toolbar.
D. [step] Your firefox should now be able to detect japanese characters. If not, you may need to install a language pack using one of the methods below.
Agent trace
D
Extracted: D · Gold: D
sample 3 · hellaswag:00006664pass · 100.0% · 701ms · eb135f444051
Question
[header] How to make cappuccino foam [title] Pour brewed espresso into your cappuccino cup. [step] Although specifics will depend on the type of cappuccino and the person making it, cappuccinos are roughly one quarter espresso and three quarters steamed milk. Pour your desired amount of brewed espresso into a large cappuccino cup before you steam the milk.

A. [substeps] Instead of cappuccino add straight milk. Espresso can be iced later, and most cappuccino companies encourage it to be iced after brewing to sweeten the milk.
B. That way, you'll have everything ready so you can pour out the steamed milk while it's still piping hot. [title] Fill a steam pitcher with cold milk.
C. This will preserve your milk-stirring capacity throughout the rest of the process. [substeps] Try the recommended amount of coffee to get the right amount.
D. Start off with hot to ensure the coffee in your cup is sufficiently cold to immerse the foam. [substeps] It's best to use skim milk.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

[header] How to make cappuccino foam [title] Pour brewed espresso into your cappuccino cup. [step] Although specifics will depend on the type of cappuccino and the person making it, cappuccinos are roughly one quarter espresso and three quarters steamed milk. Pour your desired amount of brewed espresso into a large cappuccino cup before you steam the milk.

A. [substeps] Instead of cappuccino add straight milk. Espresso can be iced later, and most cappuccino companies encourage it to be iced after brewing to sweeten the milk.
B. That way, you'll have everything ready so you can pour out the steamed milk while it's still piping hot. [title] Fill a steam pitcher with cold milk.
C. This will preserve your milk-stirring capacity throughout the rest of the process. [substeps] Try the recommended amount of coffee to get the right amount.
D. Start off with hot to ensure the coffee in your cup is sufficiently cold to immerse the foam. [substeps] It's best to use skim milk.
Agent trace
B
Extracted: B · Gold: B
sample 4 · hellaswag:00008041pass · 100.0% · 786ms · faa29905ee8a
Question
[header] How to get an adult male to use his manners [title] Consider what direct and indirect demands you have been making. [step] Reflect on the balance of personal demands you are making to your partner, and evaluate whether that balance is just. [title] Be prepared to evaluate your viewpoint of the situation as objectively as possible.

A. [title] First, determine whether your partner was accurate in his assessment of his sexual capabilities and responses. [title] Discuss there are a number of different ways that men in modern society can show discretion, professionalism, and safety in the absence of sexual restraint.
B. [step] You have also made a choice that your partner must want and must follow. When you record and evaluate your demands, be sure you are or need to be able to speak with those who are making them.
C. [title] Consider your own perspective, not personal. [step] Communicating your viewpoint will provide you with invaluable resources.
D. [step] Don't go into the impending talk convinced that anything he might say is either wrong or not heartfelt. That won't help to solve the problem.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

[header] How to get an adult male to use his manners [title] Consider what direct and indirect demands you have been making. [step] Reflect on the balance of personal demands you are making to your partner, and evaluate whether that balance is just. [title] Be prepared to evaluate your viewpoint of the situation as objectively as possible.

A. [title] First, determine whether your partner was accurate in his assessment of his sexual capabilities and responses. [title] Discuss there are a number of different ways that men in modern society can show discretion, professionalism, and safety in the absence of sexual restraint.
B. [step] You have also made a choice that your partner must want and must follow. When you record and evaluate your demands, be sure you are or need to be able to speak with those who are making them.
C. [title] Consider your own perspective, not personal. [step] Communicating your viewpoint will provide you with invaluable resources.
D. [step] Don't go into the impending talk convinced that anything he might say is either wrong or not heartfelt. That won't help to solve the problem.
Agent trace
D
Extracted: D · Gold: D
sample 5 · hellaswag:00001931pass · 100.0% · 611ms · 09555ca41c4e
Question
A boy holds up a bottle of mouth wash. he

A. bends down and throws it.
B. takes the mouthwash and blows his horn.
C. uses the mouthwash to wash himself and two other teens.
D. drinks a cap full of the mouth wash.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

A boy holds up a bottle of mouth wash. he

A. bends down and throws it.
B. takes the mouthwash and blows his horn.
C. uses the mouthwash to wash himself and two other teens.
D. drinks a cap full of the mouth wash.
Agent trace
D
Extracted: D · Gold: D
Incorrect samples
sample 13 · hellaswag:00003095fail · 0.0% · 582ms · 8e69465cf67f
Question
A man stands in the middle of a stadium. crowds

A. are gathered around him as he spins around.
B. watch and cheer the man on.
C. watch from behind a curtain.
D. flash around him.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

A man stands in the middle of a stadium. crowds

A. are gathered around him as he spins around.
B. watch and cheer the man on.
C. watch from behind a curtain.
D. flash around him.
Agent trace
A
Extracted: A · Gold: B
sample 21 · hellaswag:00008766fail · 0.0% · 722ms · 5d958c74723a
Question
[header] How to get natural looking waves [title] Wash or mist your hair. [step] When braiding your hair to create waves, it is best to begin with damp hair. You may achieve this in one of two ways : [substeps] Wash your tresses prior to braiding your hair.

A. Tie off the ends of the braid and then let the loose hair dry slightly before washing it. Gradually do this for all hair types-curly, straight, with some parting.
B. When you begin braiding your hair, do not pat your locks dry. Instead, work a small amount of shampoo (coconut hair shampoo) into your hands and massage the shampoo into your hair.
C. In one, slightly wet your hair, then mist it with water. In the other, leave your hair damp and then mist your tresses gently with the palm of your hand.
D. Mist your hair with lukewarm water. [title] Combat frizz and snarls.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

[header] How to get natural looking waves [title] Wash or mist your hair. [step] When braiding your hair to create waves, it is best to begin with damp hair. You may achieve this in one of two ways : [substeps] Wash your tresses prior to braiding your hair.

A. Tie off the ends of the braid and then let the loose hair dry slightly before washing it. Gradually do this for all hair types-curly, straight, with some parting.
B. When you begin braiding your hair, do not pat your locks dry. Instead, work a small amount of shampoo (coconut hair shampoo) into your hands and massage the shampoo into your hair.
C. In one, slightly wet your hair, then mist it with water. In the other, leave your hair damp and then mist your tresses gently with the palm of your hand.
D. Mist your hair with lukewarm water. [title] Combat frizz and snarls.
Agent trace
C
Extracted: C · Gold: D
sample 48 · hellaswag:00000502fail · 0.0% · 600ms · babc51e64319
Question
A person is seen riding on roller blades looking into a trash can. people

A. are seen riding and hitting the sand while the camera captures them by adjusting.
B. ride around the sidewalk and push down at the bottom of the trash can while laughing to the camera.
C. stand around the camera looking around.
D. look around and watch the man skate.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

A person is seen riding on roller blades looking into a trash can. people

A. are seen riding and hitting the sand while the camera captures them by adjusting.
B. ride around the sidewalk and push down at the bottom of the trash can while laughing to the camera.
C. stand around the camera looking around.
D. look around and watch the man skate.
Agent trace
B
Extracted: B · Gold: D
sample 51 · hellaswag:00003012fail · 0.0% · 620ms · 8091aafc5334
Question
An introductory title screen with the video description is shown. an individual

A. sniffs a needle and proceeded to polish the drawn out ball of the pinata with a blue polish.
B. appears riding a horse on a field alongside a bull and using a blow torch to spark a firestorm throughout the horse's mane.
C. drizzles olive oil into a pan.
D. is being brushed by a hand and gives a tutorial on how to brush your hair.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

An introductory title screen with the video description is shown. an individual

A. sniffs a needle and proceeded to polish the drawn out ball of the pinata with a blue polish.
B. appears riding a horse on a field alongside a bull and using a blow torch to spark a firestorm throughout the horse's mane.
C. drizzles olive oil into a pan.
D. is being brushed by a hand and gives a tutorial on how to brush your hair.
Agent trace
D
Extracted: D · Gold: C
sample 69 · hellaswag:00001745fail · 0.0% · 599ms · cae57f614836
Question
A close up of a fishing pole is shown with a person reeling it in. others

A. are shown capturing the fish that he has captured in the area.
B. are shown mostly off in the water followed by more people reeling the fishing pole in.
C. kneel around him as he continues to reel.
D. are seen on the bank watching a calf as it is released.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

A close up of a fishing pole is shown with a person reeling it in. others

A. are shown capturing the fish that he has captured in the area.
B. are shown mostly off in the water followed by more people reeling the fishing pole in.
C. kneel around him as he continues to reel.
D. are seen on the bank watching a calf as it is released.
Agent trace
B
Extracted: B · Gold: C
gemma-4-31B-it-GGUFQ4_K_XL · GGUFunknown protocol · unknown agent
by messerli-wallace · shard 9 · 7/24/2026, 2:57:02 AM · cmrycn86l025qo401tv7zwnwj
86.0%
665/773 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 1 · hellaswag:00001856pass · 100.0% · 634ms · 45bb9092f5a7
Question
A group of men stand on top of logs, chopping away. The chopping legend speaks to the camera again. a couple of men

A. are in a group near a river, talking.
B. stand on the log top wearing protective gear.
C. chop some wood until one of them jumps off the log and raises his hands up in excitement.
D. are hanging up rocks above a river.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

A group of men stand on top of logs, chopping away. The chopping legend speaks to the camera again. a couple of men

A. are in a group near a river, talking.
B. stand on the log top wearing protective gear.
C. chop some wood until one of them jumps off the log and raises his hands up in excitement.
D. are hanging up rocks above a river.
Agent trace
C
Extracted: C · Gold: C
sample 2 · hellaswag:00008393pass · 100.0% · 734ms · 4b49737b6a39
Question
[header] How to use movies to help kids build character [title] Pick movies by age group. [step] While some younger kids are more mature, generally, you don't want a story line that will go right over their head. Similarly, you don't want to lose older kids by showing them movies that are too simplistic.

A. [substeps] Often, the best movie lines are set in the last century. While you may have super teens, you can find lots of great movies for kids online.
B. If possible, choose movies by kids you love, not kids they already like. [substeps] For instance, in the movie audrey hepburn 2 2000, the tables at breakfast [[ (18.
C. Try to pick something that works for the age the kid is at. [substeps] For kids 2 to 7 years old, make sure the story is fairly simple, and that plot supports the idea you're trying to get across, as it needs to be glaringly obvious.
D. Most of the time, kids get what they want early on. Take your age group into consideration as you come up with a few choices for your movie series.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

[header] How to use movies to help kids build character [title] Pick movies by age group. [step] While some younger kids are more mature, generally, you don't want a story line that will go right over their head. Similarly, you don't want to lose older kids by showing them movies that are too simplistic.

A. [substeps] Often, the best movie lines are set in the last century. While you may have super teens, you can find lots of great movies for kids online.
B. If possible, choose movies by kids you love, not kids they already like. [substeps] For instance, in the movie audrey hepburn 2 2000, the tables at breakfast [[ (18.
C. Try to pick something that works for the age the kid is at. [substeps] For kids 2 to 7 years old, make sure the story is fairly simple, and that plot supports the idea you're trying to get across, as it needs to be glaringly obvious.
D. Most of the time, kids get what they want early on. Take your age group into consideration as you come up with a few choices for your movie series.
Agent trace
C
Extracted: C · Gold: C
sample 3 · hellaswag:00007942pass · 100.0% · 777ms · 9f056ac51132
Question
[header] How to choose an alternative to a flower girl [title] Invite other members of the family to join your wedding party. [step] Instead of having young flower girls, consider using older women in your family, such as your grandma or great aunt. They'll love to be included in a special way, and the sight of your grandma throwing flower petals into the air is to sure to bring a smile to everyone's face.

A. [substeps] Older women can even invite their significant other as well. Just be sure the couple makes time to discuss the details in advance.
B. [title] Let everyone know you're having a flower girl themed party. [step] In several ways, this will make your party more fun.
C. [title] Read opinions from other members of the family. [step] Reading magazines and articles on the flower girl instead of relying on your friends for opinions can help you understand what happened that night in the picture.
D. [title] Give the role to a friend who volunteers. [step] One of your adult friends may find the idea of being a flower girl enchanting.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

[header] How to choose an alternative to a flower girl [title] Invite other members of the family to join your wedding party. [step] Instead of having young flower girls, consider using older women in your family, such as your grandma or great aunt. They'll love to be included in a special way, and the sight of your grandma throwing flower petals into the air is to sure to bring a smile to everyone's face.

A. [substeps] Older women can even invite their significant other as well. Just be sure the couple makes time to discuss the details in advance.
B. [title] Let everyone know you're having a flower girl themed party. [step] In several ways, this will make your party more fun.
C. [title] Read opinions from other members of the family. [step] Reading magazines and articles on the flower girl instead of relying on your friends for opinions can help you understand what happened that night in the picture.
D. [title] Give the role to a friend who volunteers. [step] One of your adult friends may find the idea of being a flower girl enchanting.
Agent trace
D
Extracted: D · Gold: D
sample 4 · hellaswag:00007936pass · 100.0% · 643ms · 6532e0cb430f
Question
[header] How to care for your skin before and after a brazilian wax [title] Prepare 24 hours before wax. [step] The day before your wax, exfoliate the area. You don't need to use a special scrub.

A. You can exfoliate regularly with a loofah, facial bar, or regular washcloth. [substeps] Do not exfoliate right before you wax your brows.
B. All you need is a washcloth or bath pouffe with soap. Make sure to do this at least a day before your wax.
C. [substeps] As you rub on and dry your skin afterwards, avoid washing with soap or body wash. [title] Exfoliate with a facial cleanser.
D. [substeps] To get the most benefit, scrub with a pumice stone. If you are not exfoliating, begin by exfoliating with a pumice stone the day before wax.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

[header] How to care for your skin before and after a brazilian wax [title] Prepare 24 hours before wax. [step] The day before your wax, exfoliate the area. You don't need to use a special scrub.

A. You can exfoliate regularly with a loofah, facial bar, or regular washcloth. [substeps] Do not exfoliate right before you wax your brows.
B. All you need is a washcloth or bath pouffe with soap. Make sure to do this at least a day before your wax.
C. [substeps] As you rub on and dry your skin afterwards, avoid washing with soap or body wash. [title] Exfoliate with a facial cleanser.
D. [substeps] To get the most benefit, scrub with a pumice stone. If you are not exfoliating, begin by exfoliating with a pumice stone the day before wax.
Agent trace
B
Extracted: B · Gold: B
sample 5 · hellaswag:00005966pass · 100.0% · 621ms · bcf502427c6c
Question
[header] How to make resurrection rolls [title] Pour the milk into a bowl. [step] To activate the yeast, you need to mix it with a warm liquid. Add ½ cup (118 ml) of warm milk to the bowl of a stand mixer.

A. [substeps] For a longer process, you might want to run the yeast in the sink before proceeding. [title] Gradually stir the milk and mixture with a whisk.
B. Mix until the milk is very light (around 110 ml). [substeps] If the milk is too soft for this recipe, add 1/2 cup (120 ml) of sour cream.
C. The milk should be 105 ° f (41 ° c). [substeps] You can use 1% or 2% milk, but whole milk usually results in the best rolls.
D. [substeps] If you have a hand mixer, you can make the dough for the roll yourself. You'll only need to make 2 cups (500 ml) of milk.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

[header] How to make resurrection rolls [title] Pour the milk into a bowl. [step] To activate the yeast, you need to mix it with a warm liquid. Add ½ cup (118 ml) of warm milk to the bowl of a stand mixer.

A. [substeps] For a longer process, you might want to run the yeast in the sink before proceeding. [title] Gradually stir the milk and mixture with a whisk.
B. Mix until the milk is very light (around 110 ml). [substeps] If the milk is too soft for this recipe, add 1/2 cup (120 ml) of sour cream.
C. The milk should be 105 ° f (41 ° c). [substeps] You can use 1% or 2% milk, but whole milk usually results in the best rolls.
D. [substeps] If you have a hand mixer, you can make the dough for the roll yourself. You'll only need to make 2 cups (500 ml) of milk.
Agent trace
C
Extracted: C · Gold: C
Incorrect samples
sample 39 · hellaswag:00001665fail · 0.0% · 488ms · 6e197d7fe41f
Question
A boy stands on a basketball court holding a ball. other players

A. make baskets on the court.
B. swarm the boy holding the ball.
C. are chasing after the ball.
D. approach in the background, then the boy in the court runs and gives a high jump.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

A boy stands on a basketball court holding a ball. other players

A. make baskets on the court.
B. swarm the boy holding the ball.
C. are chasing after the ball.
D. approach in the background, then the boy in the court runs and gives a high jump.
Agent trace
D
Extracted: D · Gold: B
sample 47 · hellaswag:00000374fail · 0.0% · 615ms · 3b944dadf258
Question
A boy is standing outside on top of ice fishing out of a small hole in the ice. The boy struggles to get the fish out and someone comes and takes it out for him. now there

A. are two fish and one of them is focused on freezing and he waves at the fish.
B. 's a woman standing inside the water watching him.
C. 's a scoop of ice cream and he eats it and grabs a small shovel.
D. are four men visible and they all begin to fish out of the ice pond.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

A boy is standing outside on top of ice fishing out of a small hole in the ice. The boy struggles to get the fish out and someone comes and takes it out for him. now there

A. are two fish and one of them is focused on freezing and he waves at the fish.
B. 's a woman standing inside the water watching him.
C. 's a scoop of ice cream and he eats it and grabs a small shovel.
D. are four men visible and they all begin to fish out of the ice pond.
Agent trace
B
Extracted: B · Gold: D
sample 58 · hellaswag:00003889fail · 0.0% · 543ms · 0eb28a157347
Question
[header] How to take care of ladybugs [title] Purchase ladybugs. [step] If you have a garden or a greenhouse, you should consider adding more ladybugs to the environment. Ladybugs are very useful for controlling aphids.

A. Aphids are tiny insects that can destroy vegetable and flower gardens. [substeps] Ladybugs tend to hibernate during winter months.
B. Ladybugs can be sold as small boxes, pots, or bowls. [substeps] Ladybugs come in all shapes and sizes.
C. You might try to keep ladybugs away from the garden. You can also buy ladybugs from a local store that sells insect products.
D. Ladybugs can be especially helpful to keep livestock and other animals away as well as washing your vegetable and insect products. [title] Purchase the ladybugs' natural habitats.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

[header] How to take care of ladybugs [title] Purchase ladybugs. [step] If you have a garden or a greenhouse, you should consider adding more ladybugs to the environment. Ladybugs are very useful for controlling aphids.

A. Aphids are tiny insects that can destroy vegetable and flower gardens. [substeps] Ladybugs tend to hibernate during winter months.
B. Ladybugs can be sold as small boxes, pots, or bowls. [substeps] Ladybugs come in all shapes and sizes.
C. You might try to keep ladybugs away from the garden. You can also buy ladybugs from a local store that sells insect products.
D. Ladybugs can be especially helpful to keep livestock and other animals away as well as washing your vegetable and insect products. [title] Purchase the ladybugs' natural habitats.
Agent trace
B
Extracted: B · Gold: A
sample 68 · hellaswag:00002241fail · 0.0% · 515ms · b4491db15e21
Question
The man walks up and down, straight across the yard as he mows the lawn. The child watches calmly while resting on the man's back. the man

A. continues to mow the lawn with his black lawn mower.
B. stops mowing and the boy attempts to emulate.
C. lays a sticker mascot on the ground then another before raking the grass into piles around the boy.
D. takes his shirt off and begins talking while talking to the boy.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

The man walks up and down, straight across the yard as he mows the lawn. The child watches calmly while resting on the man's back. the man

A. continues to mow the lawn with his black lawn mower.
B. stops mowing and the boy attempts to emulate.
C. lays a sticker mascot on the ground then another before raking the grass into piles around the boy.
D. takes his shirt off and begins talking while talking to the boy.
Agent trace
B
Extracted: B · Gold: A
sample 69 · hellaswag:00008153fail · 0.0% · 572ms · 694cdf059bd2
Question
[header] How to relax using the feather method [title] Lie on your back and close your eyes. [title] Rest your hands on your belly. [step] You can also lay your hands on the floor, several inches from your body, with your palms facing up.

A. Feet should be spread about shoulder-width apart, or whatever is most comfortable. [title] Take a moment to get comfortable so that you can lie still.
B. [title] Bring your arms over your head and apply pressure gently to your buttocks using your palms. [title] Bring your legs up towards your head and pull them to your chest.
C. [title] Spread your hands as wide as you can and spread your fingers wide. [step] You can use only your thumb and index or middle fingers.
D. Remember that belly breathing can soothe you! [title] Breathe deeply. [step] For some people, breathing can reduce anxiety and stress.
Rendered prompt
Answer the question. Reply with only the letter of the correct choice.

[header] How to relax using the feather method [title] Lie on your back and close your eyes. [title] Rest your hands on your belly. [step] You can also lay your hands on the floor, several inches from your body, with your palms facing up.

A. Feet should be spread about shoulder-width apart, or whatever is most comfortable. [title] Take a moment to get comfortable so that you can lie still.
B. [title] Bring your arms over your head and apply pressure gently to your buttocks using your palms. [title] Bring your legs up towards your head and pull them to your chest.
C. [title] Spread your hands as wide as you can and spread your fingers wide. [step] You can use only your thumb and index or middle fingers.
D. Remember that belly breathing can soothe you! [title] Breathe deeply. [step] For some people, breathing can reduce anxiety and stress.
Agent trace
D
Extracted: D · Gold: A
Laguna-XS-2.1-INT4INT4unknown protocol · unknown agent
by SgtPooki · shard 7 · 7/23/2026, 5:45:58 PM · cmrxsyjt1016uo401qbstotjk
73.6%
273/371 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 1 · hellaswag:00007843pass · 100.0% · 96ms · 2b00dd2efd79
Question
[header] How to cite a book [title] Familiarize yourself with the accepted citation styles. [step] Use only one style when citing a book reference-each style has very specific rules about capitalization, punctuation and placement of the facts. That said, all styles are designed for the same purpose: to give proper credit where credit is due.

A. It is not necessary to include more than your attention span. You could include reading journal articles (if they were used before) or writing your own start point book reviews (if they are not never published by the author).
B. Listed below are the most common styles : [substeps] Modern language association (mla) style. This is a citation style most often used within the liberal arts and humanities departments of universities.
C. [substeps] Use a capital letter or a comma after the reference terms of a book reference. This will refer to the same format in a sentence-to-sentence format.
D. [substeps] There are several great writers' styles for reference: paul golding, glenwhelp, dell, k. paul wilson (1960s innovator), william investments investors, allan cannon (1938), bob n.
Rendered prompt
[header] How to cite a book [title] Familiarize yourself with the accepted citation styles. [step] Use only one style when citing a book reference-each style has very specific rules about capitalization, punctuation and placement of the facts. That said, all styles are designed for the same purpose: to give proper credit where credit is due.
Agent trace
B
Extracted: B · Gold: B
sample 2 · hellaswag:00006278pass · 100.0% · 150ms · c6f497c808d7
Question
[header] How to be emo in middle school [title] Learn what it means to be emo. [step] Before you can become emo, you have to understand exactly what it means. " emo " is short for emotive, and people who consider themselves to be emo are usually in touch with their emotions.

A. They don't feel the need to hide how they feel, even when they are sad or angry. [substeps] While music, clothing, hair, and makeup can play a large role in being emo, being comfortable with your feelings and expressing them is the most important part.
B. This is an important step to emo, so have a good understanding of your gender. You don't have to have what everyone else has.
C. You don't need a dictionary dictionary and stick with the " emotive ". Instead, learn what emotive means, like " i am ", " i am skinny ", " i am strong and strong ", " i'm confident in my body ", etc.
D. You can't be emo without being emo. [substeps] If you're emo, you want an emo vibe that's unique to you.
Rendered prompt
[header] How to be emo in middle school [title] Learn what it means to be emo. [step] Before you can become emo, you have to understand exactly what it means. " emo " is short for emotive, and people who consider themselves to be emo are usually in touch with their emotions.
Agent trace
A
Extracted: A · Gold: A
sample 3 · hellaswag:00004307pass · 100.0% · 111ms · 0729af895014
Question
[header] How to make a smoke bomb [title] Gather three or four ping-pong balls. [step] For this method, you don't have to buy any extra ingredients other than basic household supplies, like aluminum foil, a pencil, screwdriver, and scissors. [title] Cut a hole in the top of one of the balls.

A. [step] You can poke a hole with a screwdriver or a knife. [title] Cut the other balls into small pieces and put them in the hole.
B. [step] Use a knick knack knife to create an x around one of the balls. [title] Punch a hole in the bottom of one of the balls.
C. [step] Align the hole in the top ball with the one that's down. [substeps] The hole needs to be about six-ten inches deep and about two inches wide.
D. [step] Grab a piece of foil about 15 inches from the edge of the ball. Get a zig-zag or intricate pattern.
Rendered prompt
[header] How to make a smoke bomb [title] Gather three or four ping-pong balls. [step] For this method, you don't have to buy any extra ingredients other than basic household supplies, like aluminum foil, a pencil, screwdriver, and scissors. [title] Cut a hole in the top of one of the balls.
Agent trace
A
Extracted: A · Gold: A
sample 4 · hellaswag:00003431pass · 100.0% · 114ms · 727d315e3ed2
Question
[header] How to apply shadow on hooded eyes [title] Prime your eyelids. [step] While this is important for everyone, it's especially critical for those with hooded eyes. Primer creates a base for your makeup that helps it stay put and last all day.

A. [substeps] If you use liquid primer, try it first before you start applying it on your eyes. Otherwise, you'll try a different primer beforehand.
B. Primer also keeps your eyelids from getting heavy and swollen. To prime your eyelids : [substeps] Apply an eyeshadow to your lids using a dark gray eye liner or gel instead of a cream or gel liner.
C. Because hooded eyes are prone to smudging and smearing, primer can make all the difference. [substeps] Use your fingertip to apply the primer, and allow it to absorb into your skin for a minute or so before proceeding.
D. Apply a primer to your eyelids and eyelid over the primer you applied earlier. [substeps] Use your powder brush, primer, or a makeup sponge to carefully prime your eyelids.
Rendered prompt
[header] How to apply shadow on hooded eyes [title] Prime your eyelids. [step] While this is important for everyone, it's especially critical for those with hooded eyes. Primer creates a base for your makeup that helps it stay put and last all day.
Agent trace
C
Extracted: C · Gold: C
sample 5 · hellaswag:00004697pass · 100.0% · 113ms · 913f8d7c63a6
Question
[header] How to get favorites on your deviantart photo [title] Create your own username. [step] Have a name that can be easily recognized and defines you as a person. [substeps] Your username doesn't have to be your first name, but some well-known deviantart members remember their art style through their da names.

A. Avoid using names of famous celebrities or character from movies, video games, or animes. To change your username, you have to purchase premium membership.
B. If you don't want people to recognize you as a person, try to keep the name honest and easy to identify. You can do this by including your full name and what year it is, but stay away from androgynous names.
C. For instance, president christian founder of the white house had a username that " peter " (modern), and there are so many other names available today. In the wild, some names are simply variations of the last name.
D. [title] Create usernames for your favorite characters. [step] Depending on your theme and design, you may have many different options.
Rendered prompt
[header] How to get favorites on your deviantart photo [title] Create your own username. [step] Have a name that can be easily recognized and defines you as a person. [substeps] Your username doesn't have to be your first name, but some well-known deviantart members remember their art style through their da names.
Agent trace
A
Extracted: A · Gold: A
Incorrect samples
sample 6 · hellaswag:00002238fail · 0.0% · 106ms · fec39366eba5
Question
The man lays down and had a hard time breathing. After putting down sand he drinks a warm beverage. we

A. see the man underwater swimming.
B. see the ending credits screen.
C. then see two guys swimming nak * d and drinking.
D. see no other man and the other man is shown shirtless.
Rendered prompt
The man lays down and had a hard time breathing. After putting down sand he drinks a warm beverage. we
Agent trace
D
Extracted: D · Gold: B
sample 8 · hellaswag:00002350fail · 0.0% · 98ms · 2e760a84b9b4
Question
People are sitting on tubes in the snow. they

A. stand in a line and start going down a snowy hill.
B. are going over a large hill.
C. begin sledding down the hill on the tubes.
D. stand on the top of the tubes.
Rendered prompt
People are sitting on tubes in the snow. they
Agent trace
A
Extracted: A · Gold: C
sample 14 · hellaswag:00002753fail · 0.0% · 99ms · ac78d002ae21
Question
A mom is sitting at the top of a slide with her little baby. she

A. slides down one at a time, and she tends to make a lot of noise.
B. is moving backward on the playground, walking away.
C. pulls out several balls as she slides down.
D. is contemplating going down for a while.
Rendered prompt
A mom is sitting at the top of a slide with her little baby. she
Agent trace
A
Extracted: A · Gold: D
sample 19 · hellaswag:00000609fail · 0.0% · 101ms · 0df9993b1b59
Question
A man and a woman are in a ring wrestling while a referee is standing in the corner of the ring along with a woman holding a wrestling belt. man and woman

A. are doing wrestling movements in a ring in a dark room and a referee is waching and screaming at them.
B. are wrestling in punches and kicking in a ring of mats in front of judges.
C. are holding a wrestling pole as the referee is standing on the other side of the ring.
D. are standing on opposite ends of the ring holding each other's belts in their hands while lifting each other up in the air upside down.
Rendered prompt
A man and a woman are in a ring wrestling while a referee is standing in the corner of the ring along with a woman holding a wrestling belt. man and woman
Agent trace
C
Extracted: C · Gold: A
sample 23 · hellaswag:00000739fail · 0.0% · 97ms · a9f24f5e2f1e
Question
A man talks in front of a house. A man climbs a ladder and looks at a roof. A man goes through a stack of papers. it

A. is then announcin aluminum on a table.
B. shows the papers on the screen.
C. ends with credits and action figures on the screen attached to a fridge.
D. is then night time again.
Rendered prompt
A man talks in front of a house. A man climbs a ladder and looks at a roof. A man goes through a stack of papers. it
Agent trace
C
Extracted: C · Gold: B
Laguna-XS-2.1-NVFP4NVFP4unknown protocol · unknown agent
by SgtPooki · shard 6 · 7/23/2026, 5:28:07 PM · cmrxsblvo010yo401tbeam6ug
100.0%
1/1 correct · 1 correct traces · 0 incorrect traces
Correct samples
sample 1 · hellaswag:00000731pass · 100.0% · 224ms · 0190bd41cc1f
Question
Maria is demonstrating how to make a nyquil cocktail drink. she

A. shows the ingredients required for the drink.
B. pours it through a straw into a glass and washes it down with a cup of soda water.
C. shows each scale of the cocktail by adding different colored liquids.
D. pours the ice in a small glass and squirts lemon juice into the glass.
Rendered prompt
Maria is demonstrating how to make a nyquil cocktail drink. she
Agent trace
A
Extracted: A · Gold: A
Ornith-1.0-35B-PrismaAURA-4.75bit-vllm-MTPunspecified quantunknown protocol · unknown agent
by SgtPooki · shard 8 · 7/20/2026, 3:27:30 PM · cmrtdoxgn0875lg01o58jkzx2
83.0%
308/371 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 1 · hellaswag:00008131pass · 100.0% · 306ms · a4ac87a8acc9
Question
[header] How to flex lats [title] Learn how to isolate your lats when flexing. [step] Your lats are a pair of wide muscles that run from your armpits to your lower spine, wrapping around the back of your ribcage just beneath your shoulder blades. To figure out how to isolate your lats, first stand with your feet together.

A. Your toes should be together, with your butt facing downward. Close your abdominal muscles in the groin to get into this position.
B. Hold your ankles firmly in your right hand and your ankles together with your left hand with your hands overlapped, in a criss cross pattern of parallel and vertical lines. [title] Use a ruler to identify your lats.
C. Squat forward so that your back and front ribs are aligned and your toes are parallel with the floor. Raise your elbows at an angle and bend your elbow at 90 degrees.
D. Arch your lower back so that your chest is pushed forward and your butt is pushed out. Spread your upper back by lifting your arms slightly away from your sides and holding your hands about 6 inches (15 cm) in front of your hips.
Rendered prompt
[header] How to flex lats [title] Learn how to isolate your lats when flexing. [step] Your lats are a pair of wide muscles that run from your armpits to your lower spine, wrapping around the back of your ribcage just beneath your shoulder blades. To figure out how to isolate your lats, first stand with your feet together.
Agent trace
D
Extracted: D · Gold: D
sample 3 · hellaswag:00009876pass · 100.0% · 185ms · 019b1852fb0c
Question
[header] How to show cleavage [title] Wear tops that show off your cleavage. [step] While this may seem like a given, the right top is absolutely essential when showing cleavage. You can't show something that's covered up.

A. Also, make sure your tops are not g-rated, meaning those aren't appropriate for a particular time of day, or if you have an event coming up soon. [substeps] If you love showing cleavage, you want to show off that.
B. Make sure you're comfortable with your body in the top so you can show it up. [substeps] Studies show that flat strapless tops are considered the sexiest choice for showing cleavage.
C. While a turtleneck sweater may make your bust look fantastic, it will hide your cleavage. Make sure what you're wearing is cut for visible cleavage.
D. Ask a friend if she'd be okay with a top showing their cleavage, and if she says yes! [title] Go for a sleek cut. [step] The summer and winter months are the ideal time to show your cleavage.
Rendered prompt
[header] How to show cleavage [title] Wear tops that show off your cleavage. [step] While this may seem like a given, the right top is absolutely essential when showing cleavage. You can't show something that's covered up.
Agent trace
C
Extracted: C · Gold: C
sample 4 · hellaswag:00003669pass · 100.0% · 419ms · a1faa682bdfb
Question
[header] How to enjoy your birthday [title] If you like planning, then plan how you'll spend the day at least a week ahead. [step] Check that your friends and family are free for the day and make sure you choose things to do that can involve the people you'd like to spend time with. [title] Get plenty of sleep.

A. [step] A really good time to get a full night's sleep is about 7 am; the rest of the night should be uninterrupted. It is also good to unwind a bit in the evening and can be done so that you can think about each day.
B. [step] And make sure to get 7-9 hours of sleep a night-1 1/2 hours for adults and 4-9 hours for children. You can even make it up to four hours of sleep on weekends.
C. [step] The day before your birthday, go to bed at a reasonable hour so that you can get the sleep needed to guarantee that you'll have lots of energy on your birthday. [title] Treat yourself to an awesome birthday outfit.
D. [step] The best things you can do for a birthday is to get a good quality sleep. Your body clock is only going to be able to tell you the exact amount of sleep you need.
Rendered prompt
[header] How to enjoy your birthday [title] If you like planning, then plan how you'll spend the day at least a week ahead. [step] Check that your friends and family are free for the day and make sure you choose things to do that can involve the people you'd like to spend time with. [title] Get plenty of sleep.
Agent trace
C
Extracted: C · Gold: C
sample 5 · hellaswag:00005144pass · 100.0% · 588ms · 73c837dec8a2
Question
[header] How to succeed in distance learning [title] Familiarize yourself with the syllabus. [step] Think of your syllabus as the blueprint for your class. It should contain all of the class guidelines and rules, as well as what your instructor expects from you over the course of the semester.

A. If you're having trouble understanding all of the syllabus instructions, ask your instructor for advice. [substeps] To become a successful athlete, you have to have a strong understanding of the game and field, and then, with that knowledge, bring out your best on a single outing.
B. Include these with your notes, papers, assignments and applications so you'll be up to date and able to fully experience your classroom. [substeps] Do a quick background check on your syllabus for your instructor.
C. Bring this information to the syllabus, stating all required daily requirements, as well as any tests you may have scheduled or deadlines you must meet for your session. This way, you can easily adjust any assignments you may finish before the next session starts.
D. Make sure you read through this document in its entirety and keep track of it all semester. [substeps] Ask your instructor any questions you might have about the syllabus.
Rendered prompt
[header] How to succeed in distance learning [title] Familiarize yourself with the syllabus. [step] Think of your syllabus as the blueprint for your class. It should contain all of the class guidelines and rules, as well as what your instructor expects from you over the course of the semester.
Agent trace
D
Extracted: D · Gold: D
sample 6 · hellaswag:00005768pass · 100.0% · 526ms · a8fb349bdd6f
Question
[header] How to make spicy chicken bites [title] Preheat the oven to 450º fahrenheit (230º celsius). [title] Line a large baking sheet with parchment paper. [title] Combine the chicken bites with the garlic and paprika.

A. [title] Season the chicken bites with salt and freshly ground black pepper. [title] Stir together the chickpeas, seasoning with nutmeg, currants, cumin and dried ginger in a mixing bowl.
B. [step] Toss the ingredients together in a large bowl until fully coated. [title] Toss in the flour and fold together until well-coated.
C. [title] Stir in ground beef and chicken bouillon and stir. [title] Add the oil and cook it for 2 minutes.
D. [title] Spread the chicken bites over the coated baking sheet. [title] Bake until they've cooked.
Rendered prompt
[header] How to make spicy chicken bites [title] Preheat the oven to 450º fahrenheit (230º celsius). [title] Line a large baking sheet with parchment paper. [title] Combine the chicken bites with the garlic and paprika.
Agent trace
B
Extracted: B · Gold: B
Incorrect samples
sample 2 · hellaswag:00009130fail · 0.0% · 470ms · 493709792d71
Question
[header] How to apply makeup for a glamour photography shoot [title] Clean your face. [step] Always start with a clean face. Leave no traces of your previous makeup.

A. If you will use a pore-cleaning strip, do it the day before to avoid makeup get caught in your newly cleansed pores. Exfoliate to feel that silky smooth texture of your skin.
B. Do not blush, define, or dry dry your face before applying eyeliner in this photoshoot. [title] Apply foundation and concealer to your face.
C. Inspect your face for blemishes with a makeup primer or something like that. Use a gentle cleanser and carefully pat your face dry with a soft soft cloth.
D. A healthy glow has been built up in your face over the past day. In order to achieve the glow, begin by exfoliating your face with a gentle face cleanser.
Rendered prompt
[header] How to apply makeup for a glamour photography shoot [title] Clean your face. [step] Always start with a clean face. Leave no traces of your previous makeup.
Agent trace
C
Extracted: C · Gold: A
sample 10 · hellaswag:00004482fail · 0.0% · 452ms · 37232b0902b6
Question
[header] How to annoy your roommate [title] Leave trash around the room. [step] First of all, this will make your room smell bad. Secondly, it'll attract flies, ants, and other bugs, which will gross your roommate out.

A. She will now be forced to clean up your mess or will leave it there, and it will annoy her. [substeps] Avoid leaving anything you might also find gross.
B. Leave trash around the room until it's gone! Remind your roommate to check their room frequently for trash. Finally, move a few boxes that are not worth moving around, from your room all the way to another part of the house to make room for trash.
C. If you're really into it, leave garbage around your walls and furniture and knock around some boxes. [substeps] Alternatively, if you're in a particularly bad mood, you could hang out with a friend who's unlikely to find trash, like a teddy bear.
D. No one likes to be around these things anyway, so if you act like an annoyance right from the start, people will start to go around and pick up a lot of clutter. [substeps] Take a small trash can and fill it full of personal trash, such as toilet paper, empty paint cans, and trash bags.
Rendered prompt
[header] How to annoy your roommate [title] Leave trash around the room. [step] First of all, this will make your room smell bad. Secondly, it'll attract flies, ants, and other bugs, which will gross your roommate out.
Agent trace
B
Extracted: B · Gold: A
sample 13 · hellaswag:00008048fail · 0.0% · 439ms · c074de6e1eea
Question
[header] How to be a strict mom [title] Make it clear what the rules are. [step] Do not expect your children to read your mind. When you set rules, explain them to the entire household.

A. Enforce your limits and rules in certain ways. For example, if your child always goes to school without adult supervision, make it clear that this behavior happens every day and is to be expected.
B. For example, if there are four children in the house, explain that children should only have access to the least amount of supervision. If children are sick or injured, they should be allowed in a daycare.
C. You can even post certain rules on the fridge. This will help your children understand how to act in the house and why you are disciplining them when they get in trouble.
D. Ask your children to follow the rules one way and then follow them the other way. Make sure that everyone understands that there are consequences when the rules are ignored.
Rendered prompt
[header] How to be a strict mom [title] Make it clear what the rules are. [step] Do not expect your children to read your mind. When you set rules, explain them to the entire household.
Agent trace
D
Extracted: D · Gold: C
sample 25 · hellaswag:00008881fail · 0.0% · 445ms · 260d30dff137
Question
[header] How to keep curly hair healthy [title] Condition your curly hair. [step] Conditioning may be the most important thing you can do for curly hair. Curly hair is already more prone to dryness and damage/breakage than other types.

A. So keeping it moisturized with conditioner is essential for healthy curly hair. [substeps] You can condition your hair while in the shower, or try some leave-in conditioner.
B. To keep your hair healthy, there are several things you can do : [substeps] Condition your hair at least twice a week. If you are going out during the summer, you may need to condition your hair two to three times a week.
C. Avoid also washing damp or otherwise damaged hair. [title] Learn how much conditioner you need.
D. However, working out or air drying your hair may make it crisper and grow out longer. [substeps] If you also want to style your hair, you may choose to wear dry hair.
Rendered prompt
[header] How to keep curly hair healthy [title] Condition your curly hair. [step] Conditioning may be the most important thing you can do for curly hair. Curly hair is already more prone to dryness and damage/breakage than other types.
Agent trace
B
Extracted: B · Gold: A
sample 41 · hellaswag:00009889fail · 0.0% · 429ms · 5cba7c64cdb3
Question
[header] How to detangle hair without pain [title] Check for dryness. [step] Before you attempt to take out any tangles, make sure your hair is not severely dry. This will make it less painful when you are actually taking the knots out.

A. Run your fingers through your hair to see if it's slightly sticky to the touch. If it is, try rubbing some old hair conditioner into it.
B. Run your fingers through your hair to ensure that it is not too damp and that it is looking sleek. [substeps] Normal hair can feel rough or lifeless if it is over-dry.
C. [title] Separate the hair completely if you have it tied up and leave it free. [step] Attempt to free any loose hairs from a tangled clump.
D. Wet hair tends to dry very quickly and can cause less hair damage. [substeps] If the oil in your hair helps to detangle it while you're detangling it, hold it over a sink or tub and then gently pull the roots and tips of your hair downward to remove as much oil as possible.
Rendered prompt
[header] How to detangle hair without pain [title] Check for dryness. [step] Before you attempt to take out any tangles, make sure your hair is not severely dry. This will make it less painful when you are actually taking the knots out.
Agent trace
A
Extracted: A · Gold: C
Ornith-1.0-35B-PrismaAURA-4.75bit-vllm-MTPunspecified quantunknown protocol · unknown agent
by SgtPooki · shard 5 · 7/20/2026, 2:26:02 PM · cmrtbhvta07pglg01l2olj1hu
77.4%
287/371 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 1 · hellaswag:00004149pass · 100.0% · 607ms · 53c18ea6f57a
Question
[header] How to survive financial hardship in a marriage [title] Remember your wedding vows ,' for better, for worse, for richer, for poorer'. [step] When we make these vows to each other it probably doesn't enter our minds what could happen down the road, but nonetheless we made a promise that we had the best intentions of keeping. During difficult financial times those vows become far more prominent and necessary.

A. [title] Remember not to shut doors at all hours. [step] Shut off all lights at any time, especially those that serve to light up your wedding day.
B. Stick with it not because of the promise you made but because of the person you made it to. [title] Remember that as with everything in life, it will turn around again.
C. We finally remember that our intentions weren't necessarily good and neither were your intentions. A discharge of our vows is a second step towards firbility.
D. [substeps] Write them down for future reference (or on a reference sheet). The more you understand the meaning of financial hardship, the greater your chances at survival.
Rendered prompt
[header] How to survive financial hardship in a marriage [title] Remember your wedding vows ,' for better, for worse, for richer, for poorer'. [step] When we make these vows to each other it probably doesn't enter our minds what could happen down the road, but nonetheless we made a promise that we had the best intentions of keeping. During difficult financial times those vows become far more prominent and necessary.
Agent trace
B
Extracted: B · Gold: B
sample 2 · hellaswag:00009152pass · 100.0% · 710ms · ca9e02613e2c
Question
[header] How to plan your wedding music [title] Sit down together and talk about what music matters to you. [step] Each of you will have different ideas and preferences and this is about getting the main ideas heard, to allow both of you to plan in full knowledge of what matters to one another. [substeps] Write down a list of the songs and/or music that you would like to form a part of your wedding.

A. Make sure to decide ahead of time the location where you want to arrange your wedding music. You can discuss how many romantic themes you hope to include in each song and set up time for each venue.
B. Ask your future spouse to do the same. Bring the lists along to your discussion together.
C. Brainstorm a logical order together with your partner. Everyone should have their own wants and preferences and think about what exactly matters to them, in order to get included.
D. Or you can look through the music previously choreographed and combine the music together to put into song form. [title] Decide how the music matters to you.
Rendered prompt
[header] How to plan your wedding music [title] Sit down together and talk about what music matters to you. [step] Each of you will have different ideas and preferences and this is about getting the main ideas heard, to allow both of you to plan in full knowledge of what matters to one another. [substeps] Write down a list of the songs and/or music that you would like to form a part of your wedding.
Agent trace
B
Extracted: B · Gold: B
sample 3 · hellaswag:00009645pass · 100.0% · 919ms · 3cbf0d8ef9b1
Question
[header] How to dress in knee length skirts [title] Pick a length that flatters your height. [step] Not everyone has the same body type, and there are some ways you can make a knee-length skirt look better for yours. [substeps] Short women or girls look really good in knee-length skirts that fall right below the knees.

A. When it comes to skirts that hit at your hip, go around the waist to accentuate it. Knee-length skirts are more flattering when they have structure and a twist.
B. Not everyone likes the hemlines of a waist-length skirt, and you don't want the skirt to stop too low on their legs, or they'll look overly babyish. If you're going out to dinner, go for a skirt that reaches the mid-thigh.
C. It's not true that short women can't wear knee-length skirts. If you're very tall, you will look great in a skirt that falls right at the mid-calf point.
D. Medium petite women or girls that are fairly tall don't care if their feet are bare, and may prefer wearing knee-length skirts with ankle-length tights. For a more feminine look, opt for a length that hits at the knee and curves over your thighs.
Rendered prompt
[header] How to dress in knee length skirts [title] Pick a length that flatters your height. [step] Not everyone has the same body type, and there are some ways you can make a knee-length skirt look better for yours. [substeps] Short women or girls look really good in knee-length skirts that fall right below the knees.
Agent trace
C
Extracted: C · Gold: C
sample 4 · hellaswag:00009316pass · 100.0% · 292ms · 2ffafd95b142
Question
[header] How to look like ashley barnum from' perfect ' [title] Get the look. [step] Ashley has blonde hair that ranges from light brown to almost white-blonde near her face. It's described as being' surfer girl' hair, so you should try using the product' surfer hair' by garnier.

A. She also has super-shiny and glossy hair, and it is revealed in the book that she rinses her hair with white vinegar so that it becomes shinier. Now this actually works, so consider it: you will need 1 part white vinegar mixed with three parts cold water.
B. For kit kerr, you need to mix emo' surfer' with ash blonde, but make sure you show your true color. Ashy hair works best, but you can try out the' pure girl' look for makeup and powdering.
C. If you can't find this extremely attractive look, use straighten-up shampoo and conditioner. If you don't like the appearance of her hair, have a good waxing or body hair section done, with a paddle-like shape and wavy roots, to create this look.
D. Hazel eyes that are never quite big are her signature. Look for a hair dye made for fair to medium hair.
Rendered prompt
[header] How to look like ashley barnum from' perfect ' [title] Get the look. [step] Ashley has blonde hair that ranges from light brown to almost white-blonde near her face. It's described as being' surfer girl' hair, so you should try using the product' surfer hair' by garnier.
Agent trace
A
Extracted: A · Gold: A
sample 5 · hellaswag:00009851pass · 100.0% · 397ms · d1231ef6dbf2
Question
[header] How to set a digital watch [title] Contact the manufacturer or look up a manual for complicated watches. [step] If your watch is especially complicated and you don't have instructions, look up a digital manual online. Digital manuals can be found with an online keyword search for your watch brand and model.

A. [substeps] Some watches turn off automatically during the day. If yours is in the morning, switch it to the middle of the night.
B. Your watch may take you a while, or it may take a little while to get to a level where you won't experience any problem. [substeps] Digital watches come with online cameras and camera boxes.
C. [substeps] If a digital manual is impossible to find, look up and call the customer service number for your watch. Model and brand information are usually written or engraved on the back of the watch or somewhere on the watch face.
D. They are usually $15-$40. They cover the exact specs of your watch and include instructions to change them.
Rendered prompt
[header] How to set a digital watch [title] Contact the manufacturer or look up a manual for complicated watches. [step] If your watch is especially complicated and you don't have instructions, look up a digital manual online. Digital manuals can be found with an online keyword search for your watch brand and model.
Agent trace
C
Extracted: C · Gold: C
Incorrect samples
sample 9 · hellaswag:00008838fail · 0.0% · 798ms · a2786e15c843
Question
[header] How to wash short hair [title] Pay attention to your scalp. [step] Identify the kind of scalp you have. The type of scalp you possess affects the types of shampoo and conditioner products you should be purchasing for healthy short hair.

A. Look closely at the bottle before you buy. Whether the product is hydrating, clarifying, or moisturizing, its main properties should be listed right on the bottle for you.
B. [substeps] Note whether your hair tends to be oily or dry. Oily hair is a much higher risk for hair cancer than dry hair.
C. As part of your hair care regimen, you'll need to care for your scalp. There are many different types of hair, and choosing a hairstyle and style that reflects the kinds of hair you have will give you total control over your hair quality.
D. Depending on your hair type, you may have none/none of the following symptoms, so pay attention. [substeps] Oily hair forms oily or puffiness when it is blended with other healthy hair products.
Rendered prompt
[header] How to wash short hair [title] Pay attention to your scalp. [step] Identify the kind of scalp you have. The type of scalp you possess affects the types of shampoo and conditioner products you should be purchasing for healthy short hair.
Agent trace
C
Extracted: C · Gold: A
sample 12 · hellaswag:00002308fail · 0.0% · 749ms · 5fa57e9e6aa8
Question
A man is knelling down on snow covered ice. The man points to a whole in the ice. the man

A. then digs through some snow on the ground.
B. drills some holes in the snow.
C. then retrieves a line from the line in the ice exposing a caught fish.
D. crawls across the ice.
Rendered prompt
A man is knelling down on snow covered ice. The man points to a whole in the ice. the man
Agent trace
D
Extracted: D · Gold: C
sample 14 · hellaswag:00000895fail · 0.0% · 708ms · 3587e3fbc6f5
Question
There's an open land area with greenery around and grassy surface area. there

A. is a tennis court and a tennis court on the side.
B. is a man wearing white shirt and jungle gym shorts exercising on the green surface.
C. 's a lawn trimmer lying on the ground near some over grown patches of grass.
D. are no trees, but there is a large green grassy area.
Rendered prompt
There's an open land area with greenery around and grassy surface area. there
Agent trace
D
Extracted: D · Gold: C
sample 19 · hellaswag:00002388fail · 0.0% · 729ms · 444c2fea364b
Question
A bottle of moisturizer is shown. A woman on a beach applies it to her skin. she

A. then spits it out into the ground.
B. rubs it into her skin.
C. then shows how to lather it up.
D. rubs the moisturizer into her skin with several measured movements.
Rendered prompt
A bottle of moisturizer is shown. A woman on a beach applies it to her skin. she
Agent trace
D
Extracted: D · Gold: B
sample 22 · hellaswag:00000054fail · 0.0% · 761ms · 5373e432fcd7
Question
A man walks onto the stage to join a band. A man plays up front of a band with a harmonica. the keyboardist

A. plays the hip hop and dance moves with the band.
B. plays a spirited solo during the song.
C. plays his guitar while moving his body around in front of the band.
D. sings with the band, twirling his violin as he walks.
Rendered prompt
A man walks onto the stage to join a band. A man plays up front of a band with a harmonica. the keyboardist
Agent trace
C
Extracted: C · Gold: B
Ornith-1.0-35B-AWQ-INT4unspecified quantunknown protocol · unknown agent
by SgtPooki · shard 4 · 7/20/2026, 2:02:55 PM · cmrtao5cw07avlg01ibl7fetk
80.6%
299/371 correct · 5 correct traces · 5 incorrect traces
Correct samples
sample 2 · hellaswag:00003208pass · 100.0% · 442ms · 70afb80369cf
Question
A small baby sits in a car seat and hols a cup of food. The light shines bright on the right and scares the baby who begins to cry hard. the light

A. finally dazzles the baby and the video ends.
B. fades and the baby calms down and looks all around the car.
C. moves down to the left to shift back and shines on a woman standing in the sitting room.
D. hurts the baby's eyes as she cries and begins to wake up.
Rendered prompt
A small baby sits in a car seat and hols a cup of food. The light shines bright on the right and scares the baby who begins to cry hard. the light
Agent trace
B
Extracted: B · Gold: B
sample 3 · hellaswag:00008219pass · 100.0% · 868ms · 76a0a1963fb5
Question
[header] How to lowlight hair yourself [title] Visit a drugstore or beauty supply store to choose your colors. [step] Some hair dye manufacturers already have products made especially for " at home " lowlights. Pick this if it is available.

A. [title] Buy two bags to color in your color. [step] Homemade dye uses dye directly from the bottle.
B. If not, choose your color carefully according to your natural hair color. [title] Pick a color that is two or three shades darker than your own hair.
C. [substeps] Apply semi-permanent remover to your hair. Then, use just a little of the product to mix all the way up to semi-permanent.
D. [substeps] For lowlights that aren't sold in salon or department stores, use your credit card to buy the color you want. You can also purchase green, red, or rose blushes to match your hair color, but don't overdo it.
Rendered prompt
[header] How to lowlight hair yourself [title] Visit a drugstore or beauty supply store to choose your colors. [step] Some hair dye manufacturers already have products made especially for " at home " lowlights. Pick this if it is available.
Agent trace
B
Extracted: B · Gold: B
sample 4 · hellaswag:00002951pass · 100.0% · 783ms · beee8ecf3f9c
Question
An intro appears showing empty glasses, a card that say's "audi polo challenge 2014", an audi flag blowing in the wind, close up shots of a white horse, people with drinks, cars driving, polo riders getting on their horses, and a large group of people near and in a tent both standing and sitting. the polo match

A. is shown again, the winner of the event is a black horse, and the stands are filled with cheering fans.
B. begins and the men are riding on hroses that are going very fast as they all attempt to hit the ball with their sticks.
C. is shown again in slow motion, but the people are now sitting and riding off the field.
D. is a series of riders competing together and their opponents throwing balls around the racetrack.
Rendered prompt
An intro appears showing empty glasses, a card that say's "audi polo challenge 2014", an audi flag blowing in the wind, close up shots of a white horse, people with drinks, cars driving, polo riders getting on their horses, and a large group of people near and in a tent both standing and sitting. the polo match
Agent trace
B
Extracted: B · Gold: B
sample 5 · hellaswag:00005765pass · 100.0% · 532ms · 2594422ce165
Question
[header] How to make your hair healthy again [title] Handle your hair gently. [step] Damaged hair is very brittle, and it needs to be handled with care. If you're rough with your hair it can split or break, and it will look more frizzy than ever.

A. If you run your hand through your hair a lot, it may come out with split ends. After a week of brushing and " plucking " you might also want to start shaving off split ends.
B. To correct this, comb your hair slowly through or re-style it as needed, preferably with a diffuser or flat-top. [title] Wash your hair using warm, low-to medium and low-flow (tcas) water and rinse it well.
C. If you're gentle and controlled in this process, you'll be able to restore your hair's shape to it's normal state. [substeps] To treat split roots, use a comb to gently detangle any hair above the roots so that it is reworked.
D. Starting today, handle your hair with extra care, whether it's wet or dry. Here's what to do : [substeps] When you're applying shampoo and other products, use your fingers to gently pull the product through your hair from roots to tips.
Rendered prompt
[header] How to make your hair healthy again [title] Handle your hair gently. [step] Damaged hair is very brittle, and it needs to be handled with care. If you're rough with your hair it can split or break, and it will look more frizzy than ever.
Agent trace
D
Extracted: D · Gold: D
sample 8 · hellaswag:00009315pass · 100.0% · 358ms · 78aae1ee0415
Question
[header] How to create the perfect highly pigmented lip [title] Scrub your lips. [step] When you wear a bold, highly pigmented lip color, every dry flake in your lips is emphasized. Before putting on your lip color, use a lip scrub to remove any dead, dry skin from your lips.

A. [substeps] Lip scrub generally washes your lips and beard overnight. Facial scrubs extend beyond them and will add texture and flavor to your lips.
B. Rub it in in circular motions, and wipe it away with a damp washcloth. [substeps] If you don't want to buy a lip scrub, you can make your own with ingredients from your kitchen.
C. [substeps] These intense scrub-ins can be purchased in packs of around four at most drug stores. Combine one liter of lip scrub with up to 12 ounces of water and boil.
D. Spread the scrub onto your lips and then sweep the brush over your lips for the best coverage. [substeps] You should use this a lot after exfoliating your lips because it removes any dead or dry skin.
Rendered prompt
[header] How to create the perfect highly pigmented lip [title] Scrub your lips. [step] When you wear a bold, highly pigmented lip color, every dry flake in your lips is emphasized. Before putting on your lip color, use a lip scrub to remove any dead, dry skin from your lips.
Agent trace
B
Extracted: B · Gold: B
Incorrect samples
sample 1 · hellaswag:00008698fail · 0.0% · 274ms · 2e7f093688ef
Question
[header] How to get ready for summer using earl grey tea [title] Wash your face. [step] Your face should be squeaky clean. [title] Prepare your earl grey teabags, and a glass with a little hot water in it.

A. [step] Earl grey tea is good for any skin, not everyone's dry skin, and it's not bad for those with acne. [title] Dissolve the ginger and lemon zest.
B. [step] Use more than one teabag for optimal results. [title] Soak your teabags in the hot water to release the tea (or the color pigments you'll need).
C. [step] Take your shampoo and hair dryer and start to blast the hot water down towards your head. Since earl grey tea floats, it doesn't evaporate as quickly as you think.
D. [step] Add a spoonful of earl grey to the glass and mix well with your thumb and fingers. [title] Add a few drops of rosewater and blend well.
Rendered prompt
[header] How to get ready for summer using earl grey tea [title] Wash your face. [step] Your face should be squeaky clean. [title] Prepare your earl grey teabags, and a glass with a little hot water in it.
Agent trace
D
Extracted: D · Gold: B
sample 6 · hellaswag:00000756fail · 0.0% · 612ms · afc292dcdb1c
Question
The man puts the mop into the blue mop bucket first and then he starts mopping. First he mops near the stove and the counter. then he

A. moves around the counter the floor sweep the points near the stove.
B. starts to cut the cake mix with knife and sharpener.
C. comes back and starts mopping in the kitchen.
D. mops on the floor that between the refrigerator and another counter.
Rendered prompt
The man puts the mop into the blue mop bucket first and then he starts mopping. First he mops near the stove and the counter. then he
Agent trace
C
Extracted: C · Gold: D
sample 7 · hellaswag:00000971fail · 0.0% · 695ms · 622456bbf8b5
Question
Boy is skateboarding in a skate park at night. boy

A. is goin a hill taking a rest.
B. is in a rink skateboarding on a hill while another boy watches him.
C. is skateboarding in a sidewalk with peopl walking by.
D. is holding the ladder to lift large object from the floor of the skate slide.
Rendered prompt
Boy is skateboarding in a skate park at night. boy
Agent trace
B
Extracted: B · Gold: C
sample 10 · hellaswag:00000020fail · 0.0% · 686ms · 5d701ecf4b65
Question
A woman is reading a newspaper nearby while he speaks to her and shows her something in his hand. The man sharpens his knives using an electric tool. he

A. sharpens and cuts something on a knife board using an electric tool.
B. then continues cutting his onion.
C. travels with his knife under the table and uses it also in his shoe.
D. then shows her a spare knife that he has also used on a knife sharpener and drills into a side of the table.
Rendered prompt
A woman is reading a newspaper nearby while he speaks to her and shows her something in his hand. The man sharpens his knives using an electric tool. he
Agent trace
A
Extracted: A · Gold: B
sample 13 · hellaswag:00002302fail · 0.0% · 695ms · 9830903ea181
Question
A large group is gathered around two men playing beer pong. the men

A. take turns tossing the ping pong balls.
B. hit the ball back and forth over a ping pong table.
C. bring up a large table and stand on it in front of the game.
D. hit the balls back and forth against the wall.
Rendered prompt
A large group is gathered around two men playing beer pong. the men
Agent trace
B
Extracted: B · Gold: A