{"question_id":"hellaswag:00009544","item_index":0,"attempt":0,"prompt_hash":"ed7d3a94464d476bc059eeccd57387ebfa04f5d0e8590cdcc6ecacadb07ffe58","question":"[header] How to take action against capital punishment in your state [title] Focus your action on a specific case or cases. [step] It may help you to focus your efforts on a couple of current capital cases in your state. Focusing your work on a couple of specific cases may be more effective than tackling several cases or state laws as a whole.\n\nA. However, you should also stay on top of general issues related to capital punishment. Combining this knowledge can help you become a very informed participant in the action against these cases, which in turn may help alter capital punishment laws.\nB. Some of the cases in your state are relatively minor compared to other cases in your state. Still, you should aim to focus your efforts on a very specific case at all costs.\nC. [substeps] You can look up places to locate a case. There is a wide variety of legal resources online that can help you locate the most common cases you may want to worry about.\nD. [substeps] If you believe capital punishment in any of your cases is unjust, find a law that applies to those cases. Look at the organizations's forum links.","prompt":"[header] How to take action against capital punishment in your state [title] Focus your action on a specific case or cases. [step] It may help you to focus your efforts on a couple of current capital cases in your state. Focusing your work on a couple of specific cases may be more effective than tackling several cases or state laws as a whole.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3639,"error":null,"output":"B"}
{"question_id":"hellaswag:00000706","item_index":1,"attempt":0,"prompt_hash":"92983ab87d7a43ee33dc517929d705335052d9d5887f1dae83b079468f3c5877","question":"The boy wipes down the sole of the tennis shoe using the wet wipe towel. The boy dries off the shoe using a towel. the boy\n\nA. rinses the shoe inside the folded towel with water.\nB. picks up a high top pair and wipes it off with the same wet towel.\nC. tapes down the rear portion of the shoe.\nD. shakes his foot with music and the camera follows the movements.","prompt":"The boy wipes down the sole of the tennis shoe using the wet wipe towel. The boy dries off the shoe using a towel. the boy","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3644,"error":null,"output":"C"}
{"question_id":"hellaswag:00002205","item_index":2,"attempt":0,"prompt_hash":"9fde6021f5491082a8e5b34d5b2e4de5ea53dd9bed1426fb62a075550d89e537","question":"We see a man and the intro scene. A man in a kitchen has knifes and a sharpening block. the man\n\nA. grinds the knifes through a rod.\nB. begins sharpening the arm of a woman and releases.\nC. wets the block and runs a knife across two different blocks.\nD. then gets to chopping vegetables mainly in front of a counter with other scenes working.","prompt":"We see a man and the intro scene. A man in a kitchen has knifes and a sharpening block. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3643,"error":null,"output":"C"}
{"question_id":"hellaswag:00006200","item_index":3,"attempt":0,"prompt_hash":"24dc8d27322140625b70e53fa538df3cef73d16e148f0d57c4fcca21a7ac0845","question":"[header] How to wear yoga pants to work [title] Invest in high-quality yoga pants. [step] If you're wearing yoga pants to work, you want them to be durable while also providing comfort. Try to find yoga pants made of a thicker material, and try them on before purchasing them to ensure they aren't see-through.\n\nA. High-quality yoga pants will likely be designed for yoga or other types of work. [title] Stick to a thin, stretchy, short-waist.\nB. Sheer yoga pants are definitely something to avoid in the workplace. [substeps] Yoga leggings offer the most outfit options and tend to look the best in a professional work environment.\nC. [substeps] Make sure the pants are specifically made for yoga, preferably ones with natural fibers, such as cotton or silk. When in doubt, look for yoga pants in solid colors.\nD. Yoga pants come in a variety of shapes and sizes, and none of them fit under normal circumstances. See the rough guide section for more specific instructions.","prompt":"[header] How to wear yoga pants to work [title] Invest in high-quality yoga pants. [step] If you're wearing yoga pants to work, you want them to be durable while also providing comfort. Try to find yoga pants made of a thicker material, and try them on before purchasing them to ensure they aren't see-through.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6761,"error":null,"output":"B"}
{"question_id":"hellaswag:00002478","item_index":4,"attempt":0,"prompt_hash":"303275c321c6121bab6e4034d35b91feb35b586c0dcdd13edf46a5a2258c963f","question":"The ingredients are added to a tub and scoops of ice cream are taken from a tub after freezing. a bowl\n\nA. is filled with ice cream.\nB. of ice cream is added to the bowl and melted in four different pans.\nC. of olive oil is added as a spoon and bowls of butter are added.\nD. is used to rack ice cream.","prompt":"The ingredients are added to a tub and scoops of ice cream are taken from a tub after freezing. a bowl","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3645,"error":null,"output":"A"}
{"question_id":"hellaswag:00003152","item_index":5,"attempt":0,"prompt_hash":"4b76423bef3bd01e833ca91fa2dc10034f3d36c114c90061d06d0c2acc8d1e62","question":"A young man is outside attempting to do a highjump behind the london logo. A male man then stands in front of the mat and talks about the jump. another male\n\nA. is then shown as he takes off and clears the bar of the high jump and there's an instant replay.\nB. appears in before he is cased six times attempting to same doing the same jump in different locations and taking short breaks.\nC. joins in and the two jump from different heights, falling hundreds of times when landing.\nD. joins in and prepares to get his jump in acting steadier and hitting the mat.","prompt":"A young man is outside attempting to do a highjump behind the london logo. A male man then stands in front of the mat and talks about the jump. another male","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6761,"error":null,"output":"A"}
{"question_id":"hellaswag:00009865","item_index":6,"attempt":0,"prompt_hash":"80bdeb7973ed01d09cdf48d72bb4a8f238c27fe12d05d3b8f0600befb7e01267","question":"[header] How to make natural shampoo for your hair type [title] Boil 1 cup of spring water. [title] Remove the pan from the heat, and add 2 tablespoons of dried chamomile. [title] Let steep for 15 minutes.\n\nA. [title] Obtain suitable shampoo for your hair type. [step] These shampoos and conditioners are very absorbent and therefore should be dried separately for longer.\nB. [title] Strain the herbs from the liquid, and throw away the herbs. [title] Let the liquid sit for about 20 minutes, allowing it to completely cool down.\nC. [title] Let cool a little but leave the liquid in for 1-2 minutes. [title] Strain the liquid and store in a cool container to use again later.\nD. [title] Test and see if your shampoo mixture will lather up. [step] Add more-combine half of the mixture with 3 tablespoons of water until it bubbles.","prompt":"[header] How to make natural shampoo for your hair type [title] Boil 1 cup of spring water. [title] Remove the pan from the heat, and add 2 tablespoons of dried chamomile. [title] Let steep for 15 minutes.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4166,"error":null,"output":"A"}
{"question_id":"hellaswag:00006290","item_index":7,"attempt":0,"prompt_hash":"8f9177b33146b0aa87e3d08f63c21922fa658e1b9a74f26363abe6ca2cffa493","question":"[header] How to play sharks and minnows [title] Pick a shark. [step] Choose a player to be the shark. This player is now \" it \" and stands in the center of the pool (the \" ocean \").\n\nA. They raise their non-dominant arm straight into the water, like diving, and perform the shark trick. The shark's eyes use this trick to reflect waves and also give you a full view of the shark, since you can't see him from the board.\nB. [substeps] When playing with a lot of people in a large area, you can play with more than one shark. 10 minnows to 1 shark is a good ratio.\nC. He/she is mostly responsible for bringing the dead shark back to the water at the end of each round. [substeps] The top priority is the decision whether or not to leave the pool.\nD. You cannot choose sharks from a different species (e.g. they all have different colors, fangs, rings, claws, like fish) or any other species.","prompt":"[header] How to play sharks and minnows [title] Pick a shark. [step] Choose a player to be the shark. This player is now \" it \" and stands in the center of the pool (the \" ocean \").","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3640,"error":null,"output":"B"}
{"question_id":"hellaswag:00002393","item_index":8,"attempt":0,"prompt_hash":"e6a396ef8b4f4592c2db22a738961836b02bde727318d4a246ab30a7e07923a2","question":"The lady taps the puck with her stick before pushing it hard and pushing the puck across the court. the camera\n\nA. gets itself angled to keep the puck from hitting the boy playing while playground equipment sits nearby.\nB. zooms in on the puck and two women are standing next to it talking to each other.\nC. zooms in on cupcake detail while the lady takes a breath.\nD. follows the pucks which stops short and teeters before resting.","prompt":"The lady taps the puck with her stick before pushing it hard and pushing the puck across the court. the camera","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3644,"error":null,"output":"B"}
{"question_id":"hellaswag:00000167","item_index":9,"attempt":0,"prompt_hash":"34da43317e57923c6d59ca0c45eda2b7b29f12f90c6e0226b4ece1e09ae086ba","question":"There is a woman eating ice cream from a tub and talking to the camera man. and he\n\nA. tells her about some sticky potatoes and she eats them also.\nB. tries to look into the tub, but she pulls it away.\nC. is gone doing this all with a in a getting ice cream, while a man else with a stick is also eating from a tub and eating ice cream.\nD. is holding a baby zumba as they begin dancing around twirling their legs and dancing in tight formation.","prompt":"There is a woman eating ice cream from a tub and talking to the camera man. and he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3643,"error":null,"output":"B"}
{"question_id":"hellaswag:00009592","item_index":10,"attempt":0,"prompt_hash":"fc55092b8354dfe7a11f7bd94645e8fc27fe7a0e6e5b71d5aa6751d65f69592d","question":"[header] How to discipline your kids as divorced parents [title] Keep the rules relatively stable. [step] A divorce can upend children's lives. If you introduce a bunch of new, extremely restrictive (or extremely lax) rules following a divorce, you'll only add to their stress, confusion, and unhappiness.\n\nA. You'll be surrounding your children's faces with different rules so they have no sense of which rule is the right one to follow, and you'll open up the doors to overcrowding and inreason. [substeps] They might not agree to a rule at first.\nB. When the courts are involved, you should choose with caution what to enforce. [substeps] If you personally handle kids as well as your whole family, consider arguing that a majority of the rules are fair and consistent.\nC. Talk to your child about the rules so they know what is expected of them. [substeps] Always try to keep your rules and additions predictable-your child needs to understand that you need to consistently stay consistent with them, and you're not allowing them to relax.\nD. Once your divorce is finalized, carry over as many of the rules as possible from the time before you and your spouse divorced. [substeps] If you do decide to change a rule, then make sure that you evaluate your reasons for the rule change.","prompt":"[header] How to discipline your kids as divorced parents [title] Keep the rules relatively stable. [step] A divorce can upend children's lives. If you introduce a bunch of new, extremely restrictive (or extremely lax) rules following a divorce, you'll only add to their stress, confusion, and unhappiness.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3641,"error":null,"output":"D"}
{"question_id":"hellaswag:00005211","item_index":11,"attempt":0,"prompt_hash":"28b8a66767c2db2d4ed06cbe27deb40baad6747d6e80905df165601b44750069","question":"[header] How to do dandelion nails [title] Paint on a light-colored nail polish. [step] The color nail polish you pick is up to you, but make sure you choose a lighter color. You want the dandelion shapes you paint on to show up on your nails.\n\nA. [title] Determine if this is meant for on wood or paper nails. [step] Wood nail polish has a heavier adhesive and is easier to paint on, but does leave little ridges.\nB. Simply paint a lighter color onto your nail polish before you paint on your dandelion nails. [title] Create a dandelion outline on your top nail.\nC. [substeps] Sunflower pink nail polish works great for this, and it covers quite a lot of skin. Royal blue nail polish works perfect for this.\nD. Go for something like white or light pastel shades. [substeps] Make sure the base polish is completely dry before proceeding.","prompt":"[header] How to do dandelion nails [title] Paint on a light-colored nail polish. [step] The color nail polish you pick is up to you, but make sure you choose a lighter color. You want the dandelion shapes you paint on to show up on your nails.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3638,"error":null,"output":"D"}
{"question_id":"hellaswag:00000193","item_index":12,"attempt":0,"prompt_hash":"f742e417959c76ba9bfa4d1cbbcf3f738f52bc462651f99676f99303d4c9cc32","question":"We see the title introducing us to jesse. jesse the dog\n\nA. owner shows us how he walks and how he breathes.\nB. runs and catches frisbee.\nC. runs in circles where the dog is his mate.\nD. is shown from a shampoo advertisement one last time.","prompt":"We see the title introducing us to jesse. jesse the dog","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3644,"error":null,"output":"A"}
{"question_id":"hellaswag:00009278","item_index":13,"attempt":0,"prompt_hash":"14f1f1d8ee1dee9e1596ce9007c7babf20daeaea78b145146957473eb246ed3b","question":"[header] How to restore dll files on your computer [title] Missing dll files that somehow disappear can cause your computer to act in strange ways, such as not shutting down properly. [step] Run a system disk scan in the norton antivirus suite. For instance norton has a tool called \" windoctor \" that makes repairs automatically.\n\nA. [title] Burn dll files in a safe location if they disappear. [step] Dll files are damaged in the scanning process.\nB. [title] Notice when your case communications is missing. [step] Sometimes, when your case communications on your computer become unreadable, as administrator you can notice that the case communications are missing.\nC. [title] Do a complete reboot of your computer. [step] After reboot you are looking at dll files that are already in their original packaging.\nD. It is suggested that if you remove norton for another free antivirus program, they retain norton utilities/windoctor and speed disk for defragging. [title] Always run a scan on anything you download before you open it.","prompt":"[header] How to restore dll files on your computer [title] Missing dll files that somehow disappear can cause your computer to act in strange ways, such as not shutting down properly. [step] Run a system disk scan in the norton antivirus suite. For instance norton has a tool called \" windoctor \" that makes repairs automatically.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3640,"error":null,"output":"C"}
{"question_id":"hellaswag:00003450","item_index":14,"attempt":0,"prompt_hash":"5cdda46160bd8add08e7273ce4d8a0e87c7a26bc1e0a600bceccef6bfb08b95f","question":"[header] How to find adoption records [title] Read your state law. [step] Each state decides what kind of information it will release and the circumstances under which it will release that information. To access adoption records, the relevant state law is that of the state where the child was adopted.\n\nA. This might not be the state where the child was born or where you currently live. [substeps] Summaries of every state's adoption laws are available at the child welfare information gateway website.\nB. [substeps] States normally publish adoption records for specific counties or cities. You have the right to access this information even if the extent to which you intend adoption is negotiable.\nC. [substeps] Even if your state has a registry, you must first read and understand the law in your state. If you follow your state's registry, you may be able to get records from the state in which you adopt the child.\nD. To obtain a copy of your state law, visit. [substeps] You should be able to find copies of other laws on your state's website, or by searching \" your state \" on the internet.","prompt":"[header] How to find adoption records [title] Read your state law. [step] Each state decides what kind of information it will release and the circumstances under which it will release that information. To access adoption records, the relevant state law is that of the state where the child was adopted.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3640,"error":null,"output":"A"}
{"question_id":"hellaswag:00004751","item_index":15,"attempt":0,"prompt_hash":"919b936fb7a40f2f94aec1572fe08b3e01adc115e17448763ed17eb0533382e1","question":"[header] How to cope with depression in college [title] Get plenty of sleep. [step] Sleep is crucial for your mental and physical health, and not getting enough can make your depression worse. When you're in college, it can be hard to get the sleep you need.\n\nA. Look at that report you took about 4 weeks ago and make a note about you sleep cycle to make sure you're getting the right amount of sleep. [substeps] All people need 8 hours of sleep every night.\nB. Avoid hitting the snooze button twice in the morning and only stay up too late-until the next day! And if you fall back asleep, stay up before school so you don't have to wake up. [title] Learn to do easier on your schoolwork.\nC. Most young adults need 7-9 hours of sleep a night to be at their healthiest. [substeps] Schedule time for sleep the same way you would for studying and other obligations.\nD. If you want to regulate your depression, though, sleeping helps you to deal with the effects. [substeps] Figure out how much sleep you need each night.","prompt":"[header] How to cope with depression in college [title] Get plenty of sleep. [step] Sleep is crucial for your mental and physical health, and not getting enough can make your depression worse. When you're in college, it can be hard to get the sleep you need.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3640,"error":null,"output":"C"}
{"question_id":"hellaswag:00008493","item_index":16,"attempt":0,"prompt_hash":"e4ef6cdb6553c6a0cf4eacd0ac4649408c03ca5cf5b626868b07c01096cee117","question":"[header] How to use amadeus software [title] Go to the amadeus login page using your internet browser. [title] Type in the username and password that was provided by your account manager. [title] Tick the box to store password if you don't want to type your login information every time you need to sign in.\n\nA. [title] Read the additional instructions and clicks accept. [title] Wait for your account manager to issue a confirmation email.\nB. [step] Otherwise you can leave the box unchecked. [title] Press the ok button to sign in and go to the search page.\nC. [step] Click restart if you want to continue. [title] Make your profile area visible by tapping the button that says \" view profile.\nD. [title] Click the \" other \" icon near the top of the dashboard and continue to click the checkboxes. [title] Type recaptin for the phone you're using and click \" accept \" to continue to the other account page.","prompt":"[header] How to use amadeus software [title] Go to the amadeus login page using your internet browser. [title] Type in the username and password that was provided by your account manager. [title] Tick the box to store password if you don't want to type your login information every time you need to sign in.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3639,"error":null,"output":"B"}
{"question_id":"hellaswag:00008494","item_index":17,"attempt":0,"prompt_hash":"ea96e50e35d790138b31a1b6d8d8142d1c7dcfe8ed6d2e5c2ff5f44878ca49f0","question":"[header] How to get your parent's permission [title] Be responsible. [step] Before you even seek out your parents' permission for an activity, show them how responsible you are. You need to be doing this for a while, too, not just in the few days leading up to your asking.\n\nA. When you ask your parents to help you out, be consistent with your approach, and don't punish them for being too involved in it. The conversation will be much more productive if you don't demand that your parents' permission be restricted, too.\nB. Your parents are more likely to say yes to you if they perceive you as responsible and trustworthy. Here are some ways you can indicate you are responsible : [substeps] Do all your chores without needing to be asked or reminded.\nC. [substeps] Consistently do everything your parents say you should do, whether you feel like you're being asked to. This will show them you can really take responsibility for your actions.\nD. To be responsible, you need to be responsible for things you may not normally do, whether it's babysitting and an unexpected visit to the grocery store. In addition, keep an updated list that you can carry with you so your parents don't get lost in their head.","prompt":"[header] How to get your parent's permission [title] Be responsible. [step] Before you even seek out your parents' permission for an activity, show them how responsible you are. You need to be doing this for a while, too, not just in the few days leading up to your asking.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3639,"error":null,"output":"B"}
{"question_id":"hellaswag:00003215","item_index":18,"attempt":0,"prompt_hash":"2d7ff57fb72236318138fde6d7ff689f42510952c5d4b205876b82735c94e600","question":"A man wearing arab clothes talks on front a crow while holding a large umbrella. a person\n\nA. holds a boy holding meters on the man holding a large umbrella.\nB. walks in the street holding an umbrella.\nC. is then shown talking face to face in a fish tank.\nD. look with the man on the beach as other surfers are shown.","prompt":"A man wearing arab clothes talks on front a crow while holding a large umbrella. a person","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3644,"error":null,"output":"B"}
{"question_id":"hellaswag:00008645","item_index":19,"attempt":0,"prompt_hash":"9d837b6be3f969b6b19502f64caba2dab62500dd6def86de2a807de76a90f1fe","question":"[header] How to tie a tae kwon do belt [title] Fold the belt exactly in half to find the center. [title] Place the center of the belt at the front just your navel. [step] The two sides should be equal.\n\nA. [title] Placing your right hand on the center of the belt, make some small circles to mark where the knot should be. [title] With your left hand, make small circles to mark where the two ends of the belt will be when it is tied to your waist.\nB. [substeps] The button is on the side closest to the center. Position the first one on right side of your navel.\nC. [title] Wrap the two ends around your waist, and back to the front. [step] Make sure that the left side (new right) is crossed over the right in the back.\nD. [title] Grab the left mid part of the belt. [step] You have now mastered a very simple frog knot! [title] Pull the top edge of the belt down into front of you.","prompt":"[header] How to tie a tae kwon do belt [title] Fold the belt exactly in half to find the center. [title] Place the center of the belt at the front just your navel. [step] The two sides should be equal.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3637,"error":null,"output":"A"}
{"question_id":"hellaswag:00007580","item_index":20,"attempt":0,"prompt_hash":"7d49d0e3cbe1716748988c1edb742a86fd68626e1483017e9b58115da074b1af","question":"[header] How to wash braids [title] Choose the right shampoo and conditioner. [step] While any shampoo or conditioner will work, it is better to use moisturizing shampoos and conditioners, especially if your scalp is dry and itchy. Some products such as shea moisture african black soap deep cleansing shampoo and giovanni tea triple treat shampoo are made specifically for dry scalps and thick hair.\n\nA. [title] Apply a generous amount of shampoo onto the scalp, remembering to cover the nape and hairline. [step] Gently massage every inch of the scalp, paying close attention to itchy areas.\nB. These products cause your hair's natural oils to soak up. [substeps] Make sure to read the instructions that come with the shampoo and conditioner to make sure you use the right oils.\nC. [substeps] To prevent yourself from getting scalp damage, store your braids in a big container. Cover your braids with a towel and allow them to air dry as soon as you're done.\nD. [title] Shake the hair gently for about 20 seconds. [step] This gives the product time to seep out, and some of the water and oil will come out.","prompt":"[header] How to wash braids [title] Choose the right shampoo and conditioner. [step] While any shampoo or conditioner will work, it is better to use moisturizing shampoos and conditioners, especially if your scalp is dry and itchy. Some products such as shea moisture african black soap deep cleansing shampoo and giovanni tea triple treat shampoo are made specifically for dry scalps and thick hair.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3640,"error":null,"output":"A"}
{"question_id":"hellaswag:00009201","item_index":21,"attempt":0,"prompt_hash":"7f3e89a06819a966ea74908e2b10f87592f90bad2c90a91c0146e44b80dd9c00","question":"[header] How to make rutabaga fries [title] Preheat the oven and line a baking sheet. [step] To ensure that the oven is hot enough to bake the fries when they're ready, set the temperature to 400 degrees fahrenheit (200 degrees celsius) and allow it to heat fully. Next, use a piece of parchment paper to line a large baking sheet for the fries.\n\nA. [substeps] You can substitute foil for the parchment paper if you prefer. It's important not to place the rutabaga fries directly on the baking sheet, though, because the edges will burn more quickly.\nB. [substeps] Note that you may want to stick some heat resistant glass or steel foil over the front of the baking sheet. The glass or metal coating inside will prevent the fries from sticking together as they bake.\nC. [substeps] It's okay if the sheet burns instead of browning at this point because the fries will be softer to fry. Avoid using waxy paper because it will degrade.\nD. [substeps] You can use foil to transfer the fries from the oven to the greased baking sheet. A clean, oven-safe sheet is ideal.","prompt":"[header] How to make rutabaga fries [title] Preheat the oven and line a baking sheet. [step] To ensure that the oven is hot enough to bake the fries when they're ready, set the temperature to 400 degrees fahrenheit (200 degrees celsius) and allow it to heat fully. Next, use a piece of parchment paper to line a large baking sheet for the fries.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6761,"error":null,"output":"A"}
{"question_id":"hellaswag:00001226","item_index":22,"attempt":0,"prompt_hash":"7336370491b1925ff2022107c4c69609c9db4d13be588580d451e5143ddfa500","question":"A small group of people are seen running down a field when one scores a goal. the same shot\n\nA. is shown again in slow motion.\nB. is again shown a second time and the male is shown running in slow motion where his teammate scores a goal and another man wins.\nC. is shown again in slow motion followed by more people scoring a goal.\nD. is shown again twice in slow motion followed by several more clips in slow motion as well as children playing on the field.","prompt":"A small group of people are seen running down a field when one scores a goal. the same shot","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3644,"error":null,"output":"A"}
{"question_id":"hellaswag:00001395","item_index":23,"attempt":0,"prompt_hash":"594f44d64b0ccc564bc75480fe49c9d876d46d5da8e8804b81358902f7f622e8","question":"He stands up and pulls a fish out of the hole. He cuts the hook out of the fish and throws the fish on the ice. he\n\nA. gets back on his hands and helps pull up snow off the ice.\nB. raises his hands in the air.\nC. gets another fish out of the hole and pulls it out laying in the water.\nD. sits back down on the bucket.","prompt":"He stands up and pulls a fish out of the hole. He cuts the hook out of the fish and throws the fish on the ice. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3644,"error":null,"output":"D"}
{"question_id":"hellaswag:00000785","item_index":24,"attempt":0,"prompt_hash":"8d65af22d985b2cd3922b61254cb786f5f0a35e7c824e07fea6d37c3b4a831dd","question":"He moves a stick and pulls a fishing line out of a hole. The man pulls a fish out of the hole. he\n\nA. points at the camera and smiles.\nB. pulls a rod out of the hole.\nC. picks the fish up and shows us the fish.\nD. pauses and speaks to the camera.","prompt":"He moves a stick and pulls a fishing line out of a hole. The man pulls a fish out of the hole. he","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3644,"error":null,"output":"B"}
{"question_id":"hellaswag:00004333","item_index":25,"attempt":0,"prompt_hash":"b89c677ac8b53c4747d81251000e11d9dd44b11dfe52347ed546266412888d41","question":"[header] How to plan a trip on public transportation using google maps [title] Get on google maps, and click on the \" get directions \" tab. [title] Click on the second symbol that looks like a train. [step] It is directly to the right of the car.\n\nA. That is the tube that runs to your destination. [title] Identify the route you want to take.\nB. [title] Change the setting asking you if you want to \" leave now \" based upon your current situation. [step] For example, you might want to be at a certain place by 2 o'clock, then you would select \" arrive by \" or if you wanted to depart at a certain time you would select \" depart at.\nC. [title] On the next screen, select all stations (you can see them scattered throughout town) and type \" we need to go \" into the search field. [title] Click on it to see directly on the map what you are looking for.\nD. It will move you to an alternate route. [title] Go left and right, depending on where you are on the map.","prompt":"[header] How to plan a trip on public transportation using google maps [title] Get on google maps, and click on the \" get directions \" tab. [title] Click on the second symbol that looks like a train. [step] It is directly to the right of the car.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6763,"error":null,"output":"B"}
{"question_id":"hellaswag:00005198","item_index":26,"attempt":0,"prompt_hash":"1972cb153a90bdb515fee8a27e469d53a080d17281d0b36378f58f61c9fe8de2","question":"[header] How to put cowrie shells on dreads [title] Pick a dreadlock. [step] The width of the dreadlock needs to be small enough to fit into the front opening of the cowrie shell. Do not choose a dreadlock that is significantly smaller than the cowrie shell, though, since the shell could end up weighing down the hair and weakening the dread.\n\nA. Opt for a dreadlock that is about the same size as the hair on the dread. [substeps] You can even pick a dreadlock that is thicker and has a more coarse texture to it.\nB. [substeps] You can use a hairbrush or a comb to hold your dreadlock when you've reached your goal. Sometimes snaffle dread hair is mistakenly mistakenly called untouchy and actually means dreaded.\nC. It does not hurt to use a smaller dreadlock that matches your dreads' natural length, since that will leave the hair dense and less secure. [substeps] You could use a rubber dreadlock if the dreads are too thick and sturdy for this.\nD. [substeps] It will also be much easier to use a dreadlock with a tight end rather than one with a loose, fraying end. [title] Push the end into the center of the shell.","prompt":"[header] How to put cowrie shells on dreads [title] Pick a dreadlock. [step] The width of the dreadlock needs to be small enough to fit into the front opening of the cowrie shell. Do not choose a dreadlock that is significantly smaller than the cowrie shell, though, since the shell could end up weighing down the hair and weakening the dread.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3641,"error":null,"output":"D"}
{"question_id":"hellaswag:00002266","item_index":27,"attempt":0,"prompt_hash":"2220cd3a5599564fc5f066d676f1a5508b78b3a37944e26ebfd9424c8eaaaf5f","question":"He is standing in a kitchen in front of the counter that has a variety of raw vegetables displayed. He places a kitchen towel on the counter and then places the knife on top of it. he\n\nA. uses another knife to paint his finger tips.\nB. holds a red container and then places sticks of bread on the counter.\nC. rubs the towel against the block then scrubs it along the knife.\nD. then demonstrates how to use the rada sharpening tool to sharpen the kitchen knife by running iit back and forth in gentle and steady motion.","prompt":"He is standing in a kitchen in front of the counter that has a variety of raw vegetables displayed. He places a kitchen towel on the counter and then places the knife on top of it. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3644,"error":null,"output":"D"}
{"question_id":"hellaswag:00007249","item_index":28,"attempt":0,"prompt_hash":"913b411495ec6f6c9c8997d0fb274984cd48148062e60efa766b0e287ef3229c","question":"[header] How to add social networking icons to your blog [title] Decide which social networks you want to list on your blog. [step] The most popular icons include facebook, twitter, linkedin and pinterest. [substeps] Choose social networking spots where you and your blog readers are active.\n\nA. Try google or bing in order to find the most popular. Share your posts along with other people who share similar interests or topics as you.\nB. Create a new page every 20 to 30 days or as soon as your post/comment is received on your blog. You can adjust this time, but it doesn't have to be.\nC. For example, if you have a great facebook page, you will want to include that icon. If you want to encourage your readers to pin your blog onto their pinterest boards, be sure you have that icon present.\nD. Stand out by choosing your public, private or private profile. The more dedicated you are to making these public profiles, the more likely you are to get targeted.","prompt":"[header] How to add social networking icons to your blog [title] Decide which social networks you want to list on your blog. [step] The most popular icons include facebook, twitter, linkedin and pinterest. [substeps] Choose social networking spots where you and your blog readers are active.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3642,"error":null,"output":"C"}
{"question_id":"hellaswag:00001392","item_index":29,"attempt":0,"prompt_hash":"462384228d50c1a5d553fb64304461960fd9f092c91cdba9a70da0f153eb799e","question":"There's a man wearing a black athletic suit in a shot put field practicing shot put. He begins by taking the ball in his hand and holding it against his neck. then he\n\nA. spins around and throws the ball far ahead.\nB. shoots the ball through the hoop over and over showing how to pierce it.\nC. continues hitting the ball up against the wall.\nD. irons it down by applying a circular pattern.","prompt":"There's a man wearing a black athletic suit in a shot put field practicing shot put. He begins by taking the ball in his hand and holding it against his neck. then he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3643,"error":null,"output":"A"}
{"question_id":"hellaswag:00005018","item_index":30,"attempt":0,"prompt_hash":"b01b456701ecdb04f39553bc775105c47977feb7b3187ba5c3c0e06b1cefc695","question":"[header] How to stay up on a school night [title] Find the right time to ask. [step] When you're asking your parents to stay up on a school night, timing is everything. If you ask them when they're in a bad mood or feeling particularly stressed, you're likely going to get an automatic \" no.\n\nA. \" find a time when they're in good moods. [substeps] Check the weather or any other conditions before asking your parents to stay up on a school night.\nB. \" [substeps] Think of your last night together as a chance to see if your parents will hold up their end of the deal. You may not know how to approach your parents on a bad day.\nC. \" however, if you ask when they're in a good mood, you have a better chance of getting permission to stay up. [substeps] Pay attention to when your parents are usually in a good mood.\nD. \" in other words, if they said no and then suddenly say no later in the night, you may not have a chance to ask them the next time you know they're sleeping. It's much too early for you to ask your parents' permission.","prompt":"[header] How to stay up on a school night [title] Find the right time to ask. [step] When you're asking your parents to stay up on a school night, timing is everything. If you ask them when they're in a bad mood or feeling particularly stressed, you're likely going to get an automatic \" no.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3642,"error":null,"output":"C"}
{"question_id":"hellaswag:00003730","item_index":31,"attempt":0,"prompt_hash":"d646d8c1f6d05d4c8e57677afb849111a9f7ceb7a4a475e999f85a596a676e6c","question":"[header] How to use green tea on your face to achieve prettier skin [title] If you already drink tea, switch to green tea. [step] It will work better if you already drink it regularly. After each cup of green tea that you drink, remove the teabag and cut it open.\n\nA. This may help prevent clostridium buildup and the dark circles you'll create on your skin. [title] Follow the suggested dosages (how many cups you can expect to have with you each day).\nB. Dump the tea out into a bowl or bag and throw it away, you might not need to do this again. You can mix it all up and see how pretty you can be without it.\nC. This will save you some time as coffee could scruff and irritate skin cells. Pour some tea into a cup and drop one teabag into the cup, wait 15 minutes and strain out the tea.\nD. Empty the content into a small cup and add a little honey to make it into a paste. Make sure your face is clean before you apply the paste.","prompt":"[header] How to use green tea on your face to achieve prettier skin [title] If you already drink tea, switch to green tea. [step] It will work better if you already drink it regularly. After each cup of green tea that you drink, remove the teabag and cut it open.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3641,"error":null,"output":"D"}
{"question_id":"hellaswag:00007321","item_index":32,"attempt":0,"prompt_hash":"d2d91f8953978fc16c1f293a38184efb41afed3c3ec8cbd9317fdebb8e28037e","question":"[header] How to wall mount a plasma tv [title] Purchase a wall mount from a reputable plasma tv dealer, and ensure that it fits your tv. [step] Ensure you are purchasing the correct wall mount for your tv and for the type of wall you will be mounting your tv on. [title] Consider what you will be doing with your cables.\n\nA. [step] You will need cables to connect the tv, the tv and a router. You may want wall mounts in a different color.\nB. [step] If you are mounting your tv on a stud wall it is possible to thread your cable down the space behind the wallboard. Running the cables this way looks great but involves a lot of effort.\nC. [step] Wallpapering or removing insulation can mean removing bulbs, wires, or other items from the wall you will be moving around on. You can usually evaluate the length and height of the wall.\nD. [step] It is advisable that you weld your cables to the wall and also ensure that you find a safe, sturdy and rated drywall surface to hold your tv to. Ensure that this choice is within your budget.","prompt":"[header] How to wall mount a plasma tv [title] Purchase a wall mount from a reputable plasma tv dealer, and ensure that it fits your tv. [step] Ensure you are purchasing the correct wall mount for your tv and for the type of wall you will be mounting your tv on. [title] Consider what you will be doing with your cables.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5639,"error":null,"output":"B"}
{"question_id":"hellaswag:00008667","item_index":33,"attempt":0,"prompt_hash":"5a4f04e257a56ff6d8d238bfe3e375b525fca3c93f541b21a364dad7c8f28a28","question":"[header] How to bob for apples [title] Find a suitable tub. [step] You can use a bucket, a cooler, a basin, or a bin, as long as it is big enough to hold water and apples. Wash the tub first, then place it on a table or cart that is strong enough to hold it when it is full of water.\n\nA. [substeps] You should also check to make sure that the lid you put on it is airtight. You may want to purchase a plastic lid because there is a lot of air inside.\nB. [substeps] A plastic bucket will work just fine. Since apples are still fresh, you could use plastic cereal boxes, dishwasher trays, jars, or.\nC. [substeps] Some craft stores also sell a water pitcher, as well as a tub. [title] Pour ½ cup (118 ml) of salt in the water.\nD. The top should be about waist-high to participants in the game. [substeps] You could also place the tub on the ground and have participants kneel down to bob for apples.","prompt":"[header] How to bob for apples [title] Find a suitable tub. [step] You can use a bucket, a cooler, a basin, or a bin, as long as it is big enough to hold water and apples. Wash the tub first, then place it on a table or cart that is strong enough to hold it when it is full of water.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5638,"error":null,"output":"D"}
{"question_id":"hellaswag:00004692","item_index":34,"attempt":0,"prompt_hash":"9b28fe1387ab22b9745307e2a07fa637247ad3b5b187e827475e3b2643e40b98","question":"[header] How to help your dog through physical therapy [title] Discuss at-home physical therapy options with your vet. [step] Physical therapy may be hard for your dog, so you will need to help your dog through its therapy. Performing at-home physical therapy can help your dog feel better and get back to its normal activities.\n\nA. [substeps] Start by giving your vet a call if you are having questions or concerns about exercise. They may be able to refer you to local training programs or trainers who can give you tips on improved muscle strength and endurance.\nB. Discuss therapy options with your vet and your dog before making any major decisions such as having a physical or weight changes. [title] Commit to seeing your vet more frequently.\nC. [substeps] Your vet may suggest things like: techniques to help your dog lower its weight and reduce testosterone levels. Treatment techniques to improve the condition like diet and exercise.\nD. When your vet prescribes physical therapy, they will give you different at-home exercises to perform on your dog. [substeps] At-home physical therapy includes passive range-of-motion (prom) exercises and massage therapy.","prompt":"[header] How to help your dog through physical therapy [title] Discuss at-home physical therapy options with your vet. [step] Physical therapy may be hard for your dog, so you will need to help your dog through its therapy. Performing at-home physical therapy can help your dog feel better and get back to its normal activities.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5637,"error":null,"output":"D"}
{"question_id":"hellaswag:00003009","item_index":35,"attempt":0,"prompt_hash":"88bf712459473b3f99f8738e7a5169383476a3071bdc6956e7a088f5b35fb5ab","question":"A man is seen bending down before a set of weights with others watching him on the side. The man lifts up the weights over his head. in the end\n\nA. he drops down onto a mat with a pain f hurt.\nB. he drops the weights off.\nC. he walks away with the weights.\nD. he throws the weights down.","prompt":"A man is seen bending down before a set of weights with others watching him on the side. The man lifts up the weights over his head. in the end","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5650,"error":null,"output":"C"}
{"question_id":"hellaswag:00001093","item_index":36,"attempt":0,"prompt_hash":"09e4a9743a200b53b18596d0c59a56ea54553deb0536327dcd35aec202fbb127","question":"A girl is shown speaking to the camera with clips of women splashing water on their face. the girl\n\nA. then begins washing her face in the sink.\nB. then sports an electric razor and begins to brush her face while still speaking to the camera intermittently.\nC. continues speaking to the camera while a man soaks up the water and laughs.\nD. continues speaking while turning it towards the camera while laughing.","prompt":"A girl is shown speaking to the camera with clips of women splashing water on their face. the girl","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5649,"error":null,"output":"A"}
{"question_id":"hellaswag:00008647","item_index":37,"attempt":0,"prompt_hash":"9d2e8baabfff72ba484137f997d1512ff31ba089841d8d8630ad90049ea17a92","question":"[header] How to clean plexiglass [title] Blow dust and dirt off the plexiglass. [step] Using your own breath or a hair dryer, blow the dust and dirt off the plexiglass. If using a hair dryer, be sure it is set to its coolest setting.\n\nA. Hot air will damage the plexiglass. Position the hair dryer at a 45 degree angle several inches away from the plexiglass, running the air side-to-side down the surface.\nB. Make sure to do this quickly before anyone else has to. [substeps] Try to use a cooling chamber to avoid steam-drying the plexiglass.\nC. Otherwise, you may not be able to clean the plexiglass's interior surface. [title] Use a damp cloth to clean the surface.\nD. Using a hairdryer set to its lowest setting might damage your plexiglass or it might cause it to crack. [substeps] Dry the plexiglass with a clean towel.","prompt":"[header] How to clean plexiglass [title] Blow dust and dirt off the plexiglass. [step] Using your own breath or a hair dryer, blow the dust and dirt off the plexiglass. If using a hair dryer, be sure it is set to its coolest setting.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5638,"error":null,"output":"C"}
{"question_id":"hellaswag:00007333","item_index":38,"attempt":0,"prompt_hash":"386598987dc926199d915cb4f27cc393b0a4965801f48dcf28b19c90789ba90a","question":"[header] How to calculate the market value of equity [title] Make sure you use up-to-date information. [step] The market value of equity is constantly changing, and thus the calculation requires the newest information in order to be accurate. Stock markets are open from monday to friday.\n\nA. Stock markets are not open monday through friday, and you can't expect your stock to get \" up to date. \" [substeps] In times of the year when stocks are actually trading, there will be fluctuations between them.\nB. This means that, from mid-april through july, most popular stock markets are open. [substeps] Look online for the month when there will be two stock markets open for you to preview the strength and overall cost of your investment.\nC. If you need to calculate market capitalization on the weekend, then you will have to use friday's closing stock price. [title] Determine the total number of outstanding shares.\nD. [substeps] Stock market securities are owned by a stockbroker, not a trust. A stockbroker is a corporation with a direct stake in a certain financial institution.","prompt":"[header] How to calculate the market value of equity [title] Make sure you use up-to-date information. [step] The market value of equity is constantly changing, and thus the calculation requires the newest information in order to be accurate. Stock markets are open from monday to friday.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5638,"error":null,"output":"C"}
{"question_id":"hellaswag:00009995","item_index":39,"attempt":0,"prompt_hash":"b2043dbe589e8c6ef7c8bf86340d462dfa144acecad6b72b237a4a147733e1eb","question":"[header] How to change file dates [title] Download bulkfilechanger if you do not already have a copy in your system. [step] This utility program enables you to create lists of windows files and change their attributes. [title] Run bulkfilechanger.\n\nA. [step] As soon as your pc updates, you will be prompted to open bulkfilechanger by pressing and holding the command key at the top-left of your desktop. [title] Create a new file named bulkfilechanger.\nB. [step] This program is more sophisticated than bulkfilechanger because it can access whatever information you are deleting. [title] Right-click on bulkfilechanger's name.\nC. [step] It is a free program that runs in your pc's framework. [substeps] You must log into bulkfilechanger using your computer's external device or that of a trusted user.\nD. [step] When the main menu appears, click file and then add files. [title] Select the file (or folder) where you want to change the date/time attribute.","prompt":"[header] How to change file dates [title] Download bulkfilechanger if you do not already have a copy in your system. [step] This utility program enables you to create lists of windows files and change their attributes. [title] Run bulkfilechanger.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5637,"error":null,"output":"D"}
{"question_id":"hellaswag:00007079","item_index":40,"attempt":0,"prompt_hash":"baf45ed48c15f6179969384e89c03abb30ab10ef813b3c6d16af42ea860e01d5","question":"[header] How to transform yourself using makeup [title] Cleanse your face using warm water and your bare hands. [step] Dab it dry with a soft clean towel. [title] Use a good foundation and rub evenly on the face, being careful not to use to much in an area.\n\nA. [step] Start from the far corner of your face, as shown, but the middle of your face is always the most natural. Blend again with a moistened tissue or a swipe of clear makeup remover.\nB. [title] If you have any spots, use a good quality concealer stick and blend it in with the foundation. [title] Now get a bronzer and a brush, and gently apply over the face to give you that glowing goddess look.\nC. [step] Make sure that the foundation is left on your face for three minutes before moving on. [title] Use a makeup sponge to remove excess powder/finish from the face.\nD. [step] Show off! Good makeup affects the shape of our face, making our appearance the way it is. The first layer of foundation should be a powder, not a gel.","prompt":"[header] How to transform yourself using makeup [title] Cleanse your face using warm water and your bare hands. [step] Dab it dry with a soft clean towel. [title] Use a good foundation and rub evenly on the face, being careful not to use to much in an area.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5641,"error":null,"output":"C"}
{"question_id":"hellaswag:00006550","item_index":41,"attempt":0,"prompt_hash":"0a85635fdc827f4b3ed65afe7ba6e58357c6e73ebf2f9b1376c8326203b26075","question":"[header] How to dry out a pimple [title] Ask your doctor about retinoid treatments. [step] Topical retinoids are gels or creams that you apply to your face as an acne treatment. They are an effective treatment for reducing acne.\n\nA. These medications require a prescription from your doctor or dermatologist. [substeps] You will need to apply the medicine three times per week at first and then work up to applying it once per day as your skin adjusts to it.\nB. [substeps] You do not need to reuse a prescription retinoid. If your pimple is severe enough to warrant medical treatment, the doctor may advise blocking pores that provide deeper or more serious damage to the acne.\nC. With the residue of acne treated, the pimple may disappear on its own. [substeps] Retinoids can be applied twice a day for 2-4 weeks, twice a week for up to 6 weeks, or more if the pimple stays there for longer than a day.\nD. They are usually applied to your pimple twice a day, once in the morning and once in the night. When your doctor is treating you, they will inject the retinoids into the dermis which is where you see the blemishes.","prompt":"[header] How to dry out a pimple [title] Ask your doctor about retinoid treatments. [step] Topical retinoids are gels or creams that you apply to your face as an acne treatment. They are an effective treatment for reducing acne.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5637,"error":null,"output":"A"}
{"question_id":"hellaswag:00005823","item_index":42,"attempt":0,"prompt_hash":"7debef9263f94643a93276af710321989d21e0e1dab758c2597a5887697dba2b","question":"[header] How to get rid of laugh lines [title] Use moisturizers daily. [step] Using moisturizer can help hydrate your skin and reduce the appearance of fine lines and wrinkles, including those around your mouth. For the best results, use skin moisturizer that contains collagen, which can help plump up and repair your skin.\n\nA. [substeps] Look for a moisturizer that has a spf, such as spf 30, or including spf 30-35 with spf 15. Health professionals recommend against using products containing any higher ph than 70 or higher.\nB. [title] Apply skin exfoliants on a regular basis. [step] Exfoliating can help remove dead skin cells to reveal the healthier, younger skin cells that lie beneath and reduce fine lines caused by laughing and smiling.\nC. [substeps] The best moisturizer is calamine lotion (pho) and spf. Apply the moisturizer every night before you go to bed.\nD. [substeps] Turn off the faucet and let the water run for a minute. Set your carrier oil or bentonite oil on a towel, and splash water on your face.","prompt":"[header] How to get rid of laugh lines [title] Use moisturizers daily. [step] Using moisturizer can help hydrate your skin and reduce the appearance of fine lines and wrinkles, including those around your mouth. For the best results, use skin moisturizer that contains collagen, which can help plump up and repair your skin.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5638,"error":null,"output":"B"}
{"question_id":"hellaswag:00009475","item_index":43,"attempt":0,"prompt_hash":"eeb6df8c51948fe2d9ac0c5d019d60deac2958c9e48c374f96b5dc51feb3b876","question":"[header] How to style glitter boots [title] Go with glitter ankle boots for a versatile option. [step] Glitter boots come in all sorts of styles. If you want a low-key boot that you can wear with many different outfits, go with an ankle boot.\n\nA. [substeps] Ankle boots are similar to boots in that they have more ankle straps, their ankle band is thicker, and they are made with better flare. Ankle boots, however, are more comfortable that regular walking boots, as you can use one or more strap ankle boots.\nB. Ankle boots look great styled casually or dressed up. [substeps] This way, you won't be covered up to your knees in glitter! [title] Choose knee-high glitter boots for a bold look.\nC. This will offset some of the beauty of your boots and create a really casual, modern look. [substeps] If you want to wear a girly-girl or catwalk look, try stick to a dark purple or chocolate brown for a perfect, elegant alternative to your ankle boots.\nD. Because you won't be carrying large amounts of weight, glitter boots are a way to add volume and take away the mood on dresses, skirts, and other outfits. Choose small and large toe boots that you can wear with large outfits every day, or that match your wardrobe and budget.","prompt":"[header] How to style glitter boots [title] Go with glitter ankle boots for a versatile option. [step] Glitter boots come in all sorts of styles. If you want a low-key boot that you can wear with many different outfits, go with an ankle boot.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5638,"error":null,"output":"B"}
{"question_id":"hellaswag:00009342","item_index":44,"attempt":0,"prompt_hash":"422853ff5c96da414c8d2b5403c75d38a61b2b469a664c3bffe6920c7d247662","question":"[header] How to sign up for yahoo [title] Visit the yahoo homepage. [step] When you're ready to begin, click \" sign in \". This will take you to the login screen, where you'll be able to register a new account.\n\nA. [substeps] You will then be directed to a page where you can register your yahoo id, microsoft email address, gmail id, and password. Click \" sign up \" to load your yahoo id, and enter the information given on the screen.\nB. [title] On the next page, click \" create new account \". [step] Clicking on the \" sign in \" link should take you to a screen where you'll have the option to log into an existing account or create a new one.\nC. [substeps] If you are already logged in to yahoo, skip to step 5, because yahoo may ask you to create an account here. If you have already created an account, enter your yahoo id information on the pop-up, and tap sign in.\nD. You can also create an account on a whim. You can create one for free or for a fixed sum, depending on what you want.","prompt":"[header] How to sign up for yahoo [title] Visit the yahoo homepage. [step] When you're ready to begin, click \" sign in \". This will take you to the login screen, where you'll be able to register a new account.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5637,"error":null,"output":"B"}
{"question_id":"hellaswag:00004656","item_index":45,"attempt":0,"prompt_hash":"0b4c9eaf369dda34c68e5c87131947130be860f658bb8608590019aa7723e6eb","question":"[header] How to cut fringe bangs [title] Cleanse your hair. [step] Wash your hair thoroughly with shampoo in a shower or sink-for best results, try a high-quality one designed for your hair type. Make sure your hair is thoroughly rinsed of shampoo before proceeding.\n\nA. [substeps] Cutting your hair while it's dirty is not what stylists consider ideal. Unwashed hair has a good chance of containing patches of grease or product build-up, which can lead shears to slip and cut unevenly.\nB. [substeps] If your hair is really soaking water, you might consider getting a head or two of water added to your hair as well. [title] Create a fringe design with plastic or light colored hair.\nC. [substeps] If you're creating fringe with a hair wig, hold the wig in one hand with the wig holder dangling. Loosen and separate your wig holder without removing the wig holder itself.\nD. Rinse the shampoo with cool, running water in your hair and then close your hair up. Use a towel to dry your hair and then brush it dry with a paddle-brush.","prompt":"[header] How to cut fringe bangs [title] Cleanse your hair. [step] Wash your hair thoroughly with shampoo in a shower or sink-for best results, try a high-quality one designed for your hair type. Make sure your hair is thoroughly rinsed of shampoo before proceeding.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5637,"error":null,"output":"D"}
{"question_id":"hellaswag:00005776","item_index":46,"attempt":0,"prompt_hash":"4a42a3e2b87ef3dd21bf9c4fec8dfe5bf1709640ce0940c9988a36ef35e93933","question":"[header] How to prepare for an exam [title] Start studying early. [step] Give yourself more than enough time to review the material that was covered in class. You might want to gauge how soon to start studying by how much material you need to review.\n\nA. As long as you have some time, you can do it all at once. Having enough time before your exam will also make your studying go faster.\nB. If your work is a long one or if your courses are geared more toward multiple choice questions, begin studying two to three hours earlier in the semester. This is perhaps the simplest way to prep yourself for an exam.\nC. [title] Write down all your important information in advance. [step] If you need specific knowledge regarding how to impress and accomplish the specific goal you created, then use textspeak or did for your instructor's review of the material.\nD. For instance, if you have to review material for an entire semester, you might want to start studying a few weeks prior. However, if you are just taking a test that covers material over a few chapters, a week prior or even three to four days may be sufficient.","prompt":"[header] How to prepare for an exam [title] Start studying early. [step] Give yourself more than enough time to review the material that was covered in class. You might want to gauge how soon to start studying by how much material you need to review.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5638,"error":null,"output":"D"}
{"question_id":"hellaswag:00007504","item_index":47,"attempt":0,"prompt_hash":"a3953fcdfff8c47da536a81716a60728a0fcd18a1c35edddec2f7bad56b6e293","question":"[header] How to vacation in puerto rico [title] Think about your priorities. [step] As a relatively populous island nation with diverse regions, puerto rico offers a variety of options for travelers. Before planning the specifics of your trip, you need to decide on the purpose of your trip.\n\nA. You can opt to have it just one day or only an hour over a couple of days to keep up your travel time and avoid crowds. [substeps] Spend plenty of time trainings with your fellow travelers.\nB. Try out the following options to help you get started : [substeps] One of the great things about puerto rico is its unique sights. Since the average tourist season passes between june and november, you may want to show up with your tourist ticket and some hotels nearby.\nC. Think about if your trip to puerto rico will be : [substeps] A relaxing vacation. An adventure, full of hiking, urban exploration, and activities like snorkeling.\nD. Sometimes traveling to puerto rico is due to a desire to live on the islands or other asian culture. Other times, traveling from a city city or large city to a short, red-colored city in puerto rico can lead to de repercussions or a situation where puerto rico would be intolerable.","prompt":"[header] How to vacation in puerto rico [title] Think about your priorities. [step] As a relatively populous island nation with diverse regions, puerto rico offers a variety of options for travelers. Before planning the specifics of your trip, you need to decide on the purpose of your trip.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5638,"error":null,"output":"C"}
{"question_id":"hellaswag:00009028","item_index":48,"attempt":0,"prompt_hash":"65534d69b8e80f5ef01b109b453fd38ecd2b377aa0786a62653ab3d673a31a88","question":"[header] How to get a job as a mortuary makeup artist [title] Spread the word about your services among local business. [step] List your services online and leave business cards at salons that don't have a mortuary makeup artist on staff. Making connections within the community can lead to referrals, individual freelance jobs, or a full-time position.\n\nA. Ask if there are any publicly-traded businesses in your area that would be willing to split vacation time with you. [substeps] Most body shops hire large numbers of makeup artists, so you might be able to earn a little extra.\nB. Be selective and don't neglect to showcase a strong foundation to build a search for jobs. [substeps] If the attraction is for time-sensitive applicants, emphasize it to your clients and let them know how much they mean to you.\nC. [substeps] Keep in mind that there will be more opportunities and higher-paying mortuary makeup artist jobs in highly populated areas. [title] Reach out to funeral homes in your area.\nD. [substeps] Locate a local mortuary makeup artist and ask them to write a hiring letter and share it with prospective employers. Keep in mind that even if you have been hired for a smaller role, employers still may want to hire you again.","prompt":"[header] How to get a job as a mortuary makeup artist [title] Spread the word about your services among local business. [step] List your services online and leave business cards at salons that don't have a mortuary makeup artist on staff. Making connections within the community can lead to referrals, individual freelance jobs, or a full-time position.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6573,"error":null,"output":"C"}
{"question_id":"hellaswag:00002654","item_index":49,"attempt":0,"prompt_hash":"b160da40672d2308945a408fed476f0c579d208c7a6c7dae8f192b1f42a2d1dd","question":"The men square up and they begin hitting and kicking at one another with brief moments of stopping in between their punches and their kicks. one of the guys\n\nA. kicks the other one too hard and the one who got kicked gets mad, rips his gloves off of his hands, walks away and angrily throws his gloves.\nB. throws a tantrum and runs to the woods to get away from his mates.\nC. a man wears a brown t-shirt and the other one begins wearing a green t-shirt.\nD. runs over to where they both started and hits the man on the ground with a hammer sized hit to the head that throws him in the air high up and his head lolls against the ground until he falls off of the incline.","prompt":"The men square up and they begin hitting and kicking at one another with brief moments of stopping in between their punches and their kicks. one of the guys","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5646,"error":null,"output":"A"}
{"question_id":"hellaswag:00004003","item_index":50,"attempt":0,"prompt_hash":"3d96c16f7e4b654af442ad963a8c7d58f4b9b9f353fba3f7cc73b7e6e5955e52","question":"[header] How to fold up a cosco high chair [title] Wipe the tray clean. [step] Clean any spilled food or drink off the tray with a soft, damp cloth. [substeps] Periodically clean the tray with soapy water and a clean, soft cloth to sanitize it.\n\nA. Do not attempt to wash the tray in a dishwasher. Dry the tray with a clean towel after washing it.\nB. If you can't wipe off the tray, you can wipe its surface with a clean microfiber cloth. [title] Remove the drawer from the right side of the chair.\nC. [title] Lay the tray back down in its regular position. [title] Fasten the top, bottom, and back sides of the tray.\nD. [title] Tip the tray upside down onto a clean paper towel to dry it. [step] For large people, this method of covering a large surface can be tricky.","prompt":"[header] How to fold up a cosco high chair [title] Wipe the tray clean. [step] Clean any spilled food or drink off the tray with a soft, damp cloth. [substeps] Periodically clean the tray with soapy water and a clean, soft cloth to sanitize it.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5637,"error":null,"output":"A"}
{"question_id":"hellaswag:00004549","item_index":51,"attempt":0,"prompt_hash":"f345fadffb68c4980e6fdabc797cd5d002c97d1c73a03412d1648a533459311c","question":"[header] How to braid short hair [title] Part your hair down the center. [step] Use a comb to draw a part down the center of your hairline. Brush the hair flat on either side of this part.\n\nA. This will prevent hair from bumping against the side of your head too much as you work on braiding. Use a comb to gently divide the ends of your hair into a single strand.\nB. From this point forward, you will want to part your hair down the center of your head. [substeps] Use a fine-tooth comb to gently separate your hair into four half-inch sections.\nC. [substeps] For this style, you'll need to create two standard braids along the front sides of your head. These braids should roughly mirror one another in position, width, and length.\nD. [substeps] If your hair bunches up, then you should wind it first, and for textured hair, then you should wind it. [title] Pin the top section of hair up into a ponytail.","prompt":"[header] How to braid short hair [title] Part your hair down the center. [step] Use a comb to draw a part down the center of your hairline. Brush the hair flat on either side of this part.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5639,"error":null,"output":"C"}
{"question_id":"hellaswag:00003531","item_index":52,"attempt":0,"prompt_hash":"35b6fd262ff622715b91cc40b7f2b8912ee3a780d242b4f7a978618759a26925","question":"[header] How to smoke a cheyenne cigar [title] Remove the packaging. [step] Cheyenne cigars come in packs similar to typical cigarettes. Remove the cellophane wrapper encasing the pack and open up the lid.\n\nA. [substeps] The bag, along with all the cigars inside the baggie, should be composed of four layers. The cellophane is referred to as a \" cigar wrap.\nB. Examine the tobacco you've taken out and, if necessary, peel the seal back. You should see a few heads of tobacco.\nC. Dispose of the cellophane in the garbage once you've removed it completely. [substeps] You should be able to smell the cigars upon removing the cellophane.\nD. If you have a handheld cigar lighter, pull the cellophane back to reveal its bottom, revealing the cigars inside. You should see a microfilm film easily discharged into the zippo.","prompt":"[header] How to smoke a cheyenne cigar [title] Remove the packaging. [step] Cheyenne cigars come in packs similar to typical cigarettes. Remove the cellophane wrapper encasing the pack and open up the lid.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5646,"error":null,"output":"C"}
{"question_id":"hellaswag:00008976","item_index":53,"attempt":0,"prompt_hash":"43ff87ecd437cc0afa8ec07dc785028124183145a2d5e680dde2d8b59f5dd976","question":"[header] How to remember to do something without writing it [title] Learn how memory encodes. [step] Information from our senses must be changed into a form that we can store as a memory. If we want to remember an image that we saw, for example, it can be encoded as visual information into our memory bank.\n\nA. [substeps] Write down your impressions of these images. Try to stay comfortable with these images in your memory bank.\nB. Ask yourself if writing out a sentence makes it easier for you to remember. Think about the question's meaning for the entire time you have written it.\nC. [substeps] To make it easier for you to remember information when you're forgetting something, it should be easier to remember from a memory memory bank. It's best to work through your memory as your memory moves.\nD. The three ways that information is most often encoded are: visual (picture), acoustic (sound) or semantic (meaning). Additionally, some people can be tactile encoders.","prompt":"[header] How to remember to do something without writing it [title] Learn how memory encodes. [step] Information from our senses must be changed into a form that we can store as a memory. If we want to remember an image that we saw, for example, it can be encoded as visual information into our memory bank.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6573,"error":null,"output":"D"}
{"question_id":"hellaswag:00009182","item_index":54,"attempt":0,"prompt_hash":"0fa1065cc9977932a402a93188f168b2427f59a7db636261fcec32c48e1988b2","question":"[header] How to make a scottish rosette and sash [title] Choose your fabric. [step] You can use the tartan fabric of your choice to make your sash. Select the color and style of the fabric you want to use.\n\nA. You can use a plaid, an old plaid plaid or a striped scottish rosette along with the color of your sash. Try burlap, common cloth with small ways of stitching and embroidery.\nB. Avoid fabrics you find too recognizably scottish. [title] Create a pattern to help with accents.\nC. You will need one yard of the fabric. [substeps] Plaid is a traditional fabric for a scottish sash, but you can also use a solid color if you prefer.\nD. The color of your sash will be determined by its dimensions (for example, if you are making a tall sash and the fabric is of great length). For example, if you are making a large sash and the material is of great length and width, then you might choose this fabric.","prompt":"[header] How to make a scottish rosette and sash [title] Choose your fabric. [step] You can use the tartan fabric of your choice to make your sash. Select the color and style of the fabric you want to use.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5645,"error":null,"output":"C"}
{"question_id":"hellaswag:00008923","item_index":55,"attempt":0,"prompt_hash":"f4df5788b5be7f58548f4b786a8f1d3f82583c4c72dde1c765aa082694dd3f52","question":"[header] How to have a healthy tongue [title] Practice basic oral hygiene. [step] The best way to encourage a healthy tongue is to have a healthy mouth. Adopting good dental habits will protect your tongue as well as your gums and teeth.\n\nA. [substeps] Use purified water and a gentle soap every day. Brush your tongue thoroughly and break up the dental pieces with a metal spoon.\nB. Make these skills a daily habit : [substeps] Brush your teeth twice a day. Use a soft or medium bristled toothbrush, and brush for at least two minutes at a time.\nC. Your mouth needs to be clean, plump, and clear of bacteria and other built up food. [substeps] Floss at least twice a day, and practice good dental hygiene that emphasizes good oral health.\nD. Proper oral hygiene means that people should trim their tongue so that teeth are not exposed to air. Keep an appropriate amount of control over your tongue.","prompt":"[header] How to have a healthy tongue [title] Practice basic oral hygiene. [step] The best way to encourage a healthy tongue is to have a healthy mouth. Adopting good dental habits will protect your tongue as well as your gums and teeth.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6572,"error":null,"output":"B"}
{"question_id":"hellaswag:00009556","item_index":56,"attempt":0,"prompt_hash":"fadb9f4849a94366e04948a64b6da5c8e2b876eae718390dec369d00524bc447","question":"[header] How to prepare for eid ul adha [title] Buy the required sacrificial animal 1-2 months in advance. [step] Doing things in advance saves you so much time. Buy the animal from a verified bazaar or shop or seller.\n\nA. Check for the necessities in the animal as per the islamic shariah. Ask an islamic scholar for help if you are unsure of the type of animal to buy for sacrifice, or read the quran for further information.\nB. [title] Ensure that you do not acquire a hunting license if you do not have a clear understanding of the breed. [step] Buy a hunter's license or some health-safe business license to protect yourself while hunting eid ul adha, and keep yourself in compliance and peace.\nC. [title] Buy the farm when the time comes. [step] The first 4 months in only a 1 month period are usually time spent at a bazaar, farmer's market, barn or farm or farm work.\nD. [substeps] Many restaurants offer this phase of the process, so only buy from vendors in order to save money. If you have any questions about why your vegetarian bread is not vegetarian, ask around.","prompt":"[header] How to prepare for eid ul adha [title] Buy the required sacrificial animal 1-2 months in advance. [step] Doing things in advance saves you so much time. Buy the animal from a verified bazaar or shop or seller.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6572,"error":null,"output":"A"}
{"question_id":"hellaswag:00007148","item_index":57,"attempt":0,"prompt_hash":"312eabb1604b4b830764f138bc8e87ad875761c0d3dbdb49c3232a9e93e8f15c","question":"[header] How to do your hair like a dancer [title] Pull all your hair back into a tight ponytail, high on the back of your head. Secure with a hair elastic [title] Grasp your ponytail in your hand and begin to twist the hair to the left as you would ring out a towel. [title] Wrap your hair tightly in a counterclockwise circle around the ponytail holder. [step] Continue twisting until all hair is coiled.\n\nA. [title] Tip the hair over to the right side of the ponytail holder. [step] Center the ponytail holder on the back of your head and twist the hair to the left.\nB. Keep your hair tight and reverse this until all hair is twisted into the top of your head. [title] Twist your hair around until there is a gap in the ponytail.\nC. You will now be covering the top of your ponytail with a hair tie. [title] You can now secure all of the loose bottom part of your hair.\nD. [title] Put another ponytail holder around the bun and the loose ends of the ponytail. [step] An optional method is to skip the second ponytail holder and pin as you go, adding pins approximately every 90 degrees.","prompt":"[header] How to do your hair like a dancer [title] Pull all your hair back into a tight ponytail, high on the back of your head. Secure with a hair elastic [title] Grasp your ponytail in your hand and begin to twist the hair to the left as you would ring out a towel. [title] Wrap your hair tightly in a counterclockwise circle around the ponytail holder. [step] Continue twisting until all hair is coiled.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6572,"error":null,"output":"A"}
{"question_id":"hellaswag:00006355","item_index":58,"attempt":0,"prompt_hash":"9395c8f656027273d927742726a42f15186f70367e11087db2d755db95fddf23","question":"[header] How to clean makeup sponges [title] Dampen the sponge with fresh water. [step] In order for the soap to work into a cleansing lather, your dirty sponge must be wet. Turn on your faucet and run your reusable sponge under warm water.\n\nA. Be sure to take it out periodically to flush out any soap. [substeps] If your sponge doesn't have a handle, gently squeeze it dry with a soft, non-abrasive rag and hold it with both hands.\nB. Lather your sponge (or dishcloth) for about 30 seconds. This will start to rinse away any excess water, or at least begin to loosen up any dirt.\nC. Squeeze the sponge to remove the excess water. [substeps] Your sponge should be damp, not soaking wet.\nD. Do this gently with the sponge to make sure that all the soap is removed. [substeps] Remove as much of the makeup as possible using a soft, non-abrasive sponge.","prompt":"[header] How to clean makeup sponges [title] Dampen the sponge with fresh water. [step] In order for the soap to work into a cleansing lather, your dirty sponge must be wet. Turn on your faucet and run your reusable sponge under warm water.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6573,"error":null,"output":"C"}
{"question_id":"hellaswag:00005022","item_index":59,"attempt":0,"prompt_hash":"e05b1f771ed38e3eca253a846ef4a50f8c09cd2387aac52e874747e252fd2911","question":"[header] How to help an itching dog [title] Check for fleas. [step] If you see your dog scratching constantly, check its skin for dark brown or black specks. If you are not certain if your dog has fleas, comb it gently with a fine tooth comb, getting very close to the skin.\n\nA. Fleas can build up on bald spots and as a result, your dog's skin may become irritated. Flea bites can also lead to a rash.\nB. If it starts scratching, stop when it stops, and check for fleas at the first sign of irritation. Fleas may look like tiny red pinworms, but may be tiny feces or bite marks.\nC. These clear specks could indicate fleas. [substeps] If you spot fleas in a stray dog, wait for it to relax before you begin treating it.\nD. Tap the comb onto a wet, white paper towel and look for reddish-brown flecks, often called flea dirt. You may also catch fleas as you comb.","prompt":"[header] How to help an itching dog [title] Check for fleas. [step] If you see your dog scratching constantly, check its skin for dark brown or black specks. If you are not certain if your dog has fleas, comb it gently with a fine tooth comb, getting very close to the skin.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":8973,"error":null,"output":"D"}
{"question_id":"hellaswag:00009752","item_index":60,"attempt":0,"prompt_hash":"1e6267b14d5df67002bacfb85dade566c9ed101b9996c7ab4ea9b5942196a0ad","question":"[header] How to put boots on [title] Wear boot socks. [step] The right socks will help you put on cowboy boots. Wear boot socks or ribbed athletic ones that go up to your calf.\n\nA. The sock material will help you wiggle your feet into boots along with your body weight. [title] Sit down on a chair or bench.\nB. Use dry-wicking socks that are often worn to bed, and keep the floor area dry. [title] Wear cowboy boots with any other footwear.\nC. [substeps] Top off your boot boots with skinny leggings or a thin top. [title] Choose the right boot for your ankles.\nD. Ask a friend who has boots for their help. If you have extra socks, put on them whenever you're working out.","prompt":"[header] How to put boots on [title] Wear boot socks. [step] The right socks will help you put on cowboy boots. Wear boot socks or ribbed athletic ones that go up to your calf.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6379,"error":null,"output":"A"}
{"question_id":"hellaswag:00003415","item_index":61,"attempt":0,"prompt_hash":"35e21f40af07cbf528a874e45401e9f6376feb8f9a66f7a01c5d32fa01250b84","question":"[header] How to look and smell pretty (for preteens ) [title] Brush your teeth with a toothpaste that has a nice mint smell. [step] You want your teeth to smell fresh and look white! Floss and use mouth wash if needed. [title] Apply some chap stick so your lips are not dry and don't look flaky.\n\nA. [step] Apply some light colored, shiny lip gloss to your lips. [title] Put some scented lotion on your legs.\nB. [step] It would also be best if you had little white streaks in your teeth. [title] Put some deodorant into your body.\nC. [step] Now you are ready to look! Again, we try this way to do it all properly. [title] Wet a cotton pad, and dab it on your lips and make sure everything stays covered.\nD. [step] Also, brush your teeth with a round brush with a soft bristles, if needed, to make them looking plump instead of chapped. If you have oily lips, you can use a sugar rinse.","prompt":"[header] How to look and smell pretty (for preteens ) [title] Brush your teeth with a toothpaste that has a nice mint smell. [step] You want your teeth to smell fresh and look white! Floss and use mouth wash if needed. [title] Apply some chap stick so your lips are not dry and don't look flaky.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6378,"error":null,"output":"A"}
{"question_id":"hellaswag:00006142","item_index":62,"attempt":0,"prompt_hash":"cf39b62e6a084f094aea3155427e3be3cfe2f0cda4003a01674b7d0aae67d54f","question":"[header] How to look good at the beach [title] Start with a good shave, waxing or hair removal cream. [step] Remember that grooming that you've neglected over the winter? Remove that growth from your legs, armpits and bikini area before you do anything else. [title] Get gorgeous skin.\n\nA. [step] You know that some sunbathing will give you back your summer glow, but you don't want to make your seasonal beach debut looking pasty. [substeps] Zap any acne or blemishes with a salicylic acid cream.\nB. [step] On a large basis use conditioners, foundations, basics, these will keep your skin smooth, hydrated and ready for the sun. You don't want any sun exposure and should use synthetic products.\nC. [step] Wash your face, apply the face stabilizer before you are left in the sun. Apply an oil or lotion on your face and you smell good.\nD. [step] If you want that look, put some exfoliating lotion on your legs before going out because your skin will be drying out. Try these primer and exfoliating creams : [substeps] Full body bronzer with a spf of 15.","prompt":"[header] How to look good at the beach [title] Start with a good shave, waxing or hair removal cream. [step] Remember that grooming that you've neglected over the winter? Remove that growth from your legs, armpits and bikini area before you do anything else. [title] Get gorgeous skin.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6380,"error":null,"output":"A"}
{"question_id":"hellaswag:00007871","item_index":63,"attempt":0,"prompt_hash":"afdbea77c9333e1cd1b22b0fafdadfdc4659018e2a0da7f1c365701add9153f8","question":"[header] How to do a maple syrup hair mask [title] Mash the avocado and banana together. [step] Place ½ of an avocado and a whole, peeled banana into a medium sized bowl. Use a fork to mash the two ingredients together.\n\nA. [substeps] If you don't have avocado mash, you can use any fresh fruit that you like. Consider changing the avocado and banana recipes so that you always have the fresh fruit.\nB. [substeps] In a separate bowl, mix 1 ½ teaspoons (3.5 ml) of avocado with ½ teaspoon (1.5 ml) of olive oil. Add 2 tablespoons (30 ml) of sugar and the avocado mixture, if you'd like to add chopped fruit.\nC. Continue to mash them with your fork until they are evenly mixed together. [substeps] If your hair is very long, you may need to double all of the ingredients to end up with enough mask.\nD. [substeps] You can blend the avocado using a paddle attachment or by running it under a faucet. Make sure that you scrape the banana beneath the water, not into the bowl.","prompt":"[header] How to do a maple syrup hair mask [title] Mash the avocado and banana together. [step] Place ½ of an avocado and a whole, peeled banana into a medium sized bowl. Use a fork to mash the two ingredients together.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6379,"error":null,"output":"C"}
{"question_id":"hellaswag:00004155","item_index":64,"attempt":0,"prompt_hash":"be8c6a9f1eee91a05c30274857029302b9d005ff93b2a26b5a377a209783bc88","question":"[header] How to wear nude heels [title] Choose a heel that complements your figure. [step] Stilettos and kitten heels flatter slim body types and slim ankles. Look for chunkier, stacked heels if you're curvy or want to balance larger and more muscular calves.\n\nA. Another rule that constantly follows is that heels are usually too shoes that are too high. Heels that are too high can turn your hourglass figure into a model or your hips into a sloppy guy.\nB. Cut your heels in a nail-length for a more hourglass look. Give heels varying heights such as the height of your hips for a more exotic look.\nC. Choose a cone heel for any body type. Also look for platform wedges, which flatter most body types and can add inches if you're short.\nD. Heels that don't feature boots or platform heels can add to your image and make you look very high-fashion. [substeps] Tall women should wear heels that are closer to the knee or above.","prompt":"[header] How to wear nude heels [title] Choose a heel that complements your figure. [step] Stilettos and kitten heels flatter slim body types and slim ankles. Look for chunkier, stacked heels if you're curvy or want to balance larger and more muscular calves.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6400,"error":null,"output":"C"}
{"question_id":"hellaswag:00003387","item_index":65,"attempt":0,"prompt_hash":"26230040dfb593fa989c703da14eba492fd90520b7daa51de96b02fd3fdeb9ad","question":"[header] How to sharpen hair clippers [title] Unscrew the blades. [step] Find and unscrew the screws that attach the blades to the rest of the clippers. On most hair clipper models, there are two screws at the base of the blade.\n\nA. Once these are removed, gently detach the blades and any pieces holding them in place. [substeps] If the bottom blade does not pop out easily, use a pair of tweezers to pull it off the base.\nB. Use a flathead screwdriver to loosen these screws. [substeps] The blades of the clippers will be very sharp and will pull up strands of hair from underneath your fingernails.\nC. One screw is on the left and the other screw is on the right. Then slide off of the bottom of the blades.\nD. This is where you will place the blades, usually on the back side of the clippers. [substeps] Note: the front of the teeth is facing in the direction you cut, not the top side of the tooth.","prompt":"[header] How to sharpen hair clippers [title] Unscrew the blades. [step] Find and unscrew the screws that attach the blades to the rest of the clippers. On most hair clipper models, there are two screws at the base of the blade.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6401,"error":null,"output":"C"}
{"question_id":"hellaswag:00000194","item_index":66,"attempt":0,"prompt_hash":"2bca7964569473ddfe1f95a60624aa5e0dbadf9d3a2f6f5d35e596bec9effc34","question":"We see the title introducing us to jesse. Jesse the dog runs and catches frisbee. we\n\nA. see an opening title screen.\nB. see a guy hold a blue frisbee in the air and tell that the dog has a pink frisbee and we see its pink feet.\nC. see jesse close up in the camera.\nD. see a violinist doing a piano clip.","prompt":"We see the title introducing us to jesse. Jesse the dog runs and catches frisbee. we","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6405,"error":null,"output":"A"}
{"question_id":"hellaswag:00007251","item_index":67,"attempt":0,"prompt_hash":"2dab04ae41d0c5c42922e8f624d6433b567098571abc288cf3fed61c34d47057","question":"[header] How to practice mindful eating [title] Take at least 20-30 minutes to eat an entire meal. [step] Many health professionals, especially registered dietitians, recommend that you take at least 20 to 30 minutes to eat your entire meal. This helps support many of the other mindful eating practices.\n\nA. Trying to eat only what is needed or necessary for your health can cause your digestive system to decline and lead to nutrient deficiencies. [substeps] Take 30-60 minutes to a session of a mindful eating practice.\nB. It also gives you time to rest your eyes and slowly eat fresh, healthy food rather than snacks, which might have small starches in your food later at night. [substeps] Mindful eating practices include deep breathing, counting to 10, challenging yourself to consume fewer starches and eating smaller meals, and exercising instead.\nC. [title] Familiarize yourself with mindful eating. [step] Notice the foods that you eat, such as bread, table napkins, and bagels, in each bite.\nD. [substeps] The basis of this rule is that it takes 20 or 30 minutes for your stomach and digestive track to begin digesting and absorbing the food from your meal or snack. It takes at least 20 minutes for your gi system to signal your brain that it's had enough food.","prompt":"[header] How to practice mindful eating [title] Take at least 20-30 minutes to eat an entire meal. [step] Many health professionals, especially registered dietitians, recommend that you take at least 20 to 30 minutes to eat your entire meal. This helps support many of the other mindful eating practices.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6401,"error":null,"output":"D"}
{"question_id":"hellaswag:00004860","item_index":68,"attempt":0,"prompt_hash":"aad509df935a93ec9872f04da3f74632cb4dbfd86fe90fcd5cdc08376a6ee0d0","question":"[header] How to paint french doors [title] Lay a tarp under the doors. [step] While you can remove the doors to paint them, leaving them in the door frame makes the job easier. Protect your floors by spreading a plastic tarp out under the doors.\n\nA. You can purchase a tarp at a home improvement store or wherever else paint supplies are sold. [substeps] Cardboard and other absorbent scrap material can be used as a makeshift tarp.\nB. [title] Mix up a generous amount of acrylic paint with a lot of green. [step] Mix together 11 squirts of warm acrylic paint with 10 drops of green food coloring.\nC. This will help keep rain and wind out. [title] Pound your nails with hammer for a more precise finish.\nD. [title] Extend a pipe through the door frame. [step] If you're painting the doors with metal screws or other tools, you'll need to stretch out a pipe carefully.","prompt":"[header] How to paint french doors [title] Lay a tarp under the doors. [step] While you can remove the doors to paint them, leaving them in the door frame makes the job easier. Protect your floors by spreading a plastic tarp out under the doors.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6400,"error":null,"output":"A"}
{"question_id":"hellaswag:00006401","item_index":69,"attempt":0,"prompt_hash":"6babb1265ba10bf4943ef2b8c7f7de298b96510b4abe7dc2b7b08cec41489fd8","question":"[header] How to connect with an expectant parent as adoptive parents [title] Identify websites that connect parents. [step] Some websites allow expectant birth parents to flip through profiles of parents looking to adopt. These websites are a great way to connect expectant parents with adoptive parents.\n\nA. For instance, wechat is a hip and trendy website for families who get together to make a adoption. [substeps] Some, more advanced forums, like omewtupraminski, visticiftraxalized, have groups for determining how to link parents to their own web-pages.\nB. You can use search engines like amazon or care.com to search for that website. [substeps] Look for the websites that connect parents to moms who are still supporting the father through adoption.\nC. [substeps] There are websites specifically aimed for actual parents looking to adopt their child and parents trying to find a child who may qualify. [title] Adopt a baby that is breastfed.\nD. However, it's crucial that you research each website thoroughly to ensure that the site is safe and reputable. [substeps] One website is america adopts, which allows you to create an online profile.","prompt":"[header] How to connect with an expectant parent as adoptive parents [title] Identify websites that connect parents. [step] Some websites allow expectant birth parents to flip through profiles of parents looking to adopt. These websites are a great way to connect expectant parents with adoptive parents.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6401,"error":null,"output":"D"}
{"question_id":"hellaswag:00001748","item_index":70,"attempt":0,"prompt_hash":"e883f494ea7093545ed85279310e3abeb1f532c31fe988313e8a15a3e63b594f","question":"A man in a hardhat and goggles stands in a tiled room talking then we see the man cutting tiles with tools. We see a man tiling and another man looking at blueprints. we\n\nA. see the end screen and see a clock.\nB. watch them measure and then wax the tiles and continue to talk.\nC. see two men tiling the wall and floor in two different room.\nD. see a picture of the man and text on a wall.","prompt":"A man in a hardhat and goggles stands in a tiled room talking then we see the man cutting tiles with tools. We see a man tiling and another man looking at blueprints. we","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6404,"error":null,"output":"C"}
{"question_id":"hellaswag:00006497","item_index":71,"attempt":0,"prompt_hash":"dfc4e8e943bf8bcc66dbcbef45730f81f232240a266fea09ca7d943d4f1f1ab9","question":"[header] How to get your skin silky, smooth, soft, shining and healthy [title] Start each day with dry brushing. [step] Dry brushing is an ancient exfoliation technique designed to slough off dead skin and stimulate your body's circulation. Dry brushing every day immediately makes your skin look brighter, and if you keep up the routine your skin will begin to glow.\n\nA. [substeps] Choose a dry brush made from natural fibers, rather than plastic bristles. Natural bristles aren't as rough on your skin.\nB. [substeps] When brushing your hair, use gentle, circular motions rather than a thick brush. Doing too much brush can result in excessive bleaching and staining.\nC. [substeps] To begin with, sit on the bathroom floor with a damp cloth or towel ready. Squeeze a dollop of olive oil onto the cloth.\nD. [substeps] You can take a shower or go for a walk. If you need to do something else but eat, go for it.","prompt":"[header] How to get your skin silky, smooth, soft, shining and healthy [title] Start each day with dry brushing. [step] Dry brushing is an ancient exfoliation technique designed to slough off dead skin and stimulate your body's circulation. Dry brushing every day immediately makes your skin look brighter, and if you keep up the routine your skin will begin to glow.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6399,"error":null,"output":"A"}
{"question_id":"hellaswag:00005959","item_index":72,"attempt":0,"prompt_hash":"2d55f3b5543df1757b763a4f50de769201b8f81e7bd31186832ad4b4cb5caaa3","question":"[header] How to prevent kidney stones from recurring [title] Determine which kind of kidney stone you have had. [step] Ask your doctor to identify your particular kind. It is important to know which particular type you have had, so you can work on specific ways to prevent it from recurring.\n\nA. For example, there is a more specific type of kidney stone commonly known as chronic kidney stones. The most common type of kidney stones are a cystine kidney stone.\nB. [title] Think about who will be doing the surgery. [step] You will have to decide what type of surgery is most appropriate for you.\nC. [title] Ask your doctor to study your kidney stones. [step] Your doctor will dissect your kidney stones and perform a simple exam to determine what type of kidney stone you have.\nD. Ensure your doctor checks your parathyroid to rule it out as a factor in the formation of kidney stones. [substeps] Calcium stones are caused by unused calcium not flushed out in urine and is collected in the kidneys.","prompt":"[header] How to prevent kidney stones from recurring [title] Determine which kind of kidney stone you have had. [step] Ask your doctor to identify your particular kind. It is important to know which particular type you have had, so you can work on specific ways to prevent it from recurring.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6399,"error":null,"output":"C"}
{"question_id":"hellaswag:00007618","item_index":73,"attempt":0,"prompt_hash":"1df6e095135e4ee58b6c7f71621c017a43a4d9c237e5db735890237d9d9ce6c4","question":"[header] How to develop a good fashion sense [title] Sort through your drawers/wardrobe. [step] It may seem boring and just like a chore, but will be useful and make looking through your clothes much easier. Take out everything and put on the bed.\n\nA. It can be simple to do, such as sort through all their clothes, put them into a suitcase and immediately start rearranging your room. Or it can be something larger such as thinking up a collection and seeing what it needs, a few more items and a new outfit.\nB. [title] Draw, sketch, improve your nails...whatever you like and make sure you finish them before moving on. [title] Sleep for 15-30 minutes and if you're really dirty then get up and shower.\nC. Put your shoes away and it makes it easier to remove items from your kitchen, dining room, or bedside. Wash clothes, put them in the washer, and remove any old socks or booties.\nD. If something no longer fits or you just don't like it anymore, then put in a plastic bag ready to take to the appreciative charity shop. There's nothing wrong with getting rid of unwanted clothes.","prompt":"[header] How to develop a good fashion sense [title] Sort through your drawers/wardrobe. [step] It may seem boring and just like a chore, but will be useful and make looking through your clothes much easier. Take out everything and put on the bed.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6399,"error":null,"output":"D"}
{"question_id":"hellaswag:00005824","item_index":74,"attempt":0,"prompt_hash":"41f0c892c0a079e4f2d9c9eb3382db856b061cd1c98f60ba98c0f060fb6aacf2","question":"[header] How to change your image [title] Think about the way people see you now. [step] Before you can come up with the right new image, put some thought into the image other people have of you now. Reflect on your many interactions and how others often regard you.\n\nA. Identifying your existing image is the first step to making a gradual shift. [substeps] Ask yourself: what kind of things do people tend to say about you? What are some of your best qualities, according to your friends? See yourself from someone else's eyes.\nB. [substeps] Do you have good? Negative? Positive? People just assume what other people think. Your image should ideally change based on that.\nC. You are now recognisable and anyone can look at you. [title] Decide if you want to change yourself.\nD. [substeps] Think about the way people approach you. Have you ever seen people commenting of someone or addressing them in an unfriendly way? Of course it wouldn't feel like that today, but it might at the very least add to the image.","prompt":"[header] How to change your image [title] Think about the way people see you now. [step] Before you can come up with the right new image, put some thought into the image other people have of you now. Reflect on your many interactions and how others often regard you.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6400,"error":null,"output":"A"}
{"question_id":"hellaswag:00001952","item_index":75,"attempt":0,"prompt_hash":"32fdd37ba08fc3dfd8d6ade26a366ef170b8e7c714816b948b4b9af36311f716","question":"She points at the next set of cards. She pulls a card out and sets that on top of the cards. she\n\nA. pulls another card out to add to hers.\nB. pulls the yellow card out and puts both cards onto the white.\nC. then takes a third one to take a turn making sure the cards are all lined up.\nD. repeats the move in the next set of cards.","prompt":"She points at the next set of cards. She pulls a card out and sets that on top of the cards. she","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6401,"error":null,"output":"D"}
{"question_id":"hellaswag:00001254","item_index":76,"attempt":0,"prompt_hash":"8754580f43c500054716a31446008463701e933d429462583bb6f1149dfe578e","question":"She gets a clamp put under her eye pinching the skin. The piercer puts an earring in and she sits up. she\n\nA. does another piercing and then shows the sides of her face as she works.\nB. puts the earring back in.\nC. holds a towel to the piercing to catch the blood.\nD. gets a green pin and starts piercing the sides of her eyelid.","prompt":"She gets a clamp put under her eye pinching the skin. The piercer puts an earring in and she sits up. she","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6401,"error":null,"output":"B"}
{"question_id":"hellaswag:00008224","item_index":77,"attempt":0,"prompt_hash":"fce43d2a80d1eec7f43291f3a378203180168ed8d2c93b4d26936d7e0ba83a16","question":"[header] How to write an annotated bibliography [title] Determine the needs of your project or assignment. [step] Some annotated bibliographies are meant to summarize the sources that you have found. Others will also evaluate the main points of the sources.\n\nA. Be sure that the sources that you plan to include in the annotations aren't secondary to the points or sections. You should allocate ample space between each text and use double sided paper (both this and other copy-and-print supplies is better than paper that can't be easily scanned).\nB. Realize that you may have to fill out a few drafts to get started, though; you don't want to be towering over the page. [substeps] After you can already build a bibliography, you can start to take careful notes later on.\nC. The needs of your annotated bibliography depend on your project or assignment. Be in touch with your teacher or advisor for guidance.\nD. Figure out what your thesis usually includes before you start writing. [substeps] Read each source head to toe-they do not need to be detailed, they are just important elements of the text.","prompt":"[header] How to write an annotated bibliography [title] Determine the needs of your project or assignment. [step] Some annotated bibliographies are meant to summarize the sources that you have found. Others will also evaluate the main points of the sources.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6399,"error":null,"output":"C"}
{"question_id":"hellaswag:00000160","item_index":78,"attempt":0,"prompt_hash":"3222b6ae81fb4da765e63ad3f28ae7f49908438ec2855f0ab288b5193f19e998","question":"A woman in an orange shirt is sitting on an exercise machine. she\n\nA. gets on the machine and starts riding it.\nB. engages in some form of mount.\nC. begins working out on the exercise machine.\nD. is using a dodgeball machine.","prompt":"A woman in an orange shirt is sitting on an exercise machine. she","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6399,"error":null,"output":"C"}
{"question_id":"hellaswag:00005529","item_index":79,"attempt":0,"prompt_hash":"5350173b72721317f058b342443cd82a23f20108fe98bedd7f82e2515f23971c","question":"[header] How to clean a beauty blender [title] Prepare a dish of warm, soapy water. [step] Fill a small dish with warm water, then squeeze a dollop of gentle hand soap or shampoo into it. Mix lightly until suds form on the surface of the water.\n\nA. Soap tends to leave a film on the blades of the blender, so use cool water if you can. The hot water will soften the blades, making them easier to clean.\nB. [substeps] The dish of water can use all water or just warm, soapy water; some diluted soap should work just fine; any fresh, unused water will do. Soap with natural herbs or citrus will also work as well.\nC. Dish soap is generally not recommended, as it can irritate your skin when you need to use the sponge later. [substeps] Baby shampoo and \" gentle formula \" organic shampoos work especially well, but most soaps that are safe for use on your skin or hair can be used here.\nD. Pour the suds into a small-sized bowl. Don't fill the dish more than 3/4 full, otherwise the water will seep out and the blender will drown.","prompt":"[header] How to clean a beauty blender [title] Prepare a dish of warm, soapy water. [step] Fill a small dish with warm water, then squeeze a dollop of gentle hand soap or shampoo into it. Mix lightly until suds form on the surface of the water.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6400,"error":null,"output":"D"}
{"question_id":"hellaswag:00001323","item_index":80,"attempt":0,"prompt_hash":"a3ba0d242f67c380d9aece0cd71d0dbd14258049144884f596b7b568c131dec3","question":"A person is seen climbing up on top of a piece of exercise equipment and performing a gymnastics routine. the person\n\nA. continues spinning around on the equipment and ends by jumping down on the side.\nB. spins and gets off and performs several moves on the machine.\nC. follows the instructor all along the beam while spinning and doing acrobatics on the beam.\nD. spins his body around and around on the bar and ends by jumping off and appearing back on the side of the machine.","prompt":"A person is seen climbing up on top of a piece of exercise equipment and performing a gymnastics routine. the person","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6394,"error":null,"output":"A"}
{"question_id":"hellaswag:00009391","item_index":81,"attempt":0,"prompt_hash":"fcfd16b338b086ab704f6a5572e9d998193925f92f3c81d16411ff0d1fa5f0b4","question":"[header] How to use a pogo stick [title] Make sure that the pogo-stick is in good condition. [step] Check the pogo stick for loose parts before you get on. Make sure that everything is secured and fastened properly.\n\nA. [title] Take hold of the pogo stick's handle with one hand. [step] Place your other hand on the side of the pogo stick where the handle's prongs are.\nB. You don't want an important piece to fall off while you are jumping. [title] Make sure you are within the proper weight limit.\nC. [substeps] Slip the pogo stick around your finger and tap on any loose parts. If it's stuck, push and try again.\nD. They are also easy to disassemble if you notice any strange looking parts. [title] At the check-in point, carefully fill in the details on the pogo stick.","prompt":"[header] How to use a pogo stick [title] Make sure that the pogo-stick is in good condition. [step] Check the pogo stick for loose parts before you get on. Make sure that everything is secured and fastened properly.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6393,"error":null,"output":"A"}
{"question_id":"hellaswag:00002610","item_index":82,"attempt":0,"prompt_hash":"381a8af4631046adf090d4556d76e61cd93dcefbbb61ae58567bda480d021dd0","question":"We see people running down the street in a marathon. A motorcycle rides past going left. A lady in a bikini is followed. the camera\n\nA. pans around an airport.\nB. pans around the sidewalk.\nC. spins in people's faces.\nD. pans to the left.","prompt":"We see people running down the street in a marathon. A motorcycle rides past going left. A lady in a bikini is followed. the camera","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6392,"error":null,"output":"D"}
{"question_id":"hellaswag:00000223","item_index":83,"attempt":0,"prompt_hash":"6f7b4fadf7b303480f3d842e1f2b8651074562e66dccfbda51d108be6f963d95","question":"The audience cheers more and leads into people throwing a ball and others hitting it. more shots of the game being played\n\nA. are shown and the goalie standing on his side.\nB. are shown as well as others watching on the sides.\nC. are shown as well as men hitting and a man showing himself through the game again.\nD. are shown with a game ending in screens and then paper going and people sitting around the table.","prompt":"The audience cheers more and leads into people throwing a ball and others hitting it. more shots of the game being played","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6393,"error":null,"output":"B"}
{"question_id":"hellaswag:00002564","item_index":84,"attempt":0,"prompt_hash":"3f0ef843900c724742145063a14991398b13ebcdfcfa44dda8d18ff8b7ce28af","question":"A man jumps rope in a boxing ring in a mall as people watch. The man loses his rhythm and has to start over two times. the man\n\nA. stops jumping and his coach pulls the tape off his hands.\nB. goes over three times in circles but eventually wins, the man is beside him on the dirt.\nC. makes his way to the finish line checking his score as people walk by.\nD. attempts to fix the sin as he goes back and forth between rounds.","prompt":"A man jumps rope in a boxing ring in a mall as people watch. The man loses his rhythm and has to start over two times. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6393,"error":null,"output":"A"}
{"question_id":"hellaswag:00003983","item_index":85,"attempt":0,"prompt_hash":"952a453ba3aa135ebb796656a212ff847e844ca8ef86d0de8227f7306d224956","question":"[header] How to dispute a lot line survey as a defendant [title] Hire a lawyer. [step] As soon as you are served with the plaintiff's lawsuit, you should strongly consider hiring a qualified real estate lawyer. Specifically, look for a lawyer with experience litigating property boundary disputes.\n\nA. [substeps] While you are at the site, ask if your city or county bar association has a lawyer you can use. Alternatively, visit the website of each city or county bar association.\nB. If you absolutely must hire a lawyer, then find one that specializes in property, design, or engineering. A general rule of thumb is to hire an experienced real estate lawyer, preferably who can provide personal experience to expert witnesses.\nC. [substeps] Never hire an attorney who does not work for the state or federal government. Only hire lawyers who have experience representing local or state judges and arbitrators.\nD. A lawyer will help you negotiate a settlement, effectively cross-examine plaintiff witnesses, and retain a quality expert of your own. Start the search process by asking friends and family for recommendations.","prompt":"[header] How to dispute a lot line survey as a defendant [title] Hire a lawyer. [step] As soon as you are served with the plaintiff's lawsuit, you should strongly consider hiring a qualified real estate lawyer. Specifically, look for a lawyer with experience litigating property boundary disputes.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6524,"error":null,"output":"D"}
{"question_id":"hellaswag:00007435","item_index":86,"attempt":0,"prompt_hash":"0bf01af9e623fbcc2621a0b712bc3b37425673b4793abf40dd22d828378591af","question":"[header] How to raise an intelligent child [title] Read to and with your child. [step] Reading helps develop important communication skills, vocabulary, and grammar. In addition, reading helps children learn a lot more about the world.\n\nA. Reading helps with their critically evaluate their situation, experiences, and attempts to solve problems. [substeps] Look for online resources online and help the american academy of pediatrics program guide how to read communication updates and school talk articles on a topic available online.\nB. Reading books and learning lessons can make learning to understand and implement complex, complex grammar, and grammar. [substeps] Author: me reader: the main purpose of a book and children's literature.\nC. It expands their horizons and can bring them into contact with people and information that they might never experience otherwise. As a result, reading is one of the most important ways to raise and nurture an intelligent child.\nD. Understanding reading can help in the long run. It can help them understand how we think, why we use certain terms and phrases to solve problems, and why we enjoy nature.","prompt":"[header] How to raise an intelligent child [title] Read to and with your child. [step] Reading helps develop important communication skills, vocabulary, and grammar. In addition, reading helps children learn a lot more about the world.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6525,"error":null,"output":"C"}
{"question_id":"hellaswag:00006834","item_index":87,"attempt":0,"prompt_hash":"14bf73f7c77c3a0630a36b3bcac88e58971d1aeb4df0ae057de1d528214deb41","question":"[header] How to get rest raising a newborn baby [title] Talk to your healthcare provider about baby blues. [step] Many moms experience postpartum depression due to the sudden shift in hormones that occurs after giving birth. Difficulty sleeping can be a long-term symptom, and if you know the early warning signs and how to treat them ahead of time, you can avoid severe sleep deprivation all together.\n\nA. [substeps] Your doctor can help you find ways to block your nightly hours to the point you can't be in bed at night. Schedule an appointment with your healthcare provider and ask about adaptive sleep management approaches you can try.\nB. [substeps] Be sure to take time to read any \" baby blues \" literature your doctor gives you before baby's arrival, so that you know what to expect. [title] Make a sleep plan with your partner if you have one.\nC. [substeps] On average , 20 to 25 hours of sleep per night is a good bedtime for those with postpartum depression. Be aware that sleep disturbances during pregnancy are not necessarily an indication of baby blues.\nD. [title] Notify your healthcare provider if the mother is taking drugs that affect her sleep. [step] Drugs listed above are safe to take at around one to two weeks.","prompt":"[header] How to get rest raising a newborn baby [title] Talk to your healthcare provider about baby blues. [step] Many moms experience postpartum depression due to the sudden shift in hormones that occurs after giving birth. Difficulty sleeping can be a long-term symptom, and if you know the early warning signs and how to treat them ahead of time, you can avoid severe sleep deprivation all together.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6526,"error":null,"output":"B"}
{"question_id":"hellaswag:00004106","item_index":88,"attempt":0,"prompt_hash":"bd0c970294f3baa5a995be85b79ad173efba8a9d4d5be1b0bfd9ff978d594f9e","question":"[header] How to castrate a horse [title] Get your horse assessed by a veterinarian. [step] Your vet will need to assess your horse before the procedure and talk to you about the options for castration surgery. Some horses will be able to be castrated under sedation while at home in his stall, while others will need to be castrated under full anesthesia in a clinical setting.\n\nA. [substeps] If your horse has a medical condition that does not require treatment, he may not be given castration surgery. If the hoof was cut during the procedure, this may be more than relevant to your horse.\nB. [substeps] For many horses, castration surgery is the safest option for a horse. If your horse is unaware of the possibility of anesthesia surgery, surgery will be the safest option for them at home.\nC. Be aware that castration surgery can only be performed in certain conditions. [substeps] Like surgery, the process will require pressure on the horse's head and other internal organs to be numbed with gauze or a surgical bandage.\nD. [substeps] If the horse is healthy with two scrotal testicles, then the veterinarian may opt to to sedate the horse and operate using local anesthesia. This may be done in the field or at the clinic.","prompt":"[header] How to castrate a horse [title] Get your horse assessed by a veterinarian. [step] Your vet will need to assess your horse before the procedure and talk to you about the options for castration surgery. Some horses will be able to be castrated under sedation while at home in his stall, while others will need to be castrated under full anesthesia in a clinical setting.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6525,"error":null,"output":"D"}
{"question_id":"hellaswag:00001368","item_index":89,"attempt":0,"prompt_hash":"805d129863cda1ccf06c57f1c759bb3a7fe3e2261546d1591ac075ffc21dacf8","question":"A band is on a stage together. A man is holding a violin and inspecting it. several people\n\nA. watch him as they talk about the violin.\nB. are waiting behind him.\nC. are pushed down a passing gray line through the course.\nD. are sitting in the studio behind them recording.","prompt":"A band is on a stage together. A man is holding a violin and inspecting it. several people","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6524,"error":null,"output":"B"}
{"question_id":"hellaswag:00007293","item_index":90,"attempt":0,"prompt_hash":"43bd5c02ef7c23858b5410e954a546e8e517896194653845ed2fd774816d389c","question":"[header] How to care for bleached blonde hair [title] Shampoo less frequently. [step] Shampoos are designed to remove dirt and oil from the hair, but they can also strip your hair's natural cuticle. Bleach does the same thing, so shampooing every day just layers damage.\n\nA. It's best to start at the roots and work your way up from there. Adding more products as necessary reduces the amount of time you spend washing and moisturizing your hair and damaging it with shampoo, until your hair is no longer bleached blonde.\nB. Long haired people with bleached hair may not enjoy having to wash and condition their hair once every two or three days. When possible, designate one day a week to shampoo your hair.\nC. Think carefully about this before treating your hair with bleach, or you may end up causing more damage. [substeps] If you really want to keep your bleached hair as white as possible, shampoo more often.\nD. [substeps] Wash your hair every three to four days, especially in the first few weeks. Buy a gentle dry shampoo to spray on your roots to stretch the time you can go between washes.","prompt":"[header] How to care for bleached blonde hair [title] Shampoo less frequently. [step] Shampoos are designed to remove dirt and oil from the hair, but they can also strip your hair's natural cuticle. Bleach does the same thing, so shampooing every day just layers damage.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6525,"error":null,"output":"D"}
{"question_id":"hellaswag:00008533","item_index":91,"attempt":0,"prompt_hash":"58abfad8207883f9fa4b89173bce37c5e25b563692d5025c9bed1dac0f132a46","question":"[header] How to diagnose muscular dystrophy [title] Pay attention to frequent falls. [step] Because muscular dystrophy affects the muscles, it can cause you to fall frequently. Similarly, it can cause difficulty with getting back up, or even just getting up out of bed.\n\nA. [substeps] Check your body for left undesired things because they affect your ability to get your legs up everyday. Take the measures that you wish to take to avoid any falling.\nB. [substeps] The duchenne variety often begins in childhood. While most kids will take tumbles, note if your child is particularly clumsy, falling several times a day, for instance.\nC. If you suddenly fall on your back, you have a problem with your leg. [substeps] When your leg falls, or you trip, faint, have difficulties with staying upright, or experience disorientation, then this could be a sign of overactive leg muscles.\nD. [title] Be aware of any unexplained weight gain. [step] Any recent changes in weight are likely due to dystrophy and not just having a problem with getting back up.","prompt":"[header] How to diagnose muscular dystrophy [title] Pay attention to frequent falls. [step] Because muscular dystrophy affects the muscles, it can cause you to fall frequently. Similarly, it can cause difficulty with getting back up, or even just getting up out of bed.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7213,"error":null,"output":"D"}
{"question_id":"hellaswag:00001201","item_index":92,"attempt":0,"prompt_hash":"b5ba2b241ddc97c106115e1d60b48d5aa4febec389d1f4800758f85b48cce998","question":"A woman is holding a baby. The woman is licking an ice cream cone. the woman\n\nA. takes the ice cream cone from the baby.\nB. stops licking the ice cream cone and talks to the baby.\nC. feeds the ice cream to the baby.\nD. licked jind from her cone through a straw.","prompt":"A woman is holding a baby. The woman is licking an ice cream cone. the woman","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7213,"error":null,"output":"A"}
{"question_id":"hellaswag:00004906","item_index":93,"attempt":0,"prompt_hash":"3f207053e407526e3f9bf71a8e01f13fba31b382a6441d035515511e328ef0be","question":"[header] How to make churro apple pie bowls [title] Preheat the oven to 450 ° fahrenheit or 225 ° celsius. [title] Grease a muffin tray by flipping it on its side. [step] Instead of spraying non-stick greasing spray in the cases of the tray, grease it on the cups sides.\n\nA. [substeps] The churro bowls are made onto the cups of the muffin tray. To prevent it from sticking, it must be sprayed on that area.\nB. [title] Add the batter and butter to the muffin tray. [step] Spread the flour over the contents of the tray while cooking.\nC. Some of the liners should also be greased along the sides. [title] Add the filling ingredients except the butter into the empty muffin tray.\nD. Spread the muffin mixture out from the inner rim of each cup and sprinkle the rim lightly with flour. [substeps] You can use a non-stick greasing spray made from butter or non-stick baking flour to help break down larger, gooey chunks of dough.","prompt":"[header] How to make churro apple pie bowls [title] Preheat the oven to 450 ° fahrenheit or 225 ° celsius. [title] Grease a muffin tray by flipping it on its side. [step] Instead of spraying non-stick greasing spray in the cases of the tray, grease it on the cups sides.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7214,"error":null,"output":"C"}
{"question_id":"hellaswag:00006289","item_index":94,"attempt":0,"prompt_hash":"72807fb12da6d003de6d73fafbac38b880d3d6989d7ee7b6254955cd0b9ee83c","question":"[header] How to feed a guinea pig [title] Feed your guinea pig hay. [step] Guinea pigs love hay! They need it for digestive and dental health. Guinea pigs should have unlimited access to hay, which often means refilling a good-sized dish 3 to 5 times daily.\n\nA. The amount of hay you feed your guinea pig should always be somewhere between an over sized dish and an empty one. Remember not to feed your guinea pig hay that is not hemp hay.\nB. Timothy hay will provide good feed for them, but is not as naturally nutritious as hay from your local feed store. [title] Buy feeding bowls with straw at the bottom of them.\nC. [substeps] Combine timothy hay with grass hay (100 g or 1/2 to 1/4 cup) whenever possible. Hay is a great source of solid protein that your pet needs.\nD. [substeps] Timothy hay is the best hay for all guinea pigs in general. They will happily eat it and play with it, and it's healthy for pigs of all ages.","prompt":"[header] How to feed a guinea pig [title] Feed your guinea pig hay. [step] Guinea pigs love hay! They need it for digestive and dental health. Guinea pigs should have unlimited access to hay, which often means refilling a good-sized dish 3 to 5 times daily.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7214,"error":null,"output":"D"}
{"question_id":"hellaswag:00003291","item_index":95,"attempt":0,"prompt_hash":"947298312cb8eb0909141cc2c64b6320a972400bcddc1f76b83daf1623fd22d6","question":"[header] How to make lentil and raisin rice with meatballs [title] Puree the onions until smooth. [title] Add onion to meat and knead them with hand till mixed. [title] Add turmeric, salt and black powdered pepper to the mixture and knead again.\n\nA. [title] Turn the mixture on medium heat and cook for 5 minutes, then add the remaining ingredients and simmer till the onion has gone dark green. [title] Remove from heat and add the seasonings and sauce.\nB. [title] Set aside to rise in minced garlic and three garlic cloves until they become soft. [title] Sauté the two garlic cloves till they turn a bright green colour.\nC. [title] Put panko flour in bowl and mix with a fork on a low speed to the wet ingredients. [title] Take a 2 cup measuring cup with a lid and add this mixture into a clean jar.\nD. [step] By now the meat should be completely smooth. [title] Add broken yellow peas powdered (using a small mill) and the bread crumb (optionally) to the mixture and knead until fully mixed together and pasty.","prompt":"[header] How to make lentil and raisin rice with meatballs [title] Puree the onions until smooth. [title] Add onion to meat and knead them with hand till mixed. [title] Add turmeric, salt and black powdered pepper to the mixture and knead again.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7212,"error":null,"output":"A"}
{"question_id":"hellaswag:00001607","item_index":96,"attempt":0,"prompt_hash":"b660086e443022349caee6cf619865bc2d6c893907f140aa5b3a8cb88c118eec","question":"The argentinians steal a pass halfway across the sand field and take it down for a goal in the tight corner. This is immediately followed by another goal from argentina on a penalty kick. brazil\n\nA. takes another pass from argentina repeatedly and scores several points.\nB. wins the game which puts the argentinians in celebration.\nC. finds itself down even more on another argentinian goal on a beautiful pass and shot across the field.\nD. congratulates the argentinians and returns to the upcoming game on the monkey bars.","prompt":"The argentinians steal a pass halfway across the sand field and take it down for a goal in the tight corner. This is immediately followed by another goal from argentina on a penalty kick. brazil","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6645,"error":null,"output":"C"}
{"question_id":"hellaswag:00005129","item_index":97,"attempt":0,"prompt_hash":"79f80cea6418885e2fd4dd7752f29929e0355839b195187755e81077532d8f77","question":"[header] How to buy appliances from craigslist [title] Plan your purchases as far in advance as possible. [step] One of the downsides of buying appliances on craigslist is that you won't be able to specify the exact make and model you want. By planning your purchases ahead of time, you'll have plenty of time to browse craigslist listings each day.\n\nA. [title] Type in the number of appliances you want to buy, and then press \" purchase \" on your keyboard. [step] Browse a variety of internet search engines, like google and bing, to find what your buying price is.\nB. Set aside enough time to look through all of the products listed, which makes it easy to separate product by product if you decide to buy any of those. [substeps] Search for companies that specifically post on craigslist.\nC. [substeps] Many consumer sites also host online catalogs and promotions that are similar to what's advertised on craigslist. You may be able to schedule your purchases at an agreed-upon time, but not at a fixed date.\nD. This will allow you to wait until an appliance is listed that more closely meets your needs. [substeps] Craigslist even allows you to subscribe to an rss feed for your search results.","prompt":"[header] How to buy appliances from craigslist [title] Plan your purchases as far in advance as possible. [step] One of the downsides of buying appliances on craigslist is that you won't be able to specify the exact make and model you want. By planning your purchases ahead of time, you'll have plenty of time to browse craigslist listings each day.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6638,"error":null,"output":"D"}
{"question_id":"hellaswag:00003298","item_index":98,"attempt":0,"prompt_hash":"461aba1eb48d815d832798f9edc89807590b6a09818504396bb1f1b85fcbc42c","question":"[header] How to care for a senegal parrot [title] Choose a large cage. [step] Senegal parrots need roomy cages. The minimum size is 20 inches (50. 8 cm) long by 20 inches (50. 8 cm) deep by 28 inches (71.12 cm) high (for one parrot), but bigger is always better.\n\nA. Due to their clever dispositions, pick an escape-proof one (though some senegal parrot owners may realize \" escape-proof \" isn't necessarily true). [substeps] Bar spacing of ¾ inch (1.9 cm) is ideal.\nB. What you don't have to worry about is the size of the space your bird can comfortably move around in. If you have two birds, consider making sure the space is large enough for each parrot.\nC. Senegal parrots are huge birds who cannot stand heights and require bigger cages. Not only are they more compact than average birds, but they also do not tolerate gravity well.\nD. [title] Look at your parrot's enclosure. [step] Your senegal parrot needs an enclosure that is approximately 12 feet (1.2 m) by 14 feet (30.","prompt":"[header] How to care for a senegal parrot [title] Choose a large cage. [step] Senegal parrots need roomy cages. The minimum size is 20 inches (50. 8 cm) long by 20 inches (50. 8 cm) deep by 28 inches (71.12 cm) high (for one parrot), but bigger is always better.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6638,"error":null,"output":"B"}
{"question_id":"hellaswag:00001997","item_index":99,"attempt":0,"prompt_hash":"a14c3dbe3e539222fddbde0b9e8d5770b9b6712c0a55c0a40c9c70213be8d154","question":"An old lady knits with a red needle and red wool. An old lady stops knitting and talks continuously. the old lady\n\nA. knits again and examines her creation.\nB. is finished knitting and talks to the camera.\nC. writes on a piece of cloth and ties the ends.\nD. grabs the left wool and knits, then she knits on the hardwood floor.","prompt":"An old lady knits with a red needle and red wool. An old lady stops knitting and talks continuously. the old lady","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6642,"error":null,"output":"B"}
{"question_id":"hellaswag:00007595","item_index":100,"attempt":0,"prompt_hash":"a02c6e18e8c777d32341a879f2e3134bc836eaeaea51aea0a1ca06d64ab24bef","question":"[header] How to make memorable sales calls [title] Build rapport by listening. [step] People like to do business with people they like. The first portion of the discussion is all rapport building, or small talk.\n\nA. The more comfortable you are talking to the person, the better your natural flirting will appear. [substeps] Ask an interested opinion or ask questions about what someone says.\nB. Ask about the customer and share about your life, too-hometown, current neighborhood, college stories, favorite sporting teams, vacations, golf, books, current events, good news and goals. All of these topics lead to a deeper, more meaningful discussion.\nC. You can let the other person speak at their own pace as they talk, but remember to pay attention and listen attentively. This is one of the health benefits of working with someone you already know and trust.\nD. You want to keep things clear before you ask, and build on topics common to almost any business talk. [substeps] Pay attention to anyone's body language and tone.","prompt":"[header] How to make memorable sales calls [title] Build rapport by listening. [step] People like to do business with people they like. The first portion of the discussion is all rapport building, or small talk.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6637,"error":null,"output":"C"}
{"question_id":"hellaswag:00009114","item_index":101,"attempt":0,"prompt_hash":"a27beafe53ae19c9e64c53c687f7e7520707a574f38ad2812b900d704b61399f","question":"[header] How to grow vietnamese mint [title] Plan to grow vietnamese mint in the spring or summer. [step] Vietnamese mint can be grown in most climates so long as they have warm, sunny weather. Avoid planting your mint in the fall or winter, if possible, and wait until the weather gets warmer in your area.\n\nA. The year difference in temperature can cause leaves, flowers, and stalks to re-rot. [substeps] If you live in a temperate area, you will need to wait until spring to plant vietnamese mint.\nB. [substeps] Unfortunately, vietnamese mint doesn't thrive in cooler climates, especially in temperatures over 95 degrees fahrenheit (20 degrees celsius). [title] Plan to keep a pot full of vietnamese mint.\nC. If you live in the northern hemisphere, place your mint to grow in autumn or spring. [title] Do not plant your mint in sunny weather.\nD. [title] Propagate vietnamese mint from cuttings. [step] Vietnamese mint is best grown from clippings.","prompt":"[header] How to grow vietnamese mint [title] Plan to grow vietnamese mint in the spring or summer. [step] Vietnamese mint can be grown in most climates so long as they have warm, sunny weather. Avoid planting your mint in the fall or winter, if possible, and wait until the weather gets warmer in your area.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6641,"error":null,"output":"D"}
{"question_id":"hellaswag:00005494","item_index":102,"attempt":0,"prompt_hash":"a62d3c71f150768f66a36439b01be796ae32f4c6c905e52ac1b5c850c912bda2","question":"[header] How to get wavy hair overnight with a bun [title] Cut off the toe of an old sock. [step] Any cotton tube sock will do. If you have very long hair, you'll want a bigger sock, so choose a men's sock if you can locate one.\n\nA. Remember, curly hair tends to have more split ends. [title] Tie a headband onto the sock.\nB. [title] Try braiding it to make your layered look. [step] Braid it several times until you have about six strands of hair left, and tie the ends with a rubber band.\nC. Consider going with a round sock or rectangle sock for coarse hair. [substeps] Remember, the longer you style your hair, the shorter it will grow.\nD. Then simply snip off the toe using a pair of scissors. You now have a tube of cloth that's open at both ends.","prompt":"[header] How to get wavy hair overnight with a bun [title] Cut off the toe of an old sock. [step] Any cotton tube sock will do. If you have very long hair, you'll want a bigger sock, so choose a men's sock if you can locate one.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6637,"error":null,"output":"D"}
{"question_id":"hellaswag:00007171","item_index":103,"attempt":0,"prompt_hash":"70dde5878c050b0063514a52a44a6ee6d0473218cc69c8fb93ecb928f08b175f","question":"[header] How to remove a scratch from a stainless steel refrigerator door [title] Identify the grain. [step] Like wood, stainless steel products also have a \" grain. \" when you clean, polish, or sand stainless steel, you always need to work in the direction of the grain.\n\nA. This means that you will need to work from the outside (under the counter, on a stove, etc. ), while more frequently cleaning on the inside of the refrigerator door.\nB. Scratch-free stainless steel products usually have a grain. If you're removing leather, stainless steel cookware, and linware, make sure you work from the grain.\nC. The grain determines the overall finish of the interior of your refrigerator. [substeps] Consider wiping the inside inside and outside with a damp cloth to dull any potential scratches.\nD. To identify the direction of the grain : [substeps] Look closely at the stainless steel door. You will see tiny brush strokes-these brush strokes indicate the direction of the grain.","prompt":"[header] How to remove a scratch from a stainless steel refrigerator door [title] Identify the grain. [step] Like wood, stainless steel products also have a \" grain. \" when you clean, polish, or sand stainless steel, you always need to work in the direction of the grain.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6636,"error":null,"output":"D"}
{"question_id":"hellaswag:00001955","item_index":104,"attempt":0,"prompt_hash":"02cdbb948e2c109ee47c1a3fc884f64df8db88b2b01167bbea2f749d3d640f85","question":"A large group of cheerleaders are seen walking out on a stage while people watch on the sidelines. the group\n\nA. then runs down the aisle and throws cymbals together while others cheer them on.\nB. twirl around with one another as they cheer.\nC. then begins a routine with a gymnastic routine and leads into them jumping and performing flips and tricks.\nD. begins performing a cheer routine while jumping and flipping all along the stage.","prompt":"A large group of cheerleaders are seen walking out on a stage while people watch on the sidelines. the group","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6641,"error":null,"output":"D"}
{"question_id":"hellaswag:00006782","item_index":105,"attempt":0,"prompt_hash":"933b9b13ed6bc6fe6360dcb5fdf22ec816a3defc601b974993e25192f3ad8c42","question":"[header] How to send an international fax [title] Find a fax machine with international capability. [step] Not all fax machines or public fax kiosks can send documents abroad. When you see a fax machine with international sending options, look at the prices to be sure that you're comfortable paying the international rate, which can be a lot more than a local rate for faxes.\n\nA. Visit a specialized fax machine in some countries, such as hong kong, if you don't know how to select one. [substeps] You can also make your own, which offers fax services with international settings.\nB. [title] Figure out the country code for the destination country. [step] Each country has its own code composed of several digits.\nC. [substeps] Look to see whether there are any free fax machines around in your region. The average fax machines cost around $250, although it can range from $200 or over $200 international.\nD. [substeps] Public fax machines come in many different capabilities, which can be quite helpful for software packages. Where you purchase faxes is a matter of personal preference, but international-quality machines can be an excellent option for internet users.","prompt":"[header] How to send an international fax [title] Find a fax machine with international capability. [step] Not all fax machines or public fax kiosks can send documents abroad. When you see a fax machine with international sending options, look at the prices to be sure that you're comfortable paying the international rate, which can be a lot more than a local rate for faxes.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6640,"error":null,"output":"B"}
{"question_id":"hellaswag:00002569","item_index":106,"attempt":0,"prompt_hash":"5b16e3032b64ad09b2a58a05f1311fbe31ff19a9c2775b66c1092f12bc060385","question":"Man is walking in a field in front of a black and red inflatable. Two men are standing holding paintball huns in a field with obstacles. man\n\nA. is talking from seconds he is paintwamblening on a field holding the m&ms goal marketed by trew gym.\nB. is showing how to holds the gun in a paintball field.\nC. is wearing a medal and watted on a grassy field.\nD. is running in a parseur with hula hoops and see launch of a tire.","prompt":"Man is walking in a field in front of a black and red inflatable. Two men are standing holding paintball huns in a field with obstacles. man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6640,"error":null,"output":"B"}
{"question_id":"hellaswag:00000605","item_index":107,"attempt":0,"prompt_hash":"47c6b5657d6dbc1271b09ed59702af64d0aeaa416f559c10f4c797e2d6843b36","question":"A black screen is pictured and white words begin to bounce down on the screen. After, a group of young men are pictured playing indoor soccer. the black team's forwards\n\nA. move out of the net, resulting in their loss in a pit of mud.\nB. are shown in a buzzing tour.\nC. are lined up and the right side of the screen is flashing a green light and a red light comes on.\nD. takes the ball and begins dribbling it to make a goal and he succeeds and it is replayed in slow motion.","prompt":"A black screen is pictured and white words begin to bounce down on the screen. After, a group of young men are pictured playing indoor soccer. the black team's forwards","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6640,"error":null,"output":"D"}
{"question_id":"hellaswag:00007581","item_index":108,"attempt":0,"prompt_hash":"0c59cec4400bd0df5900105eaecf1787b09423bf35d7a6341f828aa668c235cc","question":"[header] How to hook up with a girl at a bar [title] Practice good hygiene and grooming. [step] Present your most attractive self by practicing good hygiene and grooming. Take a hot shower, style your hair, trim your beard or shave (if necessary), and put on cologne.\n\nA. Try not to throw on too much cologne, as the overwhelming smell may turn off potential dates at the bar. [title] Choose a bar that will have a higher likelihood of single women.\nB. If your feminine friend tends to wear perfumes and other negative products, make sure you go into the bathroom and spray a little one on yourself before entering a bar. Avoid eating before a meal or drinking during the time you are sitting down.\nC. Shave too, and wear a stylish outfit that makes you look fresh and confident. If you're a bit too thin, try eating lots of whole grains, fruits, and vegetables, and switching your diet completely to healthier and delicious vegetables.\nD. [substeps] Even if you have a bad habit (the girl would probably be attracted by this), be sure to keep it to a minimum and not inadvertently put on perfume. [title] Wait at least an hour before you strike up a relationship with the girl in question.","prompt":"[header] How to hook up with a girl at a bar [title] Practice good hygiene and grooming. [step] Present your most attractive self by practicing good hygiene and grooming. Take a hot shower, style your hair, trim your beard or shave (if necessary), and put on cologne.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6637,"error":null,"output":"A"}
{"question_id":"hellaswag:00004177","item_index":109,"attempt":0,"prompt_hash":"0bf8d5c2cd5361af9476cfcc73865f36deb8ccb82cc623a4147727a90da650be","question":"[header] How to buy a dog crate [title] Look for a cage based on safety and sufficient size. [step] The dog crate you choose should allow your dog to lie down and turn around comfortably in the cage. When they sit or stand in the crate, they should have about three to four inches of extra head room.\n\nA. They should have enough room to run away from you as they creep out the back door. [title] Decide whether you want a crate with a lid.\nB. Since different breeds of dogs have different proportions, weight alone cannot allow you to determine an appropriately-sized cage. [substeps] Measure your dog at its widest, tallest, and longest points, then add three inches to each of these dimensions to find the rough dimensions of the crate you'll need.\nC. The optimal height for your dog's enclosure is preferably less than a foot (4 \" x 20 \") high but it should not be higher than about an inch. [substeps] Do not use a platform or substrate to provide your dog with a comfortable seated height.\nD. With a double or triple barrel crate, an adult dog can sit and turn around comfortably in it. So ensure that the crate you choose meets the dog's basic needs.","prompt":"[header] How to buy a dog crate [title] Look for a cage based on safety and sufficient size. [step] The dog crate you choose should allow your dog to lie down and turn around comfortably in the cage. When they sit or stand in the crate, they should have about three to four inches of extra head room.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6636,"error":null,"output":"B"}
{"question_id":"hellaswag:00007229","item_index":110,"attempt":0,"prompt_hash":"2032f8f30fe8f532933a311a57977c0262f702dba51f42380cf4672181e4119f","question":"[header] How to dye men's hair [title] Choose a subtle color change. [step] Unfortunately, obviously dyed hair can be a source of embarrassment for men in many societies. To avoid this, stick with colors close to your natural hair color, at most one or two shades lighter or darker.\n\nA. If possible, try to pick off a flavor that you're going for in order to brighten your hair color. [substeps] Many women are stylists and their hair can turn out brighter and healthier if they're following the \" girl theme \" that happens during the holiday season.\nB. You don't have to dye every single strand of your hair a deep, rich color-that would totally ruin your style. [substeps] If possible, have your friend or acquaintance pick out any hair you like: this way, you're actually not having to choose two hues of hair.\nC. Aim for a dye one or two shades darker than the final result you're going for, to allow for fading. [substeps] If there are natural reddish tones in your hair, find a dye with warm tones.\nD. Many women who dye their hair do so not yet realize how their hair looks like; women who don't dye their hair should at least take the cake out of their hair when they dye. [substeps] In a wash, the dye will be applied carefully to the roots, in order to hold them back.","prompt":"[header] How to dye men's hair [title] Choose a subtle color change. [step] Unfortunately, obviously dyed hair can be a source of embarrassment for men in many societies. To avoid this, stick with colors close to your natural hair color, at most one or two shades lighter or darker.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6636,"error":null,"output":"C"}
{"question_id":"hellaswag:00001963","item_index":111,"attempt":0,"prompt_hash":"db64a1747a7df64c882c58da2e5be97c1d87cbe954d8cafb0f34f3c122e33ba5","question":"A person is seen riding a board along the water with a kite on top. more clips\n\nA. are shown of the person riding back and fourth on the board.\nB. are shown of people riding the board as well as flying off and being interviewed.\nC. are shown of people riding in the water and making a sail over the waves.\nD. are shown of them kite surfing as well as jumping off a mountain and down into the water.","prompt":"A person is seen riding a board along the water with a kite on top. more clips","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6639,"error":null,"output":"A"}
{"question_id":"hellaswag:00007998","item_index":112,"attempt":0,"prompt_hash":"e935198ccd5fa60e45842fb20ed7e5b5589281710648601502bb59baa369020e","question":"[header] How to make hot and sour soup [title] Toss the pork loin with the dark soy sauce. [step] Add 5 ounces (142 g) of boneless pork loin that's been cut in ¼-inch (6-mm) thick strips and 2 teaspoons (10 ml) of dark soy sauce to a bowl. Mix the two together well so the pork is fully coated and let the meat sit for 10 to 15 minutes.\n\nA. [substeps] Substitute blue and red lamb liver liver for the pork if you don't have any. Both meats contain many additives that can be dangerous to consume.\nB. [substeps] You can use clean chicken legs instead of the boneless pork tenderloin if you prefer. [title] Stir in 1/4 cup (60 ml) of milk.\nC. [substeps] The longer you let the pork sit, the darker it'll be. You should end up with a lighter red meat when you've finished coating it.\nD. [substeps] You can omit the pork from the soup if you want to make a vegetarian version. [title] Soak the mushrooms in boiling water.","prompt":"[header] How to make hot and sour soup [title] Toss the pork loin with the dark soy sauce. [step] Add 5 ounces (142 g) of boneless pork loin that's been cut in ¼-inch (6-mm) thick strips and 2 teaspoons (10 ml) of dark soy sauce to a bowl. Mix the two together well so the pork is fully coated and let the meat sit for 10 to 15 minutes.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6638,"error":null,"output":"D"}
{"question_id":"hellaswag:00005992","item_index":113,"attempt":0,"prompt_hash":"8f4bdc38af9f1fbbe84bdbf1079ff5f53475e2428457d61c3c6e896311c345fc","question":"[header] How to send a citizen complaint to a civil grand jury in california [title] Determine who your complaint will be against. [step] They are generally referred to as the \" respondent, \" and the person making the complaint is the \" complainant. \" [substeps] Keep in mind that county grand juries have no authority over state or federal agencies and private citizens, only county and local agencies or officials.\n\nA. [title] Download the dmv complaint form from the dmv website. [step] You can download the form and bookmark it.\nB. By contrast, the supreme court of the united states has no authority over the federal district of appeals to the supreme court (the district of appeals). States with a central hierarchy govern the district of appeals and other courts.\nC. As an exception, they also may not have direct control. Plaintiffs are not represented by an independent party, and may not have direct jurisdiction over the case.\nD. Take note of which county the agency of official is located in. [title] Determine which county grand jury would be appropriate to contact.","prompt":"[header] How to send a citizen complaint to a civil grand jury in california [title] Determine who your complaint will be against. [step] They are generally referred to as the \" respondent, \" and the person making the complaint is the \" complainant. \" [substeps] Keep in mind that county grand juries have no authority over state or federal agencies and private citizens, only county and local agencies or officials.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6639,"error":null,"output":"D"}
{"question_id":"hellaswag:00005818","item_index":114,"attempt":0,"prompt_hash":"d44898ab8b837adfd8095f4dcccafe3a6c057da485cfee2a312e927195d61db1","question":"[header] How to wear a beret [title] Tuck the brim under the beret. [step] Every beret has a brim that fits snug on your head and holds the beret in place. Tuck the brim up and under the excess fabric of the beret.\n\nA. If the beret too loose is uncomfortable and won't pull over your face, then simply leave it that way. See the instructions included with your beret.\nB. Then puff out the fabric of the beret so it hides the edge of the beret. Tilt the beret to 1 side, with the front of the beret pulled down to your eyebrow.\nC. Tug at the brim to make sure everything is centered. Adjust to fit the beret snugly and block a few inches of vertical space between the brim and the rest of the frame.\nD. Make sure you tie it in the center of the beret so that the beret is not dangling down. [title] Keep the beret well in place.","prompt":"[header] How to wear a beret [title] Tuck the brim under the beret. [step] Every beret has a brim that fits snug on your head and holds the beret in place. Tuck the brim up and under the excess fabric of the beret.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6639,"error":null,"output":"B"}
{"question_id":"hellaswag:00007996","item_index":115,"attempt":0,"prompt_hash":"a5f3e6c171ec2b711cb8a596a50297c8ddba7bf0ee1321d5da83c280d39969f2","question":"[header] How to pretend to be a girl [title] Get a wig or style your hair. [step] If you have long hair already, then you can just style it. If your hair is short, then you may want to get a wig to enhance your feminine look.\n\nA. If your hair is long, then you may want to get a wig to transform your hairstyle. Try ponytails, pigtails, or several similar hairstyles.\nB. Wigs make your hair appear like a little frizz at the top. [substeps] Do not get a wig that is very short on top, unless you're sure that you want it to be.\nC. When you are choosing a wig, make sure that you purchase one that is high quality if you want your look to be more convincing. A poor quality wig may not look as good.\nD. [title] Spend some time creating your own hair style. [step] For some girls, this may mean shopping at a hair salon; for others, that may mean posing at a hairstylist's home instead.","prompt":"[header] How to pretend to be a girl [title] Get a wig or style your hair. [step] If you have long hair already, then you can just style it. If your hair is short, then you may want to get a wig to enhance your feminine look.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6638,"error":null,"output":"C"}
{"question_id":"hellaswag:00006505","item_index":116,"attempt":0,"prompt_hash":"da8d414bfafe61617a95e09266337d9c011efa52ec357a647dcac1679cfbe4a2","question":"[header] How to decide what to wear [title] Build an outfit using one piece as inspiration. [step] If you have a new piece of clothing you've been dying to wear, consider anchoring your outfit around that one piece. If it's a nice shirt, find pants and shoes that complement that shirt.\n\nA. If your inspiration piece is a new dress, add a statement necklace to take the dress to the next level. [substeps] You can also use shoes or an accessory as your base piece.\nB. You can also add pins or a decorative necklace if the outside of the shirt is a bit beat up. The same goes for patterns, fringes, or intricate accents.\nC. If it's a nice pair of jeans you've been wanting to shop for but haven't yet worn, put them on a shopping cart instead. If it's a summer day, buy a bathing suit or sweater or skirt.\nD. If it's a nicer shirt, find a nice pair of flats. [title] Pull all your clothes together to make a base for a style.","prompt":"[header] How to decide what to wear [title] Build an outfit using one piece as inspiration. [step] If you have a new piece of clothing you've been dying to wear, consider anchoring your outfit around that one piece. If it's a nice shirt, find pants and shoes that complement that shirt.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6639,"error":null,"output":"A"}
{"question_id":"hellaswag:00008505","item_index":117,"attempt":0,"prompt_hash":"31b3c0a2da6e78118f7f17e84ad2baf4581972dfd2d4936a518cf045d65de71e","question":"[header] How to cut ceiling tiles [title] Purchase the ceiling tiles if necessary. [step] If you are conducting a replacement job on an existing ceiling, you will want to match the ceiling tile style to the existing tiles. The easiest way to do this is to order much more than you need during the initial installation and store them for later use.\n\nA. [title] Remove the ceiling tiles on the top 2-4 trim pieces to expose the paint underneath. [step] The paint will thin and give the tiles a polished look.\nB. If you are buying new tiles, start by removing them from their package and letting them rest inside overnight. The tiles need time to adjust to temperature and humidity conditions, as they can expand or contract a bit.\nC. [substeps] To finish the job: place the ceiling tiles one strand at a time in the same way you did the surrounding tiles. Grab each tile, then use a hammer or box to pound nails into the tile.\nD. [title] Cut the ceiling tiles to the correct size and hold in place with small snips or a drill. [step] This will allow the tiles to stay on the ceiling and makes it less likely that mites or small insects could escape.","prompt":"[header] How to cut ceiling tiles [title] Purchase the ceiling tiles if necessary. [step] If you are conducting a replacement job on an existing ceiling, you will want to match the ceiling tile style to the existing tiles. The easiest way to do this is to order much more than you need during the initial installation and store them for later use.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7110,"error":null,"output":"B"}
{"question_id":"hellaswag:00004206","item_index":118,"attempt":0,"prompt_hash":"e9a9d04bf693211850a84638e6908fab687c5d6879850f37bc49653eddc3f1ed","question":"[header] How to deal with children in a divorce situation [title] Tell your children as soon as possible. [step] Once it has become certain that you and your spouse will be separating, you need to let your kids know. Make sure you're calm and composed when you decide to tell your children, and have them sit down with you somewhere private.\n\nA. You should also turn off your cell phone and put it in a place that's out of the reach of children. The only time your kids know you'll be there is when their parents are in the room.\nB. If your children immediately has a lot of questions, tell them that things will be different, but that you all will address each issue as they come up. [title] Allow specific circumstances to dictate some decisions.\nC. You could give them a little guidance, such as, \" there has been a complaint about us filing for an annulment. We want to have that annulment finalized in a court of higher authority.\nD. Asking them quickly in writing can put you in contact with more than one person at once, and can make it difficult to rely on your children as a source of support. [substeps] Don't wait until the kids are younger or between the ages of eight and 12 to tell your children.","prompt":"[header] How to deal with children in a divorce situation [title] Tell your children as soon as possible. [step] Once it has become certain that you and your spouse will be separating, you need to let your kids know. Make sure you're calm and composed when you decide to tell your children, and have them sit down with you somewhere private.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7110,"error":null,"output":"B"}
{"question_id":"hellaswag:00009386","item_index":119,"attempt":0,"prompt_hash":"40cfe06c8a36f5ee4bfdafe95341cbe60147da5971889be106b9ebe248d90000","question":"[header] How to respond when your preschooler calls you names [title] Remember that your child's behavior isn't personal. [step] Your child's name-calling is about their own feelings of anger or powerlessness, not about you. Preschoolers act out because they don't have the coping skills or empathy to find better solutions to their problems yet.\n\nA. [substeps] Children with personality disorders cannot identify when their behaviors are inappropriate and cannot control their behavior. Thinking, \" they shouldn't be calling me names and saying hurtful things about me \" can be hurtful and rude.\nB. But are not afraid to ask to speak to a teacher or guidance counselor. [substeps] Call staff members at the school and see what they think of your child's behavior.\nC. Your child probably does not even know the meaning of the words they are calling you. They tend to repeat what they've heard.\nD. Just because you're not your preschooler's permanent parent or caregiver doesn't make you their parent. [title] Take some time to be understanding of your child's feelings.","prompt":"[header] How to respond when your preschooler calls you names [title] Remember that your child's behavior isn't personal. [step] Your child's name-calling is about their own feelings of anger or powerlessness, not about you. Preschoolers act out because they don't have the coping skills or empathy to find better solutions to their problems yet.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7110,"error":null,"output":"C"}
{"question_id":"hellaswag:00000250","item_index":120,"attempt":0,"prompt_hash":"5e8b57b029792d711924e5ac9d7ff63908fc8e023359117aae57d4ca4ed9d509","question":"An intro leads into a close up of a painting and a paintbrush swirling around and around. the hand holding the brush\n\nA. dips it into paint and paints while the woman continues painting.\nB. begins painting while standing next to a large piece of wood and pausing to wipe the brush on the canvas.\nC. begins painting grass into the painting slowly down the picture.\nD. is then seen using it to paint on the canvas causing it to move up and down and the logo continues flipping around but stops at the end.","prompt":"An intro leads into a close up of a painting and a paintbrush swirling around and around. the hand holding the brush","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7109,"error":null,"output":"A"}
{"question_id":"hellaswag:00006347","item_index":121,"attempt":0,"prompt_hash":"264063583a1eca90463fa9f690f4a5245cb88e6a870a4ef9b079094dc17b02b3","question":"[header] How to wear metallic lip color [title] Apply your regular foundation first. [step] To start, you should apply your regular foundation. Dab the foundation on using a makeup brush or your fingers.\n\nA. From there, mix the foundation in your face using your hands, a sponge, or a makeup brush. [substeps] If you don't normally wear foundation, you can skip this step.\nB. Working diagonally between your lips, apply powder foundation to foundation under your eyes and nose. Apply your regular foundation to your chin and cheeks.\nC. Then, dab it on the center of your cheek where your center lip should be. [substeps] Your foundation should also be evenly applied to your cheeks, at the corners of your mouth, on either cheek, and on your chin.\nD. Following the line you've drawn with the brush, wait for it to dry. However, if you're going for a classy, suave look, you can apply a bit of your favorite lipstick for added feel.","prompt":"[header] How to wear metallic lip color [title] Apply your regular foundation first. [step] To start, you should apply your regular foundation. Dab the foundation on using a makeup brush or your fingers.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7109,"error":null,"output":"A"}
{"question_id":"hellaswag:00007705","item_index":122,"attempt":0,"prompt_hash":"27f26f3f667b92bb4aebeee0ef86ee423d9bcb8b6727506edd00b5c70ebb745e","question":"[header] How to get better accuracy with a handheld gps [title] Consult the owner's manual on the type of antenna your gps has. [step] Since antenna positioning is very important for accuracy it is good to be sure about what type of antenna you have. There are commonly two types-patch and quadrifilar helix antennas.\n\nA. Patch antennas are the extra strong and large lenses that detect movement through the horizon. They also provide protection through compression to increase the display of the sky.\nB. The first one is built into the device and the other one is external. Remember to always point your antenna (regardless the type) towards the sky.\nC. Lower uglies support teleporter telescopes, low powered blow fuse rod equipment. Two of the most common antenna sizes are inportable and reliable against weather.\nD. B1952 and four different antennas are led with adapters, including video cables, ranging from 20 to 360 feet. Aaa antennas will normally be fitted with antenna slots for wifi and antenna positions.","prompt":"[header] How to get better accuracy with a handheld gps [title] Consult the owner's manual on the type of antenna your gps has. [step] Since antenna positioning is very important for accuracy it is good to be sure about what type of antenna you have. There are commonly two types-patch and quadrifilar helix antennas.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7108,"error":null,"output":"B"}
{"question_id":"hellaswag:00002060","item_index":123,"attempt":0,"prompt_hash":"3bc383dd366925c48ff67ff97736f73a9a69523ff640470aa7ff91f4647c9ead","question":"A person is seen cutting up various fruits while also seen speaking to the camera. The person mixes various ingredients into a container of water. the person\n\nA. zooms in on the vegetables and pans out to several views of the vegetables seated.\nB. then jostles the vegetables together in a bowl.\nC. rinses down the container with the water and then paints on a sandwich, as well as a sandwich of sandwich meat and cheese.\nD. spins around the container in the end.","prompt":"A person is seen cutting up various fruits while also seen speaking to the camera. The person mixes various ingredients into a container of water. the person","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6605,"error":null,"output":"D"}
{"question_id":"hellaswag:00007506","item_index":124,"attempt":0,"prompt_hash":"de801e2d43c9f6c10882e4d6da3751c360c5c84814bcbb6b52379fdd6b612acb","question":"[header] How to cook fenugreek [title] Grind the seeds to add the flavor without cooking the seeds. [step] Fenugreek seeds are quite hard, so it can be difficult to grind them by hand with a mortar and pestle. Instead, put the seeds in a small coffee grinder and let the machine do the work for you.\n\nA. Grind the fenugreek seeds until they're well-pureed. [substeps] Use a coffee grinder with a beater attachment or with the lid off to grind the fenugreek seeds to a fine powder on a flat surface.\nB. [substeps] You can also try to buy fenugreek seeds that are already ground up, directly from the store. [title] Dry roast the seeds on the stove for a more edible addition to your dish.\nC. Grind away until all of the seeds are ground, then add water to the grinder. [title] Add lemon juice to the grinder.\nD. [substeps] Fenugreek is often boiled by boiling the seeds for just about 10 minutes. It is finely ground into a fine powder and used as a spice for soups, salads, and other varieties of breads.","prompt":"[header] How to cook fenugreek [title] Grind the seeds to add the flavor without cooking the seeds. [step] Fenugreek seeds are quite hard, so it can be difficult to grind them by hand with a mortar and pestle. Instead, put the seeds in a small coffee grinder and let the machine do the work for you.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6603,"error":null,"output":"B"}
{"question_id":"hellaswag:00004856","item_index":125,"attempt":0,"prompt_hash":"8818a2d4c3a284ee126ec686016be90369f2c09233a908213073561de0c8f932","question":"[header] How to wear turtleneck sweaters [title] Choose a chunky or boxy turtleneck for a more casual look. [step] A thick, cozy turtleneck sweater and a pair of distressed jeans is the perfect way to stay warm on a cold weekend. [substeps] Straight leg or tapered jeans will elongate your silhouette more than baggy jeans will when you are wearing a bulky turtleneck.\n\nA. Avoid pairing cardigan-lined sweaters with a solid-colored turtleneck. This helps obscure or lengthen the appearance of a thinner turtleneck sweater.\nB. Cropped jeans and a pair of sneakers will make your turtleneck seem more athletic. [title] Pair a slim-fitting turtleneck with slacks for a business-ready look.\nC. [title] Wear a dress or skirt for a more formal look. [step] A dark blazer and a v-neck blouse will enhance your look while taking the focus away from your t-shirt and trench coat and the weather.\nD. A pullover or button up will also elongate your torso. Low-rise or slip-up tops can create a tapered-waist look.","prompt":"[header] How to wear turtleneck sweaters [title] Choose a chunky or boxy turtleneck for a more casual look. [step] A thick, cozy turtleneck sweater and a pair of distressed jeans is the perfect way to stay warm on a cold weekend. [substeps] Straight leg or tapered jeans will elongate your silhouette more than baggy jeans will when you are wearing a bulky turtleneck.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6604,"error":null,"output":"B"}
{"question_id":"hellaswag:00009591","item_index":126,"attempt":0,"prompt_hash":"d57615d184cc3ccea04a12f0e58779b304af22fb1695262ab2b769a4a9842f72","question":"[header] How to buy toms shoes online [title] Determine your toms shoe size. [step] Toms shoe sizes normally align with the shoe size you wear most often and are only available in a medium width. [substeps] Choose a toms shoe size identical to your size in casual or dress shoes.\n\nA. Toms shoe styles and sizes are very similar to styles of casual and dress shoes. If you wear 2 different shoe sizes that vary depending on the shoe style, choose the smaller size when buying toms shoes.\nB. If your shoes are too small to fit the shoe size you want, you may want to limit how much space you have to exercise. [title] Select a shoe for your target gait.\nC. Your feet will be bigger within a two inches of the shoe, which allows for the sole to lay between your toes. [title] Make sure your shoes do not have wide toes.\nD. This is because most soled shoes are a bit bigger than their regular size. Toms shoes are more likely to be wider, but still have more safety pin sizes.","prompt":"[header] How to buy toms shoes online [title] Determine your toms shoe size. [step] Toms shoe sizes normally align with the shoe size you wear most often and are only available in a medium width. [substeps] Choose a toms shoe size identical to your size in casual or dress shoes.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6604,"error":null,"output":"A"}
{"question_id":"hellaswag:00006415","item_index":127,"attempt":0,"prompt_hash":"1f22279a6b316d78bab6b0dce42b927b00f58e7866de89f9245a861f2d07e456","question":"[header] How to decide where to retire [title] Think about where you see yourself. [step] Have you ever pictured yourself retiring to the white sand beaches of mexico? You may have also thought of a nice college town or even just staying where you are to enjoy grandchildren. Consider a variety of options for where you want to retire from dream locations to maintaining your current situation.\n\nA. [substeps] When you're first starting your plan, you may have considered local, state, or national parks. How appealing these are and how productive you wanted them to be will largely depend on your intended goals.\nB. [substeps] Compose a list of places you're considering retiring. Make a tiered list that includes dream locations, easily attainable or alternative \" dream \" places, and staying in your current or a similar area.\nC. [substeps] If you enjoy archibitions such as graduate heaven, you may think of retirement as a useful remedy for employment. You might also think of life outside of politics or family obligations.\nD. [title] When it comes to activities, as well as how you plan your retirement goals, put them in writing and see what happens. [title] Whatever you choose to do in order to ensure your happiness and yourself and you are doing it for the right reasons.","prompt":"[header] How to decide where to retire [title] Think about where you see yourself. [step] Have you ever pictured yourself retiring to the white sand beaches of mexico? You may have also thought of a nice college town or even just staying where you are to enjoy grandchildren. Consider a variety of options for where you want to retire from dream locations to maintaining your current situation.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6603,"error":null,"output":"D"}
{"question_id":"hellaswag:00000584","item_index":128,"attempt":0,"prompt_hash":"c9b0b2a97a1680ae78c0f4ae1f4af036544ba9d7f3009206c516951dfc69ba9c","question":"The individual puts some of the ingredients in small bowls into the mug. The individual rearranges the items on the counter top. the individual\n\nA. mixes some water into the mug.\nB. puts sprinkles onto the cake.\nC. take the mug mixture out of the meal.\nD. fills the mug with ice and pours liquid from a tube into the mug.","prompt":"The individual puts some of the ingredients in small bowls into the mug. The individual rearranges the items on the counter top. the individual","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6595,"error":null,"output":"D"}
{"question_id":"hellaswag:00005582","item_index":129,"attempt":0,"prompt_hash":"15dd23cc1da1daf8aca05d82107bf5a89354fed0a24a32c0938fc9df271a836a","question":"[header] How to clean makeup brushes with vinegar [title] Create the solution. [step] Place a cup of warm or hot water in a mug or jar. Add about a tablespoon of apple cider vinegar.\n\nA. The hot water will work to disinfect the bristles and remove dirt. [title] Get out your brushes with the bristles down.\nB. [substeps] Be sure to use the correct proportions-2 equal parts vinegar and water mixed together to form a foaming solution. [title] Use a pastry brush to squeeze out excess liquid.\nC. Stir until it is dissolved, then pour the solution over the bristles to mix in. [substeps] Do not wash your brush after you try to clean it, as it can further cause damage to the oils in your brush.\nD. Pour in 1/2 a teaspoon of dishwashing soap. Stir it in to make sure the solution is mixed well.","prompt":"[header] How to clean makeup brushes with vinegar [title] Create the solution. [step] Place a cup of warm or hot water in a mug or jar. Add about a tablespoon of apple cider vinegar.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6596,"error":null,"output":"D"}
{"question_id":"hellaswag:00006676","item_index":130,"attempt":0,"prompt_hash":"c89b16a753ce920accdc312cd75f02a47da640e502ff715cd84d6601fbceb07e","question":"[header] How to organize bridal shower games [title] The first kiss fun game : [step] This is a fun game and can be a great icebreaker for the guests who do not know each other. You can start this game by asking the bride-to-be to tell the story of the first time she was kissed by her finance'. The other guests can contribute as well.\n\nA. After they've told the story, have the bride-to-be give her a kiss and greet her in a friendly way. [title] Press the invitation into place and the bride is given the option of exchanging signals.\nB. To make it fun you can have a panel of guests who will rate the best kiss story. The one who gets the highest points turns out to be the winner.\nC. Your friend's favorite foods will be the things her parents cook. Next, ask the bride-to-be to tell the story about the first time she was kissed by her finance'.\nD. Obviously, this is the safest choice for women who are are young when they aren't in a college class. Be sure to tell the story in a straightforward way and not look to the wedding cake.","prompt":"[header] How to organize bridal shower games [title] The first kiss fun game : [step] This is a fun game and can be a great icebreaker for the guests who do not know each other. You can start this game by asking the bride-to-be to tell the story of the first time she was kissed by her finance'. The other guests can contribute as well.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6582,"error":null,"output":"B"}
{"question_id":"hellaswag:00001462","item_index":131,"attempt":0,"prompt_hash":"f4ec949c2529f13e497a213c571f20b8c18785b21482489f62a26dc38519965e","question":"Several scuba divers are seen swimming around the ocean while sharks move around them underneath. the camera\n\nA. follows the scuba divers as they look on at their work.\nB. pans out and reveals many boats and people floating around.\nC. continuously follows the sharks around the water and the scuba divers interact with one another while they move.\nD. pans around the divers jumping as they swim around the area.","prompt":"Several scuba divers are seen swimming around the ocean while sharks move around them underneath. the camera","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6594,"error":null,"output":"C"}
{"question_id":"hellaswag:00007081","item_index":132,"attempt":0,"prompt_hash":"49f6f86901337564a149deec3db7a65f99e4a324159f77b91d292177f4dc2fae","question":"[header] How to personalize your wedding [title] Choose something special to you and your partner. [step] Remember that, above all, this is your day. Speak with your partner and try to settle on a theme that seems fun and interesting to both of you-a common interest or something you are both passionate about.\n\nA. [substeps] Include something that you both would desire. For example, if you want to wear a tablecloth but your partner would make a scene, think on bridesmaids and groomsmen pieces like the candles.\nB. [title] Give your partner flowers and/or flowers that are the right one for you both. [step] You can get a gift from your wedding date! A garden/hike, etc.\nC. Don't worry if the getaway feel is not perfect. If your budget is a bit more flexible, do the french toast scene with your papaya, spring the bouquet, have a fresh look of your wedding vows, or have a coffee in bed.\nD. You can structure your wedding around virtually anything. [title] Draw inspiration from your favorite books and movies.","prompt":"[header] How to personalize your wedding [title] Choose something special to you and your partner. [step] Remember that, above all, this is your day. Speak with your partner and try to settle on a theme that seems fun and interesting to both of you-a common interest or something you are both passionate about.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6583,"error":null,"output":"D"}
{"question_id":"hellaswag:00007454","item_index":133,"attempt":0,"prompt_hash":"ad568564acca3c6922e13ab29c4fd6b16cd8ffc57477cf42a1b4d137b7e764c2","question":"[header] How to reverse tooth decay [title] Brush your teeth twice per day. [step] Regular daily brushing is crucial for maintaining good dental health and it may also help to reverse the decay process. Make sure that you brush your teeth at least twice per day, such as in the morning and before you go to bed.\n\nA. [substeps] You should avoid using a toothpaste that contains fluoride. Toothpaste containing fluoride will increase the risk of tooth decay.\nB. [substeps] Choose a toothpaste that contains fluoride. Fluoride is necessary for remineralizing teeth and reversing tooth decay.\nC. This should help to prevent them from becoming overgrown. [title] Use a soft bristled toothbrush.\nD. Stop at the halfway point of your brushing routine and don't forget to brush for a few seconds and then rinse your mouth. [substeps] Brushing is especially important during the daytime when you're traveling or if your periodontal infections or fever are prevalent.","prompt":"[header] How to reverse tooth decay [title] Brush your teeth twice per day. [step] Regular daily brushing is crucial for maintaining good dental health and it may also help to reverse the decay process. Make sure that you brush your teeth at least twice per day, such as in the morning and before you go to bed.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6582,"error":null,"output":"B"}
{"question_id":"hellaswag:00001192","item_index":134,"attempt":0,"prompt_hash":"5127f35183f6524c26489cbcafba2fc430998541add6b3390b80cbd7e0108b5c","question":"The man uses an electric razor to shave his beard. The man stops shaving to talk to the camera. The man resumes shaving with the electric razor. the man\n\nA. picks up the razor again.\nB. stops shaving once more to talk to the camera.\nC. uses an electric razor to slide a shaving chain through the hairline.\nD. uses the electric razor and he talks again.","prompt":"The man uses an electric razor to shave his beard. The man stops shaving to talk to the camera. The man resumes shaving with the electric razor. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6593,"error":null,"output":"B"}
{"question_id":"hellaswag:00003572","item_index":135,"attempt":0,"prompt_hash":"fc6689f62f715b5133719405446deeff4b7fed543f6223bccba73ae5b48075b2","question":"[header] How to do a french pedicure [title] Remove any existing nail polish. [step] Soak a cotton ball with nail polish remover and gently rub the cotton ball on the surface of the nail. Keep rubbing until you see that the polish is completely removed.\n\nA. You may need to use a bit of pressure to remove the polish entirely. [title] Soak your feet in hot water.\nB. Alternatively, you can wash your hands with soap and water immediately after. You'll want to avoid accidentally rubbing the polish off.\nC. Then rinse the area with warm water. [title] Use nail polish remover to remove any sticker or adhesive that was on the base of your french pedicure.\nD. [title] Apply cuticle cream onto the polish. [step] Grip the cotton ball in one hand while gently rubbing the liquid part of the cotton ball into the nail and remaining polish to create a smooth shine.","prompt":"[header] How to do a french pedicure [title] Remove any existing nail polish. [step] Soak a cotton ball with nail polish remover and gently rub the cotton ball on the surface of the nail. Keep rubbing until you see that the polish is completely removed.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6580,"error":null,"output":"A"}
{"question_id":"hellaswag:00004275","item_index":136,"attempt":0,"prompt_hash":"9e6d495252dffe1fa1fba69893df4493e862316f9a324fa405b2878d5ff417c6","question":"[header] How to get rid of static in hair [title] Use dryer sheets to quickly get rid of static. [step] If you have dryer sheets around the house, use one to wipe your hair down when static hairs are sticking up. You can also use the dryer sheets to wipe down your brushes and combs, helping prevent static.\n\nA. [title] Wash your hair when the static is out of your hair. [step] If you have very fine or brittle hair, running your hair under cool, running water is a great way to pick up static.\nB. [substeps] Line the drawer you keep your brushes in with dryer sheets, or stick a dryer sheet under your pillow at night while you're sleeping to help prevent static. There are anti-static sheets specifically designed for getting rid of static in your hair, too.\nC. [substeps] Place dryer sheets on a clothesline on a cleaning day or on your dog's bed to help you prevent static on your brush, comb, and comb. [title] Remove static on frizz-prone hair with hairspray.\nD. [substeps] Try using a brush dipped in cold water to scrub down the brush and comb. The further away you get from these items, the worse your static problem will be.","prompt":"[header] How to get rid of static in hair [title] Use dryer sheets to quickly get rid of static. [step] If you have dryer sheets around the house, use one to wipe your hair down when static hairs are sticking up. You can also use the dryer sheets to wipe down your brushes and combs, helping prevent static.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6583,"error":null,"output":"B"}
{"question_id":"hellaswag:00007347","item_index":137,"attempt":0,"prompt_hash":"a873a5edb5fa040314f292f6ea8d9ebdb27699f2f127ccc99c546925a24a4a8e","question":"[header] How to personalize slippers [title] Decide on the color and position. [step] If you want to add your name to a slipper, this can be a nice personalized touch. However, before adding your name, stop and consider where you want to add your name and the color thread you will use.\n\nA. Before you add your name to slippers, you need to make sure that the color and position you place your name are consistent. [substeps] If you go for clear or patterned designs, you may want to remove a layer of overspray or fabric before adding the name.\nB. [substeps] Choose a color that you like and goes well with the current color of your slippers. If you have yellow slippers, for example, they may look great with blue lettering.\nC. You may also want to have a wallet or other form of identification to provide a nice touch. [substeps] Consider writing on your old socks or painting your feet or heels with a marker of your choice.\nD. [title] Fold the paper so the front of the paper is facing you. [step] Place your bottom hand on the top and pull the paper down so it's right at your midline.","prompt":"[header] How to personalize slippers [title] Decide on the color and position. [step] If you want to add your name to a slipper, this can be a nice personalized touch. However, before adding your name, stop and consider where you want to add your name and the color thread you will use.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6580,"error":null,"output":"B"}
{"question_id":"hellaswag:00009116","item_index":138,"attempt":0,"prompt_hash":"72788a111e5167b7169a35f555a7a9c295f087cd2815d1604a46775a49c0d5fc","question":"[header] How to keep in contact with your grandkids [title] Contact your grandchildren through video calls. [step] Technology is very helpful when it comes to staying in contact with your grandchildren. Grandparents can use facetime, skype, and other video chat programs to not only talk to their grandkids, but see them, as well.\n\nA. Many of these services are free and are easy to use. [substeps] Facetime is exclusive to apple products, like iphones and macs.\nB. Call your grandchildren with the intention of requesting skype over video calls and sending them pictures. Giving your grandkids the opportunity to play video chat with other grandkids while you're busy is the best way to keep in touch with them.\nC. You can just send them an email or post them on your social media platforms. If your grandkids are specifically the best at communicating with their grandparents, you may want to consider starting video calls for grandchildren who can use facetime or other video chat programs.\nD. You may find that your grandkids respond the same way or that video calls bother them-you might be able to trace those phone calls. [substeps] The best way to stay in touch with your grandkids is to share videos from pinterest, at least.","prompt":"[header] How to keep in contact with your grandkids [title] Contact your grandchildren through video calls. [step] Technology is very helpful when it comes to staying in contact with your grandchildren. Grandparents can use facetime, skype, and other video chat programs to not only talk to their grandkids, but see them, as well.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6580,"error":null,"output":"A"}
{"question_id":"hellaswag:00006945","item_index":139,"attempt":0,"prompt_hash":"214f8887ae0ab0f28ce3560507dfb0539885518831a254bbbb557d40fdbdf06f","question":"[header] How to protect your ideas as a freelancer [title] Research the entities you plan on disclosing your ideas to. [step] It is possible to get a lot of information by asking your industry contacts about and searching online accounts of the business practices used by any party or organization you hope to share your idea with. The best way to protect your ideas is to be choosy about with whom you share them.\n\nA. Be a good listener and thank everyone for their time and for their opinion of your invention. Put your problem somewhere that no one is more likely to locate it by searching for it.\nB. [title] Advertise your product or service in various publications. [step] When it comes to promotion, there are plenty of websites devoted to communicating with freelancers.\nC. [substeps] Don't create a relationship with a business by letting it grow apart. Being personally connected to a business is a great way to remain in touch with your industry partners.\nD. [title] Avoid giving away any details when pitching your ideas to third parties. [step] Otherwise, you may pitch your idea, only to find that it was developed by someone else, who was then given total credit for it.","prompt":"[header] How to protect your ideas as a freelancer [title] Research the entities you plan on disclosing your ideas to. [step] It is possible to get a lot of information by asking your industry contacts about and searching online accounts of the business practices used by any party or organization you hope to share your idea with. The best way to protect your ideas is to be choosy about with whom you share them.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6581,"error":null,"output":"B"}
{"question_id":"hellaswag:00008407","item_index":140,"attempt":0,"prompt_hash":"081dce42cd9d1295a95609cf0dbed877c956690d4679d3ab937c36d7b53dd5c4","question":"[header] How to wear harem pants [title] Select the fabric style carefully. [step] Some fabrics will drape more elegantly than others. Try opting for silks, cotton jersey, and other delicate fabrics that let the pants drape in all the right spots.\n\nA. Choose a rich, airy fabric that will also blend in with the other fabrics in the range. [title] Select a pleated skirt for a laid back style.\nB. Silks are versatile fabrics that change fairly easily or are easily pulled. But ones that drape more generously on the legs and feet, especially the waist, bring out a feminine look.\nC. If it seems like the fabric may easily wrinkle, search for a different pair. These pants already have a casual vibe because of their fit, so it is important to find fabric that will stay wrinkle-free.\nD. [substeps] Harem pants can be changed around, but you don't want to be wearing harem pants in the middle of winter. [title] Opt for a pair of thicker harem pants.","prompt":"[header] How to wear harem pants [title] Select the fabric style carefully. [step] Some fabrics will drape more elegantly than others. Try opting for silks, cotton jersey, and other delicate fabrics that let the pants drape in all the right spots.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6580,"error":null,"output":"C"}
{"question_id":"hellaswag:00002779","item_index":141,"attempt":0,"prompt_hash":"d33626fb569c9e73776c0c05b55be7be7ae3705061f6847cab0c3b702faf6b02","question":"We see the man take a handful of grass and wipe the scythe head with it. The man pulls something from his pocket and rubs it on the scythe. the men\n\nA. shake hands and walk to the car.\nB. laugh and hug each other and we see the end screen.\nC. then return to cutting the grass.\nD. close the blade on the scythe.","prompt":"We see the man take a handful of grass and wipe the scythe head with it. The man pulls something from his pocket and rubs it on the scythe. the men","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6588,"error":null,"output":"C"}
{"question_id":"hellaswag:00003822","item_index":142,"attempt":0,"prompt_hash":"cec828e486d94df447965e066058ee8df5ef93204a8cce2ca77bf1f8a7d4b585","question":"[header] How to use a lash curler [title] Get a high quality eyelash curler. [step] There are many brands, but some are better quality than others. [substeps] Use a metal curler rather than a plastic or rubber one.\n\nA. Plastic gives the illusion of weight and adheres to your skin less effectively. Alternatively, buy a wand from a beauty store.\nB. A high quality curler will be made of sturdy metal with rubber pads on the clamps. Plastic or rubber curlers don't get a good enough grip to curl eyelashes well.\nC. Plastic is better because it provides less friction. The most common point of contact is the outer or middle part of the eyelid.\nD. Plastic cubbyheads have an acidic look and liquid detergents can actually leave behind dye that discolors and could leave permanent marks on your skin. [title] Dab the curler with wet saline.","prompt":"[header] How to use a lash curler [title] Get a high quality eyelash curler. [step] There are many brands, but some are better quality than others. [substeps] Use a metal curler rather than a plastic or rubber one.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6585,"error":null,"output":"B"}
{"question_id":"hellaswag:00007272","item_index":143,"attempt":0,"prompt_hash":"5f20b70c12d00dd57405f926e945f5d3e1d4168d235ab099f3e915dea28d0b94","question":"[header] How to decorate your nails [title] Pick a good spot to decorate your nails. [step] You'll want your nails to match, so they should all be the same length. Start by cutting them down, and then use a file to smooth out the rough edges.\n\nA. Then, use nail trims around the entire nail, making sure you get all of the way around. [substeps] On the wrong nail, or get nail trims that are similar in appearance as your real nails, you'll end up with redness and blistering.\nB. [substeps] See below for tips on picking the right size nail to fit the personality of your doll. [title] Choose a decoration that is appropriate for your doll.\nC. You can also use your file to shape your nails into either a rounded or squared edge, depending on your preference. [substeps] Be sure to file in one direction, from the outside toward the center of the nail, rather than in a back-and-forth motion.\nD. [title] Decide if you want to apply cuticle oil or polish, and pick whichever works for you. [step] For example, if you have very coarse cuticles, ask to have your cuticles dry between treatments.","prompt":"[header] How to decorate your nails [title] Pick a good spot to decorate your nails. [step] You'll want your nails to match, so they should all be the same length. Start by cutting them down, and then use a file to smooth out the rough edges.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6588,"error":null,"output":"C"}
{"question_id":"hellaswag:00001328","item_index":144,"attempt":0,"prompt_hash":"4c5389ec37e398e1cb835744f9274deaf352462edbe4bf09cdefe058d0200d72","question":"Man is staying in a white room playing the violin. Man is wearing big beats headphones and playing the violin. the man\n\nA. throws a book on the room and continues playing.\nB. stops playing and starts again.\nC. stops playing the violin and makes a gesture the fingers of the hand.\nD. shoots the violin then play it.","prompt":"Man is staying in a white room playing the violin. Man is wearing big beats headphones and playing the violin. the man","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6589,"error":null,"output":"C"}
{"question_id":"hellaswag:00002843","item_index":145,"attempt":0,"prompt_hash":"08010ee4c424f21432d0edc9a6fd8e7584f72e70e85ccdf07d0b2e547447e9ca","question":"We see a girl in gold doing a baton routine. The girl does a flip. the girls\n\nA. turns and does the side kick.\nB. spoke and the girl in gold walked off stage.\nC. jump and flip and almost falls.\nD. spins repeatedly and catches her baton.","prompt":"We see a girl in gold doing a baton routine. The girl does a flip. the girls","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6588,"error":null,"output":"D"}
{"question_id":"hellaswag:00001495","item_index":146,"attempt":0,"prompt_hash":"055dfdf5c856aed19cd26d43c5676b22c337c4fe8175e69caa409eb6cbfc66c0","question":"A man claps after performing a high jump on the field. we\n\nA. are shown the instant replay in slow motion a couple of times.\nB. see headlines in the screen.\nC. see the man in purple high five the field.\nD. see a sports advertisement about high jump.","prompt":"A man claps after performing a high jump on the field. we","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6588,"error":null,"output":"A"}
{"question_id":"hellaswag:00005941","item_index":147,"attempt":0,"prompt_hash":"5426310d5bf2cdd8cd90622a9f8fdf16771080167cd3235fcc0b331fce7121db","question":"[header] How to be certain that your choice to not marry is sound [title] Read articles online about marriage experiences, from both sides. [step] Try not to read about celebrity couples, as those marriages are hardly realistic. Find blogs, vlogs, or advice articles.\n\nA. [title] Research events before making your choice to not marry. [step] By researching everything you can think of and interviewing sources that can help you form your own beliefs, ideas, and opinions, you can determine if your decision is realistic.\nB. If something seems too off, you'll have plenty of fodder to defend against it. [title] Recognize that there is more to your decision than just a general opinion (\" that's one thing \").\nC. Sign up for these sources on an iphone and watch videos of celebrity couples that you can indulge in, or read about different spouses from various magazines. It's good practice to stay informed, especially if you're not watching tv, while reading.\nD. Find things written by people who don't want to get married, and why they chose not to get hitched. [title] Talk to both married and unmarried friends and relatives, and ask them why or why not they chose to marry.","prompt":"[header] How to be certain that your choice to not marry is sound [title] Read articles online about marriage experiences, from both sides. [step] Try not to read about celebrity couples, as those marriages are hardly realistic. Find blogs, vlogs, or advice articles.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6590,"error":null,"output":"D"}
{"question_id":"hellaswag:00001078","item_index":148,"attempt":0,"prompt_hash":"f0412d8faff37ef4a5517a6cc6ee3f35931ca97dad40b11b0227a8c50874077a","question":"A group is standing in a dark room. They are playing harmonicas in unison. they\n\nA. move together slowly, one after the other.\nB. are then shown playing an instrument with other instruments.\nC. run and kneel down as they groove.\nD. continue playing to a tune.","prompt":"A group is standing in a dark room. They are playing harmonicas in unison. they","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6589,"error":null,"output":"B"}
{"question_id":"hellaswag:00009808","item_index":149,"attempt":0,"prompt_hash":"28f48edba5e956552b45716d52e29396126fd3e5e8f5592f9c0d25ea80c8459a","question":"[header] How to cope when your teenager falls in love [title] Give them space to experience their first love. [step] Your child will always be your child, even as they grow up, so you will always worry about them. But be supportive, give guidance and make an effort to be informed and involved in their life, rather than restrict or monitor everything they do.\n\nA. [title] Understand that your toddler's first love comes now. [step] Once your child informs you of its first love, the relationship will go very slow.\nB. This will allow them to make the first mistake and feel ready to tackle another. [substeps] Encouraging them to experience their first love will help your teen become comfortable exposing themselves to their first love.\nC. [substeps] Giving them space to experience their first experience of love means letting them be responsible for their emotions and choices. Some of first loves may end quickly, some may persist, but each relationship will teach your teen valuable lessons about how to be with someone else.\nD. You may be surprised to find that your teen may have developed a new love of literature or an interest in flower carving. This may be normal, but encourage your teenager to allow for this.","prompt":"[header] How to cope when your teenager falls in love [title] Give them space to experience their first love. [step] Your child will always be your child, even as they grow up, so you will always worry about them. But be supportive, give guidance and make an effort to be informed and involved in their life, rather than restrict or monitor everything they do.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5523,"error":null,"output":"C"}
{"question_id":"hellaswag:00003128","item_index":150,"attempt":0,"prompt_hash":"9abfcbf5e47fcbb0292e7d1d1a2cfc02a1481bb5d0073158d5c4fff4ad70730e","question":"A worker gives money to man who signs a paper. A man driving a yellow car is interviewed, then a reporter talks in a parking lot. men\n\nA. make hockey goals on first white room while making activities with a goal on a blue table.\nB. clean a yellow car in a parking lot.\nC. arrive at the bungee jump holding kites.\nD. put drops of water on paper, then paste a river on paper.","prompt":"A worker gives money to man who signs a paper. A man driving a yellow car is interviewed, then a reporter talks in a parking lot. men","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5524,"error":null,"output":"B"}
{"question_id":"hellaswag:00009422","item_index":151,"attempt":0,"prompt_hash":"b62e06a7b671ca950d93c5bfa3aa50f8710b38f2280c13b1859a8ae663f5c37b","question":"[header] How to apply for a marriage license in wisconsin [title] Know who can marry in wisconsin. [step] In order to obtain a marriage license in wisconsin, you must meet certain requirements regarding who can get married. The following restrictions and allowances apply in wisconsin : [substeps] Age.\n\nA. Any person 18 or older can marry as long as they are otherwise competent. If you are 16 or 17, you must obtain written consent from parents, guardians, or custodians.\nB. Wisconsin requires all people between 21 and 45 to obtain a marriage license. If you are unable to obtain a marriage license in wisconsin, the person who can get married must be a resident of wisconsin.\nC. If you are applying for a marriage license in wisconsin, you must be an adult, free to marry, and unmarried for 21 years. If you are seeking a marriage license in wisconsin, you must have been 21 years of age before you intend to apply to apply.\nD. You must be at least 21 years old when you obtain a marriage license. If you are a man, you can only get married during the two-year period established by the court in your county.","prompt":"[header] How to apply for a marriage license in wisconsin [title] Know who can marry in wisconsin. [step] In order to obtain a marriage license in wisconsin, you must meet certain requirements regarding who can get married. The following restrictions and allowances apply in wisconsin : [substeps] Age.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5523,"error":null,"output":"A"}
{"question_id":"hellaswag:00006844","item_index":152,"attempt":0,"prompt_hash":"918cdae144d3a56c373c89a6488cf6f4db0ca029c41afed4a15fea5e7f0c81db","question":"[header] How to get rid of rosy cheeks [title] Clean and moisturize your face. [step] Before you start with makeup, make sure to wash your face first with a gentle cleanser. In addition, use a moisturizer to help keep your skin hydrated.\n\nA. [substeps] It's important to be gentle with your skin, as being rough can make your skin redder or cause rosacea flareups if you have that condition. Try blotting your skin after you wash it instead of rubbing it, for instance, and try to be gentle when washing your face.\nB. You don't want to put concealer on any fresh pimples or acne, as these will ruin your plump cheeks. The moisturizer will keep your skin looking smooth and glowing, without covering up harsh redness or other blemishes.\nC. Moisturizing your face helps to seal your pores, keeping your complexion supple and glowing. [substeps] The ideal product for rosy cheeks is witch hazel, which has been shown to reduce redness.\nD. [substeps] Avoid using harsh cleansers like those found in scented body lotions and shower gel. [title] Apply an active cleanser for rosy cheeks.","prompt":"[header] How to get rid of rosy cheeks [title] Clean and moisturize your face. [step] Before you start with makeup, make sure to wash your face first with a gentle cleanser. In addition, use a moisturizer to help keep your skin hydrated.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5523,"error":null,"output":"A"}
{"question_id":"hellaswag:00004908","item_index":153,"attempt":0,"prompt_hash":"c08c37f0302d7471129ebb73762e0a265f16c9daa5d0a83bdae3a38494c5267b","question":"[header] How to get beautiful, glowing skin [title] Drink up. [step] Aim to get 6 to 8 glasses of water a day. The water will clear your skin and make it glow because it makes it easier for your body to flush out toxins quickly.\n\nA. Energy and electrolyte deficiencies shouldn't be limited to more than that. [substeps] Be sure to drink at least 13 glasses of water every day, along with a glass of purified, bottled or filtered water.\nB. What's more, it will help your skin moisturize, which helps remove open pores. The water will also help hydrate your skin, thus beginning your sun-kissed days again.\nC. [substeps] Carry a water bottle with you all day to make sure you always have water on hand. Drink herbal tea or other non-sugary beverages to hydrate when you're tired of water.\nD. Eating healthy foods rich in protein makes it easier for you to hydrate, and your body will thank you for it. [title] Speak to your doctor or a dermatologist.","prompt":"[header] How to get beautiful, glowing skin [title] Drink up. [step] Aim to get 6 to 8 glasses of water a day. The water will clear your skin and make it glow because it makes it easier for your body to flush out toxins quickly.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5523,"error":null,"output":"C"}
{"question_id":"hellaswag:00008668","item_index":154,"attempt":0,"prompt_hash":"969bf57a289440e858baa25ff9807927adf88c86018b88ab1eeb700d2de4ef34","question":"[header] How to buy a strapless bra [title] Get a measuring tape. [step] A measuring tape is necessary in order to accurately measure your bust size. The flexibility of the measuring tape will allow you to measure the circumference of your bust.\n\nA. Scoop your waist and cup it with either your hands or a measuring tape. Then, measure the width of your fullest part, i.e., your bust.\nB. It will also allow you to easily make adjustments to your bra, such as buying an artificial hemline or not purchasing one at all. [substeps] Measure your shoulders, chest, stomach, and back, including between your collarbones.\nC. Before purchasing a strapless bra, you can carefully measure out where the front of your bra will be. [substeps] Find a bra that's comfortable.\nD. [substeps] You can go to your local pharmacy, like walgreens or cvs, or a fabric store to find a measuring tape. You can also try visiting a target or a walmart; both stores have sewing sections.","prompt":"[header] How to buy a strapless bra [title] Get a measuring tape. [step] A measuring tape is necessary in order to accurately measure your bust size. The flexibility of the measuring tape will allow you to measure the circumference of your bust.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5524,"error":null,"output":"D"}
{"question_id":"hellaswag:00005794","item_index":155,"attempt":0,"prompt_hash":"0904c2402aac6f8b4bc4449b167dc795d121b33709664edcee05f9e6e39a6533","question":"[header] How to make green tea bath bombs [title] Mix the dry ingredients together but leave out the tea leaves for now. [step] Use a large ceramic mixing bowl. Combine well.\n\nA. [title] Pour the green tea flowers into the mould for the tea bomb. [step] Spread the eyes of the dough onto the mould using it.\nB. [title] Add the bran and butter in a separate bowl. [step] You do not need a lot as mixture may rise over time and become thin.\nC. [title] Add the green tea leaves and warm water in a kettle over low heat. [step] Stir constantly with a spoon until it thickens to the consistency of tea.\nD. [title] Pour the oil into the mixture and stir until it makes the entire mixture moist. [step] Once mixed in, add the use an essential oil such as rose or geranium for added benefits 1-10 drops that can be dispersed in a full bath and stir through.","prompt":"[header] How to make green tea bath bombs [title] Mix the dry ingredients together but leave out the tea leaves for now. [step] Use a large ceramic mixing bowl. Combine well.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5693,"error":null,"output":"C"}
{"question_id":"hellaswag:00005288","item_index":156,"attempt":0,"prompt_hash":"4f1a62aa855abdfb1dd85a9d6f34f0d99ab53baad6cd5528138064aa3f703455","question":"[header] How to do your makeup [title] Wash your face, then apply some toner and moisturizer. [step] Makeup can clog pores, so it is important to start with a clean face. Toner will help close your pores, while moisturizer will smooth out your skin and make it easier to apply foundation.\n\nA. [substeps] You can find toner at beauty supply stores. It will keep your skin smooth and prevent clogged pores from forming.\nB. It will also prevent your skin from drying out and flaking beneath your foundation. [substeps] When applying moisturizer, focus on areas that dry out the most, such as the forehead and cheeks.\nC. Depending on your skin type, you can use either warm or cool water to clean your face. [substeps] Toner should be applied between the brows and towards the outer edge of each eyelid; this will help guide your makeup course.\nD. Check the ingredients label on your eyeshadow or shadow before you buy it. Some of the other products may require different ingredients.","prompt":"[header] How to do your makeup [title] Wash your face, then apply some toner and moisturizer. [step] Makeup can clog pores, so it is important to start with a clean face. Toner will help close your pores, while moisturizer will smooth out your skin and make it easier to apply foundation.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5690,"error":null,"output":"B"}
{"question_id":"hellaswag:00004735","item_index":157,"attempt":0,"prompt_hash":"b02cc47a22683b2203f96c420546fc451d0a5f9f254d04b8c0e39b36e6f97e04","question":"[header] How to apologize after a heated argument [title] Avoid excuses. [step] However you choose to apologize, there are a few things you should avoid. Don't automatically try to justify your statements or behavior, or try to reiterate why you were right about whatever you were arguing about.\n\nA. Apologize gracefully, but didn't do the right thing in the first place. Avoid ones that leave room for a more appropriate apology.\nB. Pay attention to word choice and approach to prevent giving an apology that isn't actually an apology. [substeps] Avoid \" ifs \" and \" buts \".\nC. If you do want to explain yourself, then show some remorse by lying or avoiding the subject when you do apologize. [substeps] Try not to limit yourself to excuses.\nD. Instead, try to build a positive association in yourself with the other person by thinking about why you said what you said, why you did it, and whether or not you really were the primary culprit. Don't let yourself become emotionally entangled by criticizing or making excuses.","prompt":"[header] How to apologize after a heated argument [title] Avoid excuses. [step] However you choose to apologize, there are a few things you should avoid. Don't automatically try to justify your statements or behavior, or try to reiterate why you were right about whatever you were arguing about.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5691,"error":null,"output":"D"}
{"question_id":"hellaswag:00007418","item_index":158,"attempt":0,"prompt_hash":"b45b43f8bd4d9d917e34da94c2a19744043487fe1a63fa907e9344cdffe7c265","question":"[header] How to mount a led headlamp (miner's lamp) light and rear blinker on a bicycle helmet [title] Choose a bike helmet to modify. [step] If you're into bicycling, you should already have one. If you have more than one, use the old one, at least first, in case you ruin it.\n\nA. Contact a bike shop to have your helmet cut and remade by a professional. [title] Calculate a frame length, known as frame height.\nB. This is so you don't constantly have to replace it and neither does hiring a dealership. Consider that you may want a low-end, easy-to-operate, and eco-friendly helmet instead of a 4-wheel-drive one.\nC. A helmet with a flattish rear end, not a pointed one, will be easiest to use. [title] Pick out a headlamp for adaptation to the helmet.\nD. The standards for bicycle helmets differ depending on how old the bike is, and, in general, they are fairly inexpensive. [title] Attach your led headlamp to the bicycle helmet.","prompt":"[header] How to mount a led headlamp (miner's lamp) light and rear blinker on a bicycle helmet [title] Choose a bike helmet to modify. [step] If you're into bicycling, you should already have one. If you have more than one, use the old one, at least first, in case you ruin it.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5690,"error":null,"output":"D"}
{"question_id":"hellaswag:00006468","item_index":159,"attempt":0,"prompt_hash":"a7aea7dc3779da2c6a9856f0c127ff691e1fec533fd309c784c6e77c64807938","question":"[header] How to chlorine wash a pool [title] Drain your pool. [step] Before you can chlorine wash your pool, you will need to drain it. Drain your pool using the waste line of your pool pump and motor system.\n\nA. Pour your trash into your waste bin, and close your drain lid. Then, pour water through your drain hose into the garbage.\nB. Go to the gas pump and turn it on to drain the pool. Then add your pool's waste hose into the drain.\nC. Alternately, hose your pool and turn off the faucets. Let the pool drain for 1-2 minutes if you wish to pool wash it more often.\nD. [substeps] Check your water discharge regulations before draining your pool so you can dispose of the water you pump out in a responsible manner. Many municipalities prohibit pumping water into street sewers.","prompt":"[header] How to chlorine wash a pool [title] Drain your pool. [step] Before you can chlorine wash your pool, you will need to drain it. Drain your pool using the waste line of your pool pump and motor system.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5689,"error":null,"output":"D"}
{"question_id":"hellaswag:00006132","item_index":160,"attempt":0,"prompt_hash":"062830f9babd3489a9824176871fb86d7bc8c4e164f4eb1a0faafe692dcd62bd","question":"[header] How to kiss a boy passionately [title] Relax. [step] You have nothing to prove and nothing to lose. Kissing is a two way street and the guy is probably more nervous than you are.\n\nA. If he isn't ready to kiss, then it is nothing to be embarrassed about. [title] Determine how to tell him to stop.\nB. Nothing to prove from the outset is going to make him more excited to kiss you. This will only make him feel pressured into going closer to you.\nC. Take deep breaths, but don't make it too obvious that you're trying to calm yourself down. [title] Match the energy of the person you're with.\nD. Before you start feeling jittery, say to yourself, \" let's keep this at a low-stakes level, but everything is better after we're both in the moment. \" he may be nervous too, which may cause nerves to escalate on your part.","prompt":"[header] How to kiss a boy passionately [title] Relax. [step] You have nothing to prove and nothing to lose. Kissing is a two way street and the guy is probably more nervous than you are.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5994,"error":null,"output":"C"}
{"question_id":"hellaswag:00009458","item_index":161,"attempt":0,"prompt_hash":"211e162aa9a712fc98487fc6e1d0403a2f407860aedf5a7a4d969a4dd801d780","question":"[header] How to plan winter engagement photos [title] Choose between candid or staged photos. [step] You should think about what type of photos you want before setting up a session with a photographer. Consider if you would like staged or candid photos.\n\nA. [substeps] You may choose to create a professional-looking wedding scene with settings, exterior scenery, and cast created by a professional photographer. Or, you may want to choose photos with a dramatic, bohemian nature.\nB. You might also opt for stills and candid shots. [substeps] Candid shots might include blurry photos of the same colored object with colorful details, or they might include close-ups of an object such as a vase.\nC. [substeps] For example, do you want a staged frame, like an old wreath, or are you trying to photograph an actual wreath? An overcast backyard may be an ideal backdrop for an impromptu photo shoot. If you want to photograph everyday outdoor events such as a family gathering, you might not want an indoor photograph.\nD. Candid photos are great if you want a light and fun look to your photos. Staged photos are most common and are usually better for photos that will be framed.","prompt":"[header] How to plan winter engagement photos [title] Choose between candid or staged photos. [step] You should think about what type of photos you want before setting up a session with a photographer. Consider if you would like staged or candid photos.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5986,"error":null,"output":"D"}
{"question_id":"hellaswag:00009478","item_index":162,"attempt":0,"prompt_hash":"eedc1fdcbae7450e3655637f26e8e6bee8ba0e1cd141ee466ba436ba6e628869","question":"[header] How to get voted captain [title] Make sure you know the material. [step] This would include the rules, moves, names of equipment, etc. That way anyone with a question can come to you.\n\nA. [title] For sports, make sure you are flexible. [step] Flexibility can help prevent sore and pulled muscles.\nB. If you don't know the material, just start walking around like you're the center of attention. [substeps] Make sure you know what to wear and how many people to talk to.\nC. If you need a clue or idea, you'll need to know how to use it. You'll need the group's information as well as your own.\nD. If you don't know it, take this from a previous teacher, or a trusted student who does. You can pick whatever comes to mind for the specific subject matter you're being talked about.","prompt":"[header] How to get voted captain [title] Make sure you know the material. [step] This would include the rules, moves, names of equipment, etc. That way anyone with a question can come to you.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5991,"error":null,"output":"B"}
{"question_id":"hellaswag:00002731","item_index":163,"attempt":0,"prompt_hash":"ffb73232af7f1d1f444c30c5c0c95621e72ccbaa4f130a3351ed86397708a2ad","question":"A man is looking all over for his dog. He said he paid his dog 10 dollars to cut the grass and now he's gone. he\n\nA. said he was going to buy the dog esp and could give it a home on his family farm.\nB. finally finds the dog who is driving the riding lawn mower.\nC. shows his dog and pet, the dog is sitting on the ground stretching its front legs and then washing his front paws.\nD. is mowing his backyard and walking to the store.","prompt":"A man is looking all over for his dog. He said he paid his dog 10 dollars to cut the grass and now he's gone. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5990,"error":null,"output":"B"}
{"question_id":"hellaswag:00002566","item_index":164,"attempt":0,"prompt_hash":"238b65be5989916ac910efa0bdefd66e3b57cb64cf2b5a65fb29ad5b1ce4e24e","question":"You get a wash cloth and you put it under the faucet to get wet and then you rinse it out so it's not soaking. Then you use it to wash the mirror wrong back and forth and all around. with a dry cloth\n\nA. you put it under the faucet so you don't have to wash your hands.\nB. it's dry, it's dry, it's clean, it's shiny and it looks like you brushed it.\nC. you go over it and then your mirrors are clean.\nD. , you and cold water with soap and you are done.","prompt":"You get a wash cloth and you put it under the faucet to get wet and then you rinse it out so it's not soaking. Then you use it to wash the mirror wrong back and forth and all around. with a dry cloth","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5992,"error":null,"output":"A"}
{"question_id":"hellaswag:00006649","item_index":165,"attempt":0,"prompt_hash":"b9c2d72218ba812acbb4b9c6fc48f3f354850638555e474ac18f6c7f0114cdb4","question":"[header] How to build your young child's social skills [title] Give your child some light chores to do. [step] Children can already understand some instructions, as early as when they are 2 or 3 years of age. Giving a little work to do will make him or her learn to obey and follow instructions.\n\nA. As your child goes on to learn these skills, give some solid tasks and praise them. [substeps] If your child goes to the bathroom and does not flush the toilet after she does, just praise her and tell her, \" that's a good boy right there.\nB. [substeps] Let your child help with the cleaning, this helps really you identify which of the items can be washed and which can be cleaned yourself. Organizing the baby's diaper area helps your child keep organized in case of an accident.\nC. Remember, this is not to make your child feel tired or sad. It is good for your child when he or she starts learning to know that helping out is a good thing to do.\nD. It will also help your child strengthen and relax emotionally. Giving him or her some light chores can also reinforce his or her social skills.","prompt":"[header] How to build your young child's social skills [title] Give your child some light chores to do. [step] Children can already understand some instructions, as early as when they are 2 or 3 years of age. Giving a little work to do will make him or her learn to obey and follow instructions.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5989,"error":null,"output":"C"}
{"question_id":"hellaswag:00009629","item_index":166,"attempt":0,"prompt_hash":"adc9f5eed6f4eb43b6d5bd59fd6296fa500d55b9f0a56f7bd8d78a3db16af585","question":"[header] How to care for your chiminea [title] The fire in a clay chiminea needs to be built on a bed of sand so the fire doesn't actually come into contact with the clay. [step] Check your instructions carefully to make sure you set the fire properly. Not doing so may cause cracks in a clay chiminea.\n\nA. [substeps] Then, add sediment to the clay. Vinegar and salt can be added to ensure that the ash doesn't spread to your clay chiminea.\nB. [title] Dry out the chiminea before placing it in a warm place. [step] The clay chiminea should stay in a warm spot until the melting process is finished.\nC. Children's play sand will work well. Don't move your chiminea with the sand in place in the bowl.\nD. [title] Place a piece of sand on top of the entire cactus and add a few inches of soil. [step] Fill in the clay surrounding the cactus as well as the supports.","prompt":"[header] How to care for your chiminea [title] The fire in a clay chiminea needs to be built on a bed of sand so the fire doesn't actually come into contact with the clay. [step] Check your instructions carefully to make sure you set the fire properly. Not doing so may cause cracks in a clay chiminea.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5986,"error":null,"output":"B"}
{"question_id":"hellaswag:00001480","item_index":167,"attempt":0,"prompt_hash":"b27473d779307a1ed5857cbbcd599e5f7f22bffc1eea3ba765f19763577e7721","question":"We see a colorful opening scene. Kids in colorful clothes walk on a stage and get in place. the kids\n\nA. then start playing dodge ball.\nB. climb the stairs and dance.\nC. wait and then person clips their toenails.\nD. perform zumba on a stage.","prompt":"We see a colorful opening scene. Kids in colorful clothes walk on a stage and get in place. the kids","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5990,"error":null,"output":"A"}
{"question_id":"hellaswag:00004802","item_index":168,"attempt":0,"prompt_hash":"9eb88a4d87770f056b41398a1300feb9856b68c3501298893b6c397dddc51820","question":"[header] How to get rid of dark elbows [title] Use lemon juice. [step] Lemon juice contains citric acid which is a natural bleaching agent. Therefore, applying lemon juice to elbows can help to lighten the skin.\n\nA. [substeps] Squeeze lemon juice over the dark areas of your elbows. Wait 15 to 30 minutes and apply the juice to your elbows.\nB. Lemon juice can also help to remove the puffiness of those scars. [substeps] Place the lemon juice directly on your scars, then cover with a clean cotton towel.\nC. Apply lemon juice three times a day, according to the instructions on the bottle. [substeps] Alternatively, you can also spray lemon juice onto dark elbows by placing sun exposure detergent on the underarms and a load of laundry detergent in your wash machine.\nD. To use : [substeps] Cut a large lemon in half. Squeeze out some of the juice from both halves so that you have two hollow halves or \" cups \".","prompt":"[header] How to get rid of dark elbows [title] Use lemon juice. [step] Lemon juice contains citric acid which is a natural bleaching agent. Therefore, applying lemon juice to elbows can help to lighten the skin.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5987,"error":null,"output":"A"}
{"question_id":"hellaswag:00009332","item_index":169,"attempt":0,"prompt_hash":"207c9f3f6bbf6a39630e6caf9db5c032cd2266434ad9c1329f1544369dff435a","question":"[header] How to use white eyeliner [title] Draw a thick white line along the upper lid. [step] Using white eyeliner, carefully trace the lash line of the upper eyelid. This line should be very wide-up to twice as thick as normal eyeliner would be.\n\nA. Make sure your line is long enough to enter the top part of the eye. [substeps] If you would like your liner to be bigger, you can use the same technique.\nB. The line should make a tiny triangle shape in the corners of the eye. [substeps] Once you've achieved this, cover the entire lid with white eyeliner.\nC. [substeps] At the top the line should be angled downward on both sides and extend across the eyelid. Blend this line in at the very tip of the eyelash.\nD. [substeps] You can technically stop here, but the results might seem overpowering if you're not used to using white eyeliner. If you do choose to use white eyeliner as a solo liner, though, consider making this line somewhat thinner instead of opting for a super thick line.","prompt":"[header] How to use white eyeliner [title] Draw a thick white line along the upper lid. [step] Using white eyeliner, carefully trace the lash line of the upper eyelid. This line should be very wide-up to twice as thick as normal eyeliner would be.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5981,"error":null,"output":"D"}
{"question_id":"hellaswag:00006026","item_index":170,"attempt":0,"prompt_hash":"cdafa4c2f3e2f8899fb55fcd43601709218586cf5c2492434d3093abeafe9336","question":"[header] How to sell something [title] Understand how people make decisions to buy your product. [step] A lot of research shows that people make emotional decisions when they buy things. So consider what emotional triggers might make a person want to buy your product, and incorporate them in your advertising.\n\nA. Think about how you might advertise, and how you might make some money. [substeps] For example, maybe you want to sell organic toothpaste.\nB. [title] Decide if you want to advertise directly to your target market. [step] Before embarking on a heavy marketing campaign, you may want to consider if you would qualify for the advantage offered by your market share.\nC. [substeps] By applying emotional responses, your sales pitch is more effective. For example, if someone has a favorite movie that they love, encourage her to watch the movie with you.\nD. [substeps] Realize that people often respond to stories about other people, not just dry facts. For example, if you can include a testimonial from a person who has used your product, that will have an emotional response in people.","prompt":"[header] How to sell something [title] Understand how people make decisions to buy your product. [step] A lot of research shows that people make emotional decisions when they buy things. So consider what emotional triggers might make a person want to buy your product, and incorporate them in your advertising.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5977,"error":null,"output":"D"}
{"question_id":"hellaswag:00005513","item_index":171,"attempt":0,"prompt_hash":"4d019c2c9214b01b155d9f58e20196a3964a5758000bfe804def32ab72506b6c","question":"[header] How to have nice and clean hair [title] Select the best shampoo for your hair type. [step] With so many brands and choices, it is easy to be overwhelmed with selecting a shampoo. When it comes to selecting the best shampoo, decide what your hair needs most.\n\nA. Remember that changing your hair type can result in different growth patterns, as well as increased risk for hair loss. [substeps] If you have thick hair, choose a shampoo that is specifically for curly hair.\nB. If your hair is usually straight, you may need a sulfate-free shampoo. If it is generally wavy, you may want to vary this amount to balance out the curls.\nC. If you have straight or curly hair, you will need to follow the following tips : [substeps] Don't use a shampoo that is too harsh and packed with chemicals or sulfates. Avoid using a salon-specific shampoo because these will create tight, wavy hair.\nD. If you dye or add color to your hair, select shampoos that will protect your treatment. Also, most shampoos have extra benefits such as volume and shine.","prompt":"[header] How to have nice and clean hair [title] Select the best shampoo for your hair type. [step] With so many brands and choices, it is easy to be overwhelmed with selecting a shampoo. When it comes to selecting the best shampoo, decide what your hair needs most.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5977,"error":null,"output":"A"}
{"question_id":"hellaswag:00001216","item_index":172,"attempt":0,"prompt_hash":"93b157eaba3f6d855bb293fc81992a11f1db030c97a5909888e0077dd02de048","question":"An elderly man is standing in a basketball gym watching a young team of athletes practice. Another male then begins shuffling and playing defense as the kids practice their layups. they\n\nA. stay in their two lines and starts making passes to one another to make a goal.\nB. form a circle back and forth beginning their practice game.\nC. fine with practice hitting the balls and dribbling the ball in the basket.\nD. leave the gym with their practice finished and give each other a hug at the end.","prompt":"An elderly man is standing in a basketball gym watching a young team of athletes practice. Another male then begins shuffling and playing defense as the kids practice their layups. they","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5982,"error":null,"output":"D"}
{"question_id":"hellaswag:00006480","item_index":173,"attempt":0,"prompt_hash":"0cebfa925a0e43f7df6f52eba58b8e4053e2aa7c8c3997e1d3657aec30764816","question":"[header] How to find out if your friend's crush is crushing back [title] Be up front. [step] The easiest and most effective way is to just ask your friend's crush about their feelings. If you're having trouble working up the courage to ask such a personal question, there's a few ways to make the process easier.\n\nA. [substeps] Check if your friend's crush already told them that they want to get back together. If they haven't told you yet, call them and let them know it's time to talk and be upfront about your feelings.\nB. [substeps] Consider the time and place you're going to have the conversation. Make sure the crush is in a comfortable area, probably out of earshot from anyone who could listen in.\nC. Most people love sharing their feelings with someone who cares about them. If your friend has been hanging out with that same guy with their crush, they probably have feelings for someone too.\nD. [substeps] Once you've reached this point, you can use this information to give your best advice. For example, you might say, \" hey, i hear you guys are crushing back and forth now.","prompt":"[header] How to find out if your friend's crush is crushing back [title] Be up front. [step] The easiest and most effective way is to just ask your friend's crush about their feelings. If you're having trouble working up the courage to ask such a personal question, there's a few ways to make the process easier.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5977,"error":null,"output":"B"}
{"question_id":"hellaswag:00001431","item_index":174,"attempt":0,"prompt_hash":"85a75f9e9caaa8f2835beb96082bf1d1d390ffbbc68a61c070d0e745f1281016","question":"Next she adds 1 teaspoon of vanilla extract and she mixes everything up, then she adds 1 1/4 cups of all purpose flour along with 1/4 cup of unsweetened cocoa powder and salt. then she\n\nA. adds 1/3 cups of shape sugar and stirs the mixture until it glistens.\nB. adds 1/2 teaspoon of some sugar to the mix, some vanilla, and a little bit of vanilla extract and she sets it all on a pan with a lid.\nC. adds baking powder and baking soda and she whisks it until smooth.\nD. starts to roll out the mix and hang it from a frying pan.","prompt":"Next she adds 1 teaspoon of vanilla extract and she mixes everything up, then she adds 1 1/4 cups of all purpose flour along with 1/4 cup of unsweetened cocoa powder and salt. then she","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5981,"error":null,"output":"C"}
{"question_id":"hellaswag:00007336","item_index":175,"attempt":0,"prompt_hash":"942ccbad19746e284e94427f83a981f513d89811ed794a2e8be5fd7ea595ecae","question":"[header] How to stop an angora sweater from shedding [title] Determine if your sweater can be frozen. [step] This method of freezing works best for angora sweaters that are blended with another material. If your sweater is 100% angora, your sweater will still shed because of the high amount of angora.\n\nA. [title] Look for a sweater made of wool or suede. [step] These fibers won't be able to cause the sweater to shrink.\nB. [substeps] This method is particularly important to avoid in larger sizes of sweaters. It will completely melt the outer layer of the sweater.\nC. [substeps] If your sweater is made from synthetic fibers, it is not ideal at all. Synthetic fibers will degrade over time, especially in colder weather, so don't use this method if you are planning to keep this sweater.\nD. The freezing method relies on making the fibers easier to shake out. So, after you freeze the sweater you can vigorously shake out all of the fibers that will fall out so they come out at once instead of throughout the day.","prompt":"[header] How to stop an angora sweater from shedding [title] Determine if your sweater can be frozen. [step] This method of freezing works best for angora sweaters that are blended with another material. If your sweater is 100% angora, your sweater will still shed because of the high amount of angora.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5981,"error":null,"output":"C"}
{"question_id":"hellaswag:00005555","item_index":176,"attempt":0,"prompt_hash":"01496d06f31dca130eb9dfba43f94aa6f93b3b2d47175748ebf23db5e7351c91","question":"[header] How to increase your situational awareness [title] Assess your surroundings for threats. [step] The first step to increasing your situational awareness is to assess your surroundings in full. Look around you and take account of the type of situation you are in.\n\nA. If you are in a rural area, it may be a lack of air quality or steady but even bodies of water. There are differences between different types of terrain but the basic issue is the stability of plants throughout dry conditions.\nB. Different environments will require different levels of focus to maintain high situational awareness. [substeps] A city environment presents very different dangers and distractions than a rural one.\nC. You may have stopped at a nearby building or road. You may be nearby a source of water or vegetable stand.\nD. This will help you determine if you have a sensitive skin, or if a friendly coworker or friend is having a good day. Here's a list of the elements that affect your environment : [substeps] Noise: your mind and body tense and release signals when there are sounds.","prompt":"[header] How to increase your situational awareness [title] Assess your surroundings for threats. [step] The first step to increasing your situational awareness is to assess your surroundings in full. Look around you and take account of the type of situation you are in.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5980,"error":null,"output":"B"}
{"question_id":"hellaswag:00002344","item_index":177,"attempt":0,"prompt_hash":"319777e635ef5f982cfe96c49cd0efe89148dd6a9ec0b217ee97fb1255e85fc0","question":"The baby is looking at the camera and then the hand wiped his face. the hand\n\nA. wiped the baby's face and the baby opened his mouth.\nB. start wrapping a gift the hand moved down to the bottle and puts some tissue paper on the top part.\nC. lays on the table with the baby on it and then the little girl hits the camera again.\nD. talked and raised its hand.","prompt":"The baby is looking at the camera and then the hand wiped his face. the hand","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5980,"error":null,"output":"A"}
{"question_id":"hellaswag:00006471","item_index":178,"attempt":0,"prompt_hash":"1fa6a7ed7cfe2c72d7850ce317fbed280e4d1ed8dceece61ca013145abcc0527","question":"[header] How to slow the hectic pace of your life [title] Make a choice. [step] Most of us can cope with periods of frenzied activity if they're followed by inactivity and rest. [substeps] If you feel rushed and behind, then stop to notice what you've chosen to do.\n\nA. Conversely, if you feel that you're disconnected from what you're doing or react inappropriately, stop and change the rhythm. If you feel a relentless flow or portion of your day is draining of sensation, replace your hectic routine with activity.\nB. If you feel ambitious and happy, then take this opportunity to communicate with other people. [title] Don't get angry and ignore distractions.\nC. Choose to do a few extra things per day and save the rest for another day. Be realistic about the time you have and how long it takes to accomplish a certain task.\nD. Maybe you want to get up and go for a walk but no one's around. Make a decision that you feel you have to do, without second thoughts.","prompt":"[header] How to slow the hectic pace of your life [title] Make a choice. [step] Most of us can cope with periods of frenzied activity if they're followed by inactivity and rest. [substeps] If you feel rushed and behind, then stop to notice what you've chosen to do.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5977,"error":null,"output":"C"}
{"question_id":"hellaswag:00000669","item_index":179,"attempt":0,"prompt_hash":"38b0d89d0e7fbc2cde3c737c7f06198e2bf4ccebf8be681c4127605aad3b877a","question":"A man is seen wandering around a field cutting grass that pans into other people chopping up the grass as well. the camera\n\nA. pans around the area to see people cutting and ends with a close up of a person smiling and continuing to cut.\nB. shows more shots of the people cutting the grass and walk back to the camera.\nC. pans all around a row of trees and leads into him cutting up more dead grass and a man opposing him in a shoulder to shoulder hug.\nD. pans around to more people cutting up the field as well as still shots of people moving around.","prompt":"A man is seen wandering around a field cutting grass that pans into other people chopping up the grass as well. the camera","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5980,"error":null,"output":"D"}
{"question_id":"hellaswag:00006650","item_index":180,"attempt":0,"prompt_hash":"0e89dc48c3c2b216507f4b5faedbc3dd70e0d3426a3f6c74f68926052d31b9b6","question":"[header] How to paint the philippines flag on your nails [title] Paint your nails a solid white. [title] Put another coat of white to look prettier and shinier, but make sure to do it only when the first coat has already dried. [title] Make a \" v \" shape on top of your nails, with two nail tapes.\n\nA. [title] Get a toothpick, dip it in your blue nail polish, and follow the tape on the left. [title] Get a different toothpick, and a red nail polish.\nB. [title] The \" v \" shape is going to be faster for the second coat. [title] If you want for your nails to look fuller, then apply a \" v \" shape with your nail tape.\nC. [title] Work your way up the finger nail. [step] Use thinner tape (one end of the tape needs to be longer) and go up so that no \" v \" is visible after painting your nails.\nD. [title] Apply' v' over the tape. [title] Cover with another coat of white nail polish.","prompt":"[header] How to paint the philippines flag on your nails [title] Paint your nails a solid white. [title] Put another coat of white to look prettier and shinier, but make sure to do it only when the first coat has already dried. [title] Make a \" v \" shape on top of your nails, with two nail tapes.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5980,"error":null,"output":"A"}
{"question_id":"hellaswag:00001443","item_index":181,"attempt":0,"prompt_hash":"4db778b0c0690e3b7d7d6cb93a88f92d80e3a0704099e6a8b2e05d08fb5a98e9","question":"A young man is seen sitting at the bottom of a pool. he\n\nA. is holding up a hula hoop as he speaks to the camera.\nB. begins moving closer to the camera while smiling.\nC. shines shoes, and turns around.\nD. is then shown flipping multiple times while people watch on the sides.","prompt":"A young man is seen sitting at the bottom of a pool. he","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6109,"error":null,"output":"D"}
{"question_id":"hellaswag:00002586","item_index":182,"attempt":0,"prompt_hash":"5f7b4dd9efe0ef3135df20113fca82a005b2ff5985f3115b6b6fb398ad58f0be","question":"A video of scissor kick jumping at track is shown. a few people\n\nA. are now in a race track where they are practicing their legs while being led by judges.\nB. walk out into the pit.\nC. are in the stands with their scissor legs wrapped around each other.\nD. take turns at varying heights and all are successful.","prompt":"A video of scissor kick jumping at track is shown. a few people","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6109,"error":null,"output":"C"}
{"question_id":"hellaswag:00001969","item_index":183,"attempt":0,"prompt_hash":"a8e8834370de6c3aa32678b9072c90502aebef21ec5da34d5c25ae474f46c731","question":"A man bends at the knees in front of a barbell. He lifts it slowly, bringing it to his chest. he\n\nA. throws the weight down over his head.\nB. then puts it down and bends at the knees.\nC. then lowers it slowly over his head.\nD. pauses, then lifts it over his head before dropping it back to the ground.","prompt":"A man bends at the knees in front of a barbell. He lifts it slowly, bringing it to his chest. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6108,"error":null,"output":"D"}
{"question_id":"hellaswag:00009256","item_index":184,"attempt":0,"prompt_hash":"588e8e24842d1f5cc7b70d10ee7cb93e63fbe48f00bc7a2ddfef5d68fd0c206e","question":"[header] How to get around venice on the cheap [title] Walk as much as you can. [step] Venice is a walking city, with streets that are built for pedestrians only. All you need is a good map, or a willingness to get lost as you wander.\n\nA. [substeps] Take advantage of a walking tour if you want to see historic sites with the help of a guide. The twisting, narrow streets of venice might be frustrating and overwhelming if you have never been there before.\nB. If you don't have a good map, use your smart phone or cash from your bank account to start walking. Crowds can be incredibly hectic and there is no guarantee you'll find any fine restaurants or shops for walking.\nC. Tourist maps will show you pictures of areas open to tourists, and guides can provide you with locations that you would not otherwise have thought of. Germany, italy and the italian coast make plenty of good travel sites.\nD. Both eurobus and airbus carry bus or train passes, but their routes are private and much less expensive. Even if you do not drive, walk one or two to get around venice.","prompt":"[header] How to get around venice on the cheap [title] Walk as much as you can. [step] Venice is a walking city, with streets that are built for pedestrians only. All you need is a good map, or a willingness to get lost as you wander.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6109,"error":null,"output":"A"}
{"question_id":"hellaswag:00002634","item_index":185,"attempt":0,"prompt_hash":"a60337c17492f3d44096aeda3df9c11e218cce9aae4bb94225dd08ea69e5fe8e","question":"A close up of an exercise bike is shown with a person walking into frame. the person\n\nA. is then seen doing several different moves on the bike.\nB. then uses a foot to move around along the bike's pedals.\nC. climbs on the bike and begins riding furiously.\nD. smudges down one bar as the camera captures it from several angles.","prompt":"A close up of an exercise bike is shown with a person walking into frame. the person","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6109,"error":null,"output":"A"}
{"question_id":"hellaswag:00000888","item_index":186,"attempt":0,"prompt_hash":"e947f28ea65895c44233215d352dc71ed059ffbd1c498b1d231657ca67a9f5f8","question":"A kid takes a drink and swishes it around in their mouth. They spit it out into a sink. They take another drink out of a cup. they\n\nA. bend down and clean large puddles from the floor.\nB. put their mouth on a towel.\nC. sleep together in their bedroom.\nD. start to talk on the table.","prompt":"A kid takes a drink and swishes it around in their mouth. They spit it out into a sink. They take another drink out of a cup. they","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6107,"error":null,"output":"B"}
{"question_id":"hellaswag:00007154","item_index":187,"attempt":0,"prompt_hash":"d3c40c0f6d473bc807bc492205983dcf92b6795cd5f2a4d334938050d44a658e","question":"[header] How to find a person's grave [title] Know the person's full name. [step] It's best if you know the person's first, middle, and last name. Knowing this information will make him or her easier to find.\n\nA. [substeps] If the person knows that you're looking for someone you don't know, this is even better! If you don't know the name of your relative, you won't be able to get the stone taken. [title] Try the person's birth date.\nB. Because some combinations of first and last names are common, it's easier to narrow down if you have a middle name, too. [substeps] You can find a person's full name by looking through family documents if you have them.\nC. [substeps] Watch out for footnotes, which are often missed or maybe even lose easily. When writing down the name of a person's grave, keep in mind that you don't necessarily need to know what it is, but you can certainly search his or her social security number in order to figure out his or her location.\nD. This will, in turn, make your search more difficult as well. If you know their middle name or surname, you're more likely to find the right person.","prompt":"[header] How to find a person's grave [title] Know the person's full name. [step] It's best if you know the person's first, middle, and last name. Knowing this information will make him or her easier to find.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6033,"error":null,"output":"B"}
{"question_id":"hellaswag:00007602","item_index":188,"attempt":0,"prompt_hash":"848af1680bdfa73efa7cddd83ec54e1725b3355108dc8ec203c42e994a30c34b","question":"[header] How to measure for a laptop skin [title] Research the laptop skin that you want to purchase from the many that are available online. [step] You will want to investigate several websites before making a final choice because of the number of options that are out there. [title] Choose a website that will best suite your design needs.\n\nA. [step] There are quite a few services that are free of charge and you are looking for a laptop that will fulfill all of those needs. One great type of website that works well with laptop skin is cooldial.\nB. [step] This will determine how much measuring you will need to do before ordering the laptop skin. Most websites will ask you for the model of your laptop and will send you a precut laptop skin that will be close to the size of your computer.\nC. [step] If a website is bothering you, then write a list of all the options that are available and wait to compare prices. [title] Get a laptop body that can connect to your computer via usb devices.\nD. [title] Most laptops will require data about the computer, including its operating system, the size, blu-ray jack, external performance, software, available storage places, storage levels and the like. [step] Be sure to go with your design for this in order to fully embrace your laptop skin.","prompt":"[header] How to measure for a laptop skin [title] Research the laptop skin that you want to purchase from the many that are available online. [step] You will want to investigate several websites before making a final choice because of the number of options that are out there. [title] Choose a website that will best suite your design needs.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6033,"error":null,"output":"B"}
{"question_id":"hellaswag:00006086","item_index":189,"attempt":0,"prompt_hash":"76167e3f10541df093a1f160e796e6c1d9a2ebe3512eecbf0f1289bdbe45f1f2","question":"[header] How to apply creme eyeshadow [title] Choose an eyeshadow. [step] Crème eyeshadow can be difficult to apply. While you're getting the hang of it, stick to neutral or nude shades, so you can easily blend in mistakes.\n\nA. Avoid waterproof varieties until you've had some practice. [title] Check the eyeshadow for freshness.\nB. Avoid bold shades, though, and stick to mainly neutral colors. Your eyeshadow is the make up that you put on your face.\nC. [substeps] Try teaspoons or espresso colors if you want a natural look. [title] Blend concealer in the corner of your eye with the liner.\nD. Some eyeshadow colors include : [substeps] Gold, nude, deep red, purple, or reddish. [title] Prime your face and set it with some primer.","prompt":"[header] How to apply creme eyeshadow [title] Choose an eyeshadow. [step] Crème eyeshadow can be difficult to apply. While you're getting the hang of it, stick to neutral or nude shades, so you can easily blend in mistakes.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6034,"error":null,"output":"A"}
{"question_id":"hellaswag:00000265","item_index":190,"attempt":0,"prompt_hash":"55466f143fdb591c46537b5b4a3e908d60262a2dc719b7693a4ff82fd9a681eb","question":"The violinist dances as she plays the instrument along with five other dancers. They dance in a modern and hip hop style of dancing. then only two of the dancers\n\nA. talk and one dancer lightly dances.\nB. stay on the stage with the violinist and dance in a salsa style.\nC. join her as she continues playing to both the camera and the audience.\nD. are left to dance and sit together while they cover themselves with their capes and join in the dj's rendition of the fountain of youth.","prompt":"The violinist dances as she plays the instrument along with five other dancers. They dance in a modern and hip hop style of dancing. then only two of the dancers","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6033,"error":null,"output":"B"}
{"question_id":"hellaswag:00000322","item_index":191,"attempt":0,"prompt_hash":"85bd597514be3573c16aa93cf3bfb7191929bafb0ce0b29ae23fb0659df52c07","question":"A man walks up and picks up a weight. he\n\nA. lifts it up to his shoulders and then over his head.\nB. lifts it over his head and drops it onto the sand.\nC. lifts it over his head and drops it on the floor.\nD. weighs and puts it back down.","prompt":"A man walks up and picks up a weight. he","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6032,"error":null,"output":"C"}
{"question_id":"hellaswag:00005200","item_index":192,"attempt":0,"prompt_hash":"bc1b345746b1027c507d7ea260748d07accd792364b70619c059006fc99f21ef","question":"[header] How to preserve baby food [title] Know what foods will meet your baby's nutritional needs. [step] Babies need different foods at different points in their development. The younger they are, the more limited the diet.\n\nA. To protect your baby's vital nutrients, read and follow the nutritional portion labels on foods designed specifically for babies. Try to cut out any food that contains unhealthy ingredients or which is very high in sugar.\nB. Young babies may be required to eat more whole grains in order to avoid possible hunger potential. [substeps] Make sure to portion out their food at various points in their development to ensure proper portion control for their body.\nC. [substeps] Eating the same source of food at each meal, regardless of how often, will keep your baby from overeating. A basic baby food contains egg for diet, fruit, vegetables and a little broth for soups, pastas, salads, and stews.\nD. When you first begin giving them solid foods, it should be iron-fortified infant cereal and should be purchased because the homemade version lacks the iron your baby needs. Once the baby begins accepting the cereal, you can introduce more foods.","prompt":"[header] How to preserve baby food [title] Know what foods will meet your baby's nutritional needs. [step] Babies need different foods at different points in their development. The younger they are, the more limited the diet.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6378,"error":null,"output":"D"}
{"question_id":"hellaswag:00002135","item_index":193,"attempt":0,"prompt_hash":"8203255731dc8cd9b5791ea8562f211bc2f7ef44dfb119ae668ef975251b49ba","question":"A man seated in front of a tennis net turns around and faces his opponent, with a coach and camera man looking on. as they\n\nA. are watching, the man doubles over in pain.\nB. remain facing each other, the other begins to dribble while the man in front gets fet by the ball and the man behind him helps the man across the court.\nC. both rise we see that they are actually heading to a table tennis game, the table on an actual tennis court.\nD. continue to play, the goalie chops the goalie's head, then the man ends their tennis match and falls to the ground on top of his friend who cheers loudly.","prompt":"A man seated in front of a tennis net turns around and faces his opponent, with a coach and camera man looking on. as they","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6385,"error":null,"output":"C"}
{"question_id":"hellaswag:00000690","item_index":194,"attempt":0,"prompt_hash":"43f4f494bfa15e13ed91320401f97282e52335535b5267e15e482fe9704aff7a","question":"The first jump is poor, and he knocks off the bar during the jump. He clears the bar on the second jump. the third jump\n\nA. is high enough and he sits on the bar.\nB. is extremely easy, but does not last.\nC. is the most impressive, with the athlete clearing the bar again.\nD. is better, he falls off and almost regains his balance.","prompt":"The first jump is poor, and he knocks off the bar during the jump. He clears the bar on the second jump. the third jump","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6385,"error":null,"output":"C"}
{"question_id":"hellaswag:00005335","item_index":195,"attempt":0,"prompt_hash":"3eab0206c57091499bcfcaa158f03da8a3d22948d444e69f923b41384dc66207","question":"[header] How to contact kayak customer service [title] Call customer service directly at 1-978-451-0771 or 1-855-529-2501. [step] Depending on where you are located, and your phone plan, you may be charged additional fees for the call. [title] Speak to a customer service representative.\n\nA. [step] If this person works at another company, they may be able to identify you on a personal level. Simply use their id number to report the contact to kayak.\nB. [step] You'll need to speak to them directly or on the phone. [substeps] If you use the phone system for a landline, have the representative direct you to the area code where kayak customer service sells their products.\nC. [step] Inform them that you plan to call kayak customer service. Remember to start from scratch and be clear about your expectations to reach kayak customer service.\nD. [step] You may need to wait on the line to speak to a representative about your concern or complaint. Use the wait time to prepare your feedback so you are ready when the representative comes on the line.","prompt":"[header] How to contact kayak customer service [title] Call customer service directly at 1-978-451-0771 or 1-855-529-2501. [step] Depending on where you are located, and your phone plan, you may be charged additional fees for the call. [title] Speak to a customer service representative.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6377,"error":null,"output":"D"}
{"question_id":"hellaswag:00006064","item_index":196,"attempt":0,"prompt_hash":"75bc7d477111ce43cb9fdc3a9bfabbd725396c3d10c69b4c781d3d72c8c70ce4","question":"[header] How to break in patent leather shoes [title] Buy shoes that fit well. [step] Minimize the need for stretching or breaking in by getting patent leather shoes that are already a good fit for you. Since patent leather is stiff and unforgiving, you're not going to be able to change the fit very much-maybe a quarter of a shoe size, at most.\n\nA. So, if you wear vans or hiking boots/flats everyday-you don't have to get in a calf-length patent leather pair. [substeps] Aim for a shoe that can comfortably sit any volume since highly laminate patent leather is durable.\nB. Have yourself fitted for shoes in the store, and try on a few pairs before you buy. Look out for the following when you try on patent leather shoes : [substeps] Make sure that your toes do not feel squeezed or crushed.\nC. Thus, you want to make sure your shoes aren't comfortable when you go for regular walking or running shoes. Look for shoes with cuffs that are at least one-quarter a bit higher than your foot.\nD. [substeps] Don't wear cowboy-style shoes or high-heeled sandals. You'll weaken the patent leather and cause it to crack.","prompt":"[header] How to break in patent leather shoes [title] Buy shoes that fit well. [step] Minimize the need for stretching or breaking in by getting patent leather shoes that are already a good fit for you. Since patent leather is stiff and unforgiving, you're not going to be able to change the fit very much-maybe a quarter of a shoe size, at most.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6378,"error":null,"output":"B"}
{"question_id":"hellaswag:00001772","item_index":197,"attempt":0,"prompt_hash":"91f304593223068bed1b27f0d5faec02ca082740a8054990a8e4712a45e83213","question":"First the boy pumps something onto his hand from a bottle and he rubs it on his face. then he\n\nA. places items into a bucket, adds soap and scrubs it on the boy's face.\nB. separates his helmet's parts and uses it to adjust some switches and that's how he is afterwards.\nC. washes it off using the water from the bathroom faucet and he looks in the mirror to know how much of it is gone.\nD. sprays the areas on his face with a cleaner and then rubs it on the kids face.","prompt":"First the boy pumps something onto his hand from a bottle and he rubs it on his face. then he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6384,"error":null,"output":"C"}
{"question_id":"hellaswag:00003661","item_index":198,"attempt":0,"prompt_hash":"10eb202958da7acd2446b2adc9aa89d669f9ed0ec32508753818d59c37d0e0fe","question":"[header] How to blow a fuse [title] Obtain a fuse. [step] There are two main types of fuses: cylindrical and plug fuses. Cylindrical fuses consist of a ribbon of fusible metal enclosed in a ceramic or fiber cylinder.\n\nA. This type of fuse is usually used in appliances. Plug fuses are typically used to protect in home electrical wiring and consist of a metal strip through which current must flow.\nB. They come in a variety of colors and patterns. The cylindrical cylinder is the most common type of fuse to use.\nC. The cylinder is comprised of three components: a base and a cover. Plug in the plug-in fuses are one of the more expensive fuses.\nD. Stem fuses consist of fercoated or otherwise coated metal that prevents air from reaching and disperses gas through its air cells. Stick fuses essentially allow gas to enter an existing coil to be expelled and dispose of as needed before firing a new coil.","prompt":"[header] How to blow a fuse [title] Obtain a fuse. [step] There are two main types of fuses: cylindrical and plug fuses. Cylindrical fuses consist of a ribbon of fusible metal enclosed in a ceramic or fiber cylinder.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6376,"error":null,"output":"A"}
{"question_id":"hellaswag:00000058","item_index":199,"attempt":0,"prompt_hash":"7a9b3a8a517361d168fcd5f51b487bfc4c47f8988ca72cb755e7133f9b50aef7","question":"A gymnast put power chalk power on his hands and stands near the pommel horse. Then, the gymnast approach the pommel horse to spin around many times. then\n\nA. , the gymnast stand on his hand and jumps to land on the floor, while people applaud.\nB. , the gymnast put the star light on his hands and jumps high using strength.\nC. , the gymnast puts fluid on the pommel horse and sit on it to rise the body while position it.\nD. , the gymnast cross the pommel horse with the long rope while performing handsprings.","prompt":"A gymnast put power chalk power on his hands and stands near the pommel horse. Then, the gymnast approach the pommel horse to spin around many times. then","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6379,"error":null,"output":"A"}
{"question_id":"hellaswag:00003193","item_index":200,"attempt":0,"prompt_hash":"bebd22286170523e281ea5e6cde1f4ffa95ecaa37c986cd04f7d942bcaba46a3","question":"Man is doing snow fishing in a hole pulling a thread and holding a fish. Man is holding the fish by the mouth. a small piece of wood\n\nA. is on the floor of a hole doing holes with art.\nB. is involved in the spinning of the rod and a stick hold the fish with the fishing hook and wood catch it.\nC. cig is staring the man in front of him.\nD. is on hole pulling the thread and a man keeps holding it with his hands.","prompt":"Man is doing snow fishing in a hole pulling a thread and holding a fish. Man is holding the fish by the mouth. a small piece of wood","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6382,"error":null,"output":"D"}
{"question_id":"hellaswag:00004881","item_index":201,"attempt":0,"prompt_hash":"28d58f4ba5a92a6d041ce884cc2f039ad3a15b4d4183a403f3692e895ec0b378","question":"[header] How to nudge in pro tools [title] Set a nudge value. [step] Setting up nudge values will allow you to determine the value of each nudge increment. [substeps] Open the nudge value menu.\n\nA. Select nudge values from the menu that appears. Make sure that the square icon next to each nudge value stands out.\nB. At the top of the screen, you will find various achievements that will give you nudge values. Each nudge trend will be identified by a column.\nC. In your pro tools session, locate the nudge value menu by locating the toolbar on the right side of the main counter and below the grid value menu. Click on the downward arrow to open the nudge value menu.\nD. Click the \" share with default \" or \" manage default \" options, depending on what you want to share. Click the checkboxes beneath each nudge increment type, among other things.","prompt":"[header] How to nudge in pro tools [title] Set a nudge value. [step] Setting up nudge values will allow you to determine the value of each nudge increment. [substeps] Open the nudge value menu.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6382,"error":null,"output":"C"}
{"question_id":"hellaswag:00009014","item_index":202,"attempt":0,"prompt_hash":"14d1577f1a629ba8c2474fc506abb4bfce90cc8a929ada4e696ad97a2d373104","question":"[header] How to make coffee with a coffee press [title] Measure the coffee beans. [step] To ensure that you wind up with the right amount of grounds for your coffee, you must carefully measure out the coffee beans. For 32 ounces (946 ml) of coffee, which is approximately 3 to 4 servings, use a measuring cup to measure a ½ cup (51 g) of beans.\n\nA. [substeps] The standard size for most coffee presses is 32 ounces (946 ml), but you can make fewer servings if you prefer. For a single serving, measure 2 tablespoons (13 g) of beans.\nB. For 5 ounces (165 ml) of black coffee, that also includes 5 servings. [substeps] Double-check that the coffee beans you use are ok to drink in the coffee press.\nC. To measure two or four servings, use a measuring cup with a button. You should use only a pinch of sugar if your coffee contains sweet ingredients, and a pinch of salt if your coffee contains milk.\nD. [substeps] If you don't have a measuring cup, measure out 32 oz (119 ml) of coffee beans and put them into a measuring cup. Transfer that measurement to a scale or a disposable measuring cup.","prompt":"[header] How to make coffee with a coffee press [title] Measure the coffee beans. [step] To ensure that you wind up with the right amount of grounds for your coffee, you must carefully measure out the coffee beans. For 32 ounces (946 ml) of coffee, which is approximately 3 to 4 servings, use a measuring cup to measure a ½ cup (51 g) of beans.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6379,"error":null,"output":"A"}
{"question_id":"hellaswag:00009330","item_index":203,"attempt":0,"prompt_hash":"b502be27bcfcec2ae3ea637dcaed6db2114dd4bab41a7427256afd0fbff7e75d","question":"[header] How to make an emergency flash diffuser [title] Realize when you need to use a diffuser. [step] For point-and-click photographers, it is helpful to understand there are times in photography when a diffuser for your flash can make all the difference between a good photo and a poor one. Diffusers help to create more natural and softer lighting.\n\nA. [title] Spend some time thinking about what you're going to put in your flash. [step] That means: photograph...photograph...focus...use...focus on the thing you want, touch, and think about that.\nB. [title] Begin by lighting each flash in a new location. [step] This will make the colors coordinate somewhat, and also help to create a gradient look.\nC. In particular, a diffuser can help : [substeps] Taking a photo with white components. Without a diffuser, you are liable to get blown out white.\nD. Your old flash can effectively slip under the lens of the camera. If you can see the straight body of the diffuser on the camera screen, avoid letting it get down.","prompt":"[header] How to make an emergency flash diffuser [title] Realize when you need to use a diffuser. [step] For point-and-click photographers, it is helpful to understand there are times in photography when a diffuser for your flash can make all the difference between a good photo and a poor one. Diffusers help to create more natural and softer lighting.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6381,"error":null,"output":"C"}
{"question_id":"hellaswag:00001153","item_index":204,"attempt":0,"prompt_hash":"6aeffc76fd579e32b0899826924ee34436a441c5070b1ef3e26b7b5c95fffd9b","question":"A woman is talking in her kitchen. she\n\nA. cooks chocolate chip cookies and mixes them in some plastic wrap.\nB. shows how to cut and assemble multiple liquors into a glass.\nC. is in the process of making a dessert.\nD. starts drying the dishes in and out of the dishwasher.","prompt":"A woman is talking in her kitchen. she","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6380,"error":null,"output":"C"}
{"question_id":"hellaswag:00006586","item_index":205,"attempt":0,"prompt_hash":"07c02ee14e8c69efbf2ef339db50676853df82b50bb145ffb5f38efe746672d8","question":"[header] How to get to montreal, canada [title] Look for an airline with regular flights to montreal. [step] Pierre-elliott trudeau international airport, located just southwest of the city, is the largest commercial airport in the area and the main thoroughfare for air travelers. Many established airlines fly here, such as air canada or united, but you may find better deals on smaller low-cost airlines.\n\nA. [substeps] Because of their lack of miles, flying is much more affordable. When buses travel from canada to canada, bus stops get shorter and longer.\nB. [substeps] If you're about to fly to montreal from canada, look for a major outdoor air travel agency (bus, train). These agencies charge $20 per stop, so ensure you're mobile enough to get there.\nC. [title] Decide if you want a direct (nonstop) flight or one with multiple stops. [step] The latter option is usually cheaper, but far more inconvenient.\nD. [title] Fly for more than 60 or 120 minutes per day. [step] If you've never been to montreal and you are not from san diego, you may have to stay overnight.","prompt":"[header] How to get to montreal, canada [title] Look for an airline with regular flights to montreal. [step] Pierre-elliott trudeau international airport, located just southwest of the city, is the largest commercial airport in the area and the main thoroughfare for air travelers. Many established airlines fly here, such as air canada or united, but you may find better deals on smaller low-cost airlines.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6376,"error":null,"output":"C"}
{"question_id":"hellaswag:00008032","item_index":206,"attempt":0,"prompt_hash":"42e0f5c613a2fc770e9c9f5d39bf6bf55eafc2ea24d9b1c5619cb304c7dfde3d","question":"[header] How to choose and use facial wipes [title] Check the ingredients in face wipes. [step] Facial wipes can be filled with chemicals and preservatives that can be harmful to your skin. When shopping for facial wipes, read through the ingredients and select wipes that do not contain formaldehyde-releasing chemicals.\n\nA. These chemicals, such as 2-bromo-2-nitropropane-1 , 3-diol, are used to increase the shelf life of the face wipes. [substeps] Although these chemicals reduce the chance of bacteria build up, they can be damaging to the skin.\nB. [substeps] Professional facial wipes are usually free of these toxins. If you are unable to purchase facial wipes online, check with salicycare during these commercial sales days for this option.\nC. However, do not forget to check the ingredient list on the packaging for combination ingredients. Most store bought cleansers will contain a combination of these ingredients : [substeps] Palm oil face wash astringents massage oil [title] Look for an exfoliant with low-concentration ingredients.\nD. Look at what ingredients are on the bottles and in the packaging before purchasing them. [title] Familiarize yourself with the wipes' packaging.","prompt":"[header] How to choose and use facial wipes [title] Check the ingredients in face wipes. [step] Facial wipes can be filled with chemicals and preservatives that can be harmful to your skin. When shopping for facial wipes, read through the ingredients and select wipes that do not contain formaldehyde-releasing chemicals.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6379,"error":null,"output":"A"}
{"question_id":"hellaswag:00003959","item_index":207,"attempt":0,"prompt_hash":"391ff482d1e1d7bf8ed7442a918abaa5fe63d0a07fb3fa3ab0188cdb03f641dc","question":"[header] How to skin a squirrel [title] Get the squirrel wet. [step] To keep unwanted hair from getting all over your meat while you are skinning a squirrel, place the squirrel in the water at the edge of a stream or in a bucket. Lay the squirrel belly-down on a flat surface.\n\nA. [substeps] You can also douse the animal with a hose prior to continuing if you are back at home. This step is optional and dependent on a water source; however, it is a handy way to reduce the spread of hair onto the meat.\nB. If the squirrel is left outdoors your hands will rub against the water. As the squirrel ages, he will begin to form a bristle pelt.\nC. Pour water into a bucket and swish it around to saturate the squirrel. Once you have soaked the squirrel's belly, the water will flow into the bucket and spread around the squirrel.\nD. Tip the squirrel's head backwards and place the wet spot directly over its tailbone. Use a stick to gently push back the squirrel's tail which cuts off the head and separates the back and upper body.","prompt":"[header] How to skin a squirrel [title] Get the squirrel wet. [step] To keep unwanted hair from getting all over your meat while you are skinning a squirrel, place the squirrel in the water at the edge of a stream or in a bucket. Lay the squirrel belly-down on a flat surface.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6378,"error":null,"output":"C"}
{"question_id":"hellaswag:00005500","item_index":208,"attempt":0,"prompt_hash":"7ffbc131293d903f1acb183c9a0509bb85d4b747a5fe411b9413fbf9074746dd","question":"[header] How to make a clown cake [title] Cut the small sponge cake into a triangular shape. [title] Curve the bottom into a small arch instead of a straight line. [step] Use white and pink marshmallows around the cake's bottom for the ruff (or collar).\n\nA. Cut a slit in the side of the flattened form of the cake to make the collar. For the wider, bigger cuff, use jumbo marshmallows.\nB. [title] Frost the large sponge cake with the buttercream frosting. [step] Use a smooth, palette knife to give it a solid stroke all around.\nC. Use red or purple marshmallows to make duckducklings if you'd like a more realistic depiction of clowns. [title] Make straw shapes for the fake clowns at the bottom.\nD. [title] Set your eyelids together, and roll your top and bottom upwards, creating a spear (like the top and tail picture above). [title] Customize your construction by using red marshmallows for the scales, and red and pink marshmallows for the heart's ears.","prompt":"[header] How to make a clown cake [title] Cut the small sponge cake into a triangular shape. [title] Curve the bottom into a small arch instead of a straight line. [step] Use white and pink marshmallows around the cake's bottom for the ruff (or collar).","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6377,"error":null,"output":"B"}
{"question_id":"hellaswag:00002553","item_index":209,"attempt":0,"prompt_hash":"3ac65242180238b5bebad44da292a8274702b512dc809fafb2f8cc166b094614","question":"A video of bumper pool is shown. People are interviewed and then the actual pool is shown. the teams\n\nA. play against each other and then a champion is crowned.\nB. are all lined up in several locations and gather.\nC. gather and play pool while people are gather around the pool.\nD. get mixed in and interviewed and all filmed together.","prompt":"A video of bumper pool is shown. People are interviewed and then the actual pool is shown. the teams","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6377,"error":null,"output":"A"}
{"question_id":"hellaswag:00007140","item_index":210,"attempt":0,"prompt_hash":"353c2a468bb0478a7a8d5211663e4363b98d4805d6679ea33e45a815e06bf55b","question":"[header] How to apply for an american express credit card [title] Determine your credit needs. [step] The first thing to do before applying for an amex card is figure out what you want out of your card. Are you interested in accumulating rewards, or is a card with minimal fees your top priority? What do you plan to use the card for? American express offers numerous credit solutions which each have their own benefits and drawbacks.\n\nA. You'll need to know what you expect to get in return to satisfy each credit company. [substeps] Consider where you live and if you have a credit account.\nB. [substeps] Ask yourself what card/account characteristics are important to you and compare your list of needs and wants to available amex options. This will help you identify the card that is best for you.\nC. You can develop your own plan by first narrowing down your list of offers. [title] Determine what type of card you will need.\nD. Finally, as your waiting range grows, the card may become worthless. When looking at a amex card, think about the positive things you want out of it.","prompt":"[header] How to apply for an american express credit card [title] Determine your credit needs. [step] The first thing to do before applying for an amex card is figure out what you want out of your card. Are you interested in accumulating rewards, or is a card with minimal fees your top priority? What do you plan to use the card for? American express offers numerous credit solutions which each have their own benefits and drawbacks.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6379,"error":null,"output":"B"}
{"question_id":"hellaswag:00008779","item_index":211,"attempt":0,"prompt_hash":"d2a345b0b29c7a8ef516392b4ea4396e68a36076b073968454a12f3faaf29b26","question":"[header] How to plan for retirement in your 30s [title] Discuss retirement with your partner, if you have one. [step] Work with your partner to set retirement age goals, estimate retirement expense and income, and set savings goals. Factor in their retirement accounts, if they have individual savings, and social security earnings.\n\nA. [title] Evaluate if you'd like to finance an 401 (k) plan. [step] If you have any life goals beyond retirement or if you've already committed to retirement, you may want to consider a plan.\nB. Since all of you are getting into retirement, separate your retirement income from your social security account to help figure out what's in each account and where you will be spending money per year. [title] Determine whether or not to raise money by yourself.\nC. To make retirement requirements as easy as possible for both of you, you should communicate your financial situation with your partner and say the following : [substeps] \" i know you're under a bit of uncertainty about your retirement age. It doesn't hurt to set an additional financial goal for either of us to discuss.\nD. Talk about how you each foresee living in retirement and about your expectations for your living standards. [substeps] For example, discuss whether you intend to save money in retirement by moving into a smaller home or condo in a location with a lower cost of living.","prompt":"[header] How to plan for retirement in your 30s [title] Discuss retirement with your partner, if you have one. [step] Work with your partner to set retirement age goals, estimate retirement expense and income, and set savings goals. Factor in their retirement accounts, if they have individual savings, and social security earnings.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6379,"error":null,"output":"D"}
{"question_id":"hellaswag:00003138","item_index":212,"attempt":0,"prompt_hash":"3871e5c459928d5ef3fa37f0e21347abb75db6e24c8deb46e4f8eac8e5d37cfc","question":"A young man is seen sitting on a park bench playing a harmonica and tapping his foot. The man moves his foot up and down and pauses for a minutes before continuing to play. the man\n\nA. sings a bit and still plays the harmonica.\nB. moves his feet up and down and finishes his harmonica.\nC. continues playing his harmonica and ends by wrapping his hands around his mouth and appearing to moan loudly.\nD. continues playing all around the park acting as if he's a ghost and ends by walking away.","prompt":"A young man is seen sitting on a park bench playing a harmonica and tapping his foot. The man moves his foot up and down and pauses for a minutes before continuing to play. the man","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6379,"error":null,"output":"C"}
{"question_id":"hellaswag:00001763","item_index":213,"attempt":0,"prompt_hash":"35a721586bc866b1690a2833d07e2b3b6271f167025a3d9bf05c59892d57b89e","question":"Two women light each others cigarettes. They sit at a table and smoke their cigarettes. they\n\nA. take the cigarettes out and smoke.\nB. blow smoke rings into the air in front of them.\nC. dance and pose together.\nD. finish talking and pose.","prompt":"Two women light each others cigarettes. They sit at a table and smoke their cigarettes. they","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6431,"error":null,"output":"B"}
{"question_id":"hellaswag:00002285","item_index":214,"attempt":0,"prompt_hash":"974c1f9ec549c4dc64849bae76bf5118402fa07647be79395cb4d329a0d215ee","question":"A man is sitting on a frozen lake, fishing through a hole and surrounded by fishing gear. He walks away from his stuff, and a fish pulls his pole into the water, disappearing with it. he\n\nA. then stands on the ice fishing in the hole.\nB. is now fishing through a hole in the ice.\nC. stands angrily with his hands on his hips, then gets another pole to try to get it back.\nD. reels a fish in and shows how he pulled while reeling in the fish, then releases the fish and shows his accomplishment.","prompt":"A man is sitting on a frozen lake, fishing through a hole and surrounded by fishing gear. He walks away from his stuff, and a fish pulls his pole into the water, disappearing with it. he","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6431,"error":null,"output":"B"}
{"question_id":"hellaswag:00008159","item_index":215,"attempt":0,"prompt_hash":"0f22fd4284a652127c98e60e1d77fe6159510ee7dd1890e2c39287ffe0f6826b","question":"[header] How to schedule your study hours in distance learning [title] Determine what you need to get done. [step] Before scheduling study time, it is important that you are aware of all the requirements of your various classes. Use your course calendars (usually provided in some form with the syllabus) to figure out what assignments are due each week.\n\nA. Then, talk to your professor about whether she needs some time and is open to scheduling. [substeps] Pick out an area for your study time in which to work.\nB. [substeps] You can include assignments on the syllabus as you communicate to the teacher. Use these as guides when you explore the syllabus so you can make sure you know what is due to work.\nC. You may choose to choose assignments first or end them at due dates. You may also choose to try to create a schedule in advance that stretches within the scope of your class.\nD. Don't forget to include tests and exams! [substeps] It may help to write everything down and group it by day or week. This way you won't have to flip between multiple class syllabi in order to see what is due on any given day.","prompt":"[header] How to schedule your study hours in distance learning [title] Determine what you need to get done. [step] Before scheduling study time, it is important that you are aware of all the requirements of your various classes. Use your course calendars (usually provided in some form with the syllabus) to figure out what assignments are due each week.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6431,"error":null,"output":"D"}
{"question_id":"hellaswag:00001672","item_index":216,"attempt":0,"prompt_hash":"9be70c4cf59c5c6d608c60faa01902d3453481397bd88fa31cdd064fef99f438","question":"A man is standing with a fishing pole. He is fishing through a hole in the ice. he\n\nA. pulls his fishing line out of the hole and into the water.\nB. lines up a small white fish, grabbed some black and white arrowheads.\nC. reels in a large fish that he caught.\nD. chews on an ice floe as he speaks to the camera.","prompt":"A man is standing with a fishing pole. He is fishing through a hole in the ice. he","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6431,"error":null,"output":"A"}
{"question_id":"hellaswag:00006215","item_index":217,"attempt":0,"prompt_hash":"e97e940c17b72ea729e05e51fae428116a670ebc293f55978ad53d98287c162b","question":"[header] How to make a fabric belt [title] Take your waist measurement. [step] If you don't already know your waist measurement (for instance, from shopping for pants), it's very easy to find. Simply grab a tape measure and loop it around your midsection at your natural waist, which is usually at the top of your hips, just below the level of your belly button.\n\nA. [substeps] Even a little bit of extra room will allow you to write your sizing information onto the fabric. For instance, if your measurements appear to be below your navel, then your waist measurement will be somewhere between half and half.\nB. Check the measurement where the tape starts to double up on itself-this is your waist circumference. [substeps] Some women's belts are meant to be worn around the hips, rather than the waist.\nC. [substeps] Avoid putting too much pressure on your abdominal muscles. Opt for a tape measure that is specially designed for measuring waist.\nD. [title] Make a small hole around the top of your waist with the measuring tape. [step] This hole should be about one-third the size of the measurement on your belt's inside edge.","prompt":"[header] How to make a fabric belt [title] Take your waist measurement. [step] If you don't already know your waist measurement (for instance, from shopping for pants), it's very easy to find. Simply grab a tape measure and loop it around your midsection at your natural waist, which is usually at the top of your hips, just below the level of your belly button.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6431,"error":null,"output":"B"}
{"question_id":"hellaswag:00000795","item_index":218,"attempt":0,"prompt_hash":"c24b264f674a82c3b93b0f2d3b520904454b3357c5e73bdf92bd0b33ff099d95","question":"A toddler wash his hands in the sink. Then, the toddler splash water on his face and smile. the toddled\n\nA. puts water on his face, and then moves his head.\nB. then grab the ladder of the sink.\nC. while washing his hands.\nD. to the mirror and record the germs on his face.","prompt":"A toddler wash his hands in the sink. Then, the toddler splash water on his face and smile. the toddled","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6430,"error":null,"output":"C"}
{"question_id":"hellaswag:00001427","item_index":219,"attempt":0,"prompt_hash":"0d5152a5bc1ff1ed14e83f8a5b6076f9007a1acaab6c73664cb346590d034eb8","question":"The woman jumps again with an even longer run up and a more powerful skip/hop to start her run. the woman\n\nA. continues her long run and throws her arms wide and spins in the air and makes happy little faces.\nB. jumps again with the long run up and powerful skip at the beginning.\nC. runs vertically around the stadium while a host of people watch on the sides.\nD. slows down and walks up to different steps.","prompt":"The woman jumps again with an even longer run up and a more powerful skip/hop to start her run. the woman","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6411,"error":null,"output":"B"}
{"question_id":"hellaswag:00005991","item_index":220,"attempt":0,"prompt_hash":"5ae299d95d44cb64be2c77fc57b39b8e325ad0a0275ff22512561611a18fabe2","question":"[header] How to make your own unique jeans [title] Find a black or brown pair of jeans. [step] Everyone else seems to be wearing blue. [title] Design your jeans with a single theme, color, medium and technique.\n\nA. [title] You want patterns that are laid back and flatten your jeans. [title] When you are purchasing cute jeans pick ones with patterns that make them look comfortable and natural.\nB. [step] Once you have the look, you'll need the best results to really make them unique. Print your designs and stick them on a photo card or record them with your phone or a digital camera.\nC. [step] Having a theme will make your style unique! [title] Paint your jeans with an interesting design. [step] If you can't think of anything that would draw attention, such as wolves, a christmas tree, a snowman, star, goose, a reindeer, frogie, etc.\nD. [substeps] Pick a theme of interest, such as music, flowers, hearts, stars or symbols. Choose a single color, such as red, blue or purple.","prompt":"[header] How to make your own unique jeans [title] Find a black or brown pair of jeans. [step] Everyone else seems to be wearing blue. [title] Design your jeans with a single theme, color, medium and technique.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6412,"error":null,"output":"D"}
{"question_id":"hellaswag:00004043","item_index":221,"attempt":0,"prompt_hash":"7958e125c831c43bfb19a5cc2eed99354fcaee3395ba7dcb2a7bb949927e4e66","question":"[header] How to plan a bachelor party [title] Pick a date. [step] Choose a date for the party that works for the groom, groomsmen and yourself. You should choose a date in the two weeks before the wedding if possible-choosing a friday or saturday will make the bachelor party easier for anyone who has to work on weekdays.\n\nA. And if you don't have to worry about actually attending the party, consider putting some effort into planning the party. [substeps] Leave a note on the table! Put it on the ring box on your invitation.\nB. [substeps] Keep in mind locations for the party and how busy they'll be at the time of year you're going to visit. [title] Talk to the groom about what he wants.\nC. Don't feel as if you have to reserve a party for the groom personally. The groom should feel like you don't need to go to the party with him.\nD. You'll probably want to set a date around the honeymoon, when the groom will be all over the bride. [substeps] Try to arrive at the location around 75 minutes before the wedding-this gives you time to get settled and prepare to watch bridesmaids get married.","prompt":"[header] How to plan a bachelor party [title] Pick a date. [step] Choose a date for the party that works for the groom, groomsmen and yourself. You should choose a date in the two weeks before the wedding if possible-choosing a friday or saturday will make the bachelor party easier for anyone who has to work on weekdays.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6412,"error":null,"output":"B"}
{"question_id":"hellaswag:00009123","item_index":222,"attempt":0,"prompt_hash":"55a352a893e6271bc1919605ee40741736f4bdccc80048f8b3bc05a6f27f63ae","question":"[header] How to make sea salt [title] Understand the process. [step] Commercial salt makers produce sea salt in much larger scale than a home salt maker would, but knowing the commercial techniques can increase your knowledge and salt making possibilities. Here's how they do it : [substeps] Small ponds are filled with sea water and the water is left to evaporate.\n\nA. This increases the amount of salt used, even though the water themselves are largely salt free. Occasionally the water may overflow and cause a disaster.\nB. The water is dissolved and the salt floats to the surface. This is known as a sea salt maker.\nC. The product left behind after all the water is gone is sea salt. This process works best in areas where there is a lot of sun and very little rain.\nD. Fill up the shallower ponds with lye, which has been used in the field for hundreds of years. Salt water will rise slowly, then rise back up, adding to the brine water that the shallower ponds contain.","prompt":"[header] How to make sea salt [title] Understand the process. [step] Commercial salt makers produce sea salt in much larger scale than a home salt maker would, but knowing the commercial techniques can increase your knowledge and salt making possibilities. Here's how they do it : [substeps] Small ponds are filled with sea water and the water is left to evaporate.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6411,"error":null,"output":"C"}
{"question_id":"hellaswag:00002962","item_index":223,"attempt":0,"prompt_hash":"6eab1b1299d0db269fe7e725afc12f985f889b24ea5f5a82d794fa66557e737c","question":"A man in a red polo shirt demonstrates the kleva sharp sharpener. he\n\nA. demonstrates how to use the fish knife for a smooth zapping.\nB. demonstrates the baton hitting the ball which hits the man in the left eye and knocks him out.\nC. sharpens a dull butcher knife and cuts a tomato.\nD. demonstrates the knife sharpener she will use while talking.","prompt":"A man in a red polo shirt demonstrates the kleva sharp sharpener. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6410,"error":null,"output":"C"}
{"question_id":"hellaswag:00006186","item_index":224,"attempt":0,"prompt_hash":"7be2a0083b370eae180d3289befc3db1cb924ab73af6397136c3fabe1b6a63fa","question":"[header] How to relax all day (for women and girls ) [title] Gather books, magazines, mp3 players, art supplies, and nail polish. [step] Gather them legally. Do not steal these items.\n\nA. Makeup is also optional, but it's always good to have some free. But there's one thing you can do wrong.\nB. Look around your home or buy new, it doesn't matter where from or how you gather these items the important thing to aid you to relax all day is that you gather them. Get as many of each as you think you'll need, and then a little more, just in case.\nC. The law prohibits this from being permitted in the workplace. [title] Put on a simple outfit that won't draw attention to yourself or cause a panic.\nD. Stay away from this place and don't steal anything from other people. This includes anything out of the ordinary or rare.","prompt":"[header] How to relax all day (for women and girls ) [title] Gather books, magazines, mp3 players, art supplies, and nail polish. [step] Gather them legally. Do not steal these items.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6520,"error":null,"output":"B"}
{"question_id":"hellaswag:00002940","item_index":225,"attempt":0,"prompt_hash":"1493f8cd45531d826c80c1a7e8a7d5c3178dbd5269d0f2f3cab27351b8acf4cf","question":"There are many people in bathing suits standing in a lake with motor boats behind them. one girl wearing a bikini and holding a coke can in her hand\n\nA. , jumps off the side of the boat onto a row of green treaded water skis.\nB. is rubbing sunscreen on another woman's chest.\nC. floats to the tree behind them and begins taking a long drink and letting the liquid down her chin.\nD. begins to partner up with another girl.","prompt":"There are many people in bathing suits standing in a lake with motor boats behind them. one girl wearing a bikini and holding a coke can in her hand","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6519,"error":null,"output":"B"}
{"question_id":"hellaswag:00007198","item_index":226,"attempt":0,"prompt_hash":"4eeb2623c8e75231ca6c4c3fd0a6eb7e1c2f9d5b63fb16c780b15036f8bc6bab","question":"[header] How to make a moisturizing deep treatment for hair [title] Mix up the ingredients in your blender. [step] Make sure the mixture is very smooth, or it will get stuck in your hair. [substeps] If you're working with any fruits (banana or avocado), lots of honey, and other natural ingredients, it's better if they're well blended.\n\nA. Cut up fruits like bananas so they blend well. If the mixture is too thick, you can always add more conditioner.\nB. You could also use soy sauce instead of milk for a subtle fruit treatment. Mix up some honey and milk together and set it aside for now.\nC. [title] Add a moisturizing hair serum. [step] You can select these over any hair products.\nD. Be sure you find enough honey in your blender to work, such as pure brown sugar, honey, or olive oil. Milk can also work, as long as there is not cream.","prompt":"[header] How to make a moisturizing deep treatment for hair [title] Mix up the ingredients in your blender. [step] Make sure the mixture is very smooth, or it will get stuck in your hair. [substeps] If you're working with any fruits (banana or avocado), lots of honey, and other natural ingredients, it's better if they're well blended.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6513,"error":null,"output":"A"}
{"question_id":"hellaswag:00000974","item_index":227,"attempt":0,"prompt_hash":"92ea2fc5cd588b42ed7147674c6497ee908496313f885a245a7bb59e587d3765","question":"A man is seen bending over a large set of weights with another walking behind him. the man\n\nA. lifts himself backwards and the camera zooms in on his back while the other follows him.\nB. repeats this actions backwards with a height difference.\nC. lifts the bar up over his head and throws it back down.\nD. hits the weights up and down on the bar while the camera captures him from several angles.","prompt":"A man is seen bending over a large set of weights with another walking behind him. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6516,"error":null,"output":"C"}
{"question_id":"hellaswag:00002914","item_index":228,"attempt":0,"prompt_hash":"f49a2d9d318b8cc07a4f90596929ddeb41302ed0b4fa2457330b155c7cea5103","question":"The man boils milk and adds fresh vanilla and pours the solution into a bowl with other ingredients. the man\n\nA. wellops up the finished skillet and puts it in the oven, cuts up the food along with the sliced fruit and uses a spoon to taste the cooked food.\nB. joins other ingredients inside a bowl and pours it around before adding shots of rum and rum and mixing it together.\nC. adds more water and squeezes it into the mug at the end.\nD. cooks the custard and pours into a bowl and freezes to make ice cream.","prompt":"The man boils milk and adds fresh vanilla and pours the solution into a bowl with other ingredients. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6514,"error":null,"output":"D"}
{"question_id":"hellaswag:00004263","item_index":229,"attempt":0,"prompt_hash":"1daf629bdf4e70651b7832e95c3a4eca5641795b286fd9d9c29d1a8d03f66e49","question":"[header] How to hire a bodyguard [title] Set a budget before you research possible bodyguards. [step] Depending on your needs, hiring a bodyguard can set you back from $200 to over $900 a day, with expenses depending on the level of potential danger and if you need an armed guard. Using your salary as a guide, decide how much you're willing to spend on a bodyguard.\n\nA. [substeps] Bodyguards from respected agencies will cost more than less reliable bodyguards. Check many different bodyguard options to make sure you're getting the best guard for your price.\nB. [substeps] That's okay to buy a bodyguard who spends $600 a month on a later salary. However, think about how you would fit into the office or on a metropolitan area for 24 hours a week.\nC. [substeps] Hiring a bodyguard can set you back $1 , 000 in the long run, but low pay will probably only attract more attention to you. Make sure to compare the potential bodyguards to determine the skills and personality of each one.\nD. [substeps] Before deciding on a bodyguard, think about the commission you're willing to put on a guard. Hiring an armed guard will cost you a small fee, but a guard should even appear and tell you what fees they charge.","prompt":"[header] How to hire a bodyguard [title] Set a budget before you research possible bodyguards. [step] Depending on your needs, hiring a bodyguard can set you back from $200 to over $900 a day, with expenses depending on the level of potential danger and if you need an armed guard. Using your salary as a guide, decide how much you're willing to spend on a bodyguard.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6509,"error":null,"output":"A"}
{"question_id":"hellaswag:00002011","item_index":230,"attempt":0,"prompt_hash":"eba428b2de2f76053408d58489088fe307a739dde4126877cd6411b579840942","question":"We see the inside of a plant, where cookies are crushed and put into containers of vanilla ice cream, then sorted. ice cream\n\nA. is added to the hot tub, along with other ingredients.\nB. is set on an ice cream cake.\nC. with vanilla is put in a large white bowl, along with wrapped cookies for weeks and steamed milk.\nD. fills the pints, and is mixed in a giant vault before other ingredients are added and the pints filled.","prompt":"We see the inside of a plant, where cookies are crushed and put into containers of vanilla ice cream, then sorted. ice cream","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6513,"error":null,"output":"A"}
{"question_id":"hellaswag:00001321","item_index":231,"attempt":0,"prompt_hash":"ed53b74a7a9a776b54c8a3173d1337cd97d0b3512c55ed6d878d01d8fbb2c69b","question":"A young team of girls is getting coached during a practice for basketball and the girls are working on their shots. A coach is standing talking to a small group of girls who are playing basket ball. the coach\n\nA. showed the team and the girls scored several baskets.\nB. gets the ball passed to her.\nC. shoots baskets at the girls in front of her and then scores scores for the team, and her teammates applaud.\nD. connects a string then cuts the ball in half in the air and give a high ten.","prompt":"A young team of girls is getting coached during a practice for basketball and the girls are working on their shots. A coach is standing talking to a small group of girls who are playing basket ball. the coach","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6512,"error":null,"output":"A"}
{"question_id":"hellaswag:00009930","item_index":232,"attempt":0,"prompt_hash":"d19fa904e998a6d253a119a7fb4b97d1708ca435190985a55f2714f519c4469c","question":"[header] How to do a 5 strand french plait [title] Detangle your hair thoroughly. [step] If your hair is prone to tangling, dampen the ends to stop knots forming while you're working. [title] Brush your hair straight back, away from your face.\n\nA. [step] Avoid pulling your hair to one side, but gently separating your hair into three halves. [title] Wash your hair, using a clarifying shampoo if your hair is long or thick.\nB. [step] Don't use your fingers since they'll twist and make it easier to adjust. If your hair is wavy, try to use a pair of wide-toothed brushes with plastic bristles.\nC. [step] Start just above your temples, and draw your thumbs horizontally through your hair until they meet at the back of your head. You should now have a reasonably sized section of hair to start working with.\nD. [step] Make sure to brush towards the crown of your head. [title] Curl your hair with a hair straightener if desired.","prompt":"[header] How to do a 5 strand french plait [title] Detangle your hair thoroughly. [step] If your hair is prone to tangling, dampen the ends to stop knots forming while you're working. [title] Brush your hair straight back, away from your face.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6509,"error":null,"output":"D"}
{"question_id":"hellaswag:00005356","item_index":233,"attempt":0,"prompt_hash":"ba4b1dd5319ff1ae548d1cd33d7c4b19aedc3730e8e8b64eba23f3488ecf7c3e","question":"[header] How to color your converse [title] Start with a clean shoe. [step] The best converse to work on are brand new. If you can't get new ones, you will need to clean the ones you already own.\n\nA. That means you need to have a pair of converse shoes, so clean the shoes thoroughly. Then you can select any of the color combinations that were added to your converse to dye them.\nB. Clean them up by washing them in a sink full of soapy water and trying to dry them. Run the shoes under the faucet.\nC. [substeps] Wash your converse with soap and water. Use cold water as if taking a walk.\nD. This will help the ink stick and show up better. Wipe down the rubber parts with a cotton ball soaked in rubbing alcohol.","prompt":"[header] How to color your converse [title] Start with a clean shoe. [step] The best converse to work on are brand new. If you can't get new ones, you will need to clean the ones you already own.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6511,"error":null,"output":"D"}
{"question_id":"hellaswag:00007276","item_index":234,"attempt":0,"prompt_hash":"f87b5807df7c81f5a05aefc025d548b15d87aeee017d272b5849eb52c3d6bb1e","question":"[header] How to care for the face in winter [title] Hydrate. [step] Healthy, moisturized, and supple facial skin is impossible during any month of the year without proper body hydration. By taking care of your skin's hydration needs throughout the year, you will enter winter with your skin at its optimal condition.\n\nA. Drink water before and during regular daytime activities, such as showering, tanning, washing your face, and applying a new layer of makeup. [substeps] If you take public transportation to school or work, drink water to stay hydrated.\nB. [substeps] Avoid drinking coffee or liquor while outdoors. These beverages will dry out your skin and further irritate the skin.\nC. [substeps] Hydration begins from the inside out. Our bodies are comprised mostly of water, and we need to take in a large quantity of water each day to replenish what we lose through sweat, waste, and energy.\nD. [substeps] Whole, supple skin can help to ensure that you create a healthy glow, which can be helped to deepen and tone your complexion. If your skin is dry, this can increase your risk of aging and negatively affect face shapes, including wrinkles.","prompt":"[header] How to care for the face in winter [title] Hydrate. [step] Healthy, moisturized, and supple facial skin is impossible during any month of the year without proper body hydration. By taking care of your skin's hydration needs throughout the year, you will enter winter with your skin at its optimal condition.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6509,"error":null,"output":"C"}
{"question_id":"hellaswag:00005473","item_index":235,"attempt":0,"prompt_hash":"93cddbdf340649fc2467de166b3641a516f9d7c86ad4c878727361b1747368b8","question":"[header] How to change a bathtub faucet [title] Turn off the water supply to the bathtub faucet. [step] This should always be the first step when taking on any plumbing project in your home. [substeps] Also have some rags and a bucket handy in case you need to clean up water quickly.\n\nA. There is no need to wring out the toilet paper, since this will leave behind a film of residue. Towels can be used instead if you notice that liquids are pooling in the tub.\nB. You might want to consider replacing the water supply with something new. It might be a good idea to take the drain line down when making small repairs.\nC. [title] Fill a bucket up with water and make sure you put a disposal container in the water. [step] Be sure to label the bucket with your name and address on the side, so you can easily know where to dispose of it afterwards.\nD. [title] Assess what type of faucet stems you have. [step] The faucet stems are the parts of your faucet that control the flow of water.","prompt":"[header] How to change a bathtub faucet [title] Turn off the water supply to the bathtub faucet. [step] This should always be the first step when taking on any plumbing project in your home. [substeps] Also have some rags and a bucket handy in case you need to clean up water quickly.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6508,"error":null,"output":"D"}
{"question_id":"hellaswag:00001947","item_index":236,"attempt":0,"prompt_hash":"9e8150783087fa08db0c0abad7d8808cbeaa5265609319059d0d789aeb2f9665","question":"The lady shows us nail clippers. The lady clips her cats claws. The lady shows us a clipped claw. we\n\nA. take the cat to the vet office to be looked at.\nB. see her says good-bye to the cat and dog.\nC. see the lady sign a daily statement.\nD. see a paw's claw up close.","prompt":"The lady shows us nail clippers. The lady clips her cats claws. The lady shows us a clipped claw. we","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6511,"error":null,"output":"A"}
{"question_id":"hellaswag:00005969","item_index":237,"attempt":0,"prompt_hash":"c2fab15cc330bf85df6ebce8e305988a4fd30a1aec7d66642de4c4a967a548e2","question":"[header] How to play bassline when playing the piano [title] Find the key and accompanying chords. [step] Determine which key the piece is in, and learn the basic chords for each key signature. In a major key, the one, four, and five chords are always major, and together they make a common chord progression.\n\nA. [substeps] Alternatively, you can find chords tuner, jokey, or syncopated. This is specific to the piece as well as to the melody.\nB. Make your way down the keyboard until you find the four chords for each key. [substeps] If your chord is in, play the note progression key, then the chords numerical row with the two ascending steps.\nC. For example, in the key of c major, you can play the c major, f major, and g major chords. [title] Play the root of each chord with your left hand.\nD. In a minor key, there's a four-chord pattern that you can vary. [substeps] Note that if you are playing the keyboard at all, you may notice that you shuffle each chord in the same order.","prompt":"[header] How to play bassline when playing the piano [title] Find the key and accompanying chords. [step] Determine which key the piece is in, and learn the basic chords for each key signature. In a major key, the one, four, and five chords are always major, and together they make a common chord progression.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6507,"error":null,"output":"C"}
{"question_id":"hellaswag:00004463","item_index":238,"attempt":0,"prompt_hash":"59897aa809332ec34979b4391ee2783c8cc2b67fb33d436a59611e225bee84bd","question":"[header] How to write a marketing report [title] Consider why you should perform market research and write a report. [step] What information is important to you? What will you do with the marketing report after it is created? This process requires an investment of time and expense. Make sure that you have a clear plan for using the information that you collect.\n\nA. [substeps] Talk to other consumers to find out what they think and they may get an idea of what your report is likely to end up doing. For example, many marketing writers are tech savvy and planning for the next year's sales are only 75 minutes.\nB. [substeps] Market research is the process of evaluating how well your marketing efforts are working. Specifically, does your marketing get the attention and interest of prospects? Are you converting enough of those prospects into clients? [title] Identify your customer.\nC. Take this low-key approach to analyzing your report. [substeps] \" idea analysis \" is a good way to find information and introduce it into your analysis.\nD. [substeps] Maybe you are 100% sure that the company has an effective, unionized environment. Help identify how this could affect you and your business.","prompt":"[header] How to write a marketing report [title] Consider why you should perform market research and write a report. [step] What information is important to you? What will you do with the marketing report after it is created? This process requires an investment of time and expense. Make sure that you have a clear plan for using the information that you collect.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6507,"error":null,"output":"B"}
{"question_id":"hellaswag:00001420","item_index":239,"attempt":0,"prompt_hash":"95041191bbaad40ea9941c8a41d5c8ed257bc1c7e17676717721f9281401aaac","question":"A lady is feeding a baby ice cream in a car. the lady\n\nA. lays and mulch on the top of the baby.\nB. picks up the ice cream and takes it to the baby.\nC. hands the ice cream to the lady next to her and feed the baby something else.\nD. sits with the baby in the car while the baby eats ice cream.","prompt":"A lady is feeding a baby ice cream in a car. the lady","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6510,"error":null,"output":"D"}
{"question_id":"hellaswag:00009127","item_index":240,"attempt":0,"prompt_hash":"b52b476467eaf4e2f17489deeedfc13d6ee506a008e6f97174b46a691568e2c7","question":"[header] How to choose an online dog training course [title] Look for a training course recommended by the humane society. [step] Check with the humane society to see what training courses they recommend. The humane society can provide you with positive and ethical recommendations for effective online trainers and courses.\n\nA. However, you also may be able to find a blog forum that will list a variety of courses related to your specific needs. [substeps] The humane society also recommends that you read over the syllabus of your course so that you can ensure it is executed according to your specific needs.\nB. [substeps] You also want to make sure the course you choose is accessible to you at a reduced cost and with a membership fee to help you get in shape. Ask friends and family which training classes are available to accommodate your dog's size and age.\nC. [substeps] You can also check the aspca's website to see whether any online trainers are affiliated with the humane society. [title] Consider if the fees are too much for you.\nD. The humane society also offers their own courses for professionals as well as dog owners. [substeps] If you are unsure where to start, try contacting the humane society by phone or email.","prompt":"[header] How to choose an online dog training course [title] Look for a training course recommended by the humane society. [step] Check with the humane society to see what training courses they recommend. The humane society can provide you with positive and ethical recommendations for effective online trainers and courses.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6509,"error":null,"output":"D"}
{"question_id":"hellaswag:00009587","item_index":241,"attempt":0,"prompt_hash":"1d4a2e4f241afc9997f3ad965804984034887fc1554b2a2c8295195baadb26b4","question":"[header] How to paint brass [title] Remove the item if necessary. [step] Some brass objects, such as door handles, faucets, and fixtures will be easier to paint if you remove them from where they are. Other objects, like furniture, utensils, and lamps, are already portable.\n\nA. Ensure that you clean the object thoroughly before you attempt to paint it, and if you make a mistake, contact a professional to help you do so. [substeps] Do not use anything hot or flammable to paint brass objects.\nB. [substeps] If you removed any screws, nails, or other hardware, store them in a safe place so you can return the item after it's painted. It's also a good idea to test that your item is in fact brass.\nC. If you need to paint pots and pans, you may want to take apart the hardware and recycle the paint. [substeps] If your brass item is a jar or a bottle, take apart both before painting them.\nD. [substeps] If you have a piece of brass that you think will be easy to paint, such as the old lamp shade, you should always let it sit in a bucket of water. [title] Cut a separate piece of wood as long as you don't cut yourself.","prompt":"[header] How to paint brass [title] Remove the item if necessary. [step] Some brass objects, such as door handles, faucets, and fixtures will be easier to paint if you remove them from where they are. Other objects, like furniture, utensils, and lamps, are already portable.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6509,"error":null,"output":"B"}
{"question_id":"hellaswag:00001155","item_index":242,"attempt":0,"prompt_hash":"883874f5f729acae892ce62d3e7b5ee42c377a3a9fc5513212f0c2bbfa9e8fd2","question":"We approach a man on a sidewalk. we\n\nA. see some components on the outside of a building with the title of a player.\nB. then see the man at a ski lift a little boy.\nC. see a man playing the accordion on the sidewalk as people walk past.\nD. reengage in a game of arm wrestling.","prompt":"We approach a man on a sidewalk. we","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6509,"error":null,"output":"C"}
{"question_id":"hellaswag:00004309","item_index":243,"attempt":0,"prompt_hash":"3d84c29f1894b1883b4a6b690bef2e79f8973cbc8e1ce94c00756cb3836fba0e","question":"[header] How to mingle with strangers at parties [title] Introduce yourself. [step] Select a party-goer and approach them. Tell them who you are.\n\nA. You might, for instance, say, \" hi, i'm mike, \" then shake hands with them. Ensure your handshake is neither too limp nor too hard.\nB. Even if they don't understand you yet, they're going to remember your name. Roll a paper wrapper one or two times and drop one on their shoulder.\nC. Say something like, \" hi, i'm jessica, but you can call me meg. \" if a stranger asks you for the name or has any questions, just say \" hey, i'm jessica, and i'm new here.\nD. [substeps] Say \" hi \" to the person. You can say something like \" hi i'm jane smith and i'm interested in going to a party.","prompt":"[header] How to mingle with strangers at parties [title] Introduce yourself. [step] Select a party-goer and approach them. Tell them who you are.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6510,"error":null,"output":"A"}
{"question_id":"hellaswag:00002709","item_index":244,"attempt":0,"prompt_hash":"dad7359d13b3d02bf1c41f0393378624f1df955fddc31246be7b6043b5d76f24","question":"A woman in a white dress starts a lawn mower. she\n\nA. starts mowing the yard over and over again.\nB. mows the lawn in the yard of a welder shop.\nC. starts mowing the lawn.\nD. cuts the grass with a machine on the ground.","prompt":"A woman in a white dress starts a lawn mower. she","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6508,"error":null,"output":"C"}
{"question_id":"hellaswag:00000830","item_index":245,"attempt":0,"prompt_hash":"a9cf6e5d9852a782cdfad7d71e2df5ab7cde859aebdf8dbdfa956eec90480e36","question":"A girl sits in the drivers seat of a car and enjoys a ice cream cone. the girl\n\nA. sits in the driver seat of the car while holding onto a baton.\nB. laughs while trying to eat the ice cream.\nC. stops eating, but she continues eating.\nD. stands up and walks into a blondness salon while other people sit at a salon chair.","prompt":"A girl sits in the drivers seat of a car and enjoys a ice cream cone. the girl","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6552,"error":null,"output":"B"}
{"question_id":"hellaswag:00008529","item_index":246,"attempt":0,"prompt_hash":"0a7dd8a7b8efc63c4f29e754bad9e99920518f8eb0ed818b397f87fc00f93f10","question":"[header] How to apply eyelash extensions [title] Evaluate the condition of your natural lashes. [step] If your eyelashes are naturally short and thin, dramatically long or thick extensions won't last long and may damage them. Before selecting extensions, examine your own lashes.\n\nA. The longer and healthier they are, the more weight they can withstand. [substeps] If you've never applied extensions before, consider starting with a half set to see how your natural lashes hold up.\nB. Make sure they're healthy and properly maintained. Root lashes are more susceptible to damage than natural lashes.\nC. Most look best with moderately thick, delicate lashes, but a few look best full and lush. [title] Opt for extensions when you have the extra length needed to create an effect.\nD. Make sure they are in good condition and that they don't contain harmful chemicals that may cause false lashes. [substeps] Remove any lashes that appear just a shade or darker than their natural length.","prompt":"[header] How to apply eyelash extensions [title] Evaluate the condition of your natural lashes. [step] If your eyelashes are naturally short and thin, dramatically long or thick extensions won't last long and may damage them. Before selecting extensions, examine your own lashes.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6552,"error":null,"output":"A"}
{"question_id":"hellaswag:00007471","item_index":247,"attempt":0,"prompt_hash":"d6f869b65b2438b4509f2a8df491a9a602bfd1fe868ad811ba88383510ef802f","question":"[header] How to make eyes look bigger [title] Don't apply eyeliner all the way around your eyes. [step] Doing so will cause your eyes to look smaller, as they will appear fenced or ringed in by the circle line. ( this also means avoiding the).\n\nA. It is not your fault if your eyes are bigger than those of other people you are with. [title] Enunciate before each word you say.\nB. This is not necessarily a bad thing-small eyes can still look mysterious and dramatic-but not if you're looking to make your eyes look larger. [substeps] Apply eyeliner only to the outer 3/4 of your eyelid.\nC. [substeps] If you are going out to lunch, look like you have a huge smile on your face, try applying tinted mascara. [title] Using waterproof mascara, you can apply the top layer of your lashes.\nD. [title] Make sure the hairlines are not too tight. [step] There is no need to use a headphone to do this.","prompt":"[header] How to make eyes look bigger [title] Don't apply eyeliner all the way around your eyes. [step] Doing so will cause your eyes to look smaller, as they will appear fenced or ringed in by the circle line. ( this also means avoiding the).","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6552,"error":null,"output":"B"}
{"question_id":"hellaswag:00006301","item_index":248,"attempt":0,"prompt_hash":"f5414bf899be71ded2518e89c8c41e1b32c1e6dccaa1e9e54642bbae68570cfe","question":"[header] How to detangle dry, damaged hair [title] Follow your regular healthy hair routine. [step] Wash your hair using whatever method works best for you. Don't let your hair dry completely before the next step.\n\nA. For best results, rinse and condition your hair with detangling cream or hair oils to help restore moisture. [title] Keep your hair well conditioned when in the shower.\nB. Use the recommended shampoo (i.e., flue), conditioner (i.e., serums or hair masks) or sulfates. [substeps] If you choose sulfates, apply this to your wet hair first, then to your roots.\nC. [title] Be careful when drying wet or recently washed hair. [step] If you didn't want to leave hair laying around overnight, you would be surprised at how much steam it absorbs.\nD. Avoid blow-drying, as the heat can dry out and damage your hair further. [title] Massage or spray some leave-in conditioner into your wet hair.","prompt":"[header] How to detangle dry, damaged hair [title] Follow your regular healthy hair routine. [step] Wash your hair using whatever method works best for you. Don't let your hair dry completely before the next step.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6552,"error":null,"output":"D"}
{"question_id":"hellaswag:00000356","item_index":249,"attempt":0,"prompt_hash":"b713b5b860459c73401f1ffa7130ecf41d095ca8a2bf4c74f0e357116f47efd4","question":"A grey cat is sitting in a white tub of clear water, drenched in water. A hand with a tattoo appears and wipes the cat's back with a red cloth. the hand\n\nA. looks down and holds the cat's arm as-if to inspect the tattoo.\nB. continues washing the cat with a pink industrial scrub machine.\nC. then begins buffing its nails trying to clean.\nD. then wipes the cat's neck and the back of its head with the red cloth.","prompt":"A grey cat is sitting in a white tub of clear water, drenched in water. A hand with a tattoo appears and wipes the cat's back with a red cloth. the hand","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6552,"error":null,"output":"D"}
{"question_id":"hellaswag:00008013","item_index":250,"attempt":0,"prompt_hash":"862cfcf7558e0af7eb10bf26104d8b5d8e805a7ce6d5a87a9018b5ca7f1aa814","question":"[header] How to grow chrysanthemums [title] Pick a spot outside that has access to sun 5-6 hours a day. [step] Choose a spot in your yard that gets sunny-a little bit of shade is okay. Chrysanthemums like the morning sun, so try to place them in a spot that gets sunlight in the earlier hours of the day.\n\nA. [title] Don't place them outside in sunlight for more than two weeks. [step] The plant can become weak and fragile during this time.\nB. [title] Ensure the soil is well-drained and fertile. [step] Chrysanthemums don't like being constantly wet, so make sure the soil is able to drain easily and that there's plenty of air circulation.\nC. [substeps] Alternatively, you can plant in a location that gets about twice the sun's maximum exposure (3-7 hours). [title] Use sphagnum peat moss.\nD. [substeps] Chrysanthemums need at least five hours of sunlight per day, but some varieties have more. Make sure to plant them in areas that get a lot of direct sunlight, such as in the backyard.","prompt":"[header] How to grow chrysanthemums [title] Pick a spot outside that has access to sun 5-6 hours a day. [step] Choose a spot in your yard that gets sunny-a little bit of shade is okay. Chrysanthemums like the morning sun, so try to place them in a spot that gets sunlight in the earlier hours of the day.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6553,"error":null,"output":"B"}
{"question_id":"hellaswag:00009468","item_index":251,"attempt":0,"prompt_hash":"630a45c25ffc267a302f024a5a6b8e6b54b7d87c99d7e909b65039c4bd955f75","question":"[header] How to fix a sweater that has stretched [title] Determine how much of a sweater needs reshaping. [step] You only need to soak an entire sweater if you want to shrink it entirely. Sometimes, this is not necessary.\n\nA. Your sweater may only have certain portions, like the neck or sleeves, that have stretched out. In this case, you can reshape the sweater by hand.\nB. Other times, even a week-old sweater might simply need reshaping. [substeps] You can buy recuring items from thrift stores or online stores.\nC. Other times, the sweater you are trying to change looks as good or bad as it did before. Whatever the case may be, think about how much you want to stretch the sweater and check the cost of reshaping it to your taste.\nD. Other times, it may require reassembly (if intended to shrink an entire sweater, replacing the sweater stretch it out or removing removing the sweater. ) [title] Check the texture of the sweater.","prompt":"[header] How to fix a sweater that has stretched [title] Determine how much of a sweater needs reshaping. [step] You only need to soak an entire sweater if you want to shrink it entirely. Sometimes, this is not necessary.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5685,"error":null,"output":"A"}
{"question_id":"hellaswag:00009677","item_index":252,"attempt":0,"prompt_hash":"72d6900f4c4bd44155faca0bcbea3d31f7906d5c83d78c12ae804235d18a4db4","question":"[header] How to keep a baby entertained [title] Make your face entertaining. [step] Your baby will be delighted by your funny faces, and may try to imitate you if she can. Making funny sounds, sticking out your tongue, or twisting your face into a funny shape will be entertaining for a baby.\n\nA. [substeps] Babies love to be talked to and have their attention focused on you. You are likely your newborn baby's favorite toy.\nB. [substeps] Use colorful tops to highlight your whiskers. Play war games like stuffed animals or dolls, and use silly hats made of costume fabric.\nC. [title] Mention to your baby what's going on in the world. [step] The easiest way to give your baby information about your life is to genuinely put on a face, and let her take a look at it.\nD. Ideas to help entertain your baby include : [substeps] Make your face laughing or laughing. Even a simple grin can be quite entertaining.","prompt":"[header] How to keep a baby entertained [title] Make your face entertaining. [step] Your baby will be delighted by your funny faces, and may try to imitate you if she can. Making funny sounds, sticking out your tongue, or twisting your face into a funny shape will be entertaining for a baby.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5685,"error":null,"output":"C"}
{"question_id":"hellaswag:00001711","item_index":253,"attempt":0,"prompt_hash":"2b619c8b11c0b13bf4f2fa454ef09f1818f11741afc42320e5aa371f0807f28b","question":"The logo \"crossfit connect\" appears on screen. \"the olympic lifts part 2 - the clean & jerk\" appear. a woman weightlifter\n\nA. without a shirt is standing on a lift.\nB. is entered on the screen by the credit casters.\nC. is standing behind an exercise machine and pushes it forward.\nD. demonstrates the jerk followed by a list of phases.","prompt":"The logo \"crossfit connect\" appears on screen. \"the olympic lifts part 2 - the clean & jerk\" appear. a woman weightlifter","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5686,"error":null,"output":"C"}
{"question_id":"hellaswag:00004400","item_index":254,"attempt":0,"prompt_hash":"99e7323aaf801656655fc0409ed5e2d64a08d4ca082d665cc67b6e625e268baf","question":"[header] How to decorate a cheesecake [title] Take the cake out of the pan shortly before you plan on serving it. [step] It is best to decorate this cake just before you plan on serving it, so plan accordingly. [title] Consider spreading a thin layer of jam or curd on top of the cake.\n\nA. [step] Another option is to simply carve the topping with a carving knife. [title] Spread it across the top of the cheesecake.\nB. [step] Although not necessary, this will give the cheesecake extra flavor. It will also give the strawberries something to stick to.\nC. [step] This will show the flavors in it. [title] Alternatively, slice the cake in half and place it in the pan.\nD. [step] Let it set for a few minutes, to give it an opportunity to soak up any of the fondant coating. [title] Enjoy! [step] Use the fondant now to decorate the cheesecake.","prompt":"[header] How to decorate a cheesecake [title] Take the cake out of the pan shortly before you plan on serving it. [step] It is best to decorate this cake just before you plan on serving it, so plan accordingly. [title] Consider spreading a thin layer of jam or curd on top of the cake.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5686,"error":null,"output":"B"}
{"question_id":"hellaswag:00005518","item_index":255,"attempt":0,"prompt_hash":"b1b044745859072cc8c668a9f5d8073a398d4ab981ba51f202656d76c0d8fe07","question":"[header] How to get the entire class engaged in reading a play [title] Play selection. [step] Maybe you are in a school system that lets you choose your texts. If this is the case, try to make the first play you introduce students to one that contains action sequences and themes such as unrequited love, jealousy, or simply blatant comedy.\n\nA. [title] Try to have the student guess the first play so they can speak at the end and determine how you would respond. [step] This not only encourages them to associate their learning with action, but it helps the student to figure out what goes on next.\nB. This will tend to draw in interest starting at your introduction. If, like many others, you work in a school system where you are told what plays to teach, then you will have to work with what you are given.\nC. Make sure to choose a play that you like that you can use when reading, otherwise you may be breaking your school's rules. [substeps] Some classes might allow you to include objects.\nD. [title] Look at the questions that students ask. [step] It can be tempting to tell students to choose their texts for a particular audience, but this is not always the best idea.","prompt":"[header] How to get the entire class engaged in reading a play [title] Play selection. [step] Maybe you are in a school system that lets you choose your texts. If this is the case, try to make the first play you introduce students to one that contains action sequences and themes such as unrequited love, jealousy, or simply blatant comedy.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5686,"error":null,"output":"B"}
{"question_id":"hellaswag:00000369","item_index":256,"attempt":0,"prompt_hash":"9dcd91357e6cb84208ca7bb754a4edd9c01b4aafe0fd4818fa6ca65bff0ac6f1","question":"A guy runs and jumps over a pole without touching it and lands on mats, as some people in the back ground watch and others do there practices. the guy\n\nA. seems to be practicing how high he can jump by jumping over the pole many times, each time a person raises the pole higher and higher to see if the guy can jump over it.\nB. wearing a flip flops jumps over a pole many times and lands with both feet placed, but recovers and runs and jumps over the pole again.\nC. is sitting on a mat, he is working out then stands up to run.\nD. hold up around a pile of discs mixed with sticks.","prompt":"A guy runs and jumps over a pole without touching it and lands on mats, as some people in the back ground watch and others do there practices. the guy","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5766,"error":null,"output":"A"}
{"question_id":"hellaswag:00006137","item_index":257,"attempt":0,"prompt_hash":"6271cfef67027d2684d34e9dc0b786dc376473f3fb0b3243a867aa45a5a9c3d6","question":"[header] How to dress for the club [title] Groom yourself. [step] Take a shower, shave, and apply hair gel or your hair styling trick of choice. Though the club can get sweaty and hot, it's best to start the night off with a clean look.\n\nA. You want to look clean and fresh in no time, and you don't want to wait until the day before to shower or apply any makeup. [substeps] If you want to wear makeup, no makeup should be on.\nB. [title] Try to tailor your look to fit with the style of the club. [step] If you're heading to a more laid back club, loosen up your collar or opt for jeans instead of slacks.\nC. You want to be in top shape, so exfoliate, moisturize, and style your hair into an attractive hairstyle just after you arrive. [substeps] Some club-sponsored hairdressing requires your hair to be somewhat shaggy.\nD. Reach for clean clothes to keep yourself cool. If you wish, you can also apply moisturizer to your lips.","prompt":"[header] How to dress for the club [title] Groom yourself. [step] Take a shower, shave, and apply hair gel or your hair styling trick of choice. Though the club can get sweaty and hot, it's best to start the night off with a clean look.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5751,"error":null,"output":"B"}
{"question_id":"hellaswag:00002123","item_index":258,"attempt":0,"prompt_hash":"9e2fe0900dfd244900f96410d0a48f2d40aab4540d50617f268ee0bd26c10274","question":"Then we switch to more letters and roof has been sprayed with a solution. We are informed the roof has been cleaned and shown shots of the clean roof. we\n\nA. then see the front of the house.\nB. are shown a more detailed and detailed car wash story.\nC. are informed that the roof is a shiny gray color and the car is gently driven away.\nD. then see a timer over the roof and it numbers 5 and we see four rafters in the studio interviewing.","prompt":"Then we switch to more letters and roof has been sprayed with a solution. We are informed the roof has been cleaned and shown shots of the clean roof. we","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5765,"error":null,"output":"A"}
{"question_id":"hellaswag:00003687","item_index":259,"attempt":0,"prompt_hash":"2b4fd03090b09a5154e5ff8b675d8bb029a7c71cc361d88ac67edafc927092f6","question":"[header] How to keep your child healthy [title] Establish a sleeping schedule. [step] It is important for your child to get around 10 hours of sleep every night, depending on age. Going to sleep at the same time each evening will help this happen.\n\nA. Sleeping at the same time on a regular schedule will ensure there are adequate hours of sleep to wind up each night. [title] Create child-centered mornings and nights.\nB. It is also important to establish a schedule and stick with it even if it is a short one, like going to bed earlier, and then going to bed before you are expected to. Set the same times every night to establish a consistent bedtime.\nC. [substeps] Set a bedtime for each bedroom. Set your alarm to be consistent for on and off certain nights.\nD. Stick carefully to this routine and only push back the bed time if it is absolutely necessary. When possible, allow you child to sleep in if they went to bed late.","prompt":"[header] How to keep your child healthy [title] Establish a sleeping schedule. [step] It is important for your child to get around 10 hours of sleep every night, depending on age. Going to sleep at the same time each evening will help this happen.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5751,"error":null,"output":"B"}
{"question_id":"hellaswag:00001101","item_index":260,"attempt":0,"prompt_hash":"f41e226367812b2aefc14c59bc3cc23c80d1de6071fcd234cbd9564164b099a2","question":"Ice and cream are added into the toy mixer. jam and sugar\n\nA. are added in a bowl.\nB. are added and shown on a plate.\nC. are added to the toy mixer.\nD. are mixed into the mix, pickle juice is blended, beans are added to the mix, pickles, a slice of bread, a bottle of milk, and a label.","prompt":"Ice and cream are added into the toy mixer. jam and sugar","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5764,"error":null,"output":"C"}
{"question_id":"hellaswag:00009354","item_index":261,"attempt":0,"prompt_hash":"7f7448375ee654cec330445680065bca2d236d32a1a99137c5eba11460c52e37","question":"[header] How to heal ulcers fast [title] Make a doctor's appointment to determine the cause of your ulcer. [step] The type of treatment you need will depend upon what is causing your ulcer. If you don't already know the cause, talk to your doctor about diagnostic options.\n\nA. They may suggest laboratory testing, endoscopy, and/or x-rays, and will be able to refer you to any necessary specialists. [title] Use antibiotics to treat h pylori infections.\nB. [substeps] For example, your doctor might suggest congestive heart failure, bleeding out from the ulcer site, or an abscess that would have caused your ulcer. Your doctor may also recommend surgery, especially if the ulcer is very deep and serious.\nC. Your doctor can help you formulate a plan and help you determine the best form of treatment. [substeps] If you suffer from gum disease, ulcers will likely require surgery.\nD. [substeps] It's best to have had you treated 6 times each year by a doctor who has experience treating sids. Your doctor can suggest doctor's visits to treat ulcers in other areas of your body so they can formulate the appropriate treatment options for you.","prompt":"[header] How to heal ulcers fast [title] Make a doctor's appointment to determine the cause of your ulcer. [step] The type of treatment you need will depend upon what is causing your ulcer. If you don't already know the cause, talk to your doctor about diagnostic options.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5763,"error":null,"output":"A"}
{"question_id":"hellaswag:00003768","item_index":262,"attempt":0,"prompt_hash":"c423d2e74debb019b78c19fb74eee402655caff0543b48533bb37fb75c29145a","question":"[header] How to keep black jeans from fading [title] Wash your jeans by hand in cold water. [step] Cold water helps reduce the denim's dye from bleeding. Use a natural soap and don't apply too much pressure when scrubbing.\n\nA. Soak your jeans for a couple minutes and then let them dry on a hanger. [substeps] You can effectively wash and soak your jeans in your kitchen sink, as long as it's clean.\nB. Keeping it in your jeans allows the dye to penetrate deeper and will keep the fabric from drying out. [substeps] It's best not to use all-purpose wash cloths, as that will stain your jeans or leave behind a discoloration.\nC. The cold water weakens the fabric, not only making it less likely to bleed, but also opening pores. Run cool, running water through your jeans to get the excess water out.\nD. Otherwise, the bleaching process could leave a visible discoloration. To wash and dry a pair of distressed jeans : [substeps] Use a wet cloth to wipe the area dry.","prompt":"[header] How to keep black jeans from fading [title] Wash your jeans by hand in cold water. [step] Cold water helps reduce the denim's dye from bleeding. Use a natural soap and don't apply too much pressure when scrubbing.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5752,"error":null,"output":"A"}
{"question_id":"hellaswag:00005461","item_index":263,"attempt":0,"prompt_hash":"48ff47a59c7963c10075881855084a4c844e892f68ae8a454c93f4c44ab05033","question":"[header] How to apply bronzing lotion [title] Choose a bronzing lotion. [step] There are many different bronzing lotions to choose from. Choosing a bronzing lotion based on your skin type, color, and you desired tan will help you achieve the results you are looking for.\n\nA. There are three types of bronzing lotions : [substeps] Moisturizing creams. Moisturizing lotions are products that lock on your bronzing and provide additional moisture while enhancing your tan.\nB. Some bronzing lotions allow you get instant results while others take up to a week. Some are longer lasting while other are more temporary.\nC. Benchmark is one of the most popular products available, but there are also several different brands that are available and offer a variety of different shades of bronzing lotion. [substeps] You can also try different types of products to determine which bronzing lotion will work best for you.\nD. Choose between tinted or untinted if you want even more sun exposure. [substeps] Tinted or untinted bronzing lotions are for both male and female users.","prompt":"[header] How to apply bronzing lotion [title] Choose a bronzing lotion. [step] There are many different bronzing lotions to choose from. Choosing a bronzing lotion based on your skin type, color, and you desired tan will help you achieve the results you are looking for.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5751,"error":null,"output":"C"}
{"question_id":"hellaswag:00009215","item_index":264,"attempt":0,"prompt_hash":"3d4ae2b86466ce8600a0769b4a1d4f879463b92045efb1836f45b4835e981689","question":"[header] How to prevent static electricity in your hair [title] Try an ionic blow dryer. [step] Some people have had success fixing static hair by using an ionic blow dryer. This blow dryer has negative ions that neutralize the positive ions in your hair to help get rid of static.\n\nA. [title] Look for a static-free hair dryer. [step] Many people are concerned that an electric-free hairdryer will help offset the energy carried by static electricity.\nB. [substeps] You can buy it at your local hardware store or online. Blow dryers are not as efficient at removing static as heat protection units.\nC. Simply spray an air-tight, flat iron on your hair and let it cool. The resulting static electricity produced in the air passes through the iron and into the hair along with the rest of your hair.\nD. The ion molecules also break down water molecules in your hair instead of evaporating water molecules, like regular blow dryers. This helps prevent your hair of being stripped of moisture and getting staticky.","prompt":"[header] How to prevent static electricity in your hair [title] Try an ionic blow dryer. [step] Some people have had success fixing static hair by using an ionic blow dryer. This blow dryer has negative ions that neutralize the positive ions in your hair to help get rid of static.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5751,"error":null,"output":"D"}
{"question_id":"hellaswag:00006883","item_index":265,"attempt":0,"prompt_hash":"615e832b071d726fa38f47da66fed6f4ea0545ec0173d41a45950738bc13be24","question":"[header] How to create an interesting character [title] Think about personality and role in the story. [step] What does your character do for the story? What type of personality will they have? Start sketching out their rough personality and general role. A clear personality (as opposed to \" does whatever \") gives your character a stronger presence in the story.\n\nA. Avoid making tough decisions about what works and what doesn't. [substeps] Consider what your character's role in the story is-if you don't know your character well, it won't matter how well you design your character.\nB. [substeps] A list of adjectives (e.g. caring, perceptive, perfectionistic, anxious) is a good start, but it isn't nearly enough for a complex character.\nC. [substeps] Think about how long you have known your character; how often you feel inspired by their traits? [title] Create interesting characters. [step] If your characters have a specific personality, research them so that you can relate to them.\nD. [substeps] Often examples to write down include: did your character face the beast? What traits do your character possess? Keep in mind that no one is perfect. That's okay; there are positives and downs.","prompt":"[header] How to create an interesting character [title] Think about personality and role in the story. [step] What does your character do for the story? What type of personality will they have? Start sketching out their rough personality and general role. A clear personality (as opposed to \" does whatever \") gives your character a stronger presence in the story.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5752,"error":null,"output":"C"}
{"question_id":"hellaswag:00007142","item_index":266,"attempt":0,"prompt_hash":"1daaaf43ff9f686cb034dd34e9820a35b7252084ec8aa01bdb79bb0d87fe7a43","question":"[header] How to protect your skin and hair from chlorine [title] Wet your head. [step] As tempting as a dive into the pool might be, take a minute to wet your hair in the shower before you swim. Wet hair is less likely to absorb chlorinated water than dry locks.\n\nA. [title] Mix equal parts water and dechlorinated milk. [step] This simple self-dissolving solution is available at many health food stores, and it is applied to hair.\nB. [substeps] As a result, it's important that you exfoliate the skin and hair thoroughly. This can prevent the chlorine from damaging the fibers of your hair follicles.\nC. [substeps] Soak your hair in the shower until it is entirely saturated. This may feel like your head is heavy.\nD. Wet hair is less likely to mold and become brittle. [title] Spritz a bit of light body spray directly onto your hair.","prompt":"[header] How to protect your skin and hair from chlorine [title] Wet your head. [step] As tempting as a dive into the pool might be, take a minute to wet your hair in the shower before you swim. Wet hair is less likely to absorb chlorinated water than dry locks.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5752,"error":null,"output":"C"}
{"question_id":"hellaswag:00004185","item_index":267,"attempt":0,"prompt_hash":"075f4d56c9767c94b230032414d4b72a27cf3f8b353f33c56dc0c9b5c0cda4d2","question":"[header] How to prevent cat scratch disease [title] Know if you are at risk. [step] Some people are more likely to get csd than others. Organ transplant patients, infants and young children, and pregnant women are more likely to get diseases from animals.\n\nA. [substeps] African american women and their kids younger than 18 are at higher risk for developing csd. Breast cancer can be a possible cause of csd.\nB. You should also be aware that some people (generally disabled or under) are at an increased risk of getting diseases from humans. [substeps] If you are a disabled person or have a history of stress or illness, your risk for csd is higher than usual.\nC. If your cat's immune system is compromised, s/he could take some medications such as insulin, antibiotics, or a prescription pain reliever. People you know or are friends with may be at greater risk for csd than they are for humans because of their immunity to it.\nD. Those who are immunocompromised, or have a weak immune system, such as people with hiv, should be extra careful about getting csd. [title] Wash your hands after handling your cat.","prompt":"[header] How to prevent cat scratch disease [title] Know if you are at risk. [step] Some people are more likely to get csd than others. Organ transplant patients, infants and young children, and pregnant women are more likely to get diseases from animals.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5752,"error":null,"output":"D"}
{"question_id":"hellaswag:00004064","item_index":268,"attempt":0,"prompt_hash":"4d7c6c37cc4da79829991dbb09282877aac585fd5b5df04bc9a07fb2c4353569","question":"[header] How to take hormone replacement therapy [title] Talk to your doctor about your hrt plan. [step] Hrt can be a useful tool, but it also comes with some increased risks including a slightly higher risk for certain cancers, blood clots, and strokes. Your doctor will be able to help you decide if hrt is the right way to manage your menopausal symptoms.\n\nA. [substeps] Different methods of taking hormones have different associated risk levels. Your doctor will talk to you about your medical history and your menopausal symptoms to pick the right method of hormones for you.\nB. Doing research will also help you decide what type of hormone replacement therapy you might take. You may want to consider a combination of genders if your period is the only one you have had.\nC. [title] Decide on a one-time plan. [step] Successful hrt has a payoff of significantly more than it's worth to take a single hormone replacement every four to six weeks.\nD. [title] Assess whether hrt is right for you. [step] In general, hrt is not recommended for women, as menopausal women have less difficulty with switching their hormones.","prompt":"[header] How to take hormone replacement therapy [title] Talk to your doctor about your hrt plan. [step] Hrt can be a useful tool, but it also comes with some increased risks including a slightly higher risk for certain cancers, blood clots, and strokes. Your doctor will be able to help you decide if hrt is the right way to manage your menopausal symptoms.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5752,"error":null,"output":"A"}
{"question_id":"hellaswag:00002994","item_index":269,"attempt":0,"prompt_hash":"a77ab0eb16e5aee23d6e12825aea6f5adbb825322620a13cdefe2f90f78c7254","question":"A group of people clean cars at a car wash. a person\n\nA. uses a red hose to clear the puddles by the car.\nB. sticks an adult in the trunk of a car while the young boy stands by.\nC. is interviewed about the process.\nD. smokes the hookah and blows the smoke.","prompt":"A group of people clean cars at a car wash. a person","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5761,"error":null,"output":"C"}
{"question_id":"hellaswag:00003445","item_index":270,"attempt":0,"prompt_hash":"d03d58760a49206dcc1eb725b4f52c5387f8026d6f2516b7f06a9c05a7453fdb","question":"[header] How to dye your hair denim [title] Purchase a bleaching kit. [step] If you have naturally blonde hair or very light brown hair, you can skip the bleaching process. Blue dye will layer over your existing color, so it will hardly show up if you have darker hair.\n\nA. [substeps] Shop online or in bulk online for hair dye kits if you have recently dyed your hair. This is called altering the color of your hair, as restoring the color to it is one of the fastest ways to get blue dye.\nB. You can dye your hair just as you would your hair going to another hair color. The only difference is that if you dye your hair purple it will just show up on your wig to school.\nC. You can find bleaching kits at beauty supply stores, and they will have everything you need to bleach your hair, so they're great for amateur colorists. The kits will come with bleach powder, developer, a mixing bowl, and a brush.\nD. Luckily, you can also choose a kit with six colors that complement your current color. This will make the dyeing process possible.","prompt":"[header] How to dye your hair denim [title] Purchase a bleaching kit. [step] If you have naturally blonde hair or very light brown hair, you can skip the bleaching process. Blue dye will layer over your existing color, so it will hardly show up if you have darker hair.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5756,"error":null,"output":"C"}
{"question_id":"hellaswag:00006767","item_index":271,"attempt":0,"prompt_hash":"7994bd23ef69ae05640674b70e7fee594981af6ba7ab442f26dedcd83a2d15f7","question":"[header] How to reset the password of a samsung galaxy note [title] Open https: // www.google.com/android/devicemanager in a web browser. [step] If you set up your galaxy note to use the android device manager, you can use it to reset your device's lost password. [title] Sign in with your google account information.\n\nA. [step] Use the username and password you used to set up your note. [title] Click your samsung galaxy note on the screen.\nB. [step] Using your google account information will allow google to begin the reset process. If you don't have an account, you'll need to create one.\nC. [step] You'll need to provide the basic personal information about you and your device. [substeps] To create a google account, click the \" create an account \" button on the upper right side of the screen.\nD. [step] On the \" sign in \" field in the upper right of the screen, you'll find the sign in button. Tap on this to log in with your google account information.","prompt":"[header] How to reset the password of a samsung galaxy note [title] Open https: // www.google.com/android/devicemanager in a web browser. [step] If you set up your galaxy note to use the android device manager, you can use it to reset your device's lost password. [title] Sign in with your google account information.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5759,"error":null,"output":"A"}
{"question_id":"hellaswag:00003560","item_index":272,"attempt":0,"prompt_hash":"2d0ed2f58ee45d8dbfd06f82320f950eed70d6ab87817b4df0b456f117cedf8a","question":"[header] How to make a period pouch [title] Make or purchase a suitable pouch. [step] It can be a simple bag with a zipper that you sew up at home or a very small sized makeup bag. It must be able to fit into your purse, schoolbag or sleepover tote, and should be small and discreet enough to not attract much attention.\n\nA. Make sure the bag fits comfortably inside your purse and has a closed or checked top. [substeps] Make sure that the opening of the bag is small enough so that the victim cannot see out of the pouch or the pocket.\nB. [substeps] When determining the size of the pouch, think ahead of how many items you'll need to stock inside of it. This may vary amongst different periods, as some flows are heavier than others.\nC. [substeps] If you're making a pouch to go with a friend, you should opt for one made of a similar material such as fabric or felt. If this is your first time making a pouch, don't worry.\nD. Choose a fitting pouch with a zipper that would get noticed, even if not discreet. [substeps] Make the pouch off-white or have your pregnancy powder or other makeup removed if you are pregnant.","prompt":"[header] How to make a period pouch [title] Make or purchase a suitable pouch. [step] It can be a simple bag with a zipper that you sew up at home or a very small sized makeup bag. It must be able to fit into your purse, schoolbag or sleepover tote, and should be small and discreet enough to not attract much attention.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5760,"error":null,"output":"B"}
{"question_id":"hellaswag:00002756","item_index":273,"attempt":0,"prompt_hash":"f732863e0061f6e7821be2feb1f271475ca430ebfce9aa9432c6a96ebb047340","question":"People are in a rodeo arena while a man on a horse prepares a rope with a loop. then\n\nA. , two men secure the rope by doing pulleys.\nB. , the man talks hands on race line while people walk in pass next to the horses.\nC. , the bull get chased by a crowd.\nD. , the man spins the loop and throw to catch a calf, then run and tie the calf.","prompt":"People are in a rodeo arena while a man on a horse prepares a rope with a loop. then","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5760,"error":null,"output":"D"}
{"question_id":"hellaswag:00008918","item_index":274,"attempt":0,"prompt_hash":"845baec1feb76b1956403bc2c97925cdec6f9e2b2cd881593ef14f2c556c9f92","question":"[header] How to put a two year old to sleep [title] Determine a good time schedule. [step] Allow plenty of hours between nap time and bedtime. If they are too close together, your child is unlikely to sleep.\n\nA. [title] On average, a two-year-old sleeps about 11 hours at night and two hours during the day, although this is not a solid rule. [title] Be sure your child has enough physical exercise.\nB. This will not prevent them from falling asleep and waking up at the same time each night. Stick to a schedule that gives you the optimal amount of time for bed time.\nC. If they are too far apart, your child may be more prone to wanting to sleep. Always factor an hour before bedtime into their age-appropriate sleep/wake time.\nD. Make sure that the nap time and bedtime are within this timeframe. [title] Make the naps shorter than regular ones.","prompt":"[header] How to put a two year old to sleep [title] Determine a good time schedule. [step] Allow plenty of hours between nap time and bedtime. If they are too close together, your child is unlikely to sleep.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5762,"error":null,"output":"A"}
{"question_id":"hellaswag:00009322","item_index":275,"attempt":0,"prompt_hash":"ec8b775044f2e0a0a2e4062b00f9809986ba8e7b49805f625e5ed0af27407897","question":"[header] How to clean a cockatoo cage [title] Put a fresh liner in the cage. [step] Take out the old liner at the bottom of the cage and replace it with new liner. You may use newspaper or paper as liner for the cage.\n\nA. If the cleaner makes the cage feel sticky, squeeze some and dip it in a cup filled with the old liner and let it sit until it dries. Then, dip another liner and put the old liner back in the cage.\nB. [substeps] Do not put liners in the part that will be slippery, like the top of the cage. You can also use plastic or glass liners instead.\nC. Put fresh liner in the cage at the beginning or the end of the day so it is clean. [substeps] If you have layered the paper in the cage, you can remove the top sheet so the liner is clean and fresh.\nD. If you replace the old liner then you are ruining the bird's cage. [title] Place the new liner in a bucket.","prompt":"[header] How to clean a cockatoo cage [title] Put a fresh liner in the cage. [step] Take out the old liner at the bottom of the cage and replace it with new liner. You may use newspaper or paper as liner for the cage.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5760,"error":null,"output":"C"}
{"question_id":"hellaswag:00009493","item_index":276,"attempt":0,"prompt_hash":"bed5686c413594c88a0fb37702c317d417d7f408d3049192ddbc4c2658e54c9a","question":"[header] How to open a child proof pill container [title] Place the container on a flat surface. [step] This will ensure you get a good, even grip on the container. [title] Check the label to determine what type of child-proofing is on the container.\n\nA. [step] There are several possible types, including : [substeps] Push and turn down-the lid will have an arrow pointing down or a label that says push. Squeeze the sides and turn-the lid will have grooves around the lid to help you squeeze and turn it easily.\nB. [step] There are many different varieties and make available to be found over the internet. You'll just need to choose your preferred container--easy to clean, comfortable, for your child to lift and carry things easily, easy to store, and easy to clean.\nC. [step] When choosing a child-proofing carrier, you would want to choose a container with 1 outer circumference (for the container) and 1 inner diameter (for the drug). These measurements are easier to measure and you will likely get the right dimensions.\nD. [step] It could say \" non-toxic \", \" inflammable, \" or whatever else is listed on the label. [title] Open the plastic lid on the container.","prompt":"[header] How to open a child proof pill container [title] Place the container on a flat surface. [step] This will ensure you get a good, even grip on the container. [title] Check the label to determine what type of child-proofing is on the container.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5761,"error":null,"output":"A"}
{"question_id":"hellaswag:00008565","item_index":277,"attempt":0,"prompt_hash":"395be02a4600be5e93f53b3a6cf4503f5c608553d1f96a05e129c8062a280a0b","question":"[header] How to get soft lips [title] Wear a moisturizing lip balm, as needed. [step] Use a balm with shea or cocoa butter for the best results. Reapply it only when your lips feel dry.\n\nA. [title] Use lip balm with an alpha lip color. [step] Alpha lips make them look a little more natural, while the bold color creates a cute glow.\nB. [substeps] If you use lip balm too often, it can make your lips more chapped. Apply lip balm underneath any other lip products that you use, including lipstick, lip gloss, and lip stain.\nC. You can apply a glossy lip balm to wet lips in the mornings, as it will help moisturize them even in the fall. If your lips are short and matte, seek a more lightweight, scented balm.\nD. If you are looking for a more subtle trend, consider lip balm that contains shea shea or cocoa butter instead. [substeps] For a slightly softer look, use the tooth plumping formula instead of lip balm.","prompt":"[header] How to get soft lips [title] Wear a moisturizing lip balm, as needed. [step] Use a balm with shea or cocoa butter for the best results. Reapply it only when your lips feel dry.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5891,"error":null,"output":"B"}
{"question_id":"hellaswag:00000118","item_index":278,"attempt":0,"prompt_hash":"5a1dd21be02e71b0ffdca2613ba67e9a3d9bc916cbf8a75aac678ac80b777002","question":"Several people are seen drilling holes into the ice while looking down and speaking to the camera. Many pictures are shown of the people fishing as well as pulling fish out of a hole. more people\n\nA. are seen swimming around in the lake and the men speak to the camera.\nB. are seen having their pictures taken and speaking to the camera while water flows all around them.\nC. sit by the holes and continue to pull out fish.\nD. are seen drilling holes as well as swimming around in the water.","prompt":"Several people are seen drilling holes into the ice while looking down and speaking to the camera. Many pictures are shown of the people fishing as well as pulling fish out of a hole. more people","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5892,"error":null,"output":"D"}
{"question_id":"hellaswag:00004541","item_index":279,"attempt":0,"prompt_hash":"59d6d43a9f5a8a24abe0625269166d9aa5f0834b58ef7c46c5e944a7034af3d1","question":"[header] How to use cold cream to reduce acne [title] Choose a cold cream from the drugstore or make your own. [step] Recipes are available online. Ponds cold cream (the jar with the green lid) is the brand most commonly sold in the us, but some women prefer the off-brands that still contain sodium borate, an ingredient in an earlier formulation of ponds.\n\nA. If in doubt, order ice cream from the fridge that has been previously washed and wiped off. [substeps] You will also want to experiment with flavored cold cream.\nB. [title] Microwave the mixture until it is firm to the touch. [step] Stir the mixture continually if it becomes uncomfortable.\nC. Jergens all purpose face cream is another cold cream commonly available at drugstores. [title] In the morning, use your cold cream to cleanse your face.\nD. [title] Take it during the day when it's cooler outside. [step] Running, hiking, or drinking ice cream at night can help speed the process.","prompt":"[header] How to use cold cream to reduce acne [title] Choose a cold cream from the drugstore or make your own. [step] Recipes are available online. Ponds cold cream (the jar with the green lid) is the brand most commonly sold in the us, but some women prefer the off-brands that still contain sodium borate, an ingredient in an earlier formulation of ponds.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5892,"error":null,"output":"C"}
{"question_id":"hellaswag:00003631","item_index":280,"attempt":0,"prompt_hash":"8f62f710f718525a8733c191d37126daf1c307c7cc56747dcff6bfe34efdd7a5","question":"[header] How to make mini vegetable tarts [title] Prepare the puff pastry dough. [step] If the dough was frozen, have it defrost overnight. Roll out the puff pastry dough in a 12 x 8 inch rectangle using a rolling pin.\n\nA. Set the dough over parchment paper and cut it into diamond shapes. [title] Preheat the oven to 350ºf/170ºc.\nB. [title] Preheat the oven to 325 degrees f (177 degrees c). [title] Cream the butter and sugar until light and fluffy.\nC. Place it on a baking sheet covered with parchment paper. Leave the dough to chill in the fridge while you prepare the filling.\nD. Bake it on the stovetop until it turns lightly brown and won't burn. [title] Stand the tarts with a wire rack.","prompt":"[header] How to make mini vegetable tarts [title] Prepare the puff pastry dough. [step] If the dough was frozen, have it defrost overnight. Roll out the puff pastry dough in a 12 x 8 inch rectangle using a rolling pin.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5892,"error":null,"output":"C"}
{"question_id":"hellaswag:00006988","item_index":281,"attempt":0,"prompt_hash":"121c3188864a337967f9d2821ca8e3bd69170a80042aacc42b63eb16c89e3548","question":"[header] How to announce the bridal party at a reception [title] Give the mc a written list of names. [step] Your wedding party is a big part of your special day. When you introduce them, you want to make sure that they get the recognition that they deserve.\n\nA. Count up to fifty names at the reception and give each one two days' notice. [substeps] Making a list of names is a great way to tell if your reception is going well.\nB. Make sure the dj or mc has a written list of the names of the entire bridal party. [substeps] If necessary, include a pronunciation guide for the announcer.\nC. List their contact information as soon as possible, because this gives the mc time to do the actual announcement, or else you could miss out. [title] Note the formality of the wedding.\nD. Write down the names of everyone involved, the names of the bride, and the groom. Have them read it to the camera for the party and make sure all the names are correct.","prompt":"[header] How to announce the bridal party at a reception [title] Give the mc a written list of names. [step] Your wedding party is a big part of your special day. When you introduce them, you want to make sure that they get the recognition that they deserve.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5892,"error":null,"output":"B"}
{"question_id":"hellaswag:00005984","item_index":282,"attempt":0,"prompt_hash":"c4831205c003d88625f0c85ae32393aaa3f98e9d6a77bcc6d69c2f68a00c3428","question":"[header] How to get a flawless look with makeup and eyes [title] Get everything you will need and have it on hand. [title] Start with the face routine. [step] Make sure your face is clean and moisturized.\n\nA. Make sure you have a peachy skin tone. Make sure you're confident enough to do the full facial as opposed to waxing or making any external changes.\nB. That way, after a bit of cleansing you can get out. If you don't have time to soak, just wash your face with a facial cleanser (realize, though, that use of facial cleansers is completely unsanitary).\nC. If it's dry from a sunburn or something like that than you should use a face scrub. Don't forget moisturizer though! [title] Use the liquid foundation and apply a thin even coat with the foundation brush all over your face.\nD. Choose concealer, blush and clear false lashes. This makeup gives you your perfect, flawless face but masks out some of your cheekbone and eyes color.","prompt":"[header] How to get a flawless look with makeup and eyes [title] Get everything you will need and have it on hand. [title] Start with the face routine. [step] Make sure your face is clean and moisturized.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5890,"error":null,"output":"C"}
{"question_id":"hellaswag:00001391","item_index":283,"attempt":0,"prompt_hash":"b38e9996d5a04d4875cef169bd75abda32dbdec9628430d92e9a0b9d9cd1520e","question":"There's a man wearing a black athletic suit in a shot put field practicing shot put. he\n\nA. begins by taking the ball in his hand and holding it against his neck.\nB. is running down the track pulling a javelin behind him.\nC. walks over to a brown colored bag and begins taking out several pieces of exercise equipment and sets them down.\nD. begins by doing three stumbling steps to his right and then goes straight back to his original stance.","prompt":"There's a man wearing a black athletic suit in a shot put field practicing shot put. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6694,"error":null,"output":"A"}
{"question_id":"hellaswag:00007498","item_index":284,"attempt":0,"prompt_hash":"e985ff064b9d22517be71c908b7e50c3842bab400d888adb1b221d818f56339f","question":"[header] How to give a home manicure or pedicure [title] Remove the polish from your fingers and toes. [step] Before you start painting, you'll want to ensure that your finger and toenails are completely free of polish. Pour a capful of nail polish remover onto a cotton ball and use the cotton ball to clean your nails.\n\nA. [substeps] You can also use a cotton ball dipped in acetone to remove the polish from your shoes. [title] Paint your nails with liquid acetone nail polish remover.\nB. [substeps] Make sure your nail polish remover is acetone free. Never paint over already painted nails, unless you're touching up a small area.\nC. [substeps] Avoid rubbing your toenails, as this can damage them. Polish remover is especially dangerous when using a cotton ball as the cuticle oil dries.\nD. Hold the nail or toe scrapping pad flat on your skin and then rub a thin layer of polish across it, covering every surface. [substeps] Use any cotton ball sized enough to cover every nail, but be careful not to get polish on your skin.","prompt":"[header] How to give a home manicure or pedicure [title] Remove the polish from your fingers and toes. [step] Before you start painting, you'll want to ensure that your finger and toenails are completely free of polish. Pour a capful of nail polish remover onto a cotton ball and use the cotton ball to clean your nails.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6695,"error":null,"output":"A"}
{"question_id":"hellaswag:00008064","item_index":285,"attempt":0,"prompt_hash":"986b6fae018ac34015f8c53223376a1c547ba9599e533e2467e5c647a10ce2fa","question":"[header] How to get pretty eyes without wearing makeup [title] Get enough sleep. [step] One of the main reasons people have dark circles under their eyes is lack of sleep. Make sure that you get eight uninterrupted hours.\n\nA. Keep your bedroom dark so that it's easy to fall asleep and stay asleep. [title] Use moisturizer under your eyes.\nB. If your parents notice, they will pick up on this and think you aren't sleep-deprived and cause trouble for your family. [substeps] Get eight hours of uninterrupted sleep a night.\nC. You can go to bed feeling refreshed by adding sleep boosting make up and patches yourself. This is a huge red flag to worry about.\nD. Go to bed and wake up roughly the same time each day. Get up about 45 minutes before your bedtime every other day.","prompt":"[header] How to get pretty eyes without wearing makeup [title] Get enough sleep. [step] One of the main reasons people have dark circles under their eyes is lack of sleep. Make sure that you get eight uninterrupted hours.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6694,"error":null,"output":"A"}
{"question_id":"hellaswag:00001079","item_index":286,"attempt":0,"prompt_hash":"a438505d5d19edb8e3853f5691ca56b584dd93e3d3be1af8a60def02c1f3fe20","question":"Woman is holding a soap facial cleanser on her hands and a face towel. the woman\n\nA. washes her face on the beach.\nB. open the soap and started to breathe on the face.\nC. cleans the face with the liquid soap in circular motions.\nD. put the red soap in her hands, then soap on her hands and the face towel.","prompt":"Woman is holding a soap facial cleanser on her hands and a face towel. the woman","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6693,"error":null,"output":"C"}
{"question_id":"hellaswag:00000913","item_index":287,"attempt":0,"prompt_hash":"60f5ba74fd05dce434f1f37c9729871182170a2fab0ca92f8d0b6fda47ae149d","question":"A man is seen walking across a yard while holding a stick in his hand. The man is then seen playing a game of crochet while others watch on the side. the man\n\nA. continues to hit the ball around as well as bend down and measure out his shots.\nB. continues to play with another and ends by hitting a button on a device.\nC. continues to play and ends by walking up and down.\nD. continues hitting the ball around while another walks in and jumps on his back.","prompt":"A man is seen walking across a yard while holding a stick in his hand. The man is then seen playing a game of crochet while others watch on the side. the man","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6694,"error":null,"output":"D"}
{"question_id":"hellaswag:00001617","item_index":288,"attempt":0,"prompt_hash":"7f7ff73c02f1a6de46bac9e8b59128d507273f38f4c676d006ee13b1183e281f","question":"A man in a suit reads off of a paper. a woman\n\nA. sniffs a cat.\nB. speaks next to a large fire.\nC. then begins to report a news story.\nD. rolls up a towel and wipes down a chair outside in the yard.","prompt":"A man in a suit reads off of a paper. a woman","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6329,"error":null,"output":"C"}
{"question_id":"hellaswag:00006094","item_index":289,"attempt":0,"prompt_hash":"07ddc1c049dc988a2fd4ed7f3c9b99107ec19994fbf004395f900618714c17ac","question":"[header] How to feel amazing [title] Take care of personal hygiene. [step] Taking care of your body and having a little \" me \" time will help you to both look and feel amazing. [substeps] Take a nice, long, hot shower.\n\nA. You will want to shower daily, and use deodorant and perfume daily. Smell your favorite perfumes in the morning and evening.\nB. Let your hair air dry and take a bath if you don't have time to do so on the weekend. Leave dirty clothes in the hamper or throw away dirty clothes.\nC. Make sure to thoroughly wash your hair with shampoo, and leave your conditioner in for 5 minutes. Wash, rinse, and repeat as many times as you desire.\nD. Do not let it sink in that this is uncomfortable, it is light and refreshing, gentle. Use some deodorant and a nice smelling perfume.","prompt":"[header] How to feel amazing [title] Take care of personal hygiene. [step] Taking care of your body and having a little \" me \" time will help you to both look and feel amazing. [substeps] Take a nice, long, hot shower.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6324,"error":null,"output":"C"}
{"question_id":"hellaswag:00001676","item_index":290,"attempt":0,"prompt_hash":"fa6fcb8bfa4994b9f45c7864b4733ebf25599dad36ffe95f0f2b1998afb0636d","question":"More clips of people playing hockey on a rink and a man holding a microphone because talking and the blue words on the bottom of the screen read \"new russian president vladimir putin makes surprise appearance\". the men\n\nA. continue playing with all of the hockey players until the man with the microphone stops playing and walks away.\nB. are done playing hockey and are now standing in a line so they can shake hands.\nC. then discusses their movements, that is quickly changing to knitters thrown in the court of judges, and the blue words on the screen say \" link \".\nD. continue to play hockey without losing a point and after several shots to the man he's holding a microphone pretending to ignore it and just play.","prompt":"More clips of people playing hockey on a rink and a man holding a microphone because talking and the blue words on the bottom of the screen read \"new russian president vladimir putin makes surprise appearance\". the men","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6328,"error":null,"output":"A"}
{"question_id":"hellaswag:00002751","item_index":291,"attempt":0,"prompt_hash":"cfeff63f8998ce862b0d253a30d0d1f9e63b9fc2dd868ae641b96896eaf7cfbe","question":"A glass of tequila sunrise is shown. a man\n\nA. is sitting and talking on a stage.\nB. is setting up for a foulker event in his vip room.\nC. does shots of the liquid while dancing.\nD. is talking in front of a display of alcoholic drinks.","prompt":"A glass of tequila sunrise is shown. a man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6327,"error":null,"output":"D"}
{"question_id":"hellaswag:00006175","item_index":292,"attempt":0,"prompt_hash":"b12895f518257e924c01f60761ebf66784436a2cb1a0e65b278eb581d949af05","question":"[header] How to control your feelings around the person you like [title] Identify your feelings and their source. [step] This can help you figure out whether or not your feelings are a passing fancy or something more serious. It will also help you talk yourself through your feelings when you feel them.\n\nA. Is this infatuation? Lust? When you feel a warm blush start to spread across your face or a similar feeling, acknowledge the emotion, and positively talk yourself through it. [substeps] \" i'm just nervous because i think he's cute.\nB. [substeps] When you're feeling in a critical way, be positive about it. If you don't like this person, try talking to them again.\nC. Finding the source of your feelings will make you feel more relaxed and less guilty about your interactions with this person. [substeps] For example, if you have an intense dislike of the person in front of you, consider asking for advice about being more comfortable around him or her.\nD. This will lead to an easier conversation and insight for you and your partner. [substeps] Writing down thoughts and feelings about the person you like can also help, but be careful.","prompt":"[header] How to control your feelings around the person you like [title] Identify your feelings and their source. [step] This can help you figure out whether or not your feelings are a passing fancy or something more serious. It will also help you talk yourself through your feelings when you feel them.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6323,"error":null,"output":"C"}
{"question_id":"hellaswag:00008700","item_index":293,"attempt":0,"prompt_hash":"186b407eec1902b7da37a0e345bc083ae6c12e7802e13036beefcef32b1d32fc","question":"[header] How to keep high heels from slipping [title] Learn your size for heels. [step] Many people wear different sizes for heels than they do for flats, which can make getting your size right tricky! Try on heels a half-size larger than your usual size to see if they fit more comfortably. [title] Buy platform or wedge heels.\n\nA. [step] Platform heels are just legs with ankle pads and gel foot protectors. Wedge heels are not as common, but maybe they're for you! [substeps] Platforms and wedges add some depth to a flat heel, on the other hand, but generally, they're uncomfortable to walk in.\nB. [step] Stiletto heels look great, but they're much harder to walk in. Buying wedge heels or heels with a platform sole will level your foot out and make it easier to walk.\nC. [step] Platform heels are the most popular, as they don't provide enough room or provide comfort. However, wedge heels are very secure and low to the ground and usually cost around $5.\nD. [step] Platform heels fit a variety of different sizes, so go online and find the one that is right for you. You can find both platform and wedge shoes on amazon, and if you can find them in storage, you can skip this step.","prompt":"[header] How to keep high heels from slipping [title] Learn your size for heels. [step] Many people wear different sizes for heels than they do for flats, which can make getting your size right tricky! Try on heels a half-size larger than your usual size to see if they fit more comfortably. [title] Buy platform or wedge heels.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6325,"error":null,"output":"B"}
{"question_id":"hellaswag:00006269","item_index":294,"attempt":0,"prompt_hash":"02300813411db8d92b2c0bded2bf9206e3d122714db64a566d551b732e024729","question":"[header] How to make cuts heal faster [title] Wash your hands. [step] Before caring for your wound, you'll need to make sure your hands are clean so you don't transfer bacteria into the cut. Make sure you go through the correct process of washing your hands to ensure they are as clean as possible.\n\nA. [substeps] Use warm or not-too-hot water and mild soap to wash your hands so you don't worsen the cut. Lather your hands under warm or not-too-hot water and scrub your hands for five to 10 seconds.\nB. [substeps] Make sure you clean the skin around any cuts or bruised areas. Put on a pair of clean medical gloves to protect your hands.\nC. Rinse your hands gently under running water to get rid of all of the particles and dirt. [substeps] Good washing practices include: washing your hands before and after touching a wound, or after vigorous activities such as cleaning the wound with soapy water.\nD. [substeps] Wet your hands with clean, running water. Apply soap and lather your hands by rubbing them together.","prompt":"[header] How to make cuts heal faster [title] Wash your hands. [step] Before caring for your wound, you'll need to make sure your hands are clean so you don't transfer bacteria into the cut. Make sure you go through the correct process of washing your hands to ensure they are as clean as possible.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6325,"error":null,"output":"D"}
{"question_id":"hellaswag:00006214","item_index":295,"attempt":0,"prompt_hash":"28ae3b14a8c07b0b562165b675f86d152ee8aa628612affb17461a57bd770957","question":"[header] How to set up an llc in nevada [title] Choose a name for your business. [step] Under state law, every llc must contain words that indicate its business structure. Specifically, the name must include the terms \" limited-liability company, \" \" limited liability company, \" or \" limited.\n\nA. \" alternatively, you may use one of the abbreviations \" ltd., \" \" l.l.c., \" \" llc \" or \" lc. \" [substeps] The name that you select must be distinguishable from any registered names of other existing businesses.\nB. \" [substeps] You can decide how you want your llc to operate, and if you're close to the state of nevada. State llcs will not use the \" limited liability company, \" \" limited liability company, \" \" limited professional partnership, \" or \" limited.\nC. \" these words represent your llc's target operating base. [title] Determine if your llc is a traditional business or a sole proprietorship.\nD. \" you don't need to define the full name by simply stating \" llc \" instead. [substeps] To begin the name process, begin by writing down the legal name for your llc and the language in which you expect it to be written.","prompt":"[header] How to set up an llc in nevada [title] Choose a name for your business. [step] Under state law, every llc must contain words that indicate its business structure. Specifically, the name must include the terms \" limited-liability company, \" \" limited liability company, \" or \" limited.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6325,"error":null,"output":"A"}
{"question_id":"hellaswag:00001852","item_index":296,"attempt":0,"prompt_hash":"bfb089bafc6cc9d8050155d95f9c9d7c02454fef8a13d20c38a6df3007d9993f","question":"Two teams play soccer in an outdoor field in front of an audience, the black and white team makes a goal and celebrates by smiling and clapping each others hands and waiving at the audience. several more goals\n\nA. are made by the black and white team, eventually one of the players wins the game and he gets congratulated by a white lady with blonde hair and her team mates, as is her trend.\nB. are made and even two teams continues to rally opposing teams on important events such as soccer.\nC. are made and by the end of each of them a score is made and people clap while walking away.\nD. are made by the black and white team on the soccer field before the scene fades.","prompt":"Two teams play soccer in an outdoor field in front of an audience, the black and white team makes a goal and celebrates by smiling and clapping each others hands and waiving at the audience. several more goals","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6325,"error":null,"output":"D"}
{"question_id":"hellaswag:00008680","item_index":297,"attempt":0,"prompt_hash":"a252261d2d99a61023dc0df72cfcaa81983eaa632bcea08e08b56aad4910df87","question":"[header] How to make a cheek highlight [title] Scrape out your choice of old eyeshadow, pigment, or highlighter. [substeps] The more material you scrape, the brighter and more pigmented your final product will be. [title] Obtain your choice of perfume sample or rubbing alcohol.\n\nA. [substeps] If you end up with a wet, watery consistency, add more eyeshadow/pigment/highlight. [title] Spray 3-4 sprays of the perfume sample into the material.\nB. [substeps] Since blush usually doubles as lipstick, two or three days before the lighter eyeshadow you use to darken your eyes, your perfume sample is good enough to work with. Swap out the sample for orange gel you want to use for something else.\nC. [substeps] Using an eyeshadow sample is a great alternative to an alcohol print. Examine the samples and see if they contain the same fragrances.\nD. [substeps] Basic eyeshadow samples are preferred, as there is no amount of liquid needed to finish it. A few drops of perfume is usually enough, although you could start with a two ounce amount.","prompt":"[header] How to make a cheek highlight [title] Scrape out your choice of old eyeshadow, pigment, or highlighter. [substeps] The more material you scrape, the brighter and more pigmented your final product will be. [title] Obtain your choice of perfume sample or rubbing alcohol.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6317,"error":null,"output":"A"}
{"question_id":"hellaswag:00001944","item_index":298,"attempt":0,"prompt_hash":"1446132fbd7b0904a921ea087512a285471f969ca3d46e39bc9dd690a43b5939","question":"The woman then scrubs soap into her hands and continues to wash them off then taking a paper down and drying her hands and sink. the other\n\nA. steps in to demonstrate how she washes her hands and ends by laughing to the camera.\nB. woman does the same with the paper and cleaners the papers with a large towel and wipes them well with her hand.\nC. woman takes a photo of the two baths with their pets and ends with the woman waving to the camera at the end.\nD. woman adds polish to her hands and uses a pencil to draw around her nails.","prompt":"The woman then scrubs soap into her hands and continues to wash them off then taking a paper down and drying her hands and sink. the other","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6318,"error":null,"output":"A"}
{"question_id":"hellaswag:00005955","item_index":299,"attempt":0,"prompt_hash":"2f191c2e9b18e69d87243c1827f6cb51de78fe268733221e9e9d388e4425958d","question":"[header] How to get ready to go out [title] Choose your footwear. [step] Pick shoes that match your clothing and are appropriate for the event. You should also know what's going to happen that night.\n\nA. [substeps] If you choose to do something formal, you should pick shoes that suit your style. High heels and dress pants are just a couple of choices.\nB. There are a lot of water hazards in the pool, so you need a few more pairs of shoes then go out. You should consider getting waterproof shoes that you can walk in without burning your feet.\nC. [substeps] Avoid footwear that is too thick or tight, or pants that are longer than the leg bone or calf. [title] Decide what to put on before you leave.\nD. If you'll be moving around a lot, for example, comfortable shoes are a good idea. [substeps] Dressier shoes can include heels, flats, dark leather shoes, and boots.","prompt":"[header] How to get ready to go out [title] Choose your footwear. [step] Pick shoes that match your clothing and are appropriate for the event. You should also know what's going to happen that night.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6313,"error":null,"output":"D"}
{"question_id":"hellaswag:00009048","item_index":300,"attempt":0,"prompt_hash":"fc7dd5749a2f403e7976a88eb06ef07c340551a70310a3c367a52f765bb35823","question":"[header] How to install linux on a chromebook [title] Verify that all your local data has been saved to google drive or to another form of storage, such as a usb flash drive. [step] Installing linux on your chromebook will erase and wipe all local data. [title] Press and hold down the \" esc \" and \" refresh \" keys, then press the \" power \" button.\n\nA. [title] Select \" networking \" option, then select \" type of network. [title] Using the keyboard shortcut, type in the name of the network that the chromebook is running.\nB. [step] Your chromebook may now have 32 gb of saved data. [title] Press and hold down ctrl and enter the boot sequence you extracted from your chromebook onto the computer.\nC. [step] Your chromebook will reboot and enter into recovery mode. [title] Press \" ctrl + d \" when the yellow exclamation point displays on-screen.\nD. [title] Select \" software class \" from the menu at the top of the screen. [title] Select \" ip/ipv2.","prompt":"[header] How to install linux on a chromebook [title] Verify that all your local data has been saved to google drive or to another form of storage, such as a usb flash drive. [step] Installing linux on your chromebook will erase and wipe all local data. [title] Press and hold down the \" esc \" and \" refresh \" keys, then press the \" power \" button.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6314,"error":null,"output":"C"}
{"question_id":"hellaswag:00005568","item_index":301,"attempt":0,"prompt_hash":"438c6b7cae1a1ed94dc1c691acfdcf9f031f41a3d0ee5495c898c8c7623c7fd9","question":"[header] How to change a cv axle [title] Remove the vehicle's hubcap. [step] Before so much as jacking up the vehicle, you need to remove the axle nut. Start by removing the hubcap for the respective wheel of the cv axle to be replaced.\n\nA. The easiest way to remove the wheel is by prying off the hubcap. Pull the hubcap off the wheel to reveal the axle nut and unscrew it with a phillips head screwdriver.\nB. If you can't get the number right, break off the hubcap. [substeps] In most vehicles there will be a hole around the wheel that you can remove with the phillips screwdriver.\nC. [substeps] You also need to remove the wheel nut, which clips the wheel to the axle of the vehicle. If the wheel nut is not removed yet you will need to remove some other parts of the vehicle before you can replace it.\nD. For vehicles with rims, you may simply have a cap at the center of the wheel rather than a hubcap. [title] Remove the cotter pin from the axle nut.","prompt":"[header] How to change a cv axle [title] Remove the vehicle's hubcap. [step] Before so much as jacking up the vehicle, you need to remove the axle nut. Start by removing the hubcap for the respective wheel of the cv axle to be replaced.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6314,"error":null,"output":"A"}
{"question_id":"hellaswag:00007299","item_index":302,"attempt":0,"prompt_hash":"27ea0621c122e004fd14b3d7a7b4b7cb7f0986658113ab9087d0b63bfde191a3","question":"[header] How to make a candy stripe friendship bracelet [title] Buy or collect your supplies. [step] For the four string weave, you will need four pieces of embroidery thread (sometimes called embroidery floss) in two different colors. You can find these at your local craft store, or in the craft section of most major retailers, like walmart or target.\n\nA. Look for two colors that complement each other, such as blue and green or red and orange. [substeps] You can also make an ombre bracelet with two shades of a color, like blue or red.\nB. You will be able to buy wire cutters if you want to customize the bracelet. [substeps] The simplest ring necklace pattern can be found online.\nC. You will also need three sets of stitches for each color of striped yarn. Once you have used all of the yarn, tie it into a knot.\nD. All you need is enough thread to tie your stringing together to form a bracelet. [substeps] You can also sew together your pieces of embroidery floss to create a long series of small shirts or flower-print themed scarves.","prompt":"[header] How to make a candy stripe friendship bracelet [title] Buy or collect your supplies. [step] For the four string weave, you will need four pieces of embroidery thread (sometimes called embroidery floss) in two different colors. You can find these at your local craft store, or in the craft section of most major retailers, like walmart or target.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6314,"error":null,"output":"A"}
{"question_id":"hellaswag:00000875","item_index":303,"attempt":0,"prompt_hash":"65c03517365820569e7ca96850068948b67f327b214f5b798ac8e9fd80b6a433","question":"The man performs several more lifts of the bell bending at the knees first and then hoisting the bell to the shoulders before throwing the weight on the floor. the man\n\nA. shakes hands and hugs the onlookers when through with the demonstration.\nB. jumps the high bar with both sneakers.\nC. holds the bell high before resting it on his chest.\nD. then lifts the bell over his head while remaining standing and and walks away.","prompt":"The man performs several more lifts of the bell bending at the knees first and then hoisting the bell to the shoulders before throwing the weight on the floor. the man","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6317,"error":null,"output":"D"}
{"question_id":"hellaswag:00010002","item_index":304,"attempt":0,"prompt_hash":"4e7bb03233c2843e1bd06129e1254e40da95762622749fce05f80b8f68c7d348","question":"[header] How to make a travel brochure [title] Choose the destination of your potential clients. [step] If you are a professional working for a travel company, your destination of choice will be the one you work for. If you are a student, and are creating a pretend travel brochure, you will want to pick out a desirable, exotic, and interesting location.\n\nA. [substeps] A professional should already know what destination he/she is representing, or attempting to advertise for. Use this step to get to know the key features of your location: mountains, lakes, cabins, museums, parks, etc.\nB. Your destination will probably be rural country or kia, though you may also specify that you are similar to the area where you're working and travel business and book an opportunity to be a client. [substeps] Target venues in particular regarding their experience and interests.\nC. [substeps] Focus on the amount of experience you have in the field you are researching. Again, the brochures have a tendency to get expensive and inexperienced people can take quite a bit of time to prepare.\nD. This will vary from one place to another, so it will ensure that where you work is well planned and free of major distractions. [substeps] Your destination might be in an amazon shopping center, local chamber of commerce, larger chain or even the state.","prompt":"[header] How to make a travel brochure [title] Choose the destination of your potential clients. [step] If you are a professional working for a travel company, your destination of choice will be the one you work for. If you are a student, and are creating a pretend travel brochure, you will want to pick out a desirable, exotic, and interesting location.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6316,"error":null,"output":"A"}
{"question_id":"hellaswag:00008004","item_index":305,"attempt":0,"prompt_hash":"8ff8472cdee115ed000cb43f9df555a618c81bf5433c52bed50a388de4661e13","question":"[header] How to store winter clothes [title] Machine wash or dry clean all of your winter items. [step] When stored for long periods of time, dirty clothing attracts pests and can also produce unwanted odors, mold, or mildew. Unclean items that contain perfumes, lotions, oils, and/or perspiration can also cause fabric to stain and yellow.\n\nA. Anything that smells like the clothes will likely go into the garbage. [title] Go through your wardrobe to find clothes that are suitable for winter.\nB. [title] Wash your clothing with mild detergents if you want to minimize odors. [step] If you're washing your clothing for a major event, like a holiday or graduation party, using less detergent may help minimize smells.\nC. [substeps] Keep in mind that harsh detergent units are safer than harsh dryers, so wash any clothing items before storing them in the washer. [title] Store winter clothing in a central space, like a closet.\nD. Before you pack up your winter wardrobe for several months, you should properly launder each article of clothing. [substeps] Machine wash all of your non-delicate winter items.","prompt":"[header] How to store winter clothes [title] Machine wash or dry clean all of your winter items. [step] When stored for long periods of time, dirty clothing attracts pests and can also produce unwanted odors, mold, or mildew. Unclean items that contain perfumes, lotions, oils, and/or perspiration can also cause fabric to stain and yellow.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6316,"error":null,"output":"D"}
{"question_id":"hellaswag:00001471","item_index":306,"attempt":0,"prompt_hash":"f409a1bc9ca029823837e4b157774a6b582eabddfef7c35d1cf5dc1ece0d6a87","question":"She strains her juice into a bigger cup. At an oven, she prepares simple sugar with water and sugar. she\n\nA. places the empty cup into the oven to cool.\nB. drizzles some more sugar into the cup.\nC. adds juice to it the rest of the time.\nD. mixes all of her ingredients in a pitcher to make lemonade.","prompt":"She strains her juice into a bigger cup. At an oven, she prepares simple sugar with water and sugar. she","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6315,"error":null,"output":"D"}
{"question_id":"hellaswag:00003040","item_index":307,"attempt":0,"prompt_hash":"64b7b66fc9c0f819dcc0596ddb179d703779d789ae000a3c4242ad7920a8536c","question":"High ocean waves are shown from a coast. a man surfing\n\nA. in a large wave and falls into the ocean.\nB. jumprope above the waves of the ocean.\nC. on the waves using a backpack and surfboard.\nD. is seen coming to his feet on his board.","prompt":"High ocean waves are shown from a coast. a man surfing","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6314,"error":null,"output":"A"}
{"question_id":"hellaswag:00009721","item_index":308,"attempt":0,"prompt_hash":"0ac0538a475007d459117f952d1e178b34b6231d45265556c816a2a85fd3e2e7","question":"[header] How to avoid male stereotypes [title] Complete a bias test. [step] Most people have some sort of bias. That means that you likely believe in some stereotypes, whether you know if or not.\n\nA. You can take an online test to see what your personal biases are. Harvard university offers a variety of free, online tests.\nB. However, many of them are based on coercion or misinformation. That means that you are unlikely to have your bias confirmed on the first try.\nC. In this case, you're better off thinking of stereotypes as facts rather than mere myths. Stereotypes are reality, not weeds and weeds.\nD. You may imagine that you're right-handed or left-handed, or both. All of those stereotypes need to not apply to you.","prompt":"[header] How to avoid male stereotypes [title] Complete a bias test. [step] Most people have some sort of bias. That means that you likely believe in some stereotypes, whether you know if or not.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6314,"error":null,"output":"A"}
{"question_id":"hellaswag:00003575","item_index":309,"attempt":0,"prompt_hash":"c19bfeba833ac0031832c0f2687de2617a52ccfe68e0c39544c97c6e2b053f0a","question":"[header] How to wear lace up pants [title] Select the most flattering waist for your body shape. [step] When shopping for lace-up pants, you will typically have a choice between low-rise, mid-rise and high-rise waists. Make your choice based on the style that looks the best with your body shape and what you feel most comfortable in.\n\nA. [substeps] Dresses and dresses with ruffles or fringe can help add volume to your figure. Choose a dress or blouse with ruffles to make your legs look as shapely as possible.\nB. For example, high-waisted pants tend to flatter most body shapes. Choose mid-rise if you want to play down the size of your lower half.\nC. High-rise will look best for a slimmer shape, while low-rise will look best for a more skinny or slender figure. [substeps] To determine your measurements, start with an allowance of around half an inch or less.\nD. A waist that is noticeable even at the top or bottom is not suitable for clothing that is too low. [substeps] Tight jeans in a neutral material tend to create too much bulk.","prompt":"[header] How to wear lace up pants [title] Select the most flattering waist for your body shape. [step] When shopping for lace-up pants, you will typically have a choice between low-rise, mid-rise and high-rise waists. Make your choice based on the style that looks the best with your body shape and what you feel most comfortable in.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6515,"error":null,"output":"B"}
{"question_id":"hellaswag:00002773","item_index":310,"attempt":0,"prompt_hash":"b5a803e90b5710681e8004bf8e11acf731fc3f8f08a74af3cbd4a7da8ec0c97e","question":"A man is sitting on a bull in a gate. The bull is released and a man on a horse chases after it. the man\n\nA. grooms the bull and ties its legs.\nB. is helped on the horse by another man.\nC. stands over the bull and hands him a spear.\nD. jumps off the horse and ties the bull up.","prompt":"A man is sitting on a bull in a gate. The bull is released and a man on a horse chases after it. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6515,"error":null,"output":"D"}
{"question_id":"hellaswag:00009420","item_index":311,"attempt":0,"prompt_hash":"86f4cb7b355bacb1bdd07349681c1687eef3dec0bd1c659376b03c4673debe1e","question":"[header] How to wear a pin [title] Add a pin to your collar. [step] Putting a pin, or several pins, along the neckline of your dress or blouse can add pizzaz to your look. If you have one pin, place it in the center of the neckline.\n\nA. A pin with buttonholes or a small rhinestone, such as a stencil or rhinestone, can pop out perfectly. [substeps] Keep these pins in a safe place, such as between your bra and underwear.\nB. [substeps] For example, if you have a jacket with a colorful wrap, place a pin onto it. You can wrap it around the inside edge of the jacket and pin it in place.\nC. If you have several pins, place them at the top, at the armpit or corner of the collar. Draw a comma around this pin, to serve as a base or extra brace.\nD. Alternatively, you can place the pin, or pins, off-center. Be sure to space them evenly and balance out the sizes.","prompt":"[header] How to wear a pin [title] Add a pin to your collar. [step] Putting a pin, or several pins, along the neckline of your dress or blouse can add pizzaz to your look. If you have one pin, place it in the center of the neckline.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6515,"error":null,"output":"D"}
{"question_id":"hellaswag:00000172","item_index":312,"attempt":0,"prompt_hash":"ea6385863a8e6cfffcb17e1e7fe47c9ceea25819078ae456d8dd89dac1cbaf75","question":"A woman in a suit is talking to a camera. A woman is putting contact lenses into her eyes. contact lenses\n\nA. are shown inside a case onto her nose, inside and outside.\nB. are shown on a hand.\nC. are being put into elongated pupil holes.\nD. are made and put in.","prompt":"A woman in a suit is talking to a camera. A woman is putting contact lenses into her eyes. contact lenses","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6514,"error":null,"output":"B"}
{"question_id":"hellaswag:00000813","item_index":313,"attempt":0,"prompt_hash":"2166f6921a39296575e57709d5ae2c1ba8ff696848bc2ae24d1632cf0510a590","question":"The girl ends her routing a little further from the point she started and firmly drops her hands to her side and does a little bow, then opens her legs shoulder length apart and goes back into the same position as she was in when she started. after she\n\nA. circles a couple times several more times, there is still no sign of her ending her gardingaing.\nB. is done holding the position for a while the girl relaxes and then walks away.\nC. finishes, the girl moves closer to the camera on her knees while holding a cylindrical object, and then straightens her legs out and stops her bow.\nD. is done with her camera, she begins talking to the camera again and looking at the screen before finishing.","prompt":"The girl ends her routing a little further from the point she started and firmly drops her hands to her side and does a little bow, then opens her legs shoulder length apart and goes back into the same position as she was in when she started. after she","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6515,"error":null,"output":"B"}
{"question_id":"hellaswag:00000894","item_index":314,"attempt":0,"prompt_hash":"8aaed716b58b1447b587e9a8e1b2e89c37e7b74ce4eb0e91fd2fa723959101d2","question":"We see the kids riding down the street from a skater pov. A kid crashes and has a hard time recovering. a boy\n\nA. falls across the other seat.\nB. loses his board and has to run after it.\nC. is outside pretending to be a hockey player.\nD. is on the ground doing tricks.","prompt":"We see the kids riding down the street from a skater pov. A kid crashes and has a hard time recovering. a boy","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6514,"error":null,"output":"B"}
{"question_id":"hellaswag:00005574","item_index":315,"attempt":0,"prompt_hash":"6fff9436cf08af871a96b9a6dc402ae9243f92b4c419f913b3354ebfcf6de209","question":"[header] How to buy youth soccer shin guards [title] Choose shin guards with attached ankle protection for players 12 years old and under. [step] There are two basic types of shin guards. One type has attached ankle straps, which wrap under the foot and provides support and ankle protection.\n\nA. The other type simply slides under your socks and protect just your shins. For players under 12 the added ankle support is crucial for safety and comfort.\nB. The other type is attached ankle guards that end under the buttocks. [substeps] He/she is often referred to as the \" baby shin sleeper, \" except that you can also use shin guards that wrap around the upper thighs as well.\nC. Other forms of shin guards have a raised band over them that shades the shin guard and prevents it from forcing weights to grow below the level of the shin guard. Various types of shin guards include : [substeps] The dutch support splint sports leg guards low knees support and support of the shin guard strength-training units military or individual strength-training units.\nD. Both types of shin guards are adjustable, so the handles will be easy to slide out of. [substeps] These are a good choice for active adults, older children, and those playing with a younger child.","prompt":"[header] How to buy youth soccer shin guards [title] Choose shin guards with attached ankle protection for players 12 years old and under. [step] There are two basic types of shin guards. One type has attached ankle straps, which wrap under the foot and provides support and ankle protection.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6628,"error":null,"output":"A"}
{"question_id":"hellaswag:00002671","item_index":316,"attempt":0,"prompt_hash":"7aaba86ce77c65e608f2e26edea87af91c39ee5523f7378b7caab03d3feecbc7","question":"A man in a black hat is talking to the camera. He starts playing a violin on his shoulder. he\n\nA. begins to play the violin.\nB. stops playing and puts the violin down to his side.\nC. continues playing the instrument on his neck.\nD. also continues to play like he just finished playing but then stops.","prompt":"A man in a black hat is talking to the camera. He starts playing a violin on his shoulder. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6630,"error":null,"output":"B"}
{"question_id":"hellaswag:00009427","item_index":317,"attempt":0,"prompt_hash":"79564ded0fdadb618d6221817b51240670b9e4cbafc76f24663f03ac0f4cfa83","question":"[header] How to share videos on facebook [title] Navigate to facebook and log in. [step] Locate your friend's video that you want to share. [title] Click the blue \" share \" button underneath the video.\n\nA. [step] This will take you to their profile page. [title] Hover over the \" account \" tab.\nB. [title] Enter the permission code you've set up for facebook. [step] Facebook will grant you permission to share the video as long as you accept it with consent.\nC. [title] Click \" share link \" in order to post the video for your friends to see. [step] Read more about editing the video information and privacy settings in the \" editing a video's privacy and information details \" section at the end of this article.\nD. [step] This should be located on the upper right hand corner. [title] Click the new video you want to share.","prompt":"[header] How to share videos on facebook [title] Navigate to facebook and log in. [step] Locate your friend's video that you want to share. [title] Click the blue \" share \" button underneath the video.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6629,"error":null,"output":"D"}
{"question_id":"hellaswag:00003334","item_index":318,"attempt":0,"prompt_hash":"f8fd17b38e751ce5c3331e24e9e9501e5cdf2aad1b2897a43cdb3eb4d1bbed02","question":"[header] How to repair damaged, curly hair [title] Get a trim. [step] Split ends can't be repaired, no matter what a product promises. In fact, split ends will continue to split up the hair shaft if they are not trimmed, so the longer you leave them untrimmed, the more damaged your hair will become.\n\nA. [substeps] You can fix split ends by following this technique: start from the ends of your hair and work towards the tips of your hair. Ensure that the split ends are about even.\nB. [substeps] Have a stylist trim off the ends of your hair once every six to eight weeks. If you absolutely cannot go to a stylist, use the sharpest scissors you have to snip off the ends.\nC. Therefore, you cannot avoid trimming your hair every day, and taking your height into account is essential. [substeps] Regular trims every few weeks or shorter are recommended, though those with fine hairs may be less needed.\nD. Either have it trimmed or trim once a week to maintain healthy, smooth hair. [substeps] If the split ends don't heal a lot fast, you can get a trim every 20-30 weeks to realign your hair with a less-damaging treatment.","prompt":"[header] How to repair damaged, curly hair [title] Get a trim. [step] Split ends can't be repaired, no matter what a product promises. In fact, split ends will continue to split up the hair shaft if they are not trimmed, so the longer you leave them untrimmed, the more damaged your hair will become.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6628,"error":null,"output":"B"}
{"question_id":"hellaswag:00007963","item_index":319,"attempt":0,"prompt_hash":"6aafc74001b2be7f64a3f59a58f92e55a22c936f6ec3e42eb50d043107df34e8","question":"[header] How to edit a short story [title] Come to the story with fresh eyes. [step] To edit the story effectively, you need to try to read it with a fresh perspective. You may need to take a break and put the story away for a few days so you can think about other things or other stories.\n\nA. By using the first few weeks of writing as a time to come to the story, you can identify some important discrepancies in the quality of the characters. [substeps] Try to avoid goops or costumes, since these can cause many of the scene changes to be confusing and overwhelming.\nB. Then, after a few days, pull it out again and read it to yourself with fresh eyes. You may be surprised at how different it reads to you after you have not been staring at it for several hours straight.\nC. Reading different short stories may help you uncover interesting key ideas and write stories based off of them. [substeps] Before you write a short story, read it over and over again.\nD. [substeps] Even if the original story was famously very bad, you can still read it a few times and figure out if something is off. Make sure you are really committed to revising the book before you take a break.","prompt":"[header] How to edit a short story [title] Come to the story with fresh eyes. [step] To edit the story effectively, you need to try to read it with a fresh perspective. You may need to take a break and put the story away for a few days so you can think about other things or other stories.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6629,"error":null,"output":"B"}
{"question_id":"hellaswag:00000001","item_index":320,"attempt":0,"prompt_hash":"e035482fe8bd677f5175b694a06d513a02e39eaca8f88ad4a2aea1bc799ff7a0","question":"A lady walks to a barbell. She bends down and grabs the pole. the lady\n\nA. swings and lands in her arms.\nB. pulls the barbell forward.\nC. pulls a rope attached to the barbell.\nD. stands and lifts the weight over her head.","prompt":"A lady walks to a barbell. She bends down and grabs the pole. the lady","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6299,"error":null,"output":"D"}
{"question_id":"hellaswag:00001246","item_index":321,"attempt":0,"prompt_hash":"63dfa2e87cd903687dcc79669d98c4ceda92645fb5e46c6c38a7948d775ec3cd","question":"He squats down kind of and picks it up, it looks very heavy and kind of painful. He lifts up above his head and jumps his feet apart. he\n\nA. stands straight up again and drops the weight down to the ground.\nB. drops the weight until just below his knees.\nC. lets go a cap fall off and lies down on the ground.\nD. drops the weights down and bends off then up again.","prompt":"He squats down kind of and picks it up, it looks very heavy and kind of painful. He lifts up above his head and jumps his feet apart. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6298,"error":null,"output":"A"}
{"question_id":"hellaswag:00001095","item_index":322,"attempt":0,"prompt_hash":"722342c9fe1cd399832b205d034a8aeb8f57295f511133425a1e47f668076e5d","question":"A young man with bleached white hair sits at a piano. he\n\nA. backs up to the piano and begins playing.\nB. plays it quickly taking small notes.\nC. plays the piano for the crowd while other people watch him.\nD. begins playing the piano enthusiastically.","prompt":"A young man with bleached white hair sits at a piano. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6298,"error":null,"output":"D"}
{"question_id":"hellaswag:00009827","item_index":323,"attempt":0,"prompt_hash":"50331677141d6f849fffcf72b1b985ef1a4630d7f3b7cb6c8e69b7ff82e0a9a1","question":"[header] How to paint a row of frisky bunnies in watercolor [title] Prepare your paper. [step] Cut a piece of 140 lb. Watercolor paper to approximately 9 \" x 12.\n\nA. \" hold it in either orientation and with an ordinary pencil, draw a line two-thirds of the way to the top. Make it curvy to represent hills or rises in the earth.\nB. Double and double cut two 9 \" x 12 \" squares. You will likely need to use two to three square pieces.\nC. Give it 5 to 7 days to dry completely. Then, lay the paper on a plate or table so that you won't spatter the paint on the area.\nD. [substeps] Do not use regular kitchen paper unless you are painting the legs of a flower. Inspect your washable paper to make sure it is all in place and the bars are not damaged.","prompt":"[header] How to paint a row of frisky bunnies in watercolor [title] Prepare your paper. [step] Cut a piece of 140 lb. Watercolor paper to approximately 9 \" x 12.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6298,"error":null,"output":"A"}
{"question_id":"hellaswag:00004760","item_index":324,"attempt":0,"prompt_hash":"5b206a3ec3cb4b5442e96962644a8d8cc5db92e8095d2367c530ded7cb4477d4","question":"[header] How to use wella toner [title] Go with t15, t11, t27, or t35 if your hair is naturally dark. [step] If your natural hair color is brown or black and you've recently dyed it blonde, there's likely a lot of orange in the strands. The lightest wella toners won't be able to cancel out the brassiness completely.\n\nA. [substeps] If you can't find a wella toner with t15, pick up a dha shampoo and conditioner. That's one of the more straightforward ways to apply the toner.\nB. Rather, they'll just trap the orange and yellow into the strands. [substeps] Even with this technique, however, you only need to squeeze the toner up into the roots, rather than down your curls.\nC. Instead, opt for a richer beige shade to lift your color. These shades are also great if you want hair that's light but not quite platinum.\nD. [title] Use pliable copper sulfate pastels. [step] Some wella toners also say that they can be durable and washable.","prompt":"[header] How to use wella toner [title] Go with t15, t11, t27, or t35 if your hair is naturally dark. [step] If your natural hair color is brown or black and you've recently dyed it blonde, there's likely a lot of orange in the strands. The lightest wella toners won't be able to cancel out the brassiness completely.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6292,"error":null,"output":"C"}
{"question_id":"hellaswag:00008753","item_index":325,"attempt":0,"prompt_hash":"a8c8a41061d7c18bdad365ce511758d879c743eda8ff00f799c8f355faba1d10","question":"[header] How to calculate fat calories [title] Read the nutrition panel on the food package. [step] The nutrition panel on food packages has all of the information that you will need to calculate calories from fat. The panel is black and white and it should say \" nutrition facts \" at the top of it.\n\nA. [substeps] Food labels often have information about how many calories are in each serving of the food, as well as the percentage of fat added. You may be unsure which group to test the food for, so ask your doctor if you are unsure what group to test.\nB. Find this panel and then find the number of fat grams per serving. [substeps] Write down how many grams of fat are in this food.\nC. The nutrition facts panel will give you the necessary information. [substeps] Food size: the amount of calories per gram of that food should be measured.\nD. [title] Classify fat based on just a simple phrase (like, \" body weight in kilograms \"). [step] In the upper right hand corner of the nutrition panel, you should see a \" calculate fat \" button or \" calculate weights \" button.","prompt":"[header] How to calculate fat calories [title] Read the nutrition panel on the food package. [step] The nutrition panel on food packages has all of the information that you will need to calculate calories from fat. The panel is black and white and it should say \" nutrition facts \" at the top of it.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6291,"error":null,"output":"B"}
{"question_id":"hellaswag:00004687","item_index":326,"attempt":0,"prompt_hash":"7b48113f210e34bf15cf035f88e449fde877cf6f39563452ddf50b93c8b41d6e","question":"[header] How to strengthen your relationship with your best friend [title] Communicate with your best friend regularly. [step] Communication is one of the most important aspects of any friendship. If you don't talk to your friend about your thoughts and feelings regularly, you won't know what's going on in the other person's head.\n\nA. [substeps] When you talk, make sure your voice is low and clear. Use a cool tone and don't give your friend the wrong impression.\nB. You'll only only be making it worse. Make sure to eat well, exercise, and be honest.\nC. [substeps] While talking to your best friend about personal matters is important, simply chatting about school or catching up on each other's lives is all part of communicating efficiently. If your friendship is long distance, utilize technology and social media.\nD. [substeps] Communicate with your best friend by calling or texting whenever you're both on the same page. If you're at a party or event, send flowers, drop by her home office, email her, or even call her to vent to her.","prompt":"[header] How to strengthen your relationship with your best friend [title] Communicate with your best friend regularly. [step] Communication is one of the most important aspects of any friendship. If you don't talk to your friend about your thoughts and feelings regularly, you won't know what's going on in the other person's head.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6291,"error":null,"output":"C"}
{"question_id":"hellaswag:00009301","item_index":327,"attempt":0,"prompt_hash":"12f67f765097ee552ced9012cc7d33b9d93dffbcf982490d8d65e854630e5463","question":"[header] How to create a spa kit [title] Get the right basket. [step] To start, you want the right basket for your spa kit. You do not want a relaxing spa day interrupted by fumbling to find items.\n\nA. Look for a basket with multiple compartments so you can separate items by type. [substeps] Before choosing your basket, think about what you're gong to include in your spa kit.\nB. [substeps] Be sure to choose a basket with a few compartments. For example, if your spa has a large pool, it will require a large basket.\nC. Pick out a nice, safe basket for your spa kit. Make sure it's big enough to hold just the essentials you will need without being too big to fit much.\nD. Your chosen basket should contain : [substeps] Flowers: the type of flowers you desire. You can buy a variety of, so decide your own style.","prompt":"[header] How to create a spa kit [title] Get the right basket. [step] To start, you want the right basket for your spa kit. You do not want a relaxing spa day interrupted by fumbling to find items.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6297,"error":null,"output":"A"}
{"question_id":"hellaswag:00001905","item_index":328,"attempt":0,"prompt_hash":"8c941236c1290a810f5ea984d41020454e9188b5e922194c02e6e66519acd99d","question":"A man on a horse chases after a cow with a ripe. A crowd around him watches. He catches the cow with the rope. he\n\nA. enjoys the time he has with the cow.\nB. jumps off the horse and ties up the cow.\nC. puts the cow down.\nD. then cuts ropes and ropes them as well as the cow.","prompt":"A man on a horse chases after a cow with a ripe. A crowd around him watches. He catches the cow with the rope. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6295,"error":null,"output":"B"}
{"question_id":"hellaswag:00002276","item_index":329,"attempt":0,"prompt_hash":"be385c9b3bd16e8c7383906d76919771ae77a30564b363bb817ba18aa49fa9a5","question":"Then a toddler get out the home, stand on the border of the swimming pool an falls in the water. the baby\n\nA. is shown then a man gets a disc across the plastic gate.\nB. floats face up, then a man comes and take the baby.\nC. laughs when his friend visit, then he tether ropes and drinks water from the water.\nD. lets go of the pole and runs, the man jumps in the water with an adult, and they are holding the puma.","prompt":"Then a toddler get out the home, stand on the border of the swimming pool an falls in the water. the baby","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6295,"error":null,"output":"B"}
{"question_id":"hellaswag:00003383","item_index":330,"attempt":0,"prompt_hash":"6764b3c3d6719214fb68e92aaa0f8deb41e18b125066304e86f5e4cab7c2a0f5","question":"[header] How to transfer money between bank accounts [title] Ask a teller to transfer. [step] The easiest way to transfer might be to stop into the bank and ask a teller to move some money around. Have your account information available and state how much you want moved and where.\n\nA. A teller may be open to letting you move funds anywhere you prefer and how much you want to transfer. This will allow you to transfer money from another account directly into the bank.\nB. You might need to fill out some slips at the counter. The teller should be able to help you.\nC. Ask the teller if there is a \" no processing fee \" fee for this service. [substeps] This fee can range from $25 to $75.\nD. [substeps] By moving your money, you are allowing your bank to know the funds being transferred. For example, in the us, the bank receives funds one month after a certain date as part of a maintenance check.","prompt":"[header] How to transfer money between bank accounts [title] Ask a teller to transfer. [step] The easiest way to transfer might be to stop into the bank and ask a teller to move some money around. Have your account information available and state how much you want moved and where.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6291,"error":null,"output":"B"}
{"question_id":"hellaswag:00002782","item_index":331,"attempt":0,"prompt_hash":"8f017e90df2ef53217c62daff7c8354aa2aff4f917fb48b7c06db32e8d57d71b","question":"A man pushes a lawnmower on a lawn. Two children help him push it. they\n\nA. help each other jump over a rock.\nB. tug the lawnmower up to a tree.\nC. make several passes on the lawn.\nD. brush the grass with a lawn tool.","prompt":"A man pushes a lawnmower on a lawn. Two children help him push it. they","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6293,"error":null,"output":"C"}
{"question_id":"hellaswag:00002718","item_index":332,"attempt":0,"prompt_hash":"6a59d84fc4d3b82adf106063c9c0d483e3ec61608b746743c29464043c226dcf","question":"The man then pulls on the fishing wire a few times and stands there holding his fishing pole, moving it, and winding it from time to time. the end of the pole\n\nA. is shown at the sidelines.\nB. is removed from the mans camp and he walks away.\nC. is now stretched out, the man grabs another one, and he and the woman take a break together.\nD. begins to bend and the man starts pulling the pole and winding it up and he pulls a fish out of the hole and it lands onto the ice.","prompt":"The man then pulls on the fishing wire a few times and stands there holding his fishing pole, moving it, and winding it from time to time. the end of the pole","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6295,"error":null,"output":"D"}
{"question_id":"hellaswag:00005107","item_index":333,"attempt":0,"prompt_hash":"ebb91cc2aae37b4dd8aed38b743d143f52cd439fe961e067d9b00802123c5789","question":"[header] How to make a home safe for toddlers [title] Use cabinet and drawer child latches to keep little hands out of poison and sharp items. [step] Keep heavy items locked away or well out of their reach. If oven doors are not lockable, buy a latch to keep them shut unless you want to watch your child at all times when baking.\n\nA. [title] Use childproof latches for household repairs. [step] They can be safely locked, or were given, because they' stay safe' at the end of a hallway or adjoining small room.\nB. [title] If you can, leave some items accessible to toddlers that are safe. [step] They can bang on pans or sort your plastic storage containers while you cook or clean.\nC. [title] Refrain from leaving your baby inside for long periods of time. [step] Children with small hands and easily handled hands can become allergic and can even choke on poisonous foods and vegetables.\nD. Lids are very fragile and we should avoid turning them on or checking the lock. [title] Use holes big enough to allow ventilation, light from the oven, and a childproof bulb for the door.","prompt":"[header] How to make a home safe for toddlers [title] Use cabinet and drawer child latches to keep little hands out of poison and sharp items. [step] Keep heavy items locked away or well out of their reach. If oven doors are not lockable, buy a latch to keep them shut unless you want to watch your child at all times when baking.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6292,"error":null,"output":"B"}
{"question_id":"hellaswag:00009341","item_index":334,"attempt":0,"prompt_hash":"9d7716b7d7e59b6c5f4593fd44dc3720b52d40848ea3cf6f28956c044b7d485f","question":"[header] How to tell if you are an introvert or an extrovert [title] Consider your reactions around people. [step] How do you act at a party, or around a large group of people? [substeps] If you tend to be more of a listener than a talker, you might be an introvert. You would be more of a wallflower than the life of the party.\n\nA. There is a fine line between being a wallflower and being an extrovert. Introverts tend to get emotional and get angry quickly.\nB. [title] Understand you may feel anxious, overwhelmed, or risky. [step] Extroverts tend to think in big numbers about what to say and what not to say.\nC. A shy or introverted person probably tends to think about immediate matters, such as preparing food and activities, and then letting things roll by. A nervous or introverted person probably might not like that too much.\nD. If you are more talkative, you are likely to be an extrovert. People are drawn to you because you smile more and are comfortable to be around.","prompt":"[header] How to tell if you are an introvert or an extrovert [title] Consider your reactions around people. [step] How do you act at a party, or around a large group of people? [substeps] If you tend to be more of a listener than a talker, you might be an introvert. You would be more of a wallflower than the life of the party.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6292,"error":null,"output":"D"}
{"question_id":"hellaswag:00006006","item_index":335,"attempt":0,"prompt_hash":"07e1d3e8c894ec5d8be35589675ed37a972e93315bcd17bf6e66072ee27f073d","question":"[header] How to have a good family life [title] Respect what your family members have to say. [step] When someone expresses an opinion, don't dismiss it or interrupt them before they can finish speaking. Maintaining open, respectful communication channels will help your family develop trust and strengthen your bond.\n\nA. If you feel confrontational or your family member dislikes something, don't hit them back or say something you won't regret later in the relationship. Instead, listen and make an effort to understand what that person wants.\nB. [substeps] Working together is essential to demonstrate to your family that you respect their opinions. [title] Avoid spreading private information to friends and family.\nC. [substeps] For instance, avoid making fun of your siblings whenever they express an opinion. If your siblings pick on you, try telling them, \" i know that all brothers and sisters mess with each other, but it hurts my feelings when you make fun of everything i say.\nD. [substeps] Only listen to your family members if they are completely honest. For example, you could say, \" kevin, i respect your opinion, but i don't agree.","prompt":"[header] How to have a good family life [title] Respect what your family members have to say. [step] When someone expresses an opinion, don't dismiss it or interrupt them before they can finish speaking. Maintaining open, respectful communication channels will help your family develop trust and strengthen your bond.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6293,"error":null,"output":"B"}
{"question_id":"hellaswag:00000337","item_index":336,"attempt":0,"prompt_hash":"bf91348c09ec7d0e7f2f2954f06d7a49dee1ade49379f997f5855f6e2afff8ae","question":"She takes a hose and wets the dog first. There is another little puppy running around the dog and a little kitten too. the woman\n\nA. takes some liquid soap and scrubs it on the dog.\nB. rinses the dogs mouth and the leash in the sink.\nC. walks the dog out the front door.\nD. then gets in a fight with the dog being pushed out of the way.","prompt":"She takes a hose and wets the dog first. There is another little puppy running around the dog and a little kitten too. the woman","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6293,"error":null,"output":"A"}
{"question_id":"hellaswag:00004952","item_index":337,"attempt":0,"prompt_hash":"ae9ec8ba56d471a70b755e48ced147b360ce43a99c911fdbb2aeec695c929f06","question":"[header] How to make a puff pastry christmas tree [title] Thaw the puff pastry and preheat the oven. [step] Turn on the oven to 400 degrees f (204 c). Thaw 1 full package (17. 3 ounces or 490 g) of puff pastry according to the manufacturer's instructions.\n\nA. Some puff pastry brands may require you to defrost the pastry first to make it easier to prepare. [substeps] For example, some puff pastry companies allow you to defrost the pastry by making several slits in the top.\nB. They may also be labeled as \" hard puff pastry. \" [title] Preheat your oven to 400 degrees f (204 c).\nC. [substeps] Ensure that the pastry is soft enough to handle, but not too warm. If the pastry is warm, it will be sticky and difficult to work with.\nD. Carefully roll out a sheet of pastry in a position that's way larger than the pie crust. [substeps] If the puff pastry isn't floppy enough, use a pastry cutter to score the pastry before slicing it.","prompt":"[header] How to make a puff pastry christmas tree [title] Thaw the puff pastry and preheat the oven. [step] Turn on the oven to 400 degrees f (204 c). Thaw 1 full package (17. 3 ounces or 490 g) of puff pastry according to the manufacturer's instructions.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6294,"error":null,"output":"C"}
{"question_id":"hellaswag:00006344","item_index":338,"attempt":0,"prompt_hash":"c200d431aeb80adf6329e2deb88b74661c3b70a4231de364c620a08e624873f6","question":"[header] How to put photos on an ipod video [title] Plug in and configure your ipod. [step] Be sure you are using an ipod that can store and display photographs. [title] Decide which photos you will store on the ipod.\n\nA. [step] You can add the photo to existing photos or you can install safari and download your work from there. [title] Save the picture as a ctf file.\nB. [step] For instance, if your ipod has photos of your friends and family, make sure that you put them back in where the people in your pictures are. [title] Ensure you have the proper headphones if you plan to play music.\nC. [step] You can sync from iphoto, from a folder on your computer, or from some other software. [substeps] For iphoto create a new album of the photos you will want on your ipod.\nD. [step] Some people put the images on one side, then the images on the other. You may also want to specify where you will use them on the ipod.","prompt":"[header] How to put photos on an ipod video [title] Plug in and configure your ipod. [step] Be sure you are using an ipod that can store and display photographs. [title] Decide which photos you will store on the ipod.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6295,"error":null,"output":"C"}
{"question_id":"hellaswag:00007974","item_index":339,"attempt":0,"prompt_hash":"126385631f341fd55e4cc7f69f022eca96c74f75eba0006fa82f33bec16fee06","question":"[header] How to brush hair with boar and nylon [title] Comb your hair before wetting it. [step] If you have a hard time getting your hair wet, try combing it under running water as well. [title] Use a shampoo that is not damaging for your hair.\n\nA. [step] This will help prevent water buildup in your hair when it is wet, and it will help ensure that your hair gets the foamy, frizz-free volume that it needs when it is dry. [substeps] Be sure that the shampoo is made specifically for your specific haircut.\nB. [step] Some shampoos dry your hair out, making it brittle and unmanageable. Massage your scalp gently, using back and forth motions rather than circular motions that twist your hair and break it.\nC. [step] Go for a lightweight shampoo because boar/nylon shampoo is very absorbent, which makes the hair really delicate and could result in premature bleaching. [substeps] Ask your hairdresser for advice, or they may be able to recommend one that is good for you.\nD. [step] Shampoo at least spritz on your roots/tips and do not shampoo before you shampoo. [substeps] Using an oil or coconut works best.","prompt":"[header] How to brush hair with boar and nylon [title] Comb your hair before wetting it. [step] If you have a hard time getting your hair wet, try combing it under running water as well. [title] Use a shampoo that is not damaging for your hair.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6293,"error":null,"output":"B"}
{"question_id":"hellaswag:00004907","item_index":340,"attempt":0,"prompt_hash":"7a5b40726677f76f25cd1f868e15e951184d569b376f22041d52a9505cea5d70","question":"[header] How to choose a tablecloth size [title] Consider the occasion. [step] If you're picking a tablecloth for an occasion, consider the nature of that occasion. Depending on the event type, there should be different levels of drop from the edge of the table to the bottom of the tablecloth.\n\nA. In order to maintain your tablecloth size from a shallow angle, a tablecloth covering should be about the same size as the table. This means the tablecloth should start at a corner if it measures a foot in length, and extend to a further corner if there are more chairs.\nB. [substeps] For a formal occasion, such as a wedding, you want a longer drop. Have about 15 inches from the edge of the table to the bottom of the tablecloth.\nC. For the event size, choose colors such as gold, silver, and dark blue. For a more casual dinner where you'll only be using perishable food, try bright colors such as bright green, red, and pink.\nD. If there is a necklace or other item from the event that's in your style, this should be found on the tablecloth. [substeps] If your dish will cost more money than the tablecloth you're picking, consider the nuances of the occasion while locating the tablecloth.","prompt":"[header] How to choose a tablecloth size [title] Consider the occasion. [step] If you're picking a tablecloth for an occasion, consider the nature of that occasion. Depending on the event type, there should be different levels of drop from the edge of the table to the bottom of the tablecloth.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6295,"error":null,"output":"B"}
{"question_id":"hellaswag:00000668","item_index":341,"attempt":0,"prompt_hash":"02bdf756b7bb1606337aacb35494d341f02e1833fa6ac071d61c79614c027242","question":"Various tools are shown laid out while a hand picks up the tools and shows them to the camera. He then grabs another tool and begins sharpening the knife next to some sliced apples. he\n\nA. then walks over and picks up one he was slicing.\nB. shows off the knife before holding it up.\nC. then cuts the apples with the sharpened knife and leads into another person holding and showing off more knives.\nD. continues sharpening the knife through the apples.","prompt":"Various tools are shown laid out while a hand picks up the tools and shows them to the camera. He then grabs another tool and begins sharpening the knife next to some sliced apples. he","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6936,"error":null,"output":"D"}
{"question_id":"hellaswag:00002901","item_index":342,"attempt":0,"prompt_hash":"4b7c349b5295705dab8f8ab5c5919506bf2ec38626cbdd844ec8b54bbc08e251","question":"A construction worker named dave begins to show you how to grout. he\n\nA. puts duct tape on the front of a white house.\nB. starts on the wall and begins to explain what he is doing and why he is doing.\nC. puts wax on a board and scrapes it off with a long sharpener.\nD. chops old wood and put it on the floor.","prompt":"A construction worker named dave begins to show you how to grout. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6933,"error":null,"output":"B"}
{"question_id":"hellaswag:00003437","item_index":343,"attempt":0,"prompt_hash":"f068e8360abed0304310609b8ec21400480d09fb7b704b0608efe95780aa22c6","question":"[header] How to take care of twins [title] Remember that you are now eating for three : [step] Yourself and your two babies. [title] Incorporate folate and folic acid (the synthetic form of folate) into your diet. [step] Folate is a b vitamin, which is an essential ingredient during a pregnancy as it can help prevent serious abnormalities of the brain and spinal cord, and neural tube defects.\n\nA. If you don't get enough folate in your diet, the risks of a preterm delivery are increased. [substeps] Whole grain cereals and trail mix.\nB. [title] Drink plenty of water; not too much, but enough, to quench your thirst. [step] And be sure to drink almost all sodas, sports drinks, or your non-carbonated beverage.\nC. Foods such as yogurt, fish and shellfish/fish milk are rich in these vitamins and folic acid. [substeps] If you are overweight or obese you can get more than three vitamins from food.\nD. [title] Add sources of fiber to your diet following a recent injection. [title] Be careful : [step] Sparkling water is recommended to prevent dehydration even during the pregnancy.","prompt":"[header] How to take care of twins [title] Remember that you are now eating for three : [step] Yourself and your two babies. [title] Incorporate folate and folic acid (the synthetic form of folate) into your diet. [step] Folate is a b vitamin, which is an essential ingredient during a pregnancy as it can help prevent serious abnormalities of the brain and spinal cord, and neural tube defects.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6935,"error":null,"output":"A"}
{"question_id":"hellaswag:00005279","item_index":344,"attempt":0,"prompt_hash":"3eb48923e5456969746d930f5559c0e57042f96da26d5f40e56154835750edc3","question":"[header] How to change google voicemail [title] Log into google voice. [step] Visit the, and enter your google email and address in the given text fields. Click the blue \" sign in \" button to proceed to your google voice account.\n\nA. [substeps] Note that if you are already logged into any of the google products, like gmail or youtube, google will not ask you to log in again. This is because google has made it possible for users to use one account across all of their products.\nB. [substeps] If you're not already signed in, select \" sign in \" at the top menu bar in the upper right corner. [title] Select the radio/cb slider.\nC. [title] Open the settings menu at the bottom right corner of the screen. [step] You'll see a camera icon in the middle of the screen.\nD. Click the \" create account \" button to create a google voice account. You will be directed to your google voice login page.","prompt":"[header] How to change google voicemail [title] Log into google voice. [step] Visit the, and enter your google email and address in the given text fields. Click the blue \" sign in \" button to proceed to your google voice account.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6933,"error":null,"output":"D"}
{"question_id":"hellaswag:00005390","item_index":345,"attempt":0,"prompt_hash":"964cee559f395f005bbe7a91edc839518417001871a11d708925011df97620fe","question":"[header] How to fold altar linens [title] Look at the linen. [step] The purificator is the smallest of the small linens, and the post communion veil is the largest of the small linens. Both can be either square or rectangular, and both have an embroidered cross in the center.\n\nA. The pectoral cross is a crescent with a series of embroidered figures and knots. The circular fold is the largest of all of the linens, but in order to fold an altar linens, you will need to go to the vatican website or the uuruses page.\nB. [title] Place one candle on the end of the cloth for blessing. [step] Candle boxes are made for light wicks, usually silver, but be sure to choose a larger one to suit your own preferences.\nC. [substeps] The purificator is the linen used to dry the holy vessels during holy communion. The post communion veil is used to cover the chalice after holy communion has concluded.\nD. The cloth can be patterned, embroidered, or printed by the purists, as you want the simplicity of the selections not strictly decorative. Choose a patterned or rectangular cloth by drawing the corners closed.","prompt":"[header] How to fold altar linens [title] Look at the linen. [step] The purificator is the smallest of the small linens, and the post communion veil is the largest of the small linens. Both can be either square or rectangular, and both have an embroidered cross in the center.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6935,"error":null,"output":"C"}
{"question_id":"hellaswag:00003940","item_index":346,"attempt":0,"prompt_hash":"f59c729b3360a499faaf19225e819b22cf404a33c15df3ac5c722e4e670a47c3","question":"[header] How to make a naruto headband [title] Find or buy blue fabric. [step] You can purchase your fabric from your local fabric store. If you don't have a fabric store near you, you can order it online.\n\nA. For simple layered indian tops, blue fabric works. Do not use muslin, as these will insulate the headband.\nB. It won't cost you the money that templates cost. [title] Measure and cut a 2-inch (5-cm) wide piece of fabric.\nC. [substeps] Fabric stores usually have a discount section. If you're on a budget, check there first.\nD. Red and black fabrics are good as well. Get your 4-way seams on your front and back piece, and 4 squares of fabric on your back piece.","prompt":"[header] How to make a naruto headband [title] Find or buy blue fabric. [step] You can purchase your fabric from your local fabric store. If you don't have a fabric store near you, you can order it online.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6934,"error":null,"output":"C"}
{"question_id":"hellaswag:00000106","item_index":347,"attempt":0,"prompt_hash":"e31cde8d4d02474495594ba331e173f7e3913cd1652e9ffb113eae5e400c5c38","question":"A shot of a sunset set is shown followed by peaking smoking and walking around a paint ball pit. The men speak to one another and lead into running around and playing paint ball. they\n\nA. are seen outside again with one man speaking and dancing in the same spot.\nB. jump and skydive into the water at various speeds, tossing each other around.\nC. pauses for a few moments to speak to one another and the camera captures the shooters from different angles.\nD. finish and walk into each other holding a pose and cheering.","prompt":"A shot of a sunset set is shown followed by peaking smoking and walking around a paint ball pit. The men speak to one another and lead into running around and playing paint ball. they","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6803,"error":null,"output":"C"}
{"question_id":"hellaswag:00003683","item_index":348,"attempt":0,"prompt_hash":"19a3cd73e44c36d6a5f37e2847caaf8b043f96bc88a377eb3f80baa9539ce805","question":"[header] How to prepare for the arrival of a newborn as a father [title] Take parenting classes. [step] Parenting skills, like any other ability, can be learned. Taking parenting classes can help you learn what you'll need to know and do as you prepare for the arrival of a newborn.\n\nA. Most classes cover changing diapers, bathing, burping, and feeding your baby, among other skills. [substeps] For a unique bonding experience, attend these classes with your partner.\nB. Parenting classes can be offered for a lower income and also provide training with advanced skills. [substeps] Look for parenting classes at your local adoption agency (foster care facility) or talk to a family doctor.\nC. Oftentimes, you'll find that you're good at parenting even when you aren't looking. [substeps] Various educational institutions and other institutions offer classes that educate you on parenting, including parenting and parenting skills.\nD. They can also give you perspective on the world, making this experience much better. [title] Learn more about parenting and parenting your baby.","prompt":"[header] How to prepare for the arrival of a newborn as a father [title] Take parenting classes. [step] Parenting skills, like any other ability, can be learned. Taking parenting classes can help you learn what you'll need to know and do as you prepare for the arrival of a newborn.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6742,"error":null,"output":"A"}
{"question_id":"hellaswag:00009689","item_index":349,"attempt":0,"prompt_hash":"90218d89e8c1cec4cf3387a3218049cfe4feff1ed77d28db7cebd7357ebf00a4","question":"[header] How to choose bridal shower favors [title] Go for color themed favors. [step] If you are having a bridal shower that has a theme based around a color, such as the bride's favorite color, have favors that match this theme. For example, if the theme of the shower is pink, you may create pink shower favors like a pink candy bag or a pink candle set.\n\nA. [title] Trim and fluff the favors as needed. [step] Get small, delicate, and circle shaped favors that are not prone to breaking or splitting.\nB. Having the favors match the theme of the shower can make it feel cohesive and well put together. [substeps] You may try setting a budget around the favor and then looking for items in a certain color for the bridal shower favor.\nC. You can also get favors made out of recycled material, as these will likely dye the favors. [substeps] You can also use colored fabric because you do not want to take color out of pink favors.\nD. Or if the theme of the shower is pink-themed, use pink water favors. [title] Decide how much money you will need to spend on favors.","prompt":"[header] How to choose bridal shower favors [title] Go for color themed favors. [step] If you are having a bridal shower that has a theme based around a color, such as the bride's favorite color, have favors that match this theme. For example, if the theme of the shower is pink, you may create pink shower favors like a pink candy bag or a pink candle set.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6742,"error":null,"output":"B"}
{"question_id":"hellaswag:00003503","item_index":350,"attempt":0,"prompt_hash":"c7b8233d75a28e775022ae9c9dd759e628aa69d57a68586f33f13d7d8dd00026","question":"[header] How to make an advent calendar from a cookie sheet [title] Place the cookie sheet face down on a flat surface. [step] Cover the work surface with newspaper or other cover to protect it from the spraying and painting. [title] Spray the back of the cookie sheet with a krylon (or similar brand) fixative.\n\nA. [step] Making it fairly abstract [title] Sprinkle flour onto the work surface. [step] Wet your hands and hold the flour at the base of the sheet, horizontally.\nB. [step] This forms a cosy atmosphere for the advent calendar. It also helps to prevent it from falling onto the desktop or wall where you will be laying up the calendar-if you have wallpaper backing, that could split the paper.\nC. [step] The fixative gives the paint something to stick to over the non-stick surface. [title] Cover the sheet with 1 to 2 sheets of paper.\nD. [step] You can spray the edges with any type of concentrated color, including dark or light reds, yellows, or vibrant blues. [title] Sprinkle aluminum foil over the cookie sheet.","prompt":"[header] How to make an advent calendar from a cookie sheet [title] Place the cookie sheet face down on a flat surface. [step] Cover the work surface with newspaper or other cover to protect it from the spraying and painting. [title] Spray the back of the cookie sheet with a krylon (or similar brand) fixative.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6742,"error":null,"output":"C"}
{"question_id":"hellaswag:00004312","item_index":351,"attempt":0,"prompt_hash":"548c23765d59b6856677a974a0a67c0c1707bae0fbd85d360af51b99652b3214","question":"[header] How to make a fork bracelet [title] Choose your fork. [step] Look to buy forks at garage sales or thrift stores. Usually, silverware sold at garage sales and thrift stores are relatively cheap.\n\nA. The fork that you buy also needs to be made of silver or sterling silver. That's because silver and sterling silver is very easy to manipulate and bend.\nB. If you're not sure what you're putting together, ask your cashier. [substeps] Check if the fork has 2 sides: the wider the better.\nC. However, having a fork for later use is a good idea. If you'll be using the forks multiple times, ask your friends to help you find forks.\nD. You can use your fork to make smaller, fancier rings, like the ones sold at thrift stores. [substeps] Choose a fork made of stainless steel.","prompt":"[header] How to make a fork bracelet [title] Choose your fork. [step] Look to buy forks at garage sales or thrift stores. Usually, silverware sold at garage sales and thrift stores are relatively cheap.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6801,"error":null,"output":"A"}
{"question_id":"hellaswag:00005875","item_index":352,"attempt":0,"prompt_hash":"8c689b833cdbc5fad88fe59032a9f41823c47244709abd92b59f3905742eadad","question":"[header] How to ease a child's fear of the water [title] Talk with your child about why they're afraid. [step] Understanding your child's fear is essential for helping them work through it. Ask your child what in particular scares them about the water.\n\nA. What places or people make them feel afraid? What events make them feel scared? What kinds of new noises are causing the water to dip? Not all children are afraid of water. Some people may fear water as a purely psychological condition.\nB. They might have been frightened as a child that they looked like they were going to drown, and see how their phobia of the water translates back to fear of the water when they're looking at a fish. Then, you can figure out if this fear of the water translates into fear of the sea.\nC. Often, their fears are related to water, in some way. Some common fears your child might have might include : [substeps] Water disturbances business problems brushes or accidents sudden changes in home location heat cycles fear of water [title] Monitor the temperature of the water.\nD. Are they afraid of sinking? Does it make them uneasy to watch bathwater go down the drain? [substeps] Many children develop a fear of the water after having a bad experience, such as accidentally getting water up their nose. Once you know where your child's fear is coming from, you can find ways to help them cope with the particular situation that scares them.","prompt":"[header] How to ease a child's fear of the water [title] Talk with your child about why they're afraid. [step] Understanding your child's fear is essential for helping them work through it. Ask your child what in particular scares them about the water.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4758,"error":null,"output":"D"}
{"question_id":"hellaswag:00001906","item_index":353,"attempt":0,"prompt_hash":"0e7118b0eecf1ac8920413685fb0dbe62e57c7aa2c5f41245ef0479fa544ef32","question":"He catches the cow with the rope. He jumps off the horse and ties up the cow. a couple of men\n\nA. run out to the field and undo the ropes off the cow.\nB. are next to the horse.\nC. are standing beside the horse.\nD. are walking outside the barn.","prompt":"He catches the cow with the rope. He jumps off the horse and ties up the cow. a couple of men","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4761,"error":null,"output":"C"}
{"question_id":"hellaswag:00003864","item_index":354,"attempt":0,"prompt_hash":"5d23ca5e49046a11d827c29ca91949875b2f6acba4adde47b0c87381abffd5dc","question":"[header] How to teach a child to sew [title] Know what age kids can start to sew. [step] Age, in general, doesn't matter. What matters is motivation.\n\nA. [substeps] Children can start to work with children around the age of one or two, as well as with their immediate family members. If they know how to sew, they are more inclined to learn.\nB. Kids learn with age-take a look. For instance, if you are teaching kids how to sew, you can instruct them on why they can wear a hat.\nC. Not all children are able to sew faster than a lot of their peers. If your child isn't ready for the project you will have to provide him with a use for his hands, it'll be tough for him to be able to keep up.\nD. If you're child shows an interest in learning how to sew, this is the indicator that they might be ready to start learning. [substeps] Some sewing instructors, however, recommend waiting till your child is around 6 years of age before introducing them to a sewing machine.","prompt":"[header] How to teach a child to sew [title] Know what age kids can start to sew. [step] Age, in general, doesn't matter. What matters is motivation.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4762,"error":null,"output":"D"}
{"question_id":"hellaswag:00006855","item_index":355,"attempt":0,"prompt_hash":"9ce440aa5343d93d8e5f100500b79fca23de4dccf706530c366d780ea5547105","question":"[header] How to make a pdf from indesign [title] Launch adobe indesign. [title] Open the document you want to export to pdf format. [title] Click the file menu and choose the export pdf option.\n\nA. [title] Select the file folder you want to export. [step] ( the default folder is your adobe documents folder.\nB. [substeps] Adobe indesign will open an export pdf window, allowing you to customize several exporting options. [title] Choose the appropriate file size in the export pdf preset dropdown selection box.\nC. [step] Click \" file \" at the right-hand side of the pop-up window and select the \" export as page \" option from the drop-down list of options you will find. [title] Select \" file] export as page.\nD. [step] You can choose to import the ebook, text or audio file that you want to export as a pdf. Make sure to save the file in a safe place.","prompt":"[header] How to make a pdf from indesign [title] Launch adobe indesign. [title] Open the document you want to export to pdf format. [title] Click the file menu and choose the export pdf option.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4760,"error":null,"output":"B"}
{"question_id":"hellaswag:00007899","item_index":356,"attempt":0,"prompt_hash":"7bc174eabdbfcba34b407ffaea370043458e36327e37117c781b3a61dfc8483a","question":"[header] How to deploy a computer [title] Back up your files find some external drive to duplicate your old files on to. [step] This step is only necessary if you want to keep the files on your old computer. If you have files saved on a lan (local area network) then you needn't worry about those as they can be moved when the computer is hooked up to the network.\n\nA. [title] Get your files hidden in a hidden location. [step] Unplug the cords from your computer and use a mouse to lift up your current secure folders (file).\nB. You can only run windows anywhere on the computer, so be sure to type in \" administrator/isp \". Then left click on the network folder (the one with the red' x' on it).\nC. You can just find the [portrait] button next to the computer's system tray. [title] Re-boot a computer from a usb drive.\nD. It doesn't matter what you back them up on, you could use anything from an external hard drive to 158 usb thumb drives, but a hard drive is best. Once you are all backed up you may move on.","prompt":"[header] How to deploy a computer [title] Back up your files find some external drive to duplicate your old files on to. [step] This step is only necessary if you want to keep the files on your old computer. If you have files saved on a lan (local area network) then you needn't worry about those as they can be moved when the computer is hooked up to the network.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4757,"error":null,"output":"D"}
{"question_id":"hellaswag:00003762","item_index":357,"attempt":0,"prompt_hash":"7d6a40afbc93c6248a3f0cdab821c616e0d080df9f486d5c695a8862bf442998","question":"[header] How to help a highly sensitive child [title] Give your child a peaceful and calm place to recharge. [step] Highly sensitive kids need time to settle down and regroup. They also tend to be quite shy, so they usually need some alone time.\n\nA. If you can provide alternate, quiet surroundings for them, their sense of calm could last them a lifetime. [title] Avoid putting a toy, such as a big stuffed animal, in another room.\nB. They may not be able to wait to be alone after an emotionally occurring event like a road accident. [substeps] A quiet, busy and relaxing place is ideal.\nC. Create a space in your home that offers a quiet, safe place where they can relax and feel comfortable. [substeps] Since highly sensitive children like soft surroundings, fill the space with pillows and blankets that are smooth and comforting.\nD. Taking them to a peaceful place, such as a park, is highly recommended. [substeps] Keep the area quiet so they will not feel isolated.","prompt":"[header] How to help a highly sensitive child [title] Give your child a peaceful and calm place to recharge. [step] Highly sensitive kids need time to settle down and regroup. They also tend to be quite shy, so they usually need some alone time.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4755,"error":null,"output":"C"}
{"question_id":"hellaswag:00005365","item_index":358,"attempt":0,"prompt_hash":"4b087e4ca530beab13b6309b2bdd7ba7e82b7d26ad3da1254290d130bdb4c978","question":"[header] How to make a hair mask with fenugreek seeds [title] Grind the fenugreek seeds. [step] For the mask, you need fenugreek seed powder. Place 2 tablespoons (22 g) of the seeds in a spice or coffee grinder, and grind them until they are a fine powder.\n\nA. [substeps] Ensure that the seeds are finely ground or packed with so that they work well in your hair. Sprinkle the seeds in a food processor to finely ground or powder the seeds.\nB. [substeps] Fenugreek seeds can be purchased in a spice or coffee shop. You can also make them yourself using oats, arrowroot, or allspice.\nC. [substeps] Fenugreek seeds contain turmeric and other spices that help to bind kava to hair. You can find fenugreek seeds at many big grocery stores.\nD. [substeps] You can find fenugreek seeds at many grocery stores, but if your local store doesn't carry them, visit an indian grocery, organic grocery store, or a health food store. You can also order the seeds from online spice retailers.","prompt":"[header] How to make a hair mask with fenugreek seeds [title] Grind the fenugreek seeds. [step] For the mask, you need fenugreek seed powder. Place 2 tablespoons (22 g) of the seeds in a spice or coffee grinder, and grind them until they are a fine powder.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4755,"error":null,"output":"D"}
{"question_id":"hellaswag:00003356","item_index":359,"attempt":0,"prompt_hash":"df219b5178b3167ba9d79a9789931f5569b30cf4b863d6ab539927942978c210","question":"[header] How to not have a big ego [title] Stop comparing. [step] Regardless if it is directed towards the positive or the negative, playing the comparison game can result in greater anxiety, depression, and poor decision-making. There are always two sides to a coin.\n\nA. Accept the fact that you have those things instead of trying to get rid of them. The other person may think you did a good job trying to get his or her thing, but neither can you.\nB. You may look at someone and say yourself \" i'm so much better at this than him \". It is just as likely that the other person surpasses you on some other trait.\nC. True, everyone's perspective cannot be completely altered if we do not see the positive in everyone we treat. So always try to see each situation differently.\nD. Improve if possible, and choose your side. [substeps] When you compare yourself to your ideal opponent, your mind automatically picks up on the inherent validation and positivity.","prompt":"[header] How to not have a big ego [title] Stop comparing. [step] Regardless if it is directed towards the positive or the negative, playing the comparison game can result in greater anxiety, depression, and poor decision-making. There are always two sides to a coin.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4754,"error":null,"output":"B"}
{"question_id":"hellaswag:00008820","item_index":360,"attempt":0,"prompt_hash":"9427225384fe4b013d527d291674835ab4d47a9dce6f3fcdf864c0dab89da5d3","question":"[header] How to talk to your hair stylist [title] Look for images of styles you like on others. [step] Remember that the perfect windswept look of a celebrity likely took hours to style. So look for styles in magazines or online that you feel would complement your face shape and have a certain cut or color you like.\n\nA. [substeps] Name your colors according to your hair and cut. Search for images that match your hair color, or you might even be able to find an image that showcases your natural hair color! [title] Use linkedin.\nB. [title] Find hairstyles with wave or updo styles. [step] The updo (and wave style) can complement more cuts and tends to be softer than the traditional updo styles.\nC. [substeps] There's no two sides to the coin-how stark you want your hair to look is entirely up to you. [title] Decide what style you would like to use.\nD. Focus on images of the style that show the front, back, and sides so you can show your stylist the complete look. [substeps] Be prepared for your stylist to suggest adjustments or alternatives to the looks you've chosen.","prompt":"[header] How to talk to your hair stylist [title] Look for images of styles you like on others. [step] Remember that the perfect windswept look of a celebrity likely took hours to style. So look for styles in magazines or online that you feel would complement your face shape and have a certain cut or color you like.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4755,"error":null,"output":"D"}
{"question_id":"hellaswag:00002653","item_index":361,"attempt":0,"prompt_hash":"afc27b342ff8c7f38208705ca275d3b1b8d7027d8f29534fc5acb83669a58919","question":"There are two guys standing on a large blue mat in a gym and they are both wearing shorts and a pair of boxing gloves. the men\n\nA. shake hands briefly and split up the room.\nB. begin doing arm wrestling and follow through in a boxing match until one of the boxers flips out.\nC. on the floor are fighting and they're using boxing gloves without backs up.\nD. square up and they begin hitting and kicking at one another with brief moments of stopping in between their punches and their kicks.","prompt":"There are two guys standing on a large blue mat in a gym and they are both wearing shorts and a pair of boxing gloves. the men","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4758,"error":null,"output":"D"}
{"question_id":"hellaswag:00004707","item_index":362,"attempt":0,"prompt_hash":"1179e4bf2fb2d5a2fed275591643afb04b21d485a4679033f9c96b5442efe920","question":"[header] How to treat ringworm in babies [title] Recognize the symptoms of ringworm. [step] If your baby has ringworm, he made have round rash spots on his skin. These rashes will be red or pink in color and have scaly and raised borders.\n\nA. Your baby may initially have rash spots; however, your child may begin to develop their symptoms within a few days. [substeps] Ringworm shows up on the baby's scalp, which results in a raised rash covering all skin.\nB. Have the baby touch the rash to see if it appears two or three months later, after the rash has disappeared. Early detection of ringworm is based on factors like the age of your baby and the bed environment.\nC. Here are the symptoms of ringworm in your baby : [substeps] Rash rash rash fever rash rash itching unceability, irritability, restlessness, and mood. [title] Recognize the symptoms of ringworm in the mother and baby.\nD. The center of the rash may also be scaly or it may be smooth. The rash spots usually are between half an inch to an inch; however, they will slowly grow larger.","prompt":"[header] How to treat ringworm in babies [title] Recognize the symptoms of ringworm. [step] If your baby has ringworm, he made have round rash spots on his skin. These rashes will be red or pink in color and have scaly and raised borders.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4753,"error":null,"output":"D"}
{"question_id":"hellaswag:00002791","item_index":363,"attempt":0,"prompt_hash":"0d469736c54a10662dd3ff9e22ea75c27b4a154b6f7c84eff1621950e5edb62d","question":"We see a man talk standing high above a city. We then see motorcycle stunts. A bike jumps a group of people in a plaza. we\n\nA. see people in a city square.\nB. see the ending title screen.\nC. see four bikes jump a ramp one after another.\nD. see people fall off and drive around.","prompt":"We see a man talk standing high above a city. We then see motorcycle stunts. A bike jumps a group of people in a plaza. we","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4756,"error":null,"output":"B"}
{"question_id":"hellaswag:00000424","item_index":364,"attempt":0,"prompt_hash":"eaae9abbb85865fd528eb799aff634d398900f462d42ff8539be99fa724cce73","question":"Skaters are skateboarding during the day. now the same skaters\n\nA. are skating during he night time.\nB. are skateboarding at night, in intensely bright light of day.\nC. are riding at night on residential streets with friends and families as they passes other skateboarders as they pass.\nD. are skating on the day but in the night.","prompt":"Skaters are skateboarding during the day. now the same skaters","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4755,"error":null,"output":"D"}
{"question_id":"hellaswag:00006383","item_index":365,"attempt":0,"prompt_hash":"9b43077aecc11a5a90396c073a0163c6de14c8406cd3ffb0fef127999c528306","question":"[header] How to weld lead [title] Prepare the lead for welding. [substeps] Before lead can be welded, it must be cleaned. This is especially important if you are replacing an existing piece of lead, such as an old pipe.\n\nA. Be sure you don't use chemicals. [title] Remove the lead from the pipe each time you weld it.\nB. Clean the lead thoroughly with a wet cloth before welding. It is important that the lead is clean when you start doing welding.\nC. Thoroughly rinse the lead with clean water prior to welding it. Prime and clean the pipe as you go.\nD. Use a sharp knife to remove any debris or crusty lead from the area where the lead will be welded. In addition, you may want to consider using a piece of steel wool to completely polish the area.","prompt":"[header] How to weld lead [title] Prepare the lead for welding. [substeps] Before lead can be welded, it must be cleaned. This is especially important if you are replacing an existing piece of lead, such as an old pipe.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4755,"error":null,"output":"D"}
{"question_id":"hellaswag:00000837","item_index":366,"attempt":0,"prompt_hash":"3e4a21bcc2c88e07938bf7b75ad73de5812ca234a9db132bdcc38e205b37b7c3","question":"A little girl is wiping water all over her face. she\n\nA. has a damp towel and under her eyelash.\nB. is sitting in the woods laughing.\nC. gets on front a woman.\nD. looks at the camera and smiles as she continues to wash her face.","prompt":"A little girl is wiping water all over her face. she","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4755,"error":null,"output":"D"}
{"question_id":"hellaswag:00007841","item_index":367,"attempt":0,"prompt_hash":"41cbb59665fe9d5ecf939d96b6bd551a1fdc7853de8bf0750da8ed006ad333ac","question":"[header] How to change a watch battery [title] Find the small indentation on the back of the watch. [step] Turn the watch over and find the small hole or indentation that's on the side of the watch, between the watch back and the watch itself. This indentation is created specifically to help you pry off the back of the watch.\n\nA. [title] Press down on the plastic cover of the watch with your thumb. [step] Only doing this can loosen the plastic.\nB. [substeps] Skip this step if you'll be removing the watch battery. You can reattach the watch battery later by using the small indentation.\nC. [substeps] If you can't find the indentation, scan the backside of your watch with a magnifying glass. Wear a pair of dust free latex gloves when replacing your battery.\nD. [substeps] Newer watches have small, soft indentation that's easy to penetrate. You might also need to connect your fingernail to one of the small holes on the back of the watch.","prompt":"[header] How to change a watch battery [title] Find the small indentation on the back of the watch. [step] Turn the watch over and find the small hole or indentation that's on the side of the watch, between the watch back and the watch itself. This indentation is created specifically to help you pry off the back of the watch.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4755,"error":null,"output":"C"}
{"question_id":"hellaswag:00001183","item_index":368,"attempt":0,"prompt_hash":"c743d6d2ddf741123a78c00c84eb9f2ec7dc8b7e2cce5ab240cb6de34ef662c8","question":"An introduction comes onto the screen for a video about working in the kitchen. a man\n\nA. introduces sitting on a stool to an introduction about a marinating dish.\nB. walks into the kitchen and begins putting dishes into a dishwasher and several clips of ingredients are shown.\nC. comes onto the screen and shows people how to best sharpen their knives while commenting on the best way to do it.\nD. takes a contact out of his eye while he watches the woman for instructions.","prompt":"An introduction comes onto the screen for a video about working in the kitchen. a man","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4755,"error":null,"output":"B"}
{"question_id":"hellaswag:00007473","item_index":369,"attempt":0,"prompt_hash":"5533284e4885b0bfc6fdbdb12fec3bfe99b57ba2c0a1322b83137dd70abcca41","question":"[header] How to sprout quinoa [title] Gather your supplies. [step] For this project you'll need a fine mesh strainer , 2/3 cup of quinoa seeds, a bowl, water, a tray or plate, cloth, and a sealed glass container or plastic bag. [title] Rinse the quinoa seeds in a bowl.\n\nA. [step] You need to wash the quinoa seeds to remove the soapy outer coating, which is known as saponin. Quinoa should always be rinsed prior to sprouting and cooking.\nB. [step] If your rinses for purposes of sprouting can be found at hardware stores, you can also purchase a mesh strainer for this project. [substeps] Fill the tray about halfway with fresh water.\nC. [step] Place the bowl into a strainer and remove any water that may interfere with the radish seeds. Rinse each seedling on a wire strainer for about 3 minutes, until no leaks are visible.\nD. [step] For the best results, rinse every single sunflower before trying to make it whole. Most quinoa should drain well into a tight strainer, but for ease of removing water, you can use your fingers or a sieve.","prompt":"[header] How to sprout quinoa [title] Gather your supplies. [step] For this project you'll need a fine mesh strainer , 2/3 cup of quinoa seeds, a bowl, water, a tray or plate, cloth, and a sealed glass container or plastic bag. [title] Rinse the quinoa seeds in a bowl.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4756,"error":null,"output":"A"}
{"question_id":"hellaswag:00007135","item_index":370,"attempt":0,"prompt_hash":"f7226c6724a3adb7ec0c4f0caf8716138802ea6dd2153c62af2e428bf4c170a2","question":"[header] How to fade eyebrows [title] Wash your face and towel dry your brows. [step] Use a spoolie brush to brush up your eyebrows. Note the natural shape of your brows and look for any spots where your brows are sparse or uneven.\n\nA. [title] Purchase a pencil liner and apply it along your brow bone. [step] The pencil liner should slide up and out of your brow bone like dark green eyeliner.\nB. Then use a fine-toothed comb to gently comb over their natural length and shape. [substeps] While a spoolie brush works well, many people actually prefer using a wide-toothed brush.\nC. Oil and makeup can smudge eyebrow hairs and free them from the natural shape they contain. If you have oily brows, use a small amount of highlighter to smooth the hairs out.\nD. [substeps] Pluck or wax any hairs that you don't want. Trim the top of your brows if necessary.","prompt":"[header] How to fade eyebrows [title] Wash your face and towel dry your brows. [step] Use a spoolie brush to brush up your eyebrows. Note the natural shape of your brows and look for any spots where your brows are sparse or uneven.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4754,"error":null,"output":"D"}