{"question_id":"hellaswag:00000731","item_index":0,"attempt":0,"prompt_hash":"0190bd41cc1f4bc5f9a51c1e8a04257e2ceb3f1380df79387a32828c5836c74a","question":"Maria is demonstrating how to make a nyquil cocktail drink. she\n\nA. shows the ingredients required for the drink.\nB. pours it through a straw into a glass and washes it down with a cup of soda water.\nC. shows each scale of the cocktail by adding different colored liquids.\nD. pours the ice in a small glass and squirts lemon juice into the glass.","prompt":"Maria is demonstrating how to make a nyquil cocktail drink. she","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4135,"error":null,"output":"A"}
{"question_id":"hellaswag:00006203","item_index":1,"attempt":0,"prompt_hash":"3f0139f1e050ab76196d2183c138f0fee3485d7f3d331f749ddfc3d456bbfe7f","question":"[header] How to zone agricultural land for use as a wedding venue [title] Talk to your neighbors. [step] Your neighbors can be your strongest supporters or your most bitter rivals if you want to rezone your property. The best way to potentially get them on your side is to keep them informed of your plans every step of the way.\n\nA. [substeps] In the early stages, let them know about your plans and reassure them that you will keep them informed on developments. Give them the chance to ask questions, and do everything you can to assure them that events you host won't cause any disturbances to them or their property.\nB. [substeps] Make sure you know your neighbors' lives and can help them out if they disagree about your plan. Give them written correspondence so they can be helpful to you if needed.\nC. This means that if your neighbors are anything but informed, you should talk to them as well. If you don't know them well enough to keep detailed notes, reach out to them and ask them to help keep a record of your plans.\nD. [substeps] Don't forget that some of your family's acreage may need transplant rights. Always decide ahead of time what kind of liens will be put on what land you are planning on and why.","prompt":"[header] How to zone agricultural land for use as a wedding venue [title] Talk to your neighbors. [step] Your neighbors can be your strongest supporters or your most bitter rivals if you want to rezone your property. The best way to potentially get them on your side is to keep them informed of your plans every step of the way.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4124,"error":null,"output":"A"}
{"question_id":"hellaswag:00002742","item_index":2,"attempt":0,"prompt_hash":"471f69d3a29f9a0f0a9022bd55d106f993dc7719a41b485362b25bf422dcc9b5","question":". a man with a orange shirt and blue gloves\n\nA. is standing in a snow covered parking lot talking while holding a skateboard and talking.\nB. is doing gymnastics on an elliptical.\nC. is shown operating a fire torch machine.\nD. approaches a green motor vehicle that is in a pool.","prompt":". a man with a orange shirt and blue gloves","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4135,"error":null,"output":"A"}
{"question_id":"hellaswag:00009360","item_index":3,"attempt":0,"prompt_hash":"bfa9c933b4b00eaf185cab9ca8e03ee66949bb77e56adb38a1cec98fcb428d08","question":"[header] How to get ready for a party [title] Prepare anything you need to bring. [step] If the party is a potluck, you'll want to make and bring a dish to share. If the party is a birthday party or some kind of shower, you'll need to bring a gift.\n\nA. If you are throwing a birthday party or if your parents are fine with it, feel free to bring these items. [substeps] In a party, use sachets or strings to write things in.\nB. If the party is a dinner party, you should bring a bottle of wine or a hostess gift. At more casual parties in high school or college, it's generally expected that guests won't bring anything unless it's been specified on the invitation.\nC. You'll also want to bring a good notebook and a pen so you can write down ideas on how to best decorate the house. A classic approach would be to bring plastic utensils, such as wine glasses.\nD. [title] Know what you're expected to wear. [step] If you're just going to school, you might wear a dress, but if you're doing other things, such as going to a dance, you might wear some pajamas or something a little fancier like a dress.","prompt":"[header] How to get ready for a party [title] Prepare anything you need to bring. [step] If the party is a potluck, you'll want to make and bring a dish to share. If the party is a birthday party or some kind of shower, you'll need to bring a gift.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4385,"error":null,"output":"B"}
{"question_id":"hellaswag:00004111","item_index":4,"attempt":0,"prompt_hash":"b6a1d249ead73813b74be6f20b6a654cac046faab5fda919cf9df29e5c7caefe","question":"[header] How to style mom jeans [title] Pick a pair of jeans that fit your waist well. [step] Mom jeans are meant to sit high on your waist. When picking out a new pair, try them on and see how the waist fits-you don't want the jeans too tight or too loose.\n\nA. Try on any pair and see if it's even snug. [substeps] If your mom jeans tend to dig into your legs, look for jeans that come up above the knee.\nB. [substeps] Pick out jeans that are at the leg length of your legs. If you have shorter legs, opt for what looks best to you, rather than what might look nice with longer pants.\nC. [substeps] The high waist will show off your waist and elongate your legs. [title] Choose a leg style that's appropriate for your figure.\nD. Try buying pairs that are so low that you can still walk in them but they are a little looser. [substeps] Ask your mom about how to size your jeans.","prompt":"[header] How to style mom jeans [title] Pick a pair of jeans that fit your waist well. [step] Mom jeans are meant to sit high on your waist. When picking out a new pair, try them on and see how the waist fits-you don't want the jeans too tight or too loose.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4124,"error":null,"output":"C"}
{"question_id":"hellaswag:00009418","item_index":5,"attempt":0,"prompt_hash":"3accc12092a01787575421d46046615ac35fa612e842369ae3b2a48cfb209742","question":"[header] How to diagnose heel spurs [title] Locate the pain. [step] Heel spurs can show up in multiple places on your heel. This can cause the pain to be slightly different depending on exactly where the heel spur is.\n\nA. It can be hard to judge the strength of the spur and whether or not it is painful at all. When you just try to be more specific about where it is, your foot and heel will become increasingly entangled and there will be no wiggle room.\nB. They can be located at the back of the heel or under the heel, near the sole of your foot. If you are experiencing pain along the back of your foot, up through your ankle, you might have a heel spur on the back of your heel.\nC. The first position you identify is the thigh. You will see white or pinkish swelling and inflammation along the arch of the heel.\nD. The most common places you might see the sensation of the pain are your heels and your nail. [substeps] You may discover that the pain is more acute in one heel then it is in another.","prompt":"[header] How to diagnose heel spurs [title] Locate the pain. [step] Heel spurs can show up in multiple places on your heel. This can cause the pain to be slightly different depending on exactly where the heel spur is.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4135,"error":null,"output":"B"}
{"question_id":"hellaswag:00004876","item_index":6,"attempt":0,"prompt_hash":"03f45b188edba7c0e32848019fe584d6bf2b7a9de012584f94328bc03650b9f6","question":"[header] How to dye deviled eggs [title] Hard boil the eggs. [step] Place 7 eggs into a saucepan, then cover them with 1 inch (2.5 cm) of water. Add a pinch of salt, then boil them, covered, for 10 minutes.\n\nA. This will help the eggs separate from the dye and flavor them further. [substeps] If the eggs take longer than 10 minutes to boil, wait until they're cool, then run them under cool tap water.\nB. Turn the heat off, and let the eggs sit in the saucepan for another 15 minutes; don't remove the lid. [substeps] If you have a different method for hard boiling eggs that you prefer to you, go ahead and use it.\nC. [substeps] You can use any type of seafood for this recipe, including fish, shrimp, bird, breast, and octopus. If your recipe calls for steamed eggs, you will need to cook on a low heat.\nD. Drain the eggs with a strainer and discard them. [title] Smash the egg shells with a potato masher.","prompt":"[header] How to dye deviled eggs [title] Hard boil the eggs. [step] Place 7 eggs into a saucepan, then cover them with 1 inch (2.5 cm) of water. Add a pinch of salt, then boil them, covered, for 10 minutes.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4126,"error":null,"output":"B"}
{"question_id":"hellaswag:00009212","item_index":7,"attempt":0,"prompt_hash":"c482b22d53d5fb7dc36dcd369bef9d400f3a346c38a7708234622376c158fc3a","question":"[header] How to maintain your baby's curly hair [title] Use a gentle, sulfate-free baby shampoo only once or twice per week. [step] Over-washing your baby's hair can cause dryness, resulting in dull curls. When you do use shampoo, avoid rubbing or tangling the curls.\n\nA. Massage the scalp gently to wash the roots only, avoiding the ends. [substeps] Curly hair has less moisture than straight hair, so you'll want to preserve the natural oils in the hair by using shampoo sparingly.\nB. Your baby's hair will end up with tangles and curl marks in it. [title] Dry your baby's hair with a clean towel.\nC. [substeps] Additionally, make sure your baby has a moisturized hair regime such as a straightening shampoo. This will help maintain a smooth, curly hair.\nD. If your hair is already a curl long, you can just squeeze the shampoo out of it. [title] Avoid using drying sprays, sizzles, or heated water as much as possible.","prompt":"[header] How to maintain your baby's curly hair [title] Use a gentle, sulfate-free baby shampoo only once or twice per week. [step] Over-washing your baby's hair can cause dryness, resulting in dull curls. When you do use shampoo, avoid rubbing or tangling the curls.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4133,"error":null,"output":"A"}
{"question_id":"hellaswag:00000155","item_index":8,"attempt":0,"prompt_hash":"8ee692ee5d56369bed4adddbd447818c63fd144c69cb8942a28284d5e9930cc5","question":"The harmonica is held up and shown. a person\n\nA. holds a harmonica between his hands a plays a song.\nB. is flying through the sky.\nC. runs up and grabs the guitar, then begins strumming.\nD. is strumming an electric guitar.","prompt":"The harmonica is held up and shown. a person","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4135,"error":null,"output":"D"}
{"question_id":"hellaswag:00002117","item_index":9,"attempt":0,"prompt_hash":"0684eb13b568d822a6ffa7d281dc510dc5e4b2301c4cafd0cac0840cbd99d7aa","question":"The woman points to a water splash on the counter next to the sink after turning on the water at the sink. the woman\n\nA. takes out a spray bottle and a contact lens.\nB. talks to the camera and washes her hands before rinsing her hands.\nC. turns on the water with a giant sponge, grabs soap and adds some drops of the water.\nD. then grabs a paper towel and wipes down the sink counter, the inside of the sink and the faucet spout.","prompt":"The woman points to a water splash on the counter next to the sink after turning on the water at the sink. the woman","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4137,"error":null,"output":"B"}
{"question_id":"hellaswag:00000009","item_index":10,"attempt":0,"prompt_hash":"30f747182587a4813043d8c7fdfe69393ee4745d9ba7ce6a30a61c9231c09391","question":"Then he takes a small stone from the flowing river and smashes it on another stone. He starts to crush the small stone to smaller pieces. he\n\nA. cuts the center stone in half and blow it on to make it bigger.\nB. grind it hard to make the pieces smaller.\nC. eventually brings it back into view and adds it to the smaller ones to make a small triangular shaped piece.\nD. starts to party with them and throw the pieces by hand while they celebrate.","prompt":"Then he takes a small stone from the flowing river and smashes it on another stone. He starts to crush the small stone to smaller pieces. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4136,"error":null,"output":"B"}
{"question_id":"hellaswag:00006104","item_index":11,"attempt":0,"prompt_hash":"9f0e118f1747c5f5babec19cf91c25c9a2bf868ff4be17411fb7517e28d14cbf","question":"[header] How to make your own wig out of weave [title] Maintain your hair. [step] Depending on your hair type, you may need special products to condition and treat your hair. From natural oils like coconut oil to specialty products like hair mayonnaise and a protein pack, use the right combinations that work to keep your hair healthy.\n\nA. Always clean your hair before going through your stylist or straightening it. He or she should be able to tell you what styles and tonings work best for you.\nB. A light amount of conditioner or oil is fine as long as you use it regularly. [substeps] Under-washing doesn't provide moisturizer as you don't want your hair to wear down and the texture of your wig may change.\nC. [substeps] Curly hair becomes spikier and will return more easily in the first 48 hours. Loose or wavy hair may need trims or mousse to keep it on and out of the way.\nD. [substeps] Consult a stylist or haircare specialist to find what works best for you. There are natural essential oils and mixtures that can keep your hair moisturized like, lavender and rosemary if you are concerned with using a chemical formula.","prompt":"[header] How to make your own wig out of weave [title] Maintain your hair. [step] Depending on your hair type, you may need special products to condition and treat your hair. From natural oils like coconut oil to specialty products like hair mayonnaise and a protein pack, use the right combinations that work to keep your hair healthy.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4133,"error":null,"output":"D"}
{"question_id":"hellaswag:00004337","item_index":12,"attempt":0,"prompt_hash":"4c39260a00d0332077e767b959749f22ab5a07248d29696f3f3b44af2e7e3387","question":"[header] How to advertise for wedding photography [title] Purchase a small newspaper advertisement in your local paper. [substeps] Start out by placing a print ad in your town's sunday paper. Newspaper advertisement returns have been on the decline for some time, but ad space can be relatively inexpensive depending on the paper.\n\nA. If you're looking for outdoor photography, try local paintshops or area recreation areas. They also have online sites to let you advertise on a regular basis, such as forums.\nB. Increase your ad space if you get a number of calls after your first attempt. This larger ad should provide an exemplary photograph and a range of contact information.\nC. Local newspapers are also a great place to post a brief ad--whatever type of photo the print advertisement contains--for your most popular wedding photography locations. [title] Advertise your photography in local newspapers.\nD. Customers walking by today are often drawn to newspapers where images are sold relatively cheaply. If you're a reviewer of the ads on the classified sections of your newspaper, sometimes the word \" wedding \" stands out.","prompt":"[header] How to advertise for wedding photography [title] Purchase a small newspaper advertisement in your local paper. [substeps] Start out by placing a print ad in your town's sunday paper. Newspaper advertisement returns have been on the decline for some time, but ad space can be relatively inexpensive depending on the paper.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4132,"error":null,"output":"C"}
{"question_id":"hellaswag:00003466","item_index":13,"attempt":0,"prompt_hash":"c127071fab94866da78f07c0d79f93ed389719051217078aea99df77d9efcc10","question":"[header] How to use ableton live [title] Install ableton live on your computer. [step] Use the provided disc or other means to install the program and get it fully functional on your computer and operating system. [substeps] Look at all system requirements for ableton live.\n\nA. Like many other modern sound programs, this program may include requirements for an advanced sound card, graphics display card or other piece of hardware, as well as requirements for minimum memory or processing speed. Make sure that your computer has what it takes to effectively run ableton live.\nB. Most operating systems require that you access basic operating systems, but some systems, such as windows or macos, require the installation of the software on a physical cd or dvd through a system drive. This method can take up to two months.\nC. That's just the nature of the program: it should also have 1 or more power requirements. Are you operating the program from a desktop computer? If so, take your computer to the computer store as this will determine software needs and install requirements.\nD. Identifying the relevant expiration date on your system can save time and expense. [title] Access the system application to which you need to install ableton live.","prompt":"[header] How to use ableton live [title] Install ableton live on your computer. [step] Use the provided disc or other means to install the program and get it fully functional on your computer and operating system. [substeps] Look at all system requirements for ableton live.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4127,"error":null,"output":"A"}
{"question_id":"hellaswag:00000147","item_index":14,"attempt":0,"prompt_hash":"7926134a4d779edd4c184f186e6897f722899c162441cf42600a36144d7ebcce","question":"People are standing on a hill of snow. People are snowboarding down a hill of snow. people\n\nA. are pouring snowballs into tubes as they go down the hill.\nB. are skiing down a roof of a house.\nC. are skiing down a hill.\nD. are standing on a hill holding stick.","prompt":"People are standing on a hill of snow. People are snowboarding down a hill of snow. people","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7722,"error":null,"output":"C"}
{"question_id":"hellaswag:00001356","item_index":15,"attempt":0,"prompt_hash":"3e8c97e1ca2496b0958f060c050cc6a8aacd96a9cea12676be2de6a0e39b59e7","question":"A lady licks an ice cream cone and holds a baby. The lady takes the pacifier from the baby and lets the baby eat ice cream. the baby\n\nA. eats pieces of the ice cream and mixes it in a bowl.\nB. makes a face and stare at the camera as she eats.\nC. isn't happy but he feeds it from the ice cream.\nD. is getting more ice cream, and the lady keeps licking his face.","prompt":"A lady licks an ice cream cone and holds a baby. The lady takes the pacifier from the baby and lets the baby eat ice cream. the baby","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4137,"error":null,"output":"D"}
{"question_id":"hellaswag:00005743","item_index":16,"attempt":0,"prompt_hash":"d38c76c56f5401abe91fe3eea23735307824255d33f16034b60f965fc34da064","question":"[header] How to melt hershey kisses [title] Unwrap the kisses and place them in a heat-safe bowl. [step] Hershey kisses come wrapped individually in thin foil. The standard milk chocolate versions, which are the best for melting, are wrapped in plain silver foil.\n\nA. Use store-bought, homemade kisses if you wish. [substeps] If you like homemade ones, wrap them in foil, though you may wish to crush the chocolate.\nB. [title] Fill a microwave-safe bowl with 4 ounces (200 ml) of water and 1 ounce (15 ml) of sugar. [step] You can microwave the dipped kisses for just a few seconds for extra sweetness.\nC. Warm the kisses gently in the microwave at least 30 seconds before opening and unfolding them or they may melt before melting. When you open and reunwrap your kisses, the chocolate will liquefy and melt.\nD. Unwrap each one and place them into a heat-safe bowl (like glass). If you aren't sure, check your bowl for a \" heat-safe \" label.","prompt":"[header] How to melt hershey kisses [title] Unwrap the kisses and place them in a heat-safe bowl. [step] Hershey kisses come wrapped individually in thin foil. The standard milk chocolate versions, which are the best for melting, are wrapped in plain silver foil.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4127,"error":null,"output":"D"}
{"question_id":"hellaswag:00002056","item_index":17,"attempt":0,"prompt_hash":"baf467773677e04a6fe5a8ca7759f24e08c7f8e42e88da1ccad01b7809c9207f","question":"Another person moves with the camera as they watch from a window in an elevated room against a white window sill. the person blowing the leaves\n\nA. gets closer to the camera as well as the person in the street behind him.\nB. gives a final look like he is talking and ends with him dropping the leaves as soon as he makes it to the ground.\nC. leaves are shown from a small angle.\nD. moves around the yard and continues to do lawn maintenance.","prompt":"Another person moves with the camera as they watch from a window in an elevated room against a white window sill. the person blowing the leaves","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4135,"error":null,"output":"A"}
{"question_id":"hellaswag:00005641","item_index":18,"attempt":0,"prompt_hash":"0c3cfa6ca8566393a5e6ebcc8ea857db171306cbfa10a240903e34edb09e07f8","question":"[header] How to go outside naked [title] Get comfortable with your body. [step] Most people aren't used to seeing themselves naked. In order to work up the courage to go uncovered outdoors, you first have to be confident in the way you look and feel.\n\nA. Additionally, you'll be aware of the physical aspect of being naked, so you can make an educated decision in the way that your skin feels. [substeps] Being naked isn't a punishment.\nB. When you are out naked, you will be known to be an alpha and you will show others that you can be just as confident. You will also be attracted to your surroundings because of this confidence.\nC. Make time for a few hours each day to watch yourself naked. [substeps] Even the most confident people always get naked after a workout or before bed.\nD. Take some time to get reacquainted with yourself in your most natural state, without judgment or self-consciousness. [substeps] Learn to love and accept your body.","prompt":"[header] How to go outside naked [title] Get comfortable with your body. [step] Most people aren't used to seeing themselves naked. In order to work up the courage to go uncovered outdoors, you first have to be confident in the way you look and feel.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4133,"error":null,"output":"D"}
{"question_id":"hellaswag:00004314","item_index":19,"attempt":0,"prompt_hash":"bfbdc5d32b48d330f04d857d1dcf2f2bddc3a87fb50b3b5998c8564121219974","question":"[header] How to get over a girl that left you for another guy (teens ) [title] Take her off the pedestal. [step] Best girl in the world? Never met, and never will meet another like her? It's common to feel this way, but trust us, it probably is not true. So let your friends talk trash about her.\n\nA. Ask your friends what went wrong, and vent the frustration. We love you for who you are, and there's no one better than you.\nB. There are a lot of teens who would at least look at where she is on a radar to make sure she hasn't left. Start with your friends, if you have the ball.\nC. Remind yourself of the annoying things she did. Do what you need to do to remind yourself that nobody's perfect, especially her.\nD. To get over her, you need to be prepared for the worst, because she is the one. Sometimes all you can do is make your own mistake.","prompt":"[header] How to get over a girl that left you for another guy (teens ) [title] Take her off the pedestal. [step] Best girl in the world? Never met, and never will meet another like her? It's common to feel this way, but trust us, it probably is not true. So let your friends talk trash about her.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7724,"error":null,"output":"C"}
{"question_id":"hellaswag:00009328","item_index":20,"attempt":0,"prompt_hash":"e64ab0715a3de9c8a396f2049c64251542207237fffd72b9499e7eb01aff8b94","question":"[header] How to make a rope braid with only two strands [title] Brush your hair. [step] Your hair should be smooth and free of tangles when you begin to make the rope braid. Use a brush or comb to thoroughly brush any knots or tangles out of your hair.\n\nA. You can use a moisturizing cream to smooth your hair out if it is not complying. [substeps] You can also dampen your hair with water to make it easier to put back in a ponytail and braid.\nB. If you have fine, fine hair, you can skip the brush or comb and just use the palm of your hand to tousle your hair into the braid. [title] Wrap a piece of cord in a cheesecloth.\nC. Your hair should also be relaxed and even. [title] Tie a few strands of hair in two bows.\nD. [substeps] Use a texturizing hairspray or mousse, which has been approved by the american academy of hairstylists. To make the rope braid, rub a water conditioner into your hair and smooth it down using a plastic comb.","prompt":"[header] How to make a rope braid with only two strands [title] Brush your hair. [step] Your hair should be smooth and free of tangles when you begin to make the rope braid. Use a brush or comb to thoroughly brush any knots or tangles out of your hair.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4133,"error":null,"output":"A"}
{"question_id":"hellaswag:00004965","item_index":21,"attempt":0,"prompt_hash":"12d1269c4b1c99fb68f92d8856d04161dd4215acfb0123e1474c3fb8780f2ac1","question":"[header] How to report vandalism [title] Call your local emergency services to report active vandalism. [step] Do this whether the vandalism is being committed against public or private property. If you catch someone in the act of spray painting a building, stealing a street sign, or otherwise destroying or vandalizing property, call your local emergency services number immediately.\n\nA. [title] Report the vandalism by reporting the vandalism. [step] This method will require no explanation of the vandalism.\nB. The us department of buildings (ftc) has a map at https: // www.ftc.gov/free-contractor/disputes /. [substeps] If you see an officer or official police report vandalism, don't get upset or panic.\nC. Destruction of property is illegal, and the authorities will want to catch the person committing the crime if at all possible. [substeps] In the united states you should call 9-1-1 to report vandalism in progress.\nD. [substeps] A 911 call is a fast, easy way to report vandalism. If you have been on the scene for a year or more, it may be worth calling your local report center.","prompt":"[header] How to report vandalism [title] Call your local emergency services to report active vandalism. [step] Do this whether the vandalism is being committed against public or private property. If you catch someone in the act of spray painting a building, stealing a street sign, or otherwise destroying or vandalizing property, call your local emergency services number immediately.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4125,"error":null,"output":"C"}
{"question_id":"hellaswag:00003513","item_index":22,"attempt":0,"prompt_hash":"1ba13e4e2363b4cdb58c36503dbb1c63deca40f63341a0a79e79863aab8f3997","question":"[header] How to cope with your parents' custody battle (for kids ) [title] Recognize that it isn't your fault that your parents are having problems. [step] You are not the cause. They may not love each other anymore, but they love you, and that's why they are in this custody battle.\n\nA. No matter what you think, your parents are fighting to uphold their rights and responsibilities. [title] Understand that actions can affect your consequences.\nB. If you are in your parents' custody battle, your parents are not the reason why they are fighting. [substeps] They are fighting because they are angry with you.\nC. Your friends could provide you a little moral support and that could be helpful. [title] Your parents feelings are not your responsibility.\nD. However, appreciate that by recognizing what your parents are arguing about, you know that your perspective is the only way you can do better in this situation. Try your best to understand their perspective so that you have a better chance of changing your mind about custody.","prompt":"[header] How to cope with your parents' custody battle (for kids ) [title] Recognize that it isn't your fault that your parents are having problems. [step] You are not the cause. They may not love each other anymore, but they love you, and that's why they are in this custody battle.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7725,"error":null,"output":"B"}
{"question_id":"hellaswag:00002538","item_index":23,"attempt":0,"prompt_hash":"88575cd27d425f2ce10a2a643ed348e2c42d76851830a650938e412b9c966e83","question":"Various pictures of javelin athletes are shown that lead into videos of the athletes throwing the pole. a man\n\nA. then steps up and demonstrates how to properly throw the pole while bouncing up and down.\nB. talks to the camera while holding a metal ball and talking to the camera and other athletes.\nC. is watching the entire performance while people are playing the game.\nD. is then shown attempting hand stands on the rope and runs to another location and is shown again over and over throwing the pole.","prompt":"Various pictures of javelin athletes are shown that lead into videos of the athletes throwing the pole. a man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4133,"error":null,"output":"A"}
{"question_id":"hellaswag:00009494","item_index":24,"attempt":0,"prompt_hash":"5e843113bcd1a6bd9a1daead118a8aa715db65c920a7e517f1ea911f7ed90af4","question":"[header] How to apply eyeshadow that lasts [title] Wash your face. [step] Starting with a clean and fresh face will make your makeup stand out more, help it last longer, and help prevent your makeup from clumping and flaking. Use your favorite gel or cream facial cleanser and lukewarm water.\n\nA. Rub the cleanser in circular motions for 20 minutes before rinsing with warm water. [title] Apply your eyeliner to the crease of your eye.\nB. [substeps] Wet your face and pour some cleanser into your hands. Work the cleanser into a lather, and then gently massage it into your face for about 30 seconds using circular motions.\nC. Tap your face with your fingers to get the excess off, then pat your face dry with a clean towel. [substeps] Do not splash water onto your face.\nD. Pat your face dry with a clean towel. [substeps] An eyeshadow primer also works just as well.","prompt":"[header] How to apply eyeshadow that lasts [title] Wash your face. [step] Starting with a clean and fresh face will make your makeup stand out more, help it last longer, and help prevent your makeup from clumping and flaking. Use your favorite gel or cream facial cleanser and lukewarm water.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7722,"error":null,"output":"B"}
{"question_id":"hellaswag:00004289","item_index":25,"attempt":0,"prompt_hash":"95e13448af781827d9e0890beb96f7fb57c6c42c7d8b4db802026f8ab029c54b","question":"[header] How to make a louisiana seafood boil [title] Cut the head off your shrimp with a sharp knife. [title] Devein the shrimp. [step] Your shrimp will have a natural arch to it so cut along the middle of the inner arch up to the tip of the tail where the intestinal tract is.\n\nA. [title] Place the shrimp in boiling water for 3-5 minutes to loosen them. [title] Place the shrimp into a pot of boiling water.\nB. From the cut, a black line (the shrimp's vein) will become visible. Remove the vein with your hands or a fork and wash the shrimp under running water.\nC. Don't cut on the gills yet. [title] Separate the insides of the shrimp by grasping the fleshy side with fingers that are pointing away from you.\nD. [title] Carefully cut out the intestines. [step] You may have to hack at the shrimp as you cut it.","prompt":"[header] How to make a louisiana seafood boil [title] Cut the head off your shrimp with a sharp knife. [title] Devein the shrimp. [step] Your shrimp will have a natural arch to it so cut along the middle of the inner arch up to the tip of the tail where the intestinal tract is.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7722,"error":null,"output":"A"}
{"question_id":"hellaswag:00007481","item_index":26,"attempt":0,"prompt_hash":"3be2e51ac085aff8679f619faa4406c72b0993ab16dfa54a6156522013f67e7b","question":"[header] How to replace ball joints and wheel bearings on a 1996 f150 [title] Jack up vehicle and place securely on jackstands. [title] Remove front tires. [title] Remove brake caliper.\n\nA. [substeps] Do not let the caliper hang by the rubber hose. Always either tie it to something that's not coming off, or find a nice place for it to rest that it won't drop off.\nB. [title] Dump jack stands and throw them out (if it was available) in a trash can. [title] Inspect ball joints and wheel bearings for possible damage or cracks.\nC. [title] Remove alternator and module before applying fluid. [title] Remove tire, wiring, brake pads and radiator backing.\nD. [title] Replace wheel spring and brake support with new tire springs. [title] Replace wheel suspension spring and transmission plate spring with new spring springs.","prompt":"[header] How to replace ball joints and wheel bearings on a 1996 f150 [title] Jack up vehicle and place securely on jackstands. [title] Remove front tires. [title] Remove brake caliper.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4136,"error":null,"output":"A"}
{"question_id":"hellaswag:00003354","item_index":27,"attempt":0,"prompt_hash":"53de7e17fc51825b2169dab5215bb3077ded375fcdc51ffaba20f32d2ae96b7f","question":"[header] How to adopt a child in missouri [title] Assess your situation and preferences. [step] You have many options, and the type of child you want will drive who you work with during the adoption process. Spend some time considering the following : [substeps] Do you want to adopt a child in the u.s.? This is called a domestic adoption.\n\nA. You can adopt a child in the u.s. you can adopt the child in the u.s.\nB. However, you might have to wait a long time for a child to become available. Do you want to adopt a child internationally? You might have a better chance at finding a child, but international adoptions can be complicated and more expensive.\nC. You can adopt a child at the start of an adoption process, where the adopting party adopts a minor child in a different state. They provide a temporary home for the minor child, then return the child to their home country and select a permanent home from there.\nD. You are responsible for raising the child and supporting him or her through the adoption process. You would want to adopt a child under the age of two.","prompt":"[header] How to adopt a child in missouri [title] Assess your situation and preferences. [step] You have many options, and the type of child you want will drive who you work with during the adoption process. Spend some time considering the following : [substeps] Do you want to adopt a child in the u.s.? This is called a domestic adoption.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4125,"error":null,"output":"B"}
{"question_id":"hellaswag:00001371","item_index":28,"attempt":0,"prompt_hash":"260dc3775e684d7e6a2902a2a0486c0a2fb5c92fd838439989f157dd2a45066c","question":"He moves a can and touches two other cans. He picks up a can and sprays it on the woman's arm and rubs it. he\n\nA. picks up another can and talks about it then sprays her arm again and rubs the cream in.\nB. rubs foam on her arm in a circular motion that includes her eyes.\nC. leans over with his arm out and the knife enters the sleeve of her gown.\nD. then moves the can and wipe it off a row of cans.","prompt":"He moves a can and touches two other cans. He picks up a can and sprays it on the woman's arm and rubs it. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4136,"error":null,"output":"A"}
{"question_id":"hellaswag:00004689","item_index":29,"attempt":0,"prompt_hash":"a46ee2fd069ae3d865d70abbb046e4e5699125ea0ad338e8d4b2bc4ed8be4857","question":"[header] How to make gelato [title] Fill a shallow pan with water and bring it to a simmer. [step] The pan should be large enough to hold a smaller heatproof bowl. [title] Separate the yolks from the whites.\n\nA. [step] This will divide the yolks in half. [title] Add the flour , 1 tablespoon (15 ml) at a time; these should be separated from one another.\nB. [step] Sift them into a small container. [title] Add the other ingredients (optional).\nC. [step] Place 2 bowls or other containers on a flat, clean surface. One container is for the egg whites and the other is for the yolks.\nD. [step] While each piece of yolk can go in small batches, there are two basic ways to separate them. [substeps] For instance, you could break each yolk into two separate bowls, so you can place the yolks into one large bowl and the whites into another.","prompt":"[header] How to make gelato [title] Fill a shallow pan with water and bring it to a simmer. [step] The pan should be large enough to hold a smaller heatproof bowl. [title] Separate the yolks from the whites.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4125,"error":null,"output":"C"}
{"question_id":"hellaswag:00008127","item_index":30,"attempt":0,"prompt_hash":"a0464069f50dfe0e2f200d26ee617fa4c999c9a56fdcf900b02877e90d813197","question":"[header] How to host an english style high tea [title] Pick a time. [step] Consider your guests when choosing a time. People often host tea during the afternoon.\n\nA. Work and/or school may be a problem during the afternoon on weekdays, however. A weekend afternoon tea could be ideal if you let your guests know about the date you choose in enough time.\nB. The most popular time is slightly below 10pm so it would work to host the group for one hour. However, if, at the end of that time, you want your group to be very close, then a mid afternoon tea can also work well.\nC. Choose a time that is convenient for you and your guests. [substeps] For example, you might invite your friends and family over for tea at 8 pm.\nD. If people you are inviting are already well entrenched in local politics, religious issues, or other matters, pick the right time to host the tea. [substeps] Consider setting up an informational meeting or meeting in the afternoon.","prompt":"[header] How to host an english style high tea [title] Pick a time. [step] Consider your guests when choosing a time. People often host tea during the afternoon.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4135,"error":null,"output":"C"}
{"question_id":"hellaswag:00004936","item_index":31,"attempt":0,"prompt_hash":"c411c63884ab44f8dc7000e91549b99c35ba8f9d467f0057d5a864ccddad2e32","question":"[header] How to get in shape for volleyball [title] Perform lunges. [step] Start from a standing position. First, straighten your back and roll your shoulders back and down to keep your core engaged.\n\nA. Breathe in and out slowly while in the lunge position, keeping your abs tight to your body. [substeps] Brace your feet and place your hand on your hips with your fingers pointing in front of you.\nB. Raise your chin and fix your eyes at a spot directly in front of you, at eye-level. Then : [substeps] Take a step forward with one leg so that ankle is directly below your knee, with your calf and upper thigh forming a right angle.\nC. After recovering, raise up one leg with your heel slightly facing forward. Once you are standing, point your toes so that your heels are pointing towards a ball and continue working on getting the muscles in your core engaged.\nD. After you've got the hang of this, try to do two sets of lunges each on each side of your body. [substeps] To extend your lunges, jump slowly and take one lunge while keeping your feet squarely on the ground.","prompt":"[header] How to get in shape for volleyball [title] Perform lunges. [step] Start from a standing position. First, straighten your back and roll your shoulders back and down to keep your core engaged.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4126,"error":null,"output":"B"}
{"question_id":"hellaswag:00007509","item_index":32,"attempt":0,"prompt_hash":"277a43c1e121b3e6c313c70bfe76f01428df0c3397b8f723672d69e037f3581a","question":"[header] How to start dreads [title] Section your hair into small squares. [step] You'll need a friend to help with this method: they should use their hands or a comb to part your hair into multiple small squares. It's up to you how many squares your friend makes.\n\nA. Each square will become a single dreadlock, and smaller squares make thinner dreads. Generally , 1-or 2-inch (2.5 or 5 cm) squares are preferable, though it depends on what you want.\nB. [title] Use your hands to create food-sized strips from the hair you're affixing to dreads. [title] Feed the strips at medium frequency until your hair is disconnected from the dreads.\nC. A small square will fit around the core but large squares will fit on top. [substeps] Some people prefer this method because you have to supervise the center of the hair, tuck at an even rate, and it also gives them a bit more control with how often they use hair as they work.\nD. After a few minutes of doing this, switch it down to your roll curling method. [substeps] If you're struggling with how to determine how many squares you should curl, consult your friend's instructions.","prompt":"[header] How to start dreads [title] Section your hair into small squares. [step] You'll need a friend to help with this method: they should use their hands or a comb to part your hair into multiple small squares. It's up to you how many squares your friend makes.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6290,"error":null,"output":"A"}
{"question_id":"hellaswag:00002037","item_index":33,"attempt":0,"prompt_hash":"2d3a19d445ca840e40bc9739bd9a405ab24d262fd71ab87002bbf9217f06a71e","question":"A person is seen running onto a court and shooting a basketball. the man\n\nA. tosses the shot put to another player while the score of the game is shown.\nB. keeps throwing the balls as well as from behind while the camera takes pictures.\nC. is shown in several clips running down and scoring a basket.\nD. runs down a track and catches the ball and throws it off in the distance.","prompt":"A person is seen running onto a court and shooting a basketball. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6295,"error":null,"output":"C"}
{"question_id":"hellaswag:00007831","item_index":34,"attempt":0,"prompt_hash":"9a1d8ddf1d6cea9f129bc342cc07339167efb522fb0afce56916359c21647536","question":"[header] How to apply eye black for baseball [title] Remove the eye black stick from its package. [title] Take off the lid to the eye black stick. [step] A lid to eye black is removed much in the same way you remove a lid from a stick of lip balm.\n\nA. [title] Apply the eye black to your face. [substeps] Place the stick on the outside of either eye.\nB. [substeps] Lipstick is removed from the mouth prior to applying makeup. If you don't want to remove the lid, use either a q-tip, toothbrush or mini bottle.\nC. Go with the universal method: simply remove the lid. [title] File the top of the stick onto the lid.\nD. The lid may fit in the palm of your hand, but it will slide across your eye to one side. [substeps] If your lid comes in a bottle or small round plastic container, make sure you grab that first.","prompt":"[header] How to apply eye black for baseball [title] Remove the eye black stick from its package. [title] Take off the lid to the eye black stick. [step] A lid to eye black is removed much in the same way you remove a lid from a stick of lip balm.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6290,"error":null,"output":"A"}
{"question_id":"hellaswag:00009218","item_index":35,"attempt":0,"prompt_hash":"af713bc1b1ba3d585eef8d67eca0e8fdae547a7bc3aaa9f0d6086e9226cc43f1","question":"[header] How to touch up car paint [title] Buy a matching touch-up paint color. [substeps] Check the firewall (bulkhead) to find the paint code number. The firewall is the piece of sheet metal that separates the engine under the hood from the passengers inside the vehicle.\n\nA. The number will also designate the name of the engine you will be touching up. [title] Call your locksmith to make a custom primer for your car.\nB. If you don't have a matching touch-up paint code, observe the color coding in the firewall. This is usually indicated with a red and white square at the bottom of the panel.\nC. You will need to open your hood to find this number. Purchase a primer along with the touch-up paint unless the particular touch-up paint you've chosen states otherwise.\nD. The code is imprinted along the bottom of the firewall, beginning at the fully hot brights. [title] Look for the manufacturer's matching logo under the engine compartment (backside).","prompt":"[header] How to touch up car paint [title] Buy a matching touch-up paint color. [substeps] Check the firewall (bulkhead) to find the paint code number. The firewall is the piece of sheet metal that separates the engine under the hood from the passengers inside the vehicle.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6291,"error":null,"output":"C"}
{"question_id":"hellaswag:00003483","item_index":36,"attempt":0,"prompt_hash":"52048abedda98289af9291fdb107a1018169e46a91966e1763cef9054694442e","question":"[header] How to recognize obsessive compulsive disorder in children [title] Don't jump to conclusions. [step] Remember that children have quirks and often go through phases that may make you wonder whether they are normal. If you're concerned that your child may have any kind of mental disorder, it's best to talk to a pediatrician or a child psychologist before you try to diagnose a disorder on your own.\n\nA. [substeps] Your child may have borderline obsessive disorder. This may include a learning disorder, an inability to think and be articulate, or some sort of cognitive impairment.\nB. Your first instinct is to freak out when your child does something that is completely out of character. Your children may be pansensitive, compulsive, or make mistakes.\nC. If you have had your child evaluated and are still unsure, don't be afraid to get a second opinion. [title] Look out for signs of obsessions.\nD. [title] Consider the symptoms and reactions of your child. [step] Many children with ocd have behaviors that others find unappealing.","prompt":"[header] How to recognize obsessive compulsive disorder in children [title] Don't jump to conclusions. [step] Remember that children have quirks and often go through phases that may make you wonder whether they are normal. If you're concerned that your child may have any kind of mental disorder, it's best to talk to a pediatrician or a child psychologist before you try to diagnose a disorder on your own.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6290,"error":null,"output":"C"}
{"question_id":"hellaswag:00001966","item_index":37,"attempt":0,"prompt_hash":"0167986ca07154d4e5f447fe24a8123cb1525c3db4e19fb5a8e0f585bcd46276","question":"A man is sitting in a barber chair. Another man is giving him a hair cut. he\n\nA. takes the hair over the scissors.\nB. then has him cut and styled.\nC. shows off his trimmed hair from a man sitting behind him.\nD. then brushes off the mans neck.","prompt":"A man is sitting in a barber chair. Another man is giving him a hair cut. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6296,"error":null,"output":"D"}
{"question_id":"hellaswag:00008370","item_index":38,"attempt":0,"prompt_hash":"4cdb7e40b76c3fabb43d6645bbfa4a8b8f1b3685869267823bd08fb17652bf4c","question":"[header] How to start a vending route [title] Choose the type and size of the locations you want for your vending route. [step] The places you put your machines will determine which vending machines you need to purchase. You should know : [substeps] Whether your machines will be inside or outside; the type of products people at the location would be interested in vending; and approximately how many people will use your machines.\n\nA. This will help you decide what type of vending machine to have. For example, if you have gas carts with vending machines in them, you would want your machines to be able to use them.\nB. How many machines can you build on your location. Generally , 20-30 machines, so models may be nearby, larger includes retail, craft, stationary, bridge, stationery, and colored plastic/wooden buildings.\nC. [title] Choose the type and size of machines that best meet your location needs. [step] Guesstimate the size of the equipment you need by assuming a daily average of one vend, per machine, per person.\nD. Alternatively, you can purchase multiple vending machines. These might include : [substeps] Grocery stores and large grocery stores anywhere outdoors during cooler months.","prompt":"[header] How to start a vending route [title] Choose the type and size of the locations you want for your vending route. [step] The places you put your machines will determine which vending machines you need to purchase. You should know : [substeps] Whether your machines will be inside or outside; the type of products people at the location would be interested in vending; and approximately how many people will use your machines.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6293,"error":null,"output":"C"}
{"question_id":"hellaswag:00006054","item_index":39,"attempt":0,"prompt_hash":"a22b0c4e6b1c456daceefe763a0499073fed06ad9469fa67847f81761a770854","question":"[header] How to move heavy furniture [title] Invest in furniture sliders. [step] You can buy the sliders of appropriate size at your local hardware or home goods store. National chains like home depot or lowes will definitely sell furniture sliders.\n\nA. The sliders will help to ensure that the furniture doesn't sag or strain when you load it. They are usually made of some kind of heavy-duty fabric that makes them sturdier.\nB. Just make sure to check the packaging before you buy one. [substeps] Avoid chinchillas, trombone meats, and porgie foods.\nC. If you are planning to move your furniture over carpet or grass you should buy sliders that specialize in that movement. [substeps] If you don't have any sliders you can also try using frisbees.\nD. [substeps] If you're not familiar with how they are designed, try home depot or lowes. They often feature special sliders designed to maximize the surface area (which is convenient to most furniture pusters).","prompt":"[header] How to move heavy furniture [title] Invest in furniture sliders. [step] You can buy the sliders of appropriate size at your local hardware or home goods store. National chains like home depot or lowes will definitely sell furniture sliders.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6293,"error":null,"output":"C"}
{"question_id":"hellaswag:00003019","item_index":40,"attempt":0,"prompt_hash":"d9a8c7900a27d36bd6e4e1087bbfee56e2f33f7cd14ea814c7943640d92c1ddf","question":"A man is shown running down a track. he\n\nA. throws open his arms, spinning around on a dirt track.\nB. does a series of back flips while going down the track.\nC. pulls a stick as he runs.\nD. then catapults into the air over a bar.","prompt":"A man is shown running down a track. he","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6293,"error":null,"output":"B"}
{"question_id":"hellaswag:00008631","item_index":41,"attempt":0,"prompt_hash":"abc46ad0b8283b5a3aa0d6fd8668b8fc85af353623a44b11d0b39cc1d3a2b4ed","question":"[header] How to wear a camel coat [title] Choose a trench coat style for a formal look. [step] Camel trench coats are one of the most popular styles, because they can make any outfit look immediately classier. Fortunately, you can find ones in a range of materials from light cotton to heavy wool, depending on what you need for the fall weather where you live.\n\nA. [substeps] Camel jackets are typically black-gray, velvet, or beige. They have multiple sleeves in a row, and extend from the upper arms to the lower legs.\nB. You'll just need to choose one or both for all seasons. [substeps] Coat coats in light cotton are most commonly worn in the summertime, while coats in heavy wool are most often worn during the winter or during hot, sunny months.\nC. [substeps] The main reason camel trench coats are common was that they are far more casual than the traditionally worn more formal coats. Therefore, they take a great deal of the emphasis out of the outfit.\nD. [substeps] Choose a coat with a belt for the more traditional trench coat style, or find a design that doesn't use a belt, giving the overall effect a less busy finish. [title] Buy a peacoat for versatility.","prompt":"[header] How to wear a camel coat [title] Choose a trench coat style for a formal look. [step] Camel trench coats are one of the most popular styles, because they can make any outfit look immediately classier. Fortunately, you can find ones in a range of materials from light cotton to heavy wool, depending on what you need for the fall weather where you live.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6284,"error":null,"output":"C"}
{"question_id":"hellaswag:00004355","item_index":42,"attempt":0,"prompt_hash":"f719c697971aaf1d735bff87138a9870b03a0f402bcbb5a20fe393ffc8bded60","question":"[header] How to make your hair stand up [title] Wash your hair. [step] You'll have an easier time adding volume to your hairstyle if you're working with clean hair. Gently dry off with a towel when you're finished.\n\nA. [title] Blow dry your hair from head to tail. [step] Use flat dry hair is what all the adults used to say.\nB. You can also skip this step if your hair is very clean, but you can't mess with the part you just washed. You can separate your hair into four pieces and wash them all in the same washing cycle.\nC. Make sure you use products specially designed for working with hair. [substeps] If you have light hair, it's a good idea to skip products like blow drying it straight, rubbing alcohol into it, or straightening it a bit (as well).\nD. [title] Massage 1 tablespoon (15 ml) of pre-styling mousse into your damp hair. [step] Rub a palm-sized amount of mousse between your hands, and then run your fingers from the front to the back of your scalp.","prompt":"[header] How to make your hair stand up [title] Wash your hair. [step] You'll have an easier time adding volume to your hairstyle if you're working with clean hair. Gently dry off with a towel when you're finished.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6284,"error":null,"output":"D"}
{"question_id":"hellaswag:00008445","item_index":43,"attempt":0,"prompt_hash":"52edba2975d315594ae17812eae935de4af99e792d7d3286678ebf72ac7ef3ef","question":"[header] How to do eye makeup for blue eyes [title] Use eyeshadow primer under your eyeshadow. [step] This will help enhance the color of your eyeshadow, making it appear brighter, which in turn will also make your eyes appear brighter. [title] Choose eyeshadow with contrasting colors.\n\nA. [step] One of the best ways to bring out the blue in your eyes is to use contrasting colors. This means choosing eyeshadows in neutral and earth tones, warm colors, and colors opposite of blue and green on the color wheel.\nB. [step] Avoid black, or pink, however. This will only enhance the orange/black look without adding any contrast.\nC. [step] You may find it easier to use contrasting colors if your eyes are slightly different. Then, when applying eyeshadow primer for blue eyes you may choose a less neutral shade, such as beige or brown.\nD. [step] Go to the store, if you have one, and figure out what color eyeshadow you'd like. It should look one color but not several different colors, as this will make your eyes look noticeably brighter.","prompt":"[header] How to do eye makeup for blue eyes [title] Use eyeshadow primer under your eyeshadow. [step] This will help enhance the color of your eyeshadow, making it appear brighter, which in turn will also make your eyes appear brighter. [title] Choose eyeshadow with contrasting colors.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6283,"error":null,"output":"A"}
{"question_id":"hellaswag:00008759","item_index":44,"attempt":0,"prompt_hash":"9c1cb234f25f2a373f63a73e4fac900fa7283a4312d28da9b2c959a0e853b754","question":"[header] How to balance work and family [title] Decide what your values are for yourself and your family. [step] A value is a principle, standard, or quality considered worthwhile or desired. Values guide our actions and structure our lives.\n\nA. Make a list of values you feel have value to you. [substeps] Look at how often you feel valued or desired.\nB. Decide what your values are and draw inspiration to your personal goals : [substeps] Don't immediately become addicted to a book or tv series immediately. Don't give up faith in nature, visual or scientific intent of mind, decade-long solitary time despite out growing money.\nC. [substeps] Areas where we often have strong values include housework, meal times, child care, car and house maintenance, relationships between spouses and parents and children, education, money, politics, religion, etc. Specifying your values is key to managing work and family demands.\nD. Grandparents are often taught a value is something that each parent doesn't think otherwise, like \" because we are doing the best job. \" in turns, values dictate what we do and the way we live.","prompt":"[header] How to balance work and family [title] Decide what your values are for yourself and your family. [step] A value is a principle, standard, or quality considered worthwhile or desired. Values guide our actions and structure our lives.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6283,"error":null,"output":"C"}
{"question_id":"hellaswag:00002185","item_index":45,"attempt":0,"prompt_hash":"d1b417b1520dfb52a8de6e4fc4664ddecb8fcef45575e43490c3214f11d7d836","question":"Two men stand on the far left of a beer pong table. one of the men\n\nA. throws a beer pong ball across the table, into a cup.\nB. begins hitting the ball into the other dealer.\nC. throws a shot put, then two men play beer pong.\nD. hits the ball.","prompt":"Two men stand on the far left of a beer pong table. one of the men","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6285,"error":null,"output":"A"}
{"question_id":"hellaswag:00001496","item_index":46,"attempt":0,"prompt_hash":"d9283a3f52f9f10b62e571e22fbf17572c07a34cda279c6f9a64b779827fbbe1","question":"Children swings a broom handle at a pinata in a backyard. the boy\n\nA. breaks the pinata open and candy spills on the ground.\nB. smiles, then hops on the broom handle.\nC. sits on a swing set and mows the patch of grass where the boy sits.\nD. swings a mop handle at the pinata too then goes to get a different mop.","prompt":"Children swings a broom handle at a pinata in a backyard. the boy","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6287,"error":null,"output":"A"}
{"question_id":"hellaswag:00002109","item_index":47,"attempt":0,"prompt_hash":"2e58e2fcfc39b7287aab16f38f29f38f9bb04fa807fe4007f1a029138c7d4286","question":"A camera shows a person riding around a dirt track on a bike moving at a quick speed. several more shots\n\nA. are shown of people riding the dirt track on bikes while the camera captures them from several angles.\nB. are shown of people riding around on bikes and speaking to one another.\nC. are shown of people riding in different speeds and going over hills passing where there is much higher grass.\nD. are shown of people riding around on the dirt track as well as riding on a stationary bike.","prompt":"A camera shows a person riding around a dirt track on a bike moving at a quick speed. several more shots","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6285,"error":null,"output":"A"}
{"question_id":"hellaswag:00007658","item_index":48,"attempt":0,"prompt_hash":"9d2a5e9229f5bea13a386a95c11587f2d8003bb7f7c4063371e01f6b60ced817","question":"[header] How to clean white vans shoes [title] Mix up a cleaning solution. [step] Using a mixture of laundry detergent and warm water is an easy way to clean vans. Into a large bowl, mix 1/4 cup laundry detergent and 2 cups warm water.\n\nA. Dump in 1/4 cup fabric softener and 1 cup bleach. Mix together with a spoon or a rubber scrub brush.\nB. If your vans are extremely dirty, you may need to mix up a clean batch for the second shoe. If you're low on laundry detergent, here are other cleaning solutions you can try.\nC. Pour the cleaning solution into the shoe, scrub the inside of the shoe, and rub in the rinse with a cloth or rag. The rinse will return the shoe to its original clean state.\nD. Swirl the solution on a rag or cloth to get suds into the shoes. As you rub the solution in, notice if the color of your vans shoes gets darker or lighter.","prompt":"[header] How to clean white vans shoes [title] Mix up a cleaning solution. [step] Using a mixture of laundry detergent and warm water is an easy way to clean vans. Into a large bowl, mix 1/4 cup laundry detergent and 2 cups warm water.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6286,"error":null,"output":"B"}
{"question_id":"hellaswag:00007241","item_index":49,"attempt":0,"prompt_hash":"df65d045f931f5a4790f6b8f95942fce7a82dcf60d6601e04a574273fdcd7b6c","question":"[header] How to make fried smashed potatoes [title] Place the potatoes in a pot and cover with water. [step] Add 4 pounds (1.8 kg) of small red or yukon gold potatoes to a large pot. Fill the pot with enough cold water to cover the potatoes by at least 2-inches (5-cm).\n\nA. [substeps] Fingerling potatoes also work well for the recipe. [title] Season the water with salt and bring to a boil.\nB. Add salt to the water so the potatoes are submerged. The potatoes should smell fresh and fragrant as they are infused with freshly cooked onions and garlic.\nC. Let the potatoes soak for 10 to 15 minutes. [substeps] Do not use 1 pound (0.45 kg) of frozen potatoes in this recipe.\nD. [substeps] Alternatively, if you prefer to cook the potatoes on their own, you can simply boil the water for the oven to bake them in. This should be enough time so that the potatoes can brown thoroughly and cook evenly.","prompt":"[header] How to make fried smashed potatoes [title] Place the potatoes in a pot and cover with water. [step] Add 4 pounds (1.8 kg) of small red or yukon gold potatoes to a large pot. Fill the pot with enough cold water to cover the potatoes by at least 2-inches (5-cm).","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6283,"error":null,"output":"A"}
{"question_id":"hellaswag:00007033","item_index":50,"attempt":0,"prompt_hash":"07f450fa79070f99b65cffa98b4ace79423359fbcbdeaff67c47262239a312ed","question":"[header] How to clean silver jewelry with vinegar [title] Immerse your jewelry in white vinegar. [step] Place your jewelry in a clean mason jar or another suitable container. Cover it with white vinegar, so that it is completely immersed.\n\nA. [substeps] Try adding fresh tea tree oil to your vinegar solution as well. The tea tree oil helps the coating hold on to the silver.\nB. Turn on the temperature of the white vinegar and wait at least 15 minutes to let the vinegar dissolve. [substeps] Do not remove your jewelry as soon as it is submerged in white vinegar.\nC. You can let it sit for two or three hours, depending on how much tarnish you need to remove. Then, rinse and dry the jewelry.\nD. Make sure that the lid is tightly sealed before handling the glass bottle. [substeps] White vinegar will dissolve the crystals and help remove the crystal polish from your jewelry.","prompt":"[header] How to clean silver jewelry with vinegar [title] Immerse your jewelry in white vinegar. [step] Place your jewelry in a clean mason jar or another suitable container. Cover it with white vinegar, so that it is completely immersed.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6286,"error":null,"output":"C"}
{"question_id":"hellaswag:00004627","item_index":51,"attempt":0,"prompt_hash":"60ce04de4227505126204b811a5c1f64853184f886f01479c141002a34384d96","question":"[header] How to learn muay thai [title] Perfect your fighting stance. [step] The fighting stance is crucial to maintain proper balance during muay thai, and a good stance will help you protect yourself from blows while delivering the strongest hits you can to your opponent. Flexibility is important in a good fighting stance, so make sure you've properly warmed up before you begin.\n\nA. [substeps] The most common stances used in fighting in muay thai are combat stances. Many people use combat stances differently, so consider what you're using to improve your techniques.\nB. [substeps] Use your free hand to support your fists with a straight face. Try a headbutt, butt butt, or elbow teaser for extra support.\nC. [substeps] Spread your feet between hip-width and shoulder-width apart with your knees slightly bent. Keep your more-dextrous/dominant foot slightly behind you so you have more momentum when you kick with that foot.\nD. [substeps] The first basic positioning is as follows: a solid stance behind your opponent. Your dominant foot should be planted squarely on the ground, and your back straight and your head drawn up.","prompt":"[header] How to learn muay thai [title] Perfect your fighting stance. [step] The fighting stance is crucial to maintain proper balance during muay thai, and a good stance will help you protect yourself from blows while delivering the strongest hits you can to your opponent. Flexibility is important in a good fighting stance, so make sure you've properly warmed up before you begin.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6286,"error":null,"output":"C"}
{"question_id":"hellaswag:00009922","item_index":52,"attempt":0,"prompt_hash":"02ea0b9fc00cbf6311e08ebe2c2f953aad9d4d1350a1bfcbfdb1f114a4a95efb","question":"[header] How to shred lettuce and cabbage, restaurant style [title] Begin with a head of lettuce or cabbage. [step] Iceberg lettuce is usually served alongside dishes like enchiladas and on tostadas, while green cabbage is the base of many slaw salads. [title] Peel off visibly bad leaves from the outside.\n\nA. [step] Get the leaves intact, though, so that you can cut them off with a knife. Cut them out using a sharp pair of sharp kitchen scissors or a pizza cutter, then peel each leaf from the outside.\nB. [step] The exterior leaves on lettuce and cabbage tend to get damaged. The head should get fresher as you go inward.\nC. [step] This should be done quickly and clearly so that they can be removed and cooked and mixed in often using whatever ingredients you feel like using. [title] Flip lettuce with a slotted spoon, and break an avocado with your fingers.\nD. [step] If you don't have a lettuce head, you'll have some rubbery looking leaves from garbanzo leaves. These leaves may not show enough to be pulled off.","prompt":"[header] How to shred lettuce and cabbage, restaurant style [title] Begin with a head of lettuce or cabbage. [step] Iceberg lettuce is usually served alongside dishes like enchiladas and on tostadas, while green cabbage is the base of many slaw salads. [title] Peel off visibly bad leaves from the outside.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6898,"error":null,"output":"B"}
{"question_id":"hellaswag:00002050","item_index":53,"attempt":0,"prompt_hash":"b86fe87e54e23f3cd56471b9f789a211a68c47f4dc78509bf44bc6e0c2afd7a6","question":"A graphic with some keywords forming the number '2' is shown. The scene switches to a black screen with some text about an upcoming marathon. a man\n\nA. jumps onto a track with a long javelin and chalks his score, then he dismounts and runs down the track to catch the javelin.\nB. is shown jogging through a city on some sidewalks, a bridge, some paths in a park, etc, with a counter on the side showing his progress in kilometers as he goes.\nC. sock is seen with peoples name on it and videotape is shown on its surface.\nD. celebrates and then is seen on his knees on a balance beam.","prompt":"A graphic with some keywords forming the number '2' is shown. The scene switches to a black screen with some text about an upcoming marathon. a man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6285,"error":null,"output":"B"}
{"question_id":"hellaswag:00001995","item_index":54,"attempt":0,"prompt_hash":"86b5b7bfa0fd00a32130ad26aec0bce6349630529dc2b290b03b85ae98dbb04c","question":"A close up of a pool table is shown followed by a person walking around the sides. The person then hits a pool ball continuously around the table. the man\n\nA. then sinks the last ball and celebrates with a group of people.\nB. continues to play and scoop out the balls while the camera follows his movements.\nC. then positions himself and hits the cue ball and sits on the table to score the pool.\nD. then walks away as well as a woman who hits the pool ball.","prompt":"A close up of a pool table is shown followed by a person walking around the sides. The person then hits a pool ball continuously around the table. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6284,"error":null,"output":"A"}
{"question_id":"hellaswag:00003904","item_index":55,"attempt":0,"prompt_hash":"fc192932fb5b356ec6570161a42a94245d6b662e844e115872325ea8b429231d","question":"[header] How to help troubled teens [title] Talk openly with the teen. [step] While the teen may seem to ignore you, or be hesitant to share feelings, it's important to open the lines of communication. Discuss your concerns in a calm and loving way.\n\nA. Be compassionate when listening to them respond or fail to respond. The teen may not be willing to open up every time.\nB. Ask the teen if it's okay or if there is anything you can do to help. [substeps] After this, reassure the teen that adults need professional help when having difficulty.\nC. [substeps] If you feel that the teen is not sharing their feelings with you, you may request that the teen open up and talk about how they feel. If they hold back, you may have to back off a bit.\nD. At this stage, you should also let the teen know that an adult is the best way to help. [substeps] Although it is important to be compassionate with the teen and not belittle or upset them, it can be nerve-wracking for both of you to discuss your concerns.","prompt":"[header] How to help troubled teens [title] Talk openly with the teen. [step] While the teen may seem to ignore you, or be hesitant to share feelings, it's important to open the lines of communication. Discuss your concerns in a calm and loving way.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6282,"error":null,"output":"C"}
{"question_id":"hellaswag:00003400","item_index":56,"attempt":0,"prompt_hash":"53386f688c63d49e22248ac92dbfded4e28a68e5a3f9962bfac67e7c89666b92","question":"[header] How to wear short shorts [title] Get the right fit. [step] The most important part of wearing any type of clothing is its fitting. However, you should pay extra attention to fit when looking for short shorts.\n\nA. If you can find the right size, you can find shorts that fit you well. A small office, pharmacy, or department store will have a decent size.\nB. A few common problems you want to avoid : [substeps] Muffin tops. Make sure your short shorts are wide enough at the waist.\nC. Most long shorts come in a variety of styles. It's important to get the right fabric when considering what yours will fit best.\nD. [substeps] Some shorts are considered to be too baggy if you want to have shorter legs. Other shorts are considered to be too small.","prompt":"[header] How to wear short shorts [title] Get the right fit. [step] The most important part of wearing any type of clothing is its fitting. However, you should pay extra attention to fit when looking for short shorts.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6897,"error":null,"output":"D"}
{"question_id":"hellaswag:00002590","item_index":57,"attempt":0,"prompt_hash":"ca1a923ac4c35de839a8c519cdedcb8d83067f5e54a901e4ed5f9c472af770b7","question":"A boy plays a set of drums in a studio. the boy\n\nA. pitches the drums to the floor, then stands to clap.\nB. signals to a man behind him.\nC. is smiling in the mirror.\nD. uses his foot to play a base drum.","prompt":"A boy plays a set of drums in a studio. the boy","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6897,"error":null,"output":"B"}
{"question_id":"hellaswag:00006352","item_index":58,"attempt":0,"prompt_hash":"919df90b05210c41f7424cec006917c6d607a0f96131f43a824b04dbbaa396b2","question":"[header] How to stop your dog from begging [title] Understand the behavior. [step] Dogs are relatively simple creatures. If they perform a behavior that results in a reward, they will repeat the behavior expecting a reward in the future.\n\nA. They will not frequently show their teeth or act immaturely. This behavior is not a sign of submission.\nB. The taste of sugar, however, is more realistic than the taste of toy food or treats. Treats merely provide even the gift of \" happy words \" or little whoopies.\nC. If a behavior does not result in a payoff, they will have no reason to repeat it. [substeps] Some dogs will just sit and look at you, while others might whine at you until they get what they want.\nD. Treats might be an example : [substeps] A dog begs when you give it a toy. It may wait for treats at dinner to stop begging.","prompt":"[header] How to stop your dog from begging [title] Understand the behavior. [step] Dogs are relatively simple creatures. If they perform a behavior that results in a reward, they will repeat the behavior expecting a reward in the future.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6648,"error":null,"output":"C"}
{"question_id":"hellaswag:00001030","item_index":59,"attempt":0,"prompt_hash":"bd4e835f5529512b93ecbba1ba740f596ab54826b3b876d76f91cfbc81b96fa3","question":"When they are done fighting the women are shown standing close together, smiling, and the woman on the left has her arm resting on the shoulder of the woman on the right. the outro\n\nA. screen appears with a banner at the bottom, \" chor 20010's abbiya #224 amazon \".\nB. appears with an african language covered in purple lines and both begin to talk to each other.\nC. is a white screen with multiple colors on it that include logos, website and the name of the event.\nD. leads into two hockey players bare arm wrestling a woman and reeling her in and the crowd is cheering.","prompt":"When they are done fighting the women are shown standing close together, smiling, and the woman on the left has her arm resting on the shoulder of the woman on the right. the outro","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6594,"error":null,"output":"C"}
{"question_id":"hellaswag:00001941","item_index":60,"attempt":0,"prompt_hash":"68da73d5e54c9d627c78283348583f95f34e8aa27e1824c5c9d6f91da74ae902","question":"A large group of people are seen playing a soccer match on a sandy field with many watching on the side. the camera\n\nA. continues to follow the group and shows many throwing the ball to one another and the game being played for spectators.\nB. captures the game from all angles and watches as one scores a goal into the net.\nC. follows the opposing teams and leads into them kicking and pulling one another's arms.\nD. shows close ups of players as well as rackets and hitting the ball.","prompt":"A large group of people are seen playing a soccer match on a sandy field with many watching on the side. the camera","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6593,"error":null,"output":"A"}
{"question_id":"hellaswag:00008250","item_index":61,"attempt":0,"prompt_hash":"ac12b1798c7b38bb2b0fd12d752715b92c245f9eebe6eebf57ce09b27b60998f","question":"[header] How to wear chelsea boots [title] Choose black colored boots, rather than brown, if you want the classic chelsea boot style. [step] The beatles wore black boots and they fit an urban style more accurately. [title] Wear black boots with black and grey slim fit suits.\n\nA. [title] Wear retro, high top chelsea boots with dark brown mid length jeans, left over from a trip to the sporting goods store, while you are drinking soda or hot cocoa. [step] Chelsea boots must really hit the gap between black boots and tights.\nB. [title] Choose black eye rimmed hipster platform boots and black or grey skinny skinny sleeve shirts for the dressier and more modern look [title] Wear black boot cut pants for a formal look. [step] Chelsea boots wear black black or grey skinny sleeved shirts.\nC. [step] They are acceptable as both a dress shoe and a casual shoe. The chelsea boot is a slim fit boot, so it doesn't look very good with baggy suit pants, shirts or jeans.\nD. [title] Wear black boots with black and grey denim trousers and grey top and look best with black and grey top and trousers. [title] Wear black trousers, a black top, and black blazer, but black socks are not required.","prompt":"[header] How to wear chelsea boots [title] Choose black colored boots, rather than brown, if you want the classic chelsea boot style. [step] The beatles wore black boots and they fit an urban style more accurately. [title] Wear black boots with black and grey slim fit suits.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6594,"error":null,"output":"D"}
{"question_id":"hellaswag:00009815","item_index":62,"attempt":0,"prompt_hash":"79521f1f2a292fe7d0a56555c22ef91bb6486590381dd42119d4a2d7e9eb15e9","question":"[header] How to do yoga if you are overweight [title] Pick a style. [step] There are many yoga styles available to choose from. In the end, you'll need to pick a style that suits your personality, goals, and abilities.\n\nA. You can find yoga videos online and at your local yoga studio. For a good case study, look for the following : [substeps] Beginner moves active moves low to medium reach arm, wrist, and elbow speed advanced moves appropriate for your purposes [title] Take a deep breath.\nB. Popular yoga styles include \" rain rock \" for more unusual poses, \" leopard eye \" for more intense poses, and \" downhill cascade. \" [title] Figure out how often you spend time with your dog.\nC. Consider : [substeps] A vigorous yoga style that will help you become more flexible and challenge your cardio abilities. For instance, try the kundalini style.\nD. Some of the most popular poses include : [substeps] Sit or lie face down or shake your head to center movement. Keep your knees slightly bent with a mild twist to them as you stand.","prompt":"[header] How to do yoga if you are overweight [title] Pick a style. [step] There are many yoga styles available to choose from. In the end, you'll need to pick a style that suits your personality, goals, and abilities.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6593,"error":null,"output":"C"}
{"question_id":"hellaswag:00007256","item_index":63,"attempt":0,"prompt_hash":"4dd873aa4594f4dacbcd580dd0749a2fadfd9e89a34cc2eb11d4a14d968597ee","question":"[header] How to use a crock pot [title] Remove your crock pot from the package. [step] Wash the interior ceramic section and the top glass portion with dish soap and warm water. [title] Clear a space on your counter.\n\nA. [step] Your slow cooker emits heat, so safety is a priority. Make sure there is plenty of room on all sides, including the top, to vent off the heat for the duration of cooking.\nB. [step] Set your crock pot aside in a location that gets plenty of ventilation. Never return crock pots to a casing when they're not in use.\nC. [step] Remove the crock pot from the package and set it on a clean, uncluttered counter. Set the burner on the lowest setting in a place you can see the crock pot.\nD. [step] Make sure to remove the crock pot from the slit. [title] Tie a string from the fabric loop on the crock pot.","prompt":"[header] How to use a crock pot [title] Remove your crock pot from the package. [step] Wash the interior ceramic section and the top glass portion with dish soap and warm water. [title] Clear a space on your counter.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6593,"error":null,"output":"C"}
{"question_id":"hellaswag:00008034","item_index":64,"attempt":0,"prompt_hash":"4e020e2bc93a12a4e035740675c71ca9bbccf4b4f0a1958e8af81f95bdeda138","question":"[header] How to diagnose entropion in boxers [title] Look for squinting. [step] If your dog is feeling uncomfortable, you may notice him squinting or narrowing his eye. Compare one eye with the other to see if one eye is worse.\n\nA. If one eye is worse than the other, then the vet might examine the eye rather than going any further than the other to rule out entropion. The vet may want to do a rectal biopsy if they suspect entropion.\nB. If your dog's eyes are dull or faded, that is a development of entropion. [substeps] Entropion may also be an early sign of obesity or alzheimer's disease.\nC. Although entropion will look worse than any other, any eye that is unfocused, glazed, or inflamed is probably entropion. It will also look and feel different when closed.\nD. If both eyes are affected then the dog may appear to have narrow eyes. [substeps] If the eye is really bothering your dog, he may even hold his eye shut to keep the lid from rubbing against the eyeball.","prompt":"[header] How to diagnose entropion in boxers [title] Look for squinting. [step] If your dog is feeling uncomfortable, you may notice him squinting or narrowing his eye. Compare one eye with the other to see if one eye is worse.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6347,"error":null,"output":"D"}
{"question_id":"hellaswag:00007210","item_index":65,"attempt":0,"prompt_hash":"f7b875645eb9150780e2c09b698345fff77552ccf5da1e85d8d3585503976a11","question":"[header] How to pass a foreign language class [title] Work on vocabulary. [step] Knowing the meanings of words is the foundation of your language study. Try to learn a group of words that are related to each other.\n\nA. Getting numbers in your vocabulary can help ease your comprehension level. [substeps] Try to remember how they came to be pronounced.\nB. You'll be more likely to remember them. [substeps] If you already have some language experience and want to build your vocabulary, spend time reading in the language.\nC. You should be able to repeat keywords in small steps in each word. It is much harder to recall a word if you can't remember it at the root.\nD. Think about what you don't understand, see if it makes sense, and then work on developing more vocabulary. As you begin speaking, try identifying words in the language that are too complicated, or that you will have trouble understanding.","prompt":"[header] How to pass a foreign language class [title] Work on vocabulary. [step] Knowing the meanings of words is the foundation of your language study. Try to learn a group of words that are related to each other.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6352,"error":null,"output":"B"}
{"question_id":"hellaswag:00001619","item_index":66,"attempt":0,"prompt_hash":"22546112e8fe636d927f59491125804d13ed85ede1dcb8b05ce1fe93b9205b92","question":"A man is outside smoking a cigarette with his friend. They are just sitting out relaxing blowing smoke. another man with a camera\n\nA. is surrounding them all smoking.\nB. is seen smoking hitting on a girl pretty well.\nC. outside opens a door and comes in.\nD. walks by while they sit chatting.","prompt":"A man is outside smoking a cigarette with his friend. They are just sitting out relaxing blowing smoke. another man with a camera","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6351,"error":null,"output":"D"}
{"question_id":"hellaswag:00002091","item_index":67,"attempt":0,"prompt_hash":"fb00aee2f099a63f3183786640870012ae79151cfbc362c79e9c434b28f81bc2","question":"Business men walk through a busy crowd. A business man plays against a table tennis professional. two businessmen\n\nA. sit on front each other.\nB. play on a team against the table tennis pro.\nC. clap hands when they meet.\nD. stand on the stage and're talking and &having drinks.","prompt":"Business men walk through a busy crowd. A business man plays against a table tennis professional. two businessmen","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6351,"error":null,"output":"B"}
{"question_id":"hellaswag:00000428","item_index":68,"attempt":0,"prompt_hash":"060373d4b3032f5cedaba0d1bbe3eb3d23d714174600ae635cf007740b17f9c3","question":"Both of the girls are laughing and joking with one another until they start to wash their face and clean it with green rags. after\n\nA. , the girl takes a rain squeezer and washes her face that is clean and little by little.\nB. rinsing their face, they both dry their face off with a towel that is hanging down from the door and come back to the camera to show pure faces.\nC. , the person on the right cleans their face with a pink toothbrush and also cleans their other face.\nD. they have washed the potatoes the two legs turns to lather the rag and the bread cut out of the tub, followed by the pieces of proper version of what had been put in.","prompt":"Both of the girls are laughing and joking with one another until they start to wash their face and clean it with green rags. after","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6350,"error":null,"output":"B"}
{"question_id":"hellaswag:00009405","item_index":69,"attempt":0,"prompt_hash":"55177c36e6fad342eab05670cce273d3d7fba3168bbe128195ba9c77871c35bc","question":"[header] How to be awesome in your class [title] Don't use an overly large vocabulary. [step] People who do this are often referred to as geeks or nerds. \" geeks \" and \" nerds \" often do well in studying, so it's not all bad--just don't act so smart that it alienates others.\n\nA. [substeps] This applies especially to people who think you don't understand the subject. If you're studying a topic you genuinely don't understand, even a simple no is enough to make the difficulty seem less daunting.\nB. [substeps] Pretend that by emphasizing slang (like \" nerd \" when it comes to research or \" genius \"), you are disabling your genius. Using this insignificance, is actually a cute and fashionable way to act in front of your peers.\nC. [substeps] If you're in a middle or high school class, consider using a vocabulary scale like b +, u +, and s +, and studying at a larger scale later. [title] Be forward-thinking and stay up to date with current events.\nD. If you can answer a teacher's question in words everyone else knows, people will be impressed. Only then will you will be called smart rather than \" geeky \".","prompt":"[header] How to be awesome in your class [title] Don't use an overly large vocabulary. [step] People who do this are often referred to as geeks or nerds. \" geeks \" and \" nerds \" often do well in studying, so it's not all bad--just don't act so smart that it alienates others.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6346,"error":null,"output":"D"}
{"question_id":"hellaswag:00001258","item_index":70,"attempt":0,"prompt_hash":"2741c80dc2e554fa533241eb479f17cb6d8ae7cea0699d65324e1d2ee0796a20","question":"Various shots of landscapes leads into a woman standing on the beach and playing a violin. the camera\n\nA. flashes intermittently and leads into her and a man speaking to some people on the side.\nB. captures her from several angles playing the instrument as well as looking off into the distance.\nC. pans all around her while pointing to her feet and playing along with her.\nD. captures several other people using the violin as well as other instruments and end by stopping.","prompt":"Various shots of landscapes leads into a woman standing on the beach and playing a violin. the camera","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6351,"error":null,"output":"B"}
{"question_id":"hellaswag:00003751","item_index":71,"attempt":0,"prompt_hash":"41b5d419e4e61eedf4c0d1e521a3204d920b83679f5dd1e61e4dc39da75dd121","question":"[header] How to style bangs [title] Do a classic blow-out. [step] The most basic way to style your bangs it to let them loose, and simply making sure they're in order and presentable. After washing your hair, use a round brush and a blow-dryer to dry your bangs to perfection.\n\nA. Roll the brush under the bangs and pull them away from your face (on the heavy side of your part) to ensure that they are dried in the right direction. [substeps] If necessary, use a flat iron or straight iron to get your bangs just right.\nB. [substeps] Use a tousled or loose look for a messy look. If you want a more beachy look, blow-dry your hair with your fingertips, or apply one to two light-hold blow-dryers.\nC. [substeps] Run a wide-toothed comb through dry hair to create loose, bouncy waves. [substeps] Using the sides of your hair throughout the blow-out will help prevent flyaways.\nD. This will create a tousled look that should be subtle, not stark. It will not look harsh or disordered.","prompt":"[header] How to style bangs [title] Do a classic blow-out. [step] The most basic way to style your bangs it to let them loose, and simply making sure they're in order and presentable. After washing your hair, use a round brush and a blow-dryer to dry your bangs to perfection.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6347,"error":null,"output":"A"}
{"question_id":"hellaswag:00003363","item_index":72,"attempt":0,"prompt_hash":"cf8c025879c597a32f4fc0ca9bcf4a064baacbc244dc3562ea21e418e423eac0","question":"[header] How to call switzerland [title] Dial your country's exit code. [step] An exit code is a set or series of digits that allows your phone call to \" exit \" the country. In other words, the numbers let the telephone carrier known that the rest of the phone number should be directed outside of the country.\n\nA. [substeps] To see a list of common exit codes, review the section on \" calling from specific countries. \" for example, the exit code for the united states is \" 011.\nB. [substeps] For example, a phone call to switzerland from 0800 would be 0800. [title] Dial the country code from 1 of the countries listed.\nC. You can generally find this somewhere along the dial pad on all the cell phones. [substeps] The exit code on the us phone will be 93.\nD. Cell phone service stations can also be easily forgotten. [substeps] Dial the exit code for most countries within switzerland.","prompt":"[header] How to call switzerland [title] Dial your country's exit code. [step] An exit code is a set or series of digits that allows your phone call to \" exit \" the country. In other words, the numbers let the telephone carrier known that the rest of the phone number should be directed outside of the country.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6349,"error":null,"output":"A"}
{"question_id":"hellaswag:00009756","item_index":73,"attempt":0,"prompt_hash":"d0a9bba5b571262f361b2a03d64706906333b2297c65e221d43b9a460e277ec4","question":"[header] How to make the perfect dinner for date night [title] Prepare a simple appetizer. [step] While you wait for your meal to finish cooking, serve some simple appetizers. These should be light and not overly filling.\n\nA. However, there should be room for variety with varying amounts of vegetables and fruits. [substeps] If possible, offer extras like these during dessert.\nB. Try crème brûlée, crab pie, or tabasco seasoning mixed with chopped chilies, sprigs of garlic and scallions. [substeps] Serve a side of the main course instead of a stack of appetizers.\nC. The goal is to avoid getting full on appetizers while trying to not get hangry and kill the mood. [substeps] A light tossed salad makes an excellent appetizer.\nD. When you're ready to serve salad, your cocktail may be even better at refreshing produce and greens. [substeps] A nice recommendation is to have a low-sugar hand salad to pair with freshly-boiled oysters.","prompt":"[header] How to make the perfect dinner for date night [title] Prepare a simple appetizer. [step] While you wait for your meal to finish cooking, serve some simple appetizers. These should be light and not overly filling.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6345,"error":null,"output":"C"}
{"question_id":"hellaswag:00006884","item_index":74,"attempt":0,"prompt_hash":"1126fb093560070b7b65a785d348fc47442363f4a2cd5439601812d057ed24f0","question":"[header] How to care for a potted freesia after blooming [title] Understand the life cycle of a potted freesia. [step] The freesia plant has three stages to its life : [substeps] The first is the blooming phase where it produces dark green foliage and beautiful blooms. The second is a rest phase, where the plant stops producing flowers and begins to use its leaves to store energy in the form of corms.\n\nA. The third is the flowering phase, when the plant develops leaves and flowers. They start looking silvery white and look almost cloudier.\nB. During that sleeping period, the plants tend to migrate south towards the sun. During this period, the plants produce 12 or 20 corms.\nC. The third phase is a dormant phase necessary to rest the plant so it can bloom again the next year. Continue reading to learn how to care for freesias in their rest phase.\nD. The third is your plant's dormant phase, where the plant will again start to produce flowers but not the corms. [title] Know that this is known as flowering phase.","prompt":"[header] How to care for a potted freesia after blooming [title] Understand the life cycle of a potted freesia. [step] The freesia plant has three stages to its life : [substeps] The first is the blooming phase where it produces dark green foliage and beautiful blooms. The second is a rest phase, where the plant stops producing flowers and begins to use its leaves to store energy in the form of corms.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6350,"error":null,"output":"C"}
{"question_id":"hellaswag:00007359","item_index":75,"attempt":0,"prompt_hash":"2328fe96d30e2cd4468da64360426ec3acef41790659c4f293eb577d87162d08","question":"[header] How to have a cute and bubbly personality [title] Keep a positive worldview. [step] Being bubbly means developing a positive outlook to find the best in other people while lifting their spirits with your attitude. [substeps] Find ways to recast a negative situation in a positive light.\n\nA. For example, try to remind yourself that something exciting was going on at work before a big event like a new job is coming up. Remind yourself that you can't move forward simply because something bad happened.\nB. For example, if your friend did something unfortunately that cannot change, you might get angry at her or remind yourself that it wasn't her fault or isn't important to her. Focus on the positives in things that do not happen everyday, and instead focus on things that make people laugh and high energy.\nC. For example, instead of thinking \" i'm so worried about finishing this essay, \" rephrase the thought as \" i'm so excited to write a great essay. \" practice finding positive aspects to daily life.\nD. If you find yourself beating yourself up when you think the worst about yourself, start revisiting those circumstances, but then remind yourself that you are on your own, so you are not alone. Remember that for some people, having a positive outlook on life will make them happy.","prompt":"[header] How to have a cute and bubbly personality [title] Keep a positive worldview. [step] Being bubbly means developing a positive outlook to find the best in other people while lifting their spirits with your attitude. [substeps] Find ways to recast a negative situation in a positive light.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6346,"error":null,"output":"C"}
{"question_id":"hellaswag:00009406","item_index":76,"attempt":0,"prompt_hash":"4d80546e545ea2c6200638a978401024d673270ed48b8611b82b166b01367afa","question":"[header] How to eat in amsterdam on the cheap [title] Buy french fries for a snack. [step] Known as \" vlaamse (flemish) frites \" in the netherlands, and most often served with mayonnaise, french fries are sold everywhere in amsterdam from restaurants to street stands for a very reasonable price. Fries come in three or four different sizes, from small to meal-sized large, to satisfy any appetite.\n\nA. [title] Grab an èsan de vouches. [step] Handmade pre-made french fries are best used as a snack, as they are made of basic ingredients rather than other ingredients.\nB. [substeps] Optional sauces are available for an extra fee. These might include the most popular mayonnaise, as well as curry sauce, garlic sauce, gravy, cheese, and ketchup.\nC. If you are eager to please, buy french fries in seven to nine batches. The larger the frites, the larger your bag going.\nD. Remember to buy some pepper on the side, too. [substeps] You should try to purchase about one inch of french fries, or else the greasy fat might turn your plate green.","prompt":"[header] How to eat in amsterdam on the cheap [title] Buy french fries for a snack. [step] Known as \" vlaamse (flemish) frites \" in the netherlands, and most often served with mayonnaise, french fries are sold everywhere in amsterdam from restaurants to street stands for a very reasonable price. Fries come in three or four different sizes, from small to meal-sized large, to satisfy any appetite.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6345,"error":null,"output":"B"}
{"question_id":"hellaswag:00004734","item_index":77,"attempt":0,"prompt_hash":"70e71eb112508adeb640e6df4011e56faef96ab87adf18d628f49fd06ab36fbb","question":"[header] How to distinguish which brand of disposable diaper a baby is wearing [title] Walk into the child's baby nursery or bedroom. [step] Sometimes the package of diapers may be stashed in this room. Sometimes the packages can be found under the crib or bed.\n\nA. Or can sometimes even be stashed in a corner; the possibilities are endless, so look hard. [substeps] If you are 100% right that the package is impossible to find in the nursery, ask your friend where she keeps the bag of new disposable diapers.\nB. If they're not accessible by the baby, make note of where they are and why they have this particular brand. Note if the place is stocked with children's clothes, bedding, blankets, and so on.\nC. [substeps] Baby diapers, especially those with toggle backs, are located close to the baby's head and are meant to be changed every three to five days to prevent diaper rash (also known as psittacosis). [title] Look for either a diaper bag or a box.\nD. [title] Use the diaper app to find the plastic wrappers. [step] Many brands such as petit baby diapers or lize were used to use as the diapers of babies.","prompt":"[header] How to distinguish which brand of disposable diaper a baby is wearing [title] Walk into the child's baby nursery or bedroom. [step] Sometimes the package of diapers may be stashed in this room. Sometimes the packages can be found under the crib or bed.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6345,"error":null,"output":"B"}
{"question_id":"hellaswag:00000427","item_index":78,"attempt":0,"prompt_hash":"b604e09ae4ddf1a890036400616ad87d8ab07dfee7a06374cdc520151678d52d","question":"Two young girls appear in the bathroom, one has a green headband around her forehead and the other has paint across her face. both of the girls\n\nA. then start fighting with each other, screaming and throwing food in  front of the mirror.\nB. are laughing and joking with one another until they start to wash their face and clean it with green rags.\nC. begin to play on the water using tape and squeege drums to play underwater in different poses.\nD. walk in and out of frame polishing different surfaces of the bathroom, brushing their nails, and making cracks.","prompt":"Two young girls appear in the bathroom, one has a green headband around her forehead and the other has paint across her face. both of the girls","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6347,"error":null,"output":"B"}
{"question_id":"hellaswag:00000933","item_index":79,"attempt":0,"prompt_hash":"426c442b31c08233ea148c4266b904b7bbe0c7aa0082c9c5f619cd53f0ee94e5","question":"A camera pans away from a set of exercise equipment and leads into a woman's feet moving along. the woman\n\nA. progresses along her journey and ends by turning around and continuing on the floor.\nB. continues to work on the first step by bending forward, and moving her feet.\nC. is then seen using the machine as well as adjusting the settings and continuing to run.\nD. removes her sneakers and socks and runs her foot along the bench.","prompt":"A camera pans away from a set of exercise equipment and leads into a woman's feet moving along. the woman","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6347,"error":null,"output":"C"}
{"question_id":"hellaswag:00009434","item_index":80,"attempt":0,"prompt_hash":"316b2090524186631b0a550a91c38b4256f23f75d77a8946f1cb1a1db0632a28","question":"[header] How to mix colors to make dark blue [title] Squeeze the amount of blue paint you will need for your project onto your palette. [step] Make sure that you have enough paint to complete the coverage needed with this shade of dark blue. Once you have mixed the perfect hue, it could be difficult to match this shade a second time.\n\nA. For this method, let each layer of your chosen color air dry for several hours. This process will eliminate the mess on the palette floor.\nB. [substeps] Store your mix areas in well ventilated areas. Otherwise, you risk air circulation blowing spores into the mix which may quickly overwhelm your base colors, resulting in a smutchy effect.\nC. For the next step, it will be possible to mix shades from one shade to another until they have a similar look to the first shade you made. [title] Allow your paint to dry completely before selecting shade shades.\nD. It's always better to make more dark blue than you need rather than run out. [substeps] If you do not complete your project in one sitting, you can store your excess paint in an airtight container like a plastic container you would use to store leftover food.","prompt":"[header] How to mix colors to make dark blue [title] Squeeze the amount of blue paint you will need for your project onto your palette. [step] Make sure that you have enough paint to complete the coverage needed with this shade of dark blue. Once you have mixed the perfect hue, it could be difficult to match this shade a second time.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6345,"error":null,"output":"D"}
{"question_id":"hellaswag:00005820","item_index":81,"attempt":0,"prompt_hash":"96997faa642505a179f6e6ecd7e00616b61559d94178b528871ad93db60606f1","question":"[header] How to be a squidminder intern [title] Express your interest. [step] Think about the following questions and submit your answers in this questionnaire :. [title] Be open minded to what being an \" intern \" means.\n\nA. [step] In squidminders, interns are encouraged to be very self-directed, developing their own goals and activities. How genuinely interested are you in exploring the possibilities of your mind and others around you? This includes trying to help people develop skills for maximizing quality of life.\nB. [step] You are already comfortable about what the work regimen will be like, and if you aren't open minded enough you'll be at risk of this developing into something that takes off your. Working for a young swfomer is a far more interesting experience and will experience you sharing new experiences and ideas with other people.\nC. [step] Are there any learning opportunities available? Is the internship accredited (at least 2003)? Are there any particular supplemental or supervisory degrees that you will acquire? If so, define it. Are there any certifications that you will need to pass? [title] Become proficient in the software that you are working with.\nD. [step] If possible, display two different degrees of interest in various subjects related to the job you want to pursue. Be sure to do some research prior to going in for the interview process.","prompt":"[header] How to be a squidminder intern [title] Express your interest. [step] Think about the following questions and submit your answers in this questionnaire :. [title] Be open minded to what being an \" intern \" means.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6346,"error":null,"output":"A"}
{"question_id":"hellaswag:00002342","item_index":82,"attempt":0,"prompt_hash":"f1b7fca61381b5f9a681e409fe8576dbb9277f6408f199f6f36824fcf66916d9","question":"A man is running while holding a pole on a track. he\n\nA. uses the pole to vault through the air.\nB. runs down the track, wrapping his legs around the pole.\nC. runs into a sand pit and starts to jump.\nD. lands on a pile of sand.","prompt":"A man is running while holding a pole on a track. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6346,"error":null,"output":"A"}
{"question_id":"hellaswag:00005867","item_index":83,"attempt":0,"prompt_hash":"5be75c4ded17d173da0b371ac05ac07f4e19a4e79b01a255fcf4a68c87353464","question":"[header] How to wear minimalist jewelry [title] Pick simple, clean pieces. [step] When it comes to minimalist jewelry, you don't want to get too fussy. However, you don't want to make it so simple that it's boring.\n\nA. The key is picking pieces that make a statement while still being minimal, such as single stone in a clean setting dangling off a necklace or set in a ring. A single pearl on slim silver band can be this type of statement piece.\nB. Base items on what sort of jewelry to choose for your jewelry. [substeps] Jewelry already looks minimalist may be something simple like a necklace or bracelet, but it could feel bohemian and impractical if you don't remember what it says.\nC. Bring more simple pieces and ask yourself what styles and pastels would be the most ideal. Instead of refined jewelry, opt for timeless pieces instead.\nD. The key to minimalist jewelry is wearing pieces that are easy to master. This will make jewelry look more sophisticated and sleek.","prompt":"[header] How to wear minimalist jewelry [title] Pick simple, clean pieces. [step] When it comes to minimalist jewelry, you don't want to get too fussy. However, you don't want to make it so simple that it's boring.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6345,"error":null,"output":"D"}
{"question_id":"hellaswag:00008116","item_index":84,"attempt":0,"prompt_hash":"23d9693716a029b8c3604d6cfd08ac162487f71b7b87d02d896dafbc6fe946cd","question":"[header] How to slowly but efficiently become a better person [title] If you wish to ease your way into becoming a better person, you must begin by taking a look at yourself. [step] You must learn to notice your flaws and become self-aware of how your actions and words affect others. [title] Begin by asking yourself how you think.\n\nA. [step] Is this your reaction or is it someone else's reaction? Is your reaction such that it is subconscious? These are the reasons why i haven't tried this. If you are over 80 or over, then you might be over 70.\nB. [step] When did i start thinking like that? Do i actually do it that way? Recognize why you started thinking that way. Not every individual made poor choices, but most people made bad choices.\nC. [step] This is a number that will help you navigate in the future. There are many ways to help you change; by researching it, you will learn which aspects of your character (in how you (or other person) behave are better (i.e.\nD. [step] Do you think it is okay to hastily judge someone because of their appearance, ethnicity, sexuality, etc. ? Do you think it is okay to be rude to someone because they are not like you or think differently from you? [title] Ask yourself how, why, and what you speak and say.","prompt":"[header] How to slowly but efficiently become a better person [title] If you wish to ease your way into becoming a better person, you must begin by taking a look at yourself. [step] You must learn to notice your flaws and become self-aware of how your actions and words affect others. [title] Begin by asking yourself how you think.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6349,"error":null,"output":"D"}
{"question_id":"hellaswag:00000635","item_index":85,"attempt":0,"prompt_hash":"7b47e6af27f033eee0cf642cc9a5b885aa28668340e0f61b10d98e67ebcdd4cd","question":"A black boot sits on a table with several items, including a pair of handcuffs. A man's hand brings out shoe polishing supplies. the man\n\nA. sprays lotion onto the shoe.\nB. demonstrates how to polish the boot.\nC. then pours the shoe polishing supplies and lubricates his shoe with the product.\nD. rearranges the contents of the shoe on the table.","prompt":"A black boot sits on a table with several items, including a pair of handcuffs. A man's hand brings out shoe polishing supplies. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6346,"error":null,"output":"B"}
{"question_id":"hellaswag:00008743","item_index":86,"attempt":0,"prompt_hash":"5fb4d66c7d981bc08f214d246ceae94fbaedc212f3e3c1ebd353d878b0716382","question":"[header] How to get your children to eat their vegetables and fruits [title] Set the right example. [step] Children learn what they live, making it vital that parents set the right example with their own food choices. If parents are routinely eating and snacking on unhealthy foods, how can children be expected to do any differently? Setting the right example to get children to eat right requires parental self-discipline.\n\nA. [substeps] Always appreciate the value of helping your children to incorporate healthy meals into their diets. Other parents may view healthy meals as a side dish instead of a part of something fun or attention-grabbing.\nB. Parents need to provide loving and firm guidance in making healthy and wise choices regarding food and snacks. [substeps] It is most important to eat plenty of vegetables and fruits yourself.\nC. [substeps] Make your own food even if you live by the same restaurant. Try to be consistent with what you cook.\nD. [substeps] Eat with your child's best interests at heart. She's more likely to eat healthily if she doesn't have to worry about restrictions or added calories.","prompt":"[header] How to get your children to eat their vegetables and fruits [title] Set the right example. [step] Children learn what they live, making it vital that parents set the right example with their own food choices. If parents are routinely eating and snacking on unhealthy foods, how can children be expected to do any differently? Setting the right example to get children to eat right requires parental self-discipline.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6893,"error":null,"output":"B"}
{"question_id":"hellaswag:00008457","item_index":87,"attempt":0,"prompt_hash":"57150bc5284285a2db912ea9201c6dfb0b065c7b951775bef8822946235db1a0","question":"[header] How to apply a hair mask [title] Read directions carefully. [step] Many commercial hair masks come with special instructions. Some may only be designed for weekly use and some may need to be kept in for specific timeframes.\n\nA. Since there is no time limit on how long a mask will stay in place, to ease your hair's overall resistance against dryness and frizz, read any instructions carefully. [substeps] As a last resort, the mask will not stay in for longer than a day.\nB. Not all hair masks can be used interchangeably in terms of the duration and frequency of use. Make sure to read instructions carefully before applying a hair mask.\nC. Some masks may only be applied for up to six months and others may be used for up to a year. Some masks may only be applied for one or two weeks and others may last for longer periods.\nD. If you have not learned any of these steps, read the directions and consult a hair mask specialist or chiropractor. These tips may vary from product to product and are taken on a daily basis.","prompt":"[header] How to apply a hair mask [title] Read directions carefully. [step] Many commercial hair masks come with special instructions. Some may only be designed for weekly use and some may need to be kept in for specific timeframes.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6280,"error":null,"output":"B"}
{"question_id":"hellaswag:00008310","item_index":88,"attempt":0,"prompt_hash":"f24ef5e336e8212b021601f3903c13682d40258babe6c5088210a5814becc495","question":"[header] How to dress and look simple [title] Build your wardrobe with classic staples. [step] Having a few good staples goes a long way. You can mix and match them with each other and employ layers for all seasons.\n\nA. [substeps] For example, wear a cute bathing suit under a simple top for colder days, a colorful t-shirt in a color that matches your cable-knit layer, or a chic belt for a more casual time and appearance. You can also try cuffing layering.\nB. Choose classic items for an elegant appearance that doesn't require a lot of forethought. [substeps] For a dressed up event, any woman will tell you that their' little black dress' is a staple.\nC. Some good staples include : [substeps] Classic-avoid anything too subtle-modern and asymmetrical, dressy and classic. More modern staples like simple tops and skirts-have a vintage vibe but stand out from the crowd.\nD. [substeps] Slippers; consider wearing nice, neutral-colored flats or heels. Necklaces and bracelets like gold and pearl, copper and sapphire will create a seamless look.","prompt":"[header] How to dress and look simple [title] Build your wardrobe with classic staples. [step] Having a few good staples goes a long way. You can mix and match them with each other and employ layers for all seasons.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6281,"error":null,"output":"B"}
{"question_id":"hellaswag:00009356","item_index":89,"attempt":0,"prompt_hash":"ce655be7b79b7ea3a511577d8418c19f59f9ccb4fe47bdc99699c32a7994a34c","question":"[header] How to involve your kids in selecting a dog [title] Bring up the possibility of getting a dog. [step] For most kids the idea of getting a dog will be very exciting. Bring the issue up with your kids when they have time to get excited, ask questions, and generally freak out.\n\nA. Keep in mind that they may not be the most excited about the concept of having a dog, so it is important that you discuss the possibility with them a little bit before you. [substeps] For example, if you have your kids go shopping with a pet, they will probably say \" i want a dog.\nB. You could say \" we are thinking about getting a dog. What do think about that idea? \" [substeps] During that initial conversation you should bring up what your kids will have to do to care for the dog if they agree to get one.\nC. When the kids are getting excited about picking a dog, talk it over so they can be ready for anything. [substeps] You can pick a puppy, or puppies, and realize you are still just under 10 years old and don't really understand the complexity of adopting.\nD. Keep it clear with your kids that the notion of owning a dog is not a complete myth. [substeps] Instead of saying \" if you can't afford a dog, but you're still willing to try one, why not give it a try? \" let your kids know that the idea of a dog is exciting.","prompt":"[header] How to involve your kids in selecting a dog [title] Bring up the possibility of getting a dog. [step] For most kids the idea of getting a dog will be very exciting. Bring the issue up with your kids when they have time to get excited, ask questions, and generally freak out.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6281,"error":null,"output":"B"}
{"question_id":"hellaswag:00000208","item_index":90,"attempt":0,"prompt_hash":"c02b67db3ffdefe3d5ce8b9028027567c19654e5bff127e5e91cfe60964f316f","question":"Two swimmers are under water trying to reach the ball. They pass the ball around as they swim through the water. then the swimmers\n\nA. are in the pool again.\nB. are shown inside a black gym swimming.\nC. swim their way up and get out of the pool while some other swimmers are still inside the pool.\nD. head towards the ball.","prompt":"Two swimmers are under water trying to reach the ball. They pass the ball around as they swim through the water. then the swimmers","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6280,"error":null,"output":"C"}
{"question_id":"hellaswag:00003565","item_index":91,"attempt":0,"prompt_hash":"74adff0f13cd0bccba3c64e540969bb48bbe3a9d7c2bd79901dc02bdb4caec54","question":"[header] How to make a leather belt [title] Measure your belt size. [step] First, you will need to take measurements to know where to cut the leather. A simple way to do this is to put on a belt that you already have, put it on to the belt hole that fits you best, and measure the belt all the way around.\n\nA. That will give you an idea of how big and thick your belt really is. You can record yourself measurements on a measurement chart that you can buy or make for yourself.\nB. With the measurements you gave, find a fabric appropriate for your style. [substeps] For example, you may want to try a wide belt for a tight fit and a skinny belt for a soft fit.\nC. Wrap a tape measure around the belt to get the same measurement. [substeps] Make sure that you have some extra material to pull up at the bottom of the belt when you measure.\nD. You can also use a tape measurer by putting it around your hips at the place where the belt would sit and recording the measurement. [substeps] Use a measurement that feels most comfortable.","prompt":"[header] How to make a leather belt [title] Measure your belt size. [step] First, you will need to take measurements to know where to cut the leather. A simple way to do this is to put on a belt that you already have, put it on to the belt hole that fits you best, and measure the belt all the way around.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6488,"error":null,"output":"D"}
{"question_id":"hellaswag:00001782","item_index":92,"attempt":0,"prompt_hash":"b5e2ac4e999cc664318a1e4fe47c98c3ab4bbf73695784e12e43e344db756095","question":"A person is seen speaking to the camera holding a shovel. the person\n\nA. then begins pulling leaves out of a hole in the ground.\nB. uses the tool all along the bushes and leaves.\nC. shows how to use his shovel using only one hand.\nD. then begins shoveling snow around an area.","prompt":"A person is seen speaking to the camera holding a shovel. the person","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6487,"error":null,"output":"A"}
{"question_id":"hellaswag:00000098","item_index":93,"attempt":0,"prompt_hash":"e12ca10af28655b4e8a408d6dfc1aab834094f9343ba92f68dbef4ba99145630","question":"A man wearing a black clothes is sharpening the knife on a stone. The man beside him on blue long sleeves drawing something on the table with his finger. the man\n\nA. continues sharpening the knife and standing on the middle of the stone.\nB. turn the knife to sharpen the other side and then wipe it with paper towel.\nC. stops sharpening the knife and draws on the stone again.\nD. down retracts from the knife and starts sharpening the knife back up again.","prompt":"A man wearing a black clothes is sharpening the knife on a stone. The man beside him on blue long sleeves drawing something on the table with his finger. the man","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6487,"error":null,"output":"A"}
{"question_id":"hellaswag:00003549","item_index":94,"attempt":0,"prompt_hash":"3c2599ae9d61938241b43aecad49d07d30ad3857d637c211cd635da923ac833f","question":"[header] How to deal with boys [title] Understand that young boys are always on the go. [step] By the time they're toddlers, boys begin developing gross motor skills at a faster rate than girls of the same age. This means that most young boys are always moving, bouncing, running, kicking, and seemingly trying to hurt themselves in increasingly inventive ways.\n\nA. [title] When puberty advances, consider these behaviors in the moment. [step] Now, it will be decades before they are in shape, so consider their intentions before you can decide if your mothering is appropriate.\nB. Channel that energy into activities such as sports, play time, and even household chores. [substeps] Ensuring your boy gets plenty of opportunities for exercise, especially outdoor exercise, will help tire him out.\nC. Older boys are undeveloped and develop unknown control over all of their bodily functions. Yet here are some things that older boys are capable of trying out : [substeps] Making hands over funny things.\nD. Growing up in a household with a demanding life, are often at some point bound to make young boys more aggressive towards other children. [substeps] If you see a child constantly getting aggressive with other children, it can lead to violent outbursts.","prompt":"[header] How to deal with boys [title] Understand that young boys are always on the go. [step] By the time they're toddlers, boys begin developing gross motor skills at a faster rate than girls of the same age. This means that most young boys are always moving, bouncing, running, kicking, and seemingly trying to hurt themselves in increasingly inventive ways.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6488,"error":null,"output":"B"}
{"question_id":"hellaswag:00003238","item_index":95,"attempt":0,"prompt_hash":"1a68154ba5555594f0da56b9621d9aed1897986f8f47ed5ca982d5900d6f6681","question":"People sail on inflatable boats down the street in a river. A group of people take a rest having some drinks. after\n\nA. , people sail in big boats sailing in the river.\nB. , people go to the water to surf.\nC. , young men are seen fly boats and get to the shore.\nD. , people sail down the river drinking from cans.","prompt":"People sail on inflatable boats down the street in a river. A group of people take a rest having some drinks. after","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6487,"error":null,"output":"A"}
{"question_id":"hellaswag:00005527","item_index":96,"attempt":0,"prompt_hash":"d7f49d59068876e72e13305dafe347481c4d863a826b33ad07279ce81cdaddf3","question":"[header] How to start your own lawn care business [title] Get fully licensed and certified. [step] In general, the legal hurdles to starting a lawn care business are low compared to other industries. Some cities have municipal permits.\n\nA. Other levels of government may also have requirements when it comes to the regulation of things like pesticides. Consult a local lawyer to ensure that you are in full compliance.\nB. These permits are excellent substitutes for starting one of the other types of professional lawn care businesses. Once you become licensed, you'll be able to spend money on your own business.\nC. If you do not make any money running your own home, you probably can't legally open a lawn care business. To start your own lawn care business, you must obtain a license from the dmv.\nD. You are required to adhere to all necessary building codes before you start your own lawn care business. You may need to file for a permit before you ever open your lawn care business.","prompt":"[header] How to start your own lawn care business [title] Get fully licensed and certified. [step] In general, the legal hurdles to starting a lawn care business are low compared to other industries. Some cities have municipal permits.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6215,"error":null,"output":"A"}
{"question_id":"hellaswag:00003680","item_index":97,"attempt":0,"prompt_hash":"978a46277712f6bceba68f0f9040b25a83ac7d7fd5510c50077bd43cccdb0db9","question":"[header] How to make your lips stand out [title] Select three types of lipstick. [step] These should all be roughly the same color, but three different shades. Lip liner may be easier to apply for the lightest and darkest shades.\n\nA. Avoid multi-colored lipstick or different shades of black. They will blend together a little better, but they might leave your lips looking murky and muddy.\nB. [substeps] The darkest shade will be used for outlining and complementing. For a bolder look, replace it with another colour, such as purple, scarlet, or vivid red.\nC. Lip liner might be easier to apply for the whites. Warm tones: choose a lip liner that is slightly warmer than your lips so it will conform to your lips more fully.\nD. Top lip liner may be best for more intense colors, like dark reds, blacks, greens, pinks, pinks, peaches, and purples. You can use any color or style you like.","prompt":"[header] How to make your lips stand out [title] Select three types of lipstick. [step] These should all be roughly the same color, but three different shades. Lip liner may be easier to apply for the lightest and darkest shades.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6215,"error":null,"output":"A"}
{"question_id":"hellaswag:00003381","item_index":98,"attempt":0,"prompt_hash":"af35f8cf08a88e0741cf9ce1c2c9f67d20b60a6d16ce7b04233f56ee72c5ea5f","question":"[header] How to cancel a paypal buyer credit card [title] Call the paypal credit service center. [step] It may take some time to get through the automated call forwarding system and wait time. However, this is the quickest way to cancel your card.\n\nA. You can dial 011 from the phone. [substeps] If you are mobile, dial 011 from the mobile phone application or directly from the paypal wireless hotline.\nB. [substeps] Be sure to get the credit card information as soon as possible. If you wait too long, they may skip the payment entirely.\nC. Make sure you speak and answer the phone soon after the service forward number. The customer service process is quicker if you have a cellular phone.\nD. The customer service representative will hopefully be able to cancel the account while you are on the phone. They should at least be able to get all the information they need from you, even if the cancellation doesn't go into effect the same day.","prompt":"[header] How to cancel a paypal buyer credit card [title] Call the paypal credit service center. [step] It may take some time to get through the automated call forwarding system and wait time. However, this is the quickest way to cancel your card.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6214,"error":null,"output":"D"}
{"question_id":"hellaswag:00006872","item_index":99,"attempt":0,"prompt_hash":"a9b2771309f9018bfc0d0344481ce0451195922a5d90fa04815550f729891675","question":"[header] How to adjust your lantus dose [title] Track any weight loss. [step] The amount of insulin you get in each dosage of lantus is based on a lot of factors, including how much you weigh. If you've lost or gained weight, more than 1 lb (0.45 kg) or 1 lb (0.45 kg), it might be time to adjust your dose.\n\nA. [title] Keep an eye on changes in your physical activity. [step] The amount of physical activity you participate in can affect how much insulin your body produces on its own.\nB. Your doctor may indicate the following changes depending on the level of weight you are gaining : [substeps] Your blood sugar. If you're losing weight, you might want to alter how much insulin is in your blood sugar.\nC. Focus on how much you're eating or sleeping the most. If you don't look to determine whether you're losing weight, you should watch your insulin levels.\nD. [substeps] If you lose a pound during each dose, you may need to increase your dose or switch sides. Lose 1 to 2 lb (0.45 kg) of insulin each week for continued support.","prompt":"[header] How to adjust your lantus dose [title] Track any weight loss. [step] The amount of insulin you get in each dosage of lantus is based on a lot of factors, including how much you weigh. If you've lost or gained weight, more than 1 lb (0.45 kg) or 1 lb (0.45 kg), it might be time to adjust your dose.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6214,"error":null,"output":"A"}
{"question_id":"hellaswag:00000808","item_index":100,"attempt":0,"prompt_hash":"8f0d7f126694a444289ee2b13807d33795f783543b0cab6b319c064606e1e9be","question":"Multiple images and clips of many different people eating and showing their ice cream. the bald man and the lady inside of the ice cream parlour bus\n\nA. are shown working and picking up supplies as more images of patrons eat and show their ice cream treats.\nB. speaks their hot eights while the bald man talks for a bit.\nC. raise their mouths and raise their mouths to the camera with their own mouths.\nD. then the front part of the ice cream shop and end by offering the food to one another and the closing credits flashing.","prompt":"Multiple images and clips of many different people eating and showing their ice cream. the bald man and the lady inside of the ice cream parlour bus","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6217,"error":null,"output":"A"}
{"question_id":"hellaswag:00009777","item_index":101,"attempt":0,"prompt_hash":"e014d9225653b306bd7c5fdeafa850304572eb6e627d9ff0be25656b94b83b72","question":"[header] How to do a scene haircut [title] Plan your cut. [step] A scene hairstyle is composed of several layers at different lengths. Usually three main layers are used for this hairstyle.\n\nA. The top layers are usually about 40% longer than the bottom layers, although those can be thinner. The sides tend to meet in mid-length.\nB. [substeps] Don't rush fashion trends-your haircut should cover all the important features you're looking for. Each layer should be more to your liking.\nC. When creating a scene, the layers rotate around the main part of the face, cutting down the jaw and surrounding the hairline. [substeps] Unless you are going to tie in your hair, keep the layers relatively close to the same length.\nD. Before you cut, plan where you would like the layers of your hairstyle to line up. [substeps] To get ideas of cut and color, you may wish to look at pictures online to see which sort of style you prefer.","prompt":"[header] How to do a scene haircut [title] Plan your cut. [step] A scene hairstyle is composed of several layers at different lengths. Usually three main layers are used for this hairstyle.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6219,"error":null,"output":"D"}
{"question_id":"hellaswag:00006442","item_index":102,"attempt":0,"prompt_hash":"065859694a17a4f781b5c33fe119d305fc9216af18e227e27e5628f334244f35","question":"[header] How to make your eyelashes look longer without the expensive mascaras [title] Start with a clean canvas. [step] For best results, begin your makeup routine with a quick cleanse. Wipe off any old makeup and mascara so that your lashes are fresh.\n\nA. Applying new mascara is easier because your lashes will flake and look clumpy rather than long and sleek if you layer the new over old. [substeps] Cleaning your face every night before bed is a good habit to have.\nB. Brush on your moisturizer, then blend in another dab of foundation and lipstick. [substeps] For last, clear eyeshadow is usually a good idea.\nC. For now, use a little milk or a 2 ounce glass of light brown eye liner. [substeps] To get started, apply some makeup primer on your eyelids and apply a coat of mascara, eyeshadow and eyeliner to make it more neutral.\nD. Use a damp cotton swab to clean off the powder residue. [title] Apply some thin, translucent translucent-lipstick or balm.","prompt":"[header] How to make your eyelashes look longer without the expensive mascaras [title] Start with a clean canvas. [step] For best results, begin your makeup routine with a quick cleanse. Wipe off any old makeup and mascara so that your lashes are fresh.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6214,"error":null,"output":"A"}
{"question_id":"hellaswag:00005354","item_index":103,"attempt":0,"prompt_hash":"decc5f6d7839fb4ca34e8a0eb34363f0cd6cceffd84f1850a78f4362509b0e03","question":"[header] How to texture paint [title] Fill a paint roller tray with flat latex paint. [title] Pour 4 to 6 ounces (113.3 to 170 g) of silica sand into the paint. [title] Stir the paint with a wooden paint stick.\n\nA. [title] Poke 3 holes in the top of the painting surface and place 1 test disc into each hole. [step] Use the circular can opener to slide the disc through the holes.\nB. [title] Draw a small semicircle on the back of the painted canvas. [title] Using a pencil, draw a series of imaginary lines going at a 45-degree angle (like spokes on a wheel).\nC. [title] Add more silica sand for a coarser texture. [title] Repeat the process of stirring before each application because the sand will settle on the bottom.\nD. [title] Wait for some time to sift the sand, stirring every minute or so. [title] Turn on your mixer and get to work.","prompt":"[header] How to texture paint [title] Fill a paint roller tray with flat latex paint. [title] Pour 4 to 6 ounces (113.3 to 170 g) of silica sand into the paint. [title] Stir the paint with a wooden paint stick.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6214,"error":null,"output":"C"}
{"question_id":"hellaswag:00001324","item_index":104,"attempt":0,"prompt_hash":"97a072781b85d05a7f0d892d542f8f6f6c5aa0621ff31db42768366c0a097cd4","question":"A blurry black stage is present and then all of a sudden a woman appears playing the piano. while the woman\n\nA. is playing she continues to lead and begins to strum her hair and when she finishes she shows the camera her face.\nB. plays the screen moves and a 3d bass drum appears zooming over the office as well as a person's faces.\nC. is playing, the blind artist stevie wonder appears playing the harmonica into his microphone.\nD. is playing, a nurse is standing in the room beiging the keyboard and buttons.","prompt":"A blurry black stage is present and then all of a sudden a woman appears playing the piano. while the woman","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6217,"error":null,"output":"A"}
{"question_id":"hellaswag:00008368","item_index":105,"attempt":0,"prompt_hash":"3cf6c82fd94d4cff50a8878281d9da5d35825e41f568357230f9cf84561690a6","question":"[header] How to convince your spouse to help around the house [title] Discuss your expectations. [step] Talk with your spouse about what \" a clean house \" means to both of you. Tell them what level of cleanliness you need to feel comfortable, and ask them for their perspective on the issue.\n\nA. [substeps] Remember that this is your relationship, not your spouse's. If, for example, you're packing for a trip to europe while your spouse is home from work, you probably both might not want to contribute to that trip.\nB. [substeps] Some people naturally have a higher tolerance for messiness than others. A cluttered table that drives you crazy might not even seem like a mess to your spouse.\nC. [substeps] You might try saying, \" i really appreciate how valuable you are to me, but i think i'm no better off without you. Would you be ok with that? \" [title] Find common ground.\nD. Look for ways you can manage the household finances, or contribute to extra care that your spouse needs. [substeps] While prioritizing your own responsibilities will be difficult if you know for a fact that you have a child and are not able to manage household finances financially on your own, it's still worth asking.","prompt":"[header] How to convince your spouse to help around the house [title] Discuss your expectations. [step] Talk with your spouse about what \" a clean house \" means to both of you. Tell them what level of cleanliness you need to feel comfortable, and ask them for their perspective on the issue.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6213,"error":null,"output":"B"}
{"question_id":"hellaswag:00006112","item_index":106,"attempt":0,"prompt_hash":"dbfa0057b1b1cd69d5e76a43b0acddc474e6bfbaef0f19b80c94e6a342f42f79","question":"[header] How to start a baby registry [title] Work on your baby registry while you are pregnant. [step] Creating a baby registry can be a deceptively long process and may take a few revisions, so it is best to start early. [substeps] Most expectant moms start around twelve weeks, and add to their registry as they find out the gender of the baby.\n\nA. You may want to take your baby registry to a facilitator to make the process easier. This can be your doctor or midwife.\nB. One good rule of thumb is to starting at twelve weeks before the due date and extending your time between that date and then three or four weeks after. It may take a few weeks after baby steps, in order to start a new registry.\nC. Most expectant mothers include different or more than a particular baby gender in their registry. Never duplicate a baby gender before you start the registry process.\nD. Aim to have your registry completed by the time you are six months pregnant. Take into account when your baby shower is being held, and give guests a good opportunity to access your registry beforehand.","prompt":"[header] How to start a baby registry [title] Work on your baby registry while you are pregnant. [step] Creating a baby registry can be a deceptively long process and may take a few revisions, so it is best to start early. [substeps] Most expectant moms start around twelve weeks, and add to their registry as they find out the gender of the baby.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6213,"error":null,"output":"D"}
{"question_id":"hellaswag:00001451","item_index":107,"attempt":0,"prompt_hash":"d11b972d0f9fe5ac28d2d2c02df65df9cbc6e49c7f1f44b274d61a104dd1f8ec","question":"People prepare to go canoeing and then launch their boats. people\n\nA. then go down the stream.\nB. gather in a canoe at the lake.\nC. race rafts in the sand.\nD. go to a large grassy spot and start kayaking.","prompt":"People prepare to go canoeing and then launch their boats. people","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6215,"error":null,"output":"B"}
{"question_id":"hellaswag:00004235","item_index":108,"attempt":0,"prompt_hash":"9ccd45b7126250aa5b08e5a530b682bf4c4d76506cdd288fa3007c22d562eef6","question":"[header] How to look like you are wearing makeup [title] Enhance your eyelashes. [step] Instead of mascara, take a bit of lip balm (flavoured works best) on your index finger and thumb and run it through your eyelashes. This will darken and lengthen your eyelashes, and over time, it will increase their actual length.\n\nA. [substeps] Re-apply mascara daily for better results. This protects your lashes from harmful antioxidants.\nB. You can use petroleum jelly instead of lip balm. [title] Use a little bit of soft lotion of your eyelids.\nC. [title] Bring your eyebrows up so that the top of them are in the middle of your eyes, and relax them gently. [step] Don't try to contour these brows.\nD. Practice this step without a lipstick or gloss first. [substeps] If you have thick eyelashes, try using a matte lipstick or gloss.","prompt":"[header] How to look like you are wearing makeup [title] Enhance your eyelashes. [step] Instead of mascara, take a bit of lip balm (flavoured works best) on your index finger and thumb and run it through your eyelashes. This will darken and lengthen your eyelashes, and over time, it will increase their actual length.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6213,"error":null,"output":"B"}
{"question_id":"hellaswag:00000847","item_index":109,"attempt":0,"prompt_hash":"11ce69e2ccc6a5a2194d985fccf76c2d910dfb3c14d34bb3ad27858fb8431ebc","question":"A man walks up to the side of a cabin. He uses a large knife to cut through the grass. he\n\nA. places a cloth on a branch as he cuts.\nB. grabs a bag and kneels on the floor pointing to two pieces of wood.\nC. pulls up the tree off the ground before cutting it.\nD. swings the knife back and forth, cutting the grass and weeds.","prompt":"A man walks up to the side of a cabin. He uses a large knife to cut through the grass. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6215,"error":null,"output":"D"}
{"question_id":"hellaswag:00003709","item_index":110,"attempt":0,"prompt_hash":"4d0f2dd7658a35300bcf0507270ceedf475ac13d5e1000745895898d49963f94","question":"[header] How to do pressed flower nail art [title] Choose your flowers. [step] Pick some colorful wild flowers in lots of bright colors-the darker, the better. If you plan on having a non-white background on your manicure, you can get some white flowers as well.\n\nA. It's not important, but pressing flowers on your nails are best. [substeps] Gather the flowers that you want to layer on your design.\nB. The flowers should be small enough to fit on your nail. [substeps] You can buy dried flowers from the store instead.\nC. [title] Apply a base coat for a more perfect touch. [step] In any color or pattern you like, apply a base coat to finish the look of your nails, and let the primer dry before proceeding.\nD. [substeps] Choose long, flowing flowers in pink, purple, or black. You can use your creativity to print different patterns or designs on the flowers if you would like to.","prompt":"[header] How to do pressed flower nail art [title] Choose your flowers. [step] Pick some colorful wild flowers in lots of bright colors-the darker, the better. If you plan on having a non-white background on your manicure, you can get some white flowers as well.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6214,"error":null,"output":"B"}
{"question_id":"hellaswag:00004840","item_index":111,"attempt":0,"prompt_hash":"4fdaf5625823be27ad2a0315108068ecad73a008dcfb210e275119f699b62eda","question":"[header] How to plait someone's hair [title] Have the person sit in a comfortable place where you can easily reach their hair. [title] Brush the person's hair until you can easily run your fingers through it without encountering any knots. [step] A good way to brush the hair is to do it in different strips.\n\nA. [title] Apply one fast, fluid hold to the hair at the back of the victim's hairline. [step] Then have her spread her hair so that it covers her entire forehead and face except her hairline.\nB. [title] Divide the hair into three equal parts. [step] Put the outside divisions over the person's shoulders.\nC. This will ensure that the strands aren't separating on the top and bottom, which will help produce a messy, frizz-free look. [title] Use an oil-based shampoo.\nD. This will create a distorted, poofy look. Once you run your fingers through the hair strip, gently tug the material-known as a fringe-or braid it tightly.","prompt":"[header] How to plait someone's hair [title] Have the person sit in a comfortable place where you can easily reach their hair. [title] Brush the person's hair until you can easily run your fingers through it without encountering any knots. [step] A good way to brush the hair is to do it in different strips.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6213,"error":null,"output":"B"}
{"question_id":"hellaswag:00009529","item_index":112,"attempt":0,"prompt_hash":"36ecd94b68b76c7c7d99c8f6ba803493e85d328cddd176cd1294e92c7d2a4082","question":"[header] How to detangle dreadlocks [title] Cut away old portions of your dreadlocks. [step] The older a portion of your dreadlock is, the harder it will be to untangle. Cut off any old lengths of dreadlock.\n\nA. [substeps] Cut the dreadlocks down once you get out of the shower. Cut the ends of the dreadlocks at an angle.\nB. This will make it easier to untangle the newer lengths. [substeps] Generally, lengths of dreadlocks that are over four years old are considered much more difficult to remove.\nC. [substeps] If you're detangling your dreadlocks yourself, make sure that the cut-out is straight from the top of the head to the nape of the neck. [title] Rinse the dreadlocks out with water.\nD. [title] Take your left and right dreadlocks. [step] The first thing to be gently combed out is the dread.","prompt":"[header] How to detangle dreadlocks [title] Cut away old portions of your dreadlocks. [step] The older a portion of your dreadlock is, the harder it will be to untangle. Cut off any old lengths of dreadlock.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6213,"error":null,"output":"B"}
{"question_id":"hellaswag:00006035","item_index":113,"attempt":0,"prompt_hash":"f7b2b84a1d6056eb7ed23dbedfc208392ffbbf045765d0ec44ae89dc72127ef5","question":"[header] How to fold long sleeve shirts [title] Lay out the t-shirt facedown in front of you and smooth it out. [step] Straighten up the body and sleeves, brushing out any wrinkles or folds. You can use any clean, flat surface for folding, such as a table, bed, or the floor.\n\nA. [substeps] You want the entire shirt to be flat on the surface, with nothing in between the inner and outer edges. You don't want your shirt to bunch up or crease.\nB. Lay the shirt facedown so its front is facing you, with the flat area at the top facing you. [substeps] You don't need to worry about the sleeves taking up any space near the middle of the shirt.\nC. [title] Fold the shirt in half so the sleeves line up perfectly. [step] Bring 1 side over to meet up with the other, so the 2 sides mirror each other.\nD. [substeps] Don't worry if it sticks to your clothes when you lay it out. To fold short sleeve shirts, loosen the side tabs on each of the sleeves, and pull them away so that the shirt lays flat on the floor.","prompt":"[header] How to fold long sleeve shirts [title] Lay out the t-shirt facedown in front of you and smooth it out. [step] Straighten up the body and sleeves, brushing out any wrinkles or folds. You can use any clean, flat surface for folding, such as a table, bed, or the floor.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6215,"error":null,"output":"C"}
{"question_id":"hellaswag:00007683","item_index":114,"attempt":0,"prompt_hash":"83ef514a944b2685324e866288a8ca96c95b8dddbc7043fd82066a5531eb6260","question":"[header] How to get your kids to eat [title] Cook with your kids. [step] Getting your kids involved with the process is a great way to make them more interested in food. Most kids also love to \" help, \" so ask for some assistance when preparing your next meal.\n\nA. [substeps] Fill out a casserole, and be careful not to overwhelm your kids by struggling with the casserole. Don't overcook kids, as this could take years, but it can be a good way to make sure that the kids get a good deal of vitamins and minerals.\nB. Treating cooking as a fun way to spend time together will make a positive impression on your child. This will make them more curious to test the finished product, further increasing the chances of your children wanting to eat the food.\nC. [substeps] One way to make sure your food is cooked is by encouraging your kids to handle it from the outset, by calling them by their first names. If they know the name of that food and they care about it, they can surely relate to it, and it will make it easier for them to relate to you and adhere to the dish you serve.\nD. [substeps] For example, if your kids are already fascinated by pasta, have them look up recipes on the internet to save time. Make a meal out of cucumber slices, a chopped cucumber, and chopped garlic.","prompt":"[header] How to get your kids to eat [title] Cook with your kids. [step] Getting your kids involved with the process is a great way to make them more interested in food. Most kids also love to \" help, \" so ask for some assistance when preparing your next meal.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6215,"error":null,"output":"B"}
{"question_id":"hellaswag:00003258","item_index":115,"attempt":0,"prompt_hash":"25ef5f7736383313c45a1d4491ee0a0d138ebc4775f5070a84d885ce2823ec84","question":"[header] How to choose a water dish for your rabbit [title] Avoid plastic bowls. [step] Your rabbit's bowl should be made of ceramic or metal. Plastic bowls are easily scratched.\n\nA. [title] Choose a soft surface for your rabbit to hide in. [step] Rabbits like to hide on soft, fluffy surfaces.\nB. The bottom of a plastic or metal bowl can be damaged, so make sure that your rabbit is not using them. Plastic bowls are safe for humans and are easy to clean out and replace.\nC. Bacteria can then build up in the crevices and contaminate the water. Your rabbit may also pick up a plastic bowl and treat it as a toy.\nD. Instead, use a stainless steel bowl to reduce scratching on your rabbit's paws. [title] Provide a water bowl for your bunny.","prompt":"[header] How to choose a water dish for your rabbit [title] Avoid plastic bowls. [step] Your rabbit's bowl should be made of ceramic or metal. Plastic bowls are easily scratched.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6215,"error":null,"output":"C"}
{"question_id":"hellaswag:00002324","item_index":116,"attempt":0,"prompt_hash":"ea169333ceaaa8e1eee9ea29c01fdc3a8ee15c0d42a4a50222190dc31cc5f675","question":"He then takes that mixture and adds it to a larger batch of cream which he freezes until it hardens. When the ice cream is done he tastes it and describes how good the flavor is. when the video\n\nA. is finally over another text appears that present money is being credited to the man who made the smoothie, and why.\nB. ends he says again while putting the ice cream in the sink.\nC. is finished the man is shown smiling at his camera which has been in the lower left part of the screen with a screen address next to it and a website via the website pet protection services.\nD. is over the closing credits and graphics are shown on the screen.","prompt":"He then takes that mixture and adds it to a larger batch of cream which he freezes until it hardens. When the ice cream is done he tastes it and describes how good the flavor is. when the video","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6214,"error":null,"output":"D"}
{"question_id":"hellaswag:00005189","item_index":117,"attempt":0,"prompt_hash":"346325deadd6bddc6d7f76977c8bf70c28456053e32d021f4dc4e626a3a24252","question":"[header] How to prove adultery [title] Determine why you want to prove adultery. [step] Before the advent of \" no fault \" divorce, proving adultery was one of the few grounds on which a divorce could be granted. Today, however, couples can simply claim irreconcilable differences and be divorced.\n\nA. Different reasons for wanting to be married depend on where your marriage took place. [substeps] Domestic violence: if you and your partner had a relationship that lasted over two years, you can prove adultery by fighting how long you had meant to stay apart or not spending at all.\nB. [substeps] For example, if you are married with children, you might say that you wanted adultery because your spouse had stopped living in the home and no children were born. You could claim that you wanted adultery because you're the father of a child and did not expect your children to be born on time.\nC. Nevertheless, some states allow people to file for divorce immediately if they can prove adultery. If they claim irreconcilable differences, then they may have to wait a period of time (such as six months).\nD. This is why, if not proven, \" nothing \" such as adultery isn't considered a fault. As such, you must consider whether you want to have your divorce made invalid, or how often you intend to continue the marriage to win.","prompt":"[header] How to prove adultery [title] Determine why you want to prove adultery. [step] Before the advent of \" no fault \" divorce, proving adultery was one of the few grounds on which a divorce could be granted. Today, however, couples can simply claim irreconcilable differences and be divorced.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6214,"error":null,"output":"C"}
{"question_id":"hellaswag:00006441","item_index":118,"attempt":0,"prompt_hash":"49f8d764d06b55d5320e7df76f2d2f435995d91e7db15780c40497f84f964165","question":"[header] How to thicken gravy [title] Buy flour or cornstarch. [step] You can buy both of these items at your local grocery store. Flour or cornstarch will help to thicken any sauce, and gravy is no exception.\n\nA. Cornstarch becomes a common ingredient in sauce sauces, such as gravy. It works wonders, since it forms a sticky ball and stores throughout the recipe.\nB. As long as you can avoid making lumps this option is the fastest way to thicken your gravy. [title] Mix cornstarch or flour with a little water.\nC. [substeps] Do not buy flour for a recipe that used a teaspoon salt. Flour has yeasted and it has added added oils, which means it is more likely to thicken as a thickening agent.\nD. Try to use one specific type of flour, preferably one that is made from natural or vegetable sources like hemp, wheat, or rye. [title] Whisk in butter for a richer taste.","prompt":"[header] How to thicken gravy [title] Buy flour or cornstarch. [step] You can buy both of these items at your local grocery store. Flour or cornstarch will help to thicken any sauce, and gravy is no exception.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6258,"error":null,"output":"B"}
{"question_id":"hellaswag:00000297","item_index":119,"attempt":0,"prompt_hash":"4d36d9fb72455b2eb771704a5910947bb2884631a570b5f105a46d3f2f47789a","question":"A young girl is seen scrubbing a small dog down with soap next to a tub full of water. she\n\nA. shows off her rag and blow dryer while the dog mops the area around her.\nB. pours more soap onto the dog and drags him over and leads into pictures of the dog all cleaned.\nC. continues scrubbing the dog while the dogs look on.\nD. continuously spits water all along the dog while the dog sits still.","prompt":"A young girl is seen scrubbing a small dog down with soap next to a tub full of water. she","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6258,"error":null,"output":"D"}
{"question_id":"hellaswag:00009398","item_index":120,"attempt":0,"prompt_hash":"14898b0f88fbc5c6c165729bc4afc9d85e4f89865758717ee872e2ba1c8dfbb5","question":"[header] How to do a neat middle height ponytail [title] Straighten your hair or define your curls. [step] The difference between a neat ponytail and a messy one is how much care you take with your hair. Frizzy or unkempt hair will result in a ponytail that sticks out in the wrong places or hangs funny.\n\nA. [substeps] Go for a loose pompadour or a crunchy twist. Comb it back with a hairbrush when your hair is still slightly damp and achieve a messy updo with a touch of gel to it, if your normally pull back your hair into a messy bun.\nB. If you have messy or disheveled hair, it doesn't matter what much you do with it. Make sure to clean it up so the style stays neat.\nC. Since curly or messy hairstyles make your hair seem stuck, try straightening your hair if you prefer : [substeps] Twisting your hair into a ponytail or turning it upside down. These styles create a messy, curly look instead of looking neat.\nD. Depending on your hair texture, do one of the following to neaten your hair before you even put it up : [substeps] Straighten it with a straightening iron. You don't have to meticulously straighten all of your hair; focus on the ends and the sections that will be hanging down in your ponytail.","prompt":"[header] How to do a neat middle height ponytail [title] Straighten your hair or define your curls. [step] The difference between a neat ponytail and a messy one is how much care you take with your hair. Frizzy or unkempt hair will result in a ponytail that sticks out in the wrong places or hangs funny.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6258,"error":null,"output":"D"}
{"question_id":"hellaswag:00002064","item_index":121,"attempt":0,"prompt_hash":"ab010b80a0cc44746c8b8c264cba8c3b92b43e87db4b7c4c114658c94e2429bf","question":"The man pulls on his fishing pole, goes on his knees, and continues to slowly pull the fishing pole up and the camera man who had already put the camera down bends down and grabs the fish and places it onto the ice. the man\n\nA. picks the camera up and angles the camera towards the fish on the ice and the man who was fishing is smiling, unhooks the fish then holds it up in the air.\nB. then gets up and starts fishing out of a hole in the ice fishing with a fish.\nC. starts fishing fast, spinning and reeling in the fish.\nD. is fishing in the water where he is pulling on a fish and pulling onto a fish and he pulls the fish out the ker and puts it on the ice and continues to get a fish off of the ice.","prompt":"The man pulls on his fishing pole, goes on his knees, and continues to slowly pull the fishing pole up and the camera man who had already put the camera down bends down and grabs the fish and places it onto the ice. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6258,"error":null,"output":"A"}
{"question_id":"hellaswag:00002182","item_index":122,"attempt":0,"prompt_hash":"c5de44ff21391047a359e3449332d04a520c461c4968a34ebc8f132f2f8ea2cf","question":"The girl blow dry her hair with white blower, sectioned her hair, brushed her hair with roller brush while blow drying it at the same time, she roll the brush downwards and upwards. she\n\nA. styled her hair by brushing and combing it to give more volume.\nB. put complete hair style to her hair, then she turned around to show the finished hair look, she smiled.\nC. blow dry her hair to make it shiny.\nD. also blow dry her hair to effect features of curls along her head, and also blow dry her hair from front to back.","prompt":"The girl blow dry her hair with white blower, sectioned her hair, brushed her hair with roller brush while blow drying it at the same time, she roll the brush downwards and upwards. she","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6258,"error":null,"output":"C"}
{"question_id":"hellaswag:00001185","item_index":123,"attempt":0,"prompt_hash":"53dd66ef823662cdb162155edc7867f7499380ab67eed8cde18bcb464546503d","question":"Two men are standing with others in a gym. They are engaged in a game of fencing. they\n\nA. parry back and forth with their swords.\nB. hit the point with the sticks as they simulated it.\nC. knock the other male down onto the mat as he tries to recover and get back up.\nD. kick and punching each other, and run to kick or kick someone else.","prompt":"Two men are standing with others in a gym. They are engaged in a game of fencing. they","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5958,"error":null,"output":"A"}
{"question_id":"hellaswag:00009025","item_index":124,"attempt":0,"prompt_hash":"969043741f321cad3f83babb0a052cca101f64e54035b3422e1e9dcf3774d807","question":"[header] How to find the best swimsuits for your body shape [title] Try a one-piece in a solid color. [step] Solid color suits look sleek and stylish. Your figure may be curvy enough to draw attention on its own, so there is not much need to worry about accentuating certain parts or downplaying others.\n\nA. In order to make sure that your look is flattering to your body type, you can add some oomph. [substeps] Solid colors should aim to complement your body shape.\nB. Professional swimsuits in greys, olive, cotton, or dark colors can add to modern charm. [substeps] Avoid neon or pastel colors, which appear harsh and unnatural.\nC. [title] Consider a swimsuit with a deep plunge. [step] Odds are, you have a curvy bust worth showing off.\nD. A three-piece suit is one size fits all for you and should be packed in a separate suitcase when you are visiting relatives and friends. Select, include, and keep a list of key pieces that may be best tailored to your needs.","prompt":"[header] How to find the best swimsuits for your body shape [title] Try a one-piece in a solid color. [step] Solid color suits look sleek and stylish. Your figure may be curvy enough to draw attention on its own, so there is not much need to worry about accentuating certain parts or downplaying others.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5959,"error":null,"output":"C"}
{"question_id":"hellaswag:00007553","item_index":125,"attempt":0,"prompt_hash":"346e1a5b6f65b152106be18ea89971408221fe95b07f5bfa02b5f6807f3c99bb","question":"[header] How to pack for el camino de santiago [title] Store your travel documents in a protective case. [step] If you're headed to spain from another country, you'll obviously need a passport or visa. You'll also want to carry a pilgrim credential to track your progress along the way.\n\nA. For more information, check out : [substeps] Passports must be valid unless you are a u.s. citizen who is seeking to visit someone in another country.\nB. Place these documents in a safe, secure environment in spain's central region. [substeps] Packing best if you don't have it already is imperative.\nC. Put all travel documents in a plastic case that you have easy access to so they stay safe. [substeps] If you're not sure how to get a passport in your country, contact your local government.\nD. As soon as you travel, make a note of any documents you want to bring along on your trip. [substeps] To prevent yourself from inadvertently carrying anything on the way to el camino, you should have a written copy of what you're carrying along with you.","prompt":"[header] How to pack for el camino de santiago [title] Store your travel documents in a protective case. [step] If you're headed to spain from another country, you'll obviously need a passport or visa. You'll also want to carry a pilgrim credential to track your progress along the way.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5958,"error":null,"output":"C"}
{"question_id":"hellaswag:00002801","item_index":126,"attempt":0,"prompt_hash":"b1c50255fcb0b90aeb9ae456222accc818c3d1551c7dd294c80a5f7e5ddf8402","question":"A man lifts a ball in the air and holds it against his neck and throws the ball far. The man holds the ball against his neck and twists his back and throws it. the man\n\nA. grabs the ball and takes a big step behind him and throws the ball.\nB. spins around and throws the ball, lands on his feet and throws it away.\nC. continues swinging around where he doesn't score but there's people filming him.\nD. takes another shot his mouth, then drops the ball on the ground.","prompt":"A man lifts a ball in the air and holds it against his neck and throws the ball far. The man holds the ball against his neck and twists his back and throws it. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5958,"error":null,"output":"A"}
{"question_id":"hellaswag:00003121","item_index":127,"attempt":0,"prompt_hash":"ddef7399420b7ed5aee3f847e39efcdb31024d77af73420e77ef63f9178514e9","question":"A man is seated inside a building. He is quickly playing the drums and cymbals. he\n\nA. is using a voice recorder as he plays.\nB. stops drumming and rinses his hands, stepping away from the drums.\nC. bounces his drum sticks on the ground slowly.\nD. continues playing for the duration of the video.","prompt":"A man is seated inside a building. He is quickly playing the drums and cymbals. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5957,"error":null,"output":"D"}
{"question_id":"hellaswag:00000031","item_index":128,"attempt":0,"prompt_hash":"340171a69b99fff8e5057b2a2cec8c0161dabee3ccb5382363b3be3aa93d3777","question":"A cat is sitting in a cat bed. It is licking its paw. it\n\nA. then wipes its paw on its ear.\nB. crawls to the door.\nC. 's laying down about six inches away from the camera.\nD. crawls on one foot to get closer to the camera.","prompt":"A cat is sitting in a cat bed. It is licking its paw. it","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6419,"error":null,"output":"A"}
{"question_id":"hellaswag:00007101","item_index":129,"attempt":0,"prompt_hash":"51186d4f5b71673fb67652ce936df6d05f3f475b4be24b5f8795a3817d526488","question":"[header] How to look sexy when naked [title] Choose flattering lighting. [step] The right lighting can enhance the mood and improve your confidence. Aim for soft, dim lighting.\n\nA. [substeps] The right lighting will help reflect the moisture from your skin and add shine to your outfit. Never be as bright as possible-take care of your skin.\nB. This kind of lighting softens your features and many photographers recommend it for looking your best. This type of lighting can be achieved by low overhead lighting or turning on only one lamp in the room.\nC. Consider showing off your arms so that it's flattering to your body. If you are going to try to look sexy without clearly displaying these body parts, what do you do? Use your legs, which are the sides where the lips are.\nD. It can be hard to see the eyes because pose is tilted upwards at a very slight angle. [substeps] Aim for a soft, neutral day when you know the sun is out.","prompt":"[header] How to look sexy when naked [title] Choose flattering lighting. [step] The right lighting can enhance the mood and improve your confidence. Aim for soft, dim lighting.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6419,"error":null,"output":"B"}
{"question_id":"hellaswag:00008898","item_index":130,"attempt":0,"prompt_hash":"b95e14c3c8eb348af532018d77c6f256d356d21cd8485110ad8d76e9246d03a6","question":"[header] How to solve the bikini bottom problem [title] Put on your bathing suit. [step] How much of your pubic hair shows? If it's just a small amount, it can't be too hard to get rid of. But if it's a lot you might need a little extra elbow grease.\n\nA. The bikini size problem begins when you attempt to slide the bikini over your skin. Most bikini tops are designed to withstand the heat and humidity of the sun.\nB. If you have hair-removal supplies, get it out. Otherwise, go shopping at your local department store.\nC. You might want to get yourself a notebook so you can write your answer even faster. [title] Find a pair of shorts that can cover your bikini.\nD. [title] Remove the bikini bottom from its original wrapper and place it in the microwave. [step] Stir if needed to get a little grease on the bikini bottom.","prompt":"[header] How to solve the bikini bottom problem [title] Put on your bathing suit. [step] How much of your pubic hair shows? If it's just a small amount, it can't be too hard to get rid of. But if it's a lot you might need a little extra elbow grease.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6410,"error":null,"output":"D"}
{"question_id":"hellaswag:00008210","item_index":131,"attempt":0,"prompt_hash":"d4a7ff3a1772f5847a23af576a7d5207d3837d3e467a9bfa459ea031064f3c1f","question":"[header] How to conserve battery on a windows computer [title] Get to know the basic options. [step] There are generally two power plans in windows. One is the balanced power plan and the other is the power saver plan.\n\nA. A balanced power plan will deliver power to only one controller on one computer and the rest holds your computer's entire resources. Each switch is different for each computer, so it is something to be aware of.\nB. Both select available battery for the computer and then click on the power feature to save the plan. This is usually found on the front side of the screen after you have entered the power plans.\nC. The latter plan is designed so that the battery is happy. [substeps] You can make it switch between either power options with active input.\nD. In addition to this, the user has an option to also create their own power plan based on the requirements. [substeps] The balanced power plan is generally selected as the default power plan in windows.","prompt":"[header] How to conserve battery on a windows computer [title] Get to know the basic options. [step] There are generally two power plans in windows. One is the balanced power plan and the other is the power saver plan.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6409,"error":null,"output":"D"}
{"question_id":"hellaswag:00007227","item_index":132,"attempt":0,"prompt_hash":"4e25b3d220e4a23088695291386bb7f8dde623c393dedbb830d6a3e74f6521aa","question":"[header] How to pay for funeral expenses [title] Understand the cost. [step] The average funeral costs $7 , 755 according to the national funeral directors association. For some this is not much but for others it is a number they cannot hope to reach.\n\nA. [title] Go to the website of the funeral home or crematorium that works in your area. [step] They will give you a telephone number for talking to other funeral directors.\nB. [title] Know how much money you can afford. [step] Because there are hundreds of different funeral homes in the united states, there are unlimited opportunities to fill all the gaps.\nC. This number is also a pay upfront proposition, as funeral homes will no longer accept payment plans. So when you are thinking about planning ahead, know that this is the number you need to target.\nD. Contact the people who provided you with the money and ask how much is left over and how much is going towards the death of the person. Also, if the person did not call or email at the time of the death, ask to speak to a funeral director or visit a funeral home.","prompt":"[header] How to pay for funeral expenses [title] Understand the cost. [step] The average funeral costs $7 , 755 according to the national funeral directors association. For some this is not much but for others it is a number they cannot hope to reach.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6409,"error":null,"output":"A"}
{"question_id":"hellaswag:00008964","item_index":133,"attempt":0,"prompt_hash":"be4b13a8ad6122d6fdfb3f3036eb29c46c60112f409bfdb9492beb32f42ca57d","question":"[header] How to use autoshapes in word [title] Explore the auto shapes menu. [step] Click the insert tab on the menu bar, and click on the shapes button located in the illustrations menu on the formatting toolbar. Take note of the different auto shapes now visible in the pull-down menu.\n\nA. [title] Insert an auto shape into the document. [step] Select a basic shape for the purposes of this tutorial.\nB. [substeps] Browse the shape you would like to use. Select a shape size from the categories it may appear in as you open documents and files.\nC. [title] Select a shape from the diagram that's in the illustrations. [step] Select a shape by hovering your mouse over it.\nD. [title] Click on the supertools pane. [step] It's on the left side of the microsoft word format dialog panel.","prompt":"[header] How to use autoshapes in word [title] Explore the auto shapes menu. [step] Click the insert tab on the menu bar, and click on the shapes button located in the illustrations menu on the formatting toolbar. Take note of the different auto shapes now visible in the pull-down menu.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6410,"error":null,"output":"A"}
{"question_id":"hellaswag:00001653","item_index":134,"attempt":0,"prompt_hash":"4c43999c63cac42692e88b3be39015c802fc5efa46c67e64a7c68565cacf0ad0","question":"Two men are doing martial arts with others around them watching. Many other men are shown doing several flips and tricks around each other. two men\n\nA. are shown hitting people to give them an out and another man joins in.\nB. are joined by a woman.\nC. flip near each other while others around them watch.\nD. stand on a mat while a man is bartending around them.","prompt":"Two men are doing martial arts with others around them watching. Many other men are shown doing several flips and tricks around each other. two men","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6416,"error":null,"output":"C"}
{"question_id":"hellaswag:00003380","item_index":135,"attempt":0,"prompt_hash":"a3d27657d9d538354970fa9af5ad27f66e50fd4f4bfeb8c00cba6719cee6dd59","question":"[header] How to deal with your brother in law's jealous wife [title] Stand up for yourself. [step] You are a part of the family now. Your husband chose you to be his wife.\n\nA. He has taken too long to explain himself to you about the situation, but now he wants to know that you care for yourself. You should be strong and proud of yourself.\nB. Unfortunately, as a family, we all make mistakes. Your actions, though, can hurt your wife.\nC. Now doesn't mean you have to have been blamed for everything. Instead, show him the respect he deserves by taking charge and acting confidently about everything that happens in your life.\nD. You have every right to be included as a part of this new family. Building a relationship with your in-laws is very important so do make the effort to visit often and get to know them and allow them the opportunity to get to know you.","prompt":"[header] How to deal with your brother in law's jealous wife [title] Stand up for yourself. [step] You are a part of the family now. Your husband chose you to be his wife.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6416,"error":null,"output":"D"}
{"question_id":"hellaswag:00006626","item_index":136,"attempt":0,"prompt_hash":"6c095ffa4b5eae5de78784837016d8a71982a5a8e15f79d0ee91cddfb2519f3b","question":"[header] How to prepare mentally and physically to use your krav maga training [title] Realize what it is you are looking to do. [step] You are about to embark on a journey into the darker side of humanity. You're preparing for a scenario where you may be choked, stabbed, held up with guns, ambushed by multiple attackers, put in compromising situations and forced to fight blind.\n\nA. You are preparing for this scenario to be the beginning of your journey. [title] Know what what life is-long.\nB. All of these scenarios can influence your personal and individual lifestyles, so take a few deep breaths and prepare yourself to deal with these external settings. [title] Take a deep breath and prepare for inevitable reactions.\nC. [title] Recognize the details of the many stages of the process. [step] This is the whole story for you.\nD. This is street/battlefield fighting, not a sport with rules. Don't stop! Don't ever stop.","prompt":"[header] How to prepare mentally and physically to use your krav maga training [title] Realize what it is you are looking to do. [step] You are about to embark on a journey into the darker side of humanity. You're preparing for a scenario where you may be choked, stabbed, held up with guns, ambushed by multiple attackers, put in compromising situations and forced to fight blind.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6410,"error":null,"output":"B"}
{"question_id":"hellaswag:00001477","item_index":137,"attempt":0,"prompt_hash":"8f272a8029e11bbb7556f24d42fe8d66f4ce164be70b481146fd5921beaf2992","question":"A elder man is on water standing on a jet ski. The man begins to jet ski. the man\n\nA. flips in the air continuously towards the camera.\nB. begins to jump over logs.\nC. takes on hand off of the handle.\nD. takes a turn going backwards on the jet ski.","prompt":"A elder man is on water standing on a jet ski. The man begins to jet ski. the man","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6414,"error":null,"output":"D"}
{"question_id":"hellaswag:00009207","item_index":138,"attempt":0,"prompt_hash":"a964adb0ea0d49fe07e6cb4a832956353048bea97891ec4a0348e590b81d22e7","question":"[header] How to create a frequency table with spss predictive analytics software [title] Collect data and create a title for each variable. [step] For this example you want to test the correlation (if any) between an individuals affinity for cats and their overall happiness. [title] Open the spss software.\n\nA. [step] You will need to ask a random selection of people asking for samples of each product they want to make. [title] Enter your product title into the text box.\nB. [step] Click the menu button on the side of the interface and click \" add (variable name) \". A banner will appear that shows add a mouse clicks and click on \" create new page \".\nC. [title] Get a palette of colors and images and start creating the frequency table. [step] These include picture transparencies (red to white, blue, orange to orange) or color/fotocross (red to green, and blue to green, especially if new).\nD. [step] A dialogue box will appear with several different options. Select \" type in data \", this will produce a blank spreadsheet.","prompt":"[header] How to create a frequency table with spss predictive analytics software [title] Collect data and create a title for each variable. [step] For this example you want to test the correlation (if any) between an individuals affinity for cats and their overall happiness. [title] Open the spss software.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6412,"error":null,"output":"D"}
{"question_id":"hellaswag:00001382","item_index":139,"attempt":0,"prompt_hash":"d3d961fdaa8d1907c53e4fbf3dc953e572bd50d7dcafbf3a395b6b90d79ad2ad","question":"A group of people, float in a line, down a narrow stream of water in donut shaped, black inflatable rafts. One woman gets stuck on a rock in the raft and shimmies herself loose. the people\n\nA. wave at the camera.\nB. float in the water to a fast moving waterfall.\nC. reach the end of the narrow stream where a group of people help them off of the rafts and another small group of people watches them from nearby land.\nD. continue to row while perusing dry leaves, rock as well as gold dust flying in the air.","prompt":"A group of people, float in a line, down a narrow stream of water in donut shaped, black inflatable rafts. One woman gets stuck on a rock in the raft and shimmies herself loose. the people","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6416,"error":null,"output":"A"}
{"question_id":"hellaswag:00005082","item_index":140,"attempt":0,"prompt_hash":"244755e0220ccea33e7482b3981c9c4d74dafe9e1605dd6e2374cec06f01ba01","question":"[header] How to get your ferret spayed or neutered [title] Verify that your ferret isn't already spayed or neutered. [step] Ferrets sold at pet stores are almost always already spayed or neutered. Many breeders also spay or neuter kits, or young ferrets, before selling them.\n\nA. This will ensure that your ferret will not be harassed or misredirected. [substeps] Ferrets that already have an owner must also be spayed or neutered before they sell them.\nB. [substeps] You can get your ferret spayed or neutered immediately upon the transmission of your ferret, leaving a good deposit for a new starter puppy, but will have to go through much more patience. [title] Ensure that your ferret can still eat.\nC. [title] Familiarize yourself with the type of breeding location you need. [step] The first hurdle in breeding a ferret is to get it spayed or neutered.\nD. [substeps] Purchasing a ferret that's already spayed or neutered is in your best interest, since the operation can cost a lot of money. If the store is uncertain, you can also take the ferret to the vet.","prompt":"[header] How to get your ferret spayed or neutered [title] Verify that your ferret isn't already spayed or neutered. [step] Ferrets sold at pet stores are almost always already spayed or neutered. Many breeders also spay or neuter kits, or young ferrets, before selling them.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6409,"error":null,"output":"D"}
{"question_id":"hellaswag:00001436","item_index":141,"attempt":0,"prompt_hash":"1abd216f03cb026291c91dcc340ce460734aa42a71f8381f74c77a8989ee5e47","question":"Several screens are shown of bright orange lettering on a black background. a man and a woman\n\nA. are in a kitchen, working on ingredients, soaking towels and food.\nB. are standing upright in a pool, and the man is showing the woman swimming techniques.\nC. are sumo wrestling in their room.\nD. are sitting inside a booth while a customer talks.","prompt":"Several screens are shown of bright orange lettering on a black background. a man and a woman","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6413,"error":null,"output":"B"}
{"question_id":"hellaswag:00004748","item_index":142,"attempt":0,"prompt_hash":"a1250fc55828cd886bc44ba1de5f6e8e12c45c2153d3d78f396451ce03db4baf","question":"[header] How to tell if a tattoo is infected [title] Wait a few days before coming to any conclusions. [step] The day you get a tattoo, the entire area will be red, slightly swollen, and sensitive. New tattoos will be somewhat painful, about as sore as a severe sunburn.\n\nA. If the tattoo is not infected, it will likely burn within the first six days. Simply return the tattoo to a reputable pharmacy and, if an attempt to remove it looks infected, take it to a physician.\nB. This usually abates about 10 days after the tattoo, but you still may need to see your doctor. The following days will usually be the most likely day for the tattoo to appear infected : [substeps] The sun exposure is the most severe (this is usually around mid-morning or afternoon).\nC. If you are concerned about your skin's perceived ability to heal or if you suspect something is seriously wrong, avoid handling the tattoo until that day. [substeps] Best to avoid touching your tattoo if it has already healed.\nD. In the first 48 hours of getting a tattoo, it can be very difficult to determine whether or not an infection has set in, so don't jump the gun. Maintain a proper tattoo aftercare procedure and adopt a wait and see policy.","prompt":"[header] How to tell if a tattoo is infected [title] Wait a few days before coming to any conclusions. [step] The day you get a tattoo, the entire area will be red, slightly swollen, and sensitive. New tattoos will be somewhat painful, about as sore as a severe sunburn.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6410,"error":null,"output":"D"}
{"question_id":"hellaswag:00005106","item_index":143,"attempt":0,"prompt_hash":"f7393f9aa1272a7d9d79005e098a8d1b303e62f205288147cc7f46d7dbc76bf4","question":"[header] How to adopt a child as a single man [title] Make an informed decision. [step] As a single man, you must realize the difficulties involved because of both your gender and your marital status. Recent trends make adopting children by single men more acceptable, but men are likely the most prejudiced against demographic among people looking to adopt.\n\nA. [substeps] Research which countries will allow a single man to adopt a child. Not all countries are open-minded to the possibility of having one parent let alone it being a man.\nB. [substeps] If you feel like you must adopt a child by just being single, do it. Agree on a group meeting to discuss the logistics and come to an agreement regarding time and space for adoption.\nC. Work to come up with a plan that addresses these characteristics, among other factors. Keep in mind the dividends you will face with adopting.\nD. The american academy of pediatrics recommends that single men who have had two or three children over the past two years only adopt a child like one. [substeps] This does not mean you should be single alone.","prompt":"[header] How to adopt a child as a single man [title] Make an informed decision. [step] As a single man, you must realize the difficulties involved because of both your gender and your marital status. Recent trends make adopting children by single men more acceptable, but men are likely the most prejudiced against demographic among people looking to adopt.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6410,"error":null,"output":"A"}
{"question_id":"hellaswag:00007943","item_index":144,"attempt":0,"prompt_hash":"0e610240b2c7dcf461b05979c1c624e7ced90470860a85ce1d951c0313dd1599","question":"[header] How to prepare for a school dance (for girls ) [title] Decide if you're going to it or not. [step] School dances can be fun and memorable, but not everybody likes dancing in front of a lot of people. [title] Congratulations! [step] You've decided you're going to the dance.\n\nA. How long does it take to make up your mind? If you know how many people will go with you, you can really stay with that group. If you're unsure, ask! If you're not sure, ask.\nB. Now all you have to do is relax relax. Do whatever brings you joy and has your heart racing.\nC. If you need a ticket, buy one. Some schools also require you to show them your i.d.\nD. Keep that in mind in case your parents and/or siblings are there to see it. [title] In addition, you should see your dance teacher.","prompt":"[header] How to prepare for a school dance (for girls ) [title] Decide if you're going to it or not. [step] School dances can be fun and memorable, but not everybody likes dancing in front of a lot of people. [title] Congratulations! [step] You've decided you're going to the dance.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6410,"error":null,"output":"A"}
{"question_id":"hellaswag:00000393","item_index":145,"attempt":0,"prompt_hash":"316cd7923fee8a04c67eb407a9df71032e761bff9aac6edf1b7841e794a7b644","question":"A person's hands are seen sharpening a knife over a counter and the intro says \"kitchen tricks sharpening knives\". then there\n\nA. are more close up shots of knife sharpening tools and taking them off the counter and putting the squares in the pockets.\nB. is also a title screen with a logo of the restaurant and the words \" françion's clementinwe's competitive business \".\nC. is a man made knife sitting in a wooden bowl sharpening a bowl with a knife and another using a piece of iron on a fabric.\nD. is a tools needed list on the screen and a bowl, sharpener, and other tools are seen on the counter.","prompt":"A person's hands are seen sharpening a knife over a counter and the intro says \"kitchen tricks sharpening knives\". then there","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6413,"error":null,"output":"D"}
{"question_id":"hellaswag:00007056","item_index":146,"attempt":0,"prompt_hash":"be1dc9e797816a2816f1494f9b800b5e9201a316541ef22f012b19eb0f06ef29","question":"[header] How to style ripped jeans [title] Pair your jeans with a t-shirt and sneakers. [step] This a simple and quick style. Simply choose your favorite t-shirt and sneakers and pair them with your ripped jeans.\n\nA. Then, go over the jeans with your favorite t-shirt and sneakers or get two pairs and put them on over your torn jeans. [substeps] If you really want to go for other outfits, just wear small heels with your ripped jeans.\nB. Slouchy (yet casual) jeans/sneakers would be perfect, but semi-baggy skinny jeans/sneakers made out of nylon or some sort of cotton stay-on item are popular styles to pair with ripped jeans. [title] Pair skinny jeans with a pair of rolled cuff prints.\nC. Whether it's graphic or plain, any type of t-shirt will work. [substeps] For example, wear a plain white t-shirt, ripped jeans, and a pair of plain white sneakers.\nD. Make sure the t-shirt and sneakers are loose fitting for your comfort level. You can also wear either jeans or sneakers.","prompt":"[header] How to style ripped jeans [title] Pair your jeans with a t-shirt and sneakers. [step] This a simple and quick style. Simply choose your favorite t-shirt and sneakers and pair them with your ripped jeans.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6410,"error":null,"output":"C"}
{"question_id":"hellaswag:00004505","item_index":147,"attempt":0,"prompt_hash":"d6ace190022b78711b6c6f1529f05666196cb6d7fbbb39dace1eefeff6f0b209","question":"[header] How to emphasize your eye color with your clothing [title] Pick neutrals during the day for blue eyes. [step] Blue eyes can match well with a variety of shades, but classic neutral shades go great with blue during the day. Neutrals include colors like navy, olive, gold, silver, cream, and camel.\n\nA. Carry a crop top on during the day in a dark color. [substeps] If you have blue eyes, wear a crop top over top.\nB. [substeps] Neutrals tend to go with anything. If you don't want a look that's completely lacking in color, you can use neutrals to bring out your eyes while still having some bright colors in your outfit.\nC. [substeps] Invest in comfortable appropriate shoes for the office. Loafers and trainers, for example, could work great for blue eyes.\nD. [substeps] For example, consider pairing a bright lunch dress or a bright fabric blouse with a bright jeans or blouse for a fun and classic look. [title] Wear eyeshadow when you go to the mall.","prompt":"[header] How to emphasize your eye color with your clothing [title] Pick neutrals during the day for blue eyes. [step] Blue eyes can match well with a variety of shades, but classic neutral shades go great with blue during the day. Neutrals include colors like navy, olive, gold, silver, cream, and camel.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6411,"error":null,"output":"B"}
{"question_id":"hellaswag:00005622","item_index":148,"attempt":0,"prompt_hash":"5e8b0e0647f4c6a7c24d62b4b1b4cd5c8210dd0c385e7b4e9f91b4aea1a27d90","question":"[header] How to give a baby a bath [title] Wear appropriate clothing. [step] Roll up your long sleeves, remove your jewelry, and take off any other items, such as watches, that may get in your way. Know that bathing a baby can be a wet operation and be prepared to find a change of clothes afterward.\n\nA. If you are bathing a baby, then it is best to use a diaper hose or baby bubble bath. If you are bathing a baby while breastfeeding, use a diaper hose or diaper bubble bath.\nB. You want to wear something you don't care about so you can properly bathe your baby. [title] Get all of your supplies in order.\nC. [substeps] Make sure that you water both hands until your hands are completely dry before applying any gel products and soaps. [title] Avoid applying creams, oils, or serums to your baby's skin.\nD. [substeps] Using hot water can impair the baby's circulation, leading to injuries or illness. In addition, the baby may be nervous, especially since she will be moving, and may not be able to properly bathe herself.","prompt":"[header] How to give a baby a bath [title] Wear appropriate clothing. [step] Roll up your long sleeves, remove your jewelry, and take off any other items, such as watches, that may get in your way. Know that bathing a baby can be a wet operation and be prepared to find a change of clothes afterward.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6412,"error":null,"output":"B"}
{"question_id":"hellaswag:00006119","item_index":149,"attempt":0,"prompt_hash":"ec3d311058565b1fd2aed38bf77f9cb2b4bf1b293dc297c60e3cf0486bd8fe3a","question":"[header] How to do chevron nails [title] Paint your nails. [step] You can paint your nails all one color or use a couple of different colors. Make sure that you let your nails dry all the way before you apply the tape strips.\n\nA. Black will go good with the base to get the nail finish. Get down to the big pencil marks on your nails.\nB. [substeps] If you are planning on painting your nails, then your waxer will need to work on your nails while you get them painted. You don't want to paint over and over on top of the paint, as this may damage the nail box.\nC. [title] Dispense tape and cut into approximately 1 \" strips. [step] It is up to you if you want to cut enough tape strips to make a chevron design on each of your nails or just do a couple.\nD. You can paint your nails one color, or you can also use several different colors to paint your nails from. [substeps] The french colour! The chevron is the most popular.","prompt":"[header] How to do chevron nails [title] Paint your nails. [step] You can paint your nails all one color or use a couple of different colors. Make sure that you let your nails dry all the way before you apply the tape strips.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6413,"error":null,"output":"C"}
{"question_id":"hellaswag:00002035","item_index":150,"attempt":0,"prompt_hash":"4801e0eda7c7cbbc51a07d7e29057c9db6108eda94af07d4b769728d537a6111","question":"We see two reporters outdoors, talking with members of a running team. They are preparing for a race, one of which is pregnant. the crowd\n\nA. is excited for the winner, celebrating.\nB. claps and they race, passing the finish line.\nC. is shown walking to where the race will begin.\nD. cheers and claps for the child.","prompt":"We see two reporters outdoors, talking with members of a running team. They are preparing for a race, one of which is pregnant. the crowd","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6912,"error":null,"output":"C"}
{"question_id":"hellaswag:00000570","item_index":151,"attempt":0,"prompt_hash":"6e4fe45ef57c36496c7324991710a8bc0280fba81837cbeac996011aafbac3f8","question":". a shirtless man wearing long pants and red shoes\n\nA. is standing inside a garage with another man dressed like a drunk.\nB. is skateboarding on a path in a secluded neighborhood at night and appears to be about to pass on his skateboard.\nC. is bent over and has his two hands gripping onto a barbell with two very large weights on the ends of it.\nD. prepares to perform a high dive off a stair stepper but it slides but slips.","prompt":". a shirtless man wearing long pants and red shoes","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6912,"error":null,"output":"C"}
{"question_id":"hellaswag:00008389","item_index":152,"attempt":0,"prompt_hash":"6ce386411399cf99ff5cea667f30f52c162e30a20428a05231695d4a49f70516","question":"[header] How to get the most amazing legs [title] Make it a habit to walk for thirty to forty-five minutes daily, either morning or evening, whatever time suits you. [step] Try brisk walking or light jogging at one pace. [title] If you find it hard to walk by yourself, ask a friend to come along or take your dog or an ipod with you.\n\nA. [title] Find a comfortable spot to work out. [title] Since our main focus is on our thighs and calves we'll stress more on those areas.\nB. [title] Look at all the beautiful beauty on the planet. [step] There is no shame in dying yourself white! Make sure you're rich enough to do it.\nC. [step] If you aren't able to cross the street, try other ways of improving your balance and power through physical challenges. [title] Even if you're not a great person, you can still do great things on your own.\nD. [step] Try not to get too long strides, as you can get tired faster than if the walk is done slowly. Be careful not to exceed 30 minutes, as light strides often slow down.","prompt":"[header] How to get the most amazing legs [title] Make it a habit to walk for thirty to forty-five minutes daily, either morning or evening, whatever time suits you. [step] Try brisk walking or light jogging at one pace. [title] If you find it hard to walk by yourself, ask a friend to come along or take your dog or an ipod with you.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6912,"error":null,"output":"C"}
{"question_id":"hellaswag:00006093","item_index":153,"attempt":0,"prompt_hash":"30674ee98bc0be9d83950486de909225f667f1e71fc788e545c5da6388eab38b","question":"[header] How to manage layered hair [title] Blow-dry your hair when you plan on styling it. [step] Blow-drying your hair will make the layers more noticeable and bouncy. Use a blow dryer on your hair after your shampoo and conditioning routine in the shower.\n\nA. Blow dry your hair in a low ponytail low to the scalp. [substeps] To safely style your bangs from here on, brush the hair upward once and then forward as you blow dry your hair.\nB. [substeps] Be gentle with your hair as you style it. Hold your hair section horizontal and swing it back and forth a lot if using a straight section.\nC. Start drying your hair by flipping it over and going over the underside portion of your hair for a few minutes. Use your fingers to pull lightly at your roots and then move tem towards the ends of your hair as you blow dry it.\nD. Hold the blow dryer right next to your roots and let it sit for 30 seconds to a minute, then remove it. [title] Apply a leave-in conditioner with volumizing power.","prompt":"[header] How to manage layered hair [title] Blow-dry your hair when you plan on styling it. [step] Blow-drying your hair will make the layers more noticeable and bouncy. Use a blow dryer on your hair after your shampoo and conditioning routine in the shower.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6912,"error":null,"output":"C"}
{"question_id":"hellaswag:00009344","item_index":154,"attempt":0,"prompt_hash":"29e286dd953827355c38f4b5fcc116716acb7a27be34081bd2f397faeb8c51b7","question":"[header] How to measure inseam on jeans [title] Grab a pair of jeans and lay them flat. [step] Stretch them out over a solid, smooth surface that's large enough to hold the entire pair of pants. If you're trying to find your optimal inseam, pick a pair of jeans that fit you well and whose legs are your preferred length.\n\nA. [substeps] If it is possible, lay the jeans flat on a smooth surface. If one leg is dramatically shorter than the other, something with a built-in hem can help.\nB. [substeps] Measuring a pair of jeans works best if the jeans are properly fitted to your body type. [title] Smooth the pants out and, if necessary, fold them in half.\nC. Your ideal inseam measurement comes in the feet. [substeps] The feet should be the same width as your arms.\nD. [substeps] If you want your jeans to have some variation in legs' height, just go with the hottest variety of jeans. You shouldn't go for jeans that extend beyond your hips.","prompt":"[header] How to measure inseam on jeans [title] Grab a pair of jeans and lay them flat. [step] Stretch them out over a solid, smooth surface that's large enough to hold the entire pair of pants. If you're trying to find your optimal inseam, pick a pair of jeans that fit you well and whose legs are your preferred length.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6911,"error":null,"output":"B"}
{"question_id":"hellaswag:00006957","item_index":155,"attempt":0,"prompt_hash":"776d7b3d77d9c3035821b5f4ed8ef89cb8b3831c20c897f9e74e6dede3f91086","question":"[header] How to get rid of the dark area around the mouth [title] Understand why you have dark spots around your mouth. [step] These spots result from high amounts of skin-darkening melanin in certain areas of your skin. This melanin can be set off by triggers from inside and outside your body.\n\nA. These can include the following : [substeps] Exposure to natural oils. Sunburns are caused by skin exposing the mole in much the same way that exposure to ultraviolet light can expose it to full-spectrum sunlight.\nB. Darkening spots can happen early on in life, but they are commonly found around traumatic events, such as a traumatic incident at work or around religious holidays. [substeps] People with dark spots may develop dark marks during pregnancy.\nC. This melanin condition is called hyperpigmentation. The triggers might include sun exposure, melasma, and skin inflammation.\nD. This can be accompanied by skin redness, itching, swelling, bad breath, and extreme sensitivity to light. The darker the spot, the more melanin it masks.","prompt":"[header] How to get rid of the dark area around the mouth [title] Understand why you have dark spots around your mouth. [step] These spots result from high amounts of skin-darkening melanin in certain areas of your skin. This melanin can be set off by triggers from inside and outside your body.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6847,"error":null,"output":"C"}
{"question_id":"hellaswag:00006777","item_index":156,"attempt":0,"prompt_hash":"d3beea06058c93b2ae0c4ffbfe41321b556607e992f137ccef5ca6e7cb680c1b","question":"[header] How to use hair removal cream on your face [title] Read the product label. [step] While the process might seem obvious, it's important to read the directions and make sure you understand them before using the cream. Different brands of hair removal cream have slightly different instructions.\n\nA. [substeps] Additionally, this will allow you to learn of potential side effects and also check ingredients for anything you may be allergic to. Make sure the cream is designed for facial use.\nB. [substeps] If you use the product twice a week or less than the amount instructed by the manufacturer, you may experience side effects. Talk to your doctor or pharmacist about this if you're concerned about using the product too often.\nC. The less information you can provide about treatments, the better prepared you'll be for others that may have similar instructions. [substeps] For example, if a hair care product specifically intended for your face requires your hair to be chemically treated, it may take months for the product to properly get into your pores.\nD. Regardless of what product you use, be sure to read the label carefully. There may be a specific ingredient that predisposes you to having an allergic reaction.","prompt":"[header] How to use hair removal cream on your face [title] Read the product label. [step] While the process might seem obvious, it's important to read the directions and make sure you understand them before using the cream. Different brands of hair removal cream have slightly different instructions.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6848,"error":null,"output":"A"}
{"question_id":"hellaswag:00006205","item_index":157,"attempt":0,"prompt_hash":"282003fbe25d1df193d5d179115f422648be0c911303af94188936565cc39103","question":"[header] How to trim a mustache [title] Grow the hair above your lip. [step] If you can wait until it's thick and long, you'll have more to work with when it comes to achieving a certain look with your trim. [substeps] If this is your first mustache, it may take several weeks until your hair is long enough to trim.\n\nA. This will give your mustache some time to grow in and blend in with your skin. You may need to trim the area every day.\nB. You should snip your mustache as straight as possible before doing so. Loosen the hairs using a pair of sharp scissors and trim them off with a barber knife.\nC. Begin etching out the shape of your mustache by shaving your cheeks and chin while you wait for your mustache to grow. Have some patience: if you start to trim your mustache too soon, you won't get the best results.\nD. A laser-straight trimmer will be more effective but less efficient than a handheld trimmer. To make it harder to trim your mustache properly, tape down your trim to your hand.","prompt":"[header] How to trim a mustache [title] Grow the hair above your lip. [step] If you can wait until it's thick and long, you'll have more to work with when it comes to achieving a certain look with your trim. [substeps] If this is your first mustache, it may take several weeks until your hair is long enough to trim.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6849,"error":null,"output":"C"}
{"question_id":"hellaswag:00005699","item_index":158,"attempt":0,"prompt_hash":"bc357fda45ae581b46c6da7d074b746df3819a9088a52e61efc85a6db5488451","question":"[header] How to make puffy paint [title] Combine the glue and shaving cream. [step] Add equal parts of white glue and shaving cream to a large bowl. Mix the two together until they are fully combined in a thick cream.\n\nA. Spread the mixture over all surfaces of the canvas. [substeps] You will be creating the puffy paint in the shower, so mix the two at a time to ensure an even coverage.\nB. Apply this mixture to the puffy paint. [substeps] You can also use a gel; for this recipe, use one of the products described in the section below.\nC. [substeps] Be sure to use a shaving foam, not shaving gel. If you're concerned about the paint sticking to your spoons, you can use chopsticks, popsicle sticks, or even straws to mix the glue and shaving cream together.\nD. [substeps] When making your own puffer paint, follow the instructions on the package. Mix the two together until they are wet.","prompt":"[header] How to make puffy paint [title] Combine the glue and shaving cream. [step] Add equal parts of white glue and shaving cream to a large bowl. Mix the two together until they are fully combined in a thick cream.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6851,"error":null,"output":"C"}
{"question_id":"hellaswag:00009476","item_index":159,"attempt":0,"prompt_hash":"63ebfa372417b7e5d4bc7d13a63fa9fc18b522798b96322528fd8d0543120c12","question":"[header] How to teach your child the importance of gratitude [title] Start young. [step] Teach her the \" polite words \" as early as possible. When your child is a toddler, prompt her to say \" please \" whenever she asks for something and \" thank you \" whenever she's given something, no matter how small.\n\nA. Teach her by saying the \" please \" repeatedly or just let her repeat it for you. For example, ask her to replace \" please \" with \" nice \" or \" thank you.\nB. From a young age she'll understand that we must show respect and appreciation to those who do things for us. If she's too young to say the words, say them for her.\nC. In order to teach love, train your child to \" please. \" this will improve teaching and help start the process off with good manners.\nD. When you're also teaching her to associate a single action with thanks, it can help strengthen her desire to practice gratitude. Your child should have a clear goal in mind when making attempts to start a gratitude journal.","prompt":"[header] How to teach your child the importance of gratitude [title] Start young. [step] Teach her the \" polite words \" as early as possible. When your child is a toddler, prompt her to say \" please \" whenever she asks for something and \" thank you \" whenever she's given something, no matter how small.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6850,"error":null,"output":"B"}
{"question_id":"hellaswag:00006533","item_index":160,"attempt":0,"prompt_hash":"7a2b297d96af94b18eca182676df651b19704105b7251815d6023f262dcace01","question":"[header] How to keep a parenting journal [title] Write consistently. [step] When keeping a custody journal, it's imperative that you write in it regularly in order to make it as reliable and authoritative as possible. By recording information every day--or even several times a day--you can be sure that what you are writing is as accurate and useful as possible.\n\nA. [title] Review your journal every night before bed. [step] For one thing, your journal may come to an end, in which case, in order to keep track of what you are doing you will need to rewrite it.\nB. [substeps] Consider starting your journal with pictures, notes, and any other information that comes to mind during your interactions with your ex. Think about whether you have questions or need more information.\nC. [substeps] If you write less often, you run the risk of being vague and nonspecific. You may also simply forget details you meant to record.\nD. A good rule of thumb is to keep track of your journal entries. [substeps] Keep the entries in your original journal and try to keep your xeroxed journal as detailed as possible.","prompt":"[header] How to keep a parenting journal [title] Write consistently. [step] When keeping a custody journal, it's imperative that you write in it regularly in order to make it as reliable and authoritative as possible. By recording information every day--or even several times a day--you can be sure that what you are writing is as accurate and useful as possible.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5495,"error":null,"output":"B"}
{"question_id":"hellaswag:00006674","item_index":161,"attempt":0,"prompt_hash":"c92a924ed78036a53d3b8bef0faae41d3a01c87e6cbcba9c6d31267a27d4ef82","question":"[header] How to babysit a toddler [title] Don't leave them alone. [step] Be alert. Keep an eye on them all the time; you never know what one may try to do, open up, dig out ,...fall off or pull down.\n\nA. Don't even leave the room for a second. You would be surprised what kind of mess a toddler can make in the time that you take to use the bathroom.\nB. [title] Teach toddlers to do something different and exciting. [step] Whether it's silly, exciting, biblical, informative ,...etc.\nC. Call your friends and neighbors to your/their home, make sure they can hear you, if you hear nothing then they are unlikely to feel \" ok \" when you ever consider sending them a message just to listen. [title] Stay busy while you're babysitting.\nD. [title] Act when you're not around them. [step] The idea is that if the baby sees the action, they'll see you as a good babysitter.","prompt":"[header] How to babysit a toddler [title] Don't leave them alone. [step] Be alert. Keep an eye on them all the time; you never know what one may try to do, open up, dig out ,...fall off or pull down.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5495,"error":null,"output":"A"}
{"question_id":"hellaswag:00009364","item_index":162,"attempt":0,"prompt_hash":"eda5b09a776acd6d4fe1ab2cc05b40cfb87345b08f2d2fe2c20fcde6daa2e783","question":"[header] How to make hair ends softer [title] Find the right shampoo and conditioner. [step] One of the keys to making the ends of your hair softer is to keep your hair healthy and strong. In order to do that, you need to choose products that will do the job most effectively.\n\nA. Once you have a good idea of what types of products you can use, you can pick the conditioner that makes the best for your hair. [substeps] Look for shampoos that are specifically formulated for your hair.\nB. When your hair is dry, the ends of your hair can become brittle and fall out. [substeps] When your hair is dry, your hair loses its shape and texture.\nC. Each person has individual hair needs, so pay close attention to the shampoo and conditioner you use. [substeps] If your hair is dry, look for a shampoo that boasts extra hydration.\nD. [substeps] Take a good shampoo and conditioner from your local grocery store. Use a few drops of your favorite essential oil like cetaphil (or clary sage oil).","prompt":"[header] How to make hair ends softer [title] Find the right shampoo and conditioner. [step] One of the keys to making the ends of your hair softer is to keep your hair healthy and strong. In order to do that, you need to choose products that will do the job most effectively.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5495,"error":null,"output":"A"}
{"question_id":"hellaswag:00004978","item_index":163,"attempt":0,"prompt_hash":"af31fd7590a8e023501c5b1363d13ab0921f8b9a62f7af8e00047f68910557b8","question":"[header] How to build a hotel on roblox [title] Open roblox studio. [step] Go to start menu] all programs] roblox] roblox studio. When you arrive at roblox, open up the place that you want to build your hotel at.\n\nA. [title] Draw up your hotel at roblox studio. [step] You can use the create message tool on roblox studio, or you can save your design for a different site.\nB. [title] Enter your city, state, area, etc. [step] You may change your city at any time.\nC. [title] Press \" edit \" to go into edit mode. [step] Edit mode is easier than build mode, but you can go to build mode as well.\nD. [title] Make sure your sketch is in place on the studio wall. [step] This will take a few seconds as you type in the image.","prompt":"[header] How to build a hotel on roblox [title] Open roblox studio. [step] Go to start menu] all programs] roblox] roblox studio. When you arrive at roblox, open up the place that you want to build your hotel at.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5494,"error":null,"output":"C"}
{"question_id":"hellaswag:00007221","item_index":164,"attempt":0,"prompt_hash":"75f0cb032f14863ea367b4f9534af324c970aecc4d90086d3f2b9cf48dcbbb18","question":"[header] How to be a subtly sexy teen [title] Keep yourself clean. [step] You can't be sexy if you smell bad and have greasy hair. Use a good cleansing shampoo, and a hydrating conditioner.\n\nA. When you shower, rinse your hair well with warm water and then dry your hair completely. Girls who aren't clean want to smell and want to feel good, so keeping your hair clean is key.\nB. [title] Take a shower when you've got it and brush your hair. [step] Don't go looking bad; just look natural! Don't put shampoo into your hair when it's damp because it dries you out quicker and your hair is harder to grow back.\nC. Shampoo gently but firmly, and wash your hair thoroughly. [substeps] If your hair is greasy, use a natural body wash.\nD. Leave the conditioner in for about a minute, and shampoo twice if you have naturally oily hair. Shave your legs, under arms, and bikini line everyday.","prompt":"[header] How to be a subtly sexy teen [title] Keep yourself clean. [step] You can't be sexy if you smell bad and have greasy hair. Use a good cleansing shampoo, and a hydrating conditioner.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5497,"error":null,"output":"D"}
{"question_id":"hellaswag:00006636","item_index":165,"attempt":0,"prompt_hash":"63a04e30f89624036ff6083d43595f78536a0831ad27ccf0406477237ad888d6","question":"[header] How to structure your ebay auctions for maximum success [title] Choose your title carefully. [step] You need to be very clear and specific in the title. People should be able to see at a glance exactly what you're selling.\n\nA. If you give people the wrong title, then the sellers will gain a flat commission and, if your name becomes synonymous with the ebay option, you will lose future customers. [title] Select your category and set it up.\nB. The title should get your potential customers interest and grab their attention right away. Spend a good deal of time crafting a killer title for your product.\nC. You should also have a title worth both both to people and to yourself. If you are selling an item or line, you should always choose an ebay name and value that to your success.\nD. Think about your title's purpose, your sort of product, and the level of competition you can expect to have in your auction catalogue. [substeps] You may want to choose a title that is easily recognizable or that will make it stand out from the crowd.","prompt":"[header] How to structure your ebay auctions for maximum success [title] Choose your title carefully. [step] You need to be very clear and specific in the title. People should be able to see at a glance exactly what you're selling.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5493,"error":null,"output":"B"}
{"question_id":"hellaswag:00002606","item_index":166,"attempt":0,"prompt_hash":"c981de7f999a673c59258a9b1fc02fd9f9d0e61c9f7afb9cb81e6daea3a5c059","question":"An animated field of a track is shown filled with a crowd of people. an african american female\n\nA. athlete runs and meets a bus on the field.\nB. takes off running and begins to attempt a high jump but fails when the bar drops and walks off.\nC. gymnast runs up and down the field on a pair of exercise pushed ellipticals, then jumps up onto a platform in slow motion while the crowd watches from a distance.\nD. athlete runs fast towards a long pole.","prompt":"An animated field of a track is shown filled with a crowd of people. an african american female","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5496,"error":null,"output":"B"}
{"question_id":"hellaswag:00000697","item_index":167,"attempt":0,"prompt_hash":"8f0c2801052a7499252841938f74d9ccb34987c3212e00b85405695a9867fe32","question":"The man then grabs a second ball and continues to demonstrate advanced mikan drills with both balls. when the man\n\nA. is finished, he runs back to congratulate his balls and congratulate the others.\nB. is done with the drills he's talking again and the outro appears with his contact information and a subscribe button while a small video shows some of his other skills.\nC. is done with his count, he turns the pinata and begins to pour some water.\nD. 's moves are done, he drops two balls on the ground before piling them.","prompt":"The man then grabs a second ball and continues to demonstrate advanced mikan drills with both balls. when the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5496,"error":null,"output":"B"}
{"question_id":"hellaswag:00002001","item_index":168,"attempt":0,"prompt_hash":"6fd07293ea867ba6688f05aa60de805ce4b84b1836ed2e58dd74575347a1b760","question":"The camera pans around an athletic woman standing ready and others watching her on the side. she\n\nA. falls off into the sand pit.\nB. is then seen warming up and throwing a shot put and the audience yelling and cheering.\nC. stands ready demonstrating some gymnastics moves and stumbles into the pit.\nD. jumps high over a pole and walks away as her jump is shown again in slow motion and another runner steps up.","prompt":"The camera pans around an athletic woman standing ready and others watching her on the side. she","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5495,"error":null,"output":"D"}
{"question_id":"hellaswag:00003147","item_index":169,"attempt":0,"prompt_hash":"61b437e7c1b05c0771e95d15a890e7aeac9ab864b1cc776e643346db8c612214","question":"He sits at a makeup chair speaking to a makeup artist. He eats pizza with a group of friends. He sprays his mouth with mouthwash. he\n\nA. finishes and looks down at his camera.\nB. blows his breath into the air.\nC. complains that his mouth is full of toothpaste and that he is blowing bubbles at the camera.\nD. blows the smoke and does makeup jokes.","prompt":"He sits at a makeup chair speaking to a makeup artist. He eats pizza with a group of friends. He sprays his mouth with mouthwash. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5496,"error":null,"output":"B"}
{"question_id":"hellaswag:00001644","item_index":170,"attempt":0,"prompt_hash":"9bc9c049188873c1632c05adf32332ae4f8279a41585b2182180632bcfb1cb88","question":"A group of people are running down a road with various people cheering and clapping on the sides. A woman is high fiving the people as well other runners. more shots of people on the sides\n\nA. are shown while the runners continue running.\nB. of the road under the mountain are shown.\nC. are shown that leads into people eating food and celebrating.\nD. of the road are shown with views of the audience.","prompt":"A group of people are running down a road with various people cheering and clapping on the sides. A woman is high fiving the people as well other runners. more shots of people on the sides","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5495,"error":null,"output":"A"}
{"question_id":"hellaswag:00002380","item_index":171,"attempt":0,"prompt_hash":"84d35dca6a17680741f95823d240b2b1291b0f1856a5cfb63307c62cc75ec2ab","question":"Rice, flour and pepper is in a kitchen's floor. someone\n\nA. pours all of the ingredients on the floor and starts to make a cake.\nB. is holding a vacuum cleaner and turn it on to clean the floor someone is vacuuming the floor with an orange vacuum cleaner.\nC. holds a measuring cup and pour everything on the floor.\nD. is putting the rice, pepper and powder into a bucket with soap.","prompt":"Rice, flour and pepper is in a kitchen's floor. someone","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5494,"error":null,"output":"B"}
{"question_id":"hellaswag:00007907","item_index":172,"attempt":0,"prompt_hash":"5ba5d303ed0375a493e639c06790da373ad6a7092760e57507e02fa98adc9bef","question":"[header] How to create a beauty regime [title] To begin, look in the mirror. [step] Have acne? Consider wearing a primer and foundation. Dark circles under your eyes? Concealer is the answer.\n\nA. Makeup? You should consider wearing it in the morning. After applying the primer, lightly cleanse your face with water.\nB. Run a tiny powder over your face so no blemishes. Even pale pimples seem big! Find a product you like and keep it on your face but don't smudge or tear up.\nC. Want your eyes to pop? Eyeliner and mascara is for you. [title] After deciding what you need to wear, go shopping for your makeup, if you don't already have it.\nD. Try to smile) and try to keep your lips and gums clean. In the morning, you can go to sleep refreshed! Do not forget moisturizers.","prompt":"[header] How to create a beauty regime [title] To begin, look in the mirror. [step] Have acne? Consider wearing a primer and foundation. Dark circles under your eyes? Concealer is the answer.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5495,"error":null,"output":"C"}
{"question_id":"hellaswag:00006255","item_index":173,"attempt":0,"prompt_hash":"cab5b64b10d2e10dc491a0caae43f88a6cafd41d6582fcf6ab45c544eeedfaec","question":"[header] How to get car title in your name [title] Buy a car from a dealership. [step] A transfer of ownership takes place whenever a car changes registered owners. One of the most common ways this takes place is when you buy a car from a dealership.\n\nA. The dealer will then transfer the title to you. Similar to buying a foreign car, you cannot transfer your title.\nB. Different dealerships offer lower interest rates during escrow, and state exchanges schedule the transfer. You are responsible for contacting a dealer to have your title transferred into your name.\nC. However, auto dealers do not provide the same, exact contents as dealers so going into that new automotive junkyard may come with some confusion. [substeps] A dealership will not hold an item a back-lease dealer holds.\nD. Before you buy the car, it is owned by the dealership or the manufacturer. Once you purchase the vehicle, title needs to be transferred into your name.","prompt":"[header] How to get car title in your name [title] Buy a car from a dealership. [step] A transfer of ownership takes place whenever a car changes registered owners. One of the most common ways this takes place is when you buy a car from a dealership.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5491,"error":null,"output":"D"}
{"question_id":"hellaswag:00007151","item_index":174,"attempt":0,"prompt_hash":"0b391b2105c040392133fd3fa34fb219ffa7fad0fd50ae4dadd6e16a507ae206","question":"[header] How to repair jewelry [title] Assess where the hinge attaches. [step] Save all original parts; they will be used to make a pattern for the new hinge. [title] Use a ball burr to grind off the head of the rivet where the hinge attaches with a wire.\n\nA. [title] Loosen the prongs on the back of the hinge. [step] Be sure to do the same with the headstocks; if the back is too exposed, the pieces may crack.\nB. [step] A bench burr will work perfectly fine for this job. [title] Apply hydrogen peroxide or cornstarch to the body of the hinge when grinding off the head of the piece.\nC. [step] The head of the ball burr can be picked up by turning this burr up and down with a wire cutter, completing the pattern. [title] Pinch the lip of the screw with pliers.\nD. [step] Make a divot into the bracelet or necklace, where the hinge is attached. Use a small steel punch to push out the pin.","prompt":"[header] How to repair jewelry [title] Assess where the hinge attaches. [step] Save all original parts; they will be used to make a pattern for the new hinge. [title] Use a ball burr to grind off the head of the rivet where the hinge attaches with a wire.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5492,"error":null,"output":"D"}
{"question_id":"hellaswag:00008218","item_index":175,"attempt":0,"prompt_hash":"874ac9a920619afd5e9e3bad534f2bbbb21599b199a95d7abfde7141b810dff3","question":"[header] How to avoid drawing anime style [title] Realize that it might simply be that the person judging your work thinks everything is anime. [step] If so, try telling the person that what you draw isn't always anime, then show them an anime drawing and a non-anime drawing and point out the differences. [title] Look for the tell-tale signs that your work is anime : [step] Big eyes, small nose, shiny or large hair.\n\nA. Any of these may be a sign that you are having a hard time drawing anime clothes. If this is the case, look up pictures of anime outfits online.\nB. Is your work any of these things? [title] If your characters have large eyes, keep them. [step] They're expressive, which is useful in all cartoons.\nC. Watch out for the errors people make when they are critiquing that style. [title] Look for errors and ask the person to proofread their work.\nD. Big eyes are often left unaccepted, they are often your only feature. Do your research and do not identify with the anime that you have drawn.","prompt":"[header] How to avoid drawing anime style [title] Realize that it might simply be that the person judging your work thinks everything is anime. [step] If so, try telling the person that what you draw isn't always anime, then show them an anime drawing and a non-anime drawing and point out the differences. [title] Look for the tell-tale signs that your work is anime : [step] Big eyes, small nose, shiny or large hair.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5490,"error":null,"output":"A"}
{"question_id":"hellaswag:00002275","item_index":176,"attempt":0,"prompt_hash":"e7790cb025fa3127d93187f250dcd419430439729d6cbb341bda722161100a25","question":"A man is trying to lift weights in a gym. He lifts the barbell up to his chest. he\n\nA. puts it down to his belly button.\nB. then lifts it over his head before dropping it to the ground.\nC. lifts the barbell over his head and falls to the ground.\nD. lifts it over his head, then back down.","prompt":"A man is trying to lift weights in a gym. He lifts the barbell up to his chest. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5492,"error":null,"output":"B"}
{"question_id":"hellaswag:00003942","item_index":177,"attempt":0,"prompt_hash":"b292687e463efc244df4a08e48c687e33b5c58a93f6bd701152852539d0b33ba","question":"[header] How to disinfect earplugs [title] Inspect your reusable earplugs, first. [step] You are checking to see if your earplugs are torn, bent, or excessively dirty. [substeps] Your earplugs can harden and become rigid if they are covered in a lot of earwax and skin oils from your ear canal.\n\nA. If you are concerned that the earplugs do not have enough polish, consider replacing the lock caps. The lock caps on areas that get a lot of wax can become bent, or cracked.\nB. This loss of flexibility will prevent you from getting a good seal when you use your earplugs. Your ear canal connects your outer ear with your eardrum.\nC. Get a replacement pair if you plan to keep them. Tape the replacement tubes to your earplugs, (don't stuff them), clean them in/between tubes, and pop each tube out.\nD. When using a plastic cleaning device, ensure that it is fully dry before using it on your ear buds. Use a clean, lint-free cloth before applying your cleaning solution to keep your earplugs from sticking to your ear.","prompt":"[header] How to disinfect earplugs [title] Inspect your reusable earplugs, first. [step] You are checking to see if your earplugs are torn, bent, or excessively dirty. [substeps] Your earplugs can harden and become rigid if they are covered in a lot of earwax and skin oils from your ear canal.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5491,"error":null,"output":"B"}
{"question_id":"hellaswag:00000557","item_index":178,"attempt":0,"prompt_hash":"9cf867f5fdac6237785f208feb36c14f0b9523e8f0d410c1033b661d227ad7ec","question":"Person drives the red riding lawnmower across the yard. the dogs\n\nA. chasing the mower stop to watch them.\nB. protect the man as he runs around the yard.\nC. run across the yard.\nD. bends over as the person drives the lawnmower.","prompt":"Person drives the red riding lawnmower across the yard. the dogs","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5492,"error":null,"output":"C"}
{"question_id":"hellaswag:00006194","item_index":179,"attempt":0,"prompt_hash":"d76a544ee257194b5b888b4596411f3fea3150495fad4facead02b0bac6bb67c","question":"[header] How to take softgels [title] Read the instructions on your softgel package to find your dosage. [step] Your dosage will be based on age and symptoms, and your package should detail this thoroughly. Every softgel will give different instructions based on the medication type.\n\nA. For example, some common softgel's can be taken exactly as normal. Others may need to be adjusted depending on age, a particular medication, and the manufacturing process that is being performed.\nB. Sometimes a specific drug or supplement will simply have to be used. Be aware of these instructions before beginning usage.\nC. [substeps] A typical dosage could be to take 2 softgels with water every 4 hours for adults and children 12 years or over. Reviewing instructions is especially important if taking daytime or nighttime softgel products.\nD. [title] Clean and sanitize your softgel container on a regular basis. [step] It is recommended that you disinfect your container of any microorganisms, medical waste, bacteria, or infections before using it to take softgel.","prompt":"[header] How to take softgels [title] Read the instructions on your softgel package to find your dosage. [step] Your dosage will be based on age and symptoms, and your package should detail this thoroughly. Every softgel will give different instructions based on the medication type.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5492,"error":null,"output":"C"}
{"question_id":"hellaswag:00004408","item_index":180,"attempt":0,"prompt_hash":"863d703b35eec4e3aca0743327339cf072335ee855736a8888e98a96f5943977","question":"[header] How to do cute middle school hairstyles [title] Rock a ponytail. [step] Though it's a classic style, there are plenty of ways to freshen up a ponytail and make it look older for middle school. [substeps] Wear a high ponytail.\n\nA. Throw your hair up into a high ponytail to hide it from view. Lay a cute towel around your pillow to catch the bobby pins and paste.\nB. Wrap a ribbon around your shoulder and tie with a scarf. This way your ponytail will also be able to be braided.\nC. This looks especially great if you have medium-length hair. After you put your ponytail in place and secure it with elastic, pull on the top and bottom of the ponytail right beside the band.\nD. Ponytails are cute, but they will put a lot of tension on the bottom part of your hair. [title] Dress up the style with updos and conditioners.","prompt":"[header] How to do cute middle school hairstyles [title] Rock a ponytail. [step] Though it's a classic style, there are plenty of ways to freshen up a ponytail and make it look older for middle school. [substeps] Wear a high ponytail.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5491,"error":null,"output":"C"}
{"question_id":"hellaswag:00009958","item_index":181,"attempt":0,"prompt_hash":"2617a80eae696f771294a422f1e07936424450cca3944b6b64e276101699a892","question":"[header] How to calculate how much money you need to retire [title] Determine basic living expenses. [step] An important first step is to determine the amount necessary to cover basic living expenses each year. There are different perspectives on how much this will be.\n\nA. Educate yourself about how much your monthly living expenses will actually be. [substeps] Calculate your annual expenses by multiplying your spending money by your monthly contributions.\nB. Paying the conventional living allowance or about $500 dollars per month should cover relatively minor expenses, including mowing the lawn. To realize the safety issues you will need to write a few charts describing some minor expenses that you might not even consider doing if all of the above haven't been considered.\nC. [substeps] Some experts believe you should simply calculate your current expenses. Then, expect you'll need about the same amount to live on once you retire.\nD. Some sources suggest as much as five times the market cost per year. Others suggest over eight times the actual cost for basic living.","prompt":"[header] How to calculate how much money you need to retire [title] Determine basic living expenses. [step] An important first step is to determine the amount necessary to cover basic living expenses each year. There are different perspectives on how much this will be.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5491,"error":null,"output":"C"}
{"question_id":"hellaswag:00003606","item_index":182,"attempt":0,"prompt_hash":"e4a7bc897fe6dae693b38deac81d6e6e853effb72bc8c8e1d6cb7709fc011b8f","question":"[header] How to stay beautiful [title] Adhere to a proper diet. [step] Most people think of dieting as cutting back on food, or at least eating less of the things you want. Fortunately, this is not the case.\n\nA. Even though eating less is important, some people find that excessive calories will trigger body type issues like a body mass index, which is the fact that people aren't too quick to begin with a meal. [substeps] Inadequate sources of protein (such as meat or peas) are also linked to body image issues like muscle loss.\nB. Instead, a proper and sustainable diet replaces empty calories (like processed cheeseburgers and soda) with healthy alternatives. Depending on what your diet currently consists of, you can probably think of at least a few things you eat that your body would technically be better off without.\nC. You simply remember your body and the things you need to eat to stay healthy. Get enough calories and get into shape.\nD. There is a tendency of people to end up missing out on calories because they're trying to stay healthy. In fact, it can be hard to keep up with it because you simply try to make yourself feel better.","prompt":"[header] How to stay beautiful [title] Adhere to a proper diet. [step] Most people think of dieting as cutting back on food, or at least eating less of the things you want. Fortunately, this is not the case.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5389,"error":null,"output":"B"}
{"question_id":"hellaswag:00009961","item_index":183,"attempt":0,"prompt_hash":"7f8c041bf0f17f14dc84605530363999858df591135433fc312fd30658a889d3","question":"[header] How to cook a bone in ham [title] Determine the size of ham you need. [step] Generally bone-in roasts weigh 10 lbs. ( 4.5 kg) or more.\n\nA. If you plan to cook whole ham without bones, add more or less meat. [substeps] Russet, white, and red ham are the best types of ham.\nB. [substeps] Most boneless or fatter roasts are between 10-16 lbs. ( 1.5 kg) or more.\nC. Ham caramelizes greatly in its cooking process, therefore the size must be roughly the same. For this reason, most ham isn't as thick as the rest of the ham.\nD. They can feed a large family on a special occasion. [title] Pick an uncooked ham, if you want to cook it yourself.","prompt":"[header] How to cook a bone in ham [title] Determine the size of ham you need. [step] Generally bone-in roasts weigh 10 lbs. ( 4.5 kg) or more.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5390,"error":null,"output":"D"}
{"question_id":"hellaswag:00003388","item_index":184,"attempt":0,"prompt_hash":"c6be5cb4f30e6612954a4489ea87b39858c26e233d55615066e04b077f8f5b75","question":"[header] How to color hair [title] Choose the right formula. [step] Boxed hair dyes have evolved in recent years and the traditional liquid formulas are no longer your only options. Now there are foam, mousse and crème formulas, too.\n\nA. Choose a formula that works for your hair type. Here are some important factors to get the right formula : [substeps] \" aiming for 1.5 \": this formula can be found in the hair supplies section of your local supermarket.\nB. If your goal is to cover grays, a crème formula is the best option. Mousse formulas are best for touch-ups and foam formulas are ideal for those who want all-over color, but don't dye their hair often.\nC. These formulas interfere with hair absorption, resulting in less color or frizz. Because of this, you should pick a formula that is formulated for your hair type.\nD. The formula used with some products may have been slightly formulated at some point in the past, but are still good options. [substeps] The formula used with some kits will have been slightly modified at some point in the future, but it's still good choices.","prompt":"[header] How to color hair [title] Choose the right formula. [step] Boxed hair dyes have evolved in recent years and the traditional liquid formulas are no longer your only options. Now there are foam, mousse and crème formulas, too.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5390,"error":null,"output":"B"}
{"question_id":"hellaswag:00007105","item_index":185,"attempt":0,"prompt_hash":"368eccf32ead6d75f96781568a8ad1af7b789ee244641d2c71f9f2cbccbe1daf","question":"[header] How to make paint colors [title] Gather paint-mixing supplies. [step] You'll need a palette and a paint knife or brush in order to get started. Mixing paint with a paint knife results in a more even, uniform color compared to a brush.\n\nA. To prepare a painting surface, you'll need a bucket or a palette. You'll also need a measuring stick.\nB. [substeps] If you're using brushes to mix paint, make sure to clean them in between mixing. You don't want paint of a previous color mixing with the two you are currently mixing.\nC. [substeps] A palette knife is a lighter brush that comes in a few different colors and has two handles. The knife is considered safe for use with paint, although there is a high possibility of getting paint on your fingers by breaking a glass, cutting a wax paper, wiping the mirror down, or even washing the brush.\nD. If you want a more vibrant, richer tone, you'll need to buy a small amount of paint. Therefore, a palette knife or brush would work well.","prompt":"[header] How to make paint colors [title] Gather paint-mixing supplies. [step] You'll need a palette and a paint knife or brush in order to get started. Mixing paint with a paint knife results in a more even, uniform color compared to a brush.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5389,"error":null,"output":"B"}
{"question_id":"hellaswag:00007851","item_index":186,"attempt":0,"prompt_hash":"69e09b8796645ebf168dc3e7172617ce86434083cc12f418870e7b2c69d8cab4","question":"[header] How to clean druzy stone jewelry [title] Create a mixture of soap and water. [step] Use a mild soap free from fragrances or moisturizers. Dawn dishwashing liquid or dial hand soap are good choices.\n\nA. With soap, you can use just water and natural oils. [title] Take a spoon and rub it over the caked up stone piece.\nB. Pour the soap into a medium sized bowl. Add a few drops of dawn white soap to the water.\nC. Put a few drops of the soap into a bucket or bowl of warm distilled water and mix thoroughly. [substeps] Choose distilled water over tap water to eliminate dulling mineral deposits.\nD. Mix together half baking soda and half water in a small bottle. Carry this mixture through the shower to make an acidic cleaning solution.","prompt":"[header] How to clean druzy stone jewelry [title] Create a mixture of soap and water. [step] Use a mild soap free from fragrances or moisturizers. Dawn dishwashing liquid or dial hand soap are good choices.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5389,"error":null,"output":"C"}
{"question_id":"hellaswag:00004731","item_index":187,"attempt":0,"prompt_hash":"2602158c1731271c67576ac4160143ac3e93b3e496409373e613731ecd5ef6ef","question":"[header] How to minimize pores naturally [title] Consider using a steam bath. [step] Steam is recommended by estheticians for opening up the pores to clean them out. [substeps] Cleaning out your pores can help to minimize their appearance.\n\nA. One benefit of steam bathing is that it is quick and easy. Your pores are closed, and any still moist skin is cleaned out with steam.\nB. Steaming can help strengthen your immune system, which helps to prime and thin out pore skin. Steam baths can help to promote healthy skin, while improving fat and collagen production which may decrease the appearance of blemishes.\nC. Steam is a cheap and all natural way to minimize your pores. You can add herbs and scented oils for a nice scented steam bath.\nD. Steam baths usually dry your skin and open up pores. For the best results, heat the water in your steam bath for at least 30 minutes.","prompt":"[header] How to minimize pores naturally [title] Consider using a steam bath. [step] Steam is recommended by estheticians for opening up the pores to clean them out. [substeps] Cleaning out your pores can help to minimize their appearance.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5603,"error":null,"output":"C"}
{"question_id":"hellaswag:00007239","item_index":188,"attempt":0,"prompt_hash":"790e1efe3a0c0d135c7084e9b97c8e2fbdda8addad2991a17957dce4beb85bc1","question":"[header] How to write a cute note to a friend [title] Find the handwriting. [step] Find a fun type of penmanship (check out dafont.com) and use it! Practice it, writing the whole alphabet in capitals and lowercase. You can also write the sentence, \" the quick brown fox jumps over the lazy dog \", which uses every letter in the english alphabet.\n\nA. [title] Paste, scratch, nail, paint, wet, smear. [step] Add many lines so others don't get in the way.\nB. [title] Think of writing a complicated letter! [step] ) read characters from books in other languages. This step is useful for \" theme \".\nC. [substeps] If you want to write a sentimental or sincere note, use your normal handwriting or a more sophisticated or plain penmanship. Practice in your preferred writing utensil.\nD. [title] Describe the kitchen : [step] Make sure there is no blank space between the saucepan and stove top. Tray top is more desirable, as if you cooked in the big standing pan (the one with two glasses on top).","prompt":"[header] How to write a cute note to a friend [title] Find the handwriting. [step] Find a fun type of penmanship (check out dafont.com) and use it! Practice it, writing the whole alphabet in capitals and lowercase. You can also write the sentence, \" the quick brown fox jumps over the lazy dog \", which uses every letter in the english alphabet.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5603,"error":null,"output":"C"}
{"question_id":"hellaswag:00004009","item_index":189,"attempt":0,"prompt_hash":"f90a9c0249ec255dcc396399c5ee6322cfd70b44a7de51af895e96a4127d837c","question":"[header] How to use exfoliators [title] Use a scrubbing pad or cloth if you don't like exfoliating products. [step] Mechanical exfoliation involves scrubbing your skin with some sort of abrasive pad or cloth or using a product with gritty particles or beads. The simplest way to remove dead skin cells is scrubbing it with a moist or wet cotton face cloth.\n\nA. [title] Shop for a manual exfoliator. [step] Household exfoliators allow you to always mix at least one into your scrub, such as rock salt, yeast, or powder.\nB. Cotton is soft and not especially abrasive, so you're removing dead skin mainly by friction. Wash your skin with warm water and a gentle cleanser prior to exfoliating to open up your pores a little better.\nC. [title] Combine the facial cleanser and lotion. [step] Find a mild, non-acidic cleanser that is acidic but not overly abrasive.\nD. Paper towels are a good product for exfoliating. [substeps] If you have lots of skin cells under your fingernails, or dirt or grime on your hands, you could use a damp washcloth.","prompt":"[header] How to use exfoliators [title] Use a scrubbing pad or cloth if you don't like exfoliating products. [step] Mechanical exfoliation involves scrubbing your skin with some sort of abrasive pad or cloth or using a product with gritty particles or beads. The simplest way to remove dead skin cells is scrubbing it with a moist or wet cotton face cloth.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5602,"error":null,"output":"B"}
{"question_id":"hellaswag:00000977","item_index":190,"attempt":0,"prompt_hash":"c27e807f13ad44c0d7bc8241eac274ca351ade7c3cbd9dba81c7745f766531c2","question":"The person walking to one of the hole and checks the fishing line to see if he caught nay fish. the guy\n\nA. then pulls hard on the fishing line and drags it in.\nB. is holding the fishing line fishing out of there hole pulling romaine with it of course.\nC. walks back to the hole to see if he caught anything but his hooks were nowhere near the hole.\nD. gather up the fish from their hole and grabs the pole and starts fishing.","prompt":"The person walking to one of the hole and checks the fishing line to see if he caught nay fish. the guy","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5600,"error":null,"output":"A"}
{"question_id":"hellaswag:00003454","item_index":191,"attempt":0,"prompt_hash":"3760953fa37b38ca6d194ba3a0aa00f78c5fc043946e4fbf0a07cf7e27cf300c","question":"[header] How to make habanero hot sauce [title] Chop and sauté the carrots for five minutes. [step] Pour 1 tablespoon (15 ml) of olive oil into a large skillet and turn the heat to medium. Peel two large carrots and roughly chop them.\n\nA. Set them aside and sauté the other carrots for five minutes. [substeps] You can use any kind of boiled water you like, but bottled or plain tap water is more likely to work.\nB. [substeps] You don't need to mince the carrots since they're already finely chopped. You'll also need to add them to the pan.\nC. Add the carrots and sauté the carrots for five minutes. Stir them occasionally to prevent them from sticking.\nD. Sauté them in the oil for five minutes over medium-high heat. Next, cook the carrots for another three minutes to prevent them from burning.","prompt":"[header] How to make habanero hot sauce [title] Chop and sauté the carrots for five minutes. [step] Pour 1 tablespoon (15 ml) of olive oil into a large skillet and turn the heat to medium. Peel two large carrots and roughly chop them.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5600,"error":null,"output":"C"}
{"question_id":"hellaswag:00008910","item_index":192,"attempt":0,"prompt_hash":"79eaf4b8a519b0c84d3f52a1030c8f7b98dc8d3ef6d1ed7050af4935dda17058","question":"[header] How to straighten hair with milk [title] Fill a spray bottle with 1/3 cup of milk. [step] Any (fresh) milk will do, but the thicker the milk is, the better. Though it's questionable, some believe that coconut milk has extra straightening power.\n\nA. Avoid milk that has been in the refrigerator for weeks-expired milk might be what you need. [title] Press the hair with a flat paddle.\nB. [substeps] If you do use coconut milk, combine it with lemon juice from half a lemon. Let it sit in the fridge for an hour until a cream forms on top.\nC. [substeps] If you don't have an industrial-strength spray, you can use extra-virgin coconut milk. [title] Fill the bottle with about 1/2 cup (113 ml) of milk.\nD. If you plan to make your hair thick without milk, in a pinch, use a brush. [title] Obtain and heat-proof a spray bottle.","prompt":"[header] How to straighten hair with milk [title] Fill a spray bottle with 1/3 cup of milk. [step] Any (fresh) milk will do, but the thicker the milk is, the better. Though it's questionable, some believe that coconut milk has extra straightening power.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7172,"error":null,"output":"B"}
{"question_id":"hellaswag:00004532","item_index":193,"attempt":0,"prompt_hash":"53f8dd9652b97d1f8197afafff4e53cc3d647c5fc18f0da743b3ecf0d3042e6c","question":"[header] How to listen to an ipod or mp3 player at school [title] Wear a loose-fitting jacket. [title] Stick the earphones inside the jacket and down the sleeve, but keep them just inside the cuff. [title] Put the ipod in your pants pocket.\n\nA. [title] Pull out the ipod and put the volume up play on. [title] Play left and right with the ipod.\nB. [title] Leave the bag on your lap in case your ipod is lost or stolen. [title] Bring your digital phone, ipod touch or your mp3 console separately.\nC. [title] Turn the volume down low, so the teacher won't hear it. [title] Put you hand/wrist up to your ear and listen.\nD. [step] You can also slide your ipod into a sweatshirt at school if it has to be in your jacket pocket. [title] Leave the class early on because the teacher will most likely give you a warning and you will probably be late.","prompt":"[header] How to listen to an ipod or mp3 player at school [title] Wear a loose-fitting jacket. [title] Stick the earphones inside the jacket and down the sleeve, but keep them just inside the cuff. [title] Put the ipod in your pants pocket.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7172,"error":null,"output":"C"}
{"question_id":"hellaswag:00005391","item_index":194,"attempt":0,"prompt_hash":"c572e5d43748ee4f7385f1c242e3eaaa60d2a26c621cee84b62182ec38714260","question":"[header] How to help relieve gas in babies [title] Recognize the symptoms of gas. [step] Although there is no evidence for a connection between colic, or excessive crying, and gas, some people may link the two conditions. Identifying the signs of gas can help you quickly give your baby comfort.\n\nA. The symptoms of gas are : [substeps] Constant crying severe abdominal pain, which may appear clear and white and thicken after crying. Gray or dark red blood stains or swollen abdomen.\nB. Specific symptoms of gas include : [substeps] A tendency to urinate swollen kidneys difficulty breathing changes in the fetus. This may include vomiting, diarrhea, constipation, bloating, or abdominal pain headache.\nC. Symptoms of gas include : [substeps] Pulling up her legs clenching her fists squirming around as if she's uncomfortable crying a lot farting burping [title] Do the bicycle with baby. [step] If your baby seems gassy, lay her on her back on a stable surface.\nD. Gas symptoms include : [substeps] Weight difference: you might notice your baby may gain weight, but often the baby's entire body will not have gas at all. Loss of firmness while breastfeeding: you might notice your baby may cradle and sway back and forth in more ways than normal during breastfeeding.","prompt":"[header] How to help relieve gas in babies [title] Recognize the symptoms of gas. [step] Although there is no evidence for a connection between colic, or excessive crying, and gas, some people may link the two conditions. Identifying the signs of gas can help you quickly give your baby comfort.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7173,"error":null,"output":"C"}
{"question_id":"hellaswag:00009293","item_index":195,"attempt":0,"prompt_hash":"c745408d4d47d90a8f795294b37b0809646929d425fc8418af98e7ad6989a588","question":"[header] How to figure out why you aren't losing weight [title] Do a 3 day food recall. [step] A food recall is very similar to a food diary. However, it's generally only kept for a certain amount of days and is very detailed.\n\nA. A food recall is the one of the first places to start when you're not sure why you're not losing weight. [substeps] Doing a 3 day food recall will help you get a really good and solid idea of what you've been eating.\nB. It measures a food as visually as a diary, and then moves on to another food. It will mark the day and time of day when the food is likely to be the one you considered' right', but it does not mean it's' good' or' bad'.\nC. You decide what foods you need to be monitoring and use the recall to take those foods off of your body's standardized list. For example, on this list you might write down \" ingredients, utensils, tools, freezer, refrigerator etc.\nD. You can create a food recall by observing everything you ate as a person. You can ask your friends, family, partners, or coworkers what you ate.","prompt":"[header] How to figure out why you aren't losing weight [title] Do a 3 day food recall. [step] A food recall is very similar to a food diary. However, it's generally only kept for a certain amount of days and is very detailed.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7173,"error":null,"output":"A"}
{"question_id":"hellaswag:00005521","item_index":196,"attempt":0,"prompt_hash":"e0146dbb5f814d49ad591306454cb293a5a02d5fdf6c248e78140eea3c39d278","question":"[header] How to be patient with an impatient toddler [title] Prepare in advance. [step] Trying to come up with ways to entertain a toddler can be difficult when they're impatient and you aren't prepared for the situation. It'll be much easier to remain calm when dealing with an impatient toddler if you've planned ahead.\n\nA. [substeps] Reorganize your schedule by keeping more important things in mind while the toddler is under your care. Change up your schedule so that you have non-nagging things on your mind every minute or two.\nB. This way you'll already have strategies in mind to use with the toddler and you can address the situation with a positive attitude. [substeps] Try keeping a list of activities to use.\nC. Some ideas are : [substeps] Finger painting or coloring. This can usually be done with clothes on.\nD. Prepare a seat for both them and you at the table so they can be reminded about food if they decide to have a meltdown. [substeps] Make sure your toddler knows you're available if they're hungry.","prompt":"[header] How to be patient with an impatient toddler [title] Prepare in advance. [step] Trying to come up with ways to entertain a toddler can be difficult when they're impatient and you aren't prepared for the situation. It'll be much easier to remain calm when dealing with an impatient toddler if you've planned ahead.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7172,"error":null,"output":"B"}
{"question_id":"hellaswag:00001203","item_index":197,"attempt":0,"prompt_hash":"92fc7d204439d799c1664bf82273da367028e8399c59de2111e2617ab88866e8","question":"Marching band wering red uniforms are in street playing drums and doing a choreography. behind the band kids walking in circles\n\nA. on an lewle line.\nB. and end the band and leaving.\nC. are doing a choreography.\nD. in the street was a circle with a lot of marching band around it.","prompt":"Marching band wering red uniforms are in street playing drums and doing a choreography. behind the band kids walking in circles","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7177,"error":null,"output":"C"}
{"question_id":"hellaswag:00002443","item_index":198,"attempt":0,"prompt_hash":"92c24fcfd226fbff68280f8c9a26ed176581a8f83f8859a0d70463a084a32824","question":"A chef is seen speaking to the camera and begins cutting up ingredients. he\n\nA. cuts up several pieces of food using different knives.\nB. cooks out of a large pan and slowly shapes up a meal.\nC. pours pasta into a pot and begins dipping ingredients into the pot mmons boiling water.\nD. receives the ingredients in a bowl followed by a bowl filled with salad.","prompt":"A chef is seen speaking to the camera and begins cutting up ingredients. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7175,"error":null,"output":"A"}
{"question_id":"hellaswag:00002139","item_index":199,"attempt":0,"prompt_hash":"834a6e2acea596e21c827aa0ae6d588a110fa45aef8e1a6fa743426b750af4fc","question":"A woman stands on front a young lady and talks. then\n\nA. , the woman spread the hair in front of the camera, the woman use makeup brush to create some lines and a face.\nB. , the woman unrolling carpet on her floor.\nC. , the woman takes plaster from a wall and measures it.\nD. , the woman and the young lady perform fitness exercise while the woman talks.","prompt":"A woman stands on front a young lady and talks. then","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7174,"error":null,"output":"D"}
{"question_id":"hellaswag:00004931","item_index":200,"attempt":0,"prompt_hash":"0eacf4d7a4bfc78b4ba3248389a7da3609252701e2c9b69b8e1e35007257a8ed","question":"[header] How to get a detention [title] Talk when you're not supposed to. [step] Want to get in trouble real quick? Just talk when your teacher is talking, or when you're supposed to be quiet. Especially if you channel your inner heckler.\n\nA. Whenever you're overdue for an assignment, speak up. [substeps] Never, ever say anything just because.\nB. Here are a bunch of ways to talk when you're not supposed to : [substeps] Wait until you've just been given a test, then start raising your hand and asking questions about the homework. Interrupt other students when they're reading out loud, or if they're presenting their science/biome projects.\nC. [title] Change it up a bit if you come across as mean. [step] If you know you are a bully, switch it up for a more serious, constructive approach.\nD. [substeps] Use in the middle of a lesson, such as when you're doing homework and you cannot get it resolved in class. [title] Understand that detention is different from always being quiet.","prompt":"[header] How to get a detention [title] Talk when you're not supposed to. [step] Want to get in trouble real quick? Just talk when your teacher is talking, or when you're supposed to be quiet. Especially if you channel your inner heckler.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7170,"error":null,"output":"B"}
{"question_id":"hellaswag:00000038","item_index":201,"attempt":0,"prompt_hash":"f9f8aa68dcb1c8bbe82e5d27a316a183232e668a792009faf37c63f80254e68d","question":"We see a person holding face wash then putting it on their face. They rinse the face and add the face wash with a brush. we\n\nA. see a closing title screen.\nB. see a black screen with the credits.\nC. see an illustration on how to add the wash using a brush.\nD. then see a replay then the person putting the face wash on.","prompt":"We see a person holding face wash then putting it on their face. They rinse the face and add the face wash with a brush. we","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7173,"error":null,"output":"B"}
{"question_id":"hellaswag:00000817","item_index":202,"attempt":0,"prompt_hash":"47ccc2c8d7d1428043b5e5f112190796fc143c5d7fe4d7df114608d50ef62725","question":"A man rides a bicycle behind three men who are running in a professional race on a paved freeway lined with onlookers, spectators and participants. a man\n\nA. is riding a bicycle down a freeway wearing a helmet and sporting a paper tag on the front of the bicycle.\nB. wearing a helmet and hooded sweatshirt runs beside the other dirty or dirty bikers, followed by his younger body of sea-wort that hangs from trees from his chest.\nC. and a man dressed in white run together in the racetrack.\nD. wearing a black jacket lines up at the center of a sea filled crowd in the starting line waiting for the starting line.","prompt":"A man rides a bicycle behind three men who are running in a professional race on a paved freeway lined with onlookers, spectators and participants. a man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7174,"error":null,"output":"A"}
{"question_id":"hellaswag:00006637","item_index":203,"attempt":0,"prompt_hash":"563a71f6dceb9a3afa266a250ff96fd68e0f2ff79213eae4f448b049f4595134","question":"[header] How to slur words [title] Keep your lips as close together as you can when you speak. [step] The less open your mouth is, the less you'll be able to enunciate your words and the more you'll slur. Speak while keeping your lips only partially open.\n\nA. [title] Enunciate words carefully before you pronounce them. [step] You can emphasize word structure more easily when you pronounce each word in the sentence.\nB. This should add to the slurring of your speech. [title] Push your tongue up to the roof of your mouth.\nC. [title] Enunciate your words clearly. [step] When you're saying slurred words, enunciate each word.\nD. Keep your vocal cords taut and pretty much straight. [title] Keep your enunciation and timing consistent.","prompt":"[header] How to slur words [title] Keep your lips as close together as you can when you speak. [step] The less open your mouth is, the less you'll be able to enunciate your words and the more you'll slur. Speak while keeping your lips only partially open.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7175,"error":null,"output":"B"}
{"question_id":"hellaswag:00005380","item_index":204,"attempt":0,"prompt_hash":"3b6027fce269adefceb2c9627d2f3065a736ed29bd572dab32b7c57c8fba158e","question":"[header] How to use and install a live cd of linux [title] Make sure your computer is booting from the cd drive. [substeps] Either turn on or restart your computer. While doing this, hold the delete button to enter the bios.\n\nA. [title] Once you've backed up your computer, press a. [step] After a few seconds the bios should pop up on your screen.\nB. If you don't hear any result after a few seconds of working (or if the cd drive isn't booting freely from the computer), you may need to reboot your computer. [title] Run the cd containing the bios.\nC. Use your left and right arrow keys to navigate to the boot tab. * once on the boot tab use your down arrow keys to navigate to the \" boot device priority \" menu.\nD. The program is now on the cd drive and it cannot be added to the system at any time. To quit the program via the bios), enter the status key and select restart the computer.","prompt":"[header] How to use and install a live cd of linux [title] Make sure your computer is booting from the cd drive. [substeps] Either turn on or restart your computer. While doing this, hold the delete button to enter the bios.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7170,"error":null,"output":"C"}
{"question_id":"hellaswag:00006008","item_index":205,"attempt":0,"prompt_hash":"83255d7c522e0e1f6e343a2540aac5e6f9a68e08caf95e438897f4430e90595e","question":"[header] How to measure chest size [title] Locate a fabric tape measure. [step] This type of tape measure is easy to wrap around your body, and accurately measuring curves. If you don't have one, you can instead wrap a string around your body, then measure that with a ruler.\n\nA. [title] Find a friend to help you measure. [step] It can be a little tricky to take an accurate chest measurement on your own, so if possible, see if you can find a friend to help you.\nB. Be sure to stand about 6 feet (1.8 m) behind any furniture you intend to walk on. [substeps] You can try double-sided tape for a stronger measurement.\nC. Hang the tape tape down the front of your chest, rubbing its edge along your ribs. [substeps] Mark down the shirt width using an invisible pencil.\nD. [substeps] Some fabrics can be tested with built-in test tape, but other fabrics and materials can be found online or in craft stores. You might use test tape for smaller forms of chest measuring, such as sweaters, tutus, and shirts.","prompt":"[header] How to measure chest size [title] Locate a fabric tape measure. [step] This type of tape measure is easy to wrap around your body, and accurately measuring curves. If you don't have one, you can instead wrap a string around your body, then measure that with a ruler.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7169,"error":null,"output":"A"}
{"question_id":"hellaswag:00010030","item_index":206,"attempt":0,"prompt_hash":"03d9488e8358ad51db189261e6d0d73027e4ff81166d8eded24168e3c843d366","question":"[header] How to prepare children for a new sibling [title] Get your child/children to sit with you in a quiet place where you are unlikely to be disturbed. [title] Ask them what they would think of a new little brother or sister. [step] Most children would be excited by the prospect, but some may be a little confused.\n\nA. [title] Provide a book to them and read it to them. [step] Talk to them and help them to put the pages of every book in their mind.\nB. [title] Have a conversation about happy or sad things. [step] Remind them that this should be an ordinary little day, and include all the new things we have planned and keep an eye on them through meals, playdates, overnight events.\nC. [title] Explain that you/the mother is going to have a baby. [step] This may be awkward if it is with a stepparent, or a boyfriend.\nD. Keep talking to them until they are comfortable with the idea of the new little brother or sister. [substeps] Ask how the twin feels about the new little brother or sister, and describe the scent that smells like the new small brother or sister (it should be sweet).","prompt":"[header] How to prepare children for a new sibling [title] Get your child/children to sit with you in a quiet place where you are unlikely to be disturbed. [title] Ask them what they would think of a new little brother or sister. [step] Most children would be excited by the prospect, but some may be a little confused.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7172,"error":null,"output":"D"}
{"question_id":"hellaswag:00007176","item_index":207,"attempt":0,"prompt_hash":"311da327ef0eabf2b32536da0bc6e8289f0cb672be5b920fc9257f21319442e5","question":"[header] How to choose the best products for african american hair [title] Know your hair type. [step] To begin with, it helps to know what your hair type is likely to be and therefore, what the best products for it will be. To test your hair type, try a daily density test to determine how well your hair absorbs moisture : [substeps] Wash your hair.\n\nA. Follow up with a cool/humid shower. For those with wavy hair, use a dry shampoo and conditioner; for those with straightened hair, use a blow dryer.\nB. Place your fingertips in your hair to look for dampness. If water is not readily available, make a dry shampoo and conditioner known as \" micro water \" by licking the shower gel out of your hair.\nC. Wash it with shampoo every day (optional), or every 2 days if you have very dry hair. Wrap it up in a towel and then shampoo until you no longer get an oily look.\nD. Allow your hair to air dry, to observe hair's natural moisture level. Use water-based conditioners and oils that penetrate the hair shaft.","prompt":"[header] How to choose the best products for african american hair [title] Know your hair type. [step] To begin with, it helps to know what your hair type is likely to be and therefore, what the best products for it will be. To test your hair type, try a daily density test to determine how well your hair absorbs moisture : [substeps] Wash your hair.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7169,"error":null,"output":"D"}
{"question_id":"hellaswag:00000907","item_index":208,"attempt":0,"prompt_hash":"6dd1608e380e98b427e7f4ac5d3ee5891b2ad0a271b87cbafd5411119c3c0fba","question":"There are two young girls holding violins in their hands, standing with a music stand and a book with music notes behind them. the older girl\n\nA. begins singing while playing the violins as the girl behind her starts to dance.\nB. then attempts to play the violin, the older girl holds the violin too while the younger continue playing.\nC. is helping and teaching her younger sister to properly hold the violin by changing her head position.\nD. on the left reaction to a song by lisa anthony and the older girl on the left react to the song by annie jones.","prompt":"There are two young girls holding violins in their hands, standing with a music stand and a book with music notes behind them. the older girl","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7173,"error":null,"output":"C"}
{"question_id":"hellaswag:00009343","item_index":209,"attempt":0,"prompt_hash":"7e32503f6d165b84934732d71808f678e2e4ef45d2ef31797b8fbb9fcd0adaee","question":"[header] How to anoint oil [title] Check with your judicatory or a religious authority for specifics. [step] Each denomination has its own guidelines governing the way that oil is blessed for anointing purposes, as well as the way that anointing oil is used. [substeps] The most common restriction regards who may bless or anoint the oil.\n\nA. Believers must purchase and learn the following mandates before they may apply. For example, in a specific denomination or field, someone can anoint the oil for religious purposes or for ritual purposes.\nB. It is a guideline that will vary fairly widely in species. To find any non-non-buddhist group, you will likely need to look up nifty non-buddhist societies.\nC. In some denominations, only a priest or similar clergyman can bless the oil. Some denominations even limit which clergymen are authorized to consecrate oil.\nD. Anabolic physiology physiology, for example, prohibits shopping for oils at that exact minute. The consensus is that fertilizers and liquids must be consumed in order to make anabolic physiology.","prompt":"[header] How to anoint oil [title] Check with your judicatory or a religious authority for specifics. [step] Each denomination has its own guidelines governing the way that oil is blessed for anointing purposes, as well as the way that anointing oil is used. [substeps] The most common restriction regards who may bless or anoint the oil.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7172,"error":null,"output":"C"}
{"question_id":"hellaswag:00004395","item_index":210,"attempt":0,"prompt_hash":"357dd672e324b6f4779ced0a8932d34253c653022bf766886bede7d9ec6f71cb","question":"[header] How to obtain the percent complete numerical value from an audible audiobook from the audible for ios app [title] Turn on the device and get to your home screen. [title] Make sure that you've installed and/or updated audible's official ios app. [step] Check the updates tab on the appstore integrated app to ensure you are completely updated.\n\nA. [title] Browse for and then install audible. [step] Below the initial listings, you should see a prompt to download audible.\nB. [title] Head to the app drawer and locate the \" app drawer \" button. [title] Enter the app name and press enter.\nC. [title] Go to https: // www.audio.google.com in a web browser. [title] Click on the menu at the top of the screen and select \" appaller.\nD. This will ensure the feature has been installed. [title] Open and log in to the audible app from your ios device.","prompt":"[header] How to obtain the percent complete numerical value from an audible audiobook from the audible for ios app [title] Turn on the device and get to your home screen. [title] Make sure that you've installed and/or updated audible's official ios app. [step] Check the updates tab on the appstore integrated app to ensure you are completely updated.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7173,"error":null,"output":"D"}
{"question_id":"hellaswag:00008419","item_index":211,"attempt":0,"prompt_hash":"e4a46b3235049eef5cd99e6bb0b754f0ef0b70c7237d532810f8719938c0ba33","question":"[header] How to recognize someone with paranoid personality disorder [title] Look out for four or more of the following character traits in people who are diagnosed with paranoid personality disorder. [title] Pay attention to the suspicious nature of the person concerned. [step] Do they suspect, without sufficient basis, that others are exploiting, harming or deceiving them in some way or another.\n\nA. [title] Notice if there is a preoccupation with unjustified doubts about the loyalty of friends and other associates. [step] These types will see hidden agendas to every action performed by others, even though this may not be the case.\nB. [title] Pay attention to people who accuse them of these things. [step] They may never let you know what is going on.\nC. It may even be caused by the feeling of being undervalued, contacted, manipulated, or ignored in a company or role they may invest in. The more suspicious and paranoid the person becomes, the stronger the problem is.\nD. [title] Look for techniques that might help you identify a paranoid person. [step] There is a vast range of techniques for identifying someone with paranoid personality disorder.","prompt":"[header] How to recognize someone with paranoid personality disorder [title] Look out for four or more of the following character traits in people who are diagnosed with paranoid personality disorder. [title] Pay attention to the suspicious nature of the person concerned. [step] Do they suspect, without sufficient basis, that others are exploiting, harming or deceiving them in some way or another.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7173,"error":null,"output":"D"}
{"question_id":"hellaswag:00008995","item_index":212,"attempt":0,"prompt_hash":"ecd446ea800e8523fca9d776598f360386b54416c6655ed97d313fe4605db094","question":"[header] How to remove odor from your shoes with baking soda [title] Measure out at least 1 tablespoon of baking soda into each shoe. [step] You need to use enough baking soda to cover the entire insole of the shoe. If you have big shoes, you may need to use more than 1 tablespoon of baking soda.\n\nA. [title] Shake the shoe to distribute the baking soda across the insole. [step] Tilt the shoe forward and back to help spread the baking soda across the insole.\nB. [title] Pour the baking soda into a spray bottle and shake it up thoroughly. [step] Pour the baking soda over the insole of the shoe.\nC. [title] Soak a cotton ball in baking soda. [step] Place the fresh teaspoon of baking soda in your shoe.\nD. [substeps] When packing your shoe in the bag, use a funnel to ensure that the baking soda contains all of the baking soda. [title] Pre-soak the shoes for 30 minutes.","prompt":"[header] How to remove odor from your shoes with baking soda [title] Measure out at least 1 tablespoon of baking soda into each shoe. [step] You need to use enough baking soda to cover the entire insole of the shoe. If you have big shoes, you may need to use more than 1 tablespoon of baking soda.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7175,"error":null,"output":"A"}
{"question_id":"hellaswag:00001840","item_index":213,"attempt":0,"prompt_hash":"7cea680f3701a5424069e2d43b876a919b8869af21264c3e31b4e71184f67ce1","question":"The man not fishing inserts his hand into the water several times. The man not fishing helps to pull a fish out of the ice fishing hole. the two men\n\nA. first check their gauges for air.\nB. are in a kitchen talking about their fishing experience.\nC. shake hands while a third walks behind them.\nD. pull the fish out of the hole afterward.","prompt":"The man not fishing inserts his hand into the water several times. The man not fishing helps to pull a fish out of the ice fishing hole. the two men","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7172,"error":null,"output":"B"}
{"question_id":"hellaswag:00003123","item_index":214,"attempt":0,"prompt_hash":"9f8c8b79925521e3fa382dd1485e1201a7352680eae46cc8b082f9a160d9cb9f","question":"A man is lifting weights in a weight room, pulling a large barbell up to his chest. He stands shaking before lifting it above his head, trying to hold it in place. he\n\nA. stands and lets it fall slowly to the ground.\nB. drops the barbell hard onto the ground.\nC. drops the barbell and lowers it down messily.\nD. then lifts it over his head to let it land on the floor.","prompt":"A man is lifting weights in a weight room, pulling a large barbell up to his chest. He stands shaking before lifting it above his head, trying to hold it in place. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6831,"error":null,"output":"B"}
{"question_id":"hellaswag:00006854","item_index":215,"attempt":0,"prompt_hash":"8f274b8c24004e98a38fea73a985adb8a4e68bb5d2b627edf82a0b3b62e4251d","question":"[header] How to wear skirts [title] Choose a pencil skirt if you want a classic, streamlined silhouette. [step] The pencil skirt starts at the waist and ends just above the knee. It is fitted, tapering down to the knees, and has clean, tailored lines.\n\nA. Pencil skirts end up looking slimmer in the fall, when the weather will be cooler. They have to fall well below the knee for a classic look.\nB. You may also choose a pencil skirt if you already have shorter, more formfitting skirts. Pencil skirts also have beadwear and streamlined fronts.\nC. They are perfect for formal occasions, including office settings. Here are some chic outfit ideas : [substeps] Pair a black pencil skirt with a black, fitted shirt.\nD. Pencil skirts pair nicely with nearly any skirt, from skirts with a crisp knee or plunging waist or short, form-fitting skirts. [substeps] Pencil skirts also come in colors that are flattering to most body types.","prompt":"[header] How to wear skirts [title] Choose a pencil skirt if you want a classic, streamlined silhouette. [step] The pencil skirt starts at the waist and ends just above the knee. It is fitted, tapering down to the knees, and has clean, tailored lines.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6830,"error":null,"output":"C"}
{"question_id":"hellaswag:00007892","item_index":216,"attempt":0,"prompt_hash":"c352027edc69994c82baae7dcc687368bee75eb6503b1bd6ce3f037f917bb3e7","question":"[header] How to tell an interesting travel story [title] Keep an updated log of your travel adventures. [step] Make it a habit to keep a notepad so that you can jot down different experiences you have while you travel. This will make it easier to remember experiences that you can later use in your stories.\n\nA. [substeps] Notice which places you are most likely to see in your travels. Read stories that you have never heard of.\nB. [substeps] Keep it on your phone or tablet, or on your computer if your cellphone is sim-safe. Write down at least one trip or possible trip every week, whenever you're thinking about them.\nC. Sign up for a travel journal with descriptions of places you have visited. This will help you remember detailed details without giving away too much information.\nD. Jot down the most important points of the story rather than trying to chronicle all the details during your travels. [substeps] You can also take photos or video of your experiences to help you remember them, or you can even integrate them into the telling of your story.","prompt":"[header] How to tell an interesting travel story [title] Keep an updated log of your travel adventures. [step] Make it a habit to keep a notepad so that you can jot down different experiences you have while you travel. This will make it easier to remember experiences that you can later use in your stories.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6831,"error":null,"output":"D"}
{"question_id":"hellaswag:00006298","item_index":217,"attempt":0,"prompt_hash":"cf84c46bef0524c9588e74d532037202eaac175766abb39e365658598dfc2b0b","question":"[header] How to adopt a child from ukraine [title] Choose an adoption services provider. [step] An adoption agency can help insure that you have all the documents and information you need to successfully adopt a child. Many ukrainian adoption agencies also provide guidance and translation services once you are in ukraine.\n\nA. [substeps] In addition to medical and financial services, your adoption services provider may also offer practical and travel assistance in addition to child support. You can research adoption services online or at a local ukraine adoption agency.\nB. [substeps] These agencies help parents get familiar with each other and provide information about their needs in life and their policies when adopting a child. [substeps] Adoption agencies also assist parents each with finding a child first.\nC. [substeps] You can find adoption services through the national adoption services center's portal at http: // www.ncfa. Nc/downloads/city/ukcа04.aspx.\nD. [substeps] If you're american and you want to adopt a child from ukraine, u.s. law requires you to use a licensed and registered adoption services provider.","prompt":"[header] How to adopt a child from ukraine [title] Choose an adoption services provider. [step] An adoption agency can help insure that you have all the documents and information you need to successfully adopt a child. Many ukrainian adoption agencies also provide guidance and translation services once you are in ukraine.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6831,"error":null,"output":"D"}
{"question_id":"hellaswag:00001140","item_index":218,"attempt":0,"prompt_hash":"c7a1ea99b7ae44e99593794690601afcf1fc177c42cd37f1f50a00f20f847aad","question":"During the game, several attempts are made at the goal but the goalie isn't able to stop all of them and japan keeps losing. the game\n\nA. ends with an easy goal as the men congratulate each other on a five to zero game.\nB. continues and shows a replay of the previous game.\nC. becomes more aggressive and then the goalie is called a winner and it's time to end the match.\nD. continues on and the goalie continuously fails to make his shot.","prompt":"During the game, several attempts are made at the goal but the goalie isn't able to stop all of them and japan keeps losing. the game","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6830,"error":null,"output":"B"}
{"question_id":"hellaswag:00005866","item_index":219,"attempt":0,"prompt_hash":"23d2d12b178016e98b641496e38f26b1f6baac7c241728e9b148126c00525d9b","question":"[header] How to have glowing skin and perfect lips [title] Cleanse your skin every morning and night using a gentle cleanser. [step] Make sure your cleanser is suited to your skin type. [title] Using a soft towel, pat your face dry.\n\nA. [step] Rub a thicker moisturizer over your lips. Apply an hydrating lip scrub and apply it to your lips.\nB. [step] Do not rub as it could lead to irritation. Even better, leave your skin to dry naturally.\nC. [step] Apply lip moisturizer to your lips and apply a little heat to your lips. Then apply some lip balm and lip liner.\nD. [step] Moisturize with a soft-cream face cream. [title] Apply an orange eyeshadow plumping powder.","prompt":"[header] How to have glowing skin and perfect lips [title] Cleanse your skin every morning and night using a gentle cleanser. [step] Make sure your cleanser is suited to your skin type. [title] Using a soft towel, pat your face dry.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6692,"error":null,"output":"B"}
{"question_id":"hellaswag:00003855","item_index":220,"attempt":0,"prompt_hash":"fd383e0ec186406e334df4217af06ef3c436ee615389d36eda1c8d9ca06c8e80","question":"[header] How to conserve ammo in resident evil [title] Master the knife. [substeps] If you are less than proficient with the knife, there is still a way for you to use it, but will still require some ammo. Shoot zombies a few times until they fall, but not die.\n\nA. Go over and slash their fallen bodies as quickly as possible. If they get up before you are done, finish them off with a gun.\nB. If you are at a dead end (conserving low damage) try reinstalling your secondary and switch tactics. [title] Access the stamina hole (the one you know where all the zombies surround you except you).\nC. [title] Shoot someone until they fall, especially if you know they will try to retaliate against you, or die. [step] You need to kill them first, so go around town and see if anyone is.\nD. Congratulate yourself when you win, and accept the reward. The only way you can live with empty ammo is using both knives, but its possible to recharge them without changing the swap method.","prompt":"[header] How to conserve ammo in resident evil [title] Master the knife. [substeps] If you are less than proficient with the knife, there is still a way for you to use it, but will still require some ammo. Shoot zombies a few times until they fall, but not die.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6690,"error":null,"output":"A"}
{"question_id":"hellaswag:00001117","item_index":221,"attempt":0,"prompt_hash":"d100a41495ee432aa2f82377c27fbf2233f52edb5333a28b6492e648df56cf99","question":"He turns the tool and uses the cutting edge to slice the carpet at the wall seam and tears off the cut strip. he\n\nA. removes the bleaches and pulls out the window.\nB. turns the tool around and uses the roller end to tuck the carpet in at the wall seam.\nC. moves down the wallpaper on the wall to show more work cuts.\nD. then cuts the old piece of wallpaper with the scissors.","prompt":"He turns the tool and uses the cutting edge to slice the carpet at the wall seam and tears off the cut strip. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6690,"error":null,"output":"B"}
{"question_id":"hellaswag:00005338","item_index":222,"attempt":0,"prompt_hash":"9646c4c8644fb83d56465847a12139a03cd554ccb2bb14019807a2303e631654","question":"[header] How to wax ear hairs [title] Buy an at-home waxing kit. [step] Purchase a wax that is designed to be used on sensitive areas. You can use heated or non-heated wax.\n\nA. Leave the kit in the same spot and wait three to six hours before using the product. [substeps] You can wax just one area of the ear.\nB. Purchase an at-home waxing kit that is made to be used on sensitive skin. You can request wax directly from your dermatologist or pharmacist.\nC. Non-heated wax removal products often come with the wax already on a strip, which can provide a more precise application of wax on ear hairs. This is the safest way to wax your ear hair.\nD. A wax at home wax kit should contain waxing supplies, and work beside a heated wax tray in a cool, dark area. Your wax kit will also be used on affected areas, such as the backs of your nose and temple.","prompt":"[header] How to wax ear hairs [title] Buy an at-home waxing kit. [step] Purchase a wax that is designed to be used on sensitive areas. You can use heated or non-heated wax.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6691,"error":null,"output":"B"}
{"question_id":"hellaswag:00003990","item_index":223,"attempt":0,"prompt_hash":"3cfafc44d480a9de2222bc152364943391758718e74eab8ca93d35bf92e58791","question":"[header] How to get your husband to listen to you [title] Choose a location where you can talk without distractions. [step] While it might be tempting to call him on his cell phone in the very moment you become upset, calling at work or while he is busy is not going to end well. [substeps] The best location is somewhere quiet, without cell phones or the tv on.\n\nA. Lean against a wall if necessary, or lean against a wall if you can. Listen to music or a movie in another room to help you relax.\nB. You will also be able to have a conversation without distractions. Place your husband's phone near his bed or office if you would like privacy.\nC. A flat, car can give him some privacy. Your husband may also be busy at work, so it's best to pick an indirect location.\nD. If you have children wait until they are asleep or out of the house to avoid any little ears. [title] Ask whether it is a good time.","prompt":"[header] How to get your husband to listen to you [title] Choose a location where you can talk without distractions. [step] While it might be tempting to call him on his cell phone in the very moment you become upset, calling at work or while he is busy is not going to end well. [substeps] The best location is somewhere quiet, without cell phones or the tv on.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6691,"error":null,"output":"D"}
{"question_id":"hellaswag:00005455","item_index":224,"attempt":0,"prompt_hash":"ee658f859c7cfc03017c5150d8ae65dba05e59ca6a4acedaab09e85fbd80abf4","question":"[header] How to can tomatoes [title] Choose your tomatoes. [step] Any variety of tomatoes can be used, but make sure the fruit is not too ripe. Overripe tomatoes are bad for canning due to their high acid content.\n\nA. [substeps] If you can find fresh tomatoes, cut them into quarters or slices. Make sure your cutting board is clear and that you can picture the fruits.\nB. Gently press on the tomatoes to make sure they are still firm. Look over your tomatoes to make sure they do not have any bruises.\nC. [title] Use chinese cabbage, lettuce or kale. [step] Ask your butcher to repot or feed your vegetables any cabbage you choose.\nD. That said, you can always have loose tomatoes cook over the stove, or simmer over the stove without them if that is the case. [substeps] Dried fruits are also good tomatoes to use for canning.","prompt":"[header] How to can tomatoes [title] Choose your tomatoes. [step] Any variety of tomatoes can be used, but make sure the fruit is not too ripe. Overripe tomatoes are bad for canning due to their high acid content.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5401,"error":null,"output":"B"}
{"question_id":"hellaswag:00001241","item_index":225,"attempt":0,"prompt_hash":"ad96947b63fbf83ee2ad267339c7d6b16e4eebb8b5e9be979e41b8416f6deae9","question":"As he does this he explains his methods. Using the same rag he moves to the other pane to clean the frame. finally\n\nA. , he turns around and begins rubbing the umbrella on the window.\nB. , the water sports appears and he sprays it with water before shooting his glass and setting it in the trashcan.\nC. he cuts the tape on he window and gets a custom paste and paints the window with it.\nD. , he ends by scrubbing the bottom right corner of the door.","prompt":"As he does this he explains his methods. Using the same rag he moves to the other pane to clean the frame. finally","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5399,"error":null,"output":"D"}
{"question_id":"hellaswag:00000488","item_index":226,"attempt":0,"prompt_hash":"b9f52618c5b7caa60a9526eb8ede392667b534218493fd6e7c702cd13e3213d5","question":"A man is seen speaking to another holding a violin. The man speaking them walks out of frame. the other man\n\nA. holds the bow back up to speak to the camera.\nB. begins playing the piano followed by the person speaking more while they play.\nC. walks back in and the picture speeds up.\nD. begins playing the violin.","prompt":"A man is seen speaking to another holding a violin. The man speaking them walks out of frame. the other man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5399,"error":null,"output":"D"}
{"question_id":"hellaswag:00004651","item_index":227,"attempt":0,"prompt_hash":"af4ff28e81312a6ebcc66ed8b3e0885f68235d0bfa97e5981307bab1c67808e0","question":"[header] How to find a life partner [title] Put yourself out there. [step] The more time you spend dating and socializing, the more likely you will find someone you like. Put yourself out there by attending social events organized by your friends, by taking classes and chatting with classmates, and by signing up for dating sites, apps, and services.\n\nA. [substeps] Constantly keep the number of dates you've already had in mind handy, using a number of these to boost your social sense. Though you'll find somebody attractive, you'll definitely land someone who doesn't have the same hang of social interactions as you.\nB. Be adventurous and open minded: try speed dating, for instance. [substeps] The most common way to meet a future partner is through mutual friends.\nC. The more you find a partner, the more time you will have to devote to them. [substeps] This can be any musician, movie star, or comedian.\nD. [substeps] For example, invite your friends to sleepovers, go to a movie theater with a friend, or take up reading in your own class. Take a class in your chosen subjects so you feel like you have a real connection with the person.","prompt":"[header] How to find a life partner [title] Put yourself out there. [step] The more time you spend dating and socializing, the more likely you will find someone you like. Put yourself out there by attending social events organized by your friends, by taking classes and chatting with classmates, and by signing up for dating sites, apps, and services.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5393,"error":null,"output":"B"}
{"question_id":"hellaswag:00004054","item_index":228,"attempt":0,"prompt_hash":"4d8e135a4ec6b0aa79312a44daabae3b9147fd1418b5ac891b9620f6ac92f445","question":"[header] How to pretend to be a wizard [title] Decide what kind of wizard you want to be. [step] Research your choice. Perform an image search for the character or style that appeals to you the most.\n\nA. You can also peruse books with wizards in them for costume ideas. [substeps] For example, maybe you want to be like harry potter, or maybe like merlin, gandalf, or perhaps your very own kind of wizard.\nB. It might help to take some time to research the other wizards in your town. By choosing a particular character or the culture of your place of origin you can get inspiration for the type of potions you will be creating.\nC. Find knowledge regarding the character, background, style and personality and come up with a response to each. [substeps] Subscribe to an online wizardry comparison site to find online examples of wizard you wish to emulate.\nD. Keep doing this until you find the type of wizard you want to be. [substeps] Try to predict if your character will be a wizard or not because it might have some qualities.","prompt":"[header] How to pretend to be a wizard [title] Decide what kind of wizard you want to be. [step] Research your choice. Perform an image search for the character or style that appeals to you the most.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5397,"error":null,"output":"D"}
{"question_id":"hellaswag:00006478","item_index":229,"attempt":0,"prompt_hash":"a4d46bc7a45304816d26b4f59e153c794ee3257b435d7556dff752f08783a5d3","question":"[header] How to convince your husband to have a baby [title] Think about any prior discussions you have had about children. [step] One thing you may want to think about before you talk to your husband about having a baby is any previous conversations you have had about them. This may be information you can use to help your case.\n\nA. [substeps] You may want to refresh your memory regarding past conversations with your husband, or you may need to discuss how many children both of you have. Write down everything you remember about your conversation about the children and discuss when you spoke about the children to help you reconcile the information with your husband.\nB. [substeps] Did he say he wanted children before you got married? Did he say he never wanted children? If he said he wanted children, you can discuss this knowledge with him. If he said he never wanted to have children, you should discuss how you believed he would change his mind after being married for awhile.\nC. [substeps] If one of you had any discussions about your children recently, take this into consideration. If you have ever talked about child custody with your husband, there is a good chance he is interested.\nD. [substeps] For example, you may have had conversations with your husband about having children because you are concerned about their child's wellbeing. You may have conversations discussing the pros and cons of having kids before you have talked to him, as they are important to him.","prompt":"[header] How to convince your husband to have a baby [title] Think about any prior discussions you have had about children. [step] One thing you may want to think about before you talk to your husband about having a baby is any previous conversations you have had about them. This may be information you can use to help your case.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5393,"error":null,"output":"B"}
{"question_id":"hellaswag:00005322","item_index":230,"attempt":0,"prompt_hash":"47c3e72f52e36dbea26f508e45f96bd4143e6e4ad4c14f1ec91ee2df311f92f6","question":"[header] How to choose a modest wedding dress [title] Set your budget. [step] Before buying any wedding dress, it's important to determine your budget. You should figure out the total cost of everything involved in the wedding and figure out how much you can set aside for a dress.\n\nA. [substeps] Get rid of your \" a \" and \" b \" items. Instead, look into what makes your dress worthy.\nB. Keep in mind that you need to take into account, what decorations are required, and what attire you will need for the wedding. [substeps] You also should spend some time considering what to expect.\nC. This may be something like $10 , 000 or $250 , 000 or 5800 , 000, depending on your budget. [substeps] It's also important to know how long you expect your gown to be.\nD. If your parents or spouse's parents are paying for the wedding, discuss a reasonable budget with them. [substeps] Keep in mind you can always rent a dress if purchasing one is out of your budget.","prompt":"[header] How to choose a modest wedding dress [title] Set your budget. [step] Before buying any wedding dress, it's important to determine your budget. You should figure out the total cost of everything involved in the wedding and figure out how much you can set aside for a dress.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5397,"error":null,"output":"D"}
{"question_id":"hellaswag:00002504","item_index":231,"attempt":0,"prompt_hash":"afd58c633eb8cb30a44586d245d20ea7df97f7b669240eaed6b5f1981666fbc2","question":"A snowmobile speeds over icy terrain. A woman is interviewing a man inside a tent. they\n\nA. get caught in a trap and start fighting.\nB. drill a hole in the ice, and use their gear to fish.\nC. are riding their mounts down an empty road.\nD. camp outdoors, enjoying a snowy day.","prompt":"A snowmobile speeds over icy terrain. A woman is interviewing a man inside a tent. they","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5398,"error":null,"output":"B"}
{"question_id":"hellaswag:00001943","item_index":232,"attempt":0,"prompt_hash":"0f19f1afac011d830cc0770e958f6c6db99b520fb31e072cd6d898a6f3d9a0d5","question":"The man brings the bottle closer to his mouth and laughs some more and then pours the liquid from the small bottle onto his mouth and begins to gargle until bubbles come out of his mouth. the man\n\nA. grabs more water and continues trying to gargle and spit some more.\nB. smiles while backing away and wiping his mouth with his hand.\nC. moves his body a little bit and spits on the empty bottle.\nD. then quickly leans forward and spits it all out of his mouth.","prompt":"The man brings the bottle closer to his mouth and laughs some more and then pours the liquid from the small bottle onto his mouth and begins to gargle until bubbles come out of his mouth. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5398,"error":null,"output":"D"}
{"question_id":"hellaswag:00009454","item_index":233,"attempt":0,"prompt_hash":"c7e2b8d9c9b2a1c6d13a6392a71779182bf1cccef67a22726a8145e51878969b","question":"[header] How to store watches at home [title] Keep your watches in a storage case if you want to keep them in sight. [step] These cases are similar to a jewelry box, but the compartments are designed to fit watches. They are often lined with felt or another plush fabric that will prevent your watches from getting scratched or dented.\n\nA. [substeps] Watch boxes often have glass tops, allowing you to see your collection. This is a good option if you like to wear a different watch every day.\nB. If you store your watches in one particular place, you do not have to worry about having them easily moved from one place to another. [title] Store watches in a sealed book or case.\nC. [title] Keep watches in a mesh wrapping case. [step] Mesh is a piece of mesh or insulation made from no more than two strands of material.\nD. [title] Wrap your watches with felt or batting. [step] This will keep your watches safe and keep them from peeking over your shoulder.","prompt":"[header] How to store watches at home [title] Keep your watches in a storage case if you want to keep them in sight. [step] These cases are similar to a jewelry box, but the compartments are designed to fit watches. They are often lined with felt or another plush fabric that will prevent your watches from getting scratched or dented.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5391,"error":null,"output":"A"}
{"question_id":"hellaswag:00008530","item_index":234,"attempt":0,"prompt_hash":"64490eeada6fa0a30639a70f91c924412793669ca9895d58d3f8a8f3094d2f72","question":"[header] How to appear more authoritative [title] Know and accept your actual authority. [step] While there can be benefits to appearing authoritative in excess of your actual authority, the focus in this article is on enhancing the appearance to match the reality of your authority. To do so, especially in a business setting, it is important that you are clear on exactly what authority you do possess.\n\nA. This can be accomplished through presenting evidence to illustrate your position, convincing your audience that you are willing to respond to the issue with blatant evidence. [substeps] Rise above that cliché by keeping an air of authority despite a lack of it.\nB. As shown in the previous article, you must be clear on the person you are addressing. You may be perhaps nervous about issuing authority to an important person, such as a boss.\nC. [substeps] We all know the precise powers we have (or possess, if we know them). Power of attorney is no different, nor is it more formal.\nD. [substeps] If your powers and responsibilities have not been clearly defined, discuss the particulars of your authority with your boss. Having things specifically laid out for you may make it easier for you to accept and represent this authority.","prompt":"[header] How to appear more authoritative [title] Know and accept your actual authority. [step] While there can be benefits to appearing authoritative in excess of your actual authority, the focus in this article is on enhancing the appearance to match the reality of your authority. To do so, especially in a business setting, it is important that you are clear on exactly what authority you do possess.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5393,"error":null,"output":"D"}
{"question_id":"hellaswag:00004230","item_index":235,"attempt":0,"prompt_hash":"1e72539c2d7b4832400c17d837c62b7080b3396c16a7666fc4ac32e9668939ab","question":"[header] How to defend yourself in a fight [title] Try talking it out first. [step] If the other party is riled up and looking for violence, words might not do any good, but it's usually a good idea to try. Physical fights can very likely end people in the er, so avoiding them at all costs is advisable.\n\nA. [substeps] Whenever you decide to assert yourself, put on your protective mask and speak frankly. Think about your argument from the perspective of an ally as opposed to a single bystander trying to say you're wrong or problematic.\nB. Try to determine why the other person is angry, and see if you can talk them down. [substeps] Some people may be motivated not by anger but by something else, like a mugger whose motivation is money.\nC. [substeps] Don't enter a fight in a fury. Anger will only worsen the situation and be able to escalate and cause further conflict.\nD. [substeps] If the other person starts screaming, swearing, and acting out, that's it. Suggest we go somewhere else, or rather do something else.","prompt":"[header] How to defend yourself in a fight [title] Try talking it out first. [step] If the other party is riled up and looking for violence, words might not do any good, but it's usually a good idea to try. Physical fights can very likely end people in the er, so avoiding them at all costs is advisable.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5391,"error":null,"output":"B"}
{"question_id":"hellaswag:00006371","item_index":236,"attempt":0,"prompt_hash":"280493338b0cdbda88408352708a52bef01ef800107998d9a8cd906b264444e0","question":"[header] How to save ifunny videos on pc or mac [title] Go to https: // ifunny. Co in a web browser. [step] You can use any browser on your pc or mac. [title] Search for a video to save.\n\nA. [step] Hover over \" video \" on the bottom menu and select \" video stream \" from the drop-down menu. [substeps] Many video streams will be listed as \" uploadable \" and some may have micro-itunes restrictions.\nB. [step] Click the magnifying glass at the top-right corner of the box to open a page with trending hashtags and categories, or type what you're looking for into the search bar. [substeps] Videos have a play button (a sideways triangle) at the center of their thumbnails.\nC. [step] You can find video in any genre, such as documentaries, books, or just movies on your screen. Popular search engines include tinder, tumblr, and facebook.\nD. [step] It may be pertains to a theme, which can add mood to any video. If you're searching for a science show, enter the year in your query box (without the quotes), the name of the show's main administration, and a few general identifiers.","prompt":"[header] How to save ifunny videos on pc or mac [title] Go to https: // ifunny. Co in a web browser. [step] You can use any browser on your pc or mac. [title] Search for a video to save.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5394,"error":null,"output":"B"}
{"question_id":"hellaswag:00002116","item_index":237,"attempt":0,"prompt_hash":"78a8dd37d88422a2ceb16626b4d045346883c5a19eb79b5e450b121cd7a3a2cb","question":"A woman demonstrates how to dry a counter top and sink of water splashed from a sink faucet with a paper towel. A woman stands in a kitchen next to a sink and talks to the camera. the woman\n\nA. stands in a kitchen, takes out a sander, and uses it to brush the surfaces of a counter surface, underneath and below the sink.\nB. points to a water splash on the counter next to the sink after turning on the water at the sink.\nC. washes water bottles after mixing and liquids in a sink.\nD. drops a floating dollar in a bucket then soaks in water and dries with paper towel.","prompt":"A woman demonstrates how to dry a counter top and sink of water splashed from a sink faucet with a paper towel. A woman stands in a kitchen next to a sink and talks to the camera. the woman","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5395,"error":null,"output":"B"}
{"question_id":"hellaswag:00004157","item_index":238,"attempt":0,"prompt_hash":"4a746f14ae9ec69a261e21eaf647d9f4caf1de5a6d6198b838065f47275d1d64","question":"[header] How to get bug bites to stop itching [title] Cool the bite with a cold compress. [step] The cold will reduce the swelling and numb the area a little bit. [substeps] If you don't have an ice pack ready and waiting for use in the freezer, you can quickly make one by wrapping ice cubes or a package of frozen peas in a towel.\n\nA. Put one wrapped towel over the bite site. Cover the area with a thin layer of ice for a couple minutes, then remove the wrap and let the cool completely.\nB. Immediately apply ice to the affected area for approximately five minutes at a time. Repeat this process every ten minutes until the area stops itching.\nC. Be careful not to hold the ice on your skin for too long. Limit it to 15 to 20 minutes at a time.\nD. [title] Apply a thin layer of ice over the affected area. [step] The cold will help neutralize the insect bite after the fact.","prompt":"[header] How to get bug bites to stop itching [title] Cool the bite with a cold compress. [step] The cold will reduce the swelling and numb the area a little bit. [substeps] If you don't have an ice pack ready and waiting for use in the freezer, you can quickly make one by wrapping ice cubes or a package of frozen peas in a towel.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5393,"error":null,"output":"B"}
{"question_id":"hellaswag:00001149","item_index":239,"attempt":0,"prompt_hash":"06c8863d7c13e4b6b9f8db12f332acf7db3e14637a209e1d7a63e0e2d2f4a51f","question":"A woman is standing inside a bathroom. She is using the rollers on the bottom of a machine. she\n\nA. uses a curling iron to scrub the side of her face.\nB. sticks her boots in the machine and they get polished.\nC. uses them to blow dry her hair.\nD. begins to groom dogs using the product.","prompt":"A woman is standing inside a bathroom. She is using the rollers on the bottom of a machine. she","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5393,"error":null,"output":"C"}
{"question_id":"hellaswag:00005205","item_index":240,"attempt":0,"prompt_hash":"6dd361ab86b2931bbd173c13a3d9a7857d73bc1c9aacdfec20fa73e62c177c96","question":"[header] How to grow bird's nest fern as an indoor plant [title] Plant the bird's nest fern in good quality potting soil. [step] Be sure to use soil that permits excellent drainage as this fern does not like waterlogged feet. [substeps] Also ensure that the pot is able to drain water with ease.\n\nA. Filling the saucer with marbles, glass pebbles or real pebbles can help make drainage better, as well as allowing for evaporation to create humidity. [title] Water very generously during warmer months and in warm indoor environments.\nB. Add mulch to the planting site to deter rot. [title] Set aside at least 8 hours before heading out to shake branches or brush for fertilizers.\nC. Do not fertilize it with cat litter, wood chips, garden compost, or manure as the species can rot or die during this time. [title] Allow the fern to recover for 24 hours.\nD. You want the fern to retain water until it is ready to grow to avoid becoming too soggy. [title] Put on gardening gloves to keep ferns wet.","prompt":"[header] How to grow bird's nest fern as an indoor plant [title] Plant the bird's nest fern in good quality potting soil. [step] Be sure to use soil that permits excellent drainage as this fern does not like waterlogged feet. [substeps] Also ensure that the pot is able to drain water with ease.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5393,"error":null,"output":"A"}
{"question_id":"hellaswag:00001459","item_index":241,"attempt":0,"prompt_hash":"66b416d897e0cc8e3c252f54578d353d933de67d0f1d922a61764bd9c9440c2a","question":"A man is riding a surfboard in the ocean. he\n\nA. then sits on a board and holds onto it's handle.\nB. rides around wearing his surf board.\nC. is holding on to the sail.\nD. stands up on the board during a wave.","prompt":"A man is riding a surfboard in the ocean. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5394,"error":null,"output":"D"}
{"question_id":"hellaswag:00007295","item_index":242,"attempt":0,"prompt_hash":"ca11564cad556ec32fc1065dfc7d70eea408e09ba02fd8217c26fa511b977dcf","question":"[header] How to select shoes to wear with an outfit [title] Choose shoe colors that match your clothes instead of colors that compete with your clothes. [substeps] Wear simple black heels or flats when wearing a dress with a bright, bold pattern. If you were to wear a more complex shoe, it would be too overwhelming in the opinion of some.\n\nA. Wear a long, flowing skirt or floor length dress to a very casual event like a party or easter fest. Wear a strappy top or thin leggings if a more feminine style can be worn with this article of clothing.\nB. Limit your visibility that way and you'll be drawing more attention to your outfit as a result. Wear socks with long pant legs to hide your feet from the rest of the outfit.\nC. Instead, wear a thin pink or magenta knee-high shoe to match your dress or suit pants. [title] Order a few sneakers in a shade of pink, or heels to match your outfit.\nD. Obviously, you can wear whatever shoes you like provided there is no dress code to obey or health and safety issues to consider. Consider neutral or' nude' heels or flats if you have on a sparkly evening top.","prompt":"[header] How to select shoes to wear with an outfit [title] Choose shoe colors that match your clothes instead of colors that compete with your clothes. [substeps] Wear simple black heels or flats when wearing a dress with a bright, bold pattern. If you were to wear a more complex shoe, it would be too overwhelming in the opinion of some.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5391,"error":null,"output":"B"}
{"question_id":"hellaswag:00007608","item_index":243,"attempt":0,"prompt_hash":"943affbf23893833a294d2c23b57a1653cb7f382fffbe49215ad1b3898da3c71","question":"[header] How to become a police officer in florida [title] Meet basic qualifications to apply to a police training academy. [step] Florida law has established a set of minimum qualifications to become a police officer. If you meet those requirements, you can register for a police training academy, which is the first step you need to take to become a police officer in florida.\n\nA. [substeps] You must be at least 19 years of age and a citizen of the united states to meet the state requirements to enter an academy. Most agencies will require that you have a valid driver's license too.\nB. [title] Complete a 6-page application form. [step] This will require you to provide a valid driver's license and to provide your full legal name.\nC. [substeps] During the course of the training academy, you will have an intense one on one communication and forensics training. At the end of the training academy, you will receive a certification from the department of police.\nD. [substeps] You must be at least 18 years of age. When you apply for a police training academy through the private police academy district, you will need to use the \" serve area \" special training system.","prompt":"[header] How to become a police officer in florida [title] Meet basic qualifications to apply to a police training academy. [step] Florida law has established a set of minimum qualifications to become a police officer. If you meet those requirements, you can register for a police training academy, which is the first step you need to take to become a police officer in florida.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5393,"error":null,"output":"A"}
{"question_id":"hellaswag:00006250","item_index":244,"attempt":0,"prompt_hash":"4931b2673124c32e9579647504d3ece4dfe9c0a0bdecfed857f9cbd84a52ee4d","question":"[header] How to make longganisa [title] In a large bowl, add the soy sauce, vinegar, garlic, bay leaves, paprika, brown sugar, salt and pepper. [step] Stir the ingredients together until the brown sugar is dissolved. [title] Add the ground pork and pork fat to the bowl, and mix well.\n\nA. [title] Heat the rice under boiling water for five minutes. [step] Add the rice to the rice mixture.\nB. [title] Cover the bowl with plastic wrap and refrigerate overnight. [step] You can refrigerate it for up to 12 days.\nC. [title] Refrigerate for at least two hours or overnight. [step] This will allow the meat to firm and the flavors to blend well before forming the sausages.\nD. [title] Strain the mixture into a fine sieve or strainer. [step] Stir gently to incorporate all the liquids.","prompt":"[header] How to make longganisa [title] In a large bowl, add the soy sauce, vinegar, garlic, bay leaves, paprika, brown sugar, salt and pepper. [step] Stir the ingredients together until the brown sugar is dissolved. [title] Add the ground pork and pork fat to the bowl, and mix well.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5391,"error":null,"output":"B"}
{"question_id":"hellaswag:00002890","item_index":245,"attempt":0,"prompt_hash":"c9e491402d275d47289468445695564ccd4c1be081e89f2c43b2a8cc9ee0a5dd","question":"She goes to the grass and she begins hooping in circles, she is having a really good time dancing and hooping. then\n\nA. she starts to quick dance on the grass of the room.\nB. , other people are dancing around and hooping also, doing tricks and just enjoying the day.\nC. her mom comes back and she begins hooping herself doing cardio and her mom comes to help.\nD. she takes a break and they both finish dancing and shake hands.","prompt":"She goes to the grass and she begins hooping in circles, she is having a really good time dancing and hooping. then","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5391,"error":null,"output":"D"}
{"question_id":"hellaswag:00003101","item_index":246,"attempt":0,"prompt_hash":"1dfbf002082e42cfd23e2989bdd3715038c6210278e14666d43fc5b704b6981f","question":"A camera pans around several snow mobiles and shows people sitting around a hole. A man swims up in the water and speaks to the camera. another man\n\nA. is water skiing and showing several different tricks and how to do various flips into the depth.\nB. comes out of the water wet.\nC. climbs into the water next to him and they begin skiing.\nD. sits while he speaks and holds a frisbee already hanging from his saddle.","prompt":"A camera pans around several snow mobiles and shows people sitting around a hole. A man swims up in the water and speaks to the camera. another man","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4957,"error":null,"output":"C"}
{"question_id":"hellaswag:00004775","item_index":247,"attempt":0,"prompt_hash":"cfa5e05beab48a86f86e97e247ffe8ec06be814b3de5e944cb71dba4a164968a","question":"[header] How to know why a child (under 2) is crying [title] Listen for a loud cry, almost a wail. [step] This usually means \" i'm in pain \" or \" i'm sick. \" the baby will pause, then scream an entire lungful, and repeat the process.\n\nA. This sort of crying is normally just a warning sign of hunger. A baby will also cry slightly more if it is feeling hungry.\nB. This can be very upsetting to watch, so get a parent to come and help the baby. [substeps] Only a parent can judge their child's age.\nC. This will usually end after about three minutes. [title] Pay attention to the baby's closed eyes.\nD. It's persistent, piercing, and unmistakable. If you hear this cry, go to the child immediately.","prompt":"[header] How to know why a child (under 2) is crying [title] Listen for a loud cry, almost a wail. [step] This usually means \" i'm in pain \" or \" i'm sick. \" the baby will pause, then scream an entire lungful, and repeat the process.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4957,"error":null,"output":"C"}
{"question_id":"hellaswag:00000209","item_index":248,"attempt":0,"prompt_hash":"aeee9d6b34d88cdf350b7f89c136b764bdbe9aeebb68bf04b728df9d1ae34822","question":"A knife is put into a sharpener and moved back and forth and used on a circular sharpener. a pair of scissors\n\nA. is sharpened on the sharpener.\nB. is scraped away and then covered with a paper.\nC. is used and used on trimming a cheek.\nD. are placed into a glass of liquid and cut along skin.","prompt":"A knife is put into a sharpener and moved back and forth and used on a circular sharpener. a pair of scissors","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4957,"error":null,"output":"A"}
{"question_id":"hellaswag:00009776","item_index":249,"attempt":0,"prompt_hash":"6df1141ead9c8bf3d679ee0dcb6fb26688eafa59b342ce46c3671a5ac7052a40","question":"[header] How to give flat hair volume [title] Reduce your conditioner use. [step] Conditioner restores hair health with beneficial oils. If you have naturally greasy hair, however, the extra weight of these oils can make your hair limp.\n\nA. [substeps] Go for a massage during the week. Pour some olive oil into a lather and massage your scalp.\nB. Removing all these oils from your hair shows off your overall health and stimulates hair growth. [substeps] Greasy hair does not take very much, but you will need to make sure to limit the amount.\nC. Minimize this effect with this approach : [substeps] Use a small dollop of conditioner. Apply conditioner only to to the tips of your hair, not at the roots.\nD. To reduce this, reduce the amount of conditioner that you use throughout the day. [substeps] Lots of times dry hair will feel too moisturizing and try focusing more onto volume.","prompt":"[header] How to give flat hair volume [title] Reduce your conditioner use. [step] Conditioner restores hair health with beneficial oils. If you have naturally greasy hair, however, the extra weight of these oils can make your hair limp.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4957,"error":null,"output":"C"}
{"question_id":"hellaswag:00005263","item_index":250,"attempt":0,"prompt_hash":"28a1d520f182379f7a5c312ba2e776617aaca0444b280c221c6bb142cd45d91e","question":"[header] How to sign up for file storage at dropbox and use public folder at dropbox [title] Go to http: // www.dropbox.com it is available for free for windows, mac, linux and mobile devices. [step] Click on setup file to run it on your computer. A welcome screen will appear.\n\nA. Click on \" install \" button to start the installation process. After the dropbox installation will start running on your computer, you will see a progress bar that will show the progress of the installation.\nB. [title] Confirm the actions that you want to take. [title] Locate your default file type in the box, download file explorer and double click on file explorer.\nC. [title] Select \" file \" and then a settings dialog box will appear. [step] Click \" file privacy protocol \" to enable public folder sharing.\nD. [title] Click on install file and you are ready to start. [title] Make sure you have the appropriate computer name (vconfig or php), the computers file format (php) and your url if desired, and the folder name for the file to which you have the appropriate installation preferences.","prompt":"[header] How to sign up for file storage at dropbox and use public folder at dropbox [title] Go to http: // www.dropbox.com it is available for free for windows, mac, linux and mobile devices. [step] Click on setup file to run it on your computer. A welcome screen will appear.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4957,"error":null,"output":"A"}
{"question_id":"hellaswag:00009897","item_index":251,"attempt":0,"prompt_hash":"b74d698666c56e1d544b77f11964be2f5bcfbb20db6c16bed680874bb4bdae87","question":"[header] How to work at a recording studio [title] Be friendly and adaptable. [step] Working at a recording studio requires a passion for music and the ability to adapt quickly to new technology. You should be able to get along well with others and work on a team.\n\nA. You'll also need to be able to convert input from producers and musicians into results that are both technical and provoke an emotional response. [substeps] Working at a recording studio requires both technical and artistic skills.\nB. Also, you should be able to develop communication skills with others if necessary and share learning styles with the others who work with you. [title] Get experience before you move on to more complicated projects.\nC. You may not be good at music management or used to working outside of a team. If you work with a band, perform a regular show at their studio and earn money in the process.\nD. Consider pursuing an accredited degree, if you want to get certified as a director. [substeps] To get an accredited degree or certification, talk to an accredited music video or recording studio director.","prompt":"[header] How to work at a recording studio [title] Be friendly and adaptable. [step] Working at a recording studio requires a passion for music and the ability to adapt quickly to new technology. You should be able to get along well with others and work on a team.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4012,"error":null,"output":"A"}
{"question_id":"hellaswag:00002403","item_index":252,"attempt":0,"prompt_hash":"f4c7bff3f66d676a2329de033d44452568296d5449d53c3e495aaf07bb367f97","question":"A woman speaks to the camera while turning her face back and fourth. she\n\nA. holds up a product and wipes it all along her face.\nB. demonstrates how to properly use abdominal crunches.\nC. then laughs, waves, and sets her face back in the water.\nD. continues speaking while showing a \" you'll never make it \" text across the screen and the woman turns around.","prompt":"A woman speaks to the camera while turning her face back and fourth. she","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4011,"error":null,"output":"A"}
{"question_id":"hellaswag:00000966","item_index":253,"attempt":0,"prompt_hash":"dcec68c47b801bc4645091ebe8831240547f26b6cd630972152ec41bd5f6f042","question":"A news segment begins and black and white picture of a smiling girl appears on the screen and the banner on the screen says \"henrico teen dies after race father recounts daughter's final moments\". the newscaster\n\nA. tells viewers how her team is running and she hugs a cheerleader for showing her final moments.\nB. appears with her parents talking about the girl while clips of the race play, and still shot pictures of the girl appear on the camera.\nC. returns to his station.\nD. talks about news reporting and young children dying and is shown talking to a curly haired girl from england who is wearing a jacket and smiling.","prompt":"A news segment begins and black and white picture of a smiling girl appears on the screen and the banner on the screen says \"henrico teen dies after race father recounts daughter's final moments\". the newscaster","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4011,"error":null,"output":"B"}
{"question_id":"hellaswag:00000941","item_index":254,"attempt":0,"prompt_hash":"a9e5f424ea249559490b9334e1a6507c58ca6c9edf54a57fccaad42dbb106da2","question":"A boy is seen speaking to the camera holding two basketballs and leads into him holding them up to his face. the boy\n\nA. then moves up and down the court while dribbling the balls and shoots a basket off in the end.\nB. grabs the ball and begins dribbling the ball over and over again and hitting the ball with the basketball.\nC. then throws the ball around while speaking to the camera and hitting a ball around.\nD. begins taking off his gear and throwing the ball to the other team while swinging around the stick.","prompt":"A boy is seen speaking to the camera holding two basketballs and leads into him holding them up to his face. the boy","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4010,"error":null,"output":"B"}
{"question_id":"hellaswag:00002437","item_index":255,"attempt":0,"prompt_hash":"137425c1e16042784c4ddcc0f2018241bef6288579a0f8c52df4d122a994cf2f","question":"The female athlete is standing and then positioned. the athlete\n\nA. jumped and pulled the rope using her feet.\nB. then spins and kicks with both legs, trying to throw the ball across the field.\nC. ran and jumped over the pole the female athlete get off the mat and walked away.\nD. jumped from a first base position and landed on the shallow field, while her teammates take turns to hit her.","prompt":"The female athlete is standing and then positioned. the athlete","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4010,"error":null,"output":"B"}
{"question_id":"hellaswag:00003510","item_index":256,"attempt":0,"prompt_hash":"bf1163fda1b383d9d4465a48f43ec3d30d8a986ed0adde47922c0ab0de724eb6","question":"[header] How to get an autograph at a baseball game [title] Try to get seats up front close to the field but not behind the net. [step] You should also get up front during batting practice or warm-ups. Players will sign items before the game with generosity.\n\nA. [title] If you are upset that someone has signed your paper, and it becomes wet by the time the game is over, raise your hand, or have someone do it for you. [step] Scream, peel off your shirt and say \" got the paper? That works for us! \" [title] Be sneaky, and glance over at the ball.\nB. [title] , remember to have a blanket on your lap at the first inning. [step] You must also have a towel nearby to prevent getting wet.\nC. [title] Observe where the players stand, wait, or offer autographs at games. [step] You may not be able to get the autograph during the game but can try to catch a player after the game, before the game during warm ups or post game near the player exit.\nD. [title] Make sure there is a spectator statue in front of the field. [step] During the game you should see a big statue either behind the line of scrimmage or next to the pitch.","prompt":"[header] How to get an autograph at a baseball game [title] Try to get seats up front close to the field but not behind the net. [step] You should also get up front during batting practice or warm-ups. Players will sign items before the game with generosity.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4246,"error":null,"output":"C"}
{"question_id":"hellaswag:00005284","item_index":257,"attempt":0,"prompt_hash":"a4ddba33642d2568e65639bdf90d5feb776f8f6ab13a91d52cee338756dd3562","question":"[header] How to take care of military boots [title] Use the on and off brush to remove muck from your boots, and to get the sides shiny. [step] First of all, get your on brush and put a bit of polish on it, although not too much. Don't wet the polish or anything yet.\n\nA. Let it dry before proceeding with the next step. [title] Use the toe brush to insert up and down the inside of the boot, then (preferably on the longest side) up to and around the top edge, trying to get the cuff down smooth.\nB. [title] Apply the polish on the boot, around the sides, heel and toe until the boot is dull [title] Take your off brush and brush really really hard for ages until the boot is shiny. [step] You won't be taking the polish off, but rather making it shiny.\nC. When getting a fresh pair of boots, put on a little bit, to start. [title] Once you've gotten the dirty boots to your liking, you might want to do a side brush on them (you'll never know it how dirty they usually get).\nD. Also, you will want to put newspaper underneath to dry before putting polish on the boots. After a good soaking, wipe off the excess with a microfiber cloth to get the coat looking shiny again.","prompt":"[header] How to take care of military boots [title] Use the on and off brush to remove muck from your boots, and to get the sides shiny. [step] First of all, get your on brush and put a bit of polish on it, although not too much. Don't wet the polish or anything yet.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4246,"error":null,"output":"B"}
{"question_id":"hellaswag:00002689","item_index":258,"attempt":0,"prompt_hash":"b6754dd6df3554d8dc220f9b5805cf047642e18d1080e1cae468039f2198ff05","question":"The lady from china has a second turn and also knocks over the bar and looks mad. the second lady from uzbekistan\n\nA. rock has the third turn but isn't successful because the bar is stuck.\nB. performs and clears the bar and almost cries.\nC. has her turn and then realizes she was supposed to have a turn and try to do the winning one but in the end fell off the bar.\nD. has her third turn and she looks angry as she adds another five before leaving.","prompt":"The lady from china has a second turn and also knocks over the bar and looks mad. the second lady from uzbekistan","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4251,"error":null,"output":"B"}
{"question_id":"hellaswag:00008356","item_index":259,"attempt":0,"prompt_hash":"5d0b01f5f273ef7fb2734862fdc206b9a426cab5361c7050a4cc2ec378284d41","question":"[header] How to stop kids from being afraid of monsters [title] Search for monsters. [step] Ask your child where she thinks the monsters are-under the bed, in the closet, and so on. Then look for the monsters together, and show your child they don't exist.\n\nA. [substeps] You don't want to just say, \" there's nothing there; go to sleep, \" nor do you want to encourage the fear by pretending the monsters are there and you can remove them. The best way to tackle the fear is to actually show your child that the monsters aren't real and they have nothing to fear.\nB. Obviously it's easier said than done, but you are bound to be surprised at how many creatures there are. [title] Make room for horror that your child hasn't ever seen.\nC. If some kids are scared of monsters, look for them too, to have fun with them. [substeps] You can draw pictures of the monsters for your child to draw as she watches it all.\nD. [substeps] Give your child suggestions so they think about what to look for. Something from your own childhood can help teach your child exactly where monsters are.","prompt":"[header] How to stop kids from being afraid of monsters [title] Search for monsters. [step] Ask your child where she thinks the monsters are-under the bed, in the closet, and so on. Then look for the monsters together, and show your child they don't exist.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4245,"error":null,"output":"A"}
{"question_id":"hellaswag:00000441","item_index":260,"attempt":0,"prompt_hash":"daf158a111d5c481bc10c02b481f523c2f6e91a61f0c7b486d70e396d58e6db5","question":"The man uses stacked pennies to get the angle of the knife to be sharpened and demonstrates how to sharpen kitchen knives using a stone. The man slides the knife on the stone while pressing the knife with his fingers. the man\n\nA. points and sits next to the knife down.\nB. then begins sharpening a metal tool to sharpen.\nC. begins to sharpen knives using regular knives.\nD. demonstrates the proper way to sharpens a knife using a steel.","prompt":"The man uses stacked pennies to get the angle of the knife to be sharpened and demonstrates how to sharpen kitchen knives using a stone. The man slides the knife on the stone while pressing the knife with his fingers. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4249,"error":null,"output":"D"}
{"question_id":"hellaswag:00005092","item_index":261,"attempt":0,"prompt_hash":"1c49ee2df8190be53edd5f7f208c70de368e04b979f21d018d49e5ad7ef2ad3d","question":"[header] How to be close with your teen child [title] Always keep your promises. [step] When you promise something, for example, to buy them something, or to take them some place make sure you always keep your word. [substeps] Never make empty promises.\n\nA. Even a promise to do your teen's chores or to go to a club with them could end up being meaningless. Always promise to make the effort, even if you aren't serious about it.\nB. Avoid doing people favors that you know they will never do again. If you have to promise something, say it respectfully but never avoid it.\nC. If you say you are going to do something, do it. If you have a genuine reason breaking a promise, be sure to explain it to your teen-aged child.\nD. If your kid is planning on going to school, write down what you promise them and stick to it. Say this in the afternoon or when you're ready to pick your child up for school.","prompt":"[header] How to be close with your teen child [title] Always keep your promises. [step] When you promise something, for example, to buy them something, or to take them some place make sure you always keep your word. [substeps] Never make empty promises.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4244,"error":null,"output":"C"}
{"question_id":"hellaswag:00006644","item_index":262,"attempt":0,"prompt_hash":"043084ad75c8cc6cc1cbe2af9da00b988ddcbf681f8aa92da3d2fd1d7c709b01","question":"[header] How to condition your knuckles [title] Get in a standard pushup position. [step] Make sure your back is straight and your feet are shoulder length apart. Complete knuckle pushups on a soft surface like carpet, a workout mat, or even grass.\n\nA. [substeps] To lift your nails, make sure you don't lift your nails up past their width, or you'll put more strain on your joints. If you're doing handstands, place your feet shoulder width apart and your hands on the ground instead of in the pushup position.\nB. If you perform them lying down, you can choose to do them on the floor. [title] Roll your toes forward so that your toes are pointed.\nC. [substeps] If you can't do the sit-ups that you used on touch-ups, lay down on a carpeted floor on your hands and knees, and hold the breath. Imagine a straight line running straight to your fingertips.\nD. Over time, once these surfaces feel comfortable and are not challenging anymore, you can do knuckle pushups on a hard floor or concrete. [title] Place your weight on the first two knuckles of each hand.","prompt":"[header] How to condition your knuckles [title] Get in a standard pushup position. [step] Make sure your back is straight and your feet are shoulder length apart. Complete knuckle pushups on a soft surface like carpet, a workout mat, or even grass.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4245,"error":null,"output":"D"}
{"question_id":"hellaswag:00004723","item_index":263,"attempt":0,"prompt_hash":"048dd16a2f03d2a77f00bd09cce331ef37cf6e91988cfe4251a1f6696acfc421","question":"[header] How to make avgolemono soup [title] Pat the chicken dry with paper towels. [step] This will help promote a better sear later on. [title] Season the chicken with salt and pepper.\n\nA. [step] This will help it bind together easier. [title] Transfer the chicken to a large flour-based saucepan.\nB. [step] Do this with both wet and dry seasonings. Seasoning the chicken will keep it fresh for longer.\nC. [step] Pepper is best because you can experiment with different seasonings. [title] Mix 1 teaspoon of cumin, garlic and 1/4 teaspoon of white pepper in a quart (1.1 l) of boiling water.\nD. [step] ( you can add your own spices if you want. ) [title] Place the chicken in a pot of moderately hot oil.","prompt":"[header] How to make avgolemono soup [title] Pat the chicken dry with paper towels. [step] This will help promote a better sear later on. [title] Season the chicken with salt and pepper.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4249,"error":null,"output":"D"}
{"question_id":"hellaswag:00006361","item_index":264,"attempt":0,"prompt_hash":"e395963ac1d43e8efef6e008711b5da1bc2b809908201637255413d4e6ce76a6","question":"[header] How to make your appearance reflect on who you are [title] Take some time to think about who you are. [step] Consider what colors interest you, what kind of style you like and what simply makes you feel good about yourself. [title] Consider dyeing your hair.\n\nA. [step] Most people's natural hair color matches their skin tone anyway, but if it doesn't you should think about dyeing it a color slightly darker than your skin tone. You should also take good care of your hair by bathing every day, grooming hair and using a good shampoo and conditioner.\nB. [step] There is a wide variety of ways to do this, from coloring your hair purple to dyeing it white. Make sure you are not copying the hues of your former hair and that the shade matches perfectly.\nC. [step] Although color can be important in creating visual interest, it can also help you understand your upkeep and make your look more personable. [title] If you have acne you're more likely to have it.\nD. [step] Going for a sleek and sophisticated look can be a fun way to get some gear to you but it is the other little things that stop a major trend from coming. [substeps] Dyed hair can make you as kissable as bleached hair.","prompt":"[header] How to make your appearance reflect on who you are [title] Take some time to think about who you are. [step] Consider what colors interest you, what kind of style you like and what simply makes you feel good about yourself. [title] Consider dyeing your hair.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4246,"error":null,"output":"A"}
{"question_id":"hellaswag:00001470","item_index":265,"attempt":0,"prompt_hash":"041fa6c10e50aee8a29a967ea99d4140e423903905b208b6f47c6844ff845335","question":"A woman on screen talks about lemonade. She juices her lemons on a cutting board, with a knife and bowl. she\n\nA. cuts a flower with the cutting block.\nB. grinds the lemons green, with a wooden spatula.\nC. strains her juice into a bigger cup.\nD. pours jello in the bowl onto a white plate and along with several different fruits and vegetables.","prompt":"A woman on screen talks about lemonade. She juices her lemons on a cutting board, with a knife and bowl. she","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4248,"error":null,"output":"C"}
{"question_id":"hellaswag:00007243","item_index":266,"attempt":0,"prompt_hash":"a37b256db653c2b4b079d30a258a9a47158a4e189f34c8b2b5b4aa2bba6dc89a","question":"[header] How to have a nice hairstyle when you are obese [title] Take good care of your hair. [step] Some basic things you need are shampoo, conditioner, brush, wide toothed comb, heat protection, and whatever else you think is necessary. [substeps] It is important to keep your hair clean and conditioned.\n\nA. Use a heat protecting shampoo once a week, but if you have very short, longer hair, use a different type. It is a good idea to use conditioner once a week for medium to long hair.\nB. This does not mean scrubbing the living daylights out of your hair day in and day out, however. This simply means washing as often as needed.\nC. But if your hair is oily try a leave in conditioner, cumin powder, or apple cider vinegar in shampoo. Not enough of those would work but a little spritz of cleanser can give you volume.\nD. Doing that, often, will make you look like a slob. This is especially true if you are constantly changing your hair.","prompt":"[header] How to have a nice hairstyle when you are obese [title] Take good care of your hair. [step] Some basic things you need are shampoo, conditioner, brush, wide toothed comb, heat protection, and whatever else you think is necessary. [substeps] It is important to keep your hair clean and conditioned.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4245,"error":null,"output":"B"}
{"question_id":"hellaswag:00001488","item_index":267,"attempt":0,"prompt_hash":"167ce3922c0ade4c8afa003f367aea4eacb2a5a8066442bb8b7cd36c453a101d","question":"A man prepares himself on a dock to go water skiing. He launches into the water as the boat guns off. the man\n\nA. goes down a small waterfall at high speed.\nB. runs and jumps off the dock to drink a cup of water.\nC. rhythmically mounts the wide, flat-bottomed ski.\nD. holds onto the rope while gliding through the water, looking very confident and skilled.","prompt":"A man prepares himself on a dock to go water skiing. He launches into the water as the boat guns off. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4247,"error":null,"output":"D"}
{"question_id":"hellaswag:00002616","item_index":268,"attempt":0,"prompt_hash":"476084cfc1183b477bb1842bd0cf6c0b9496f00951a31d632176c922135f0514","question":"The girl holds the cats paw in her left hand. The cat groomer grabs a cat nail clipper in her right hand. the cat groomer\n\nA. squeezes the cats paw and proceeds to clip the cats nails.\nB. shines the nail of the girl.\nC. stops to give the cat paw back to the woman.\nD. takes the cat's left paw and with her open hand cut the nails with latex less nail clippers.","prompt":"The girl holds the cats paw in her left hand. The cat groomer grabs a cat nail clipper in her right hand. the cat groomer","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4247,"error":null,"output":"A"}
{"question_id":"hellaswag:00005576","item_index":269,"attempt":0,"prompt_hash":"d73fd05db92856aefe311fc3dc29fec54deb54014b5567f2721cbc60eec91610","question":"[header] How to cook vegetarian chili [title] Take a large, heavy bottomed pot, add 2 tbsp olive oil, cumin, chili powder, oregano, and cayenne pepper. [title] Sauté over medium heat for about 1 minute, stirring to toast the spices. [title] Add the soya crumble, tofu or tvp, and stir well to coat.\n\nA. [title] Add the garlic and chopped onion, stir, and sauté, adding up to two more tablespoons of olive oil if needed. [title] Cook, stirring occasionally for about three minutes, making sure the mixture does not scorch.\nB. [title] There you go! [step] Remove from the pan and let cool. [title] Heat vegetable oil in a large pot over medium low to medium heat.\nC. [title] Drop in the precooked chili powder. [title] Pour the tomato puree from the skillet and stir or shake well.\nD. [title] Add soy sauce (non-flavored). [title] Remove from heat, cover to simmer and simmer for 30 to 60 min.","prompt":"[header] How to cook vegetarian chili [title] Take a large, heavy bottomed pot, add 2 tbsp olive oil, cumin, chili powder, oregano, and cayenne pepper. [title] Sauté over medium heat for about 1 minute, stirring to toast the spices. [title] Add the soya crumble, tofu or tvp, and stir well to coat.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4246,"error":null,"output":"A"}
{"question_id":"hellaswag:00005151","item_index":270,"attempt":0,"prompt_hash":"bcf2a354eccac60095a45a0b03ea8bebf95fb81c30d7fc0666bb6e25f1cd5e85","question":"[header] How to cook beef joint [title] Buy your beef joint according to how you want to cook it. [step] Tender cuts of meat, like sirloin and fore rib, are better for roasting, while leaner cuts of meat, like topside and silverside, are better for braising. [title] Pick out a fresh cut of beef.\n\nA. [step] When purchasing beef, make sure the meat is a deep, dark red. Look for beef containing little streaks of fat throughout leaner muscle, and with an outside layer of firm, creamy-white fat.\nB. [step] Choose a cut of beef that's still fresh but that's not dripping any liquid. Remove any fat, and dice the beef till it's sweet-smelling.\nC. [step] You can get the joint from your local butcher's store if you're not planning on braising it yourself. Think about purchasing very fresh cuts of beef, then pick the one that matches your braising preferences.\nD. [step] For example, tenderloin or prime sirloin are flat cuts of beef. This helps prevent overcooking, for example, by roasting for a minimum of 30 minutes.","prompt":"[header] How to cook beef joint [title] Buy your beef joint according to how you want to cook it. [step] Tender cuts of meat, like sirloin and fore rib, are better for roasting, while leaner cuts of meat, like topside and silverside, are better for braising. [title] Pick out a fresh cut of beef.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4245,"error":null,"output":"A"}
{"question_id":"hellaswag:00005553","item_index":271,"attempt":0,"prompt_hash":"6b4a30f93a1d71401870ddd63d24e6992db91ee0e4948a2676a55b2b1d3439f7","question":"[header] How to fly an rc quadcopter [title] Find a safe, open place to practice. [step] The best place to practice is a spacious field. Survey your area for public parks, sports fields, and outdoor recreational areas until you come across a suitable takeoff point.\n\nA. Make sure you are not isolated from your group of friends, family, or anybody else that is going to be in your copter. [substeps] In case a gamblers anonymous flyer is on a first-come, first-served basis, you will want to know your private details about the flyer.\nB. However, avoid any places that have high fences or other other obstructions. [substeps] You should also clear your road by walking around in place.\nC. Grass is also a major plus for first time flyers, since it will cushion crash landings in the event that something goes wrong. [substeps] Stay away from concrete until you're confident in your piloting skills.\nD. Whatever you use to practice, make sure to observe the permit process and verizon law. To obtain your permit, you must go to a government agency (such as the governmental assembly of ngutheir international transmes at).","prompt":"[header] How to fly an rc quadcopter [title] Find a safe, open place to practice. [step] The best place to practice is a spacious field. Survey your area for public parks, sports fields, and outdoor recreational areas until you come across a suitable takeoff point.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4244,"error":null,"output":"C"}
{"question_id":"hellaswag:00008559","item_index":272,"attempt":0,"prompt_hash":"842a3638af5dcb34da12edce0ca80c245aed174d1d2adccba95b2eb82bb40d7d","question":"[header] How to tell a freshwater crocodile from a saltwater crocodile [title] Check how big the crocodile is. [step] One of the major differences between a freshwater crocodile and a saltwater crocodile is size. Freshwater crocodiles are much smaller than saltwater crocodiles, growing to about two meters, with males reaching up to three meters, and a maximum weight of 90 kg.\n\nA. This is a lot larger than saltwater crocodiles, who'll generally stand around 83 kg. [substeps] Skip the smaller crocodiles for now.\nB. In contrast, saltwater crocodiles can grow to be as long as six or seven meters and weigh over 1000 kg. [substeps] Saltwater crocodiles as so big they are considered the largest reptiles in the world in terms of mass.\nC. Because of this, freshwater crocodiles are smaller than saltwater crocodiles. [title] Look for the crocodile's feeding habits.\nD. Check your local reptile or marine food store for better identification of your saltwater crocodile. [title] Identify any other crocodile's habitats.","prompt":"[header] How to tell a freshwater crocodile from a saltwater crocodile [title] Check how big the crocodile is. [step] One of the major differences between a freshwater crocodile and a saltwater crocodile is size. Freshwater crocodiles are much smaller than saltwater crocodiles, growing to about two meters, with males reaching up to three meters, and a maximum weight of 90 kg.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4246,"error":null,"output":"C"}
{"question_id":"hellaswag:00004170","item_index":273,"attempt":0,"prompt_hash":"a18ed4350f5dafb10690f6e524d8d9e250b53ccd83613f1148054f96f9f9a4e1","question":"[header] How to tame curly hair [title] Throw out the hairbrush. [step] Using a hairbrush regularly will dismantle your curls and cause breakage and frizz. Use a wide-toothed comb or your fingers while your hair is wet to detangle.\n\nA. Then, use your fingers to gently comb the hair so it's smooth and tangle free. [substeps] The goal is to get through all the tangles-no frizzier hair! Use your fingertips to gently brush the ends of your hair to make them tangle as you do this.\nB. If you need to detangle during the day, wet that section of hair and use your fingers to gently comb through it. [substeps] Never brush wet hair--use a wide-toothed comb or your fingers out of the shower.\nC. Comb it out as necessary to avoid thinning it. You can apply a good conditioner to wet hair when your hair is dry, but don't use conditioner so often to tame it.\nD. Comb sections upward, then down to loosen frizz. [title] Apply anti-frizz serum in the shower.","prompt":"[header] How to tame curly hair [title] Throw out the hairbrush. [step] Using a hairbrush regularly will dismantle your curls and cause breakage and frizz. Use a wide-toothed comb or your fingers while your hair is wet to detangle.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4248,"error":null,"output":"B"}
{"question_id":"hellaswag:00002399","item_index":274,"attempt":0,"prompt_hash":"739842f08e2150dc8427ea032fd5cbad66ffcd57c8e264a178e12ce28cc7d4b1","question":"The interior of an rv is shown. a man\n\nA. is talking while standing on skis.\nB. takes off his wet undershirt and washes his chest.\nC. takes a rag and wets it in the sink.\nD. places places a bucket on the ground.","prompt":"The interior of an rv is shown. a man","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4247,"error":null,"output":"A"}
{"question_id":"hellaswag:00000901","item_index":275,"attempt":0,"prompt_hash":"3ec262433ffeb5f5f9ab8adbe0c0a95561f05a1b19130c92369fcf443e8d3b31","question":"Old man is raking leaves outside his home in the fall. Man is narrating the action. there\n\nA. is a fire going around the yard on the back yard.\nB. is a camera filming the little boy.\nC. is give yourself a break till the movie time.\nD. is a bike parked next to the house.","prompt":"Old man is raking leaves outside his home in the fall. Man is narrating the action. there","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4246,"error":null,"output":"D"}
{"question_id":"hellaswag:00007645","item_index":276,"attempt":0,"prompt_hash":"9dc3f04fdfe4668103da7a6f06c28df9de76e5d655f053a98d2ec24a6839774b","question":"[header] How to apply powder blush [title] Use powder blush if you have oily skin. [step] Forms of blush made out of liquid and cream do not hold up well on oily skin. Powder blush, on the other hand, deals with natural oil extremely well, lasting far longer without the need for constant reapplication.\n\nA. It allows access to more pigments, or even gives you a little boost in saturation. Type of blush works for both oily and dry skin, but figure out what type of blush works best for you.\nB. [title] Pick a blush shade that matches your skin tone. [step] If you're going for a natural look, you'll want to pick a blush color that complements your skin tone.\nC. For this reason, powders are best for any skin type. [substeps] For oily skin, avoid powder blush, since it likely contains less oil than you want to achieve.\nD. If you are putting powder blush on your face, apply a clear base blush (a solid shade) at least four shades lighter than your skin. Dip your finger into the liquid or cream blush and begin dabbing the powder into your face.","prompt":"[header] How to apply powder blush [title] Use powder blush if you have oily skin. [step] Forms of blush made out of liquid and cream do not hold up well on oily skin. Powder blush, on the other hand, deals with natural oil extremely well, lasting far longer without the need for constant reapplication.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4247,"error":null,"output":"B"}
{"question_id":"hellaswag:00002036","item_index":277,"attempt":0,"prompt_hash":"f011989657b2e6f78a9b9c140e52145d4a7404c8165cbf189fef88b3ed3b5ea9","question":"A man is sitting on the stairs playing a harmonica. he\n\nA. is sitting on a set of stairs with drums behind him.\nB. stops playing to play a riff.\nC. waves his hands over his shoulders as he plays.\nD. stands up and continues playing it.","prompt":"A man is sitting on the stairs playing a harmonica. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4246,"error":null,"output":"D"}
{"question_id":"hellaswag:00005095","item_index":278,"attempt":0,"prompt_hash":"0fff0d02c6d20b8eec67c0333e137d9b4f1628d5d7cbf4fbc85b8d63da521484","question":"[header] How to plant pitcher plants [title] Use a premade mix specially created for carnivorous plants. [step] Using these mixes is highly recommended. Pitcher plants typically grow in damp soils that are devoid of most regular nutrients.\n\nA. They evolved to catch bugs to fulfill their nutritional needs. Therefore, even the tiniest bit of chemical fertilizer is a recipe for disaster and most of the time will cause fatalities.\nB. Consult your local garden nursery or cpa to determine if a specific mix will create a good environment for your pitcher plant. [substeps] Nearly any brand of bark can be grown in a dry, temperature controlled environment.\nC. This, in turn, contributes to the breakdown of leaves and are the core of the garden's drainage. [title] Wash the plant in lukewarm water before transplanting.\nD. However, other types of plants may grow in colder soils. These kinds of plants do not require aeration but they can be grown indoors.","prompt":"[header] How to plant pitcher plants [title] Use a premade mix specially created for carnivorous plants. [step] Using these mixes is highly recommended. Pitcher plants typically grow in damp soils that are devoid of most regular nutrients.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4364,"error":null,"output":"A"}
{"question_id":"hellaswag:00008396","item_index":279,"attempt":0,"prompt_hash":"e21cf1fafb3ff8d962016d5fdc139a548833ae15c56032749c245dc98f65d649","question":"[header] How to identify a termite infestation [title] Look for swarming near lights. [step] This is because light attracts flying termites, especially during the night. Also look for swarming near windows and doors.\n\nA. Termites enjoy the light and are more likely to squeeze through caulk. [substeps] Avoid termites in buildings with windows.\nB. If you see a large congregation of termites in the air, then the termites are exhibiting swarming behavior. Swarming can occur during the day or night.\nC. [title] Look for termites near ponds or trash cans. [step] Termites have nests inside these containers.\nD. Notice whether you find white dots or mounds on surfaces near your home or any areas that appear full of termites' excrement. [substeps] Nearby buildings, sheds, entryways, telephone poles, and driveways are common places where termites live.","prompt":"[header] How to identify a termite infestation [title] Look for swarming near lights. [step] This is because light attracts flying termites, especially during the night. Also look for swarming near windows and doors.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4366,"error":null,"output":"B"}
{"question_id":"hellaswag:00007458","item_index":280,"attempt":0,"prompt_hash":"df4025587028d83b8c380af455cb029ffffb1efd8ca29ece291992fe12748fd4","question":"[header] How to remove grease from clothes [title] Try dish detergent. [step] The best and simplest way to remove anything oily or greasy from any fabric is by generously applying a concentrated dish detergent directly to the stain. It contains no harsh chemicals so you don't have to worry about it damaging the fabric.\n\nA. [substeps] Have your stain removed by a commercial stain remover that is specifically formulated for fabrics and garments. You can replace the dish detergent or laundry detergent by saturating a rag with the dish detergent.\nB. However, try to use dish detergent that is specifically designed for the use of grease. In one study, while 4% solutions were successfully removed with much less residue , 32% solutions still were unsuccessful.\nC. Choose a small batch of dish detergent and test it on a small strip of clothing before cleaning it. You can find dish detergent at most local department stores and big-box stores.\nD. Dish detergent is formulated to cut through grease plain and simple. Using a small brush (old toothbrushes work great! ) scrub the stain in a circular motion for several minutes.","prompt":"[header] How to remove grease from clothes [title] Try dish detergent. [step] The best and simplest way to remove anything oily or greasy from any fabric is by generously applying a concentrated dish detergent directly to the stain. It contains no harsh chemicals so you don't have to worry about it damaging the fabric.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4365,"error":null,"output":"D"}
{"question_id":"hellaswag:00000405","item_index":281,"attempt":0,"prompt_hash":"f277b8fffd22134aefd86aaa7ada4917bf6d87a0e7736fae9cce6ffcc785cc2f","question":"The men flip then the ladies flip and the men lift them in the air and hold them up as the crowd claps. The people forum groups and flip three girls in the air. the people\n\nA. stop spinning and poles except for the right and right girl.\nB. put the ladies on their shoulders and flip them into the air and spin then do arm wrestling as people run running up.\nC. form two groups and hold up two girls who hold up one girl together.\nD. flip and the men stand on them.","prompt":"The men flip then the ladies flip and the men lift them in the air and hold them up as the crowd claps. The people forum groups and flip three girls in the air. the people","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4364,"error":null,"output":"C"}
{"question_id":"hellaswag:00004948","item_index":282,"attempt":0,"prompt_hash":"88e1ae58d9b336d12bbcd724c5cfcc012d4b32dec4d7fb2301548ef94c890991","question":"[header] How to convert and combine office documents to one pdf [title] Open adobe acrobat pro. [step] Find the \" create \" function and click it to drop download the list, and you'll see \" combine files into a single pdf \". [title] Add office documents.\n\nA. [step] Click \" create pdf \" while you're downloading the list. The file files will be added after you click \" import \" on the drop down list, and the dialog box will appear with all the documents you've separated and/or converted to one pdf.\nB. [step] Use the command \" adobe acrobat pro \" and click on the name of the file in \" add the first \" drop down menu. This will open a pop-up asking the for a pdf, click get started, and the username of your pdf in the top right corner on the pop-up.\nC. [step] At the top left of the \" combine files \" dialog box, click \" add files \" and choose the office documents to include. [substeps] In the thumbnail view, you can directly drag and drop files and pages into the desired order.\nD. [step] Click \" create \" to create a new pdf by dragging and dropping your pdf. [step] Add also pages, columns, charts, languages, etc.","prompt":"[header] How to convert and combine office documents to one pdf [title] Open adobe acrobat pro. [step] Find the \" create \" function and click it to drop download the list, and you'll see \" combine files into a single pdf \". [title] Add office documents.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4364,"error":null,"output":"C"}
{"question_id":"hellaswag:00002184","item_index":283,"attempt":0,"prompt_hash":"bae70632b567c11f53d480f9949ba5e441a92c1a4aa5068846f82acfbfa46068","question":"A little girl sits on front a cup of ice cream, then she takes a piece of food from the cup and eats it. then\n\nA. , the girl takes a large stick and holds the stick to the edge of a pumpkin, then place the stick on top of the board and then play ice cream.\nB. , the girl finishes the ice cream and drinking the cup of ice cream she threw to the ground.\nC. , the little girl goes outside and eat apples close to the table, then she grabs nail tools and starts clipping the nails of a baby sitting.\nD. , the girl grab the spoon and eats the ice cream.","prompt":"A little girl sits on front a cup of ice cream, then she takes a piece of food from the cup and eats it. then","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5711,"error":null,"output":"D"}
{"question_id":"hellaswag:00006929","item_index":284,"attempt":0,"prompt_hash":"ce0f0cc948d237b1845826887bd99e89435ef7ecfbe00870dac9c7803e40124f","question":"[header] How to delete videos on an iphone [title] Open the videos app. [step] Tap its application (movie clapboard) icon from your iphone's home screen to open the app and show all the video files currently saved on your device. [title] Choose the video you want to delete.\n\nA. [step] Tap the video to select the option to permanently delete it (or save it). You can choose anywhere from 8-20 video files, or you can delete more video files at once.\nB. [step] Swipe across the screen to scroll along the list of videos of look for the files you want to remove from your iphone. [title] Swipe the video you want to delete to the left.\nC. [step] Depending on your phone model, you may be able to delete lots of video at once or a couple videos at once. Tap the video you want to delete, then tap \" delete \".\nD. [step] Tap on the one you want to delete by tapping on it. [substeps] Alternatively, tap the four-colored dots in the upper-right corner of your screen and tap).","prompt":"[header] How to delete videos on an iphone [title] Open the videos app. [step] Tap its application (movie clapboard) icon from your iphone's home screen to open the app and show all the video files currently saved on your device. [title] Choose the video you want to delete.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5710,"error":null,"output":"B"}
{"question_id":"hellaswag:00001691","item_index":285,"attempt":0,"prompt_hash":"f846cbab77b4c898d8cb44a33215f9670b9c61a12f73bdc5e470e2b2ef11f2d7","question":"Graphics with the \"isport com instructional video lessons\" appears on screen. young men\n\nA. play basketball in an indoor court and \" how to do a spin move in basketball \" overlays the video.\nB. talk on lacrosse field while several players speak on link to a different video.\nC. compete soccer soccer on a soccer field competition.\nD. are seen taking into a tutorial on how to do \" boots \" and how to make them comfortable on hard dirt while several people cheer.","prompt":"Graphics with the \"isport com instructional video lessons\" appears on screen. young men","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5709,"error":null,"output":"A"}
{"question_id":"hellaswag:00005659","item_index":286,"attempt":0,"prompt_hash":"a3d8707dd1cdf0fe6f83cfd048e87ab549c099cee2bf4da143eb762d570e83a4","question":"[header] How to get a good indoor tan [title] Understand how indoor tanning beds work. [step] The natural sun emits 95 percent uv-a and 5 percent uv-b rays at noon during the summer months. Most indoor tanning beds also emit 95 percent uv-a and 5 percent uv-b rays, similar to the exposure of the summer sun.\n\nA. [substeps] Comprehend how the indoor tanning equipment tans your skin. The skin's epidermis, or top layer, contains melanocytes, which are cells that produce melanin when stimulated with ultraviolet light.\nB. [substeps] A good indoor tanning bed should be made of porous, soft material, like nylon or fabric. Corn can absorb uv rays, which will speed up the shedding of some of your skin, when it dries off.\nC. Remember that you will first use these sunscreens throughout the day. [substeps] These sunscreens work better if you only wear them when you are swimming or tanning.\nD. When it comes to outdoor tanning beds, temperature extremes are more focused on the natural upkeep of your house than the location of your home. [substeps] Indoor tanning beds start at 95 percent humidity and cook for about 36 hours.","prompt":"[header] How to get a good indoor tan [title] Understand how indoor tanning beds work. [step] The natural sun emits 95 percent uv-a and 5 percent uv-b rays at noon during the summer months. Most indoor tanning beds also emit 95 percent uv-a and 5 percent uv-b rays, similar to the exposure of the summer sun.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5711,"error":null,"output":"A"}
{"question_id":"hellaswag:00003046","item_index":287,"attempt":0,"prompt_hash":"0b497a7ddc0d3327242ddfc7e4704fbec0f05e33c43bd6e056a51acdd390ad27","question":"A young man runs towards a pole in the air and hurls himself over it in a high jump. the man\n\nA. is seated at the side of the pole.\nB. lands on the other side of the bar onto a gym mat where he rolls and then gets up and walks.\nC. recovers after landing and his face is set in marble statue demeanor.\nD. doesn't raise his arms for a couple of minutes.","prompt":"A young man runs towards a pole in the air and hurls himself over it in a high jump. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5709,"error":null,"output":"B"}
{"question_id":"hellaswag:00001342","item_index":288,"attempt":0,"prompt_hash":"9ce71093b86485010c7bf8dd8afd33e21e85acc314f14237e81f70ce5ebcb1bc","question":"A man in a beekeepers outfit walks around a field of colorful boxes. he\n\nA. sits down in front of each box.\nB. picks up a lawn cutter and places the strap attached to his lawn cutter around his body.\nC. lassoes an individual with a black straw around the neck.\nD. opens one of the boxes and dumps out kites inside.","prompt":"A man in a beekeepers outfit walks around a field of colorful boxes. he","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5505,"error":null,"output":"A"}
{"question_id":"hellaswag:00008589","item_index":289,"attempt":0,"prompt_hash":"cce7a54604abaa1501a1e42c97484895300863fc5a32bbee7bdfd65dbdeb69fd","question":"[header] How to kill black ants [title] Buy commercial ant bait. [step] Using a commercial bait is a very effective way to kill black ants. Ants are attracted to the bait, which is a mixture of poison and a sweet substance.\n\nA. If you find the bait stuck in the ants' hole, you can try to move the bait closer to the hole. Mosquitoes are repelled by the bait, so they'll leave you alone.\nB. The ants eat some bait and bring some back to the nest, where the rest of the colony is also exposed to the poison. Over the course of one or two life cycles, the ants will be destroyed.\nC. They will flock to the bait and sting, so when the bait is used, the ants will swarm and destroy the bait. [substeps] Unlike commercial bait, wasp bait is designed for you to use.\nD. At a minimum, there are two types of bait that ants naturally use to hunt their prey. The one type is cyanide-which is a pesticide that attracts black ants.","prompt":"[header] How to kill black ants [title] Buy commercial ant bait. [step] Using a commercial bait is a very effective way to kill black ants. Ants are attracted to the bait, which is a mixture of poison and a sweet substance.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5498,"error":null,"output":"B"}
{"question_id":"hellaswag:00003609","item_index":290,"attempt":0,"prompt_hash":"04a7078f96c1e87f0425194d62ffaec50618819c4ebe6bf8ebb729089d823370","question":"[header] How to train cavalier king charles spaniels [title] Start training right away. [step] Don't wait to house train your cavalier. You should begin toilet training him as soon as you bring him home.\n\nA. [substeps] Always start training young before you adopt him. This will help your dog adapts to training with you so that it begins to understand to \" sit \" if he wants to use the bathroom.\nB. Start by first spraying him with a fresh coat of dog shampoo as soon as you bring your cavalier home. The shampoos and lotions will hopefully help get his fur and skin clean every time you groom him.\nC. Take him to the spot where you'd like him to toilet and set him down on the floor or ground. [substeps] Choose a toileting spot before you bring your dog home and be consistent.\nD. He will start by being potty trained and may respond better to the need to urinate. [substeps] If you know that he hates to go to the bathroom, have a hideaway house where he does not have to go to the bathroom.","prompt":"[header] How to train cavalier king charles spaniels [title] Start training right away. [step] Don't wait to house train your cavalier. You should begin toilet training him as soon as you bring him home.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5498,"error":null,"output":"C"}
{"question_id":"hellaswag:00000181","item_index":291,"attempt":0,"prompt_hash":"dbe242734490df0d9e699783415c3bf9a654913bb52493bc6ceb2f8f9ed05100","question":"A group of people wearing swim caps move around a pool and toss a ball around with a man on the sides waving a flag. they\n\nA. play a game of water polo and throw the ball around to each other.\nB. dive backward into the pool causing it to crest and swim to the edge of the pool.\nC. continue to throw the object around while holding the hold the fish.\nD. continue playing with one another in the water and join stand by to watch the game.","prompt":"A group of people wearing swim caps move around a pool and toss a ball around with a man on the sides waving a flag. they","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5504,"error":null,"output":"A"}
{"question_id":"hellaswag:00007211","item_index":292,"attempt":0,"prompt_hash":"aedb8f43cf25675c2dcf841243fea51eaac85e9f66910147377f608361aeb596","question":"[header] How to look good in front of your crush [title] Make sure you have a friend or two close by. [step] Avoid one-on-one encounters with your crush until you've practiced getting your nerves under control. Hanging around in a group setting will take a lot of pressure off of you and make you feel a lot less self-conscious.\n\nA. [substeps] If your friend invites you over for coffee or to a movie, go with her. Many people like to have other opportunities to spend time with their friends.\nB. [title] Be ready for your meet-up! [step] Visit your closet to find inspiration. Find sweatpants, leggings, or any other loose fitting clothing that will make your legs twitch.\nC. Even if they don't know about your crush, your friends being there will lend the moral support you'll need to relax. [title] Try dressing up a bit if you think you may run into your crush.\nD. [substeps] If you do see him or her standing with your friends and trying to get your attention, move away from the two of them. If he or she is with someone else, simply ignore them.","prompt":"[header] How to look good in front of your crush [title] Make sure you have a friend or two close by. [step] Avoid one-on-one encounters with your crush until you've practiced getting your nerves under control. Hanging around in a group setting will take a lot of pressure off of you and make you feel a lot less self-conscious.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5499,"error":null,"output":"C"}
{"question_id":"hellaswag:00007673","item_index":293,"attempt":0,"prompt_hash":"b733f583d22049848fde48d96efdf750cc1726a75fb5710e6f47b2822412c8f0","question":"[header] How to attract abundance [title] Acknowledge the positive things in your life. [step] The prevalent philosophy on attracting abundance encourages that you focus on what you do have, rather than what is lacking in your life. For example, instead of focusing on not having reached your monetary goals, acknowledge the things you already possess, such as a roof over your head, food in your refrigerator and your good health.\n\nA. [substeps] By doing this, you can reduce the temptation for expecting big debts on your credit score. Instead, recognize all the good things in your life that you have instead of expecting similar things to grow.\nB. Take some time to think about all of the things you want in life and what you can see in it. Do not focus on the big picture aspects of your life.\nC. [title] Express gratitude for the blessings in your life. [step] Being thankful for what you already possess allows your energy to flow in the direction of creating and attracting more good things.\nD. Feeling positive and can also make you more popular at work, or the body in the workplace. [substeps] Make a list of all the good things about your life and reach for those.","prompt":"[header] How to attract abundance [title] Acknowledge the positive things in your life. [step] The prevalent philosophy on attracting abundance encourages that you focus on what you do have, rather than what is lacking in your life. For example, instead of focusing on not having reached your monetary goals, acknowledge the things you already possess, such as a roof over your head, food in your refrigerator and your good health.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5498,"error":null,"output":"C"}
{"question_id":"hellaswag:00006339","item_index":294,"attempt":0,"prompt_hash":"a62954d8a7ff940b12b12da9905dcec164d42c800c06bc1d0d132cd121b4f8d7","question":"[header] How to choose fall makeup colors [title] Ditch the bronzer. [step] Summer is all about airy, light coverage and a sun-kissed bronze. When fall arrives, it's time to swap out the shimmery tan for soft, creamy skin and rosy cheeks.\n\nA. Eyeliners are absolute no-no! If you wear full coverage, your eyes glaze over quite a bit during the day. Decide on the best shade for your complexion.\nB. Not only will your skin look rosy, the natural skin will appear plumper and softer. [substeps] Enjoy the fantastic summer look without feeling like you can't accessorize.\nC. For cooler shades like pink and orange, go with beige or camel for a smooth, youthful look. [title] Cleanse, apply, and rinse your face if necessary.\nD. You can bring out your full coverage foundations that you skipped in the hot summer months, and use a beauty blender to apply it smoothly and evenly. [substeps] Opt for lighter colored concealers and foundations that you used in the summer, so that your skin looks natural in the less sunny weather.","prompt":"[header] How to choose fall makeup colors [title] Ditch the bronzer. [step] Summer is all about airy, light coverage and a sun-kissed bronze. When fall arrives, it's time to swap out the shimmery tan for soft, creamy skin and rosy cheeks.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5498,"error":null,"output":"D"}
{"question_id":"hellaswag:00006364","item_index":295,"attempt":0,"prompt_hash":"3347f44cfaafc8ac15c5aef37e275180ef85e2bf49f5ca3ca17c02f0b40d33c1","question":"[header] How to teach kids to wash their hands [title] Place a stool in front of the sink. [step] If the child cannot reach the sink by themselves, place a stool in front of the sink to facilitate regular hand-washing practices [substeps] Look for a stool with rubber grips on the steps. When young children are involved, washing hands can get extremely messy, and you don't want the child slipping on the step.\n\nA. The stool should be in a position that allows the child more room to grip the sink. This allows them to sit in a more comfortable position after a long trip, and prevents accidents.\nB. You can lay an unused towel over the stool before you work so it can maintain a comfortable grip. [title] Point the child in the direction of the tip of their hand.\nC. You can also use a step stool or window seat for this purpose, especially during weekdays when accidents are less likely. [title] Place the child in a seated position.\nD. The feet of the step should also be equipped with rubber to limit the risk of slipping or sliding across a slick floor. [title] Talk about germs and how they live on hands and surfaces.","prompt":"[header] How to teach kids to wash their hands [title] Place a stool in front of the sink. [step] If the child cannot reach the sink by themselves, place a stool in front of the sink to facilitate regular hand-washing practices [substeps] Look for a stool with rubber grips on the steps. When young children are involved, washing hands can get extremely messy, and you don't want the child slipping on the step.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5497,"error":null,"output":"D"}
{"question_id":"hellaswag:00007387","item_index":296,"attempt":0,"prompt_hash":"149664282d0cb1076e0be2a435e5e56ca052a090648d7ee73438612c2fb10786","question":"[header] How to carry a toddler [title] Have your toddler stand in front of you. [step] It is harder to pick them up from the side of from the back, for sometimes they will squirm or slip out of reach. [title] Place your hands under their armpits.\n\nA. [title] Gently lift the toddler off of the ground. [title] Bring the child closer to you, and hoist them to your side.\nB. [step] The side of their body that has full access to your hands is their face. Make sure that they cannot get out of your hands.\nC. [step] When they start squirming and jumping around, they will jump on your hands and legs. [title] Check how big you can support their weight as you carry them.\nD. [step] Try not to force them to stand, just turn toward them and help them move towards their stroller. [title] Search the stroller for familiar plastic items and throw them in.","prompt":"[header] How to carry a toddler [title] Have your toddler stand in front of you. [step] It is harder to pick them up from the side of from the back, for sometimes they will squirm or slip out of reach. [title] Place your hands under their armpits.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5495,"error":null,"output":"A"}
{"question_id":"hellaswag:00004070","item_index":297,"attempt":0,"prompt_hash":"313be12cfbb9916efcc2766f45d0e954446cffebabfbcdeaf413fc67bdb1599b","question":"[header] How to apply nail wraps [title] Remove any nail polish. [step] It is important to remove any old nail polish before you apply your nail wraps. Use some nail polish remover to take off your old nail polish.\n\nA. If it remains, you risk bacteria causing nail polish from reusing it. [substeps] Rub a damp cotton ball with nail polish remover over a small area of your nail.\nB. Even if you do not have any nail polish on, you might want to swipe each of your nails with a little nail polish remover to ensure that they are totally clean. [substeps] You can also wash your hands and nails to ensure that they are free of grease and dirt.\nC. After you clean your nails, allow them to dry completely before applying your nail wraps. [substeps] Nail polish remover can sting if you do not drink water.\nD. This will remove the old polish remover onto your skin. [title] Place the nail wraps onto your skin.","prompt":"[header] How to apply nail wraps [title] Remove any nail polish. [step] It is important to remove any old nail polish before you apply your nail wraps. Use some nail polish remover to take off your old nail polish.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5502,"error":null,"output":"B"}
{"question_id":"hellaswag:00009732","item_index":298,"attempt":0,"prompt_hash":"bc628d879b70ff155e33f113a9245ff4a3f5fef6bde20fad320366849e58d838","question":"[header] How to apply pencil eyeliner [title] Consider what type of pencil eyeliner to use. [step] There are many different types of eyeliner that come in a pencil form, each of which has a different consistency and texture, application, and effect. [substeps] Powder-based pencils, sometimes called kohl, offer less intense color.\n\nA. Single tbsp pencils, called sable pencil, are an alternative choice for wide eyes. Laser pencils, such as od, focus more harsh, dark tones.\nB. They're also a good option if you want to smudge the eyeliner for a \" smoky eye \" effect. Gel-or cream-based pencils offer simple and excellent application.\nC. Stick with softer shades like red, cream, and champagne or sapphire to avoid blending many shades together. Red pencils have also been coated with synthetic pigments to create eye-catching colors.\nD. Pearl-based pencils (and powder pencil pencils for shades of black) provide a more intense, subtle shade. [title] Look for a pencil that's loose and form-fitting.","prompt":"[header] How to apply pencil eyeliner [title] Consider what type of pencil eyeliner to use. [step] There are many different types of eyeliner that come in a pencil form, each of which has a different consistency and texture, application, and effect. [substeps] Powder-based pencils, sometimes called kohl, offer less intense color.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5497,"error":null,"output":"B"}
{"question_id":"hellaswag:00004061","item_index":299,"attempt":0,"prompt_hash":"826a0bd22e53780b98e09c5e222511367b00f727f7c518c8f70c94fdbcfa55f4","question":"[header] How to use a fan brush [title] Create an airbrushed effect by fan brushing foundation. [step] Use a large fan brush to even out your powdered foundation, using the top of the brush to gently buff the foundation on your entire face. This will leave you with a streak-free complexion.\n\nA. [title] Contour your cheekbones using a large fan brush. [step] To contour your cheekbones, apply some bronzer to your brush and swipe the brush back and forth between the top of your ear and the edge of your mouth.\nB. It will also take some of the shine out of your foundation. [title] Angle your brush at a 45 degree angle.\nC. [title] Clean up blemishes with either large-bristled makeup brush or a small brush. [step] For added effect, rinse makeup brushes in clear water and use a brush with brush bristles to apply a light coat of foundation.\nD. To begin applying foundation, dab a light powder brush under foundation from the base of your cheekbones down past your nose. Use your fingers to brush foundation onto your temples, down your jaw line, along your forehead, across the bridge of your nose, and underneath your chin.","prompt":"[header] How to use a fan brush [title] Create an airbrushed effect by fan brushing foundation. [step] Use a large fan brush to even out your powdered foundation, using the top of the brush to gently buff the foundation on your entire face. This will leave you with a streak-free complexion.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5496,"error":null,"output":"A"}
{"question_id":"hellaswag:00009909","item_index":300,"attempt":0,"prompt_hash":"66580842f432e239f6caecf615bedd5352b60a75eddfae6d5b098b1311dc9504","question":"[header] How to apply for ssi for your child [title] Understand the eligibility criteria. [step] Ssi payments are for unmarried children under age 18 who have a qualifying disability. A qualifying disability is defined as having a physical or mental impairment (s) which results in marked and severe functional limitations and has or can be expected to last for a continuous period of at least 12 months or result in death.\n\nA. [title] Apply for ssi as required by your state. [step] Every state has a specific waiting period for ssi, regardless of whether you live in another state or are on a municipal or state government disability program.\nB. [substeps] The ssa publishes a listing of impairments for children. If the child's disability is on the list, then this is usually enough to establish that the child has a qualifying disability.\nC. When you apply for ssi, you must be an immigrant from a country or countries where you have resided for at least three years. Have a medical or psychological condition and present for the nhs or evaluator's evaluation.\nD. The requirements for ssi are similar to those described in the following sections. [title] Take the following steps to meet disability requirements.","prompt":"[header] How to apply for ssi for your child [title] Understand the eligibility criteria. [step] Ssi payments are for unmarried children under age 18 who have a qualifying disability. A qualifying disability is defined as having a physical or mental impairment (s) which results in marked and severe functional limitations and has or can be expected to last for a continuous period of at least 12 months or result in death.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5498,"error":null,"output":"B"}
{"question_id":"hellaswag:00009271","item_index":301,"attempt":0,"prompt_hash":"f1b9ab522a4c7aa6d2bbe728f52990688285e3b95cb39f80e24be704233d8dd2","question":"[header] How to care for the most delicate fabrics [title] Treat stains promptly. [step] Fresh stains are easier to remove than old ones. If the stain is on a no washable fabric, take it to the dry cleaner as soon as possible, and describe the nature of the stain and the fiber content of the garment.\n\nA. The dry cleaner will be able to determine if it's dry cleaning or if the stain is fresh by looking for spots on the fabric, such as a dirt mound or a knot in the thread. [title] Protect delicate fabrics from mildew by applying a stain remover.\nB. [title] Read and carefully follow package directions when using any stain removal product. [title] Always test stain removers on an inside seam or other hidden part of garment for color fastness.\nC. If it is especially difficult to remove the stain, or you suspect that stain could be directly related to the fabric, you can treat the stain with hydrogen peroxide, or spot treatment hydrogen peroxide. The hydrogen peroxide will kill germs and remove on-the-spot stains from the fabric.\nD. [substeps] Alternatively, you could take the garment to the dry cleaner and ask them for a recommendation. They should be able to recommend a good reputable dry cleaner in your area.","prompt":"[header] How to care for the most delicate fabrics [title] Treat stains promptly. [step] Fresh stains are easier to remove than old ones. If the stain is on a no washable fabric, take it to the dry cleaner as soon as possible, and describe the nature of the stain and the fiber content of the garment.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5498,"error":null,"output":"B"}
{"question_id":"hellaswag:00008481","item_index":302,"attempt":0,"prompt_hash":"5a38903c22e2f2e9f0b2a05cf6b5c7002e75333086e022e1d8d390872fc10b1e","question":"[header] How to write apocalyptic stories [title] Decide what kind of apocalyptic story it is : [step] Before the world ended, while the world ended, or after the world ended. [title] Decide why the world ended and how : [step] Plague, nuclear war, aliens contact, etc. [title] Decide how your character (s) survived the apocalypse (e.g. [step] Immune).\n\nA. [title] Choose what magic you want to use. [step] Does your character (s) have human characteristics or is he more (non-human)? What sort of to use? [title] Decide what genres (gothic, fantasy, goth, sci-fi, etc.\nB. If you don't, you'll leave major holes in your story. If you're writing a story placed before or during the apocalypse, you can use a character's survival method to your advantage by making it the story line.\nC. If you enjoy your fictional world a bit more intensely, and plan to repeat the story where it ended, then you're good to go. If you don't and can no longer enjoy it, then you're heading for serious trouble with apocalypse.\nD. [title] Send me a description of the ending and how it affected the reader's perception and thinking at the time. [title] Write a foreshadowing of what happens next.","prompt":"[header] How to write apocalyptic stories [title] Decide what kind of apocalyptic story it is : [step] Before the world ended, while the world ended, or after the world ended. [title] Decide why the world ended and how : [step] Plague, nuclear war, aliens contact, etc. [title] Decide how your character (s) survived the apocalypse (e.g. [step] Immune).","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5497,"error":null,"output":"B"}
{"question_id":"hellaswag:00000740","item_index":303,"attempt":0,"prompt_hash":"ecdcda32bd21d67d445c4f20d5e37a8111d36f22a75ab915c1c483aab364b406","question":"A man is seated on a couch, playing a harmonica. he\n\nA. lowers it from his mouth when he is done.\nB. is playing the acoustic guitar with his eyes closed.\nC. cuts his fingers and plays intensely as he plays on the harmonica.\nD. switches sides quickly, continuing to play the instrument.","prompt":"A man is seated on a couch, playing a harmonica. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5498,"error":null,"output":"A"}
{"question_id":"hellaswag:00009575","item_index":304,"attempt":0,"prompt_hash":"b5f68c5385400340c2591445582981108b20639b70aa5bf9368c23a651669e17","question":"[header] How to adopt a dog from overseas [title] Make sure you have adequate funds to adopt a dog overseas. [step] Adopting a dog overseas is not cheap. In addition to an adoption fee, there are costs like travel, veterinarian exams, and fines and fees for transporting a dog in or out of the country.\n\nA. On average, it can cost anywhere between $150 and $2 , 000 to adopt a dog overseas. Make sure you can get this money together before committing to the adoption process.\nB. To get this far, it's essential to feel financially prepared for the trip, then bring along the dog with you. [substeps] To stretch out your budget, visit an online adoption site or visit a pet shelter.\nC. If you have enough money to sustain yourself, your adult dog or pet will be much more cost-effective than the other animals living in your household. [substeps] Calculate your budget, then determine your rest of the expenses and how much you can provide to your pet.\nD. [substeps] Start your own dog carrier early on in your adoption journey so that you have enough money to provide extra for vaccinations and dog food after you arrive in the country. After you return home, investigate the shipping costs of your carrier.","prompt":"[header] How to adopt a dog from overseas [title] Make sure you have adequate funds to adopt a dog overseas. [step] Adopting a dog overseas is not cheap. In addition to an adoption fee, there are costs like travel, veterinarian exams, and fines and fees for transporting a dog in or out of the country.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5498,"error":null,"output":"A"}
{"question_id":"hellaswag:00000092","item_index":305,"attempt":0,"prompt_hash":"4edeb8371b184b221f0d097d9ed6de65e5f06d1d3170c89083c9265ada48bb14","question":"A pilates instructor on skates explains how to strap your risk protectors tight and shows that you need knee pads to protect your knees. the lady\n\nA. starts with two small straps and it puts her into a squatting position with feet fully placed over her knee and her hands suspended and ready.\nB. then explains the skates movement and how to use the breaks that is in the back of the skates.\nC. then puts lotion in both of her hands and rubs them along your knees while the instructor washes them.\nD. shows how to harness and strap your weight on properly and she moves away from the mounting as she reaches for her left knee.","prompt":"A pilates instructor on skates explains how to strap your risk protectors tight and shows that you need knee pads to protect your knees. the lady","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5500,"error":null,"output":"B"}
{"question_id":"hellaswag:00009395","item_index":306,"attempt":0,"prompt_hash":"b9bec789fb4cb14a3cf2a615d5f36a4f1be46f2d0a0abbf18e5e1900facd93fb","question":"[header] How to get a safe summer glow (works best on fair skin tones ) [title] Shave your legs in the shower. [step] Make sure to use a new razor blade and shaving cream for the closest shave. This will help the tanner stick to your legs.\n\nA. Leave them waxed and crunchy but don't go too drying, at any rate. You don't want to end up with suds on your legs.\nB. It will also help remove bumpy skin for a more even application. [title] Now exfoliate everywhere you will be applying self tanner, but spend a little extra time on elbows, knees, hands, and feet, which are the roughest areas.\nC. Shave your legs to get rid of excess grease. This will let you test the new razor on a small patch of skin.\nD. You could also leave your pubic hair unshaved for 1 to 2 weeks. This will give the tanner the chance to penetrate the hair and stimulate the follicles.","prompt":"[header] How to get a safe summer glow (works best on fair skin tones ) [title] Shave your legs in the shower. [step] Make sure to use a new razor blade and shaving cream for the closest shave. This will help the tanner stick to your legs.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5499,"error":null,"output":"B"}
{"question_id":"hellaswag:00009527","item_index":307,"attempt":0,"prompt_hash":"acec1980f727840fec9b95446ce98bc488bca1a23b2df3e66f4eb8c32efae6db","question":"[header] How to apply liquid metal eyeshadow [title] Get a liquid metal eyeshadow kit. [step] They are often labeled as \" metal foil. \" these kits contain a metallic, pressed powder eyeshadow and a clear, liquid primer.\n\nA. [substeps] Look for a kit in a color that complement your metal eyeshadow, such as maroon. If your eyeshadow is neutral/deep toned, like metallic, you can also opt for opaque eyeshadow kits.\nB. You can use these kits to mark your eyelids so you can make your eyeshadow sooner if you purchase one [title] Swipe the liquid metallic eyeshadow over your eyelid. [step] Tilt your head back and carefully swipe the powder eyeshadow over your eyelid.\nC. [title] Consider applying some basic eyeshadow first. [step] Liquid metal eyeshadow is best applied just on the lid or inner corners of your eye.\nD. Fill them up with your liquid ingredients, including the powder and primer, and secure with a clip before waiting. [substeps] A tissue box is a good place to pack liquid eyeshadow.","prompt":"[header] How to apply liquid metal eyeshadow [title] Get a liquid metal eyeshadow kit. [step] They are often labeled as \" metal foil. \" these kits contain a metallic, pressed powder eyeshadow and a clear, liquid primer.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5499,"error":null,"output":"C"}
{"question_id":"hellaswag:00002346","item_index":308,"attempt":0,"prompt_hash":"609fcbc3e27e8a162b8f819bab9a066ffab6b041924267a9d7e33993dc03639d","question":". \" a camel ride - desrt of doha - qatar \"\n\nA. appears on screen \" demonstrate how a camel rides using its front legs in the desert.\nB. appears on screen and it's a treadmill with kind words on it at the bottom of it.\nC. is shown on screen.\nD. appears on screen while people wash their hands in the sink in a center of a large bathroom bathroom with a sink.","prompt":". \" a camel ride - desrt of doha - qatar \"","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5499,"error":null,"output":"D"}
{"question_id":"hellaswag:00000083","item_index":309,"attempt":0,"prompt_hash":"90ab570a9f14b4315095cd1407de8c8ed09c8d9910170fceeacfb35e19ed941f","question":"As the young lady continues her performance, she twirls the baton between her legs and over her head, catches it and does a gallop. the rest of the young girl's performance\n\nA. is replayed in slow motion.\nB. is a set of twirls and twist and then she runs off the floor to finish her performance.\nC. is shown several times and ends by showcasing her back to the both the camera and a final focus of her performance.\nD. is replayed, ending with the young lady clapping her hands joyfully.","prompt":"As the young lady continues her performance, she twirls the baton between her legs and over her head, catches it and does a gallop. the rest of the young girl's performance","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5498,"error":null,"output":"A"}
{"question_id":"hellaswag:00009423","item_index":310,"attempt":0,"prompt_hash":"64820c2d4df39f7d7637e0d847e91a5a032dafa993e2598e1b81fbd143cb5955","question":"[header] How to make hair spray [title] Bring 1 cup (240 milliliters) of water to a simmer in a saucepan. [step] Try to use filtered or distilled water if you can. Ordinary tap water contains too many chemicals and minerals, which may cause buildup in your hair overtime.\n\nA. [substeps] For lighter hair, you could use a glass bottle with lip balm, mineral oil, or coconut oil. For medium or thick hair, heat up your water using a curling iron.\nB. The added sugar will increase the mineral content of the water, which will give your hair boost. [substeps] For extra aroma, add a pinch of salt to the water mixture.\nC. Heating the water will make the salt easier to dissolve. [title] Stir in 1 tablespoon of sea salt.\nD. [substeps] Adding lemon juice as a fragrance enhancer can help to calm the frizz in your hair as well. [title] Soak your hair in the water for 5 minutes.","prompt":"[header] How to make hair spray [title] Bring 1 cup (240 milliliters) of water to a simmer in a saucepan. [step] Try to use filtered or distilled water if you can. Ordinary tap water contains too many chemicals and minerals, which may cause buildup in your hair overtime.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6354,"error":null,"output":"C"}
{"question_id":"hellaswag:00009588","item_index":311,"attempt":0,"prompt_hash":"10243150adadba7c92105dde71f4a08ab92b673c7eab6313f3f8d0dd8422c80a","question":"[header] How to get commercial property insurance [title] Check your state's law. [step] The state in which your business is located may require specific minimum amounts of commercial property insurance. These requirements represent the minimum commercial property insurance you must purchase for your business.\n\nA. Check the state's law to get the maximum commercial property insurance coverage for your business. [substeps] Some states for example are kansas, illinois, maine, massachusetts, idaho, oklahoma, oregon, and tennessee.\nB. [substeps] If your business is located in someone else's state, you must purchase commercial property insurance for the business in which you live within the state. In illinois, for example, you must purchase commercial property insurance with the permission of your business's zoning division.\nC. [substeps] The article provided is a guide to making sure you meet the minimum required minimum required by your state. Generally, the amount of commercial property insurance you must purchase is $13 , 920.\nD. [substeps] You can contact your state's insurance commission to find out if there are any insurance requirements that apply to your business. You also might check with a local small business association or chamber of commerce.","prompt":"[header] How to get commercial property insurance [title] Check your state's law. [step] The state in which your business is located may require specific minimum amounts of commercial property insurance. These requirements represent the minimum commercial property insurance you must purchase for your business.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6354,"error":null,"output":"D"}
{"question_id":"hellaswag:00000876","item_index":312,"attempt":0,"prompt_hash":"5bd28fcca670d8ee0b6b126a7bb26e1b33fb7c70540d5b0580a94d8251542232","question":"A small group of people are seen swimming in a pool and leads into various clips of children learning how to swim. A man helps them practice in the water as well as jumping off a diving board. more shots\n\nA. are shown of kids swimming and leads into a girl grabbing objects under water and swimming with another.\nB. of kids play as well as holding up silver objects and getting worn down.\nC. are shown of the crowd cheering and showing the girls swimming successfully.\nD. are shown of kids swimming and lie down afterwards while the boy keeps in time with everyone inside the pool.","prompt":"A small group of people are seen swimming in a pool and leads into various clips of children learning how to swim. A man helps them practice in the water as well as jumping off a diving board. more shots","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6353,"error":null,"output":"C"}
{"question_id":"hellaswag:00000773","item_index":313,"attempt":0,"prompt_hash":"ce9ba941562616b5ed79beb24ebea342cfc1dc4027448e433fc3c31a8068795c","question":"A cowboy is riding on a horse in a large arena and is swinging his rope. the view\n\nA. turns to a view of a canyon with large rocks around.\nB. cuts away to him jumping off of the horse and pulling a woman off.\nC. changes to show a group of men with their horses and they're each holding onto their ropes and doing various things with them like tying and swinging them.\nD. changes to see the man fly over the arena.","prompt":"A cowboy is riding on a horse in a large arena and is swinging his rope. the view","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6352,"error":null,"output":"C"}
{"question_id":"hellaswag:00001722","item_index":314,"attempt":0,"prompt_hash":"1243fe03bcc62272823c245b3eeb57caf4a9da9ab65629826cf6546fe3ae0389","question":"A woman is standing in front of a truck in a driveway. she\n\nA. is then shown snowboarding down a hill.\nB. picks up several bricks from the ground.\nC. is using a large shovel to move snow.\nD. uses a lawn mower to sweep the snow off the front of the truck.","prompt":"A woman is standing in front of a truck in a driveway. she","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6351,"error":null,"output":"B"}
{"question_id":"hellaswag:00005349","item_index":315,"attempt":0,"prompt_hash":"9c7afe276e5fb9b89adb7e3a653348b64fd79cdd466d2306f44907f49fca43b4","question":"[header] How to retire in the republic of panama [title] Show financial ability to support yourself. [step] Prove that you have $1 , 000 for a single person (or $1 , 250.00 for a couple) of income per month. [substeps] This can be social security, retirement funds or other.\n\nA. Let this addition to your gross household determine your net income and housing expenses. However, do not only have you the money, but also realize how healthy you are overall.\nB. Make sure your savings account works with that of your main bank. There are usually some deductions for self-employment purposes, and they can change depending on your financial situation.\nC. However, act as a member and provide means to buy and control certain group as well as raise funds. To make bringing material people who aren't working together easier, demonstrate that you can promise both in staying in the rented panama enterprise facility program.\nD. You can also demonstrate that you are a person of means. This is defined as having $100 , 000.00 or more in funds and will meet the financial requirement.","prompt":"[header] How to retire in the republic of panama [title] Show financial ability to support yourself. [step] Prove that you have $1 , 000 for a single person (or $1 , 250.00 for a couple) of income per month. [substeps] This can be social security, retirement funds or other.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6161,"error":null,"output":"D"}
{"question_id":"hellaswag:00003112","item_index":316,"attempt":0,"prompt_hash":"09db3d3c40860e21f86f268c05d88680c9c6b65a405ad09376746a1ae3093171","question":"A man shows a bottle of cream, and then he puts gel on his hand. then\n\nA. , the man rub his face and neck with the cream.\nB. , the man inks the side of the mans hand with gel.\nC. , the man takes playing jars from a bag and rubs stage cream on his hand.\nD. , the man puts a chemical on top of the gel bottle.","prompt":"A man shows a bottle of cream, and then he puts gel on his hand. then","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6160,"error":null,"output":"A"}
{"question_id":"hellaswag:00006226","item_index":317,"attempt":0,"prompt_hash":"daee0d701c582756615fb638b1e0956750cca59595606e3215283ae59e92ff3f","question":"[header] How to repair an air mattress [title] Listen and look for the leak in the air mattress. [step] If you don't know where the hole is, keep the air mattress inflated as you look. Holes most often appear on the bottom or in the seams, so stand the mattress up against a wall to check.\n\nA. If the leak is noticeable, cover the pipe with something sturdy. You may wish to wear goggles and gloves to protect your eyes from the smoke.\nB. If it appears cloudy or filled with debris, it may have been hit by someone. [substeps] Check the right lever to ensure no air escapes.\nC. If you find one, fix the leak. [title] Replace the air mattress when you're finished.\nD. Listen for the sound of rushing air to detect tiny leaks. [title] Use soap and water if you can't find the leak.","prompt":"[header] How to repair an air mattress [title] Listen and look for the leak in the air mattress. [step] If you don't know where the hole is, keep the air mattress inflated as you look. Holes most often appear on the bottom or in the seams, so stand the mattress up against a wall to check.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6161,"error":null,"output":"D"}
{"question_id":"hellaswag:00008649","item_index":318,"attempt":0,"prompt_hash":"d6e6d06003af8718e64dddad64eea9defce03e44f70dbb33ef34eee1287c34ae","question":"[header] How to babyproof your home [title] Install baby-proof/child-proof products around your home. [step] Products include locks for cabinets, outlet covers, corner guards, and furniture straps. [title] Keep hazardous products away form children.\n\nA. [step] Store chemicals, glass, medicines, knives, forks, pots, pans, spoons, spices, and other sharp or hazardous objects away from the reach of children. Instead, put these items in locked cabinets or drawers.\nB. [step] Dangerous products include adhesives, varnish, and graphite shavings. Inspect your home and ask if there are hazards in plain sight before you entrust your child with it.\nC. [step] Make sure children are not exposed to potentially hazardous products, like smoke, fire, and poisonous bugs. [substeps] Clothing and clothing may contain pesticides, insect or disease-preventing agents, and homes with bugs in these items may be a breeding ground for insect mites.\nD. [step] Children are often startled by products with dangerous chemicals in them. [title] Keep your personal safety concerns in mind.","prompt":"[header] How to babyproof your home [title] Install baby-proof/child-proof products around your home. [step] Products include locks for cabinets, outlet covers, corner guards, and furniture straps. [title] Keep hazardous products away form children.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6160,"error":null,"output":"A"}
{"question_id":"hellaswag:00000708","item_index":319,"attempt":0,"prompt_hash":"e3c8558809e8fa43babe7ca4a0a9b56bb0c27b3ccce2b55379772286bb30bc52","question":"A toddler pulls a string from a hole in the ice while a man is kneels on the ice. then\n\nA. , the toddler covers the hole with snow.\nB. , the child slope down the ice to get to the hole.\nC. , the colorful snow man is seen standing on the ice.\nD. , a fish come out the hole.","prompt":"A toddler pulls a string from a hole in the ice while a man is kneels on the ice. then","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6159,"error":null,"output":"A"}
{"question_id":"hellaswag:00006034","item_index":320,"attempt":0,"prompt_hash":"4249fa689d320b5b05c63dd458112e3e02eae9f3548b5f75c36e1f341ceb9a33","question":"[header] How to train your cat to fetch a toy mouse [title] Pick the right sized toy mouse. [step] In general, cats like to fetch things that they can easily grab with their paws or put in their mouths. If you do not already have a toy mouse at home, you can purchase one at your local pet store.\n\nA. Cats will enjoy playing with smaller toys, like paper snowflakes. [substeps] To put the toy mouse into the right mouth, position your cat in your lap and place one paw on top of the other.\nB. Consider the size of your cat as you are deciding which toy mouse to purchase-a kitten would need a smaller toy mouse than an adult cat. [substeps] If possible, pick a toy mouse that does not have plastic eyes.\nC. These are small rubber balls with bones on them that, depending on the size of your mouse mouse, can be bigger than its normal size. It's important to make sure that the toy mouse is not too small for the mouse mouse.\nD. Most cats go for small, lightweight toys that you can put in their mouths. Make sure your toy mouse is thin enough so it is easy to pick up.","prompt":"[header] How to train your cat to fetch a toy mouse [title] Pick the right sized toy mouse. [step] In general, cats like to fetch things that they can easily grab with their paws or put in their mouths. If you do not already have a toy mouse at home, you can purchase one at your local pet store.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6231,"error":null,"output":"B"}
{"question_id":"hellaswag:00008737","item_index":321,"attempt":0,"prompt_hash":"442fc098cdac258b807dfe2736247398ebf36ebeb4310a80317cf56b9a038234","question":"[header] How to use talcum powder safely [title] Use it for male irritation. [step] Talcum powder appears to safe for males to use for genital sweating and chafing. It has not been linked to any cancers of male genitalia.\n\nA. However, it is a naturally occurring product. [substeps] Note that talcum powder can also be toxic if inhaled.\nB. Women and men still use talcum powder. [title] Note that talcum powder cannot be easily removed.\nC. To test for any sensitivity to the powder, rub talcum powder over your genitals and on your vagina. This can cause a reaction in the male and female partners.\nD. Talcum powder can keep you dry if you have issues with chafing or other friction irritation. [substeps] If you are a male using talcum powder on his genitals, do not use it before having sex with a woman.","prompt":"[header] How to use talcum powder safely [title] Use it for male irritation. [step] Talcum powder appears to safe for males to use for genital sweating and chafing. It has not been linked to any cancers of male genitalia.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6229,"error":null,"output":"D"}
{"question_id":"hellaswag:00004797","item_index":322,"attempt":0,"prompt_hash":"2fdbc3c4734167f8cd9102cf4537c076d0b77ca35054781b954968f6f4881389","question":"[header] How to make your hair look naturally curly [title] Avoid brushing out your freshly washed hair. [step] Brushing or combing out your hair after washing it will encourage your hair to look straighter. Instead, apply conditioner in the shower, then use a wide-toothed comb to detangle and distribute it.\n\nA. Doing this ensures the strands are thoroughly detangled and tangle free without adding additional breakage. [substeps] Make sure to use a wide-toothed comb to get away from your roots when brushing.\nB. [title] Cut your hair at the roots if your hair is shorter. [step] Split-ends can lead to damage to hair follicles, so try cutting your hair at the roots during your wash, when possible.\nC. Rinse as usual, and avoid brushing it out after that. [title] Towel-dry gently using the \" twist and press \" technique.\nD. For deep-conditioning spray, look for a conditioner that contains a built-in protein, such as a extract or mineral supplement. Follow the instructions on the spray bottle for accurate application.","prompt":"[header] How to make your hair look naturally curly [title] Avoid brushing out your freshly washed hair. [step] Brushing or combing out your hair after washing it will encourage your hair to look straighter. Instead, apply conditioner in the shower, then use a wide-toothed comb to detangle and distribute it.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6229,"error":null,"output":"C"}
{"question_id":"hellaswag:00007134","item_index":323,"attempt":0,"prompt_hash":"bb625e64adccac30d83e865abd24b73310098fa779a2a51488aaf2ebbc808d41","question":"[header] How to tan in the sun [title] Change into clothes that expose your skin to the sun. [step] The most common clothes to wear for tanning is a bathing suit, but you can wear shorts, tank tops, or nothing at all if you are somewhere amenable to nude sunbathing. Any exposed skin, left out under direct sunlight, will tan.\n\nA. No one needs to notice you in the bathing suit. [title] Choose clothes that suit your particular complexion.\nB. [title] Tan before 10 am or after 4 pm for the safest results. [step] This avoids the scorching peak hours and is recommended by the american cancer society for the safest tanning.\nC. [substeps] If you must wear a bathing suit, make sure it is as tight as possible around the stomach, or consider a sports bra or tank top over top. Anything with long sleeves will work, however.\nD. [title] Change into a light cotton sports bra or tank top. [step] A lightweight, plain bra can cover up your skin and instead catch the sun's rays, while making it harder for the sun to penetrate the clothing underneath.","prompt":"[header] How to tan in the sun [title] Change into clothes that expose your skin to the sun. [step] The most common clothes to wear for tanning is a bathing suit, but you can wear shorts, tank tops, or nothing at all if you are somewhere amenable to nude sunbathing. Any exposed skin, left out under direct sunlight, will tan.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6230,"error":null,"output":"B"}
{"question_id":"hellaswag:00007420","item_index":324,"attempt":0,"prompt_hash":"2f8ddeaf8ce1453ea40c90a36f30716bdfb8fce5e410855cdb13c5087e2d2bcf","question":"[header] How to care for a leather jacket [title] Make your leather water resistant. [step] There are many \" leather protector \" or \" leather waterproofing \" products, but read the label carefully before buying. A silicone polymer spray, or an acrylic copolymer spray, should preserve your leather's appearance and shine.\n\nA. This will help prevent eczema from forming or off spotting. The oil will improve the pores, and reduce scars from absorbing oil.\nB. Grease-based or wax-based products are more protective, but not recommended for jackets due to their potential effect on color, longevity, shine, and smell. Check the label on your product to find out how long the water resistance lasts.\nC. After you apply your oil, rinse your leather with lukewarm water. If you use a spray bottle, the bottom of the bottle must be clean, and there must also be no running water when you spray your leather when you apply the oil.\nD. [title] Wrap your jacket in a towel to squeeze the excess water out. [step] Secure the towel with a rubber band and apply pressure as you remove the towel.","prompt":"[header] How to care for a leather jacket [title] Make your leather water resistant. [step] There are many \" leather protector \" or \" leather waterproofing \" products, but read the label carefully before buying. A silicone polymer spray, or an acrylic copolymer spray, should preserve your leather's appearance and shine.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6228,"error":null,"output":"B"}
{"question_id":"hellaswag:00003731","item_index":325,"attempt":0,"prompt_hash":"63b0a1ac8c083908cf9b26827ba4daf390231158f74d9e6360947b109d43d345","question":"[header] How to hook up roku [title] Plug the hdmi cord into the tv. [step] Doing so will give you audio/visual once the device is turned on. [substeps] The hdmi cord will be about 3 or 6 feet long and has 2 plugins on each end that are pentagon shaped.\n\nA. You can typically find your hdmi input behind the tv on its right side, and some tvs will have more than one input. Older tv's will have a component or composite connection instead of hdmi.\nB. Here are some general plugins you'll need to know : [substeps] Vapor vacuum-this is the best choice. It's very simple-this takes a steady stream of air coming out and turning into a sound, indicating that there are two tones coming out.\nC. You will plug the wires into a socket until you are ready to use the hdmi cord. [title] Connect the cables to the hdmi cables.\nD. For anything greater then 3.5k, attach them together with another-smaller hdmi cord and connect the two to the back of the tv. [title] Plug in the other hdmi cord.","prompt":"[header] How to hook up roku [title] Plug the hdmi cord into the tv. [step] Doing so will give you audio/visual once the device is turned on. [substeps] The hdmi cord will be about 3 or 6 feet long and has 2 plugins on each end that are pentagon shaped.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6229,"error":null,"output":"A"}
{"question_id":"hellaswag:00004960","item_index":326,"attempt":0,"prompt_hash":"c9c5b5781cc39aa5c86dc62a2679026a6df2c73de6ad2d63efafa61e81617e14","question":"[header] How to play big bear's den (ukrainian snow game ) [title] Draw a large square on a flat area of snow. [step] Use a large stick or the end of a shovel. See note under \" tips \" for square size needs.\n\nA. [title] Mark all four cubes at the points where they meet. [step] Place vertically one cube on each of the matching squares.\nB. [title] Draw another square inside the large square. [step] The inner square is the big bear's den.\nC. [title] Next, draw a long, thin notch. [step] Don't make it to the tip; you will have to guide them at the bottom of the square so as to leave room for an entrance.\nD. It can serve as a circle, while large square may be included as well. [title] Draw a large circle and two circles for each animal and make a line between them.","prompt":"[header] How to play big bear's den (ukrainian snow game ) [title] Draw a large square on a flat area of snow. [step] Use a large stick or the end of a shovel. See note under \" tips \" for square size needs.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6229,"error":null,"output":"B"}
{"question_id":"hellaswag:00000442","item_index":327,"attempt":0,"prompt_hash":"c39427061af5fe7cf22cdb5d2669ae0b7d31bf96f523cdc6c292328677f1479a","question":"A kid takes mouthwash and swishes it in his mouth. he\n\nA. spits it out followed by another kid.\nB. spits it out onto his hand.\nC. spits it out and then starts crying.\nD. puts it back and puts it down.","prompt":"A kid takes mouthwash and swishes it in his mouth. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6234,"error":null,"output":"C"}
{"question_id":"hellaswag:00002920","item_index":328,"attempt":0,"prompt_hash":"c7c346b13333bc2ad3431084419b421c9bb8a5d5250b2740489036e2487ff63b","question":"A man introduces the basketball concepts of the video to the camera. the player\n\nA. starts from one side of the court and runs up and lays up the basketball in the net.\nB. tries to make several balls, but only makes very high number of balls.\nC. , then sits on the end of a pallet and explains that he is demonstrating make boxing moves.\nD. drops land, bounces, and lands on the green basketball court.","prompt":"A man introduces the basketball concepts of the video to the camera. the player","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6232,"error":null,"output":"A"}
{"question_id":"hellaswag:00009071","item_index":329,"attempt":0,"prompt_hash":"7919b27e93b665d652ac5626aba35f5570548020e5bb9af1df43a50470751bd9","question":"[header] How to lift objects when pregnant [title] Decide whether you should lift the object in question based on its weight. [step] The amount of weight that is generally safe to lift is connected to your stage of pregnancy. The more advanced your pregnancy, the more you should avoid lifting heavy objects.\n\nA. The frequency of lifting also matters, with occasional lifting being less risky than everyday actions. [substeps] Up until the 24th week, it is generally okay to lift more than 51 lbs.\nB. [substeps] Children under six months of age are permitted to lift heavier objects. Young women (between six and 14 months) are not allowed to lift heavy objects.\nC. [substeps] If you are pregnant, you should avoid lifting heavy objects such as heavy purses, books, furniture and toys. Keep in mind that carrying heavy items as a pregnant woman could throw off your chances of growing and having a baby.\nD. Your weight will fluctuate as you advance, so ask your doctor for advice. [substeps] A pregnant woman should lift virtually any object once it has reached your optimal weight so that it can be used during pregnancy.","prompt":"[header] How to lift objects when pregnant [title] Decide whether you should lift the object in question based on its weight. [step] The amount of weight that is generally safe to lift is connected to your stage of pregnancy. The more advanced your pregnancy, the more you should avoid lifting heavy objects.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6228,"error":null,"output":"B"}
{"question_id":"hellaswag:00002733","item_index":330,"attempt":0,"prompt_hash":"7eb51735f933b4fce511f9ebc02c0d43fef4bebb9bab57f524c2f2c5fac0d64e","question":"He finally finds the dog who is driving the riding lawn mower. According to the owner, the dog is asking for a raise. he\n\nA. continues mowing in not very straight lines as the other dogs watch.\nB. walks back to his lawn mower and starts to mow.\nC. explains, and then let the owner poke holes holes in his lawn for the dog to go.\nD. is raking the leaves in the wet lawn.","prompt":"He finally finds the dog who is driving the riding lawn mower. According to the owner, the dog is asking for a raise. he","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6230,"error":null,"output":"B"}
{"question_id":"hellaswag:00007164","item_index":331,"attempt":0,"prompt_hash":"4cf627a6c977f7e086eab86c9244b7c1a28494e41e1dcc371d8cc51a39c07d1d","question":"[header] How to get a new york beer or liquor license [title] Decide whether you need a wholesale or a retail license. [step] The first decision you need to make is the overall type of license that you require. There are two basic categories for liquor licenses in new york.\n\nA. These categories break down as follows : [substeps] Wholesale. A wholesale license gives you the power to manufacture, warehouse and distribute alcohol.\nB. Both require local licenses, but you may still need a retail license. [substeps] Retail licenses require a licensed shop and a retail store license, but retail licenses require a retail store.\nC. Wholesale licenses allow parties to buy liquor but wholesalers can also take on retail licenses. Retail licenses require the driving vehicle department to drive under the road.\nD. [substeps] The retail or retail license has to be a commercial form of the state license issued on the premises of the establishment of the liquor store. The retail or retail license is a commercial form of the license issued on the premises of the establishment of the establishment of the establishment.","prompt":"[header] How to get a new york beer or liquor license [title] Decide whether you need a wholesale or a retail license. [step] The first decision you need to make is the overall type of license that you require. There are two basic categories for liquor licenses in new york.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6228,"error":null,"output":"A"}
{"question_id":"hellaswag:00006589","item_index":332,"attempt":0,"prompt_hash":"c31b8facd80a83a80c5af039b10ed7002b9e45e49c94d8297e3f95520cdf6ef5","question":"[header] How to stop your throat from hurting [title] Stay hydrated. [step] Drink a lot of water and other liquids designed to replenish nutrients in your body. Sports drinks are normally great options.\n\nA. They rehydrate you fast and come in great flavors. [substeps] Drinking a lot of liquids can also help wash away thin layers of mucus or other allergens that may have coated your throat, resulting in inflammation and discomfort.\nB. However, make sure to stay hydrated and avoid sugary drinks, which make you tongue-tied. [substeps] Adding sweetened beverages can actually help to make your throat feel more manageable and less scratchy.\nC. Some examples of sports drinks are : [substeps] Water sports drinks. Sports drinks offered by sports teams coffee and tea sports drinks.\nD. Carbonated beverages such as seltzer and gatorade, for instance, are also excellent options because they're high in electrolytes (a sugar your body needs to produce). This will help keep your throat from getting sore.","prompt":"[header] How to stop your throat from hurting [title] Stay hydrated. [step] Drink a lot of water and other liquids designed to replenish nutrients in your body. Sports drinks are normally great options.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6227,"error":null,"output":"A"}
{"question_id":"hellaswag:00000962","item_index":333,"attempt":0,"prompt_hash":"fe2cfede3f84e88d4cf9342ee4fe43e045b9788038a6ba902cb4cd37504b98fb","question":"He picks up a large weight and lifts it over his head. He drops the weight onto the ground. a man\n\nA. kneels next to him water boarding.\nB. rushes up and throws the weight onto the field.\nC. collapses on a mat.\nD. in a blue shirt claps.","prompt":"He picks up a large weight and lifts it over his head. He drops the weight onto the ground. a man","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6230,"error":null,"output":"B"}
{"question_id":"hellaswag:00009735","item_index":334,"attempt":0,"prompt_hash":"ef9e2040d660fa197bec68df54ea3e4485718a4af554abb5f635aac52651a1fa","question":"[header] How to adopt in australia [title] Determine your eligibility. [step] Many people in australia hope to adopt a child born in the country. The process begins with determining whether you are eligible.\n\nA. The process depends on the country and the circumstances surrounding the child's arrival. [substeps] For example, if your child is a refugee from an alien country, you must be at least 18 years of age when you adopt the child.\nB. If you are declared mentally ill, you may be legally adopted. Additionally, some countries do not permit mothers who are closely related, such as ireland's family and the united kingdom, where the status quo was often determined by a prior age.\nC. If your condition requires either, then this is a personal decision you will want to make. [substeps] In order to adopt a child in australia, you need a passport or a birth certificate.\nD. [substeps] You must be married or have been in a stable relationship for two years. You must be able to provide evidence of this.","prompt":"[header] How to adopt in australia [title] Determine your eligibility. [step] Many people in australia hope to adopt a child born in the country. The process begins with determining whether you are eligible.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6227,"error":null,"output":"D"}
{"question_id":"hellaswag:00004518","item_index":335,"attempt":0,"prompt_hash":"6c7512baff007e31c9897d54002f9c3ea725b74db03e0735587699abc2fa4293","question":"[header] How to become a jag lawyer [title] Consider your career goals. [step] The judge advocate general (jag) corps of the five military branches is as competitive and demanding as any military career. Along with completing the educational and licensing requirements of the legal profession, you must also be able to meet the same standards as any prospective officer.\n\nA. [title] Attend a \" beverage therapy/training \" (psa) program. [step] The psa program provides a summer course related to cases in \" state court, \" which may be a career choice you are pursuing later in life.\nB. Your education must be high priority for you as an jag lawyer. [substeps] To qualify for public service benefits (cep), you must be at least 21 years old or older.\nC. [substeps] Applicants must be a citizen of the united states. Jag candidates must be able to pass a security clearance.\nD. [substeps] The jag enforces the following ideals: \" a responsible, capable worker, a good monarch, and a good citizen. \" [title] Graduate from a rigorous job search program.","prompt":"[header] How to become a jag lawyer [title] Consider your career goals. [step] The judge advocate general (jag) corps of the five military branches is as competitive and demanding as any military career. Along with completing the educational and licensing requirements of the legal profession, you must also be able to meet the same standards as any prospective officer.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6229,"error":null,"output":"C"}
{"question_id":"hellaswag:00008442","item_index":336,"attempt":0,"prompt_hash":"c49875f5893d382cba5c374b9b2a22a4ad983cb7fb05b1e0c8b7884e94f537eb","question":"[header] How to epilate legs [title] Choose a good quality epilator. [step] The quality of your epilator matters. Pick a well-known, well-reviewed brand that you know you can trust.\n\nA. The best brands that can reach you are probably revlon, trulia, and smith. [substeps] If possible, get an epilator that is ovulatory (even if there is a lower quality one like making ice cream with it).\nB. You can find them in both consumer and automotive stores, and you should be able to find one at no charge. [substeps] A quality epilator should be made for comfortable handling and cleaning, and should not be expensive.\nC. Do some research on the manufacturer and the types of epins used. Whether it's a branded epilator, or used worsted epilators, you'll want to start by reading reviews online.\nD. A cheap epilator may not work as well and it may be more painful to use. Since your epilator will last a long time (and you won't have to buy wax and fancy razors) it's worth springing for a good one.","prompt":"[header] How to epilate legs [title] Choose a good quality epilator. [step] The quality of your epilator matters. Pick a well-known, well-reviewed brand that you know you can trust.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6228,"error":null,"output":"D"}
{"question_id":"hellaswag:00005755","item_index":337,"attempt":0,"prompt_hash":"d9456f2ffa5f0c505bac51aff113e0b57f8ab2b3ed351c69aa75655dcc97de25","question":"[header] How to paint your nails with the opposite hand [title] To start, if you are using colour, apply a good base coat. [step] Keep in mind that dark polishes stand out, as does red and metallic polishes show up mistakes easily. Solid colours and chunky glitter are forgiving.\n\nA. [title] Get your nails wet and take a small amount of the base coat off your nails. [step] Apply an even amount to the bottom edge of your nails and apply in one straight line, use a clear topcoat or just add a little and let dry.\nB. [title] Using gloves, mix the base coat. [step] Apply base coat in 1/8 inch margins all around the nail, but blend it into the smooth polish.\nC. [title] Determine your colours : [step] Apply a base layer that is thick and shiny that shows up easily. Use a roller (such as glossy, dark or satin) to do this.\nD. It is recommended that you use a clear nail strengthening varnish, for a couple of reasons : [substeps] People aren't going to notice the mistakes you'll likely have made. It'll serve as a protective coat and help keep your nails strong as you practice polishing regularly.","prompt":"[header] How to paint your nails with the opposite hand [title] To start, if you are using colour, apply a good base coat. [step] Keep in mind that dark polishes stand out, as does red and metallic polishes show up mistakes easily. Solid colours and chunky glitter are forgiving.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6228,"error":null,"output":"D"}
{"question_id":"hellaswag:00004234","item_index":338,"attempt":0,"prompt_hash":"96006c8e02893474949002918e871489892b6811663e1878903af436c159fb7f","question":"[header] How to roast tomatillos [title] Place the tomatillos in a heavy pan. [step] Arrange the tomatillos in a single layer in a large, heavy frying pan. [substeps] Do not stack the tomatillos in multiple layers.\n\nA. Hold them between your thumb and forefinger, whether they're stacked or not. [title] Cook the tomatillos over high heat for 2 to 6 minutes.\nB. A cast iron frying pan will give you the best results, but if you do not have a cast iron pan, any heavy pan should work well enough. This method draws out the \" earthy \" flavor undertones of the fruit.\nC. You can pile the tomatillos on top each other. Do not leave too much of a gap between the tomatillos and the pan.\nD. You only want to roast the tomatillos according to your taste and the season. If you are not sure how large you want them, just err on the side of caution and move them all to different pans.","prompt":"[header] How to roast tomatillos [title] Place the tomatillos in a heavy pan. [step] Arrange the tomatillos in a single layer in a large, heavy frying pan. [substeps] Do not stack the tomatillos in multiple layers.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6227,"error":null,"output":"C"}
{"question_id":"hellaswag:00003651","item_index":339,"attempt":0,"prompt_hash":"9c19efcb3d317af40cda9971987d89b0da238d0b773e1aea038103d3cd21ef0d","question":"[header] How to dress well for a week [title] Day 1 : [step] Wear an off the shoulder t-shirt with flowy patterns with denim shorts. Wear a brown braided belt and wedge heels. Wear your hair down and wavy, if possible.\n\nA. [title] Day 2 : [step] Wear a sparkly tank top with skinny jeans. Accessorize with a nice, practical bag with a bit of pattern on it.\nB. Keep in mind, you can always make your hair super wavy if you want it longer. Wear it down in a low ponytail, high ponytails, or pigtails, if you want.\nC. They keep you looking effortlessly popular! Day 2: consider a fitted jacket with loose fit of shoulder strap with a scoop neck. Wear a black or deep purple dress with skinny jeans and a studded belt.\nD. Pull your hair back into a loose ponytail or knot it up in a, at the nape of your neck; leave it up the whole day. Again, create a similar look by layering blocks of natural brown hair around you for this \" morning tousled look.","prompt":"[header] How to dress well for a week [title] Day 1 : [step] Wear an off the shoulder t-shirt with flowy patterns with denim shorts. Wear a brown braided belt and wedge heels. Wear your hair down and wavy, if possible.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6229,"error":null,"output":"A"}
{"question_id":"hellaswag:00007170","item_index":340,"attempt":0,"prompt_hash":"c3810287d7db3522ecf34e74c7af43c31542f89beeb45daf42222b8d94697f1d","question":"[header] How to be curious [title] Read widely. [step] Choosing something you are already interested in makes it easy to start becoming more curious. Reading is one of the best ways to widen your knowledge base.\n\nA. There are books and articles written on every conceivable topic, from politics, to cooking, to gardening. Think about your interests, and choose one to start learning about.\nB. [substeps] If you are not already old enough to read the human food market, read blogs, books, and magazine articles. You can decide which articles are good for you.\nC. Learn as much as you can about math, science, and technology. If you have a lot of free time, you can learn things like physics, world history, and geography! [title] Enjoy the learning.\nD. You can check out blogs, magazines, newsletters, and even online opinions to increase your curiosity. Find a way to build learning skills, however, and you may discover that curiosity isn't all about the world.","prompt":"[header] How to be curious [title] Read widely. [step] Choosing something you are already interested in makes it easy to start becoming more curious. Reading is one of the best ways to widen your knowledge base.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6229,"error":null,"output":"A"}
{"question_id":"hellaswag:00002968","item_index":341,"attempt":0,"prompt_hash":"a0871150dafef7895a9339a30bba78894f91341e5643a68a2d8d73b888c31233","question":"A person is seen riding around on a dirt bike that leads into him speaking to the camera. He shows off his bike as well as camera and leads into several clips of him riding around. he\n\nA. shows off his workout and pans around on the bike and his take on how to use it.\nB. begins to ride around and show people he is skateboarding.\nC. continues riding around then walks away.\nD. continues to ride around on the track while also showing close ups of himself as well as the camera.","prompt":"A person is seen riding around on a dirt bike that leads into him speaking to the camera. He shows off his bike as well as camera and leads into several clips of him riding around. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6228,"error":null,"output":"D"}
{"question_id":"hellaswag:00008845","item_index":342,"attempt":0,"prompt_hash":"33f5c4cef2d9ee882a389a951f44c36828b1f4a046415212df74b363263dea2b","question":"[header] How to check backlinks to any website with moonsearch tool [title] Enter \" moonsearch.com \" into the address bar of your web browser. [title] Type the address of the website you want to check into the input field (we have typed example.com). [step] Hit the \" search \" button.\n\nA. Now you can see how with this search field of options you can be sure the website is on. But chances are, it isn't.\nB. [title] In the \" summary \" tab you can see valuable information on the website's alexa's rank, moonrank, page rank, server's ip address, location, etc. [title] In the \" backlink \" tab you will see the list of linking domains to the website.\nC. If you've disabled it, you'll be prompted to enter your domain code first. [title] Scroll up to the website you just want to check.\nD. You will be redirected to the website's main page. [title] Enter the search terms and a period into the text fields for the website's return address or url.","prompt":"[header] How to check backlinks to any website with moonsearch tool [title] Enter \" moonsearch.com \" into the address bar of your web browser. [title] Type the address of the website you want to check into the input field (we have typed example.com). [step] Hit the \" search \" button.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6909,"error":null,"output":"B"}
{"question_id":"hellaswag:00002463","item_index":343,"attempt":0,"prompt_hash":"cabd8b73a2daff6e5d15a11c98906311ecac7e6030fbb019ee2fd6fe6410c520","question":"An intro leads into a basketball player holding a ball and then running up and shooting a basket. the man\n\nA. continues running back and fourth while shooting the baskets and ends with text across the screen.\nB. continues throwing the bobble ball pattern with the ball while a timer starts to go off in the back.\nC. then gets rolls to the ground during a break for a ball to roll back and forth over the field.\nD. then runs down the court and throws the ball between two parallel bars, and again enters the game.","prompt":"An intro leads into a basketball player holding a ball and then running up and shooting a basket. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6909,"error":null,"output":"A"}
{"question_id":"hellaswag:00002964","item_index":344,"attempt":0,"prompt_hash":"368736b3c2e4902da2a8c5650143800f65a9824b033d11fd8ca1864de325ed9f","question":"The kleva sharpener is dismounted from the table and stored in a cabinet drawer. The kleva sharpener is mounted on the refrigerator on a wall and on the table. he\n\nA. demonstrates the locking mechanism.\nB. clips the wire securely together the your eyesight and then plugs the gun in, the lady tucks the new gun into a harness with a black cloth.\nC. moves very quickly explaining and showing how to put the alcohol and beer ingredients into a mixer.\nD. is food stepping on other shoes in a clear manner and pulls them off his feet.","prompt":"The kleva sharpener is dismounted from the table and stored in a cabinet drawer. The kleva sharpener is mounted on the refrigerator on a wall and on the table. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6910,"error":null,"output":"A"}
{"question_id":"hellaswag:00009228","item_index":345,"attempt":0,"prompt_hash":"927aa8c770df1230963cadfcb2f018f35ed407f2e632af209e86e95a0da372e9","question":"[header] How to live apart and stay married [title] Take advantage of technology if you're in a long-distance marriage. [step] You or your spouse might have to live in another state or country for work, school, or another obligation. If living apart isn't a choice, schedule regular video and phone chats to keep in touch.\n\nA. [substeps] There are websites that host face-to-face conversations in languages both can use to communicate often. There are also paid chat services that can help couples talk without having to share information or use video with their dependent family members.\nB. Text messages and video chats help even out the long distance relationships. [title] Set up your own social media accounts.\nC. [substeps] Skype or email is a great solution. You can even change your phone number as well.\nD. [substeps] Additionally, try sending each other texts or emails here and there such as, \" hey babe, hope you're having a great day! Love you. \" technology can also help you keep your sex life alive.","prompt":"[header] How to live apart and stay married [title] Take advantage of technology if you're in a long-distance marriage. [step] You or your spouse might have to live in another state or country for work, school, or another obligation. If living apart isn't a choice, schedule regular video and phone chats to keep in touch.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6909,"error":null,"output":"B"}
{"question_id":"hellaswag:00001830","item_index":346,"attempt":0,"prompt_hash":"cf4e4c4daefb7c818c98e8c620bbffbd6933769867a5c7c35cdfa0fe35686fb5","question":"A person throws a frisbee and the dog runs to get it. The man lays down and put his feet up, and the dog jumps on the feet. then\n\nA. , the man throws the frisbee and the dog runs to get it.\nB. , people run to bag the frisbee, and the people are standing behind the man talking.\nC. , the man puts his feet down and a dog jumped on his feet.\nD. , the man the man toss the frisbee and the dog runs to fetch it.","prompt":"A person throws a frisbee and the dog runs to get it. The man lays down and put his feet up, and the dog jumps on the feet. then","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6908,"error":null,"output":"A"}
{"question_id":"hellaswag:00000176","item_index":347,"attempt":0,"prompt_hash":"b5f3d763ad40ba89872e2b1fd6121677cd2389883df3ebdbb1a46e20cc703d03","question":"There's a woman with red colored hair wearing a black tank top and printed black tights skating on roller blades through the streets of a city. the streets\n\nA. showed various advertisements advertised for skate skating.\nB. are covered hip hop and dressed in purple and red leotards as the individual rhythmically spins around.\nC. are filled with crowds, people and cars, and she is wearing a white and orange outfit.\nD. don't have any cars but people are riding bikes or skateboarding.","prompt":"There's a woman with red colored hair wearing a black tank top and printed black tights skating on roller blades through the streets of a city. the streets","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7027,"error":null,"output":"D"}
{"question_id":"hellaswag:00002489","item_index":348,"attempt":0,"prompt_hash":"4d04d817a4884b08bd12df123274d913629adfc5c872abf2c5c510d06a249a71","question":"A few people are seen standing around on a basketball court. The group then begins shooting baskets and passing to one another. a man\n\nA. claps on the side and the group continues practicing.\nB. throws the ball in the air with his feet.\nC. scores a basket and walks back to the game where he is playing.\nD. returns to play court with the team.","prompt":"A few people are seen standing around on a basketball court. The group then begins shooting baskets and passing to one another. a man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7027,"error":null,"output":"A"}
{"question_id":"hellaswag:00007872","item_index":349,"attempt":0,"prompt_hash":"d7d31e21af52e5a67e9830ac275d999806bed9fdadc523e05059afc54bc98301","question":"[header] How to hide a double chin [title] Opt for a low or broad neckline. [step] High necklines frame your neck and face, and as a result, they also tend to draw more attention to a double chin. Direct the eye away from your chin by opting for low v-necks and similar collar styles.\n\nA. The idea is to keep the collar of your shirt as far away from your chin as possible. [substeps] If you're wearing a button down shirt, leave the top two or three buttons undone.\nB. [substeps] Remember that this technique is subtle, so wearing it so high doesn't attract too much attention. If you are pale or medium-in-color, the tilted chin look may be avoided entirely, although a more dramatic, industrial-cut neckline can be attractive.\nC. [substeps] Some accentuate a double chin, while others draw attention to it. If the occasion calls for high necklines for your grandmother's embroidery, choose low necklines to draw interest from.\nD. [substeps] Lower necklines may attract more attention than a v-neck. Worry about appearing modest; this can make you stand out from the crowd, too.","prompt":"[header] How to hide a double chin [title] Opt for a low or broad neckline. [step] High necklines frame your neck and face, and as a result, they also tend to draw more attention to a double chin. Direct the eye away from your chin by opting for low v-necks and similar collar styles.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6971,"error":null,"output":"A"}
{"question_id":"hellaswag:00007947","item_index":350,"attempt":0,"prompt_hash":"6def98826db99801aafc59553c37e5e11304e900931a6c2897d1256cb74059ba","question":"[header] How to do ghana cornrows [title] Apply a hydrating cream to your hair. [step] Focus on the roots, where your hair is most likely to turn dry. It would be best if you use a cream specially formulated for curling hair or braiding cornrows.\n\nA. [substeps] If you have very oily hair, you may want to use an oil like almond or safflower oil instead. These oils will help to hydrate your hair, not dry it out.\nB. [title] Place a chunk of your hair between your pointer fingers. [step] Set your hair off to one side.\nC. [substeps] Choose a natural cream, such as a coconut curling cream. [title] Decide how many cornrows you want.\nD. A moisturizing type cream will prevent your hair from drying out and make your hair more secure when it's dry. [substeps] You can also apply coconut oil in a spray bottle to your hair before you set up your starrows.","prompt":"[header] How to do ghana cornrows [title] Apply a hydrating cream to your hair. [step] Focus on the roots, where your hair is most likely to turn dry. It would be best if you use a cream specially formulated for curling hair or braiding cornrows.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6971,"error":null,"output":"A"}
{"question_id":"hellaswag:00009380","item_index":351,"attempt":0,"prompt_hash":"504a39854e9375e8ee744c4b1245b9cb393dfdc369be84542340b1fd062a977b","question":"[header] How to make magic mouthwash [title] Talk to your doctor. [step] Radiation and chemotherapy treatments for cancer often cause a side effect called oral mucositis. This is a painful and sometimes debilitating side effect, which causes soreness, swelling, and sensitivity in the mouth, sores on the mouth, gums, or tongue, and difficulty swallowing or talking.\n\nA. If you take oral medication, talk to your doctor before taking it. [substeps] Chemotherapy is also commonly used to treat oral cancer.\nB. It can make it hard or even impossible to eat. [substeps] Ask your doctor whether a magic mouthwash could help to relieve some of your discomfort.\nC. [substeps] May result in a variety of side effects. Consider going to your dentist's office to have your mouth checked.\nD. Ask your doctor to prescribe an opioid that will help to prevent this side effect. [substeps] The typical dosage for mouth mucosa treatments is four to six tablets per six hours in 15 minutes.","prompt":"[header] How to make magic mouthwash [title] Talk to your doctor. [step] Radiation and chemotherapy treatments for cancer often cause a side effect called oral mucositis. This is a painful and sometimes debilitating side effect, which causes soreness, swelling, and sensitivity in the mouth, sores on the mouth, gums, or tongue, and difficulty swallowing or talking.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6971,"error":null,"output":"B"}
{"question_id":"hellaswag:00003397","item_index":352,"attempt":0,"prompt_hash":"62395e282dbe9cffd2ba18d6aa97049f823cf00e5c2b77562ae4a5b423391a06","question":"[header] How to erase crow's feet [title] Moisturize daily. [step] Fine lines and wrinkles appear as you age because your skin becomes drier. If you use a moisturizer every day, you help trap water in your skin.\n\nA. Wearing a thick moisturizer that is designed to keep your skin moist can block the lines and wrinkles that shape crow's feet, which are uneven lines. Make sure your moisturizer is waterproof, also, to remove excess moisture from your skin when you add a darker product.\nB. Reapply moisturizer after you shower or before bed. [substeps] Another good option is to lay a wet cloth over your face the night before and then switch when you wake up.\nC. Once your skin feels as light and fresh as it should and you have plenty of time to rejuvenate it, the lines will start to disappear. [substeps] Use a light moisturizer that contains a broad range of nutrients.\nD. This can help keep the skin around your eyes hydrated and supple, which can minimize the appearance of crow's feet. [substeps] Look for a moisturizer that contains plant oils, shea butter, ceramides, or glycerin.","prompt":"[header] How to erase crow's feet [title] Moisturize daily. [step] Fine lines and wrinkles appear as you age because your skin becomes drier. If you use a moisturizer every day, you help trap water in your skin.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5388,"error":null,"output":"D"}
{"question_id":"hellaswag:00008072","item_index":353,"attempt":0,"prompt_hash":"2fd5fe5a24a1f7a7868e1a9648abd7afc09cdec0a500a69df2e4b652b7096368","question":"[header] How to maintain ash brown hair [title] Use products made for color-treated hair. [step] Using the wrong product is a leading factor behind fading hair dye. Because regular shampoos and conditioners contain ingredients that may cause hair dye to fade, you should use shampoos and conditioners made for color-treated hair instead.\n\nA. [substeps] For hair with a light-brown color, avoid using light-color hair products. Dark-colored hair tends to degrade faster than light-brown hair.\nB. [title] Request a color-safe shampoo. [step] You can also condition your hair with lightening shampoo or a name brand shampoo that contains lightening body oils such as moroccan oil.\nC. [substeps] Products labeled \" color-treated \" are more expensive than \" hair-treated \" when it comes to color and texture. Read the ingredient ingredient list of your hair product.\nD. [substeps] Avoid clarifying shampoos, as they tend to wash out hair dye. Avoid anything that contains sulfates, as they can also remove hair dye.","prompt":"[header] How to maintain ash brown hair [title] Use products made for color-treated hair. [step] Using the wrong product is a leading factor behind fading hair dye. Because regular shampoos and conditioners contain ingredients that may cause hair dye to fade, you should use shampoos and conditioners made for color-treated hair instead.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5388,"error":null,"output":"D"}
{"question_id":"hellaswag:00008150","item_index":354,"attempt":0,"prompt_hash":"80dc4ad6826690fbd72f7781a08d66365287c38f99cdc10ad1ae48c1e2318c70","question":"[header] How to get shiny hair [title] Use an egg. [step] It may sound strange, but an egg can do wonders for your hair. The yolk nourishes your hair by adding moisture so that it doesn't look dull.\n\nA. Some people find that egg cures their hair faster than egg yolks. [substeps] Egg yolks contain vitamins a, e.\nB. Just like eggs, eggs also seal up your hair, allowing you to have the best hair possible. It also helps to keep your hair dry so that it is already shiny.\nC. Just about any egg will do, as long as you keep your hair healthy and soft to start with. [title] Use bed noodles (or oatmeal).\nD. The white clarifies your hair by helping to remove any buildup. The egg also has protein in it, which strengthens your hair.","prompt":"[header] How to get shiny hair [title] Use an egg. [step] It may sound strange, but an egg can do wonders for your hair. The yolk nourishes your hair by adding moisture so that it doesn't look dull.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5393,"error":null,"output":"D"}
{"question_id":"hellaswag:00009832","item_index":355,"attempt":0,"prompt_hash":"3fd42d82c80abdcdcb966c2672d73f1523283305ba4defa1e10c9004c91cad17","question":"[header] How to pack a diaper bag for a plane trip [title] Pack a few of your child's favorite toys. [step] Bring age-appropriate items like chew toys, cars, doll, or stuffed animal. Try not to bring things with many pieces, so there's less to keep track of.\n\nA. [title] Pack a first aid kit so you are prepared for any injury. [step] Make sure your kit includes bandages, gauze, antibacterial ointment, cotton swabs, tissues, alcohol wipes, thermometer, and hand sanitizer.\nB. Be sure that the toys are first-line or that your child has enough of each type. [substeps] Buying early is also recommended because it saves you a lot of time and effort on buying too many toys than you're likely to have at some point.\nC. Keep the variety of toys you pack in mind, either in a suitcase or buying. A suitcase will save a lot of unneeded space on the plane and enough room for the litter box, shampoo, and first aid kit.\nD. Prioritize what you'll need each time, so that you're quick about bringing everything your child needs the most. [substeps] These things are usually things that you don't want your child to forget in case an emergency arises.","prompt":"[header] How to pack a diaper bag for a plane trip [title] Pack a few of your child's favorite toys. [step] Bring age-appropriate items like chew toys, cars, doll, or stuffed animal. Try not to bring things with many pieces, so there's less to keep track of.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5389,"error":null,"output":"A"}
{"question_id":"hellaswag:00006829","item_index":356,"attempt":0,"prompt_hash":"73a1a43a37006113c8b0f42ed2199e81c9d9317c467c85e85a165b4a4491a369","question":"[header] How to save a marriage after infidelity [title] Be accountable for your actions. [step] If you were the unfaithful party, it's important that you take responsibility for your actions and end the affair. You should stop all communication with the person you were unfaithful with and prevent any future interactions with them.\n\nA. [substeps] This principle is easy to change if there is no prenuptial agreement. [title] Do not accept the chance to have children with your spouse.\nB. Remember to always explain the actions you took, and keep your partner safe. [substeps] For example, if your partner cheated on you with someone else, you could say, \" anna, i'm sorry.\nC. If your spouse was unfaithful, you should confirm that they have ended the affair and intend to avoid all contact with they were unfaithful with. [substeps] Avoid deciding right away if you are both going to stay in the marriage.\nD. [substeps] Keep in mind that once you end the affair, you can't go back, and the married party must know that, too. You may not have been especially open to communication after a cheating affair.","prompt":"[header] How to save a marriage after infidelity [title] Be accountable for your actions. [step] If you were the unfaithful party, it's important that you take responsibility for your actions and end the affair. You should stop all communication with the person you were unfaithful with and prevent any future interactions with them.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5388,"error":null,"output":"C"}
{"question_id":"hellaswag:00003374","item_index":357,"attempt":0,"prompt_hash":"965651ed5bb6d525d7578a9d9734b407753ffe836f264d7da9abcee4f4e21e66","question":"[header] How to prepare your dog for a road trip [title] Visit your veterinarian. [step] Perhaps the first step in preparing your dog for a road trip is to make sure that he or she is healthy enough to take the trip. This should be no problem if you regularly take your dog to the vet for checkups and other issues.\n\nA. If you have not : [substeps] Make an appointment with the vet. Talk to your vet to make sure your dog is up to date on important vaccinations.\nB. Your veterinarian will be able to provide you with vaccinations, in addition to taking your dog in for professional vet visits. [substeps] Although canine infection is rare, your veterinarian may be able to provide vaccine for canine infectious diseases, like measles.\nC. Mention that your dog needs to follow a certain food regimen and behaviors during the trip. [substeps] Do not try to \" test \" your dog's blood for any diseases.\nD. Make sure your veterinarian shows that your dog is up to date on his or her vaccinations. [substeps] Be sure to bring your dog's current medication and acceptable lab coat.","prompt":"[header] How to prepare your dog for a road trip [title] Visit your veterinarian. [step] Perhaps the first step in preparing your dog for a road trip is to make sure that he or she is healthy enough to take the trip. This should be no problem if you regularly take your dog to the vet for checkups and other issues.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5390,"error":null,"output":"A"}
{"question_id":"hellaswag:00007664","item_index":358,"attempt":0,"prompt_hash":"d78b8daa7617aefcc162a9e52c691e176e4831983ba5f7009e19a262a8dcb279","question":"[header] How to choose versatile bridesmaid dresses [title] Remember not every person will look good in the same type of dress. [step] It's difficult to pick one dress that will fit multiple body types. You may find one that fits several people well and not others.\n\nA. When assessing bridesmaid dresses, think about how fashion designers like kate brown, kate peoples, angelina jolie, ella calliato, and october smith all have attractive bodies. [substeps] Don't pick one dress that doesn't shift quite as much as others.\nB. But look for one that is flattering to most body types, regardless. [title] Choose a size 1 do not overdress.\nC. Keep that in mind as you look for a universally \" good \" dress. [substeps] For instance, a dress with a well-defined waistline looks good on a person with an hourglass figure, while a pear-shaped person looks better in a dress with a fitted top and a slightly fuller skirt.\nD. The important thing is that the dress is versatile enough to stand out on the runway. [substeps] Focus your search on women around you and women at different stores.","prompt":"[header] How to choose versatile bridesmaid dresses [title] Remember not every person will look good in the same type of dress. [step] It's difficult to pick one dress that will fit multiple body types. You may find one that fits several people well and not others.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5390,"error":null,"output":"C"}
{"question_id":"hellaswag:00006743","item_index":359,"attempt":0,"prompt_hash":"662151a466367b6d72fbe5c2b50b1d9239c7ddaa1ad917cc15ee5f87784564a1","question":"[header] How to apply eyeliner around the entire eye [title] Wash your face thoroughly to ensure that you don't transfer any dirt or grime that may be on your hands onto your face. [title] Sharpen the eyeliner pencil to get rid of any bacteria that may be present on the pencil. [step] Do not make the pencil too sharp or pointy.\n\nA. [title] Put the eyeliner into your eyes. [step] Start at the crease where the eyelid meets the pupil.\nB. After sharpening it you can move on to the next step. [title] Make sure that your skin is still well hydrated from the bath or shower.\nC. It should have a flat point on the end, similar to the point on a ballpoint pen. [substeps] If your point is too sharp, you can easily soften the point with a match (a lit one).\nD. [title] Use liquid eyeliner that comes in a tube. [step] This will serve both to outline the line on your eyelid.","prompt":"[header] How to apply eyeliner around the entire eye [title] Wash your face thoroughly to ensure that you don't transfer any dirt or grime that may be on your hands onto your face. [title] Sharpen the eyeliner pencil to get rid of any bacteria that may be present on the pencil. [step] Do not make the pencil too sharp or pointy.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5388,"error":null,"output":"A"}
{"question_id":"hellaswag:00005357","item_index":360,"attempt":0,"prompt_hash":"cbe92a77d9b36e7db13096c0e645acac46c79b1b6fed017418dbc6c5998fe435","question":"[header] How to label clothes for daycare [title] Stay organized by creating a system for labeling your child's clothes. [step] When you begin, pull out every item of clothing that your child will take to daycare, including gloves, boots, and coats in the winter, and sunglasses, swimsuits, and shorts in the summertime. [title] Label everything at once.\n\nA. [step] This will help keep track of what has been done. As your child grows and more clothing is purchased, label each item as it joins the closet.\nB. [step] If you have a particularly large group of kids, label every outfit individually. Always be sure to keep a notebook with all important information about each child's clothing choices.\nC. [step] Hold all items labels one box, and label each item by shape. Put \" diapers \" and \" potty \" in the lettered end.\nD. [step] Label all items in your child's closet, backpack, or backpack for the day. [substeps] Use labels like \" contains presets, \" \" has preset care, \" and \" wants \" to label the materials in the clothing.","prompt":"[header] How to label clothes for daycare [title] Stay organized by creating a system for labeling your child's clothes. [step] When you begin, pull out every item of clothing that your child will take to daycare, including gloves, boots, and coats in the winter, and sunglasses, swimsuits, and shorts in the summertime. [title] Label everything at once.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5388,"error":null,"output":"A"}
{"question_id":"hellaswag:00001474","item_index":361,"attempt":0,"prompt_hash":"87f127730cba5d7498e06b1cc6115cead6e1dcff949f007074f81e9c96789717","question":"A boy in blue shirt hit the drum, but the big drum keeps moving, then it fell. A boy with earphones is drumming and the stick got stuck in the instrument. the band\n\nA. stopped drumming and the tall boy named cuanspwe was smiling and tapping his left foot as the drummer continue to drum.\nB. is playing, while the man is singing, the drummer fell off the his chair.\nC. stopped playing, one of the boys put the earphones on the drums.\nD. pass the road, then cars are passing by the hill.","prompt":"A boy in blue shirt hit the drum, but the big drum keeps moving, then it fell. A boy with earphones is drumming and the stick got stuck in the instrument. the band","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5390,"error":null,"output":"C"}
{"question_id":"hellaswag:00002178","item_index":362,"attempt":0,"prompt_hash":"e49f7024ffe376ed6cf37e72eb3582eeb0786d2e6f8054cc4fe1174424453545","question":"A man in a grey track suit performs jump rope tricks in front of a group of students. the man in the grey track suit\n\nA. finishes and the children clap.\nB. performs jump rope tricks indoors on a wooden stage with the audience watching in support.\nC. successfully jumps free from jump rope to comes back to the rope and repeats the jump rope tricks in reverse while demonstrate multiple jumps with different people walking onto the rope.\nD. spins on the rope with close ups of his feet as he performs flips the rope.","prompt":"A man in a grey track suit performs jump rope tricks in front of a group of students. the man in the grey track suit","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5389,"error":null,"output":"B"}
{"question_id":"hellaswag:00006376","item_index":363,"attempt":0,"prompt_hash":"8f978461511bbf8036f698b3aadbfcaf2ac9515947a425ed24345a2f1734a9a6","question":"[header] How to make it through valentine's day without a valentine [title] Make the day about self-care. [step] Do things that make you feel good and relaxed. Focus on activities that soothe you and put your mind at ease.\n\nA. [substeps] For example, some things you can do to make valentine's day better include, reading books, baking something, taking a short break, watching family movies, and listening to music. Try to immerse yourself in activities that make you happy or feel good and make valentine's day a family event.\nB. The most important love you'll ever have is the love of self. Be kind to yourself in mind, body, and spirit.\nC. [substeps] For example, say, you enjoy walking, playing your favorite sport, and taking care of a pet. Make sure to do these activities daily to prevent stress and help you think deeply on the subject.\nD. [substeps] Turn off all electronics, shut down anything you think you'll be doing in the near future, allow yourself to sleep. This might mean getting up and going for a walk or drinking some water instead.","prompt":"[header] How to make it through valentine's day without a valentine [title] Make the day about self-care. [step] Do things that make you feel good and relaxed. Focus on activities that soothe you and put your mind at ease.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5387,"error":null,"output":"B"}
{"question_id":"hellaswag:00004492","item_index":364,"attempt":0,"prompt_hash":"5ee258e3b3e75c61c4be01ed5073e25bdc02a12f6a4357b475ba25e1ad07dfcb","question":"[header] How to get a toddler to take antibiotics [title] Ask the doctor for an antibiotic that is taken less frequently. [step] This includes giving medicines that are taken once or twice a day as opposed to four times a day. There are different brands of antibiotics that act in a similar fashion, but some of them have to be taken more frequently than others.\n\nA. Always check with the doctor before giving antibiotics to ensure you are not over-taking a certain antibiotic. [substeps] There are different brands of antibiotics that act in a similar fashion, but some of them have to be taken more frequently than others.\nB. For example, the antibiotic cture' nsaids' is commonly taken only once a day for one to two days. [title] Deal with improperly fed or undernourished toddlers.\nC. [substeps] Others, including famciclovir (paroxetine), ramax (macular), and zoloft (hellinganide) can be taken eight times a day. Not all help your toddler with symptoms.\nD. [substeps] Talk to your doctor about the situation if your child won't take her medication, and see if there is another pill that your child can switch to. [title] Request an antibiotic with a more appealing taste.","prompt":"[header] How to get a toddler to take antibiotics [title] Ask the doctor for an antibiotic that is taken less frequently. [step] This includes giving medicines that are taken once or twice a day as opposed to four times a day. There are different brands of antibiotics that act in a similar fashion, but some of them have to be taken more frequently than others.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5387,"error":null,"output":"A"}
{"question_id":"hellaswag:00009482","item_index":365,"attempt":0,"prompt_hash":"2e5e584a6a5675d315a3c09d27f9a5096bae704bd4a6df9363f6d02f5c55c18e","question":"[header] How to safely clean a diamond ring [title] Make a soap and water mixture. [step] Squirt a little dish soap into a bowl. Fill the bowl with warm water.\n\nA. Use approximately one part soap and six parts water. [substeps] You can begin pouring water into the bowl at any time of day.\nB. Swish it around a bit to produce some suds. [substeps] Use a gentle dish soap, preferably one made with natural ingredients, to avoid harming your ring with chemicals.\nC. Put your ring in the water and let it soak for as long as you like , 5 minutes. This is the soap solution for cleaning a diamond.\nD. Then, stir in a few drops of dish soap. [title] Dip your ring into the soapy solution.","prompt":"[header] How to safely clean a diamond ring [title] Make a soap and water mixture. [step] Squirt a little dish soap into a bowl. Fill the bowl with warm water.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5389,"error":null,"output":"D"}
{"question_id":"hellaswag:00007652","item_index":366,"attempt":0,"prompt_hash":"16c1af6b7eb86b23d57e046368a5cdd95f64065bd3c2909c671e7b5892594aa8","question":"[header] How to care for tan leather shoes [title] Wipe down your leather shoes with a soft cloth after each wear. [step] Run the cloth over the surface of your shoes each time you take them off for the day. Doing this regularly will prevent dirt and grime from building up on your shoes.\n\nA. [title] Remove any debris from your shoes after they've been in the sun. [step] Sand your shoes carefully after wearing them.\nB. [title] Polish your leather shoes once a month. [step] A regular polishing routine will keep your shoes looking new.\nC. On the other hand, if you have a lot of leather, dirt and grime can build up on the fabric. [substeps] Gently wipe up any dirt that has built up on your shoes each day with a soft cloth, such as a washcloth, towel, or soft cotton towel.\nD. [title] Apply a soft solution of a shoe stripper oil to the soles of your shoes. [step] Have the mixture slowly soak for at least 20 to 30 minutes, then rinse and dry with a clean towel.","prompt":"[header] How to care for tan leather shoes [title] Wipe down your leather shoes with a soft cloth after each wear. [step] Run the cloth over the surface of your shoes each time you take them off for the day. Doing this regularly will prevent dirt and grime from building up on your shoes.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5387,"error":null,"output":"B"}
{"question_id":"hellaswag:00008074","item_index":367,"attempt":0,"prompt_hash":"b81e9767778f53ce49bdc2440a6386fe2135ae138ed60c2bda580827e18baee6","question":"[header] How to grow thick curly hair [title] Deep condition your hair. [step] Moisturizing your hair is extremely important for curly hair. Curly hair has more difficulty getting moisture to the ends because of the shape.\n\nA. When you take the time to deep condition your hair, your hair can get much drier. In fact, hair can actually become drier at the roots by drying quickly or overheating.\nB. Removing excess moisture by conditioning your hair once a week can help with this while making your thick hair thicker. [substeps] Water your hair with a clarifying shampoo once a week to lock in moisture.\nC. This can cause dryness and dandruff. Moisturizing your hair helps prevent this.\nD. Your hair will usually need to be hydrated. [substeps] When choosing shampoo, store-bought brands may be too harsh on curly hair.","prompt":"[header] How to grow thick curly hair [title] Deep condition your hair. [step] Moisturizing your hair is extremely important for curly hair. Curly hair has more difficulty getting moisture to the ends because of the shape.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5389,"error":null,"output":"C"}
{"question_id":"hellaswag:00008556","item_index":368,"attempt":0,"prompt_hash":"3ff1868d94ec013499413b48cfce89b4c2787db5511f0ac40e1972b814aab5ea","question":"[header] How to manage student behaviour when relief teaching [title] Anticipate compliance. [step] Generally most students want to behave and very few really want the tension of getting into trouble. So if you give instructions, anticipate that the student will comply and act as if this is the case.\n\nA. Even more, stay true to your own beliefs and values. In a mess with rules, expectations or general public opinion, the teacher in charge of classroom behaviour can very well become a jerk! When they improve just enough, students learn more.\nB. [substeps] Assume that the student will follow what is told. You should like to cross say protocol, and show that incompliance is a constituent of behaviour.\nC. [substeps] Try to counteract this by breaking things up. For example, if a student gives up an activity they promised they would finish doing in one week, and their new responsibilities are met in week one, cut the work that comes along.\nD. For example, in your travels around the room you notice gina is reading a book and not doing her math. You take the least intrusive path, \" thanks for putting that book away, gina.","prompt":"[header] How to manage student behaviour when relief teaching [title] Anticipate compliance. [step] Generally most students want to behave and very few really want the tension of getting into trouble. So if you give instructions, anticipate that the student will comply and act as if this is the case.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5387,"error":null,"output":"D"}
{"question_id":"hellaswag:00001543","item_index":369,"attempt":0,"prompt_hash":"ff211ae97c49db75d53788b2d70f9c4f3682b87b6f298a76f909652fb827b1d3","question":"A man is seen holding a blow torch and begins using it while pulling his mask down. the man\n\nA. continues to blow the camera in various ways and give close ups to the camera when he is done.\nB. continues to put on his mask while blowing up his mask.\nC. continuously sparks with the machine on a piece of metal.\nD. turns off the torch and uses it again with his face up and moves down his body.","prompt":"A man is seen holding a blow torch and begins using it while pulling his mask down. the man","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5387,"error":null,"output":"B"}
{"question_id":"hellaswag:00003364","item_index":370,"attempt":0,"prompt_hash":"d205bc91f797c180f11b1471639e2289952adf02cc07bb5a10a269829b6cab34","question":"[header] How to raise a transgender child [title] Don't fight your child's gender. [step] Your child is already transgender, and you can't change that. Fighting your child's identity is exhausting, and it will cause them tremendous pain.\n\nA. Resist the urge to argue with your child about her gender. Arguments are insignificant and can't hurt.\nB. [title] Support your child through this difficult time. [step] Your child should not feel pressured into being transgender, but they should be as supportive as possible.\nC. Instead, it's important to accept that your child is different, and let them live as they are most comfortable. [substeps] Trans youth who are accepted and able to live openly as their true gender are at much lower risk of mental health problems.\nD. Children who are transgender are often stubborn when it comes to love and acceptance. Arguing will only clog up your loved one's already blocked birth canal.","prompt":"[header] How to raise a transgender child [title] Don't fight your child's gender. [step] Your child is already transgender, and you can't change that. Fighting your child's identity is exhausting, and it will cause them tremendous pain.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5388,"error":null,"output":"B"}