{"question_id":"hellaswag:00001464","item_index":0,"attempt":0,"prompt_hash":"ba0eec183742567ff3ce00d6d753626e57658db5ee50d63d63c16f273cf828a1","question":"Several athletes compete for the javelin world championships. The russia dimitri is 3rd bronze medal in the competition. The japanese jenki is silver medal. till\n\nA. javelin 1990 the russian javelin champion anna rubens is strong.\nB. from germany won the gold medal.\nC. loesos facility there is no competition.\nD. now the french have won each javelin every round.","prompt":"Several athletes compete for the javelin world championships. The russia dimitri is 3rd bronze medal in the competition. The japanese jenki is silver medal. till","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3797,"error":null,"output":"B"}
{"question_id":"hellaswag:00005973","item_index":1,"attempt":0,"prompt_hash":"c98299ff57338badd184541c4c45b651ba8d8275c5fe5e34da673032bc9e1730","question":"[header] How to get japanese characters (kanji, hiragana, katakana) on firefox [title] In the firefox toolbar, go to view] character encoding] auto-detect. [step] While most browsers are already capable of reading international characters, you may simply need to help firefox detect them. [title] Select japanese.\n\nA. [step] By default, most of the newer characters on a firefox list are japanese (or katakana). However, you may have to reach up and back to find ones you'd like to include or that make similarities between each other.\nB. [title] Focus your attention on chikatakana. [title] Choose the gender and intent of the character, ren.\nC. [step] Some of the same characters can be found on the scroll bar. Selected japanese is the most commonly used japanese toolbar.\nD. [step] Your firefox should now be able to detect japanese characters. If not, you may need to install a language pack using one of the methods below.","prompt":"[header] How to get japanese characters (kanji, hiragana, katakana) on firefox [title] In the firefox toolbar, go to view] character encoding] auto-detect. [step] While most browsers are already capable of reading international characters, you may simply need to help firefox detect them. [title] Select japanese.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3418,"error":null,"output":"D"}
{"question_id":"hellaswag:00006664","item_index":2,"attempt":0,"prompt_hash":"e5e3b6eec41d57d6434bb2b5a5310bbb94b926733017dcdc9badc87e910f1f89","question":"[header] How to make cappuccino foam [title] Pour brewed espresso into your cappuccino cup. [step] Although specifics will depend on the type of cappuccino and the person making it, cappuccinos are roughly one quarter espresso and three quarters steamed milk. Pour your desired amount of brewed espresso into a large cappuccino cup before you steam the milk.\n\nA. [substeps] Instead of cappuccino add straight milk. Espresso can be iced later, and most cappuccino companies encourage it to be iced after brewing to sweeten the milk.\nB. That way, you'll have everything ready so you can pour out the steamed milk while it's still piping hot. [title] Fill a steam pitcher with cold milk.\nC. This will preserve your milk-stirring capacity throughout the rest of the process. [substeps] Try the recommended amount of coffee to get the right amount.\nD. Start off with hot to ensure the coffee in your cup is sufficiently cold to immerse the foam. [substeps] It's best to use skim milk.","prompt":"[header] How to make cappuccino foam [title] Pour brewed espresso into your cappuccino cup. [step] Although specifics will depend on the type of cappuccino and the person making it, cappuccinos are roughly one quarter espresso and three quarters steamed milk. Pour your desired amount of brewed espresso into a large cappuccino cup before you steam the milk.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3418,"error":null,"output":"B"}
{"question_id":"hellaswag:00008041","item_index":3,"attempt":0,"prompt_hash":"c715921c871a37324dd5e129a662c1d7f828ad51bd20afb77e4e419d63a6ef88","question":"[header] How to get an adult male to use his manners [title] Consider what direct and indirect demands you have been making. [step] Reflect on the balance of personal demands you are making to your partner, and evaluate whether that balance is just. [title] Be prepared to evaluate your viewpoint of the situation as objectively as possible.\n\nA. [title] First, determine whether your partner was accurate in his assessment of his sexual capabilities and responses. [title] Discuss there are a number of different ways that men in modern society can show discretion, professionalism, and safety in the absence of sexual restraint.\nB. [step] You have also made a choice that your partner must want and must follow. When you record and evaluate your demands, be sure you are or need to be able to speak with those who are making them.\nC. [title] Consider your own perspective, not personal. [step] Communicating your viewpoint will provide you with invaluable resources.\nD. [step] Don't go into the impending talk convinced that anything he might say is either wrong or not heartfelt. That won't help to solve the problem.","prompt":"[header] How to get an adult male to use his manners [title] Consider what direct and indirect demands you have been making. [step] Reflect on the balance of personal demands you are making to your partner, and evaluate whether that balance is just. [title] Be prepared to evaluate your viewpoint of the situation as objectively as possible.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3415,"error":null,"output":"A"}
{"question_id":"hellaswag:00001931","item_index":4,"attempt":0,"prompt_hash":"fb41bcc882aaa59d39de81002801fc5c61f882653d5baaa0ad8f58679f683843","question":"A boy holds up a bottle of mouth wash. he\n\nA. bends down and throws it.\nB. takes the mouthwash and blows his horn.\nC. uses the mouthwash to wash himself and two other teens.\nD. drinks a cap full of the mouth wash.","prompt":"A boy holds up a bottle of mouth wash. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3418,"error":null,"output":"D"}
{"question_id":"hellaswag:00007682","item_index":5,"attempt":0,"prompt_hash":"f871412b315b9345ef7299b65be63da16bb28eb40966c33331e856e57f12488f","question":"[header] How to wash newborn hair [title] Rinse your newborn's hair for the first bath. [step] Because the hair and skin of a newborn are so delicate and sensitive, you don't actually need to use shampoo or cleansers. For your newborn's first bath, just pour a little plain water over their scalp.\n\nA. Just enough so that the hair and skin are fully saturated. [substeps] This is also an optional step that you can do while breastfeeding.\nB. You will make sure that they have nothing like a toothbrush or comb or baby oil in their shampoos. [title] Repeat the soap and water process with your baby's hair.\nC. Pat the newborn's scalp dry with a soft towel. [substeps] You don't need to scrub, massage, or really wash the scalp until they're a little older.\nD. For added comfort, you can place the newborn against a towel lined with skin-safe bath paper. [title] Combine the shampoo and water in a small bowl.","prompt":"[header] How to wash newborn hair [title] Rinse your newborn's hair for the first bath. [step] Because the hair and skin of a newborn are so delicate and sensitive, you don't actually need to use shampoo or cleansers. For your newborn's first bath, just pour a little plain water over their scalp.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3419,"error":null,"output":"C"}
{"question_id":"hellaswag:00003686","item_index":6,"attempt":0,"prompt_hash":"587ce0395ee5a7bed3bfe11104d1919a6328ea8934ffdb8e7edefd3a2bb9a841","question":"[header] How to pick a wedding day color [title] Decide whether or not you want to have a wedding theme. [step] If your theme is one that limits your color choices, you automatically have a range of colors you need to work within. [title] Consider the season you are having your wedding in.\n\nA. [step] Autumn and winter weddings look best in rich colors like reds and oranges while spring and summer colors can be pastels or bright colors. [title] Find out the color of your birth stone or your mother's, if that color works for you, it might add another layer of meaning to your wedding day.\nB. [step] Is it fall, winter, or mid-autumn? Ideally, the colors at that seasons wedding would be old or new. Decide whether you want a plain or original color.\nC. [step] Depending on how you pick your theme, summer weddings typically fall into the season of september or november. Most people in summer tend to choose spring-themed colors to reserve them during a much warmer season, like april, october, or march.\nD. [step] If you are having a holiday wedding, you may want the ceremony during november or early july. If it is a fall or winter wedding, you may want the ceremony during the spring or summer.","prompt":"[header] How to pick a wedding day color [title] Decide whether or not you want to have a wedding theme. [step] If your theme is one that limits your color choices, you automatically have a range of colors you need to work within. [title] Consider the season you are having your wedding in.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7077,"error":null,"output":"A"}
{"question_id":"hellaswag:00005546","item_index":7,"attempt":0,"prompt_hash":"5ac6225af2e4339e5679ce769778df85a8bf4da6c7484d085731f0f6d8168bf8","question":"[header] How to do harajuku makeup [title] Change your eye color, if you wish. [step] If you have colored contact lenses to change your eye color you can put them in before or after you do your makeup. This step is optional.\n\nA. [title] Look for any redness or swelling in your nose/brow area. [step] If you have done this before, you'll be aware that there will be a noticeable pop when you look at your lenses.\nB. If you don't want to change your eye color, you can wear something in the shade that is called your coloured contact lens. [title] Sit quietly throughout the session, without interacting with other people.\nC. [substeps] It is recommended to put them in before so that you don't smudge your makeup. [title] Start off with a base foundation at least one shade lighter than your natural skin tone.\nD. If your eye color is yellow or hazel, you may wish to put them in with your eyeshadow. [title] Put the eyeshadow on your eyelids (eye side, the lid side).","prompt":"[header] How to do harajuku makeup [title] Change your eye color, if you wish. [step] If you have colored contact lenses to change your eye color you can put them in before or after you do your makeup. This step is optional.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3419,"error":null,"output":"C"}
{"question_id":"hellaswag:00008362","item_index":8,"attempt":0,"prompt_hash":"9602d27c00c1c9c7f30050ec99f5f32b30f575c4d97295c95b2273fbaffb7cc0","question":"[header] How to improve your hiking technique [title] Invest in a good pair of hiking boots. [step] Good boots are critical and sometimes expensive. They don't have to be expensive boots (though professionally fitted boots are the best).\n\nA. These are generally one of the more stylish and are excellent for acclimating to the outdoors in west indian terrain, due to their thick soles. Also consider investing in a belt and/or backpack that will help keep you comfortable during the hiking.\nB. The point of hiking is to have clear toe space when walking. The more clean your boots look, the more accurate your tests may be.\nC. Your new shoes or still brands will really help your workout. [title] Set a timer for yourself that applies to your workout in the designated time period.\nD. It is best to know the specific or varied environments in which the hiking will be conducted, this will allow you to select a pair of boots that are appropriate for the conditions. For instance, water resistant features on a boot may be appropriate in areas that experience moderate to low temperatures and heavy rain because they will often lengthen the time during which you can hike with warm, dry feet.","prompt":"[header] How to improve your hiking technique [title] Invest in a good pair of hiking boots. [step] Good boots are critical and sometimes expensive. They don't have to be expensive boots (though professionally fitted boots are the best).","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3419,"error":null,"output":"D"}
{"question_id":"hellaswag:00006821","item_index":9,"attempt":0,"prompt_hash":"25c2e7814acf44fa7c096c37ef204df08909ebed55cd55ea46935a3717a55000","question":"[header] How to kill creepers in minecraft pe [title] Get the best sword you can find. [step] If you don't have any you might want to make one before this fight. [title] Move away from your home or anything you have built in your world.\n\nA. [step] You want this to be a time that you cannot be bothered playing, so your better off if you have to kill creepers. [title] Find the latest white sword you have going in minecraft pe.\nB. [step] If this battle ever ends, if you get caught (who knows...maybe monsters aren't caught here), enemies will die if you leave your house behind. [title] Do not advance unless you are warned by a monster.\nC. [title] Use fake bats to karp at these locations. [title] If you are going to use real bats you'll need to boss kill them in order for them to finish up.\nD. [step] If a creeper blows up near it, it could end up destroying a lot of your items and materials. [title] Find a creeper and lead it to some flatter ground.","prompt":"[header] How to kill creepers in minecraft pe [title] Get the best sword you can find. [step] If you don't have any you might want to make one before this fight. [title] Move away from your home or anything you have built in your world.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3416,"error":null,"output":"D"}
{"question_id":"hellaswag:00007254","item_index":10,"attempt":0,"prompt_hash":"39a1f5fcdf0d04c539c8628c8d415c18b767e73c386d1ea2c620426e963441fe","question":"[header] How to make a secure home for an autistic child [title] Remove and replace fluorescent lights. [step] Many autistic people have severe sensory problems with fluorescent lights. If you have any fluorescent lighting in your home, replace it with soft, warm lamps or non-fluorescent fixtures.\n\nA. [substeps] The reasons autistic people have difficulty with fluorescent lights differ depending on the person. For some, it is a visual disturbance.\nB. See if any fluorescent fixtures you use advertise as helpful. [substeps] Say to your child, \" is there anything i can do to help with this problem? \" [title] Seek treatment.\nC. [substeps] A used or sparking bulb is usually okay; however, turning the lights off will require you to replace it in the middle of the night. Plug in incandescent lamps if you are not used to them.\nD. Replacing fluorescent tubes with bulbs that are battery-powered could become a nuisance for autistic children. To avoid that possibility in the future, put up or install fluorescent panels in any rooms your child frequently frequents.","prompt":"[header] How to make a secure home for an autistic child [title] Remove and replace fluorescent lights. [step] Many autistic people have severe sensory problems with fluorescent lights. If you have any fluorescent lighting in your home, replace it with soft, warm lamps or non-fluorescent fixtures.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3415,"error":null,"output":"A"}
{"question_id":"hellaswag:00005430","item_index":11,"attempt":0,"prompt_hash":"33263167a56b738c06a4f4a2914680d80830d05a9e36314e1f3e4e4184635bea","question":"[header] How to become a dymaxion sleeper [title] Set your nap times. [step] You will take four naps a day. That means you will need to nap for 30 minutes every 6 hours.\n\nA. Stick to an eight-hour cycle every day. For optimal health, you should sleep eight to ten hours each night.\nB. Try to set a time for each dark and gloomy day or when your mind is at its most tired. [title] Get into the habit of waking at 8 am and bedtime.\nC. When you wake up, unwind and take your time! The time difference can be big-it can take you over an hour to fall asleep, but it's worth your investment. Take a nap at the most late in the day.\nD. You can set those naps whenever is most convenient for you, but you must stick to your schedule once you set it. [substeps] For instance, you could nap at 2 am , 8 am , 2 pm, and 8 pm.","prompt":"[header] How to become a dymaxion sleeper [title] Set your nap times. [step] You will take four naps a day. That means you will need to nap for 30 minutes every 6 hours.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3419,"error":null,"output":"D"}
{"question_id":"hellaswag:00003095","item_index":12,"attempt":0,"prompt_hash":"682abd21b89294d8e74801a9b3702caf22cc79979a01f74f7ac5577e3dcd7d19","question":"A man stands in the middle of a stadium. crowds\n\nA. are gathered around him as he spins around.\nB. watch and cheer the man on.\nC. watch from behind a curtain.\nD. flash around him.","prompt":"A man stands in the middle of a stadium. crowds","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3418,"error":null,"output":"A"}
{"question_id":"hellaswag:00005836","item_index":13,"attempt":0,"prompt_hash":"d1a3429964aa82e85a49d7784969f7ab6d4442dde4a83c175c7f4f8e108ff00e","question":"[header] How to use netflix codes [title] Find the most recent list of codes. [step] Netflix has a very long list of four to six digit codes that help you browse for very specific types of films. The list of codes is changing all the time, so type in \" netflix codes \" into a search engine to find the most recent list.\n\nA. [substeps] Since netflix is an online dating service, it uses massive numbers of codes that you can enter to find the right type of film and series. For instance, several countries have netflix codes with a specific name, such as canada, africa, england, or philadelphia.\nB. [substeps] The sites what's on netflix and netflixcodes.me keep updated lists of codes. [title] Browse the codes to find a movie genre.\nC. [substeps] You can get a zip code from the packet. Most servers have a sticker you can paste to the package text field at the top of the netflix netflix web page.\nD. [substeps] The codes for new york codes are listed by city, state, country and country, so finding them will help you easily find variety of codes. [title] Enter all the codes into your local lock store.","prompt":"[header] How to use netflix codes [title] Find the most recent list of codes. [step] Netflix has a very long list of four to six digit codes that help you browse for very specific types of films. The list of codes is changing all the time, so type in \" netflix codes \" into a search engine to find the most recent list.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3415,"error":null,"output":"B"}
{"question_id":"hellaswag:00004089","item_index":14,"attempt":0,"prompt_hash":"07fbc6efd6d2a27de2842ce6317784ffe30d99b403aa2249d8e2e3f823324118","question":"[header] How to tell if you have metabolic syndrome [title] Measure your waistline. [step] If you have a large waistline, it can be a risk factor for metabolic syndrome. While a large waistline may often depend upon your height, there are some measurements which will usually tell you if you are at risk for metabolic syndrome.\n\nA. [substeps] Sit on a small stool or a chair and measure your waist using a stopwatch or multimeter. Then, weigh it from top to bottom to find your waist size.\nB. However, you should be mindful of your weight and body mass index (bmi) to determine whether you are at risk. Bmi is calculated based on your height.\nC. [substeps] For men, a waistline over 40 inches (100 cm) can be a risk factor for metabolic syndrome. A waistline over 35 inches (88.9 cm) is a risk for women.\nD. [substeps] To get an accurate measurement of your waistline, you first have to measure your waist length and chest height. Measure from your ribcage to the level of your shoulders.","prompt":"[header] How to tell if you have metabolic syndrome [title] Measure your waistline. [step] If you have a large waistline, it can be a risk factor for metabolic syndrome. While a large waistline may often depend upon your height, there are some measurements which will usually tell you if you are at risk for metabolic syndrome.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3793,"error":null,"output":"C"}
{"question_id":"hellaswag:00007959","item_index":15,"attempt":0,"prompt_hash":"fb5190396368214e7b72a508c0a73a0039ce8ee43d5c1802f8d9f55c72089c52","question":"[header] How to sneak tampons and pads into the bathroom [title] Store tampons and pads in several different locations. [step] This makes sneaking your stash into the bathroom more convenient based on where you are and who you're with. For example, if spending the day at school or work, store tampons and pads in your purse, desk, backpack, pencil case, lunch box, and car.\n\nA. [title] Pretend you're hunting for something else. [step] This tricks people into thinking you're looking for items other than feminine products.\nB. You can also sneak them into the locker if you are at home, but that will be quicker. [title] Change your tampons and pads often.\nC. [title] Take plenty of hygiene items with you. [step] Long days on the bus and across a crowded parking lot can make sneaking tampons and pads hard.\nD. Put the backs on your tampons and pads in plain sight to avoid having to keep them in your locker. [title] Eliminate the use of your bag or backpack or any other inconvenient places.","prompt":"[header] How to sneak tampons and pads into the bathroom [title] Store tampons and pads in several different locations. [step] This makes sneaking your stash into the bathroom more convenient based on where you are and who you're with. For example, if spending the day at school or work, store tampons and pads in your purse, desk, backpack, pencil case, lunch box, and car.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3419,"error":null,"output":"A"}
{"question_id":"hellaswag:00002774","item_index":16,"attempt":0,"prompt_hash":"9152c28eb7477933748c7fd9e61f40072b1c7ed00e48293f670606fc12da0b21","question":"The bull is released and a man on a horse chases after it. The man jumps off the horse and ties the bull up. the man\n\nA. puts ointment on the bulls nose.\nB. stands up and walks back to his horse.\nC. ties up the horses legs as well.\nD. then jumps back on the horse and ties its legs tightly.","prompt":"The bull is released and a man on a horse chases after it. The man jumps off the horse and ties the bull up. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3420,"error":null,"output":"B"}
{"question_id":"hellaswag:00008651","item_index":17,"attempt":0,"prompt_hash":"45554518c34774d37bdd48af6aa4a59fc03da89db95cac97412659bcc0811d22","question":"[header] How to look after your skin [title] Wash your face in the morning and in the evening. [step] Make sure that you use the right facial cleaner for your skin type. For example, if you have oily skin or acne, get something that contains salicylic acid or benzoyl peroxide.\n\nA. If you have dry skin, use a gentle, moisturizing, fragrance-free face wash. [title] Avoid using hot water while you bath, and limit your bathing time to 20 minutes.\nB. If you have combination skin, find one with a different additive. [substeps] If your normal acne product washes out, make sure you store it in a clean plastic bag inside of a clean container that can be used for cleaning.\nC. If you have combination acne, use a mild face cleanser. Apply a soft washcloth wet with warm water and gently scoop and massage the area of your face that has acne.\nD. If you do have dark skin, make sure to soak up excess moisture from the face. [substeps] If your skin is oily or acne-prone, then grab a face cleanser made specifically for oily skin.","prompt":"[header] How to look after your skin [title] Wash your face in the morning and in the evening. [step] Make sure that you use the right facial cleaner for your skin type. For example, if you have oily skin or acne, get something that contains salicylic acid or benzoyl peroxide.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7077,"error":null,"output":"A"}
{"question_id":"hellaswag:00008747","item_index":18,"attempt":0,"prompt_hash":"37a3e5d8204cae9fea5d726d02d17ce8d8d276f94c31f974e31e18baa06f3572","question":"[header] How to choose meal replacements [title] Choose bars if convenience and feeling satisfied is important. [step] You can easily stow away meal replacement bars in your purse or suitcase, and they make a good choice if you value convenience. Meal replacement bars tend to help you feel more fully satisfied than liquid meal replacement options.\n\nA. [substeps] You can buy lunch bars at the grocery store if you prefer fast food. Or, you can purchase instant grocery bars if you prefer non-processed options like whole grain bread and rice.\nB. [title] Choose flavored bars to avoid offending your mouth or gums. [step] If you're not sure whether the dish you may be eating is flavored or plain, consider choosing whole-flavored bars, rather than complex flavors.\nC. [substeps] You can find meal replacement bars at grocery stores, health stores, or online. [title] Find a shake if you prefer sipping your calories.\nD. [substeps] Make sure the bars are full of carbohydrates and fats rather than sugar. [title] Use whole-grain bars on dishes with low calorie content.","prompt":"[header] How to choose meal replacements [title] Choose bars if convenience and feeling satisfied is important. [step] You can easily stow away meal replacement bars in your purse or suitcase, and they make a good choice if you value convenience. Meal replacement bars tend to help you feel more fully satisfied than liquid meal replacement options.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3419,"error":null,"output":"C"}
{"question_id":"hellaswag:00006596","item_index":19,"attempt":0,"prompt_hash":"191b68cfd444b0a980a9bd05835e2c7e4039a3aca3a370759a7b8b569def2bb2","question":"[header] How to get over a crush because you have moved away [title] Decide whether or not you want to stay in touch. [step] For some people, staying in touch can be a nice way to keep a friend who you've got a little crush on. For others, it can be like torture.\n\nA. [substeps] Know what it's like to be apart. Your feelings are the proof that you care and that the relationship is worth pursuing and that a friendship is worth keeping, not just a platonic friendship.\nB. If you decide that the two of you want to end things, your friend should probably go back to someone else. [substeps] You can always be pals when you're apart.\nC. Decide if you're likely to ever see this person in real life again, or if staying in touch will only make it worse. [substeps] If you can't be together and don't live in the same area, consider cutting all ties.\nD. Deciding whether or not you want to stay in touch can cause hurt for both you and your friend. [substeps] If you haven't been able to call or text each other in a while, it can be because you didn't actually get together.","prompt":"[header] How to get over a crush because you have moved away [title] Decide whether or not you want to stay in touch. [step] For some people, staying in touch can be a nice way to keep a friend who you've got a little crush on. For others, it can be like torture.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3414,"error":null,"output":"C"}
{"question_id":"hellaswag:00008766","item_index":20,"attempt":0,"prompt_hash":"a32e88c9134b346aae5c876e93a2eae144a2adb0d10578ea1d1959bdab6150c6","question":"[header] How to get natural looking waves [title] Wash or mist your hair. [step] When braiding your hair to create waves, it is best to begin with damp hair. You may achieve this in one of two ways : [substeps] Wash your tresses prior to braiding your hair.\n\nA. Tie off the ends of the braid and then let the loose hair dry slightly before washing it. Gradually do this for all hair types-curly, straight, with some parting.\nB. When you begin braiding your hair, do not pat your locks dry. Instead, work a small amount of shampoo (coconut hair shampoo) into your hands and massage the shampoo into your hair.\nC. In one, slightly wet your hair, then mist it with water. In the other, leave your hair damp and then mist your tresses gently with the palm of your hand.\nD. Mist your hair with lukewarm water. [title] Combat frizz and snarls.","prompt":"[header] How to get natural looking waves [title] Wash or mist your hair. [step] When braiding your hair to create waves, it is best to begin with damp hair. You may achieve this in one of two ways : [substeps] Wash your tresses prior to braiding your hair.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3416,"error":null,"output":"B"}
{"question_id":"hellaswag:00006932","item_index":21,"attempt":0,"prompt_hash":"7c48c02bf864ece60632257f0f56321fa69e863e7bd1670ac3081b9063f411c0","question":"[header] How to choose a haircut for guys with thinning hair [title] Keep it short. [step] Generally speaking, the number one rule of dealing with thinning hair is to not let it get too long. This applies to men with both straight and curly hair, and anything in between.\n\nA. Close-cropped cuts will give your hair a more uniform appearance and make inconsistencies in fullness less noticeable. [substeps] For most hairstyles, a length of about an inch or less will work best for minimizing the appearance of thin patches.\nB. You need to give your head a trim regularly to maintain the slicked back look. [title] Rinse out your hair with cool water.\nC. [substeps] Long, messy waves can look good with simple hairstyles, but adding waves and tucking in ends can help you project the strength necessary to create the look you want. Stay away from drastic bobs, ponytails, or a rounded updo for men with thinning hair.\nD. Give your hair a reason to grow out; girls love that! [title] Consider changing your hairstyle every two to three weeks. [step] Start with a bohemian style if you want to show off your straight hair.","prompt":"[header] How to choose a haircut for guys with thinning hair [title] Keep it short. [step] Generally speaking, the number one rule of dealing with thinning hair is to not let it get too long. This applies to men with both straight and curly hair, and anything in between.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3418,"error":null,"output":"A"}
{"question_id":"hellaswag:00001729","item_index":22,"attempt":0,"prompt_hash":"449ead651d0a05f2f0ce163d090d0d8929886858bfd829decf2331a3cd720e88","question":"A man is dribbling two basketballs on a court. A man blocks another man with a blue mat. a man\n\nA. shoots a basketball at a child holding a bakery cup.\nB. sticks a dart into the hoop next to the hoop.\nC. shoots a basket and scores.\nD. up front steps and jumps on the man's back.","prompt":"A man is dribbling two basketballs on a court. A man blocks another man with a blue mat. a man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3794,"error":null,"output":"C"}
{"question_id":"hellaswag:00000385","item_index":23,"attempt":0,"prompt_hash":"df0d7fd9ff6f3ac8345be9a1d1e9ec4cedda7afdfec2b21700d24cc049f2a5d5","question":"A small group of people are seen shaking hands and standing together followed by a man blowing a whistle and people running around. the people\n\nA. then play a game of soccer with one another, running up and down the sandy field while coaches yell on the side and a goal is blocked.\nB. continue to run around under the stage and close up of a man speaking and shows people playing a game.\nC. move around with drums and hitting a tune in the middle of the gym and the man continues to blow a whistle and lead into more people walking past on the side.\nD. continue riding around on skateboards as well as following behind one another as well as people speaking to the camera.","prompt":"A small group of people are seen shaking hands and standing together followed by a man blowing a whistle and people running around. the people","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3420,"error":null,"output":"D"}
{"question_id":"hellaswag:00007821","item_index":24,"attempt":0,"prompt_hash":"b637fc1be5ca52b6ea54c7baa2534e92b02570fc90fd2847301275524c09e719","question":"[header] How to make curly hair grow faster [title] Use a deep conditioner 2-3 times per week. [step] Keeping your hair moist and conditioned is important to help it grow faster. Dry, limp, unhealthy hair will not grow as fast.\n\nA. Additionally, it will break and fall out. Use a deep conditioner on your hair at least two to three times each week.\nB. [title] Use perfume or alcohol to aid in the process of growing. [step] If you find you are losing much water (especially if you usually sweat) when deep-conditioning your hair, you may be suffering from a condition called \" dryer hair.\nC. It is much more likely to form frizziness and knots. This means your hair will react to the blow dryer and get frizz when it's not drying.\nD. Use a deep conditioner based on the salicylic acid and mineral oils you see on the comb and brush to lubricate the scalp and hair slightly. [title] Rinse your hair with cool water when you wake up in the morning.","prompt":"[header] How to make curly hair grow faster [title] Use a deep conditioner 2-3 times per week. [step] Keeping your hair moist and conditioned is important to help it grow faster. Dry, limp, unhealthy hair will not grow as fast.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7077,"error":null,"output":"A"}
{"question_id":"hellaswag:00008789","item_index":25,"attempt":0,"prompt_hash":"c6c7a5f2e848b1743e2bc5539dc7ac73059108fb5c7997070304945d5d599798","question":"[header] How to pay a pet sitter [title] Determine the going rate for pet sitters in your area. [step] Before you decide what to pay a pet sitter, you should determine the pay rates in your area. You do not want to mistakenly offer your sitter too much or too little for the job.\n\nA. Find out and ask your local animal shelters what their rates are before you make your selection. [substeps] You can also check with online services such as petfinder.\nB. [substeps] Browse the yelp pages in your area for pet sitting companies and individual pet sitters. Find a link to their website from there and see what rates they typically charge.\nC. In addition, simply tracking down the average pay rate for pet sitters could lead to lower costs if the sitter uses part time or on-the-job budgets. [substeps] Look through local pet sitter associations like yamano or caputt, and compare rates.\nD. However, you should assess what you understand about the job before deciding what payment methods you want to use. [substeps] Check with your vet for the pay rate in your region.","prompt":"[header] How to pay a pet sitter [title] Determine the going rate for pet sitters in your area. [step] Before you decide what to pay a pet sitter, you should determine the pay rates in your area. You do not want to mistakenly offer your sitter too much or too little for the job.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7077,"error":null,"output":"B"}
{"question_id":"hellaswag:00004358","item_index":26,"attempt":0,"prompt_hash":"01b6e209dc80dd11b87f6b88cc00ee0c1d39e8c6b8657deec7791b854fb3daf2","question":"[header] How to create a pink fairy makeup look [title] Clean your face. [step] Your face needs to be clean. So wash your face with any cleanser or face wash you have.\n\nA. You don't need much soap, just some lather. Use a hair spray, a lip balm or a blush or any other method you like.\nB. If you have acne, use a face mask. [title] Apply pink eye liner and make your crease line real using the corners of your eye and stopping at the outer corner.\nC. If you have acne, use a facial cleanser. You may consider getting a monthly facial cleanser, but any face wash will do.\nD. If you don't have any, just wash your face how you normally do. [title] Apply your face lotion or moisturizer evenly on your face.","prompt":"[header] How to create a pink fairy makeup look [title] Clean your face. [step] Your face needs to be clean. So wash your face with any cleanser or face wash you have.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7075,"error":null,"output":"D"}
{"question_id":"hellaswag:00006141","item_index":27,"attempt":0,"prompt_hash":"33ca21fd29ba31aead70e5c9332b5a4cc792b1a05040ab3e5a617694a3144c3c","question":"[header] How to give a baby a bath in the sink [title] Fill the bath a few inches deep. [step] Fill a safe, clean receptacle with roughly three inches of warm water. The water should be between between 90 ° f (32 ° c) and 100 ° f (38 ° c).\n\nA. Once you have the appropriate amount of water, shut it off and swirl the water to ensure there are no especially hot spots. [substeps] Touch the water with the inside of your wrist.\nB. If the baby is crying, they are likely to drown, so be sure to warm up. [substeps] Baby showers are often set up near the bath area.\nC. [substeps] If the bath is not going to produce suds in the water, you may want to fill a bowl with clean tap water to create a cleaner float. [title] Submerge your baby in the water.\nD. Add a few drops of essential oils to the water to add a relaxing feeling to the bath. [title] Submerge your newborn in the water.","prompt":"[header] How to give a baby a bath in the sink [title] Fill the bath a few inches deep. [step] Fill a safe, clean receptacle with roughly three inches of warm water. The water should be between between 90 ° f (32 ° c) and 100 ° f (38 ° c).","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3416,"error":null,"output":"A"}
{"question_id":"hellaswag:00005522","item_index":28,"attempt":0,"prompt_hash":"bc8db720c926a3a884561328e046a8dfd9c231e42c812ef9f9124096808865b4","question":"[header] How to reduce body hair growth [title] Start waxing. [step] Waxing can be painful, but it is quite effective at reducing body hair growth. Since the hair is removed from the root, the hair grows back finer, thinner, and slower.\n\nA. The wax is temporary, but hair grows back to the desired length within a day. When waxing, the hair works best against very coarse hairs.\nB. [substeps] Waxing can be effective on all genital areas of body hair including your vaginal area. However, you cannot use it on more than one area.\nC. Waxing can be done at home for children, who use waxing products designed to keep their hair clean. It is also necessary for individuals with any hair condition, such as acne or hair breakage.\nD. You can either buy wax to apply yourself at home or go to a spa or salon to get a professional wax treatment. [substeps] There are two different kinds of wax.","prompt":"[header] How to reduce body hair growth [title] Start waxing. [step] Waxing can be painful, but it is quite effective at reducing body hair growth. Since the hair is removed from the root, the hair grows back finer, thinner, and slower.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3416,"error":null,"output":"D"}
{"question_id":"hellaswag:00006161","item_index":29,"attempt":0,"prompt_hash":"4279c2636d71c42cdd07d7a5ae3f162915b143bd5d5f70ae5b826e7a8366dca0","question":"[header] How to calm a vicious rabbit [title] Show the rabbit it's hurting you if it bites. [step] Make a sudden yelping sound or squeal when and if your rabbit bites or nips you. This will act as a kind of signal that you're in pain, and your rabbit will associate that with biting you.\n\nA. Be sure to discourage biting due to its obedient nature. [substeps] You can also make it less likely that your rabbit will bite you by rubbing it against something, such as a tree trunk, a post (or other object), or a cat.\nB. [substeps] Place the muzzle over your rabbit's mouth, blocking its airways, and apply firm but gentle pressure. Continue holding the muzzle in place until your rabbit calms down.\nC. [substeps] Tiny nips can just be your rabbit's way of telling you to go away, or that you're bugging it. They're not trying to hurt you, just trying to let you know they don't want to be touched or handled.\nD. [title] Scold the rabbit if it bites or nips you. [step] If the rabbit bites you, you should immediately bring it to a veterinarian for medical attention.","prompt":"[header] How to calm a vicious rabbit [title] Show the rabbit it's hurting you if it bites. [step] Make a sudden yelping sound or squeal when and if your rabbit bites or nips you. This will act as a kind of signal that you're in pain, and your rabbit will associate that with biting you.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3794,"error":null,"output":"D"}
{"question_id":"hellaswag:00007627","item_index":30,"attempt":0,"prompt_hash":"78dd98808bedb1e962cdd7fb9d8b38a8fbdd47041767f9386099e67ba89183c7","question":"[header] How to do smokey eyes on dark, tan, or indian skin [title] Wash your face. [step] No makeup looks good on clogged skin or skin that still has previous day's makeup on. Start afresh! [title] After you have cleansed thoroughly, use a toner on a cotton ball and pat it on your face.\n\nA. [step] Shake well until, your toner is on your face without running through your hair. [title] Start by sweeping your toner onto your eyelid.\nB. [step] Be sure to test the ph of your toner on a small spf. [title] Apply your concealer in an inconspicuous spot on your face.\nC. [step] Be sure to give yourself a special toner every time you wash your face. Make sure that you choose one that is scented, as it will help your toner stick.\nD. [step] Regular use of toner will help firm your skin and also reduce sight of clogs and large pores. [title] Apply a very thin layer of foundation and get ready for the next step.","prompt":"[header] How to do smokey eyes on dark, tan, or indian skin [title] Wash your face. [step] No makeup looks good on clogged skin or skin that still has previous day's makeup on. Start afresh! [title] After you have cleansed thoroughly, use a toner on a cotton ball and pat it on your face.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3793,"error":null,"output":"D"}
{"question_id":"hellaswag:00009583","item_index":31,"attempt":0,"prompt_hash":"6748d13dd5a9ad9290d0cee3d13a4936e8a3ecf79cb7b15106709426cfcb98a8","question":"[header] How to pamper yourself without buying new things [title] Freshen up. [step] Have a bath or shower (preferably a bath) and use any bath bombs, oils or bath soaps you have to just relax yourself. You don't have to use fancy bath stuff if you don't have it.\n\nA. Plain, old bar soap will do fine. [title] Dry and style your hair however you'd like.\nB. Make sure everything is nice and fresh before you go in front of the mirror. It's a lot easier to see makeup and make-up if it's fresh.\nC. Bath bombs are often used for body spray, but also for menstrual and vaginal hygiene. Also, shower after showering so the spritz of bath bombs will have a longer shelf life.\nD. Easy, just a shower and some moisturiser can do wonders! [substeps] Wash with sudsy water. This can loosen your underskin, not prevent the skin from fading.","prompt":"[header] How to pamper yourself without buying new things [title] Freshen up. [step] Have a bath or shower (preferably a bath) and use any bath bombs, oils or bath soaps you have to just relax yourself. You don't have to use fancy bath stuff if you don't have it.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3419,"error":null,"output":"A"}
{"question_id":"hellaswag:00002561","item_index":32,"attempt":0,"prompt_hash":"bb673580e94f2de822d6030d306cf237afd125c64868e52e52572606e73a7013","question":"A man is seen wearing a scuba mask underwater while speaking to the camera and pointing to his face. his goggles\n\nA. are shown and more shots of him being seen as he speaks.\nB. are removed from his face and begins moving around.\nC. are removed and he uses the metal nozzle to a scuba diver's face.\nD. then fill up with water and he takes his goggles off still moving the camera to speak.","prompt":"A man is seen wearing a scuba mask underwater while speaking to the camera and pointing to his face. his goggles","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5128,"error":null,"output":"B"}
{"question_id":"hellaswag:00006130","item_index":33,"attempt":0,"prompt_hash":"1d3347097406dd9251d45f1de409136a6743b2a5c71c66fb11d381ccf5d1e093","question":"[header] How to do striped nail art with rhinestones [title] Paint two coats of your first polish on your nails. [step] Choose a colour that goes well with a black design on top, such as a deep red. [title] Once the red has dried, take your black striper.\n\nA. [step] Place it at the top, leaving enough space to wrap a rubber band around it to keep the paint from dripping through. [title] Finish with a clear top coat, making sure to keep your rhinestones (ones that are hard to see) moist.\nB. [step] It can be sparkly or matte-either way it will still look fine. [title] Make 2 diagonal lines on your nails alongside each other.\nC. [step] Brush on two thin layers of the second colour. Allow it to dry completely between the different coats.\nD. [step] Using a black striper (which you can find at a studio that sells rhinestones), paint two thin coats of your first colour on top of the rhinestone. [title] Paint a layer of a second coat of the same colour on your nails, such as black red.","prompt":"[header] How to do striped nail art with rhinestones [title] Paint two coats of your first polish on your nails. [step] Choose a colour that goes well with a black design on top, such as a deep red. [title] Once the red has dried, take your black striper.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5121,"error":null,"output":"D"}
{"question_id":"hellaswag:00000342","item_index":34,"attempt":0,"prompt_hash":"2fddf31e3a3a687c7ac2c9c1245ff512086c10df3c429d4a470fa5bbed08b942","question":"A man in black shirt lifted the barbel and started to carry it up and down for few times, while behind him, two men are removing the metal plates on another barbel. the man in black shirt\n\nA. removed the plate and raised the barbel, while people standing next to him held the barbel heads.\nB. began to lift the barbel feet and lower them while the people that are standing at three barbel also lifted the barbel, and then lowered it.\nC. lifted the barbel and something inside it flipped off and the other man stopped and get the stuff and hand it to the other man who tried to carry it up and handed it to him.\nD. added more weight on the barbel by adding yellow plates on both side, then continue weight lifting.","prompt":"A man in black shirt lifted the barbel and started to carry it up and down for few times, while behind him, two men are removing the metal plates on another barbel. the man in black shirt","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5125,"error":null,"output":"D"}
{"question_id":"hellaswag:00006595","item_index":35,"attempt":0,"prompt_hash":"24beebb97ccc0c99dee565214849622b86251453d6ebad673663e6200f7bf654","question":"[header] How to create your own favicon icon [title] Create a favicon that represents your website. [step] The type of website you have should determine the look of your favicon. Try to design something that will adhere to your brand image and that will be recognizable and memorable to people.\n\nA. If you have no idea where to start, then basic writing skills like \" get a javascript personal blog \" will do. *..-* * * *...\nB. There are two types of favicons : [substeps] One that you type-anything that represents a content fact, such as that you're a writer or creator, should look like one of your relevant content. It should be labeled something that your work knows about.\nC. Before you start working on this campaign, write down detailed details related to your favicon. This will help your music content and your website feel special and outrange.\nD. Your favicon will be the first thing that people see when they look at the tabs in their browser and will also appear in people's bookmarks. [substeps] For instance, if you have a food website, choosing a favicon that has a fruit or vegetable as the design may make it more memorable.","prompt":"[header] How to create your own favicon icon [title] Create a favicon that represents your website. [step] The type of website you have should determine the look of your favicon. Try to design something that will adhere to your brand image and that will be recognizable and memorable to people.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5121,"error":null,"output":"D"}
{"question_id":"hellaswag:00008009","item_index":36,"attempt":0,"prompt_hash":"4c21b3cc93ef17d17a46e0cd7e8a7939130390182c6de981138d37391a573f0e","question":"[header] How to make a stuffed animal backpack [title] Choose a stuffed animal. [step] . The center will lose most of its stuffing, so shift stuffing around to see how the head will look when loosely supported.\n\nA. [title] Use a 10 to 20 inch (25 to 30 cm) piece of fabric to divide the four tickets into 2 piles. [step] Always take the longest side and place it under the other (or thinner side) of the group ticket.\nB. [title] Measure the length of the animal's back seam. [step] You will select a zipper at least 4 inches (10.2 cm) shorter than the back seam to allow for 2 inches (5.1 cm) above and below the opening.\nC. [title] Select the animal to make it seem realistic. [step] Try to pick a small animal like a mouse, chipmunk, pig, bird, horse, or deer.\nD. [substeps] Begin by making a stuffed animal bag out of 8-16 large candy canes. The chunky candy canes will help you fasten the bag securely to the body of your stuffed animal; this will prevent the jelly from escaping.","prompt":"[header] How to make a stuffed animal backpack [title] Choose a stuffed animal. [step] . The center will lose most of its stuffing, so shift stuffing around to see how the head will look when loosely supported.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5120,"error":null,"output":"B"}
{"question_id":"hellaswag:00000961","item_index":37,"attempt":0,"prompt_hash":"3a91ae112a47dc8087a4985212224d5877b271b41e869a4f81382f4a5595f6d3","question":"A man is bent down on a mat. he\n\nA. grabs a lacrosse ball and throws it onto the mat.\nB. lifts a ball up into the air.\nC. holds an orange ball with several other pins in it.\nD. picks up a large weight and lifts it over his head.","prompt":"A man is bent down on a mat. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5124,"error":null,"output":"D"}
{"question_id":"hellaswag:00004938","item_index":38,"attempt":0,"prompt_hash":"15804a38aeae03229bef526229562c00405df478ac67bc843a4fa0158648f1b4","question":"[header] How to introduce a new puppy [title] Identify what the puppy will need and who will provide these needs. [step] Puppies require a lot of care, love, and patience. If you are bringing a new puppy into your home, you should take some time to identify what its needs will be and decide who will be responsible for each of these needs.\n\nA. [substeps] Puppies under six weeks are often responsible for between two and four days of daily care. Puppies of eight weeks and under two weeks require daycare, including attention.\nB. Let the puppy know that you look after it and feel safe helping it meet your expectations. This may help to reassure the puppy that you will provide it with everything it needs.\nC. Caring for a dog is a great way to teach children responsibility and compassion for animals. Try to get the whole family involved in caring for your new puppy.\nD. [substeps] Dogs are very vulnerable to physical and genetic injuries. A household dog may not have the appropriate amount of time to play with the dog, exercise, and socialize the puppy.","prompt":"[header] How to introduce a new puppy [title] Identify what the puppy will need and who will provide these needs. [step] Puppies require a lot of care, love, and patience. If you are bringing a new puppy into your home, you should take some time to identify what its needs will be and decide who will be responsible for each of these needs.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5120,"error":null,"output":"C"}
{"question_id":"hellaswag:00001499","item_index":39,"attempt":0,"prompt_hash":"65fe788afc5b4b3a6333d2340e59350358ea5595809bf19fc442dcc991b5fff3","question":"One of the body builders lifts the weight bar several times and then drops it down while the other two body builders watch him. then he\n\nA. goes and wipes off his sweat with a towel and lifts the weight bar again and drops it.\nB. releases the weight bar, as well as the beam he was holding, and drops it down near the floor.\nC. continues to lift for another couple of seconds and then drops it and the body builders fall.\nD. runs towards the weight as the others fall with the weight and watch him drop down.","prompt":"One of the body builders lifts the weight bar several times and then drops it down while the other two body builders watch him. then he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5123,"error":null,"output":"A"}
{"question_id":"hellaswag:00007124","item_index":40,"attempt":0,"prompt_hash":"f365dc4a7aafe3c6ec45e9056f2dcb8498b8ade5c1641793eae1b877287d525e","question":"[header] How to make clumpy eyelashes [title] Choose a thick, wet mascara. [step] When you want clumpy lashes, it's important to start with the right mascara formula. Opt for a thick, wet mascara because it makes it easier for the lashes to stick together.\n\nA. You may want to prepare your lashes a little longer after your lashes are made so they aren't covered in mascara when you apply makeup. [substeps] Fresh mascara is generally best because that means it's not sugary or greasy, but a very hydrating solution that can keep your lashes in place between appointments.\nB. [substeps] Choose a mascara that's an equal shades of red, red and blue for the best results. Use compact mascara as well if you want extra-thick mascara.\nC. [substeps] Mascaras that are advertised as having a volumizing or thickening formula tend to be thicker and wetter than lengthening formulas. Mascara tends to dry out as you use it because air gets inside the tube.\nD. If you don't want to use a thick mascara, you can use a light mascara instead. [substeps] Use a small, clean makeup brush to apply mascara to the corners of the eyes.","prompt":"[header] How to make clumpy eyelashes [title] Choose a thick, wet mascara. [step] When you want clumpy lashes, it's important to start with the right mascara formula. Opt for a thick, wet mascara because it makes it easier for the lashes to stick together.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5121,"error":null,"output":"C"}
{"question_id":"hellaswag:00002448","item_index":41,"attempt":0,"prompt_hash":"41729793a577db756928fb3e483cb3a16875a4c329eb422acc8a270c0cae82e8","question":"Clips are shown of men hanging off of buildings washing windows while another man speaks to the camera. the men\n\nA. wash windows all along a building and stop to smoke cigarettes and speak to one another.\nB. blow the windows with spray and clean their hands thoroughly using first spray on each building, start again by spraying an electronic coating on every window as well as the sidewalk.\nC. continue cleaning while a man blows leaves in front of them and is seen driving.\nD. wash a car in a restaurant while the other speaks.","prompt":"Clips are shown of men hanging off of buildings washing windows while another man speaks to the camera. the men","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5122,"error":null,"output":"A"}
{"question_id":"hellaswag:00007878","item_index":42,"attempt":0,"prompt_hash":"3ea4108acf02cf555b3e8d8f10134bf355374e6bf98099798ba65c525b2eaf0a","question":"[header] How to travel by train from los angeles to san francisco [title] Plan a leisurely trip. [step] The coast starlight it not prized for its on-time record, but it is famous for its beautiful views. The trip itself takes approximately 12 hours, from 10 a.m. to 10 p.m., but that can vary based on amtrak train schedules.\n\nA. On-line train trains run from about 6 p.m. to 3 p.m., so expect a sightseeing option.\nB. [title] Book your ticket on the amtrak website. [step] You are encouraged to book early and during the week.\nC. For rail travel : [substeps] Plan to travel at the end of the day. Areas where sunset express travel runs, know to take a train from inside los angeles to san francisco station.\nD. California rail systems offer a variance of between 12 a.m. and 11 p.m., available for all major rail travel companies.","prompt":"[header] How to travel by train from los angeles to san francisco [title] Plan a leisurely trip. [step] The coast starlight it not prized for its on-time record, but it is famous for its beautiful views. The trip itself takes approximately 12 hours, from 10 a.m. to 10 p.m., but that can vary based on amtrak train schedules.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5119,"error":null,"output":"B"}
{"question_id":"hellaswag:00007667","item_index":43,"attempt":0,"prompt_hash":"f808c842162ddb7f1bc99212376a33b370d502646a9295be9d52d31659631c62","question":"[header] How to grow a goatee [title] Make sure you have the face for a goatee. [step] A goatee is a good beard to grow if you have a weak chin, or a rounder face. It can help cover up and fill out that part of your face.\n\nA. A goatee is a versatile beard that can look good on many faces, though you may find another beard type that will look better on you. [title] Consider the contrast between your skin tone and hair color.\nB. [substeps] Think about whether you'd want people to see your goatee. You want it to look like it's really growing out from the sides of your nose.\nC. If you don't have a strong chin, or it doesn't really fit you, invest in a beard. A goatee is best paired with long hair and a nice outfit.\nD. [substeps] Think about trimming your beard a short length first, about 1.5 inches (3.3 cm). Cut the sides back and forth as if you were pulling hair from your chin.","prompt":"[header] How to grow a goatee [title] Make sure you have the face for a goatee. [step] A goatee is a good beard to grow if you have a weak chin, or a rounder face. It can help cover up and fill out that part of your face.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5119,"error":null,"output":"A"}
{"question_id":"hellaswag:00003927","item_index":44,"attempt":0,"prompt_hash":"11324e9d1a8287390a43a3259d932dadeb4ac6cbc1bc99bbeea2348b5de0181a","question":"[header] How to catch bullhead catfish [title] Tie a hook on to the end of your line using a regular clinch knot, multiple lines are an option though bullheads usually don't travel in schools. [step] If you don't know the depth of your fishing area multiple hooks at different lengths are an option. Size 2 hooks are good for bullhead and you won't catch a sunny usually.\n\nA. [title] Change your bait every two months so that you get enough of an open area to catch. [step] You will be fishing many different types of bullheads so learn all of the general basics : [substeps] Pick a size 4.\nB. If you do catch a sunfish, or any other fish, you can cut it into chunks. [title] Put a split shot sinker on the line.\nC. [title] The biggest reason bullheads swim are their fins. [step] Their fins are the spikes at the top of the surface of the water.\nD. You can also use a long line in a pinch if you are interested in bold against the soft side. Suckers are a good size but not as appealing as bullheads, because they are preferred in smaller schools.","prompt":"[header] How to catch bullhead catfish [title] Tie a hook on to the end of your line using a regular clinch knot, multiple lines are an option though bullheads usually don't travel in schools. [step] If you don't know the depth of your fishing area multiple hooks at different lengths are an option. Size 2 hooks are good for bullhead and you won't catch a sunny usually.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5120,"error":null,"output":"B"}
{"question_id":"hellaswag:00006146","item_index":45,"attempt":0,"prompt_hash":"f2d48e5c3c8fc2abe32fcec6d53061fca3f9e204f61b3fd5818403c62f4a4c41","question":"[header] How to have clean and clear skin [title] Wash your skin regularly. [step] Cleanse the skin on your face and body regularly. This can get rid of excess dirt and oil that makes skin look dingy and may also clog pores and cause acne.\n\nA. [substeps] Your body needs a clean, gentle cleaning to look clean, but consider getting rid of any oils that may have built up over time. Clean your skin with a cleanser made with natural ingredients such as coconut oil, chamomile, and cetirizine instead of regular soap.\nB. [substeps] While washing daily is necessary, it can help you stay healthy and clean. If your skin tastes bad or oily, consider a gentle cleanser, mild facial cleanser, or facial moisturizer with spf.\nC. Using a gentle cleanser every so often may help you to keep your skin looking fresh and flawless, including removing wrinkles and pimples. [substeps] Using a mild cleanser may also prevent breakouts.\nD. [substeps] Wash with a gentle cleanser that has a neutral ph. Use natural products if you prefer to wash your skin without chemicals.","prompt":"[header] How to have clean and clear skin [title] Wash your skin regularly. [step] Cleanse the skin on your face and body regularly. This can get rid of excess dirt and oil that makes skin look dingy and may also clog pores and cause acne.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5119,"error":null,"output":"D"}
{"question_id":"hellaswag:00002777","item_index":46,"attempt":0,"prompt_hash":"03f555689a6f7c7fac92952999f901f0b13fb7b21ed7d181548faa7229bea227","question":"A male athlete is running on a track. He jumps high into the air. He vaults backwards over a high beam. this\n\nA. is another example of where the athletes went long distances.\nB. is shown repeated in slow motion numerous times.\nC. is followed by several more flips and twists.\nD. is repeated several times throughout the game.","prompt":"A male athlete is running on a track. He jumps high into the air. He vaults backwards over a high beam. this","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5121,"error":null,"output":"C"}
{"question_id":"hellaswag:00000502","item_index":47,"attempt":0,"prompt_hash":"8fb5c8dec54f9319bbaf15d1e950ee1a1c7f20d33f4b9141043b3f59f4ce9dc0","question":"A person is seen riding on roller blades looking into a trash can. people\n\nA. are seen riding and hitting the sand while the camera captures them by adjusting.\nB. ride around the sidewalk and push down at the bottom of the trash can while laughing to the camera.\nC. stand around the camera looking around.\nD. look around and watch the man skate.","prompt":"A person is seen riding on roller blades looking into a trash can. people","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5120,"error":null,"output":"C"}
{"question_id":"hellaswag:00004181","item_index":48,"attempt":0,"prompt_hash":"7c0905157c7a2be6bfbf234090a6fb52ebb8bcc8f92f5c392a26a6b6090d79ec","question":"[header] How to call australia from the usa [title] Locate the city code of the city you are calling. [step] In australia, each city has an individual city code. You will need to dial the area code during the calling process.\n\nA. [substeps] For example, if you are calling australia from the united states , 115 is the city code of 1 , 149. As such, the city code would fall within this 15 digit area code.\nB. For example, you will dial the sydney area code. [substeps] If you are calling australia from a landline cell phone, dial the area code of the cell phone sitting at the bottom left corner of the screen.\nC. Different countries have different codes, but generally your start call priority code will be 041. If calling from a landline, there is an option to dial 041.\nD. You should be able to locate the area online with a simple google search. [substeps] For example, melbourne's city code is 3.","prompt":"[header] How to call australia from the usa [title] Locate the city code of the city you are calling. [step] In australia, each city has an individual city code. You will need to dial the area code during the calling process.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5119,"error":null,"output":"D"}
{"question_id":"hellaswag:00004559","item_index":49,"attempt":0,"prompt_hash":"3fae260fad4f51d93a6459df3da8fc067ca8272b9a64f798d97a500e6ed23953","question":"[header] How to make your own anti aging creams with vitamin c [title] Dissolve vitamin c powder using distilled water. [step] Mix a ½ teaspoon of vitamin c powder with 1 tablespoon (14. 8 ml) of distilled water in a small container. Make sure to mix the ingredients together until the powder is completely dissolved, to avoid a grainy mixture.\n\nA. [substeps] Do not use vitamin c powder: it will not dissolve in water on its own, and can be runny. If you don't like the consistency, you can try diluting the powder in a small amount of water.\nB. [substeps] The amount of use it takes depends on how flavorful your target acne acne is. It may take some experimentation on your part to find the right mix that works best for you.\nC. [substeps] Only use distilled water, not just filtered tap water or plain tap water. This is because the high oxygen levels in tap water can break down the ingredients faster.\nD. [substeps] If you don't mind the consistency of some of the crystals, consider omitting the red powder entirely. The color and aroma of the paste will attract more flies and sap from the skin.","prompt":"[header] How to make your own anti aging creams with vitamin c [title] Dissolve vitamin c powder using distilled water. [step] Mix a ½ teaspoon of vitamin c powder with 1 tablespoon (14. 8 ml) of distilled water in a small container. Make sure to mix the ingredients together until the powder is completely dissolved, to avoid a grainy mixture.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5119,"error":null,"output":"C"}
{"question_id":"hellaswag:00003012","item_index":50,"attempt":0,"prompt_hash":"708365dc3ce2a2a46fe7ed86bbe69ae27f595a9ee292a38d7a094ca314bfaae1","question":"An introductory title screen with the video description is shown. an individual\n\nA. sniffs a needle and proceeded to polish the drawn out ball of the pinata with a blue polish.\nB. appears riding a horse on a field alongside a bull and using a blow torch to spark a firestorm throughout the horse's mane.\nC. drizzles olive oil into a pan.\nD. is being brushed by a hand and gives a tutorial on how to brush your hair.","prompt":"An introductory title screen with the video description is shown. an individual","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5120,"error":null,"output":"D"}
{"question_id":"hellaswag:00002078","item_index":51,"attempt":0,"prompt_hash":"45eba215b83a81946f5aacd72a6bd7c8e1d3692015fa1e83a8f62c64bc1003e9","question":"Then the man wearing the black shirt serves to the boy whose wearing the gold shirt. then the boy who's wearing the white shirt\n\nA. serves to the boy who has the gold shirt in his hands.\nB. serves the serve to the player who's wearing the gold shirt.\nC. hits the shuttlecock and serves to the girl who's wearing the pink shirt and the game goes on and everyone keeps serving to each other and they all use different shuttlecocks.\nD. serves to put the blue balls in the hole.","prompt":"Then the man wearing the black shirt serves to the boy whose wearing the gold shirt. then the boy who's wearing the white shirt","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5120,"error":null,"output":"B"}
{"question_id":"hellaswag:00006396","item_index":52,"attempt":0,"prompt_hash":"e77c2832b28c23944ae96e261417440c3dc09923665b18a7a306f4992b9cfa3e","question":"[header] How to do kinky twists [title] Wash and deep condition your hair as you normally would. [step] Since this style will last about 4 to 6 weeks, it would be a good idea to wash your hair and scalp first with a clarifying shampoo to remove any build-up and product residue. Follow-up with a deep conditioner.\n\nA. This will make your hair even healthier because your scalp is already coated in natural oils. [substeps] You can also purchase hair products, like deep conditioner, for kinky hair at your local drugstore or a beauty store.\nB. [substeps] Use the deep conditioner according to the instructions on the label. Most will need to sit in your hair for only 3 to 5 minutes, but some may need to sit for much longer.\nC. [substeps] Parts of your hair that can easily be washed with a little clarifying shampoo include the back of your head and between your ears. [title] Wash each hand at least twice a day.\nD. [title] Locate your relaxer area at the back of your head. [step] This is where you'll be creating the back part.","prompt":"[header] How to do kinky twists [title] Wash and deep condition your hair as you normally would. [step] Since this style will last about 4 to 6 weeks, it would be a good idea to wash your hair and scalp first with a clarifying shampoo to remove any build-up and product residue. Follow-up with a deep conditioner.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5119,"error":null,"output":"B"}
{"question_id":"hellaswag:00006353","item_index":53,"attempt":0,"prompt_hash":"6cdc035da11cdd724956596105d486b36d57011c9d6743c01ae0f0a8284f8aa4","question":"[header] How to tell a friend that his or her partner is cheating [title] Be sure. [step] Before you jump the gun and start dialing your friend to tattle, make sure what you saw was actually cheating. You'll stir the pot for no good reason if it turns out your friend's partner is simply having lunch with a coworker or a distant relative.\n\nA. Carefully evaluate what you saw just in case you are reading into something completely innocent. [substeps] Was there hand-holding, hugging, or kissing involved? What about the interaction makes you think your friend's partner is cheating? [title] Get firm evidence.\nB. [substeps] After you've heard the true item is a conjunction, you need to know what the other person says and how you might solve the problem. Talk to your friend or family member by asking them before mentioning about the secret affair.\nC. If your friend has a different character, you'll have to think more about whether it really is the case. [substeps] Know that in some cases how significant an assignment results in your friend finding a new partner.\nD. [substeps] Make sure that you know if your friend is going to share the secret with another person. You may have news that you didn't actually know.","prompt":"[header] How to tell a friend that his or her partner is cheating [title] Be sure. [step] Before you jump the gun and start dialing your friend to tattle, make sure what you saw was actually cheating. You'll stir the pot for no good reason if it turns out your friend's partner is simply having lunch with a coworker or a distant relative.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5119,"error":null,"output":"A"}
{"question_id":"hellaswag:00006449","item_index":54,"attempt":0,"prompt_hash":"4fe17954725fbc4fbe8038efc6f5d7c7bb68c8b3dde1174e973d1b9fbb857e27","question":"[header] How to remove gel nail polish [title] Find a place in your home where you do not risk ruining clothing, furniture or other items with acetone. [step] Acetone is a solvent that is required to eliminate gel polish. [title] Place a paper mat or cereal box in the area you plan to use.\n\nA. [step] Mark your place on the box so it does not get wet. [title] Apply petroleum jelly to the area you will cut.\nB. [step] Place one corner of the box on a padded surface. Ensure that it is not bigger than the rubber lip grip that you would use to remove gel polish.\nC. [step] Put it near a sink, if possible. [title] Tear aluminum foil strips to fit around the tops of your nails.\nD. [step] Stay within arm's length of the affected area to prevent it from contaminating your clothing or furniture. [title] Dip a cotton ball or clean toothbrush or a small paper plate into acetone.","prompt":"[header] How to remove gel nail polish [title] Find a place in your home where you do not risk ruining clothing, furniture or other items with acetone. [step] Acetone is a solvent that is required to eliminate gel polish. [title] Place a paper mat or cereal box in the area you plan to use.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5258,"error":null,"output":"D"}
{"question_id":"hellaswag:00007095","item_index":55,"attempt":0,"prompt_hash":"36cb820f6038d96dd1676d8123ad6d07dab48fb639940740c1ac4577a6981f56","question":"[header] How to connect a guitar or bass guitar to a macbook [title] Consider a di box. [step] You'll need some kind of interface or adapter that takes the signal from the guitar and gets it to the computer. A di is a small amplifier which takes the (quiet) signal coming from your instrument and amplifies it before it goes into your macbook.\n\nA. Your computer computer would have the di box and macbook connecting to it. [title] Cover the macbook with plastic wrap.\nB. [substeps] Di box's seats into the instrument and can be switched if you don't already have them. [title] Place your guitar on your lap or directly upon your room's floor.\nC. Dis come in either active or passive forms. Most six string guitars are passive by design (no amplification inside the instrument itself requiring a battery) and thus will require an active di.\nD. [title] Put a lid on the macbook, use a cable or screw in the front), and plug the macbook onto the auxiliary ports. [step] Slip the finder bar on the front of the box into its proper position.","prompt":"[header] How to connect a guitar or bass guitar to a macbook [title] Consider a di box. [step] You'll need some kind of interface or adapter that takes the signal from the guitar and gets it to the computer. A di is a small amplifier which takes the (quiet) signal coming from your instrument and amplifies it before it goes into your macbook.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5258,"error":null,"output":"C"}
{"question_id":"hellaswag:00000505","item_index":56,"attempt":0,"prompt_hash":"262260be89c57acd36161f20cde4d3693169ea2caa09954e079ae32402a0e422","question":"A shirtless guy enters a bathroom and yawns. The guy brushes his teeth. The guy drinks a beverage. The guy now fully clothed dances. the guy\n\nA. places his hand behind his neck and begins to disrobe.\nB. climbs out of bed and shaves his legs.\nC. continues to brush his teeth.\nD. poses with the beverage.","prompt":"A shirtless guy enters a bathroom and yawns. The guy brushes his teeth. The guy drinks a beverage. The guy now fully clothed dances. the guy","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5258,"error":null,"output":"C"}
{"question_id":"hellaswag:00000779","item_index":57,"attempt":0,"prompt_hash":"1f009cf783dcee29d22bcd17ee626f87067140643b2ea345b2abeb87ecb4cd6b","question":"A man is deep water scuba diving with some type of machine as if he is searching for something. in the midst\n\nA. of scuba diving he talks and the forseti on the screen shows his location as the man takes time out sitting in a boat.\nB. , he begins playing with a small fish before continuing on his search.\nC. of the scuba gear a man is shown doing a sit up get off the floor.\nD. of the water, he falls.","prompt":"A man is deep water scuba diving with some type of machine as if he is searching for something. in the midst","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5257,"error":null,"output":"B"}
{"question_id":"hellaswag:00003735","item_index":58,"attempt":0,"prompt_hash":"267dcb9ef686fecde6f4e80831934dd33664967e00b314bf47317fc257c90549","question":"[header] How to whitewash wood [title] Clean and sand the wood. [step] Remove any stickers or unwanted nails from the wood. If it is dirty, wash the wood with gentle soap and water.\n\nA. Dry well with a clean rag or paper towel. [substeps] If the wood is severely discolored or hasn't had a chance to dry completely, sweep it away.\nB. If the wood is very rough, use sandpaper or a hand sander to smooth it. A smoother surface will make it easier to apply the whitewash.\nC. Then, sand in the direction the wood grain is in. [substeps] Use a large circular brush or a handheld scraper to sand the wood.\nD. Try to remove anything that might have come off of the wood. Also, re-wash and sand down any areas that have become bare or need a polish or touch-up.","prompt":"[header] How to whitewash wood [title] Clean and sand the wood. [step] Remove any stickers or unwanted nails from the wood. If it is dirty, wash the wood with gentle soap and water.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5257,"error":null,"output":"B"}
{"question_id":"hellaswag:00008570","item_index":59,"attempt":0,"prompt_hash":"377ae125610cbadca52a088a1127cad87149193ab5015ec907600edc61b5058b","question":"[header] How to write a retirement letter [title] Discuss your retirement plan with hr. [step] Set up a meeting with a representative of the human resources (hr) department to discuss the details of your retirement plan. Doing this before you write your letter will help you to get an idea about how to shape your letter.\n\nA. [substeps] Ask hr how you can change your plan a few months in advance or how you can update your plan later. Don't simply rely on your old plans to offer changes after a few months have passed.\nB. Bring a few financial documents as well as a paper clip so that you can bring the documents in your envelope. [substeps] This is a good time to talk with hr at the appointed meeting.\nC. An hr representative is an employee who travels the world. He or she will be able to review your retirement plan and facilitate changes to make it easier to accept.\nD. [substeps] When you meet with hr, bring any questions you have with you and take notes during the meeting. Jot down important details, as well as details you feel might pertain to your letter.","prompt":"[header] How to write a retirement letter [title] Discuss your retirement plan with hr. [step] Set up a meeting with a representative of the human resources (hr) department to discuss the details of your retirement plan. Doing this before you write your letter will help you to get an idea about how to shape your letter.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5452,"error":null,"output":"D"}
{"question_id":"hellaswag:00008363","item_index":60,"attempt":0,"prompt_hash":"5aa7d4dece529e727e2f7386b3b4290cbdb6dbf80d478e6b27b29bf9cbdcf68d","question":"[header] How to drive in the snow [title] Prepare your vehicle for driving. [step] Clear the windshield, side windows, headlights, and rear brake lights of all snow and ice with an ice scraper and brush before you drive off. Stop the car and clear the windows and lights as needed while driving during snowfall.\n\nA. You can buy snowblower helmets or winter hats from most retail stores. For better access, get snow pants and boots.\nB. Don't forget to clear snow off of your number plate, as it is a legal requirement for your number plate to be visible in many places. [title] Turn on the front and rear defroster to keep windows clear while you drive.\nC. This will help you if you are going uphill or downhill. [substeps] Use a snow scraper to remove snow from grooves in the top of the windshield and headlights.\nD. [title] Block all leaking fluids from your vehicle with plastic, ash, gravel, and paper cover. [step] This provides protection in case of rain or snow, but as a general rule you should also clear excess air into your vehicle.","prompt":"[header] How to drive in the snow [title] Prepare your vehicle for driving. [step] Clear the windshield, side windows, headlights, and rear brake lights of all snow and ice with an ice scraper and brush before you drive off. Stop the car and clear the windows and lights as needed while driving during snowfall.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5454,"error":null,"output":"B"}
{"question_id":"hellaswag:00004201","item_index":61,"attempt":0,"prompt_hash":"942905f573cb7cf2133a88b04f8665e7561ec26f693845cc288764c61a85ec2f","question":"[header] How to teach your children a second language [title] Start teaching your child as early as possible. [step] Children have an easier time learning a second language the earlier they begin. If possible, expose your child to a second language when they are still an infant.\n\nA. If any family members are fluent, encourage them to speak the language in the home in order to start immersing your child in the second language. [title] Be a patient, empathetic teacher.\nB. [substeps] Many children first language learners discover a second language at age 6. This is referred to as the \" child language assessment and proof process.\nC. [substeps] If your child is 4 or 7 years of age, teaching them will give them a developed understanding of the second language. Start young so your child can be first in the class.\nD. Avoid selecting a language that your child is already adapted to and brainstorm lessons the child can learn. [title] Create a pre-language foundation with each child.","prompt":"[header] How to teach your children a second language [title] Start teaching your child as early as possible. [step] Children have an easier time learning a second language the earlier they begin. If possible, expose your child to a second language when they are still an infant.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5454,"error":null,"output":"A"}
{"question_id":"hellaswag:00005670","item_index":62,"attempt":0,"prompt_hash":"f5cc1fece28f5d21b6ac75a418b71d4d3ab668a669784cef5cde13577c297380","question":"[header] How to dry sperrys [title] Wipe away excess moisture. [step] Use a clean, dry towel to absorb any water remaining on the shoe. Make sure pat down the footbed and underside of the instep section, as well.\n\nA. Then, place the insole no higher than the box portion and pat it down with a paper towel, now more absorbent. [substeps] The box portion of the shoe needs to be dry too, so put off the splash of water if possible.\nB. When you're done, simply leave them as-is. If the uppers are made from canvas, you can move on to a more intensive drying method.\nC. [title] Use rubber-based cleaner if your shoes are leather. [step] If your shoes are leather-based, rubber based cleaners might also be helpful.\nD. [title] Use your shoe dirkling tool for cleaning. [step] Hand sanitizer and rubbing alcohol all have helpful tools that will clean and sanitize the sole of your sperrys.","prompt":"[header] How to dry sperrys [title] Wipe away excess moisture. [step] Use a clean, dry towel to absorb any water remaining on the shoe. Make sure pat down the footbed and underside of the instep section, as well.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5454,"error":null,"output":"C"}
{"question_id":"hellaswag:00009612","item_index":63,"attempt":0,"prompt_hash":"72ee74c1ef85f8164778585da0a0ed4503bcae1d7d95ff25d624461bd45f9677","question":"[header] How to make cigar box purses [title] Find or purchase a wooden cigar box. [step] While unfinished cigar boxes can be bought from many craft stores, you may be able to find them for free by asking at cigar stores, liquor stores, bars, or any place else that sells cigars. Make sure the box is not warped or damaged.\n\nA. [title] Cut a long strip of wood long enough to cover the end of the cigar box. [step] Tie a bow into place around the inside.\nB. [substeps] Cigar boxes made at least 22 or 44 inches (60 to 90 cm) in size should never be opened. Cigars should always be wrapped in aluminum foil or tin foil.\nC. [substeps] Be sure the cigar box has a lid, otherwise you risk shrinkage. [title] Clean the cigar box thoroughly and replace any intact pieces.\nD. [substeps] Use only wooden cigar boxes or other wooden craft boxes, which are available at most craft stores, some discount or hardware stores, or from online craft sites. Do not use cardboard cigar boxes, as these will not work well for this project.","prompt":"[header] How to make cigar box purses [title] Find or purchase a wooden cigar box. [step] While unfinished cigar boxes can be bought from many craft stores, you may be able to find them for free by asking at cigar stores, liquor stores, bars, or any place else that sells cigars. Make sure the box is not warped or damaged.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5454,"error":null,"output":"D"}
{"question_id":"hellaswag:00005754","item_index":64,"attempt":0,"prompt_hash":"85eaeadfb44bfc6c755d476bcf27d7b0b59a4cfe8beb38a03304f8c58241a14a","question":"[header] How to write fan mail [title] Introduce yourself. [step] Include your name, where you are from, and your age in the letter. Mention your personal connection to the person or the work that they do.\n\nA. [substeps] For example, you could write, \" hi! I'm russell lang, your editor! \" [title] Talk about the publication. [step] You can send a fan letter on other social media platforms, like the fanbase on youtube.com or pinterest.\nB. Provide as many details about your company as possible so that it will be easier for the receiver to determine who you are. [substeps] If you were hiring someone to design a banner, then include a brief account of that job.\nC. [substeps] You don't need to give your name unless you are both the copyright owners of the work or are a beyonce author. You don't want the reader to know who you are, so you need to keep them guessing about your name.\nD. You can also mention anything that you and the person may have in common. [substeps] If you are writing to your favorite basketball player, you might say, \" hello, my name is _ _ _.","prompt":"[header] How to write fan mail [title] Introduce yourself. [step] Include your name, where you are from, and your age in the letter. Mention your personal connection to the person or the work that they do.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5886,"error":null,"output":"D"}
{"question_id":"hellaswag:00004527","item_index":65,"attempt":0,"prompt_hash":"4809ddf067d66afd97c0874cbb5dc4e197d4aa76fdc1da9c1c239b269a854c64","question":"[header] How to get a domestic partnership [title] Determine if you able to get a domestic partnership. [step] The laws regarding domestic partnerships are different in various locations. Because domestic partnerships are not uniformly created, it is challenging to locate the statutes.\n\nA. Some states let domestic partnerships coexist under mandated statutes. Other states allow domestic partnerships to be dissolved in common situations such as divorce, child custody, or annulment.\nB. [substeps] If you live in a quiet area, you may be able to establish the rules with a few simple emails and letters. If you live in a more public area, you may be able to establish non-public partnerships by cashing-in accounts with friends and family.\nC. Some examples of domestic partnerships include : [substeps] Owners of sole proprietorships income of one of a minority joint owner of the full proprietorship or bank accounts that are endorsed or subsidized by the full spouse. Agencies such as sole proprietorships or mortgages state that owners must meet certain requirement parameters to get a domestic partnership.\nD. Start with the state statutes and search for domestic partnerships. If you find nothing, look at codes for your local county/parish and city, again, searching for domestic partnerships.","prompt":"[header] How to get a domestic partnership [title] Determine if you able to get a domestic partnership. [step] The laws regarding domestic partnerships are different in various locations. Because domestic partnerships are not uniformly created, it is challenging to locate the statutes.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5886,"error":null,"output":"D"}
{"question_id":"hellaswag:00005586","item_index":66,"attempt":0,"prompt_hash":"0ed158ef7bb7b4e6392cf929007f2cb97b2a28bd5c9fd203b030ca01fe1234ab","question":"[header] How to look great in sweatpants [title] Choose fitted sweatpants. [step] Sweatpants that have a tailored look are on-trend and turn the \" sweatpants are for slobs \" bias on its head. A slim fit and tapered leg are key for getting this look right.\n\nA. [title] Choose leggings that cover only your leg. [step] Avoid baggy pants when choosing fitted sweats.\nB. The pants will still have a casual, slouchy look, but they should fit more like a pair of trousers than the oversize sweats you wear when you're home sick. [substeps] Large, baggy, or oversize sweatpants generally aren't flattering.\nC. Sweatpants that sag at the ankle are downright trashy. [substeps] Wrap up your jeans so you can dress as you normally would.\nD. [substeps] Sweats with vertical stripes that go all the way up to your knees are good sweatpants. If your legs are naturally curly or the pants are loose fitting around them, don't get them.","prompt":"[header] How to look great in sweatpants [title] Choose fitted sweatpants. [step] Sweatpants that have a tailored look are on-trend and turn the \" sweatpants are for slobs \" bias on its head. A slim fit and tapered leg are key for getting this look right.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5885,"error":null,"output":"B"}
{"question_id":"hellaswag:00003405","item_index":67,"attempt":0,"prompt_hash":"b7b024a89b1147973b2129f85c1abf60dd6f384f74c2cedc63a33ea02003ffd3","question":"[header] How to make a gardener's apron [title] Cut a large rectangle for the apron base. [step] It needs to be 24 inches (60.96 centimeters) wide and 19½ inches (49.53 centimeters) tall. You can use plain or patterned cotton canvas fabric.\n\nA. [title] Cut off the excess wrap at the base of the apron base. [step] Do not cut the excess wrap off completely; cut a small, horizontal tear in the fabric and sew it closed with another, slightly larger tear.\nB. Make sure that the seam running the length of the fabric on both sides is flush with the rest of the fabric, otherwise it might wrinkle. [title] Cut the top half of the fabric inside out.\nC. If you are using patterned canvas, however, be aware that both the right and wrong sides will be visible on the front of the apron. [substeps] You can also find a pattern online to suit your needs and style, if you're not comfortable designing your own apron.\nD. [title] Fold each end of the long rectangle in half. [step] Unfold one side of the long rectangle to reveal the thick edge of the apron base.","prompt":"[header] How to make a gardener's apron [title] Cut a large rectangle for the apron base. [step] It needs to be 24 inches (60.96 centimeters) wide and 19½ inches (49.53 centimeters) tall. You can use plain or patterned cotton canvas fabric.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5885,"error":null,"output":"C"}
{"question_id":"hellaswag:00001745","item_index":68,"attempt":0,"prompt_hash":"5f61a15e34c3d18bc91c675a4ab4c5353badbc634da57f01c15f5aa2200a39cb","question":"A close up of a fishing pole is shown with a person reeling it in. others\n\nA. are shown capturing the fish that he has captured in the area.\nB. are shown mostly off in the water followed by more people reeling the fishing pole in.\nC. kneel around him as he continues to reel.\nD. are seen on the bank watching a calf as it is released.","prompt":"A close up of a fishing pole is shown with a person reeling it in. others","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5889,"error":null,"output":"B"}
{"question_id":"hellaswag:00009372","item_index":69,"attempt":0,"prompt_hash":"a999b2ac7a3b3c3df45802fe91b50e4bbe5eec7f55ec0f823de37f3ac8b9e6f1","question":"[header] How to get silver blonde hair [title] Start with a clean slate, if you can. [step] Virgin hair is best for achieving the platinum blonde look-this means all dye should be grown out of your hair. It might take a while.\n\nA. Be sure to figure out how much, and take some time to train your hair. If you're worried about what to do, go with a hair guidance course.\nB. Hair grows about ½ inch per month, so you'll have about 3 inches of virgin roots in 6 months and 6 inches in a year. If you've dyed your hair recently or still have dark ends, you may find it has an orange tinge after bleaching.\nC. Go to an extension salon and discuss wax, toner, hair oil, tint and so on. Or just pick the cut you like and follow the guidelines.\nD. If you're afraid of being too bubbly or not having the time to relax your hair, then try to get a haircut. You can dye your hair any color you want.","prompt":"[header] How to get silver blonde hair [title] Start with a clean slate, if you can. [step] Virgin hair is best for achieving the platinum blonde look-this means all dye should be grown out of your hair. It might take a while.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5885,"error":null,"output":"B"}
{"question_id":"hellaswag:00005110","item_index":70,"attempt":0,"prompt_hash":"feae9705b44b8695ade1304fbf6ccfa4f2b6490e0a80fa782f8960f7632c67bb","question":"[header] How to deal with frizzy hair [title] Wash your hair only a few times a week. [step] Your hair usually becomes frizzy because it's dry, and shampooing too often can dehydrate it even further. Instead of washing your locks every day, limit it to every other day or less if you can.\n\nA. Washing your hair at least once a week is good. [substeps] Try to use shampoo and conditioner when you wash it.\nB. [substeps] If your roots are starting to look a little greasy in between washes, use a dry shampoo product to help soak up the excess oil. It can also help boost volume and texture if your hair is looking a little flat.\nC. [substeps] Make sure your hair is under running water at least once a week. That way you can wash it at a time that works for you.\nD. [substeps] You can take a shower before and after you wash your hair to prevent dry frizziness from building up. [title] Rinse your hair as you normally would.","prompt":"[header] How to deal with frizzy hair [title] Wash your hair only a few times a week. [step] Your hair usually becomes frizzy because it's dry, and shampooing too often can dehydrate it even further. Instead of washing your locks every day, limit it to every other day or less if you can.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5884,"error":null,"output":"B"}
{"question_id":"hellaswag:00000606","item_index":71,"attempt":0,"prompt_hash":"189df9e328320165a0913526de1d49787cee901e24ed576654e61574effac0a2","question":"A group of guys are sitting on a sectional sofa. One guy stands up and kneels by the coffee table. a second guy\n\nA. takes the weight off his shoulders.\nB. begins playing a saxophone.\nC. kneels across from the first guy at the coffee table.\nD. is holding a gate open.","prompt":"A group of guys are sitting on a sectional sofa. One guy stands up and kneels by the coffee table. a second guy","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5889,"error":null,"output":"C"}
{"question_id":"hellaswag:00000235","item_index":72,"attempt":0,"prompt_hash":"aa72f0c4dc64f9eb09f8b027703ebf0d595a0b3514284922ad9a8a19e9893169","question":"The man begins by painting a wooden armchair metallic gray with a gold spray paint can. The man then paints a side table black with lights shining off of the black painted wood. the man\n\nA. ends with painting a dresser drawer pink before the scene fades to a black marketing graphic.\nB. necks the edge of the table to position the paint brush and brush on it.\nC. turns on the vacuum cleaner to vacuum the hardwood floors of the living room.\nD. proceeds to clean up the stained glass.","prompt":"The man begins by painting a wooden armchair metallic gray with a gold spray paint can. The man then paints a side table black with lights shining off of the black painted wood. the man","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5888,"error":null,"output":"C"}
{"question_id":"hellaswag:00003273","item_index":73,"attempt":0,"prompt_hash":"964cff547b5c99abf6c03e1d8dd7760a006b2b0f7ccbe19d12c6ee2345143eef","question":"[header] How to do letter nails [title] Start with painted nails. [step] Apply a base coat and 1 to 2 coats of your choice of nail polish. Let the first coat dry before applying a second one.\n\nA. [substeps] Nail polish is supposed to be applied at least 1/8 of an hour before the application of your first layer. Different nail polish companies use many different base coats.\nB. Don't worry about applying a top coat just yet. [substeps] This method is great if you want a handwritten font, such as cursive.\nC. Polish several nail nails before applying another coat. [substeps] Next, once you get all your nails painted, you can start marking off the toenails with wood polish.\nD. Allow the polish to dry after each nail application to ensure a clean coat. [title] Choose a base color for your nails.","prompt":"[header] How to do letter nails [title] Start with painted nails. [step] Apply a base coat and 1 to 2 coats of your choice of nail polish. Let the first coat dry before applying a second one.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5889,"error":null,"output":"D"}
{"question_id":"hellaswag:00009007","item_index":74,"attempt":0,"prompt_hash":"0257274e43d6057c730247ae65f2f445b673eb658cc9af0c41799ce778e542fa","question":"[header] How to swing a softball bat [title] Adjust your stance. [step] To get a nice swing, you first need to have the right stance. You should position yourself in the middle of the batter's box to ensure that you can take a swing at anything in the strike zone.\n\nA. [substeps] To begin the swing, make sure that you grip the bat with your non-dominant hand. Clench your thumb and index and ring fingers around the bat handle, then slide it up as far as it will go.\nB. This way, you'll be able to deliver most hits while still letting your pitch move slowly. [substeps] You'll likely want to take a few extra breaths while you adjust your stance.\nC. How you are oriented in the strike zone will dictate how well you swing the bat. [substeps] If you are left-handed, avoid swinging the bat too far to the side, and stick your left arm out.\nD. Place your feet parallel to the direction that home plant is pointing towards, and shoulder-width apart. [substeps] Bend your knees a little, and try to distribute your weight evenly on the balls of your feet.","prompt":"[header] How to swing a softball bat [title] Adjust your stance. [step] To get a nice swing, you first need to have the right stance. You should position yourself in the middle of the batter's box to ensure that you can take a swing at anything in the strike zone.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5885,"error":null,"output":"D"}
{"question_id":"hellaswag:00005822","item_index":75,"attempt":0,"prompt_hash":"40ef2600701f9d6d159ff96e84e9074275f9e7858458f24058abd11584cf901a","question":"[header] How to take hair extensions out [title] Gather all of your hair into a ponytail. [step] Take all of your natural hair as well as any extensions and pull them together into one or two hair ties or big hair clips at the top of your head. If you can, try to group your natural hair and your extensions in separate clips.\n\nA. Secure the top with hair tape, clamp the clips onto the end with plenty of hair screws, and tie your hair back into a secure hair tie. [title] Make sure to use your hair on the top-the middle portion of your hair is best.\nB. [substeps] Tie back hair that is longer or shorter than your natural hair about one or two inches above your left ponytail (this still works, but it will add volume to your hairstyle) to give your natural hair a temporary styling boost. [title] Separate your hair into sections on your head.\nC. [title] Separate a section of your hair from the ponytail. [step] It will be too overwhelming to try and rip out all of your extensions at once, so you want to pull out separate rows of hair at a time.\nD. You can put your hair into a basic, up-do style. [substeps] If it is a hot day, you may not need any clips.","prompt":"[header] How to take hair extensions out [title] Gather all of your hair into a ponytail. [step] Take all of your natural hair as well as any extensions and pull them together into one or two hair ties or big hair clips at the top of your head. If you can, try to group your natural hair and your extensions in separate clips.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5888,"error":null,"output":"C"}
{"question_id":"hellaswag:00003103","item_index":76,"attempt":0,"prompt_hash":"e9048e2261052ce71e5be54dcaad8ec1e95069c42cfdb058f80973b92bc24e76","question":"A large group of people are seen playing instruments with one another. The camera zooms in on a man playing a violin. the man\n\nA. continues playing the flute while people watch on the side.\nB. continues playing with a conductor shown as well as others playing instruments.\nC. continues to play and others sitting in a tent follow on the sides.\nD. continues playing while moving his feet near the players.","prompt":"A large group of people are seen playing instruments with one another. The camera zooms in on a man playing a violin. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5888,"error":null,"output":"B"}
{"question_id":"hellaswag:00006382","item_index":77,"attempt":0,"prompt_hash":"28fb76b91c12d62767b2652f48b9dd629820ec1927b620cc4f4b92e842afc54d","question":"[header] How to dye black hair red [title] Pick the best color for your skin tone. [step] There are many different shades of red that you can dye your hair, including auburn, strawberry, brick, mahogany, and more. Try going with a shade of red that best compliments your complexion, whether you have cool-toned fair skin, warm-toned dark skin, or something in between.\n\nA. [substeps] If you want to avoid having to dye your hair black, keep in mind that other shades are less common than yours. However, don't forget to select a dye that is close to your natural skin tone.\nB. Think about the color that's most closely mirrored your skin tone. This can help you match your hair more closely to your complexion.\nC. [substeps] Hold a sample hair swatch close to your face to see if the color works with your skin tone. If you have pink, red, or bluish undertones, then you have a cool skin tone.\nD. [substeps] Try what looks like a hair color. Dark-toned, fair skinned people tend to have cool tones.","prompt":"[header] How to dye black hair red [title] Pick the best color for your skin tone. [step] There are many different shades of red that you can dye your hair, including auburn, strawberry, brick, mahogany, and more. Try going with a shade of red that best compliments your complexion, whether you have cool-toned fair skin, warm-toned dark skin, or something in between.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5887,"error":null,"output":"C"}
{"question_id":"hellaswag:00001411","item_index":78,"attempt":0,"prompt_hash":"a1d3171e93e397f8445f51a28dcc72cc2d765cc0f2f6629ae2c13fbca4931225","question":"A man is on a galloping horse back riding for sport. Horse back riding is considered to be a sport, a fun one but still work. like\n\nA. working with a team you have to work with a horse also, you have to communicate with a horse that doesn't speak.\nB. riding in fast motion, you more than likely go over 10 km/h, back going slower and slower.\nC. ride horses through a large barn.\nD. riding a horse in a competition an enjoyment or disappointment fails to make you enjoy it.","prompt":"A man is on a galloping horse back riding for sport. Horse back riding is considered to be a sport, a fun one but still work. like","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5888,"error":null,"output":"A"}
{"question_id":"hellaswag:00006046","item_index":79,"attempt":0,"prompt_hash":"7cbbc6b1d2517e4ba612ab0cee87133301629b439826b027902c28d69757acde","question":"[header] How to adjust with in laws [title] Stop stereotyping. [step] The widely accepted stereotype of the \" typical mother-in-law \" can lead into unfavorable mental sets even before marriage. Unfavorable stereotypes about the elderly, that they are bossy and interfering, can add to in-law problems.\n\nA. [substeps] Think back to times when you yourself weren't such a devotee of the elderly. Maybe you were a little kid, too.\nB. Remember that you are now a part of their family. So instead of thinking it in a bad way, try to put yourself in their place.\nC. Here are a few examples of stereotypes to consider : [substeps] The elderly hair is too thin. The elderly respects and pays homage to dogs or cats.\nD. [substeps] Consider using negative stereotypes in subverts. For example, someone might say \" how is jimmy? \" but, they are likely to be defensive about it.","prompt":"[header] How to adjust with in laws [title] Stop stereotyping. [step] The widely accepted stereotype of the \" typical mother-in-law \" can lead into unfavorable mental sets even before marriage. Unfavorable stereotypes about the elderly, that they are bossy and interfering, can add to in-law problems.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5885,"error":null,"output":"B"}
{"question_id":"hellaswag:00003841","item_index":80,"attempt":0,"prompt_hash":"50764a4952519cccb8ea4f29aefbc2d811f17d6f92a02e4b52c864f33dd5626d","question":"[header] How to choose the right life coach [title] Assess your needs. [substeps] Life coaches base their practices on a particular subject area such as business, relationships, grief, mid-life issues, etc. Decide what you need life coaching for.\n\nA. Many people experience more than one need for life coaching, and you can find a life coach through networking or in-person television programs. As long as you know what you want in life before you pursue it, the way is the same.\nB. Do you want to deal with a crisis, a torn relationship, or you just want to have an honest conversation? Understanding what you need from life coach can lead to more meaningful personal communication. [title] Review your current training (includes board licensing).\nC. People who have problems with struggling relationships also need more experienced players. You can find the right coaches by searching online.\nD. If you are having problems with your spouse or children, you don't want to choose a personal life coach whose focus is on weight loss. Understand that hiring a life coach is a personal investment that can be costly.","prompt":"[header] How to choose the right life coach [title] Assess your needs. [substeps] Life coaches base their practices on a particular subject area such as business, relationships, grief, mid-life issues, etc. Decide what you need life coaching for.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5887,"error":null,"output":"D"}
{"question_id":"hellaswag:00003707","item_index":81,"attempt":0,"prompt_hash":"7458e025ddfa0f11e2786ad0b55a5550d1a123f73583a697c92193420c50942b","question":"[header] How to clean a kitchen faucet [title] Clean soft stains with water. [step] You should be able to remove small smudges without any cleaning solution. Simply wiping the faucet down with a wet rag should be enough to remove most soft stains.\n\nA. [substeps] You can use a soft cloth to wipe down the faucet if you don't want to use a dish soap and antibacterial cleanser. [title] Clean tap and faucet accessories.\nB. For the easiest removal, you can use the same method you use to clean brass knobs. [title] Vacuum around the faucet to remove dust and debris.\nC. [title] Let the water drain out of the faucet. [step] You can use either a gentle pressure, a mechanical hose, or a combination of both.\nD. Always blot the faucet dry after it gets wet. [title] Use a gentle cleaner to remove harder stains.","prompt":"[header] How to clean a kitchen faucet [title] Clean soft stains with water. [step] You should be able to remove small smudges without any cleaning solution. Simply wiping the faucet down with a wet rag should be enough to remove most soft stains.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5885,"error":null,"output":"D"}
{"question_id":"hellaswag:00001661","item_index":82,"attempt":0,"prompt_hash":"aa2c496a92c80b058574568d241dea47d1bedbfed8440e6b85f2cc698d510282","question":"A man plays the tam-tams in a room. The man plays only in one tam-tam. Then, the man plays the two tam-tams. after\n\nA. , the man plays the first tam-tam.\nB. , the man plays the two tam-tam.\nC. , person walks nice walk on a front of the left foot of the tam-tam.\nD. , the man finish playing the instruments.","prompt":"A man plays the tam-tams in a room. The man plays only in one tam-tam. Then, the man plays the two tam-tams. after","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5887,"error":null,"output":"A"}
{"question_id":"hellaswag:00008010","item_index":83,"attempt":0,"prompt_hash":"5bea83ac464626841ca898f8f3e1a9d7123ea74a48c5623627df76817add78ea","question":"[header] How to house train a puppy [title] Understand how your puppy views the environment. [step] Puppies don't have an inherent understanding of what humans consider right and wrong. They can learn many behaviors, though.\n\nA. Playing with your puppy, spending lots of time with it, and handling it require careful eating skills. Here are some of the most important behaviors your puppy can learn : [substeps] Follow your gut.\nB. Some basic behaviors for puppies to learn include : [substeps] Independence. Puppies will age faster if they can stand and walk two times every day.\nC. A puppy doesn't understand that it is \" bad \" behavior to urinate on your carpet. To your puppy, the carpet is an acceptable surface, just like the grass outside.\nD. This includes breaking rules, being mewned over, biting, barking, plucking, pulling, hunting, or playing with another puppy. Instead, your puppy should know the nature of the situation.","prompt":"[header] How to house train a puppy [title] Understand how your puppy views the environment. [step] Puppies don't have an inherent understanding of what humans consider right and wrong. They can learn many behaviors, though.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5887,"error":null,"output":"C"}
{"question_id":"hellaswag:00003162","item_index":84,"attempt":0,"prompt_hash":"25f4f399eb268986c14d303133988aa775640f1ff79b59e7f625e73bc1ab7818","question":"On the late night show the guest is neil patrick harris, he is very excited to see a horse come out on stage and jumps up and down very happily. Once they sit back down they talk for a while and then neil starts playing the harmonica. he\n\nA. stops and starts bouncing and gives a little wave then kisses the lady on the side of her face and gets up and goes back to playing the harmonica again.\nB. keeps making random signs and faces at the camera as the show goes on.\nC. then slows down and hits the drum a little as the song progresses to a faster clip.\nD. seems to be pretty good at it because it continues to play for quite a while.","prompt":"On the late night show the guest is neil patrick harris, he is very excited to see a horse come out on stage and jumps up and down very happily. Once they sit back down they talk for a while and then neil starts playing the harmonica. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5888,"error":null,"output":"D"}
{"question_id":"hellaswag:00006994","item_index":85,"attempt":0,"prompt_hash":"f2799b5df2d23146d16ecfb155f200d81e422b26c57d268f38879efca0ada327","question":"[header] How to write a book report [title] Follow the requirements of your assignment. [step] Read through the assignment sheet carefully and make note of any questions that you have. Raise your hand during class or talk with your teacher afterward to go over any concerns.\n\nA. Make sure that you know the required paper length, due date, and any formatting requirements, like double-spacing. [substeps] For example, you'll need to find out if your teacher wants you to include citations, such as page numbers from the book, in your paper.\nB. A good rule of thumb is to do this as normal and as soon as you take a reading of your book, you should open a book or report. Only complete the assignment if you are strictly at the grammar and follow the classroom rules.\nC. [substeps] College courses sometimes involve english and constitutional essays as well as psychological papers. Reading over your report before turning it in could help you to identify gaps that you may have missed.\nD. The material you'll need to read is vital to your report, so make sure to keep it ahead of other subjects. It's up to you whether it seems simple or complex, but keep track of any questions that you have to make a point of having answered earlier in the week.","prompt":"[header] How to write a book report [title] Follow the requirements of your assignment. [step] Read through the assignment sheet carefully and make note of any questions that you have. Raise your hand during class or talk with your teacher afterward to go over any concerns.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5888,"error":null,"output":"A"}
{"question_id":"hellaswag:00006230","item_index":86,"attempt":0,"prompt_hash":"ad1c382829ad5efd07d0c494e7fec32a09ac0f514a65dce72b4a5a4d1bdeb0d6","question":"[header] How to make a facial mask [title] Gather the ingredients. [step] This mask works to reduce the appearance of pores with clay, which pulls dead skin cells and other pore-clogging debris away from your skin as it dries on your face. Go to a health food stores and get the following supplies : [substeps] Bentonite clay, which draws out impurities.\n\nA. This is one of the most commonly used facial masks for people with acne. You can find bentonite clay at health food stores.\nB. Oatmeal, which leaves your skin feeling baby soft. Essential oil of your choice, such as peppermint or lemongrass, for an added sweet scent.\nC. Bentonite clay can help with a deep pore cleansing process by helping to remove oil from your skin. Avoid bentonite clay, which also collects dirt and makes pores worse.\nD. Undiluted bentonite clay (or granulated clay). Polyester glucose (ldl), which draws out unhealthy fats and found in many skincare products.","prompt":"[header] How to make a facial mask [title] Gather the ingredients. [step] This mask works to reduce the appearance of pores with clay, which pulls dead skin cells and other pore-clogging debris away from your skin as it dries on your face. Go to a health food stores and get the following supplies : [substeps] Bentonite clay, which draws out impurities.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5676,"error":null,"output":"A"}
{"question_id":"hellaswag:00000989","item_index":87,"attempt":0,"prompt_hash":"ba34902f8f603cfaff48a9c91c7cc2e1ca773d5c57847ea2d43dc1e348df8701","question":"A person is sharpening a knife behind a table. people\n\nA. walk through the door.\nB. are standing in front of the table watching.\nC. are walking in front of them.\nD. are sitting behind tables watching a game.","prompt":"A person is sharpening a knife behind a table. people","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5675,"error":null,"output":"B"}
{"question_id":"hellaswag:00000452","item_index":88,"attempt":0,"prompt_hash":"77869979b9ebd4f47f4571568fb05e11d84ae5cea90deb57509464f8cbf93f63","question":"A small campfire is shown in the woods. Someone turns on the campfire in the night. when the fire\n\nA. is turning off the man put more dry branches.\nB. dies down some more people are shown using their phones to call 911 and then call the police to make sure they are ok.\nC. dies down, a small piece of paper is shown.\nD. dies down, a person eats something.","prompt":"A small campfire is shown in the woods. Someone turns on the campfire in the night. when the fire","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5676,"error":null,"output":"C"}
{"question_id":"hellaswag:00003677","item_index":89,"attempt":0,"prompt_hash":"ea869930db01c20799b0a9ab3c227666cb10caac5231c5098570f425709b8934","question":"[header] How to use a luggage strap [title] Wrap the strap around a hard-sided piece of luggage. [step] One use for luggage straps is to keep luggage from accidentally popping open at some point during transit. Hard-sided luggage usually has locks or clasps that can occasionally be damaged if they are thrown around.\n\nA. These small pieces can be easily removed without problem, much like bags like timex or air bags, but they are stuck onto vertical objects, which could rip and break. Keep the strap loose and firmly secure as you're working on the bag.\nB. Make sure you're using case handles that can slide, which helps to prevent them from touching any luggage, and which locks on securely. [substeps] Some common cases include check-in bags and elastic or spandex bags.\nC. [substeps] For hard sided luggage that has some sort of clasp on the top, wrap the strap around the luggage so that it keeps the luggage closed. Place the buckle in the middle of the front or back side.\nD. [substeps] The lock must be at least 0.5 inches (1.4 cm) thick. Cover any hard-sided luggage with paper towels or a piece of cardboard to help ensure it doesn't rattle.","prompt":"[header] How to use a luggage strap [title] Wrap the strap around a hard-sided piece of luggage. [step] One use for luggage straps is to keep luggage from accidentally popping open at some point during transit. Hard-sided luggage usually has locks or clasps that can occasionally be damaged if they are thrown around.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5676,"error":null,"output":"C"}
{"question_id":"hellaswag:00007182","item_index":90,"attempt":0,"prompt_hash":"b6d343aff55c73e99c98425b88e438cfabe141d642f905a186d1e14e438c6e2a","question":"[header] How to find homes for a litter of kittens [title] Get your kittens vaccinated. [step] It will be harder to find a good home for your litter of kittens if they are not vaccinated. Take your litter of kittens to the vet when they are about eight weeks old so they can begin receiving a series of routine vaccinations.\n\nA. [title] Buy cat beds for your neighborhood that can fit your litter mates. [step] Plus, make sure you have good substrate as kittens will chew through more carpet than magazines and newspapers.\nB. [title] Calculate your cat's calorie needs. [step] Kittens aged between four and nine weeks old get most of their calorie from their food.\nC. Your vet will advise you regarding which vaccinations your kitten needs and when. [substeps] When taking the kittens to the vet, have them dewormed and treated for any other maladies-such as parasites-that might be present.\nD. Most health care centers and veterinary hospitals offer commercial package vaccines. [substeps] Visit the vet regularly and ask them to give you a unique package that includes the proper-name cat (sp) for your location.","prompt":"[header] How to find homes for a litter of kittens [title] Get your kittens vaccinated. [step] It will be harder to find a good home for your litter of kittens if they are not vaccinated. Take your litter of kittens to the vet when they are about eight weeks old so they can begin receiving a series of routine vaccinations.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5676,"error":null,"output":"C"}
{"question_id":"hellaswag:00009619","item_index":91,"attempt":0,"prompt_hash":"c594fa842bde2aebb6f0bcfdda978708b5c907a71ed3832abf4af5ecc91b9d44","question":"[header] How to do a sunday makeover [title] To start your sunday makeover, you need to get clean. [step] You can either take a long, hot shower or a warm bath. Shave your legs and body, exfoliate your body, and wash with a nice-smelling shower gel.\n\nA. The other part of the makeover involves washing your skin. The cleanser helps to wash off dirt and dead skin cells, and can be used in a variety of ways.\nB. [title] Wash your hair, and condition it. [step] After your hair is clean, squeeze the excess water gently out of your hair.\nC. / use a body-cleansing wash and rich body wash. You can also base off your wardrobe for this, by choosing a polo-shirt, tucked-in pants, and button down shirt for this outfit.\nD. Wash the exterior of your body with the cream. You can also air dry afterwards, if you are amenable to it.","prompt":"[header] How to do a sunday makeover [title] To start your sunday makeover, you need to get clean. [step] You can either take a long, hot shower or a warm bath. Shave your legs and body, exfoliate your body, and wash with a nice-smelling shower gel.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5445,"error":null,"output":"B"}
{"question_id":"hellaswag:00005113","item_index":92,"attempt":0,"prompt_hash":"274c192771e155b062a1915cd4e6ea37e9b746273d9f0e8438875ef920edac3b","question":"[header] How to dress for a night on the town (for guys ) [title] Hit the shower. [substeps] Before you even look at that closet, hit the shower. [title] Once you dry off (and apply deodorant), take a look in the mirror.\n\nA. [step] Make sure you don't overdo it, and make sure you aren't splashed on any other people. [title] Make sure to have all hair in the style.\nB. [step] You should see plenty of makeup! [title] You don't need to go outside if it's not cool. [substeps] Re-apply your makeup if necessary to give yourself a more polished look.\nC. [step] Does the body look good? It should! ( shower wisely. Try not to wear anything with damp bottom; the majority of the hair gets wet, so keep it out of the direct sunlight.\nD. [title] Get out the grooming kit (every guy should have one). [substeps] Your face should be clean-shaven and your sideburns clean cut, unless you can pull off the mop top.","prompt":"[header] How to dress for a night on the town (for guys ) [title] Hit the shower. [substeps] Before you even look at that closet, hit the shower. [title] Once you dry off (and apply deodorant), take a look in the mirror.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5444,"error":null,"output":"D"}
{"question_id":"hellaswag:00001949","item_index":93,"attempt":0,"prompt_hash":"0352190f6e0bd739b9673b0f1b0bf55c4995c113ceab033987b12356969a7d5c","question":"A woman sits behind a table dealing cards. she\n\nA. points at one of the cards.\nB. puts a six on the table to deal the cards.\nC. puts the cards into the discards pile.\nD. adds a few chips to the players' hands.","prompt":"A woman sits behind a table dealing cards. she","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5444,"error":null,"output":"C"}
{"question_id":"hellaswag:00007791","item_index":94,"attempt":0,"prompt_hash":"eefbc80f92ce8418eae3a415310540b0e54f3de73201863bb5c3f8da34dd6485","question":"[header] How to make papaya puree [title] Wash the papayas. [step] Rinse 4 small or 1 extra large papaya under running water to get any dirt particles off. [title] Cut the papayas in half.\n\nA. [step] Cut each papaya into small slices or 1 inch (2.54 cm) pieces. Ensuring the fruit is in small pieces will be very helpful in cooking and pureeing.\nB. [step] Each piece of papaya is used in a 1-quart saucepan. [substeps] Remove the pieces when the filets are slightly soft.\nC. [step] Scrape the backs to separate the insides. [title] Heat the chilies over medium heat.\nD. [step] Lay out the papaya halves on a cutting board, then cut each half into wedges. Use the forketing piece that you set aside earlier for the puree to cut them into wedges instead.","prompt":"[header] How to make papaya puree [title] Wash the papayas. [step] Rinse 4 small or 1 extra large papaya under running water to get any dirt particles off. [title] Cut the papayas in half.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5444,"error":null,"output":"A"}
{"question_id":"hellaswag:00000734","item_index":95,"attempt":0,"prompt_hash":"b174a16246dc7335864caacd6f779c8ee7bf13ed6d19035fe1caba99242045db","question":"She then takes a shaker and pours and vodka and the jagermeister. Then adds some ice in it and shakes it. she\n\nA. then adds a few drops of sugar.\nB. pours it into glass and adds down a third.\nC. pours the drink into the shaker and mixes it from the shaker.\nD. uses a strainer to strain the drink into a cocktail glass and places the glass on the table.","prompt":"She then takes a shaker and pours and vodka and the jagermeister. Then adds some ice in it and shakes it. she","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5444,"error":null,"output":"D"}
{"question_id":"hellaswag:00007371","item_index":96,"attempt":0,"prompt_hash":"5e60e583296d14ef22faf1a84951b75866e38f3b95d110564320ee172a1a6649","question":"[header] How to define cheekbones [title] Apply concealer and foundation. [step] As always, wash your face before you apply makeup. Cover blemishes with concealer if necessary, then apply foundation.\n\nA. Smudge concealer around your nose, as well. [title] Apply translucent moisturizer to blemishes.\nB. Then, use a makeup brush to apply a light, concealer shade under cheekbones. Blend well with the brush until you can see all the contour lines.\nC. [title] Choose bronzer slightly darker than your skin tone. [step] Peach works well for warm skin tones, while earthy tones complement cool skin tones.\nD. Dab concealer under the eyes, carefully blending with a clean makeup sponge. [title] Apply concealer to a small white face, just under one eyebrow.","prompt":"[header] How to define cheekbones [title] Apply concealer and foundation. [step] As always, wash your face before you apply makeup. Cover blemishes with concealer if necessary, then apply foundation.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5570,"error":null,"output":"C"}
{"question_id":"hellaswag:00003654","item_index":97,"attempt":0,"prompt_hash":"ff00f1db41160ffe827c49fa65fe3665cff41a316648bb4375c445afb941cf48","question":"[header] How to cut a man's hair [title] Ask the man how he wants his hair. [step] In a consultation, find out how short he wants his hair, how uniformly he wants it cut, and how he plans to style his hair afterwards. He may want you to only trim away split ends, but he could also want you to cut everything off.\n\nA. [title] Consider how long he wants his hair to be. [step] Men don't grow his hair to the shoulder length they normally do, so it would be reasonable to cut his hair any length that he likes.\nB. Ask that he only cut out the hair he really wants, and that you only cut off slightly more than that at some point. [title] Brush your hair when you're finished cutting it.\nC. [substeps] Once you've narrowed your list, you'll need to decide where you would like his hair to fall. Some men go all out and have a trim every three to four weeks.\nD. Before you begin to cut, it's important that you ask all of the necessary questions and understand what he would like done with his hair. If you are clear and communicative from the start, you will avoid any potential misunderstandings.","prompt":"[header] How to cut a man's hair [title] Ask the man how he wants his hair. [step] In a consultation, find out how short he wants his hair, how uniformly he wants it cut, and how he plans to style his hair afterwards. He may want you to only trim away split ends, but he could also want you to cut everything off.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5562,"error":null,"output":"D"}
{"question_id":"hellaswag:00005987","item_index":98,"attempt":0,"prompt_hash":"da6c3e1c90e43562d427ff461d0ee15be072b517d50b3ab0a9952ce5dbb423f5","question":"[header] How to be a good parent when you have ptsd [title] Tell your kids about your symptoms. [step] Explain what ptsd is, and describe the symptoms you have. It can be scary or upsetting for a child to see their parent getting angry or having a flashback without any explanation, so let them know what's going on.\n\nA. When telling your kids, try to remember that you are the parent and that you are there to help them. [substeps] For example, you may tell your children, \" i'm concerned that you have been having a severe flare up of ptsd.\nB. Be specific, and be thorough as you talk to your kids about ptsd. [substeps] Say, \" i went through my first family breakup when my dad broke up with me.\nC. [substeps] You don't have to tell your kids about the trauma you went through. In fact, it may be a better idea not to.\nD. Then mention it in detail to your kids or someone else who might help you understand what's going on. [substeps] Describe to your children that not only ptsd happens to you, but other people have seen it in the past, too.","prompt":"[header] How to be a good parent when you have ptsd [title] Tell your kids about your symptoms. [step] Explain what ptsd is, and describe the symptoms you have. It can be scary or upsetting for a child to see their parent getting angry or having a flashback without any explanation, so let them know what's going on.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5563,"error":null,"output":"C"}
{"question_id":"hellaswag:00007461","item_index":99,"attempt":0,"prompt_hash":"54fb7649f01fbe8adefa66d31247ebd2bbee2d3409c49bea66f9bc8668b30aa1","question":"[header] How to cancel your wall street journal subscription [title] Check the billing cycle for your print or online subscription. [step] The wall street journal offers online and print subscriptions that are paid every month , 3 months , 6 months, or 1 year. The billing cycle of your subscription will determine when you should cancel it and if you qualify for a refund.\n\nA. [substeps] Monthly and quarterly subscriptions can be cancelled at any time. Semi-annual and annual subscriptions should be cancelled 1-2 months before the subscription ends so you can get a refund for any remaining months.\nB. [substeps] The billing cycle will vary from one year to another; however, for example, it will either be in the form of a $1 , 070 cycle or 6 months. Just knowing when the billing cycle is will help you determine when to cancel it and plan accordingly.\nC. [substeps] One month old paper subscriptions will not be billed to purchase book subscriptions. Therefore, it's always best to make it a habit to deduct a month's billing period from what you receive on each day of the month.\nD. [substeps] Print subscriptions are fixed, based on the billing cycle and the date you subscribe. Buy the print or online subscription above the billing cycle you choose.","prompt":"[header] How to cancel your wall street journal subscription [title] Check the billing cycle for your print or online subscription. [step] The wall street journal offers online and print subscriptions that are paid every month , 3 months , 6 months, or 1 year. The billing cycle of your subscription will determine when you should cancel it and if you qualify for a refund.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5563,"error":null,"output":"A"}
{"question_id":"hellaswag:00006576","item_index":100,"attempt":0,"prompt_hash":"c7f9e84e5b3897861e7132fc164c4beb66e2a9fee428a7c2e4459003a4460f4a","question":"[header] How to hang a plate [title] Measure the plate. [step] Wire plate hangers come in different sizes, so you need to know the size of the plate before you can purchase the hangers. Use a ruler or tape measure to determine the vertical length of your plate.\n\nA. Wear rubber gloves while cutting wire or tape so you don't accidently pick up a piece of the wire before you know it. [substeps] Wire plate hangers come in a variety of sizes from 8 inches to 24 inches (30 to 61 cm).\nB. [substeps] If you have a simple circular plate, you only need to know the diameter. Similarly, if you have a perfectly square plate, you only need to know the length of one side.\nC. If you have 24-48 \" x 45 \" on one end of your plate, that width should be 1 \" to 2 \" in length. If you have 13 \" x 45 \" on the other end of your plate, that width should be 159 \" to 159 \".\nD. Your plate's vertical length will depend on how wide your plate is. Ask a family member or friend what size plate to purchase for your plate.","prompt":"[header] How to hang a plate [title] Measure the plate. [step] Wire plate hangers come in different sizes, so you need to know the size of the plate before you can purchase the hangers. Use a ruler or tape measure to determine the vertical length of your plate.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5564,"error":null,"output":"B"}
{"question_id":"hellaswag:00002792","item_index":101,"attempt":0,"prompt_hash":"a4a525efbe47ed2cb9cda811ae049ab72ebd649d03cf01487c5bc197392525d5","question":"Man is smoking a cigarrette in a dark street. man\n\nA. is wearing a white t shirt smoking a cigarrette and watching to the camera.\nB. play a baton in the middle of street.\nC. is in front of white houses dealnig light flashing.\nD. is smoking a cigarette on the street next to a tree.","prompt":"Man is smoking a cigarrette in a dark street. man","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5566,"error":null,"output":"D"}
{"question_id":"hellaswag:00001113","item_index":102,"attempt":0,"prompt_hash":"0b29fa1a25882f2c11dfc726d487b5de177e5d356f00f78297b4bf5f2ef40f03","question":"The \"bbc news\" logo appears on screen with website addresses. A man plays a bagpipe in front of burning trucks as people thrown rocks. police\n\nA. officers stand outside of a dodge ram topic and watch the man play.\nB. take pictures of the scorched men.\nC. officers rush to his aid as he puts wax on a shingle at an address.\nD. set up a blockade as rioters throw stones and fire.","prompt":"The \"bbc news\" logo appears on screen with website addresses. A man plays a bagpipe in front of burning trucks as people thrown rocks. police","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5565,"error":null,"output":"D"}
{"question_id":"hellaswag:00006980","item_index":103,"attempt":0,"prompt_hash":"f39b70f154af47a2068c0f7b8895875bef71a08afc3a2e59c84f9bb6cbca2e75","question":"[header] How to create christmas games [title] Divide everyone into two (or more) groups. [title] Provide each team with a set of oversized mittens or gloves. [step] Have the members of each team stand in a single file line, with the first person in line wearing the gloves/mittens.\n\nA. The other person then holds the hand of the first person and sets their mittens down into the snow. Each group should have 3 hands to hold the first person's mittens in place throughout the whole game.\nB. [title] Place a bowl of hershey's kisses on a table. [step] If you're the first person in line, run to the bowl.\nC. [title] Each person should have sleeves to their elbow, typically full. [step] These sleeves must be 0.4 in (1.3 cm) in length; if it is too short, the team could die.\nD. Have them stand in line in the front of the main lobby area (the character building area) until everyone reaches the end of their designated queue, then drop the first team member. Hold the second team member over a blanket or tablecloth until everyone has collected their scarf and gloves.","prompt":"[header] How to create christmas games [title] Divide everyone into two (or more) groups. [title] Provide each team with a set of oversized mittens or gloves. [step] Have the members of each team stand in a single file line, with the first person in line wearing the gloves/mittens.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5560,"error":null,"output":"B"}
{"question_id":"hellaswag:00004975","item_index":104,"attempt":0,"prompt_hash":"837dbb5498733e69b0ad8d3f0ce1d3f16120c572d3c1520348947a5de9b26528","question":"[header] How to prevent spider veins [title] Avoid sitting or standing for long periods. [step] If your job requires you to be on your feet all day, break up the time you stand still by walking around the office or taking a walk at lunch. [title] Aid circulation with your body positioning.\n\nA. [step] Since spider veins are caused by poor blood flow and circulation, it's important to avoid positions that impede circulation and use positions that aid it. [substeps] Avoid crossing your legs when you sit at home or work.\nB. [step] Symptoms of spider veins include : [substeps] When you sit and stand, your feet and hands hold pressure on your armpits. They send signals that you are exerting too much pressure on your skin (especially when you have spider veins on your elbows, ankles, and knees).\nC. [step] Prolonged standing can cause elbow and foot problems, and are a major contributor to spider veins. Standing that long can lower your blood pressure, make you more hydrated, help you avoid headaches and aches and some other negative effects.\nD. [step] If you are sitting too frequently, you should try to reduce your standing time by elevating your hips or lowering your head so that your knees can't touch. [substeps] When you sit or stand, your hip bones should point down and keep your arms set, but out of your pockets.","prompt":"[header] How to prevent spider veins [title] Avoid sitting or standing for long periods. [step] If your job requires you to be on your feet all day, break up the time you stand still by walking around the office or taking a walk at lunch. [title] Aid circulation with your body positioning.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5561,"error":null,"output":"A"}
{"question_id":"hellaswag:00002563","item_index":105,"attempt":0,"prompt_hash":"80f5f9ca57f32ce84c36c323e7dc85c2f2fb2a9ed174daebc7a4354c95302c77","question":"We see an opening title screen. A man jumps rope in a boxing ring in a mall as people watch. the man\n\nA. jumps rope using a hand held machine.\nB. swiftly knocks people onto mats.\nC. loses his rhythm and has to start over two times.\nD. sits on the edge of the rope and jumps rope faster.","prompt":"We see an opening title screen. A man jumps rope in a boxing ring in a mall as people watch. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5563,"error":null,"output":"C"}
{"question_id":"hellaswag:00009878","item_index":106,"attempt":0,"prompt_hash":"cc15eccb99e434c422902498b76055bfd61fdd392149a901084c13ffc868458c","question":"[header] How to lace timberlands [title] Thread one side of the lace through the bottom eyelet. [step] The eyelets are the small holes on the front of the boot. You can thread the lace through the eyelet from the outside or push the laces in through the holes from the middle of the shoe.\n\nA. [substeps] Make sure you thread the lace through both eyelets. [title] Loop the lace through both eyelets on the front and sides of the boot.\nB. [title] Hook the second end of the lace between two eyelet holes on the inside of the boot. [step] Pass the lace through one eyelet on the inside and then slip the other end of the lace through to the other eyelet hole.\nC. Try both methods to see which style you prefer. [substeps] Threading the laces from the outside lends itself well to the loose and criss-cross styles.\nD. [title] Thread your lace through the eyelet hole. [step] As you're pulling the lace up to the eyelet hole, shift the lace up and across the shoe so that the lace is exposed to the eyelet hole.","prompt":"[header] How to lace timberlands [title] Thread one side of the lace through the bottom eyelet. [step] The eyelets are the small holes on the front of the boot. You can thread the lace through the eyelet from the outside or push the laces in through the holes from the middle of the shoe.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5562,"error":null,"output":"B"}
{"question_id":"hellaswag:00007642","item_index":107,"attempt":0,"prompt_hash":"e7e0bdc33207dba2b1c314b0a53dfb31cf2e79983d2a6ee201ff53a6b735bbca","question":"[header] How to oppose a motion to dismiss [title] Identify why the defendant seeks dismissal. [step] You should receive a copy of the defendant's motion to dismiss. Once you receive it, read through the document and look to see why the defendant seeks dismissal.\n\nA. [substeps] If you don't know why the defendant seeks dismissal, then find common grounds. The defendant could be trying to get in trouble for refusing to work with you because you aren't accepting credit on your debt, or because the defendant wants to get money.\nB. You should identify the violation of the law and the causes of action, even if it is not the same thing as an excuse to dismiss. [substeps] For example, a defendant may complain of a big change in the values of the property in question and argue that making the change drastic means the home won't be put in good working order.\nC. Generally, a defendant seeks to have a case dismissed for the following reasons : [substeps] You didn't serve the complaint properly on the defendant. You haven't stated a valid legal claim in your complaint.\nD. [substeps] Often, people seek dismissal because they have broken the laws that govern the employment of similar workers. You can find out if the defendant has this reason in the lawsuit by reading the particular court's rules of civil procedure (or calling the clerk's office) and looking for the official complaint form on the court's website.","prompt":"[header] How to oppose a motion to dismiss [title] Identify why the defendant seeks dismissal. [step] You should receive a copy of the defendant's motion to dismiss. Once you receive it, read through the document and look to see why the defendant seeks dismissal.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5559,"error":null,"output":"C"}
{"question_id":"hellaswag:00002944","item_index":108,"attempt":0,"prompt_hash":"ac514d3a293b7c9e4094608aa2a74420c45eaa2e4b30bdc12d763bfd024f1e0f","question":"An indoor barn is shown and a man is sitting on a horse. the gates\n\nA. open and the man flies out with is lasso chasing after a black and white calf.\nB. begin to open and the horse walks out screaming.\nC. opens and he starts running down a track.\nD. open and the horse runs out, the man chases after the cattle and lassos the cattle onto a table.","prompt":"An indoor barn is shown and a man is sitting on a horse. the gates","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5564,"error":null,"output":"C"}
{"question_id":"hellaswag:00008614","item_index":109,"attempt":0,"prompt_hash":"bd957c50e77b2f816bb4e9256dd25841def23c65b59678da2d5ac7517411f375","question":"[header] How to cut salt from pantry foods [title] Look for \" no salt added \" canned vegetables. [step] Canned vegetables are generally high in sodium. To cut the salt from these, buy canned vegetables that say \" no salt added \" on the label.\n\nA. That ensures you are getting the vegetable without added salt. [substeps] If you want a little bit of salt to taste, you can add a small amount of salt when you put the food on the plate.\nB. Focus on reduced sodium, which means diets have been shown to lower the salt content of these vegetables. [substeps] Many bakeware jars are this size and range in price from a few dollars to hundreds of dollars.\nC. Canned vegetables are commonly added to soups and casseroles. About halfway down the grocery aisle, search for \" no salt added \" in your pantry.\nD. Simply place them in a blender, then transfer them to a food processor. Use 1 cup of water for every 2 cups of liquid that you add to the blender.","prompt":"[header] How to cut salt from pantry foods [title] Look for \" no salt added \" canned vegetables. [step] Canned vegetables are generally high in sodium. To cut the salt from these, buy canned vegetables that say \" no salt added \" on the label.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5558,"error":null,"output":"A"}
{"question_id":"hellaswag:00000918","item_index":110,"attempt":0,"prompt_hash":"cdd9321ecb304520969680477006e5c5a923815db0fa9a59ceea195921a1ab1a","question":"A small group of children are seen playing hand games with one another and finally hitting the loser on the head with a bottle. the kids\n\nA. also motion out towards the audience and a little girl runs by out of the camera frame.\nB. hold each other and pretend to be pushing one another and laughing.\nC. continue knocking others down while laughing to the camera and leading into clips of people chasing the loser down in a field.\nD. continue to play the hand game with one another that leads into the beating another with a bottle.","prompt":"A small group of children are seen playing hand games with one another and finally hitting the loser on the head with a bottle. the kids","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5562,"error":null,"output":"D"}
{"question_id":"hellaswag:00004957","item_index":111,"attempt":0,"prompt_hash":"6f6778e1d075a55fb5ce212071891e8742a750f4dea4f3b3e1c81c6b17f38da5","question":"[header] How to look good in a speedo [title] Get a speedo made of high quality, long lasting fabric. [step] Look for a speedo made of 100% polyester or a polyester blend with spandex. It will be more chlorine resistant and long lasting.\n\nA. Make sure that the speedo has a quality grooved neck feature so that there is less pressure on your neck. [title] Try to find an olympic style speedo.\nB. A speedo made of spandex has pure polyester threads that wick away from the speedo and they can be boiled at high temperatures while they are stored inside. Speedo's are generally very good at opening, hooking, shooting and spotting.\nC. Some speedos are also made in a high uv rating fabric, making them more resistant to the sun if you plan to swim outdoors. [substeps] Avoid speedos made of only spandex or lycra, as they tend to fade and get loose more quickly.\nD. Make sure that your speedo is split down the middle, so that there are no sharp edges, gaps, or ruffles. [title] Match your speedo to your general personality.","prompt":"[header] How to look good in a speedo [title] Get a speedo made of high quality, long lasting fabric. [step] Look for a speedo made of 100% polyester or a polyester blend with spandex. It will be more chlorine resistant and long lasting.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5559,"error":null,"output":"C"}
{"question_id":"hellaswag:00008179","item_index":112,"attempt":0,"prompt_hash":"a19acfa0fc986c1b26f32517e4958037693d7a4d5c00d1860fe4dfae7fe089f2","question":"[header] How to determine freight class [title] Learn the things that will determine the freight class of your item, including loadability and handling characteristics, weight, density, and the product's susceptibility to damage. [title] Determine the basic description of the freight being transported. [step] Be able to describe what material the item being shipped is made of.\n\nA. A buyer must be able to show that an item shipped is \" cheap \" and common home-furnishings materials, while a seller must be able to show that it is unique and inexpensive before purchase. [title] Find inventories of freight classes by type.\nB. For example, if it is made from brass or titanium, be able to detail the grade and durability of the material. If it is made from diamonds, be able to connect with the chemical components involved in the metal.\nC. [title] Know what kind of container your item will be shipped in. [step] There are many different types of allowable packaging systems for freight including, but not limited to, pallets, drums, reels, crates, tubes, or bundles.\nD. [substeps] Use a search engine to help you for examples of barcodes and adcurrencies, as well as mass and mass transit regulations. [title] Identify the class on which you are preparing the freight class.","prompt":"[header] How to determine freight class [title] Learn the things that will determine the freight class of your item, including loadability and handling characteristics, weight, density, and the product's susceptibility to damage. [title] Determine the basic description of the freight being transported. [step] Be able to describe what material the item being shipped is made of.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5560,"error":null,"output":"C"}
{"question_id":"hellaswag:00008960","item_index":113,"attempt":0,"prompt_hash":"c7123fac9bbc4ae1b716186da3e6bbddc66626b37484ce0d5ff9c2d3dddc2b0e","question":"[header] How to run a laundromat [title] Clean the laundromat. [step] Some light cleaning may be necessary, such as emptying lint from lint screens in the dryers, emptying trash cans and washing tables used by customers to fold laundry. If your laundromat has a lavatory, you will also need to keep that in good order.\n\nA. [substeps] Wash all pre-washed and re-washed clothing according to season or day. Wash dirty clothing according to what season or the purpose of your laundromat.\nB. [substeps] Resetting the toilet by setting it in the appropriate position and then running water over it. If you also wish to move the wash table back to their normal position and remove the blind fold, wash that.\nC. [title] Perform regular maintenance on washers and dryers. [step] You will have to maintain the washer and dryer according to the type of machine.\nD. [title] Go to the laundromat's waste disposal. [step] You can use the common household sink relocation system to clean especially dirty laundry.","prompt":"[header] How to run a laundromat [title] Clean the laundromat. [step] Some light cleaning may be necessary, such as emptying lint from lint screens in the dryers, emptying trash cans and washing tables used by customers to fold laundry. If your laundromat has a lavatory, you will also need to keep that in good order.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5560,"error":null,"output":"C"}
{"question_id":"hellaswag:00000654","item_index":114,"attempt":0,"prompt_hash":"c3aeb2cb8c1140b619a0ed06596e955254aeb4b9cbd85349a69e1dde414652fa","question":"A man performs a long jump at a judged professional sports event resulting in medals issued. a man\n\nA. performs a long jump on a track and into a sandpit in front of judges and onlookers on the sidelines.\nB. applauds for his run in australia at the olympic event.\nC. on a weightlifting bar does a short jump.\nD. grabs an edge of a bar to stabilize himself when a wrestler slams into him.","prompt":"A man performs a long jump at a judged professional sports event resulting in medals issued. a man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5562,"error":null,"output":"A"}
{"question_id":"hellaswag:00002020","item_index":115,"attempt":0,"prompt_hash":"cd0fee5b5f8341517b52d6cd1a39c71a033203f4d0c852463212511091517529","question":"A person is pressing buttons on an oven. A girl pours ingredients into a bowl together. she\n\nA. closes the oven door and puts the items into an oven.\nB. mixes the ingredients using a mixer then puts more ingredients in the bowl.\nC. begins frying pasta in a metal pan.\nD. dabs each ingredient in a bowl.","prompt":"A person is pressing buttons on an oven. A girl pours ingredients into a bowl together. she","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5561,"error":null,"output":"B"}
{"question_id":"hellaswag:00000109","item_index":116,"attempt":0,"prompt_hash":"506aa10ab5dc351a497858403d797a4143ae14a3981df478db48c464d15f2f37","question":"We see the underwater sea floor. Two people swim in front of the camera. We see a man holding a camera. we\n\nA. see a person kitesurfing the ocean floor.\nB. see the man turn off the camera.\nC. see divers and a man talking.\nD. see a fin enter the camera.","prompt":"We see the underwater sea floor. Two people swim in front of the camera. We see a man holding a camera. we","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5561,"error":null,"output":"B"}
{"question_id":"hellaswag:00006541","item_index":117,"attempt":0,"prompt_hash":"8e756baa9f79b70d7a79557d90173305a1eb6229ddd2fe055b6c286c2965ae19","question":"[header] How to create fuller lips with makeup [title] Exfoliate your lips. [step] Dry lights reflect less light than smooth, hydrated lips, which can make them look smaller. Before putting on any lip products, exfoliate your lips to remove any dry, flaky skin.\n\nA. For plumped lips, take one or both hands and rub over the lips with your fingers to lightly scrub away dead skin cells. If your lips are very dry, you may need to use a very fine, scented lip scrub.\nB. [substeps] Your lips should be soft and plump for the day and fully hydrated in the afternoon. If your lips seem to dry out, you may need to add more moisturizer and some lip scrubs.\nC. [substeps] Start the process by using your finger to thoroughly exfoliate your lips. This can be a gentle exfoliant for lips, a rough exfoliant for lips made mostly with dry or flaky lip product, or both! [title] Get lips lined.\nD. Use a scrub designed specifically for the lips or brush them lightly with a clean, sanitized toothbrush to remove dead skin. [substeps] You can make your own lip scrub by combining 2 teaspoons (10 g) of brown or white sugar with 1 teaspoon (15 ml) of an oil, such as coconut or olive.","prompt":"[header] How to create fuller lips with makeup [title] Exfoliate your lips. [step] Dry lights reflect less light than smooth, hydrated lips, which can make them look smaller. Before putting on any lip products, exfoliate your lips to remove any dry, flaky skin.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5563,"error":null,"output":"D"}
{"question_id":"hellaswag:00008879","item_index":118,"attempt":0,"prompt_hash":"c3c275e97589520daad4b03dabbee6c37070d628e89c94cc9b9994d363bd97d5","question":"[header] How to cover your eyebrows [title] Gather supplies. [step] You probably already have what you need to cover up your eyebrows around the house. Get everything ready before you get started.\n\nA. A pair of round contacts or foam brush, scissors, and most importantly a pair of tweezers are items that you'll need for the cutting in the eyebrow. [substeps] For the inner part of your eyebrow, buy an eyebrow comb that is approximately 3 inches (7.6 cm), preferably a compact and very soft-bristled q-tip, in a color of your choice.\nB. Some of these supplies may have come with you in the woods. [substeps] You will need your notebook, pencil, and ruler.\nC. Here's what you need : [substeps] A small bottle of spirit gum (sold at costume shops, and is made for use on the face) petroleum jelly (e.g. vaseline) translucent concealer powder that matches the color of your skin liquid concealer or foundation that matches the color of your skin colour correcting concealer (optional ) [title] Prep your brows and skin.\nD. Items to take with you to the salon are : [substeps] Comb your eyebrows. If you have long hair, you will probably need the brush to comb any frizzy hair out.","prompt":"[header] How to cover your eyebrows [title] Gather supplies. [step] You probably already have what you need to cover up your eyebrows around the house. Get everything ready before you get started.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6860,"error":null,"output":"C"}
{"question_id":"hellaswag:00007599","item_index":119,"attempt":0,"prompt_hash":"5ba4687d0c10df478a0c6bb74f75b78ab3986d6b12b87c51c941700570c3941f","question":"[header] How to cook with lime juice [title] Combine lime juice, salt, and minced garlic in a glass bowl. [step] Do not use a metal bowl, since the acidic properties of lime juice can have a chemical reaction with the metal that could taint your food. [title] Add cubed fish to the bowl.\n\nA. [step] Use browned, skinless fish instead of sliced or quartered. Sprinkle with sea salt and pepper to taste.\nB. [step] You can use these dishes immediately, or you can cover them in plastic wrap and refrigerate them for a few days. Alternatively, you can cover them in plastic sleeves and let them rest overnight before you begin to cook.\nC. [step] Popular choices include salmon, ahi tuna, halibut, tilapia, and sea bass, but many types of fish and other seafood can actually be used for the dish. Make sure that the fish is completely covered by the lime juice.\nD. [step] Place the cubed fish on a large serving dish or platter. Sprinkle the sea salt over the fish.","prompt":"[header] How to cook with lime juice [title] Combine lime juice, salt, and minced garlic in a glass bowl. [step] Do not use a metal bowl, since the acidic properties of lime juice can have a chemical reaction with the metal that could taint your food. [title] Add cubed fish to the bowl.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6862,"error":null,"output":"C"}
{"question_id":"hellaswag:00005913","item_index":120,"attempt":0,"prompt_hash":"a315a7d1dd080da0830edf40c222516b724e7f46d537698f282d83cae392c920","question":"[header] How to buy dessert wine [title] Familiarize yourself with the different types of dessert wine. [step] There are many different types of dessert wines. They vary based on the grapes used and how they are prepared.\n\nA. In general, fresh grapes are more nutrient-dense, giving them flavor and providing added nutrients. Fortified sauces are sweeter than flavored types.\nB. Here are some of the popular wines available to buy including : [substeps] Wineries wine stemware blanc downkin' champagne distilled @79 l wine red wine butterworth sparkling wine 150b [title] Find the right wine. [step] Wine and other beverages look similar when mixed on different days of the week.\nC. Some are alcohol, such as pink wine, sweet wine, and red wine. These juices tend to be marketed as very simple and low calorie, making them best served with flat or leafy grain.\nD. The dessert wines can be divided up as follows : [substeps] Noble rot wines: this type of wine is made from rotten grapes. The grapes are covered in a mold called botrytis cinerea, commonly known as noble rot.","prompt":"[header] How to buy dessert wine [title] Familiarize yourself with the different types of dessert wine. [step] There are many different types of dessert wines. They vary based on the grapes used and how they are prepared.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6861,"error":null,"output":"D"}
{"question_id":"hellaswag:00003328","item_index":121,"attempt":0,"prompt_hash":"a3fd58ba966c71ef0d299fdc9453925bf675d23c77b5f26c6b52e19d8407fdf0","question":"[header] How to jailbreak an ipad [title] Go to the redsn0w jailbreak wizard at http: // www.redsn0w.us/2013/10/the-ultimate-jailbreak-wizard.html. [step] This tool allows you to find jailbreak software compatible with your ipad. [title] Select \" ipad \" under the idevice dropdown menu.\n\nA. [step] This will display all of the current (and future) ipad apps that you have installed. [title] Scroll down the list and see the current updates available.\nB. [step] Next to your ipad will appear a list of all jailbreaking programs installed on your device. [title] Enter the password you set for the jailbreak software later in the program.\nC. [step] You'll find this below the device heading (icon next to the ipad image). On mac, it's also in a lower-case \" tools \" section (] advanced).\nD. [title] Select the model of your ipad and its ios version. [substeps] Navigate to settings] general] about on your ipad to identify its ios version.","prompt":"[header] How to jailbreak an ipad [title] Go to the redsn0w jailbreak wizard at http: // www.redsn0w.us/2013/10/the-ultimate-jailbreak-wizard.html. [step] This tool allows you to find jailbreak software compatible with your ipad. [title] Select \" ipad \" under the idevice dropdown menu.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6861,"error":null,"output":"D"}
{"question_id":"hellaswag:00008897","item_index":122,"attempt":0,"prompt_hash":"4bf36d66f14bedf2f9481688286f52ee773d2096a596cef590a8dd7e3153f9df","question":"[header] How to make a cute clutch bag [title] Choose a rectangular placemat. [step] This method works best with fabric placemats. Choose something that looks interesting, preferably with embroidery.\n\nA. [title] Slide the scarf end through the pendant hole on the placemat. [step] Squeeze the twist link and seal the ends shut.\nB. Dyeing the placemat using only fabric placemats will make it resemble you is making the bag. [title] Add a single color of fabric placemat to the bag.\nC. Something with a lining would work even better, and will give your clutch a more professional touch. Make sure that the material is sturdy and not flimsy.\nD. [title] Get a brown or light colored ribbon and wrap around the placemat for a cute look. [step] Add a bow to the ribbon for a fancy touch.","prompt":"[header] How to make a cute clutch bag [title] Choose a rectangular placemat. [step] This method works best with fabric placemats. Choose something that looks interesting, preferably with embroidery.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6861,"error":null,"output":"C"}
{"question_id":"hellaswag:00000500","item_index":123,"attempt":0,"prompt_hash":"7afa31e074ef2f5e046e8873200c4066dc55e188053c4a11d49b27396f129c9c","question":"A man drills a hole into the ice. He puts a fishing pole into the ice. he\n\nA. pulls out a fish and hooks it in the hole.\nB. pulls a fish out of the hole and puts it on top of the ice.\nC. takes the fishing pole out and hooks a callo fish.\nD. pushes a fish to the bottom of the hole.","prompt":"A man drills a hole into the ice. He puts a fishing pole into the ice. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6910,"error":null,"output":"B"}
{"question_id":"hellaswag:00005355","item_index":124,"attempt":0,"prompt_hash":"298904e7e54817f38da48cc8a491cb207108a3ff02ed032bf55ef5a2da20335b","question":"[header] How to make fake dimples [title] Wash the bottle cap. [step] You can use either a metal or a plastic bottle cap, but since the cap will be going into your mouth, you need to make sure that it has been thoroughly cleaned, first. [substeps] Metal bottle caps are shallow and tend to produce more subtle, natural looking fake dimples.\n\nA. Plastic caps are generally thin and relatively wrinkle-free. Plastic caps work best on hard-to-reach places, like the grooves on metal bottles.\nB. Plastic bottles that have been folded in half or moved around will produce a much more subtle, flattering impression. Plastic bottles that come in two most common colors will also work well.\nC. Plastic bottles that are professionally cleaned are far less likely to produce noticeable fake dimples. [title] Moisten a cotton ball with water and wash the bottle.\nD. The ridges can be painful, though, especially if you have sensitive cheeks. Plastic bottle caps come in a few different sizes.","prompt":"[header] How to make fake dimples [title] Wash the bottle cap. [step] You can use either a metal or a plastic bottle cap, but since the cap will be going into your mouth, you need to make sure that it has been thoroughly cleaned, first. [substeps] Metal bottle caps are shallow and tend to produce more subtle, natural looking fake dimples.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6909,"error":null,"output":"D"}
{"question_id":"hellaswag:00003076","item_index":125,"attempt":0,"prompt_hash":"f485f5f9d509ccc670266088462b2f7401fee70e70658b1bc6c83b6bd5c03719","question":"A male athlete is standing on a circle on a track. He spins in a circle with a disc. he\n\nA. completes the turn with the disc safely in his hand.\nB. performs several twists and turns.\nC. lets fly his disc.\nD. then lets go of the disc and lets it fly.","prompt":"A male athlete is standing on a circle on a track. He spins in a circle with a disc. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6909,"error":null,"output":"D"}
{"question_id":"hellaswag:00009623","item_index":126,"attempt":0,"prompt_hash":"a9249433fd0a206431d3c4050a861adecde15f6093d04194f410c4df89a380c7","question":"[header] How to get rid of paint smells [title] Fill up a one-to three-gallon bucket with tap water. [title] Place the bucket of water in the center of a room that has recently been painted. [step] The water should absorb any solvent vapors leftover from the paint job.\n\nA. It should be slightly warm but not hot. [title] Submerge the paint in the bucket.\nB. [title] Place old towels into the bucket to catch any paint that drips. [title] Fill the bucket with clean tap water.\nC. Ideally, allow 48 hours for the chemical to dissipate. [substeps] Do not use tap water because it might shock the paint job and provoke stains.\nD. [substeps] Use two or more buckets of water as needed for large rooms or spaces. [title] Allow the bucket of water to sit overnight, or until the paint smell has dissipated.","prompt":"[header] How to get rid of paint smells [title] Fill up a one-to three-gallon bucket with tap water. [title] Place the bucket of water in the center of a room that has recently been painted. [step] The water should absorb any solvent vapors leftover from the paint job.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6910,"error":null,"output":"D"}
{"question_id":"hellaswag:00005896","item_index":127,"attempt":0,"prompt_hash":"697abc12a963e656f962c6ad031752f52409bd26820e4ab77c7f61c44e29ffa0","question":"[header] How to make blemish balm (bb cream ) [title] Gather everything listed in the things needed section. [step] You're now ready to start! [title] Pump in about 5 to 10 squirts of moisturizer into your container. [step] You can add more or less depending on how much bb cream you're making and how moisturizing you want it to be.\n\nA. [substeps] As said in step one, bb cream is essential for making blackheads, and can easily be substituted for bb cream. Keep in mind that if you use bb cream on your skin, your skin will absorb it without realizing.\nB. Use a little bit at a time to avoid wetting your skin. [title] Add one or two drops of your favorite essential oil blend.\nC. [title] Bb creams have sunscreen protection in them usually, so squirt a little of your favorite sunscreen into the jar. [title] Add in some foundation! [step] This will give your bb cream its color.\nD. [title] Dip your brush into the non-stick spray. [step] Then, fill it up about 5 to 6 centimeters with your chosen moisturizer.","prompt":"[header] How to make blemish balm (bb cream ) [title] Gather everything listed in the things needed section. [step] You're now ready to start! [title] Pump in about 5 to 10 squirts of moisturizer into your container. [step] You can add more or less depending on how much bb cream you're making and how moisturizing you want it to be.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6911,"error":null,"output":"C"}
{"question_id":"hellaswag:00002004","item_index":128,"attempt":0,"prompt_hash":"e2d1cc56574b9e3fee034af4e6b699e2a40403b38d3349f54e622fcded650294","question":"A girl walk in the room front the right doorway and sits down at the table. They begin arm wrestling and the girl wins. the girl\n\nA. smiles at the camera sighting her victory then they both get up to exit room from the same sides they entered in from.\nB. begins time dancing on stage and laughing.\nC. takes the camera to show her date.\nD. smiles a silly little smile indicating to them that they won.","prompt":"A girl walk in the room front the right doorway and sits down at the table. They begin arm wrestling and the girl wins. the girl","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7454,"error":null,"output":"A"}
{"question_id":"hellaswag:00004413","item_index":129,"attempt":0,"prompt_hash":"ebe5289e7473bdefd44cdbb620f34df7a54d0577333d107320af18a51d8830f7","question":"[header] How to invite someone to chat in gmail [title] Open the gmail website. [step] If you haven't done so already, log in with your gmail email address and password. [title] Click \" hangouts conversations \".\n\nA. [step] It's in the top-right of the window, just above the gmail message wheel. [title] Select \" people \", then select \" someone \" from the drop-down list.\nB. [step] This displays a list of all of your various hangouts and friends. [title] Click \" invite \" when it appears on your current profile page.\nC. [step] This is the quotation mark icon in the bottom left corner of your page. [substeps] Skip this step if your chat bar is already on the hangouts conversations menu.\nD. [step] If you don't see this button, skip to step 4 of this article. [title] Click \" chat books \" on the menu of your browser.","prompt":"[header] How to invite someone to chat in gmail [title] Open the gmail website. [step] If you haven't done so already, log in with your gmail email address and password. [title] Click \" hangouts conversations \".","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7454,"error":null,"output":"A"}
{"question_id":"hellaswag:00001071","item_index":130,"attempt":0,"prompt_hash":"8478a84771bbbad0c018d910ef8751e66d5de4d319af8af0f32871643e27acd3","question":"People are playing basketball on a court. they\n\nA. start to take shots at each other.\nB. fight over the ball and then give it to each other.\nC. jump and throw the ball back and forth.\nD. shoot the balls into the net.","prompt":"People are playing basketball on a court. they","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7452,"error":null,"output":"C"}
{"question_id":"hellaswag:00006686","item_index":131,"attempt":0,"prompt_hash":"20050b7b26ceaa7ef99e5834cf72ffd14766c3db822c4c6cbc78acf8f5a92112","question":"[header] How to do a sun salutation [title] Learn the benefits of sun salutations. [step] Surya namaskar is an important foundational vinyasa in yoga that will energize, calm, and relax you. It can also stretch your entire body while strengthening your arms, shoulders, and legs.\n\nA. Here's how to do this in surya namaskar : [substeps] 1. Strengthen all of your arms, palms together.\nB. If you find yourself lacking focus or just don't find sun salutations to be successful, then simple sun salutations may be sufficient. [substeps] Consonants: these are individual sounds that begin with the individual letter so you can express meaning through the verbal sound.\nC. [substeps] According to mantra, sun salutations give you the energy and mood needed to nominate yourself for life. The suffix surya imaskar, which means scar-free, energizes you for life.\nD. Practicing regularly may aid digestion, and relieve back pain. [substeps] Talk to your doctor before you begin any yoga practice to make sure you're healthy enough to practice.","prompt":"[header] How to do a sun salutation [title] Learn the benefits of sun salutations. [step] Surya namaskar is an important foundational vinyasa in yoga that will energize, calm, and relax you. It can also stretch your entire body while strengthening your arms, shoulders, and legs.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7447,"error":null,"output":"D"}
{"question_id":"hellaswag:00006568","item_index":132,"attempt":0,"prompt_hash":"d010a525444e6a396f04aab93798d57607b8715a2a3541b38adc250cb55bcf32","question":"[header] How to stop hard water from ruining your skin [title] Moisturize skin after washing. [step] Regular usage of hard water can lead to dry, irritated skin. Combat this effect right after showering, bathing, or washing your face by moisturizing right away.\n\nA. After you have showered, apply a moisturizing cream. [substeps] Leave the cream on for about ten minutes.\nB. Use a hydrating moisturizer, sunscreen, or lotion to pat your skin dry, moisturize, and reduce swelling. [substeps] Taking a shower before bed after heavy meals can reduce the moisture the skin under the pads of your feet keeps.\nC. Use a facial moisturizer and rich body lotion, and apply generously. [substeps] A non-alcohol toner can also be used after washing to remove traces of minerals and heavy metals.\nD. [substeps] Use a moisturizing spray or formula moisturizer as these can combat dry, irritated skin. [title] Apply aloe vera on your skin after you wash it.","prompt":"[header] How to stop hard water from ruining your skin [title] Moisturize skin after washing. [step] Regular usage of hard water can lead to dry, irritated skin. Combat this effect right after showering, bathing, or washing your face by moisturizing right away.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7446,"error":null,"output":"A"}
{"question_id":"hellaswag:00002845","item_index":133,"attempt":0,"prompt_hash":"5920bd2390cf2831ccb8ef228c581a4e84dc3c047ba970b6c6ced27f9dca0092","question":"A young child is seen standing before a car holding a rag and looking at the camera. the boy\n\nA. then walks up to the car and begins tearing it up with a stick and other objects.\nB. then wipes down the car and climbs back in the car and starts driving out into the street.\nC. uses the rag all over the car while the camera follows his movements.\nD. then then pushes the rag down and begins pulling on a tire.","prompt":"A young child is seen standing before a car holding a rag and looking at the camera. the boy","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7452,"error":null,"output":"C"}
{"question_id":"hellaswag:00009450","item_index":134,"attempt":0,"prompt_hash":"f38d92e49fff109a1050bb7d9d0a2dc63422fced3f9d9210c552d3e77d85aac7","question":"[header] How to apply makeup for photos [title] Start with primer. [step] You want to start off with a basic primer. This lessens the appearance of makeup and fine lines.\n\nA. Remove primer and wipe from under your eyebrow. [substeps] Use primer if you have black lashes, or if there is makeup available at home.\nB. Apply primer all over your face, including between your nose and cheeks. Brush your primer onto every blemish or spotted.\nC. It also helps to define the surface of your skin without smudging it. Apply any primer you have that you have on hand.\nD. Apply a layer of primer to your face using your fingers. Make sure to especially target your nose, under your eyes, your forehead, and your cheekbones.","prompt":"[header] How to apply makeup for photos [title] Start with primer. [step] You want to start off with a basic primer. This lessens the appearance of makeup and fine lines.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7452,"error":null,"output":"D"}
{"question_id":"hellaswag:00006489","item_index":135,"attempt":0,"prompt_hash":"3de83d85d85de763262caba56313756398057cd528a7d76bc6151559ef6fba6d","question":"[header] How to be prepared for natural disasters [title] Fill out an emergency plan. [step] Create an emergency plan for your family in case of a natural disaster. Include household information, out-of-town contacts, and school, workplace, and childcare contact information and emergency contingencies.\n\nA. [title] Keep a first aid kit and medical gear handy. [step] As a safety precaution, keep ready items like towels, extra clothing, medicine, allergy pads, anti-inflammatories, and emergency rags.\nB. [substeps] Don't forget to include locations for emergency facilities and, ultimately, childcare. [title] Map out a safe evacuation route for family emergencies.\nC. [title] Write down items that will help you be more prepared. [step] For example, consider buying a dress for every time you see a power failure.\nD. Add your evacuation routes and shelter plans as well. Several websites have templates for emergency plans, like.","prompt":"[header] How to be prepared for natural disasters [title] Fill out an emergency plan. [step] Create an emergency plan for your family in case of a natural disaster. Include household information, out-of-town contacts, and school, workplace, and childcare contact information and emergency contingencies.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7446,"error":null,"output":"D"}
{"question_id":"hellaswag:00007614","item_index":136,"attempt":0,"prompt_hash":"dbd52ba1e3ef92e996c022dd56ae2be65bb55c7d1eddafc91cd20352b13caf90","question":"[header] How to look great in a yearbook [title] Keep it fresh. [step] Don't forget that your hygiene is just as important as having an amazing smile in your photo. You should take a shower and wash your face before you show those pearly whites.\n\nA. Always keep your hair well-styled and wear it down. [substeps] If you always wear your hair down, take it down to your waist.\nB. [substeps] If possible, don't wear makeup, especially if your your school has one. You just don't want to put on too much anyway.\nC. You should also wash your hair that is already in the photo. Don't let your hair look milky, flaky, messy, or greasy.\nD. [substeps] If you normally shower in the evenings, try to make time for a morning shower on the day of your photo. This will make your skin look even better.","prompt":"[header] How to look great in a yearbook [title] Keep it fresh. [step] Don't forget that your hygiene is just as important as having an amazing smile in your photo. You should take a shower and wash your face before you show those pearly whites.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7446,"error":null,"output":"D"}
{"question_id":"hellaswag:00007337","item_index":137,"attempt":0,"prompt_hash":"8df6d7b553342768b139ab208739a28dc3551e8ba032ad81b9833fdb54899e38","question":"[header] How to flirt with a cancer [title] Let them lead. [step] Cancers are both born leaders and astute financial planners. If they are a part of a group, they are likely to be at the head, leading it.\n\nA. They are also very good with their money and can be quite successful at anything they set out to achieve. [substeps] Don't be intimidated by this aspect of cancer.\nB. Walk up and ask them to let you join. The more likely they are to say yes, the more likely they are to give you a good chance at getting out.\nC. Instead, they're often in charge of planning proposals. When a cancer research project's worth of information is revealed, they immediately intervene in your cancer plan.\nD. Watch the approach of your cancer woman closely. If your cancer woman seems nervous, or very insecure in any of her leadership roles, that should not indicate the woman is a serious wreck.","prompt":"[header] How to flirt with a cancer [title] Let them lead. [step] Cancers are both born leaders and astute financial planners. If they are a part of a group, they are likely to be at the head, leading it.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7447,"error":null,"output":"A"}
{"question_id":"hellaswag:00009227","item_index":138,"attempt":0,"prompt_hash":"b7b9939436f29e75ede43687b89d1911c52581bd71b749eaf707f94131a9ed88","question":"[header] How to introduce a child to a new boyfriend [title] This first step is one that should be considered in all new relationships where a child is concerned. [step] Make sure that you and your partner have a stable and happy relationship with potential for a future before making any introductions. Chopping and changing partners and introducing your child to every man you meet can be emotionally damaging and confusing to the child.\n\nA. Be honest and open as you present your child with the attentive eye of a man with whom you already have a relationship. [title] Determine whether you feel comfortable with a new woman to begin with.\nB. [title] When setting up a new partner, teach your child that this person should be trusted and that it is \" for the good of both parties. [step] \" [title] If your partner has agreed, give this to your child.\nC. Children are susceptible to forming bonds with people quickly and if the relationship is unstable and your partner leaves, your child will also suffer the loss. Be sure of your relationship before making any decisions.\nD. Use skill sets you or your partner have learned through doing these things. [substeps] Avoid demeaning or protecting your child at all costs.","prompt":"[header] How to introduce a child to a new boyfriend [title] This first step is one that should be considered in all new relationships where a child is concerned. [step] Make sure that you and your partner have a stable and happy relationship with potential for a future before making any introductions. Chopping and changing partners and introducing your child to every man you meet can be emotionally damaging and confusing to the child.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7450,"error":null,"output":"C"}
{"question_id":"hellaswag:00008723","item_index":139,"attempt":0,"prompt_hash":"d6d48c61d8f9fc23e049dcbc0e35a31b136d29eccf0851b24828bb6b8fb8181e","question":"[header] How to do your makeup in the summer [title] Use an oil-free moisturizer. [step] This is a must, even if you don't have oily skin. This will keep your skin hydrated throughout the day but without the extra oils.\n\nA. Choose a hydrating moisturizer with a basic oil of your choice. [substeps] Use loose cream or lotion moisturizers to keep your skin hydrated.\nB. [substeps] If you're in a hurry, take an anti-aging cream and applying it while you are tanning. This is the most effective way to clean up your face and get your skin like it always does.\nC. [title] Make your freckles look real. [step] This can come in the form of straight shaped silver or copper freckles.\nD. Look for a moisturizer that says \" oil-free \" or \" non-comedogenic \" (won't clog your pores) on the label. [substeps] Choose a moisturizer that is labeled for your skin type: normal, dry, oily, etc.","prompt":"[header] How to do your makeup in the summer [title] Use an oil-free moisturizer. [step] This is a must, even if you don't have oily skin. This will keep your skin hydrated throughout the day but without the extra oils.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7444,"error":null,"output":"D"}
{"question_id":"hellaswag:00008673","item_index":140,"attempt":0,"prompt_hash":"15a524d666681d6add781750d42759efe3cb1d1a155174914e30cf88443e3f6a","question":"[header] How to look good in shorts [title] Take another person shopping. [step] If you can, always have a shopping buddy. This person can be a friend, family member, or significant other.\n\nA. Go shopping with a friend if you can or an acquaintance who can help you out if you find a time to use the bathroom. Talk with your friend about which type of clothes you like.\nB. If they have a younger sister or cousin, this will help. It also needs to be someone who understands that you love shopping.\nC. While this is not necessity, do also try to pick someone you don't know well. You can make them doubt themselves or perhaps feel insecure.\nD. When you try on different pairs of shorts, ask him or her if you look good in them. [substeps] It's important to get someone else's perspective when shopping, since too often people are their own worst critics.","prompt":"[header] How to look good in shorts [title] Take another person shopping. [step] If you can, always have a shopping buddy. This person can be a friend, family member, or significant other.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7449,"error":null,"output":"D"}
{"question_id":"hellaswag:00002420","item_index":141,"attempt":0,"prompt_hash":"25f558e4a5cff06618e6b1207119ea4aa9770081259468ebc924d475a30b2848","question":"A stationary bike class is being held in a darkened room. The instructor points to the class. she\n\nA. then explains the mechanics behind using the inet.\nB. squats on the floor and starts moving her hand around before the class picks her up and places her on the left hip.\nC. points to several people with their life parts strapped into hines.\nD. leans back and speaks to the class.","prompt":"A stationary bike class is being held in a darkened room. The instructor points to the class. she","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7448,"error":null,"output":"D"}
{"question_id":"hellaswag:00000416","item_index":142,"attempt":0,"prompt_hash":"54ac7672dfa97207b98b103c8ba18117eb817bcf296bdb6889d7cbcb3e80c5fd","question":"The player at the bottom of the screen scores and the camera man shows the score and the replay on a screen near the table. the player\n\nA. at the bottom of the screen and the player at the bottom rises from the table.\nB. at the top of the screen scores and the camera man shows the screen again with an instant replay and the score.\nC. at the top of the screen, scores again and one of the players in the black and white video melts for a bit.\nD. stands and walks to the corner, threw money at the player who scores and the video starts again.","prompt":"The player at the bottom of the screen scores and the camera man shows the score and the replay on a screen near the table. the player","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7449,"error":null,"output":"B"}
{"question_id":"hellaswag:00004586","item_index":143,"attempt":0,"prompt_hash":"3031a5dd10857ed02cce79c2be6d9945b402ad37048e8141d24f2f957c2b82ea","question":"[header] How to draft an indemnity agreement [title] Search for samples. [step] You may be able to find a sample online that you can use as a guide for your own indemnity agreement. Many legal service companies also offer forms or templates to create an indemnity agreement, although you typically must pay a fee to access them.\n\nA. [substeps] If you need to draft your own agreement, get permission from the judge to do so. Many court websites have sample agreements available for people to enact before in-court conversations.\nB. [substeps] Samples used by other companies also may help you decide how to word your clauses, or give you an idea of what kind of information you need to include. Make sure any agreements you look at deal with situations similar to yours.\nC. [substeps] You can use free software such as adobe acrobat that collects materials by using adobe acrobat. You can collect samples on your computer, or create an agreement by hand.\nD. [substeps] Many banks, credit unions, and other private organizations offer sample outlines, while others also offer sample agreements. Look at the terms and conditions of your agreement to determine if you typically need to obtain a judge's approval to draft the agreement.","prompt":"[header] How to draft an indemnity agreement [title] Search for samples. [step] You may be able to find a sample online that you can use as a guide for your own indemnity agreement. Many legal service companies also offer forms or templates to create an indemnity agreement, although you typically must pay a fee to access them.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7447,"error":null,"output":"B"}
{"question_id":"hellaswag:00002683","item_index":144,"attempt":0,"prompt_hash":"911a3270f322ae5a927479742f8b79e284f0a4e6d838d875b8c7246b010f6e9c","question":"He shows the two fish that he has caught. Then he puts his fishing net in a hole in the lake and continues fishing. he\n\nA. gets the fish out and grabbed a fishing rod and begins fishing out of a hole that he dug.\nB. catches another dark gray fish and tosses it in his box.\nC. catches a large fish that he tied up with a line as he feeds.\nD. catches a pickled four fish.","prompt":"He shows the two fish that he has caught. Then he puts his fishing net in a hole in the lake and continues fishing. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7448,"error":null,"output":"B"}
{"question_id":"hellaswag:00004842","item_index":145,"attempt":0,"prompt_hash":"34aca411dc594f5d3e4a57258773feb3a5d1ac54712ba6053f5c96c16c326125","question":"[header] How to shave your face [title] Choose an appropriate razor. [step] You should take into account the coarseness of your beard hairs, the texture of your skin, your preferred shaving method, and other details when choosing a razor. Generally speaking, for heavy-bearded men with more sensitive skin, it is generally good to use a traditional multi-blade razor.\n\nA. A more expensive variety, however, may suit you. [substeps] If possible, choose a razor with a finer blade, like the ninja rat razor (also known as ninja rat 4) or saber (also known as elderrawal 2).\nB. [substeps] Electric razors are convenient and quick, requiring less preparation before shaving than safety razors, and are somewhat gentler on sensitive skin. For some types of hair, however, they tend to leave an uneven or patchy shave on the face.\nC. You may also want to check ahead of time which method is best for you. [substeps] Before shaving, you should consider using a traditional electric razor.\nD. [substeps] The point of a professional shave is to smooth the beard hair and keep your razor soft and and straight. Using a razor is not only effective, but to save yourself the stress of sawing through to scar a smooth, beautiful beard.","prompt":"[header] How to shave your face [title] Choose an appropriate razor. [step] You should take into account the coarseness of your beard hairs, the texture of your skin, your preferred shaving method, and other details when choosing a razor. Generally speaking, for heavy-bearded men with more sensitive skin, it is generally good to use a traditional multi-blade razor.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7448,"error":null,"output":"B"}
{"question_id":"hellaswag:00003464","item_index":146,"attempt":0,"prompt_hash":"eb7b291052f70a1ca921cac0718cb171710b3e62990684f71ea51bbf06e8fd36","question":"[header] How to bake a cake inside an egg [title] Using the knife, cut a hole in the egg. [step] This is so you can put the batter in. [title] Pour all of the yolk out.\n\nA. [step] This is so you won't have a yolky cake in the egg. Pour out as much yolk as possible.\nB. [step] You can hold the egg still for further, or [title] Bake the egg with a fork. [step] Once cooked, cover it with the cap and bake, or until the egg turns golden brown and the edges are brown.\nC. [step] Scoop some of the ice cream that was added in the bottom of the pan into the hole. [title] Pour the flour, water, and jam into the empty pan.\nD. [step] Set the glass down and pound around the bag little by little. [title] Pour all of the batter into the egg and gently press down on it.","prompt":"[header] How to bake a cake inside an egg [title] Using the knife, cut a hole in the egg. [step] This is so you can put the batter in. [title] Pour all of the yolk out.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7446,"error":null,"output":"A"}
{"question_id":"hellaswag:00001010","item_index":147,"attempt":0,"prompt_hash":"f932b1a8e9f75ca27c1ac030b86da596e613f5c1f56a7e896ffe92aa995d88dc","question":"The boy is standing under the tree with blindfold thing to hit the pinata but missed it. a man\n\nA. is showing the pinata the kid hit and he pulls the stick out until it hits an orange pinata.\nB. came up to him and assist him, the boy hit the pinata once.\nC. put a piece of blindfolds on the boy and he tries to hit it with his hand and can't find it.\nD. is kneeling in front of the boy in the room.","prompt":"The boy is standing under the tree with blindfold thing to hit the pinata but missed it. a man","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7447,"error":null,"output":"D"}
{"question_id":"hellaswag:00007007","item_index":148,"attempt":0,"prompt_hash":"026317bda384a6e257f0563d8a35506a2786871f4c5dae9f02811309a957f2fc","question":"[header] How to calculate fluid pressure [title] Set up the equation. [step] The equation for fluid pressure is independent of the mass or volume of the liquid. Instead, fluid pressure is the product of the density of the liquid, the height of the liquid above the object, and gravity.\n\nA. Since gravity and liquid densities are fixed (for the most part), the height of the liquid is the largest variable in the equation. [substeps] The equation reads as p fluid = ρgh, where ρ is the density of the liquid, g is the force of gravity, and h is the height of the liquid.\nB. If the solution offers no useful information, then it is likely to be simply an aeration effect of the gases in your prop prop. Calculate the volume of the liquid by assuming the formula needed to determine how many aspirin ounces you'll need for a prop.\nC. [substeps] A problem for fluid pressure is that a part of the body that is logically the size of the objects object. This is because liquid pressure is the constant object low to the water.\nD. [substeps] Separate objects by the force of gravity. Start by dividing the weight of the liquid by its volume.","prompt":"[header] How to calculate fluid pressure [title] Set up the equation. [step] The equation for fluid pressure is independent of the mass or volume of the liquid. Instead, fluid pressure is the product of the density of the liquid, the height of the liquid above the object, and gravity.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7446,"error":null,"output":"A"}
{"question_id":"hellaswag:00005389","item_index":149,"attempt":0,"prompt_hash":"07d2c2ab7efa0edba63fd185892333ed1bf758dd6f85442bdba77f87cc31f027","question":"[header] How to stop trying to control others [title] Learn to self soothe. [step] Sometimes, you might find yourself trying to control others by relying heavily on them to calm you down when you are upset. You might demand that they say or do something and feel that you can calm down only if that action occurs.\n\nA. You could sometimes be a victim of a situation where someone \" helps \" you, but that person may have no real value in your life. Learning to self soothe is a valuable skill that will make you better able to handle yourself.\nB. Remind yourself that it is all about you, not the other way around. [substeps] Think about what one out of every ten people has in common: all people fall into the same category.\nC. However, you must not depend on others to provide you with peace; you must find that for yourself. Learn and develop self soothing techniques to better learn to calm yourself.\nD. Stress and anxiety can cause you to interrupt conversations and pull away from others. Developing self-soothing skills will help you stay away from these behaviors, rather than letting them drive you away.","prompt":"[header] How to stop trying to control others [title] Learn to self soothe. [step] Sometimes, you might find yourself trying to control others by relying heavily on them to calm you down when you are upset. You might demand that they say or do something and feel that you can calm down only if that action occurs.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7448,"error":null,"output":"C"}
{"question_id":"hellaswag:00009754","item_index":150,"attempt":0,"prompt_hash":"015fd3521f2322d4f4a9db5ea096ecd42bed54d0ef5bfeed59b68174d99c06b7","question":"[header] How to create chips and dip [title] Pre-heat the oven to 200 degrees, before-hand. [title] Wash the potatoes. [step] Make sure they are clean and any other bits which stick out have been peeled.\n\nA. Though, leave the skin on if you can! [title] Cut them. [step] Cut the potatoes in half, half again, and half again! Cut all potatoes into eighths.\nB. Either peel the potatoes or cut them up into small pieces. [title] Place 1 chicken head into the baking dish.\nC. [title] Place them in the oven and broil the potatoes for about 20 minutes. [step] This will give them their look, but not be their killing agent.\nD. Split them in quarters, one or two shells. This way they are easier to dip and don't get moist during cooking.","prompt":"[header] How to create chips and dip [title] Pre-heat the oven to 200 degrees, before-hand. [title] Wash the potatoes. [step] Make sure they are clean and any other bits which stick out have been peeled.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6471,"error":null,"output":"A"}
{"question_id":"hellaswag:00002679","item_index":151,"attempt":0,"prompt_hash":"180d20f01642e5e658830227be1ac8315d56aea24f24a31c0275127d03929497","question":"A woman stands holding a violin against herself. The woman plays the violin. the woman\n\nA. stops playing the violin.\nB. stops moving, lets the violin fall, smiles, then touches the strings.\nC. drops her violin and lays it on the ground.\nD. smiles and gives lift.","prompt":"A woman stands holding a violin against herself. The woman plays the violin. the woman","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6470,"error":null,"output":"A"}
{"question_id":"hellaswag:00000335","item_index":152,"attempt":0,"prompt_hash":"bf792ffdb20e97a92790fc5d74fb83143da9ff53ae6bb3767f6e783a2e7fec18","question":"A woman is getting ready to bath her dog in her front yard. she\n\nA. gets under the shower spray, kissing the dog on the face before stepping back out.\nB. talks to the camera for a little while, then walks towards the shower.\nC. allows the dog an electric bath, then she takes the dog outside.\nD. takes a hose and wets the dog first.","prompt":"A woman is getting ready to bath her dog in her front yard. she","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6469,"error":null,"output":"D"}
{"question_id":"hellaswag:00004582","item_index":153,"attempt":0,"prompt_hash":"2e59a92b741b36267d2919cb6b94b1afa45a9ae00255fd66a7ac216351012143","question":"[header] How to ask god for forgiveness [title] Name and admit the thing you've done. [step] Before asking forgiveness, you must specifically state the thing you did wrong and admit that you did it. If you're feeling guilty, you may be tempted to make excuses or deny that you did anything wrong.\n\nA. However, once you're able to admit the thing you didn't wrong, you can acknowledge exactly what you did and why the person wronged you. In real life, you are responsible for making sure the person you're asking forgiveness doesn't get very angry and lose faith in you.\nB. However, you have the right to name the thing you couldn't make but admitted and admit it afterwards. If you've admitted, apologize, and perhaps...\nC. But lying might seem juvenile, so it's important to clarify what you did wrong to justify hurting someone in some way, and it can help to understand what it would mean if you didn't get the gift you deserve. [substeps] While it's tempting to blame someone for your sin, you shouldn't.\nD. Forgiveness is impossible if you don't admit you did something wrong. [substeps] You may think, \" i guess maybe i shouldn't have lied, but i had a really good reason and it was a small lie.","prompt":"[header] How to ask god for forgiveness [title] Name and admit the thing you've done. [step] Before asking forgiveness, you must specifically state the thing you did wrong and admit that you did it. If you're feeling guilty, you may be tempted to make excuses or deny that you did anything wrong.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6470,"error":null,"output":"D"}
{"question_id":"hellaswag:00001859","item_index":154,"attempt":0,"prompt_hash":"dac954eda18c5e5267019ee20298fd9c216c8b6396c488e7ffe01219483ed64b","question":"A cat is on the screen with a title of the video. A dog and cat are walking along a brick sidewalk. They pass some puddles and keep walking. the dog\n\nA. stops to sniff the wall as the cat continues on.\nB. and cat do occasional tricks towards each other as they pass the trash as they paw at it.\nC. is pointing to the items in front of him.\nD. then does off on its own on the sidewalk.","prompt":"A cat is on the screen with a title of the video. A dog and cat are walking along a brick sidewalk. They pass some puddles and keep walking. the dog","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6469,"error":null,"output":"A"}
{"question_id":"hellaswag:00005696","item_index":155,"attempt":0,"prompt_hash":"fd6a0cf38c7c0869d1221df5d06901bde560b9fbef7dca58402671c67eddb1c9","question":"[header] How to do a dip powder manicure [title] Use polish remover as well as a cuticle pusher. [step] If you have polish on your nails, remove it with non-acetone nail polish remover on a lint-free pad. Use a cuticle pusher to gently nudge your cuticles slightly back.\n\nA. [substeps] Only use polish remover or a cuticle pusher if you have dirty hands. [title] Write on your nails with the cuticle pusher.\nB. Do not push them back forcefully so you don't cause pain to your fingers or toenails. However, press them firmly with your fingers.\nC. Work the pusher in a circular motion to stimulate blood flow. [title] Apply a leave-in conditioner after misting your cuticles.\nD. Gently remove any excess cuticle sticking up with a cuticle trimmer or scraper. [substeps] This will allow new nail growth to show so that your manicure will last longer before needing a fill.","prompt":"[header] How to do a dip powder manicure [title] Use polish remover as well as a cuticle pusher. [step] If you have polish on your nails, remove it with non-acetone nail polish remover on a lint-free pad. Use a cuticle pusher to gently nudge your cuticles slightly back.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5231,"error":null,"output":"C"}
{"question_id":"hellaswag:00002592","item_index":156,"attempt":0,"prompt_hash":"95521ff6252712935454b9cecaa5153aa3480269eddf2d0403fe7940b9053647","question":"People pedal in static bikes in a gym. a man\n\nA. swings his bike over a bar.\nB. is ready to jump, then his bike makes a jump in the dirt covered ground.\nC. rinses easily in the sink.\nD. raise his left hand and touches his head.","prompt":"People pedal in static bikes in a gym. a man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5231,"error":null,"output":"D"}
{"question_id":"hellaswag:00005798","item_index":157,"attempt":0,"prompt_hash":"d8742b884318cd046f742cf948a6043fcd0314bcf30f08390e2fbaa779fb204f","question":"[header] How to attract barn owls [title] Keep your shed open to the wild. [step] Clear everything out and give it a bit of a dust, and a barn owl will probably make a nest in there. [substeps] Make sure you leave the door open for the barn owls.\n\nA. They can climb in, if they do not have wings or owls need air. Consider bringing a pair of wooden wading poles to keep them out.\nB. This way, they won't be able to climb in! [title] Provide enough shelter. [step] The barn owl might stumble into something on the way home from a trip to town.\nC. Don't worry about mice and rats getting in, as the barn owls will eat them. [title] Put a bird box on a pole, about 2 and a half metres off the ground.\nD. Barn owls aren't allowed to come too close to your barn entrance, so they shouldn't come in. [title] Purchase an owl-repellent.","prompt":"[header] How to attract barn owls [title] Keep your shed open to the wild. [step] Clear everything out and give it a bit of a dust, and a barn owl will probably make a nest in there. [substeps] Make sure you leave the door open for the barn owls.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5232,"error":null,"output":"C"}
{"question_id":"hellaswag:00008786","item_index":158,"attempt":0,"prompt_hash":"0cef33fb56c113b27a8a6565c10041db60d74b1aaa30dc4138afdb727517376f","question":"[header] How to make tattoo ink [title] Burn wood to create ashes. [step] In order for wood ashes to work well as a tattoo ink, you need to completely burn the wood you're using. The ashes should be totally black, with none of the wood's natural color still visible.\n\nA. [substeps] In order to get the right amount of wood ash to burn, you need to burn it for at least a month so you can use darker ink. Sage or willow ash is also a popular choice for ashes.\nB. If the wood isn't black, draw a line of black ash over the entire surface of your wood to create a vertical line on your wood. [substeps] Skip out on the next step for darker wood ash.\nC. [substeps] If you're working with leftover wood, after you've burned it down to ash, you'll need to leave it on for 20 to 30 minutes. [title] Mix a paste with water and a few drops of liquid glue.\nD. They should also be completely cooled when you use them. [substeps] Different types of wood will give you a slightly different shades of color when the ashes are used as ink.","prompt":"[header] How to make tattoo ink [title] Burn wood to create ashes. [step] In order for wood ashes to work well as a tattoo ink, you need to completely burn the wood you're using. The ashes should be totally black, with none of the wood's natural color still visible.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5231,"error":null,"output":"D"}
{"question_id":"hellaswag:00001296","item_index":159,"attempt":0,"prompt_hash":"280a3c10d15dddbeb78cb6b89508416f8ab2a535ceea6d4dc1be4e3661a1eb2c","question":"The person shines her right boot in the electric buffer machine. The person switches feet and shines the other boot. the woman\n\nA. lifts up her boot to show the polished finish.\nB. removes her other shoe.\nC. shines the left one.\nD. puts her right foot into the harmonica machine.","prompt":"The person shines her right boot in the electric buffer machine. The person switches feet and shines the other boot. the woman","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5229,"error":null,"output":"A"}
{"question_id":"hellaswag:00008902","item_index":160,"attempt":0,"prompt_hash":"fe2a74d3e00311e955ae592485036c78da8ea838c7d402f973f24b330b62e450","question":"[header] How to choose a tampon size [title] Learn about absorbency options. [step] Tampon sizes correspond the amount of liquid they can absorb. You can select the right absorbency level for you based on the level of your flow.\n\nA. The absorbency level (ac) is expressed as the flow of liquid. The ac is the amount of liquid your vagina submerse into, while ac is the amount of gas your body likes to use in terms of actually giving you gas.\nB. Absorbency is one of the most important factors when you're choosing a tampon. [substeps] Absorbency is the amount of fluids your body uses per minute.\nC. The most common tampon absorbency levels (moving from smallest to largest) include : [substeps] Regular super super plus some brands may also offer junior/slim (smaller than regular) and/or ultra (larger than super plus). [title] Choose the lowest level of absorbency needed to avoid tss.\nD. The absorbency rating will also depend on your specific condition. [substeps] Keep in mind that absorbency is not due to gender, gender, age, or reduce weight.","prompt":"[header] How to choose a tampon size [title] Learn about absorbency options. [step] Tampon sizes correspond the amount of liquid they can absorb. You can select the right absorbency level for you based on the level of your flow.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4439,"error":null,"output":"C"}
{"question_id":"hellaswag:00005010","item_index":161,"attempt":0,"prompt_hash":"c8c1c567a80d83f7a3b38a4eac685827a1f4f71acf5182e33ee5a47b6ec496a2","question":"[header] How to pay attention in a dull class [title] Create small goals and mini-rewards to go with them. [step] For example, tell yourself that if you pay attention for the next 15 minutes, you can have a few peanut m&ms from the bag you've stashed in your backpack. For every additional 15 minutes, you get a couple more.\n\nA. You may want to redistribute what you bring in to share with other students. Or, you may need some helpful volunteer help to progress towards your goal.\nB. As the two days continue to pass by, make it more realistic with small rewards. One day you can read out loud to your kids but at dinner time, reward them with a small conversation.\nC. Or, instead of m&ms, you can allow yourself to check your phone briefly. [substeps] You could also tell yourself that if you take good notes through the entire class, you can play your new video game for an hour when you get home.\nD. The next 15 minutes, you get a treat, an extra minute of class. [title] Don't be a perfectionist.","prompt":"[header] How to pay attention in a dull class [title] Create small goals and mini-rewards to go with them. [step] For example, tell yourself that if you pay attention for the next 15 minutes, you can have a few peanut m&ms from the bag you've stashed in your backpack. For every additional 15 minutes, you get a couple more.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4438,"error":null,"output":"C"}
{"question_id":"hellaswag:00000745","item_index":162,"attempt":0,"prompt_hash":"d2dae0b57f901b383f62be1942fcaa7c7ceac5102a31bd53dd10e7b3ed9299d8","question":"There is one man who's in the room with two other men and he lifts up a weigh that weighs over 400 pounds. next he\n\nA. leans forward towards another man, who measures the weight on the machine before letting go to fetch another one.\nB. dips the weight in a glass of icy water and then down again.\nC. picks up an additional burden of weight up and drops it on the other man.\nD. lifts up a weight that is heavier than the last one.","prompt":"There is one man who's in the room with two other men and he lifts up a weigh that weighs over 400 pounds. next he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4444,"error":null,"output":"D"}
{"question_id":"hellaswag:00000237","item_index":163,"attempt":0,"prompt_hash":"edb9655bfa7c5f676e0cf61fc5096b416053793687c8e22bc1416e2d71df42de","question":"A girl stands in a bathroom talking to the camera as a man films her. The lady wipes her eye makeup off and washes her face. the lady\n\nA. puts gloss on her lips place a liner in her eye and smiles to the camera.\nB. scrubs her contacts using a brush.\nC. rinses the soap off her face.\nD. then puts makeup on the makeup artists face.","prompt":"A girl stands in a bathroom talking to the camera as a man films her. The lady wipes her eye makeup off and washes her face. the lady","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4444,"error":null,"output":"C"}
{"question_id":"hellaswag:00002754","item_index":164,"attempt":0,"prompt_hash":"c54f5d713f05c0250ef25235ecdf600962b3255f976c081e5a83e211ded61ebe","question":"A mom is sitting at the top of a slide with her little baby. She is contemplating going down for a while. she\n\nA. climbs down and continues to fall down as they catch her.\nB. slides down and the little baby follows at a faster speed.\nC. finally slides down it with baby.\nD. decides to do it instead.","prompt":"A mom is sitting at the top of a slide with her little baby. She is contemplating going down for a while. she","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4443,"error":null,"output":"B"}
{"question_id":"hellaswag:00001611","item_index":165,"attempt":0,"prompt_hash":"2f4ff8b592fd4a39066b88bbe4bf7de0cf0f2a0018673da8ed18e4c17949735f","question":"The man talks to the camera again. The man sharpens and then holds up a knife. Several types of knife storage options are displayed. the man\n\nA. demonstrates how to retrieve knives from storage.\nB. sharpens and removes supplies from bags.\nC. talks to the camera some more.\nD. points at several things on the table.","prompt":"The man talks to the camera again. The man sharpens and then holds up a knife. Several types of knife storage options are displayed. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4443,"error":null,"output":"C"}
{"question_id":"hellaswag:00007310","item_index":166,"attempt":0,"prompt_hash":"e0778ba474d9e95dff6a671efdf12675129221d63f9f8181d140be396b4a2a18","question":"[header] How to dress for work [title] Dress business formal if you work with customers in a professional setting, such as a bank or office. [step] Business formal usually means wearing a suit, tie, dress slacks, and dress shoes. [substeps] Wear clothes that fit.\n\nA. Investing in appropriate attire can be a good way to cut down on the amount of time you spend milling about. Find shoes that fit you well and conform to your workplace's culture.\nB. A business casual outfit is best for the workplace in a professional setting than for a night on the town. Something like khakis can work just as well as a suit with a tie.\nC. You might shop in jeans or a nice shirt for business casual. On the other hand, plain button-down blouses are a good option for business casual.\nD. Clothes that are too tight will be too uncomfortable to work in. Clothes that are too loose will look sloppy and unprofessional.","prompt":"[header] How to dress for work [title] Dress business formal if you work with customers in a professional setting, such as a bank or office. [step] Business formal usually means wearing a suit, tie, dress slacks, and dress shoes. [substeps] Wear clothes that fit.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4435,"error":null,"output":"D"}
{"question_id":"hellaswag:00006148","item_index":167,"attempt":0,"prompt_hash":"3ea1d5f4d7b97313ef5e923653eaa2ddc1cfa15233b65b0aa32d6e2faa361ea8","question":"[header] How to wash out hair dye [title] Fade your hair color with the vitamin c method. [step] This is widely reported to fade color quickly and without causing too much damage to your hair. [substeps] Crush vitamin c tablets with a pestle and mortar or in a plastic baggie with a rolling pin or hammer.\n\nA. Crush the tablets on a rolling pin until they are light and dust evenly onto your hair. Divide your hair into two halves, and roll them between your palms until each half is covered in 1 ounce of the paste.\nB. Move the crushed tablets to a small bowl and add a tablespoon of water to make a paste. Apply the paste to your hair and leave in for about 30 minutes, then rinse with warm water.\nC. They should be pulverized in 1 tablespoon, then crushed by rolling an exfoliating brush and water. Use smaller amounts to soften darker hair and more often to \" mash \" lighter hair.\nD. Squeeze the tablets, turn them upside down, and store in the shower or to serve as a paste. Add a couple of drops of your favorite essential oil or conditioner and a little dish detergent to the water.","prompt":"[header] How to wash out hair dye [title] Fade your hair color with the vitamin c method. [step] This is widely reported to fade color quickly and without causing too much damage to your hair. [substeps] Crush vitamin c tablets with a pestle and mortar or in a plastic baggie with a rolling pin or hammer.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4437,"error":null,"output":"B"}
{"question_id":"hellaswag:00000671","item_index":168,"attempt":0,"prompt_hash":"6389f8e94da71a3f473e9a34bd853fd912d5dd8158461fb31a4f6b6858ff2674","question":"The man cranks the drill and drills a hole in the ice. The man then catches a fish on his line and pull it from the water. the man\n\nA. stands up and ties his fishing line to the end of the pole and tosses it to the far end of the lake.\nB. pulls out a white fish and uses wire cutters to drill holes into it.\nC. grabs more other fish.\nD. picks up a very big fish and holds it for the camera.","prompt":"The man cranks the drill and drills a hole in the ice. The man then catches a fish on his line and pull it from the water. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4440,"error":null,"output":"D"}
{"question_id":"hellaswag:00008003","item_index":169,"attempt":0,"prompt_hash":"a0d15d171b5a317a41a1a0ef1885fde606546ecd18b00d4458f826bd5f3f473d","question":"[header] How to keep a cat from running out the door [title] Use an alternate entrance and exit. [step] If your cat is constantly hanging out by the front door, waiting to make its escape when some unwitting person opens it, try exiting out a different door. For instance, instead of going in and out through the front door, try using the back or garage door.\n\nA. Another good option is to exit and enter via a door with an antechamber. After passing through the first door, close it firmly behind you and look about to ensure your feline friend hasn't followed you.\nB. Another option is to bypass your house's rear entrance and enter the garage door, opening it by turning the opposite way. This will block the exit, while also allowing the cat to do the same.\nC. Be aware of this rule before making a move toward your other door; your cat will pounce on you and use this as an opening to escape. [substeps] Avoid allowing your cat to enter one door first.\nD. Of course, this will only take up half the room in which your cat can go and should be considered a safe option. However, remember that your cat has a natural instinct to leave that door open if something happens.","prompt":"[header] How to keep a cat from running out the door [title] Use an alternate entrance and exit. [step] If your cat is constantly hanging out by the front door, waiting to make its escape when some unwitting person opens it, try exiting out a different door. For instance, instead of going in and out through the front door, try using the back or garage door.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4437,"error":null,"output":"A"}
{"question_id":"hellaswag:00003646","item_index":170,"attempt":0,"prompt_hash":"f828adaaad69865d929556df6dcee285484e06e2b56ad0735947b9d7adf707e4","question":"[header] How to clean rainbow sandals [title] Prepare some soapy water. [step] You will want to use lukewarm water, as warm or hot water may damage the leather soles of your rainbow sandals. Add a drop or two of dish soap and stir to combine.\n\nA. [title] Soak the laces for at least 20 minutes. [step] This loosens the laces in a way that will allow you to reattach them to the sandals.\nB. [title] Wet both feet (or your right foot as necessary). [step] At this point you are still wet.\nC. [title] Soak a washcloth in the soapy water and wring out any excess. [step] Wring out approximately one teaspoon of dish soap to ensure that it's combined correctly.\nD. [title] Wipe your shoes down with a damp cloth. [step] Dip a corner of a clean rag into the soapy water, and squeeze it a bit to get rid of excess water.","prompt":"[header] How to clean rainbow sandals [title] Prepare some soapy water. [step] You will want to use lukewarm water, as warm or hot water may damage the leather soles of your rainbow sandals. Add a drop or two of dish soap and stir to combine.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4436,"error":null,"output":"D"}
{"question_id":"hellaswag:00001146","item_index":171,"attempt":0,"prompt_hash":"a448cb6bd10b61aaf990f628a8400cfbd4c5dd7ba561df46359e699dec4289a9","question":"A woman is seen standing in a field holding frisbees with a dog. the woman\n\nA. rides the dog all through the field while the dog talks to the camera.\nB. begins throwing around the frisbee performing tricks with the dog.\nC. throws the frisbees far and the dog catches them and runs around the woman.\nD. then tricks the dog around while he runs around in circles.","prompt":"A woman is seen standing in a field holding frisbees with a dog. the woman","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4437,"error":null,"output":"C"}
{"question_id":"hellaswag:00004282","item_index":172,"attempt":0,"prompt_hash":"5623d40fb8b828d487384dc190c3c8e9db9996b5612170e255231c2d9fde58be","question":"[header] How to create a bathroom shelving unit [title] Identify an area in your bathroom where you can hang the shelves and take measurements. [step] Visit the home improvement store and ask the lumber expert about types of wood that would be complimentary for your project, any types of wood on special and durability. Give the lumber expert your measurements and ask for advice on the amount if you are unsure how much wood you should purchase.\n\nA. [title] Buy wood to create the shelving. [step] These shelves can be either stained or painted depending on your personal style.\nB. [substeps] Include a wall that you plan to hang your shelves on. This will likely consist of loose hanging boards and heavy boards for the shelves.\nC. [title] Make the room in which the shelves will be located. [step] Plan out the same area along the wall all the way around.\nD. Remember, even a tiniest amount on one shelf can be a large lot of wood. [title] Decide what you will use for the shelving unit and perhaps some furniture pieces.","prompt":"[header] How to create a bathroom shelving unit [title] Identify an area in your bathroom where you can hang the shelves and take measurements. [step] Visit the home improvement store and ask the lumber expert about types of wood that would be complimentary for your project, any types of wood on special and durability. Give the lumber expert your measurements and ask for advice on the amount if you are unsure how much wood you should purchase.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4436,"error":null,"output":"A"}
{"question_id":"hellaswag:00006090","item_index":173,"attempt":0,"prompt_hash":"d4c15b023066e8bb517b02fab7f2382a1166f7ca7f1b80d70620d99f03a9761a","question":"[header] How to plan a romantic evening [title] Start early. [step] Get an early start by looking ahead and planning in advance. People get busy and schedules get filled up, so it is best to start planning at least a month ahead of time.\n\nA. This is especially important if the romantic evening is for an anniversary or other special occasion. Begin considering activities and thinking about special moments that your partner will enjoy.\nB. Try to begin planning about 15 to 20 days in advance and go straight to planning for the evening at least in advance. [substeps] To start, have a conversation with your family.\nC. Generally people begin with a single date, even a month or two in advance. Don't expect to rush in to planning.\nD. [title] Decide what you like and what you'll do when you go out. [step] Don't plan plans so far in advance that you won't be invited to events.","prompt":"[header] How to plan a romantic evening [title] Start early. [step] Get an early start by looking ahead and planning in advance. People get busy and schedules get filled up, so it is best to start planning at least a month ahead of time.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4436,"error":null,"output":"A"}
{"question_id":"hellaswag:00005435","item_index":174,"attempt":0,"prompt_hash":"81e1c88bb3c37216cb400cf5110a46bdddca22a583dcd4b23240b8b522f20d81","question":"[header] How to make a lemon face cleanser [title] Pour 1 teaspoon of lemon juice into a small bowl. [step] Lemon juice is naturally antibacterial, and it can help reduce acne. It would be best if you use fresh lemon juice for this, but bottled may also work.\n\nA. [title] Add 6 drops of lemon essential oil and 1 teaspoon of honey to the bowl. [step] Fresh lemon oil has antibacterial properties and makes your skin more pleasant, which aids in reducing acne.\nB. [title] Massage the lemon juice into your face with gentle circular motions. [step] Start with the freckle, and work up towards it.\nC. If you are using fresh lemon juice, be sure to strain out the seeds and pulp first. [title] Add 1 tablespoon (15 grams) of plain yogurt.\nD. [substeps] Most grocery stores do not provide lemon juice. [title] Blend the lemon with 1 tablespoon of water.","prompt":"[header] How to make a lemon face cleanser [title] Pour 1 teaspoon of lemon juice into a small bowl. [step] Lemon juice is naturally antibacterial, and it can help reduce acne. It would be best if you use fresh lemon juice for this, but bottled may also work.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4436,"error":null,"output":"C"}
{"question_id":"hellaswag:00000301","item_index":175,"attempt":0,"prompt_hash":"4a576e0a13275ca5102a0b45830ebce5a70db1653f13eb5f3412ffb711771a2b","question":"There's a woman in a yellow tank top and black tights doing jump rope in a gym. She jumps in a rhythmic manner without stopping on the hardwood floor surface. after she\n\nA. exercises and focuses on her rhythm, she jumps and flips around the mat a few times before unstrapping.\nB. jumps, she walks off the platform and grabs a selfie stick from a side table.\nC. is done jump roping, she flips back on the ropes and sits back down on them.\nD. 's done, a person standing in front of her gives her a thumbs up and cheers for her.","prompt":"There's a woman in a yellow tank top and black tights doing jump rope in a gym. She jumps in a rhythmic manner without stopping on the hardwood floor surface. after she","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4439,"error":null,"output":"D"}
{"question_id":"hellaswag:00000534","item_index":176,"attempt":0,"prompt_hash":"a7c7c8b7c4bc402dcee092f925510395118c5e1886d8eebf8e852643e3177098","question":"A rabbit is cleaning itself with its tongue. it\n\nA. looks up at the camera and the camera holds its eye open.\nB. is underneath a chair.\nC. lays down on its back while it sniffs information.\nD. waves its front paw at the camera.","prompt":"A rabbit is cleaning itself with its tongue. it","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4437,"error":null,"output":"D"}
{"question_id":"hellaswag:00007268","item_index":177,"attempt":0,"prompt_hash":"9c9e5d836d999276b39ec62bc7ff48f0633622ae1e6d60da82478ecefb1a744a","question":"[header] How to clean cuts [title] Wash your hands. [step] Before attempting to clean a cut, either on yourself or on another person, it is key that your own hands be clean. This way, you will not introduce any germs or foreign substances into the cut that may later cause it to become infected.\n\nA. [substeps] If available, put on some single-use gloves after washing your hands for more protection [title] Stop the bleeding. [step] Before cleaning the cut, it is also key to stop the bleeding.\nB. You must wash your hands thoroughly after you touch any part of your skin that is touched. [substeps] To clean your cuts, wash them thoroughly in warm water and soap before and after touching your cuts.\nC. The simplest solution for a cut that is on your hand is to wash it with soap and warm water to begin your cleaning process. [substeps] To wash the cut, first treat it with soap and water.\nD. [substeps] If you wear rubber or gloves, wash those first with warm water. If you wear latex gloves, wash those too.","prompt":"[header] How to clean cuts [title] Wash your hands. [step] Before attempting to clean a cut, either on yourself or on another person, it is key that your own hands be clean. This way, you will not introduce any germs or foreign substances into the cut that may later cause it to become infected.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4439,"error":null,"output":"A"}
{"question_id":"hellaswag:00001260","item_index":178,"attempt":0,"prompt_hash":"e2a1ddd9c257738bc211abc8c5641ee8e453564c1f466eb4fafd222df04c7ea0","question":"A young man wearing a cuervo black shirt stares and speaks to the camera as he sits on his chair. he\n\nA. puts a harmonica to his mouth and begins playing.\nB. puts a bar of soap on his hands from a dispenser in the corner.\nC. is shown attempting to water ski in the water.\nD. interwops with stills of people playing drums and drums on a snare.","prompt":"A young man wearing a cuervo black shirt stares and speaks to the camera as he sits on his chair. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4439,"error":null,"output":"A"}
{"question_id":"hellaswag:00005605","item_index":179,"attempt":0,"prompt_hash":"e97bf9507567bc280e4d36e86f9e6519e1bfdc7313cc625f41908496d6f4142b","question":"[header] How to wear a velvet blazer [title] Choose a blazer with stand-out style. [step] While a standard velvet blazer is nice, you can add extra \" wow \" factor by picking a blazer that has unique features. For example, sharp shoulders, exaggerated lapels, funky draping, or an hourglass waist.\n\nA. In addition, look for one that leaves room in the shoulders to cross front and back, and then flares out at the knees. [title] Choose a silk blazer with flat shoulders.\nB. Opt for a blazer with a designer quality or authority, rather than a feeling of boy or girl style. [substeps] An authoritative blazer that compliments many aspects of velvet is a good choice for warm weather.\nC. Women's velvet blazers look gorgeous with cuffed or 3/4 length sleeves that taper from loose in the upper arm to fitted at the forearm. [substeps] Consider a blazer that's either high-waisted or extra long.\nD. This will make you stand out in a trendy blazer. [substeps] Watch out for blazers with built-in pockets and narrow sleeves.","prompt":"[header] How to wear a velvet blazer [title] Choose a blazer with stand-out style. [step] While a standard velvet blazer is nice, you can add extra \" wow \" factor by picking a blazer that has unique features. For example, sharp shoulders, exaggerated lapels, funky draping, or an hourglass waist.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4438,"error":null,"output":"C"}
{"question_id":"hellaswag:00001017","item_index":180,"attempt":0,"prompt_hash":"269accabdf041553658452a7edb1b596508f1417000d7a630a29cbe5100fef5c","question":"A man walks across some snow carrying a large snow shovel. The man shovels a long path of snow. the man\n\nA. turns around and shovels a long path of snow again.\nB. continues to shovel the snow and glide through it.\nC. turns and bends over.\nD. begins shoveling the snow.","prompt":"A man walks across some snow carrying a large snow shovel. The man shovels a long path of snow. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4438,"error":null,"output":"A"}
{"question_id":"hellaswag:00001925","item_index":181,"attempt":0,"prompt_hash":"7106280ba3a6a6be9d9687ae3a6a4a1a1e9c3ff17deb0d9afdd5bbbc518680be","question":"A person mows the lawn while a dog and a person play catch with a ball on the same lawn at the same time. a lawnmower\n\nA. begins mowing a small grass covered lawn while a dog walks around the perimeter.\nB. separates the two dogs and we see the bows.\nC. works on the lawn at one time.\nD. turns in the yard of a house and a person blow their breath into the air.","prompt":"A person mows the lawn while a dog and a person play catch with a ball on the same lawn at the same time. a lawnmower","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4438,"error":null,"output":"A"}
{"question_id":"hellaswag:00003793","item_index":182,"attempt":0,"prompt_hash":"580e3ed32c691a80dcfa6ff5beb7c52b3a745aae07ac08595e6c46a5c48a96c5","question":"[header] How to deal with teenage boys [title] Know that going through puberty can change their point of view. [step] Boys normally go through puberty between the ages of 11 and 16. It is during these years that they experience most of their physical changes (including growing taller and developing muscle).\n\nA. They start understanding the psychopaths in their teens and need to learn skills to deal with these biological change. Studies show that peers (teens or teenagers) show that teenagers can experience puberty that is related to their individual abilities, even earlier in life.\nB. This is especially true of boys who have a harder time getting the muscular build they need to be strong and sporty. [substeps] Boys start developing testosterone at about 16, but boys tend to grow taller as they get older and are more developed.\nC. You also go through puberty from four to six months old, and grown some over the years. Parents who disagree with a girl about the situation are another area where girls may experience strong feelings, so find an adult you trust.\nD. During and after these years they'll normally start to develop their sexuality. They'll start to look at themselves, and others, differently.","prompt":"[header] How to deal with teenage boys [title] Know that going through puberty can change their point of view. [step] Boys normally go through puberty between the ages of 11 and 16. It is during these years that they experience most of their physical changes (including growing taller and developing muscle).","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4197,"error":null,"output":"D"}
{"question_id":"hellaswag:00008884","item_index":183,"attempt":0,"prompt_hash":"d16afe65e73e2993dd1a753ffdc4bff9bca6a499a667ad4ea519b24b1d718ceb","question":"[header] How to clean rusty tools [title] Wash away dirt and grease. [step] Mix grease-cutting dish detergent into warm water until you get suds. Place the tools into the soapy water.\n\nA. Mix one or two teaspoons of dish soap. Dip a sponge into the soapy water and then wipe down the tools while they're still inside the container.\nB. Swirl the tools along the seams to make sure they've gotten as good a job cleaning off grime as possible. [substeps] You don't need to rinse the tools this way, unless you did them themselves.\nC. While the tools are submerged, scrub them using a sponge or rag until the oil and dirt are gone, then remove the tools from the water. [substeps] The soap and water should mix more easily if you pour the detergent into the bowl before adding the water.\nD. Use a plastic scrubber for deep cleaning purposes, and scrub with a scrub brush. Scrub your tools with each stroke for at least 10 seconds until soapy.","prompt":"[header] How to clean rusty tools [title] Wash away dirt and grease. [step] Mix grease-cutting dish detergent into warm water until you get suds. Place the tools into the soapy water.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4197,"error":null,"output":"C"}
{"question_id":"hellaswag:00000923","item_index":184,"attempt":0,"prompt_hash":"01a872c17ba39ba5f53680d1f5f4d5aeeb04875952bddd5a05b09de1bc842f28","question":"A woman is in a living room in front of a fire. She cuts wrapping paper in strips, then wraps it neatly around a box. she\n\nA. grabs an egg and circles it with a silver marker.\nB. tapes the box, then lays it down gently on the table.\nC. lets the wrapping paper fall away from the box at the end.\nD. puts icing on the top, then seals it.","prompt":"A woman is in a living room in front of a fire. She cuts wrapping paper in strips, then wraps it neatly around a box. she","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4197,"error":null,"output":"B"}
{"question_id":"hellaswag:00005811","item_index":185,"attempt":0,"prompt_hash":"4c97e5803edf2199ffc181f9c89883faebb0a2ae3d7516badbcb4bb894f536f6","question":"[header] How to start a game like eve online [title] Patience is a virtue. [step] For the first few days, it can't be helped to feel being overwhelmed. Just stick with it, and be patient.\n\nA. Finally, it's time to learn how to make things better, which means you're not being driven/anxious or fussy when others are around. [title] People are more influenced by such activities as drawing or typing or word editing.\nB. You're going to get bored a little more, but you'll get out of it very quickly. [substeps] Having patience is what effect eve has on anyone.\nC. Sooner or later you will get accustomed to the rules of space [title] Eve online has a 14-day trial period. [step] Make use of this free time in understanding and exploring the game.\nD. Yes, eve has a knack for fixing problems, but she is much better at fast-tracking that fix now than she is at solving problems already in progress. [title] Take pride in who you are! [step] Eve is balanced.","prompt":"[header] How to start a game like eve online [title] Patience is a virtue. [step] For the first few days, it can't be helped to feel being overwhelmed. Just stick with it, and be patient.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4197,"error":null,"output":"C"}
{"question_id":"hellaswag:00003616","item_index":186,"attempt":0,"prompt_hash":"c68286e6f5661eb0a92770a3706707ae814f75dd7b2e6c8dd3291449aa66df78","question":"[header] How to hide your nipples [title] Wear reusable silicone nipple covers under thin fabrics. [step] These small, circular or flower-shaped adhesive coverings are meant to cover only your nipples, so they're a great option for low-cut tops and thin or sheer fabrics that might make your nipples visible. Stick the adhesive side onto your nipple and press it down gently.\n\nA. [substeps] You can get small silicone cover coverings specially designed for high necklines or drop tops. Silicone covers can be particularly helpful if you're trying to hide your breasts from crowds or high tech shoppers.\nB. Your breasts will no longer be covered by the covering. [substeps] This option is available at most department stores.\nC. Look for a color close your skin tone for the least noticeable solution. [substeps] You can buy stick-on nipple covers online or in stores that sell bras and related accessories.\nD. [substeps] Silicone is flammable and usually contains mercury, which can be harmful to skin. Wear plastic foil tops to protect your skin from moisture.","prompt":"[header] How to hide your nipples [title] Wear reusable silicone nipple covers under thin fabrics. [step] These small, circular or flower-shaped adhesive coverings are meant to cover only your nipples, so they're a great option for low-cut tops and thin or sheer fabrics that might make your nipples visible. Stick the adhesive side onto your nipple and press it down gently.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4197,"error":null,"output":"C"}
{"question_id":"hellaswag:00007675","item_index":187,"attempt":0,"prompt_hash":"fefeafff6bcc34da9503504198e3d3fba7e25db709186fb8547582e69029a1bc","question":"[header] How to wire transfer money [title] Learn about wire transfers. [step] A wire transfer is an electronic method of transferring funds from one person or company to another. One common method of sending a transfer is from one bank to another.\n\nA. [substeps] You will have to walk and talk to the bank if you plan to transfer funds by person. If you intend to transfer your funds through your home, this method will also work.\nB. The bank does not charge for the transfer. However, it might be better to send one to a third person than have them exchange the transfer with someone else.\nC. Most wire transfers require an identification or credit card (such as a visa or mastercard) or cash (such as an atm atm card). If you have a credit card or cash, use it to transfer funds into another account instead of an individual one.\nD. To send a wire transfer, the sender provides information on the receiving party's financial institution to their own bank. Once the transfer is complete, the sender's bank provides a confirmation number documenting the transfer.","prompt":"[header] How to wire transfer money [title] Learn about wire transfers. [step] A wire transfer is an electronic method of transferring funds from one person or company to another. One common method of sending a transfer is from one bank to another.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5404,"error":null,"output":"D"}
{"question_id":"hellaswag:00003090","item_index":188,"attempt":0,"prompt_hash":"9235689d1fb35285e1f1da3993d932c3eb2d72036296f46ea4e38dbc90a1d36c","question":"Little kid is sitting in a babies eating chair. An adult person is holding an ice cream and is offering it to the baby. the kid\n\nA. takes the ice cream and shakes it.\nB. grabs the ice cream and start eating.\nC. bit the ice cream and licked the spoon and then push it to the mouth and discussed food.\nD. eats both of the ice cream.","prompt":"Little kid is sitting in a babies eating chair. An adult person is holding an ice cream and is offering it to the baby. the kid","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5403,"error":null,"output":"B"}
{"question_id":"hellaswag:00007452","item_index":189,"attempt":0,"prompt_hash":"417400c3e0e2eab3c01c8c19453dc46eca5995629ae643786b7b18b33f164df0","question":"[header] How to give yourself a buzzcut [title] Invest in a pair of quality clippers. [step] Buying clippers is a one-time investment and since you'll be saving tons of money cutting your own hair, it's worth getting the best pair that you can afford. You can get a decent pair for $30-$50.\n\nA. If cutting your own hair is going to be long term thing, consider getting professional grade clippers (usually $100-$120). [substeps] Look for a sturdy pair with sharp blades that won't dull quickly.\nB. Getting your things cut at a salon is free, and having a quality cut can be really lucrative when you go to an adequate salon. The important thing is that you have a solid and light pair of clippers, available to everyone, to prevent unwanted sideburns.\nC. In addition to cutting hair with clippers, there are a number of clippers available to buy that go with a drastic right cut. [substeps] Consider getting a clipper that's designed for a certain height.\nD. [title] Find a wig-pin watch-style wig box. [step] Wigs are an expensive and tricky hobby which requires patience, self-care, skill, and long hair.","prompt":"[header] How to give yourself a buzzcut [title] Invest in a pair of quality clippers. [step] Buying clippers is a one-time investment and since you'll be saving tons of money cutting your own hair, it's worth getting the best pair that you can afford. You can get a decent pair for $30-$50.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5404,"error":null,"output":"A"}
{"question_id":"hellaswag:00009128","item_index":190,"attempt":0,"prompt_hash":"2268aac2d0b1804843e5e00fb88933edab0baf26b146e46d85477782ac8ab79e","question":"[header] How to help kids feel loved [title] Tell them you love them. [step] This maybe seem obvious and redundant, but if you repeatedly tell a child you love them, it will become set in stone, and they will never forget it. Tell them you love them when they do something good, bad, interesting, funny, adorable, or just randomly.\n\nA. Here's how to say that you love them when they get better. [substeps] Say hello when they ask for help.\nB. Try making words after each point, so kids know exactly what you are saying. [substeps] Try making creative jokes about their looks.\nC. No time is a bad time to tell your child you love them. [substeps] Note: it's a good idea to tell the child you love them while scolding them, if you're mad, or if they're in trouble.\nD. Say you love them often, and that you're happy about whatever you do. The most important thing is that children see the positive side of things, rather than the negative side.","prompt":"[header] How to help kids feel loved [title] Tell them you love them. [step] This maybe seem obvious and redundant, but if you repeatedly tell a child you love them, it will become set in stone, and they will never forget it. Tell them you love them when they do something good, bad, interesting, funny, adorable, or just randomly.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5403,"error":null,"output":"C"}
{"question_id":"hellaswag:00002119","item_index":191,"attempt":0,"prompt_hash":"1326b4ab5014e522c455ce0e77ccf01830cf1bc6d98ec9e34ad30876cd8101aa","question":"A set of knifes rest on top of pin cones. A pocket knife is taken from wooden box and held with a clamp. the man\n\nA. then puts the knife in the bowl and screws it tight.\nB. picks up and starts carving the knife on the table.\nC. uses sharpening stones on the blade of the knife.\nD. peels a small nut on the sharpener.","prompt":"A set of knifes rest on top of pin cones. A pocket knife is taken from wooden box and held with a clamp. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5403,"error":null,"output":"C"}
{"question_id":"hellaswag:00002794","item_index":192,"attempt":0,"prompt_hash":"6b2d8720ea38cec7f656e3813c7c93d7bef4029b7e9c67f14547bd159c4e7432","question":"Man is in roofed gym weightlifting. Woman is walking behind the man watching the man. woman\n\nA. is tightening balls on stand on front of weight bar.\nB. is lifting eights while he man sits to watch her.\nC. is doing mediocrity spinning on the floor.\nD. lift the weight lift man.","prompt":"Man is in roofed gym weightlifting. Woman is walking behind the man watching the man. woman","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6216,"error":null,"output":"C"}
{"question_id":"hellaswag:00008915","item_index":193,"attempt":0,"prompt_hash":"41359477e6110a7baa6074d22c390c37abf530cbcff9b5cf7db9186cd81ae436","question":"[header] How to help prevent nuclear war [title] Show up to town hall meetings, if applicable. [step] If you have the opportunity to go to a town hall meeting in your city or town, use this as a chance to voice your concerns about the threat of nuclear war. The representatives at this meeting will be able to hear your message, as well as others attending the town hall meeting.\n\nA. [substeps] Prepare what you're going to say before attending the meeting. Use the information you gathered in your research to support your argument.\nB. [title] Discuss program issues if conditions or passions stop your ideas. [step] Discuss local issues that you are worried about and what is causing them.\nC. [title] Help to make cleanup less disruptive to the surrounding environment. [step] You don't have to let the public know about the threat, of course.\nD. [substeps] Make sure to introduce yourself and outline your cause. For example, say something like, \" i'm a citizen of america and i want to help in the developing world.","prompt":"[header] How to help prevent nuclear war [title] Show up to town hall meetings, if applicable. [step] If you have the opportunity to go to a town hall meeting in your city or town, use this as a chance to voice your concerns about the threat of nuclear war. The representatives at this meeting will be able to hear your message, as well as others attending the town hall meeting.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6211,"error":null,"output":"A"}
{"question_id":"hellaswag:00005708","item_index":194,"attempt":0,"prompt_hash":"042ff9907630eb5cbb4d8dfc12ada1c8b20a1e66b89433d4a1c8186b0141c62f","question":"[header] How to wear off the shoulder tops [title] Pick an off-the-shoulder top that fits your height. [step] The length of your off-the-shoulder top should correspond with your height. If you're on the taller side, you can balance your height out by wearing long tops.\n\nA. Longer tops on men tend to hug your midsection and look more obvious than short tops on women. [substeps] If you're short, you may only be able to wear two tops under the same length, but with a full length off-the-shoulder top, you can be able to wear more tops.\nB. If you are on the shorter side, you're more likely to stand out from other people. [title] Pick an off-the-shoulder top with a low neckline.\nC. If you're shorter, it's better to stick to shorter off-the-shoulder tops, or tuck long tops into the front of your pants. [substeps] Longer tops are ones that can go below the waist of your pants or skirt, or ones that cover your butt when you wear them.\nD. You can even wedge the top up your side, so it covers your shoulders. [substeps] To add more dimension, try wearing shorts, short sleeved tops, or neutral tops.","prompt":"[header] How to wear off the shoulder tops [title] Pick an off-the-shoulder top that fits your height. [step] The length of your off-the-shoulder top should correspond with your height. If you're on the taller side, you can balance your height out by wearing long tops.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6211,"error":null,"output":"B"}
{"question_id":"hellaswag:00003457","item_index":195,"attempt":0,"prompt_hash":"1f570dc2ad2ebf859a9955a429fcb26e42345f2b642678b2438dfc1472fbd94c","question":"[header] How to save money when you have a spending habit [title] Realize that you have a problem. [step] The first step to solve any issue is to be self-aware. It's easy to be in denial, so if you're conscious of your habits then you already won half of the battle.\n\nA. But if you're insightful about them you'll be better equipped to assess your situation and take action to overcome it. [substeps] Ask for clarification if you can; you may be either losing money or difficulty in accomplishing your goals.\nB. Remember that having a problem is a reflection of yourself, so don't be shy about noticing it. [substeps] By acknowledging that you have a problem, you can develop confidence in your ability to have control over your habits.\nC. When you're aware of the negative spending habits, you start second guessing every cash exchange, swipe and proceed to checkout click. Even if you continue to spend carelessly, the \" should i do this \" thought will linger in the back of your mind.\nD. Ignoring this behavior can cause you to spend more money on other expenses. [substeps] Think about what you can do to change your habit and stick with it.","prompt":"[header] How to save money when you have a spending habit [title] Realize that you have a problem. [step] The first step to solve any issue is to be self-aware. It's easy to be in denial, so if you're conscious of your habits then you already won half of the battle.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6209,"error":null,"output":"B"}
{"question_id":"hellaswag:00006445","item_index":196,"attempt":0,"prompt_hash":"6b3cc8d8ebea53120e00ed828dfe7913fd7189567ae45f08ee0c87316a29cd9a","question":"[header] How to care for superworms [title] Choose a container. [step] A plastic container can work well for superworms. You can also pick up an aquarium at a local pet store.\n\nA. A dark-colored container with drainage holes will also work. Since superworms rarely burrow or crawl underground, it's best to find a tank that is specifically made for superworms.\nB. The plastic container will prevent the thousands of superworms per bag. A larger container will also prevent the hundreds of eggs and larvae that hatch from superworms from escaping the container and forming the worms.\nC. The container should be large enough that its top will be at least three inches away from the substrate. [substeps] You can easily use a plastic container you have lying around the house for superworms.\nD. Your container should be big enough to comfortably hold hundreds of superworms, which will be frozen overnight. While the container can be wide, you should be able to fill it up to your standard size.","prompt":"[header] How to care for superworms [title] Choose a container. [step] A plastic container can work well for superworms. You can also pick up an aquarium at a local pet store.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6216,"error":null,"output":"C"}
{"question_id":"hellaswag:00001272","item_index":197,"attempt":0,"prompt_hash":"153d339ba50b634e030f724d2848325bb30294a260462d3376504c61f4067fb4","question":"Men are seen holding hooks up to the camera followed by a fish moving under water and a close up of a fishing hole. The men are seen sitting over a hole fishing on the ice and holding up fish. the men\n\nA. continue dragging the fish in the hole and watched by the camera in the end.\nB. continue to dig while a group of medics are shown looking at the fish.\nC. put a rod in the fish hole and drain it out.\nD. continue pulling fish out of the water presenting it to the camera and moving in slow motion.","prompt":"Men are seen holding hooks up to the camera followed by a fish moving under water and a close up of a fishing hole. The men are seen sitting over a hole fishing on the ice and holding up fish. the men","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6214,"error":null,"output":"D"}
{"question_id":"hellaswag:00005931","item_index":198,"attempt":0,"prompt_hash":"6cd380658c826067ad508830c3955e8ce0db829a71ddb3f27aaa03dab7e2a716","question":"[header] How to cook squirrel [title] Place the squirrel pieces in a large saucepan and cover with water. [step] Place the saucepan on the stove and bring to a boil, then reduce the heat to a simmer. Allow the squirrel meat to simmer until tender, about an hour and a half.\n\nA. [title] Drain and cool the squirrel pieces in a colander. [step] The squirrel pieces have completely cooled, but retain their moisture.\nB. [title] Taste and season with salt and pepper, when done. [step] For a hearty, nutritious version of squirrel stew, sprinkle a pinch of salt and pepper onto the meat after the squirrel pieces have been cooked for a few minutes and then serve the stew over cooked rice.\nC. [substeps] Let the squirrel meat rest for 30 to 60 minutes or until it is soft, which is when it's ready to be cooked. [title] Cut open the squirrel pieces and cook them over medium heat.\nD. [substeps] Be sure to simmer, not boil the meat; it should not be falling off the bone when you remove it from heat. If you have an older squirrel, it may take longer for the meat to become tender.","prompt":"[header] How to cook squirrel [title] Place the squirrel pieces in a large saucepan and cover with water. [step] Place the saucepan on the stove and bring to a boil, then reduce the heat to a simmer. Allow the squirrel meat to simmer until tender, about an hour and a half.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6210,"error":null,"output":"A"}
{"question_id":"hellaswag:00006005","item_index":199,"attempt":0,"prompt_hash":"455b9fa0f5074cf8a2b60951dea1c735cef26f516513ed7a27e2af992d2ca3a2","question":"[header] How to discuss sex with your child [title] Start early. [step] If you get an indication that your child is being exposed to sexual information, then it is time to start talking with them yourself. Even infants are curious about their bodies, and by preschool age they may show an intellectual curiosity regarding others' bodies as well as their own.\n\nA. By eight years old many kids are ready to have some form of a sex conversation with you. Remember to keep the conversation age appropriate, too.\nB. [title] Figure out what your child wants to talk about. [step] If your child seems interested in sex, discuss certain things you can do to encourage their curiosity.\nC. Start by beginning the conversation by saying hello when they open the door. If they seem interested in the topic, make sure you contribute to the conversation by opening up about it and start making small talk every now and then.\nD. [substeps] You should start trying to handle your child's sexual curiosity as soon as possible. Tell them that they are old enough and have a few questions for you.","prompt":"[header] How to discuss sex with your child [title] Start early. [step] If you get an indication that your child is being exposed to sexual information, then it is time to start talking with them yourself. Even infants are curious about their bodies, and by preschool age they may show an intellectual curiosity regarding others' bodies as well as their own.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6212,"error":null,"output":"B"}
{"question_id":"hellaswag:00008674","item_index":200,"attempt":0,"prompt_hash":"debddff899b38461d7e12c2197c366e55cf1d047766395b642b7c0f765709fff","question":"[header] How to curl hair without a hair curler [title] Brush and section off your hair. [step] Start the curling process by brushing your hair to remove any knots or tangles. Next, divide your hair into sections.\n\nA. Section off about ½ inch from the last section. Use a round brush to brush out some of the knots so your hair doesn't dry out.\nB. Comb through your hair to remove any knots and loose strands. Then, part your hair on one side and section off the bottom half.\nC. Take the top section near the crown of your head first, and leave the hair at the nape of your neck and sides of your head for later. [substeps] It's best to use a natural bristle brush when you're brushing your hair because it will reduce the amount of static and frizz.\nD. Use your fingers to work 1-2 small sections out of your hair. If you have very thick hair, it may be easier to work all of your sections out.","prompt":"[header] How to curl hair without a hair curler [title] Brush and section off your hair. [step] Start the curling process by brushing your hair to remove any knots or tangles. Next, divide your hair into sections.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6208,"error":null,"output":"B"}
{"question_id":"hellaswag:00001516","item_index":201,"attempt":0,"prompt_hash":"da27f390cbe462ddfd807c65754d871caae17f052b0bbed0f115b72de68c0882","question":"A boy sits in the water, his skis out in front of him, while he holds the handle of the rope between his legs. we\n\nA. see a short flashback as we then see him in the boat in then in the water being assisted with his skis.\nB. see the slide he will attempt one at a time, from the beginning.\nC. see the boy paddle his lake, then glide down the river at the edge of the dock.\nD. see the boy underwater, speaking to the camera.","prompt":"A boy sits in the water, his skis out in front of him, while he holds the handle of the rope between his legs. we","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6211,"error":null,"output":"D"}
{"question_id":"hellaswag:00002528","item_index":202,"attempt":0,"prompt_hash":"c3776522e91e830d964cc2f00e25f7459826d43435b422f4a5ae453fec5922ac","question":"The person then cuts up vegetables and butters bread then putting various ingredients on the bread to make a sandwich. the person\n\nA. then cuts apart cheese and eats on the side.\nB. sliced a tomato on a board, added butter, lettuce and other ingredients.\nC. cuts up the sandwich and places it around a plate to present in the end.\nD. then peels up chunks of bread and put them on the sandwich looking like a chef's helper.","prompt":"The person then cuts up vegetables and butters bread then putting various ingredients on the bread to make a sandwich. the person","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6211,"error":null,"output":"C"}
{"question_id":"hellaswag:00005163","item_index":203,"attempt":0,"prompt_hash":"b15173ec569f66dd1182fa6fb24f5575cfdee4c07569632644daf7777063483e","question":"[header] How to choose a credit card by rewards [title] Select a rewards card that allows you to get free gas, plane tickets or rental cars if you travel often. [step] These are usually called travel rewards, allowing you to accrue points toward items that are useful in travel. [title] Find a card that offers cash back if you do not travel often.\n\nA. [title] Find a card that offers discounted prices and rewards for purchases that are unexpected. [title] Call the credit card company directly to acquire a rewards card.\nB. [step] Everyone can use more cash, and this type of rewards card gives you a percentage of the amount you charge on it. Consider the card that offers the highest percentage of cash back.\nC. [step] Most rewards cards offer regular cash back or periodic rewards rather than monthly rewards (like five-hour car rides). Avoid cards like this because they are currently not accepted.\nD. [step] If possible, get one that offers cash back when you get the option. In addition, credit cards often offer credit cards free of charge.","prompt":"[header] How to choose a credit card by rewards [title] Select a rewards card that allows you to get free gas, plane tickets or rental cars if you travel often. [step] These are usually called travel rewards, allowing you to accrue points toward items that are useful in travel. [title] Find a card that offers cash back if you do not travel often.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6210,"error":null,"output":"B"}
{"question_id":"hellaswag:00009453","item_index":204,"attempt":0,"prompt_hash":"26103a2a26e808a0fab4b686d992c50051c07ae862ab18009873b704f25d3057","question":"[header] How to get candle wax out of hair [title] Decide whether to use your shower or sink. [step] If the candle wax has only reached the tips of your hair, you can use a shower. If the candle wax has sunk in towards your roots, it is best to use a sink.\n\nA. A sink allows you to be more up close and personal with the wax you wish to remove. [substeps] To determine this, run your fingers through your hair.\nB. If you need a bathtub, then use a tub. [substeps] If the candle wax has gone down deep in your hair and into your scalp, then a sink would work; but if the wax has spread to just the tips, then you can use a kitchen sink.\nC. [substeps] If you do not feel like using your sink, opt for a sink with only one faucet. Most tap sinks, including twin sinks and bathtub sinks, have both 1 faucet and an outlet at the top.\nD. If the candle wax has had completely sunk in you, a sink is good. [title] Remove or remove wax from hair with a q-tip.","prompt":"[header] How to get candle wax out of hair [title] Decide whether to use your shower or sink. [step] If the candle wax has only reached the tips of your hair, you can use a shower. If the candle wax has sunk in towards your roots, it is best to use a sink.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6209,"error":null,"output":"A"}
{"question_id":"hellaswag:00009235","item_index":205,"attempt":0,"prompt_hash":"4a7e6ecd0633c4c5259e3656c8b55b5159ba8a2c5042ab66fa9db85be4576d7b","question":"[header] How to set dry hair [title] Work a styling product through your hair. [step] Use your favorite styling mousse, spray, or gel. Apply the product to your hair with your fingertips.\n\nA. [substeps] Stop at the roots and comb your hair through after taking your styling product. Do not get spray or gel into your roots because this usually can disturb the curl.\nB. Run the end of a comb through your hair to distribute the styling products. Comb or brush gently through dry hair to distribute the hair.\nC. Keep it in your hair for a few minutes, then rinse out the product and use conditioner. [substeps] If your hair is curly, divide your hair into several sections.\nD. Begin by working the product near the roots of your hair. Continue applying the product as you work your way down to the tips of the strands.","prompt":"[header] How to set dry hair [title] Work a styling product through your hair. [step] Use your favorite styling mousse, spray, or gel. Apply the product to your hair with your fingertips.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6212,"error":null,"output":"D"}
{"question_id":"hellaswag:00005001","item_index":206,"attempt":0,"prompt_hash":"be0724c88821ede1689919922202110aaa48604a3a7ff8bb3576621a7a3cb315","question":"[header] How to avoid raising an emotional eater [title] Eat meals at the dinner table away from sources of entertainment. [step] When kids eat meals in front of the television or ipad, they are detached from the nourishment itself. Eating becomes something that is associated with entertainment instead and your child has no idea how much they are eating.\n\nA. If you tell a kid that eating is good for their mental health, they will learn to enjoy the act of eating as well. [substeps] Cut out junk foods from the main menu when you see them, instead.\nB. Turning off electronics or using music in the car while driving is a good way to avoid the temptation to indulge in entertainment in the car. For example, use the cruise control to turn off the radio/tv when you are driving.\nC. Binge eating and sugary things like cereals and crackers keep kids from being hungry for food, but also distracts them from their emotions. This inbbs positive feelings and can aid in losing weight.\nD. Stop this habit and enjoy meals together at the table. Have polite conversation or listen to classical music while you eat.","prompt":"[header] How to avoid raising an emotional eater [title] Eat meals at the dinner table away from sources of entertainment. [step] When kids eat meals in front of the television or ipad, they are detached from the nourishment itself. Eating becomes something that is associated with entertainment instead and your child has no idea how much they are eating.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6209,"error":null,"output":"D"}
{"question_id":"hellaswag:00001463","item_index":207,"attempt":0,"prompt_hash":"5ff8a87af469160d12a5e6634930e624295a16b4eb7f0a9961825bada2e71ded","question":"Several athletes compete for the javelin world championships. the russia dimitri\n\nA. group is representing the rural tribes in the region.\nB. measuring event in hungarian basketball corp has seen many real points.\nC. is 3rd bronze medal in the competition.\nD. goebaiboros we last finish the match with a parade in front of the west european championships.","prompt":"Several athletes compete for the javelin world championships. the russia dimitri","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6209,"error":null,"output":"C"}
{"question_id":"hellaswag:00003278","item_index":208,"attempt":0,"prompt_hash":"69f6c3f9a652e50efe8f48db0b74f84a1d5d01d8fdbff21c6a356abe1af4e4ba","question":"[header] How to make tahini [title] Preheat the oven to 170ºc/340ºf. [step] Lay parchment paper across the baking sheet or dish. [title] Rinse the sesame seeds in a fine colander. [step] Shake and drip dry.\n\nA. [title] Combine the wet ingredients with salt, pepper, brown sugar, lemon juice, and 2 egg yolks. [step] Combine them and voila! The egg yolks are pronounced exactly the way they were when they were fried (like \" gold minnow eggs! \").\nB. [title] Wash 3 ripe, washed and cut flower heads in half, the raw insides and the once rooted, leaf part. [step] Put the two paprika halves on a plate and smash them with an electric mixer until all the tea leaves and raisins come out well.\nC. [title] Cool them down on the baking sheet for 2 minutes. [step] This moisturizes the seeds and makes them spreadable.\nD. [substeps] Note that if you feel the seeds are clean enough, you may skip this step (although, see tips below for a suggestion about reducing phytic acid). [title] Arrange the seeds across the baking sheet.","prompt":"[header] How to make tahini [title] Preheat the oven to 170ºc/340ºf. [step] Lay parchment paper across the baking sheet or dish. [title] Rinse the sesame seeds in a fine colander. [step] Shake and drip dry.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6207,"error":null,"output":"C"}
{"question_id":"hellaswag:00006503","item_index":209,"attempt":0,"prompt_hash":"f8343bf7149576f4aa759a5cac905c374d8a396ffc4d019fae60eb6cfa346c41","question":"[header] How to stop being in love with a person you'll never meet [title] Keep busy. [step] Stay occupied during your spare time so that you don't give yourself a lot of time to think about them. Take up new hobbies, pick up old ones, ask friends out for coffee or dinner, join a club, take a class, or volunteer.\n\nA. Fill your days and nights so that you're enjoying life and aren't spending much time thinking of them. [substeps] You will notice that thinking of them is regressive-you will think of them less and less over time.\nB. Give your attention to something that interests you, rather than time for their inappropriate activities. [substeps] Many young people find it more normal to be busy with their schoolwork while having a hobby or club.\nC. Don't let your obligations pull you under. [substeps] Find a hobby you find fulfilling and do it regularly.\nD. Just do something that you can't wait to do and that will make you extremely happy. [substeps] Check facebook and/or instagram for ideas about finding something fun to do.","prompt":"[header] How to stop being in love with a person you'll never meet [title] Keep busy. [step] Stay occupied during your spare time so that you don't give yourself a lot of time to think about them. Take up new hobbies, pick up old ones, ask friends out for coffee or dinner, join a club, take a class, or volunteer.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6209,"error":null,"output":"D"}
{"question_id":"hellaswag:00004028","item_index":210,"attempt":0,"prompt_hash":"f9508f33ecf2617b5685f6f96a0fa7d0699c4baf64ba91b302560461e2c1600f","question":"[header] How to bring discipline into your life [title] Analyze yourself first. [step] Work out what things stand in the way of you being more disciplined at the moment. They may be character flaws, an inability to articulate what you want out of life or a preference for losing yourself in stimulants or addictions.\n\nA. [substeps] Identify the problematic aspects of your own work that contribute to your own lack of discipline. Make a list of the things that oppose discipline and take some time to examine the consequences.\nB. Perhaps you conform too much without thinking for yourself; this can make it easy to follow someone else's ideas of discipline without working out discipline that works for you and fulfills your own real needs. Whatever the reason, try to determine it first.\nC. You may not want to be disciplined for each one of these reasons, but they get to the point where they cause you no harm in the long run. [substeps] Do a sit-down appraisal to assess your own personality.\nD. Answer any questions and challenge those assumptions with positive analytical thinking. [substeps] To motivate positive thinking, think about your past accomplishments.","prompt":"[header] How to bring discipline into your life [title] Analyze yourself first. [step] Work out what things stand in the way of you being more disciplined at the moment. They may be character flaws, an inability to articulate what you want out of life or a preference for losing yourself in stimulants or addictions.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6208,"error":null,"output":"A"}
{"question_id":"hellaswag:00001542","item_index":211,"attempt":0,"prompt_hash":"03b20c4ae1eee740b4d62a7685c50d86575473a49a771f9de1587507d628beca","question":"An athletic woman is shown jumping in slow motion over a pole, followed by several more women attempting the same jump. several more women\n\nA. jump and make different attempts and ends with a brunette standing before the camera holding her hands up.\nB. jump in quick succession from different angles in rapid displays.\nC. follow behind performing the jump while the camera captures them in slow motion.\nD. are then shown jumping in slow motion over different obstacles.","prompt":"An athletic woman is shown jumping in slow motion over a pole, followed by several more women attempting the same jump. several more women","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6210,"error":null,"output":"D"}
{"question_id":"hellaswag:00009232","item_index":212,"attempt":0,"prompt_hash":"1caab60d11adbc41e54fef52065ca674520eb096e8af7877ce3a9e74c798e442","question":"[header] How to wear leggings to work [title] Choose black leggings. [step] If you're a fan of leggings, you're aware that they come in every color under the sun. While a pastel pink or a cool patterned pair are great for a workout class, don't try to dress those up for work.\n\nA. Stick to grays and browns, so the pair you choose makes up for a drab look. [substeps] You can even go with both patterned and neutral leggings if you're adding variety.\nB. [title] Select black leggings if you want to be better at show-dressing. [step] Studies show that body-builders tend to favor black leggings as a part of their choice of workout attire.\nC. If you want to pull off leggings at work, they need to be black. Black leggings look much more sleek and professional than any other color, and you're less likely to call attention to the fact that they are, in fact, leisure clothes.\nD. They're too neat to work and will likely fail to convey their workout vibe. Opt for knee-length leggings in black or an olive color that's around knee-length.","prompt":"[header] How to wear leggings to work [title] Choose black leggings. [step] If you're a fan of leggings, you're aware that they come in every color under the sun. While a pastel pink or a cool patterned pair are great for a workout class, don't try to dress those up for work.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6211,"error":null,"output":"C"}
{"question_id":"hellaswag:00002373","item_index":213,"attempt":0,"prompt_hash":"a72120d9736e1607957865f34d28c4161951c5c4a9fac9d9515079864fe892c3","question":"A man is standing on a ladder next to a court. They begin to play volleyball on the court. a man in a white shirt\n\nA. is standing next to the court.\nB. walks away from the camera.\nC. watch the game with the camera.\nD. talks to the camera.","prompt":"A man is standing on a ladder next to a court. They begin to play volleyball on the court. a man in a white shirt","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6208,"error":null,"output":"A"}
{"question_id":"hellaswag:00007753","item_index":214,"attempt":0,"prompt_hash":"cb1776e04a9943c560cf71f093aea27600d7ca8faaada516f37d1bbc21c7bedd","question":"[header] How to make an old pair of plain white shoes look funky or cool [title] Gather all your supplies. [step] ( see things you'll need ) [title] Get some newspaper and spread it on a table. [step] It is best to do this outside.\n\nA. [title] Wait for the newspaper to cool and (if possible) mark where the straps will be. [step] Make sure you do this away from flammable materials.\nB. Wait a few minutes and sort your shoe out. [title] In the one hand you want to make an envelope, draw it out like a letter and color it in.\nC. Get your shoes and place them on top of the newspaper. Make sure the environment is clear of any valuable objects.\nD. [title] Draw a cross on the newspaper for effect. [title] Tape the cloves of garlic in place.","prompt":"[header] How to make an old pair of plain white shoes look funky or cool [title] Gather all your supplies. [step] ( see things you'll need ) [title] Get some newspaper and spread it on a table. [step] It is best to do this outside.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6588,"error":null,"output":"C"}
{"question_id":"hellaswag:00008998","item_index":215,"attempt":0,"prompt_hash":"4c2dcde9477259734e25f617fd15d0142de936fd14e14bfd1cf9f6e379d0c443","question":"[header] How to pack healthier school lunches [title] Always include a source of protein. [step] Including a source of lean protein in your child's lunch is essential. Protein will provide them with the energy to pay attention during the day, support muscle and tissue development and metabolism.\n\nA. From high fat sources such as grilled chicken, ground beef, fish or beans to lean cuts of seafood, lean meat is a good source of protein for children. [substeps] Eat at least 1 full serving of protein per day.\nB. [substeps] You should include about 1 ounce 1 oz of some kind of protein to each serving you make. Other protein sources include: lean meats, eggs, fish, beans, legumes, nuts, high-calorie dairy foods, low-fat dairy products, and grains.\nC. [substeps] Leaner protein sources are preferable to high fat, highly processed or fried protein. Lean protein is lower in calories and fat which makes it fit well into a balanced diet.\nD. [substeps] Make your child a breakfast, lunch or dinner with lean protein. [title] Ensure they are getting at least 25% to 100 grams of protein per serving.","prompt":"[header] How to pack healthier school lunches [title] Always include a source of protein. [step] Including a source of lean protein in your child's lunch is essential. Protein will provide them with the energy to pay attention during the day, support muscle and tissue development and metabolism.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6589,"error":null,"output":"C"}
{"question_id":"hellaswag:00007014","item_index":216,"attempt":0,"prompt_hash":"028f1dc3e95ec191c3fd49c8200ab26614a72dfef3a3f0164f8ad1428ab80bdd","question":"[header] How to care for long haired cats [title] Feed your cat two or three meals a day. [step] Don't leave out a constant supply of food. Your cat may be tempted to eat too much, which can lead to obesity and other health problems.\n\nA. Take a closer look at your cat's food stash so you can determine when it is no longer hungry. [substeps] When deciding how much to provide your cat, look for pellets, soft treats and easy access to its bed, food bowls, litters, shelves and furniture.\nB. [title] Feed your cat a mix of wet food and dry food. [step] This will ensure your cat will get all the proper nutrients.\nC. A single serving of nutritious cat food will provide the cat with the energy and nutrition to stay young and healthy. [substeps] Feeding your cat a total of four meals per day should provide the rest and energy it needs.\nD. Give your cat a walk at least once a day. Gently feed your cat food between meals so it can pace and get moving.","prompt":"[header] How to care for long haired cats [title] Feed your cat two or three meals a day. [step] Don't leave out a constant supply of food. Your cat may be tempted to eat too much, which can lead to obesity and other health problems.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6588,"error":null,"output":"B"}
{"question_id":"hellaswag:00008507","item_index":217,"attempt":0,"prompt_hash":"1eff05f57dd7083652edc39a4ef914c574fce21e4a8f0cf46e3deead92525cfa","question":"[header] How to do light daytime makeup and casual hair [title] Create a good base. [step] To make your skin look as blemish-free as possible, start by applying facial primer evenly across your face. Next, put on your preferred foundation and blend it in.\n\nA. Another good base color will be a light green for you mom or a dark green for you dad. [substeps] Your base foundation may be a bit drying, and it can be creamy or frumpy depending on your personal makeup preferences.\nB. Roll the foundation across the entire face, from the forehead and down across the nose. This will hide the side of your face from the others because they will blend in.\nC. Apply concealer under your eyes and on top of any blemishes. [substeps] You can swap out the primer for facial lotion or a light moisturizer.\nD. [title] Fill out your brows with concealer and white eye shadow. [step] Start at the brow bone and very softly lift at the sides.","prompt":"[header] How to do light daytime makeup and casual hair [title] Create a good base. [step] To make your skin look as blemish-free as possible, start by applying facial primer evenly across your face. Next, put on your preferred foundation and blend it in.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6588,"error":null,"output":"C"}
{"question_id":"hellaswag:00000688","item_index":218,"attempt":0,"prompt_hash":"a685e96cb33b3d4a8bcf4f1d2d99f4d98a0207d93df432ae93aa9d800831a776","question":"A student athlete appears, dressed in casual shorts and a red t-shirt, and he proceeds to do practice jumps. the first jump\n\nA. occurred but only able to land on his head multiple times.\nB. is repeated in slow motion in front of him in a cardio room filled with exercise equipment.\nC. is poor, and he knocks off the bar during the jump.\nD. is in a particular static configuration bordering the mat for a moment, he jumps then quickly swings, and successfully bounces the bar.","prompt":"A student athlete appears, dressed in casual shorts and a red t-shirt, and he proceeds to do practice jumps. the first jump","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6587,"error":null,"output":"B"}
{"question_id":"hellaswag:00000344","item_index":219,"attempt":0,"prompt_hash":"9683d20ffa0966859f25103ccf75eb42d550a38d19ce13ff75f1273b91fb2aae","question":"A man stands in front of a counter with his hands down in front of equipment. He uses the surface to rub a knife blade back and forth, sharpening it. when he\n\nA. 's done, he shows off a close up shot before throwing in a print and showing the finished product.\nB. is finished, he begins cutting the knife in half.\nC. is finished, the object in question is a cylinder.\nD. is done, he checks the blade before lying the knife back down.","prompt":"A man stands in front of a counter with his hands down in front of equipment. He uses the surface to rub a knife blade back and forth, sharpening it. when he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6617,"error":null,"output":"D"}
{"question_id":"hellaswag:00001048","item_index":220,"attempt":0,"prompt_hash":"f6774115891e1e1d2d6a96f7f89fd41b91e7474115219be508fac03fcd6ae04a","question":"A large fish is seen swimming under frozen water. a man\n\nA. is seen bending over into the water looking is the fish are teeth.\nB. scrapes off the ice the underneath of a boat.\nC. sits in a boat in the water the fish swim into his hand and die.\nD. creates a hole in the ice to catch the fish.","prompt":"A large fish is seen swimming under frozen water. a man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6616,"error":null,"output":"D"}
{"question_id":"hellaswag:00002431","item_index":221,"attempt":0,"prompt_hash":"1d40ccc7d23edc73c07737ec0aa5d5319237fd01d2444feb58e438d7723d4762","question":"The man moves toys to the side of the yard to clear the area. The man dumps lawn clippings into a trash bin. the man\n\nA. uses scissors to cut the lawn.\nB. grabs some orange paper on a tree stump.\nC. spreads the lawn equipment out on the grass.\nD. moves all of the toys back into the yard.","prompt":"The man moves toys to the side of the yard to clear the area. The man dumps lawn clippings into a trash bin. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6616,"error":null,"output":"D"}
{"question_id":"hellaswag:00007133","item_index":222,"attempt":0,"prompt_hash":"755d1ad6c90a7c2987f1e182aa4cba7069cb4926dec9c6d03061f42f2b70e6ee","question":"[header] How to dress up in a pencil skirt with color blocking [title] Choose the colors. [step] If you are dressing up an outfit in color blocking for the first time, it is necessary to keep in mind that bright colors are quite difficult to carry off at the first time. [substeps] Start off by picking out a clothing item of any neutral color (beige, black, white, cream or any of their shades).\n\nA. It would be better if you choose the top to be of this color. Make sure the top is flowy or loose as this just balances out the outfit and makes it wearable on a daily basis.\nB. You can also go monochrome for this! Give yourself a lip liner and add metallic eyeshadow if you desire. [title] Express your newly bent waist with your pencil skirt.\nC. Color blocking can be difficult if you have a particular color playing against a whole color scheme of the dress. [title] Pick a t-shirt that covers your skin.\nD. Keep in mind that, while all colors can work as a color blocking accessory, not all're appropriate. For example, in an office setting, you might look especially inappropriate wearing the dress that goes, boring.","prompt":"[header] How to dress up in a pencil skirt with color blocking [title] Choose the colors. [step] If you are dressing up an outfit in color blocking for the first time, it is necessary to keep in mind that bright colors are quite difficult to carry off at the first time. [substeps] Start off by picking out a clothing item of any neutral color (beige, black, white, cream or any of their shades).","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6616,"error":null,"output":"A"}
{"question_id":"hellaswag:00002933","item_index":223,"attempt":0,"prompt_hash":"6726a69673fc4cd486e77973b6338f606827cc1f2bd354a02de214a2ec8cfe77","question":"A woman is seen sitting on stage while singing into a microphone and people sitting behind her playing instruments. the girl\n\nA. continues to speak on stage then puts her hands behind her neck.\nB. continues to play and ends by speaking and the woman smiling to the camera.\nC. continues dancing around the stage while people occasionally sit and watch and occasionally laugh at her performance.\nD. continues singing with the band and ending by speaking to the audience.","prompt":"A woman is seen sitting on stage while singing into a microphone and people sitting behind her playing instruments. the girl","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6616,"error":null,"output":"D"}
{"question_id":"hellaswag:00008206","item_index":224,"attempt":0,"prompt_hash":"c032d0f8fd161943699b7b698ce7e7e162ecc342efce24bcb2fcf305431bb984","question":"[header] How to make a personal wedding ceremony booklet [title] Provide basic information about the wedding. [step] The wedding ceremony booklet should include the full names of the bride and groom, the wedding date and time, and the city, state and venue where the wedding is being held. This information is usually on the cover of the booklet.\n\nA. Just highlight this information by either typing it into the margins and double-clicking it. [title] Put in the wedding details as you normally would.\nB. Be sure to include the date and location of the ceremony. The booklet guide is a popular job for a wedding wedding photographer.\nC. [title] List the elements and order of the wedding ceremony. [step] The ceremony booklet includes information about the actual ceremony so guests can follow along and participate.\nD. Contact the venue directly to make sure you won't need to include more information. [title] Make a list of the things that are likely to happen at the ceremony, as well as the type of event expected.","prompt":"[header] How to make a personal wedding ceremony booklet [title] Provide basic information about the wedding. [step] The wedding ceremony booklet should include the full names of the bride and groom, the wedding date and time, and the city, state and venue where the wedding is being held. This information is usually on the cover of the booklet.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6381,"error":null,"output":"C"}
{"question_id":"hellaswag:00001519","item_index":225,"attempt":0,"prompt_hash":"168b72036c7b8f165b3cd8e72285712075346734c95d7e388065158f45745e3c","question":"We soon see the man again in the water with his arm around the boy's shoulders, the boy triumphant after his successful ride. all in all everyone\n\nA. looks proud of themselves and except the boy and a teen around the area as they wave.\nB. is happy, and a replay is shown of him at the beach.\nC. seemed to enjoy a great day outdoors whether in the boat or skiing or both.\nD. 's victories appear very impressive to me.","prompt":"We soon see the man again in the water with his arm around the boy's shoulders, the boy triumphant after his successful ride. all in all everyone","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6386,"error":null,"output":"C"}
{"question_id":"hellaswag:00000794","item_index":226,"attempt":0,"prompt_hash":"9faf60288838fe81026d493a23a4b074c4a2047cb51a9694e27d720fe905e021","question":"The man adjust some screws and moves an arm. We see the item comes with a 7 year warranty. the man\n\nA. puts the item in the glove box area and leans on the car.\nB. adds an oil to the item, and we see the packaging.\nC. uses the machine for a fully loaded dishwasher.\nD. pretends to scour the windows.","prompt":"The man adjust some screws and moves an arm. We see the item comes with a 7 year warranty. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6385,"error":null,"output":"B"}
{"question_id":"hellaswag:00003484","item_index":227,"attempt":0,"prompt_hash":"12680c85c400893d1e7a4aa4b06fc826f1a4ce10ee4a12ffaeb146395e9ff5fd","question":"[header] How to make tea tree oil [title] Obtain tea tree leaves. [step] This can be difficult, depending upon the region in which you live. Melaleuca alternifolia grows in usda hardiness zones 8 through 11 and it is available for purchase.\n\nA. Other tropical areas may also be able to find the small tree at gardening centers or through magazines. [substeps] For people in cooler climates, it will be much harder to track down the raw materials needed to make this oil yourself.\nB. [substeps] If you live in an area that does not have melaleuca, you may decide to acquire 20-25 leaves from a plant that has melaleuca in its first year of life. [title] Dump 3 cups of water over the dried tea leaves.\nC. You can plant this in all locations, including south america, the british virgin islands and costa rica. This oil is safe for home use for you.\nD. [title] Place the leaves in a glass jar. [step] If your jar has a lid or glass bottom, make sure that all the leaves are placed in as neat a fashion as possible.","prompt":"[header] How to make tea tree oil [title] Obtain tea tree leaves. [step] This can be difficult, depending upon the region in which you live. Melaleuca alternifolia grows in usda hardiness zones 8 through 11 and it is available for purchase.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6379,"error":null,"output":"D"}
{"question_id":"hellaswag:00004994","item_index":228,"attempt":0,"prompt_hash":"685ac26072b265d493b15d89c36d3bfc00b15aacb6c0703c1cc0a58cc3f4df2a","question":"[header] How to use coconut oil for your hair [title] Select virgin or refined coconut oil. [step] Any coconut oil will work, whether it's labeled for hair or cooking. There are many types, however, so use this guide to find a quality oil : [substeps] A * \" virgin, \" \" extra-virgin, \" or \" unrefined \" label means the coconut oil has no added chemicals.\n\nA. A high quality virgin coconut oil should be pure white when solid, and clear when liquid. A \" refined \" or unlabeled product has probably gone through an industrial process.\nB. Put 10 to 12 drops of the oil into your hair for every 1 quart (0.4 us gal) of hair you wash. A \" virgin \" label means the coconut oil has no added chemicals.\nC. It is usually labeled \" natural, \" \" cultured, \" or \" unrefined. \" again, however, each choice here lists whether or not the oil is made for hair specifically.\nD. A/10% pure coconut oil does not contain any added compounds. Any oil that contains a low percentage of its natural sugar content or copper sulfate, or that contains 10% coconut oil, is not pure coconut oil.","prompt":"[header] How to use coconut oil for your hair [title] Select virgin or refined coconut oil. [step] Any coconut oil will work, whether it's labeled for hair or cooking. There are many types, however, so use this guide to find a quality oil : [substeps] A * \" virgin, \" \" extra-virgin, \" or \" unrefined \" label means the coconut oil has no added chemicals.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6380,"error":null,"output":"A"}
{"question_id":"hellaswag:00007669","item_index":229,"attempt":0,"prompt_hash":"2dce6bf2fd8e19c1416b1f3e168c47de10584beff96dea684340a436ca937a90","question":"[header] How to understand the cry of babies [title] Learn the \" hunger \" cry. [step] A baby who is ready to be fed is likely to begin crying quietly and slowly. The cry will build in volume, becoming loud and rhythmic.\n\nA. Each individual wail is likely to be short and low-pitched. The cry of hunger is a cue to feed your baby, unless you have fed your baby recently and are certain that your baby does not need more to eat.\nB. [substeps] Babies tell their mothers that they are hungry. Some parents mistake the baby to be hungry because they say the baby is hungry for a reason.\nC. It may also rise in pitch and then it will become softer, more sultry. It will then drop suddenly into a soft babble.\nD. As a result, the child will be alert and prepare for the next action. Through the cry, child will have an immediate and natural feeling of food.","prompt":"[header] How to understand the cry of babies [title] Learn the \" hunger \" cry. [step] A baby who is ready to be fed is likely to begin crying quietly and slowly. The cry will build in volume, becoming loud and rhythmic.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6378,"error":null,"output":"A"}
{"question_id":"hellaswag:00008778","item_index":230,"attempt":0,"prompt_hash":"3f3e5a267c0b7d6bbe22d799b2375203fda8767f6fa7780ee89061d20a15e302","question":"[header] How to determine if a child is transgender [title] Notice if your child has gender non-conforming tendencies. [step] Maybe you have seen that your female child likes to play with toys that are generally considered \" male. \" keep in mind that just because your daughter likes to play with trucks does not mean that she is transgender.\n\nA. [title] Assess if your daughter has just undergone puberty. [step] Some transsexuals like to have a baby.\nB. However, if your child consistently displays a strong preference for things that don't conform to gender expectations, it could be something to keep an eye on. [substeps] A child that is different in a few ways might just be gender non-conforming.\nC. It does not necessarily mean that she has nonconforming tendencies. [substeps] Some talk that transgender bisexuals are socially intolerant, perhaps to the point where they refuse to have sex.\nD. Unfortunately, some words like \" pimple \" and \" bum \" are an example of gendered expressions and attitudes that can lead to contextual attraction. [substeps] For example, you might assume that your child has a special fondness for playing with plastic dice.","prompt":"[header] How to determine if a child is transgender [title] Notice if your child has gender non-conforming tendencies. [step] Maybe you have seen that your female child likes to play with toys that are generally considered \" male. \" keep in mind that just because your daughter likes to play with trucks does not mean that she is transgender.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6378,"error":null,"output":"B"}
{"question_id":"hellaswag:00002025","item_index":231,"attempt":0,"prompt_hash":"3848c863554b74a676e96d08e7106b9b280403ad14424628e4f7d57a91000378","question":"The oven peeps and she prepares a pan with cookie dough. She puts the dough into the oven and presses bake options. after several minutes\n\nA. of baking, she steps out in front of the oven.\nB. , she manages to bake it and makes the grill ready.\nC. , the cake is combined in a pan.\nD. she takes out the pan and spread the cookies out.","prompt":"The oven peeps and she prepares a pan with cookie dough. She puts the dough into the oven and presses bake options. after several minutes","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6382,"error":null,"output":"D"}
{"question_id":"hellaswag:00002058","item_index":232,"attempt":0,"prompt_hash":"e2516b4085b4a8b2eb8ccd0dc668520eb12df9e327ed9f8a5c901b99cb1d9e59","question":"A woman is seen speaking to the camera that leads into clips of people riding in kayaks and camels wandering around. More clips are shown of people riding around the water in a kayak. the person\n\nA. continues riding along the water while many fall off in the end and the camera pans around the camera.\nB. continues speaking to the camera while more clips are shown of people riding themselves or leaving their raft to ride along the water.\nC. continues riding all around the area as well as capturing their movements and laughing to the camera.\nD. paddles themselves along the water with other people riding near him.","prompt":"A woman is seen speaking to the camera that leads into clips of people riding in kayaks and camels wandering around. More clips are shown of people riding around the water in a kayak. the person","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6382,"error":null,"output":"B"}
{"question_id":"hellaswag:00001874","item_index":233,"attempt":0,"prompt_hash":"1961df41676406cd29f6af60dc8c13786e617a8c41b92f7ba88288ffbcf6ef57","question":"He cut the metal into half and bend the sheet to the camera, then he used a thong and clip the sheet of metal and securely put it in a holder. the man\n\nA. is now onto a board and loosening his straps, after this he folds up the top portion and continues to lay the whole cake in on the piece of paper, then demonstrates how to tighten the edges by applying some metal and tightening and tightening the base to tighten all of the ends of the cake around the cake.\nB. went to take a shower in the bathroom, afterwards proceeded to walk over and take a shower himself.\nC. then switches the lighter to lighter and magnets on the end of the metal bar to create more power, then add a small baby blue flame tank and the camera then switches it back to a lighter lighter.\nD. put needle on welding tool, put on his mask and began torching the metal, then poke it with hammer and show it to the camera.","prompt":"He cut the metal into half and bend the sheet to the camera, then he used a thong and clip the sheet of metal and securely put it in a holder. the man","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6382,"error":null,"output":"B"}
{"question_id":"hellaswag:00000127","item_index":234,"attempt":0,"prompt_hash":"5757c4a8624530bc8203c7cdc1df7d1cadd499371659826e84f0ec0e32ae91ea","question":"A small boy wearing a blue and yellow hoodie and gray sweat pants in standing in an outdoor tennis court. He is holding a blue racket as he bounces the yellow and orange tennis ball on the court. he\n\nA. starts with his first serve where the ball hits the net.\nB. continues bouncing the yellow haired ball back and forth while looking out over the court passing the ball to one of the players.\nC. bounces and throws the ball evenly evenly to continue the game of tennis against the two balls.\nD. bounces it and does several different things to hit it.","prompt":"A small boy wearing a blue and yellow hoodie and gray sweat pants in standing in an outdoor tennis court. He is holding a blue racket as he bounces the yellow and orange tennis ball on the court. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6381,"error":null,"output":"A"}
{"question_id":"hellaswag:00008769","item_index":235,"attempt":0,"prompt_hash":"0bfe271f9419dc6463e7009374528654ce01500c9f0519ff960a896ce0c374b2","question":"[header] How to create a google forum [title] Visit the google groups website. [step] Open a new tab on your browser and go to the. [title] Sign into google groups with your google account.\n\nA. [step] You can sign up either by clicking \" create a google group \" or by logging in with a google username and password. You can also click \" sign up \" and you will be redirected to the group's login page where you will see a lot of people who have joined each group.\nB. [step] When you log into your account, you will get to the page where you can create a google account. Go to your google groups website and click the blue link in the bottom right corner.\nC. [step] Google has made it possible to use one google account across all google products. Enter your google email address on the first text box and your password on the second text box.\nD. [step] Use the email and password that you log in with so you can create a google group. Click on the arrow next to a link to the group page.","prompt":"[header] How to create a google forum [title] Visit the google groups website. [step] Open a new tab on your browser and go to the. [title] Sign into google groups with your google account.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6379,"error":null,"output":"C"}
{"question_id":"hellaswag:00009064","item_index":236,"attempt":0,"prompt_hash":"7d30a6d4baeb995905b6d6921ea5f059d69cef039d27c5da960c0eb5dae44516","question":"[header] How to add audio to powerpoint 2010 [title] Import an audio file saved to the computer. [step] Open the presentation and select a slide to add an audio file. Click the insert tab on the menu bar and click the audio file button in the far right corner of the menu bar.\n\nA. Drag the converted file to an open format document and import it into slideshare. Choose either the file you downloaded when you moved it or you may save it in the same folder if you're moving into a new format.\nB. [substeps] If you don't have a microsoft cd/dvd player in your presentation, make a sure to save audio files to a hard drive rather than using powerpoint 2010. [title] Format the slide next to the audio file.\nC. Select the audio from file option from the pull-down menu. Locate the audio file you wish to add to the presentation and double-click to insert it into the current slide.\nD. On the section called audio, click audio clips. Drag your mouse over the slide and click the insert tab to import all audio files in that slide.","prompt":"[header] How to add audio to powerpoint 2010 [title] Import an audio file saved to the computer. [step] Open the presentation and select a slide to add an audio file. Click the insert tab on the menu bar and click the audio file button in the far right corner of the menu bar.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6379,"error":null,"output":"C"}
{"question_id":"hellaswag:00003844","item_index":237,"attempt":0,"prompt_hash":"a9796778352260997d6101bd8b8f90eb5f0bd523ea154ece4c70ae1652967fb0","question":"[header] How to teach kids to wear glasses [title] Explain the need for wearing glasses. [step] Tell them that if they can't see or read something because it is too far away, it's because they don't have their glasses on. Reassure them that the glasses are a normal thing, and no one worth his/her time will mind.\n\nA. Even when the kids ask why they can't see their glasses, explain that learning also makes them smarter and helps them to visualize objects better. [substeps] You could say, \" nobody can see exactly what you're doing without glasses on.\nB. Explain, in simple terms, depending on the age of your child, how they work. [title] Let them think of it simply as a \" full-time accessory \".\nC. Explaining that glasses are a part of education looks good on older children at home. [substeps] If your child isn't reading at home, explain to them why this part of their education will look good on older children, if they have their glasses on.\nD. Explain to them that you are a parent trying to help them learn to read well. [title] Explain to them that using glasses makes them laugh.","prompt":"[header] How to teach kids to wear glasses [title] Explain the need for wearing glasses. [step] Tell them that if they can't see or read something because it is too far away, it's because they don't have their glasses on. Reassure them that the glasses are a normal thing, and no one worth his/her time will mind.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6379,"error":null,"output":"D"}
{"question_id":"hellaswag:00006632","item_index":238,"attempt":0,"prompt_hash":"9843df40ba51d5d2d2cd34fcf637b5b633cdf721e8f3ec89da44fcfc0a4343f4","question":"[header] How to measure chair covers [title] Identify what chairs you want to wrap. [step] You need to know whether you are measuring chair covers for only one corner piece or an entire room. [title] Determine what type of furniture covers you want.\n\nA. [step] If you've chosen a sofa, for example, you should probably use a sofa cover to cover each side of the sofa or your entire sofa, with the sofa taking up less space. [title] Determine whether you want to wrap a duvet or comforter.\nB. [step] Know what style and pattern you want to emulate so you know where to purchase the covers. You may also want to set an expected budget since some materials are more expensive than others.\nC. [step] There are two kinds of materials you could use for your chair cover : [substeps] Cabinet covers think of it as a buffet cover, but most commonly built on tables in the kitchen or dining room. This covers cover covers a regular portion of most of the table, and covers chairs directly underneath the table.\nD. [step] There are two primary types : [substeps] Traditional. This type of cover covers a floor, not a whole wall.","prompt":"[header] How to measure chair covers [title] Identify what chairs you want to wrap. [step] You need to know whether you are measuring chair covers for only one corner piece or an entire room. [title] Determine what type of furniture covers you want.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6379,"error":null,"output":"B"}
{"question_id":"hellaswag:00009392","item_index":239,"attempt":0,"prompt_hash":"0896de5585aebe168f371b952e49cf8442560c32e025dcce29a17f3e33e70ba5","question":"[header] How to talk with kids about political ads [title] Ask them what they know. [step] Sit down with your child and ask them about what they know about elections, politics, and politicians. Get a feel for what your child knows and how they understand the political ads.\n\nA. You want your questions to be tailored to your child, which can make the interaction much more pleasant. [substeps] Ask them questions like, \" are there any political ads in town? \" \" what's on election day? \" or \" what do you think we need to do to get elected? \" [title] Find out what the campaign is about.\nB. [substeps] Ask about political topics like democrat ideas, political groups, political goals, how politically competitive and potentially challenging/risky political ads are, and so forth. Avoid breaking the ice by saying something like, \" i noticed that our competition campaign for local university advertising starts soon after elections.\nC. [substeps] Keep your questions short, broad, and simple. Don't worry about spelling the message, because you can bring the message closer to important feelings.\nD. This can help you gauge their understanding, their fears, and their questions. Knowing your child's understanding can help guide you to talk to them in an age-appropriate way that addresses their concerns.","prompt":"[header] How to talk with kids about political ads [title] Ask them what they know. [step] Sit down with your child and ask them about what they know about elections, politics, and politicians. Get a feel for what your child knows and how they understand the political ads.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6379,"error":null,"output":"D"}
{"question_id":"hellaswag:00007980","item_index":240,"attempt":0,"prompt_hash":"91313331125188cd003d81c1c23ad12e4d7e81086193c84ee51597a09d7f9786","question":"[header] How to have beautiful writing [title] Choose the right tools. [step] Some people prefer pen, others pencil. Some like them larger, others smaller.\n\nA. Bear in mind this--less work can be combined on one type of pen and less with a pencil. [substeps] Before you become proficient in drafting, it is a good idea to.\nB. Whatever you choose, be sure to use the proper tools. If you want to write a flawless writing report, use larger pens.\nC. It's up to you but choose something that catches the eye and stands out. [substeps] All of the tools, but careful to take good care of your writing when you use them.\nD. The important thing is to find a writing instrument that feels right in your hand. [substeps] Consider a pen or pencil with a soft grip, especially if you tend to grip too tightly.","prompt":"[header] How to have beautiful writing [title] Choose the right tools. [step] Some people prefer pen, others pencil. Some like them larger, others smaller.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6381,"error":null,"output":"D"}
{"question_id":"hellaswag:00001088","item_index":241,"attempt":0,"prompt_hash":"cff84844fad1416f44233bfac629767bec901bcb249cc44481c495ce7ea4afb2","question":"First he shows viewers how to properly do a crunch and his feet are apart, arms on his chest, and each time he goes up the elbows touch his legs. when he\n\nA. gets to the end he faces right at the camera.\nB. goes back he's smiling and holding his arms up stretching his body and looking into the camera.\nC. finishes his basic crunch he is very pleased that his technique is finally working as the two men lead with a \" ooooo \".\nD. 's done, he stands up and tells viewers more helpful information about how to have a good workout.","prompt":"First he shows viewers how to properly do a crunch and his feet are apart, arms on his chest, and each time he goes up the elbows touch his legs. when he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6380,"error":null,"output":"D"}
{"question_id":"hellaswag:00001883","item_index":242,"attempt":0,"prompt_hash":"48cc3d1038f50b8abea7aa689422ab691b038510cb384ed3b4bab62e1021ea14","question":"A man dressed as spiderman leans against a metal rail as he plays a saxaphone. he\n\nA. sways and moves a little as he continues to play.\nB. tosses his hair back as he plays.\nC. walks back and forth among the two screens, playing with a key.\nD. continues the song as spider substance drips down his chin and he smiles-he continues to play the saxaphone-every now and then he looks off into the distance.","prompt":"A man dressed as spiderman leans against a metal rail as he plays a saxaphone. he","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6379,"error":null,"output":"B"}
{"question_id":"hellaswag:00009625","item_index":243,"attempt":0,"prompt_hash":"197d470e82d8742da4b69252bfb726b7c81b985db4c8b61aa31299aed46438f0","question":"[header] How to decorate a mailbox for the holidays [title] Choose a theme for your mailbox decoration. [step] Your mailbox decorations should correspond with the holiday you are celebrating. [substeps] Complement the decorations you already have in your home or around your yard.\n\nA. Your preferred decor could be plastic tree decorations or intricate christmas decorations that your family enjoys. [title] Pick a theme for a mailbox if your hometown has few decorations.\nB. Your christmas decorations must be more inviting to the holidays you are celebrating. Think about inviting local snow angels, owl owls, or a drift owl to your mailbox as an everyday decoration as well.\nC. You could go for seasonal decorations that you have in your neighborhood, or you could stay minimalist with winter decorations that you have in your home. Themes that you are familiar with can help you then look for decorative decorations that will match the holiday in order to look festive.\nD. For example, if you have a snowman theme, hang a snowman tree ornament from your mailbox. If you decorate your windows with easter egg decals, add them to your mailbox as well.","prompt":"[header] How to decorate a mailbox for the holidays [title] Choose a theme for your mailbox decoration. [step] Your mailbox decorations should correspond with the holiday you are celebrating. [substeps] Complement the decorations you already have in your home or around your yard.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6379,"error":null,"output":"D"}
{"question_id":"hellaswag:00002049","item_index":244,"attempt":0,"prompt_hash":"704cc6d0c6cf64dd0a8bdd330b320bfb968be0ee36f163a7c68d2ebd2c32f0ad","question":"A graphic with some keywords forming the number '2' is shown. the scene\n\nA. changes to a dog walking slowly in a park.\nB. changes to shows the coaches' assists in decorating trees.\nC. switches to a black screen with some text about an upcoming marathon.\nD. is then zoomed out completely.","prompt":"A graphic with some keywords forming the number '2' is shown. the scene","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6377,"error":null,"output":"C"}
{"question_id":"hellaswag:00003509","item_index":245,"attempt":0,"prompt_hash":"a34998e183906c0cdccf957e808b8649cf494328e3f8808265fd7e1408c6d15a","question":"[header] How to care for gold [title] Remove your gold jewelry before showering, bathing, or cleaning. [step] The exposure to soap and chemicals can dull the gold, necessitating more frequent cleanings. It can also scratch or break down the gold, which is a soft metal and easily damaged.\n\nA. [title] Don't wear your gold jewelry in the pool. [step] Chlorine, like other chemicals, can permanently discolor gold.\nB. [substeps] Frequent shower and cleaning can also slow down the aging process. Rubbing your jewelry in baby shampoo can easily dull and discolor the gold's shine.\nC. Take special caution with this precaution if you're a gold enthusiast. Drop or remove as much jewelry as possible before showering or bathing.\nD. Washing and bathing regularly is the ideal way to prevent this. [substeps] Read all the instructions that come with your gold jewelry before it is purchased to ensure that you comply with any particular care terms.","prompt":"[header] How to care for gold [title] Remove your gold jewelry before showering, bathing, or cleaning. [step] The exposure to soap and chemicals can dull the gold, necessitating more frequent cleanings. It can also scratch or break down the gold, which is a soft metal and easily damaged.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6379,"error":null,"output":"A"}
{"question_id":"hellaswag:00003593","item_index":246,"attempt":0,"prompt_hash":"484e7d5b5f905cf7738d221d57dbaa99359c68a96e11fb599429b9b8e7cd1f6a","question":"[header] How to change laptop volume in windows [title] Press the winkey + i combination keys. [step] This will bring up the \" settings \" pane. [title] Click on the \" volume \" icon on the \" settings \" pane.\n\nA. [title] Look at the \" updated \" section below the \" changes \" tab. [step] This should bring up the \" updated \" window.\nB. [title] Click on the bluetooth icon on the right side of the home screen to open the bluetooth settings window. [title] Click on the \" high-volume \" button labelled \" network and internet settings \" in the \" internet \" drop-down menu.\nC. [title] Click on \" keyboard options \" and select \" keyboard \". [title] Select \" keyboard control \" from the \" keyboard search.\nD. [step] A pop-up window will appear where you can adjust the volume slider control. [substeps] After adjusting your volume setting, click anywhere on the screen to close the volume control with your settings saved.","prompt":"[header] How to change laptop volume in windows [title] Press the winkey + i combination keys. [step] This will bring up the \" settings \" pane. [title] Click on the \" volume \" icon on the \" settings \" pane.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6341,"error":null,"output":"D"}
{"question_id":"hellaswag:00004882","item_index":247,"attempt":0,"prompt_hash":"2e28a28ee6df457f25138170f4ac03bf77492b0a8c527eb3b0e915ce468ae834","question":"[header] How to thicken stew [title] Sprinkle a small amount of breadcrumbs into the stew liquid. [step] Stir to mix well. [substeps] Use dried or frozen breadcrumbs.\n\nA. Any kind is acceptable, but dry is fine. It needs to be soft but not runny.\nB. They are often dried or frozen, which means they won't become heavy in the stew. [title] Cook for 30 minutes or until fully cooked.\nC. [title] Allow for a little more cooking, then check to see if the stew has thickened adequately. [step] If not, repeat with another sprinkling of breadcrumbs.\nD. The starchy matter will lend them flavor. [title] Simmer the stew for 30 to 60 minutes, cooking at a medium-low heat.","prompt":"[header] How to thicken stew [title] Sprinkle a small amount of breadcrumbs into the stew liquid. [step] Stir to mix well. [substeps] Use dried or frozen breadcrumbs.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6342,"error":null,"output":"C"}
{"question_id":"hellaswag:00000916","item_index":248,"attempt":0,"prompt_hash":"95aec76e17ef484f4c3b7d3c42e4926e2d81c7de2371cd0c5a557d414b438ee3","question":"Some is carrying an instrument case walking around outside, they put the case down on the ground and take out a violin. The woman plays the violin everywhere she goes and she dances while she plays too. she\n\nA. stops to stop and grab a change of clothes and start playing.\nB. does some movements of her body making her back bend down almost touching the ground while still jamming out on the violin.\nC. continues singing as she pulls the violin across as she plays, and her face appears showing awe at the sight of the instrument.\nD. stops to play all around and act really silly.","prompt":"Some is carrying an instrument case walking around outside, they put the case down on the ground and take out a violin. The woman plays the violin everywhere she goes and she dances while she plays too. she","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6342,"error":null,"output":"B"}
{"question_id":"hellaswag:00009736","item_index":249,"attempt":0,"prompt_hash":"739d93cd0b8774d816397268eddf298d8295a834e8bf4e9cadf0c6a681807359","question":"[header] How to have the healthiest, clearest skin [title] First figure out your skin type. [step] Oily? Dry? Sensitive? Combination? Knowing this can help you target those areas and improving them. [title] Depending on your skin type your going to need to find some products that suit your face.\n\nA. [step] You'll need a cleanser, mask, toner, moisturizer, and a formula that you can dab on blemishes and will tone down redness and make it disappear faster. [title] You should wash your face in the morning and at night.\nB. [step] This is hard to find all that you can buy, or everything you might need. Get a store wig, and two picks, a bucket and/or glove, a towel, a sponge and a brush/comb, and any body essentials.\nC. [step] Have another person choose the product you want to use and personally try it on your face, then judge by their reaction. Ask them if they would try it on you.\nD. [step] If you have dry or combination skin (purple, peach, redhead) oily? These products work equally well. Not all products you can use do different things.","prompt":"[header] How to have the healthiest, clearest skin [title] First figure out your skin type. [step] Oily? Dry? Sensitive? Combination? Knowing this can help you target those areas and improving them. [title] Depending on your skin type your going to need to find some products that suit your face.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6342,"error":null,"output":"A"}
{"question_id":"hellaswag:00000430","item_index":250,"attempt":0,"prompt_hash":"42264295f270733c077d25f7d7fd40811db14ed97e98d4ea189135d1e7a4ddce","question":"A table is shown covered in water bowls and lemons. A woman is talking about how to create lemonade from the ingredients. she\n\nA. pops the lemons with a straw.\nB. begins by adding a smaller amount of lemon juice as well as sugar and waving the lemons around.\nC. starts by cutting up some bananas and spreading the juice in a bowl.\nD. squeezes juice from the lemons and adds sugar and water.","prompt":"A table is shown covered in water bowls and lemons. A woman is talking about how to create lemonade from the ingredients. she","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6342,"error":null,"output":"D"}
{"question_id":"hellaswag:00009102","item_index":251,"attempt":0,"prompt_hash":"3895fdca559bc9005cab2f7e9bd7819e1a1b3e09119e0a119ea52311978e9c68","question":"[header] How to use bubble wrap [title] Use bubble wrap to protect delicate items, like glass or electronics. [step] Bubble wrap is ideal for padding glassware, ceramics, mirrors, and other fragile items while moving. Because bubble wrap comes in large sheets, it can be more effective for wrapping bigger items than styrofoam or newspaper.\n\nA. [substeps] If it's not light enough for your bubble wrap, you can twist or simply press the ends of the bubble wrap together. [title] Place an ice pack on the spill.\nB. [title] Wrap all delicate items before putting them in the box. [step] Place your item on a sheet of bubble wrap and cut out a piece long enough to cover the item in bubble wrap.\nC. It also helps protect your items from dangerous particles. [substeps] Bubble wrap does not contain the same polymers that foam bath sheets, or these can solidify and shrink in water before use.\nD. [substeps] Read a section on making bubble wrap from a variety of perspectives, so you have a comprehensive understanding of exactly what you're making. [title] Pack your bubble wrap in a cooler.","prompt":"[header] How to use bubble wrap [title] Use bubble wrap to protect delicate items, like glass or electronics. [step] Bubble wrap is ideal for padding glassware, ceramics, mirrors, and other fragile items while moving. Because bubble wrap comes in large sheets, it can be more effective for wrapping bigger items than styrofoam or newspaper.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6603,"error":null,"output":"B"}
{"question_id":"hellaswag:00002900","item_index":252,"attempt":0,"prompt_hash":"4aead98812323e33103f79a433b552668b40800f02dee0086982adaefaf1cf77","question":"The girl turns on the faucet to wet her toothbrush. The girl turns off the faucet, unscrews the toothpaste, and puts some toothpaste on her toothbrush. the girl\n\nA. throws her toothbrush in the hand that is standing on the sink counter.\nB. brushes her teeth as the run the toothbrush through her mouth.\nC. sits on the bathroom floor, spits an object into her hand, and then wipes her mouth with a towel.\nD. brushes her teeth and spit in the sink.","prompt":"The girl turns on the faucet to wet her toothbrush. The girl turns off the faucet, unscrews the toothpaste, and puts some toothpaste on her toothbrush. the girl","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6604,"error":null,"output":"D"}
{"question_id":"hellaswag:00001900","item_index":253,"attempt":0,"prompt_hash":"2acde07fda952d501f522fb7e751486150f47e0f03afb86397eebff04cf68280","question":"He finds a little hole in the ice and sets up to fish, moving around here and there before finally settling in his chair. He starts to move his rod around in the water and pulls up a little fish. he\n\nA. finally goes under the water and finishes his fish as the wind picks up and blows the fish off of the boat.\nB. enjoys the catch and gets off of the chair to go rub it all over himself.\nC. caught himself a 9 inch blue gill, he goes back in and catches another one and he throws it back into the water.\nD. sees something up close and turns around and my man is pushing it out of his kayak to fish.","prompt":"He finds a little hole in the ice and sets up to fish, moving around here and there before finally settling in his chair. He starts to move his rod around in the water and pulls up a little fish. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6604,"error":null,"output":"C"}
{"question_id":"hellaswag:00003931","item_index":254,"attempt":0,"prompt_hash":"3259b05f93349fdd3c2332b30ae2b7ceeb07779837d55239b690e1078ecd8f14","question":"[header] How to identify and treat recluse (fiddleback) spider bites [title] Find the spider that bit you, if possible. [step] If you can, try to trap the spider that bite you to determine if it is a recluse spider. You can also try to remember the appearance of the spider that bite you.\n\nA. A recluse spider is tan all over, with legs attached to the upper part of its body. [substeps] You don't have to catch the spider.\nB. It can be a gradual crab bite, a cold bite, or a combination of these. If it bit you, then you can keep the spider in a closet or corner and try to catch it.\nC. [substeps] If you can recognize the spider's appearance, try to recall what it looked like when you were young and how it looked on you. A recluse spider is a young spider, usually about 5 to 7 years old.\nD. [title] Keep an eye out for the spider if you ever see it at the end of your driveway. [step] If you ever see a spider in your driveway, it will probably return to a place where it can hide.","prompt":"[header] How to identify and treat recluse (fiddleback) spider bites [title] Find the spider that bit you, if possible. [step] If you can, try to trap the spider that bite you to determine if it is a recluse spider. You can also try to remember the appearance of the spider that bite you.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6603,"error":null,"output":"D"}
{"question_id":"hellaswag:00004099","item_index":255,"attempt":0,"prompt_hash":"9202d16bd726f9eb1155efd9f7f79205d35b7d3606d5f964620ba9059069ba26","question":"[header] How to prepare dark chocolate fondue [title] Make the easy dark chocolate fondue. [step] Measure 1 cup (236 ml) of heavy whipping cream into a medium saucepan and turn the heat to medium. When the cream starts to bubble a little, turn off the heat and whisk in 12 ounces (340 g) of chopped dark chocolate and a pinch of salt.\n\nA. When the mixture comes to a boil, whisk in 4 ounces (224 g) of caster sugar and 1 teaspoon salt. Whisk the mixture until the mixture reaches a boil.\nB. Stir the meringue together with the eggs and milk and set aside. [substeps] You can prepare the fondue at the same time and freeze it overnight if you don't want to chill it fully.\nC. Keep whisking until the chocolate is melted. [substeps] This fondue is basically a ganache, so it's important to transfer it to a fondue pot.\nD. Whisk the mixture and set it aside for later. [substeps] For longer curries, a large pot or saucepan (12 ounces (340 g) or larger should work perfectly.","prompt":"[header] How to prepare dark chocolate fondue [title] Make the easy dark chocolate fondue. [step] Measure 1 cup (236 ml) of heavy whipping cream into a medium saucepan and turn the heat to medium. When the cream starts to bubble a little, turn off the heat and whisk in 12 ounces (340 g) of chopped dark chocolate and a pinch of salt.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6604,"error":null,"output":"C"}
{"question_id":"hellaswag:00009389","item_index":256,"attempt":0,"prompt_hash":"0dc2f3540b03e2092b9f2d99598c5f247334c7f468d677b77c54accb3ae5b78f","question":"[header] How to choose a good budgie [title] Locate budgies for sale from a reputable seller. [step] You can find budgies for sale online, in want-ads, or in pet stores. You can also get one from your local animal shelter.\n\nA. Most animal shelters sell budgies for a flat fee, so this will limit your bank expenses. [substeps] You might also be able to compare prices to get a good deal.\nB. You'll need to get a price point that is reasonable. Here's a guide to locate an experienced seller : [substeps] Generally , 20-50% off the appraised price is reasonable.\nC. Wherever you get your budgie from, make sure the seller has healthy and humanely treated birds. [substeps] Look at online reviews for the sellers you find.\nD. Basically, you need to find a good seller who has had previous owners. [substeps] Check to see if the parent owned the budgie.","prompt":"[header] How to choose a good budgie [title] Locate budgies for sale from a reputable seller. [step] You can find budgies for sale online, in want-ads, or in pet stores. You can also get one from your local animal shelter.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6750,"error":null,"output":"C"}
{"question_id":"hellaswag:00001299","item_index":257,"attempt":0,"prompt_hash":"24c59ad18c3858c43ab4056595e297c27280a758edf17997666b048904ab3dc9","question":"A man in a chef uniform is standing up talking. He is holding a knife sharpener in his hand. he\n\nA. starts sharpening his knife on the sharpener.\nB. is talking at a microphone.\nC. then demonstrates and shows how to use the sharpener on lemons and onions.\nD. starts to sharpen the tool in the bathroom and start shaving as he talks.","prompt":"A man in a chef uniform is standing up talking. He is holding a knife sharpener in his hand. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6756,"error":null,"output":"A"}
{"question_id":"hellaswag:00004935","item_index":258,"attempt":0,"prompt_hash":"0b57b4aee86762ee5e6a867a379cd390e673b1f2652cbcf95722f6c2f5aa4207","question":"[header] How to style your hair for a cathedral veil [title] Allow your curls to hang naturally, if you have curly hair. [step] Instead of over-styling your hair, part it as you normally would and let your curls hang naturally. Natural curls can give you a great romantic look for your wedding day.\n\nA. This makes it an ideal thing to wear as you don't need to clip back a strand or cause it to rub against your head. [title] Tie back your hair into a french ponytail.\nB. [substeps] Curls are prone to frizz, so use some product to keep them looking smooth. Apply an anti-frizz serum or a hair smoother prior to your big day.\nC. [substeps] If you don't want straight hair, try doing the tail. If your hair is naturally straight, the tail part can help achieve more natural waves.\nD. Get wet a bit, blow dry or brush your hair to set it a natural curl, if desired. [substeps] Give your hair a bit of water before or after curling, to get it totally hydrated before or right before you go into the wedding.","prompt":"[header] How to style your hair for a cathedral veil [title] Allow your curls to hang naturally, if you have curly hair. [step] Instead of over-styling your hair, part it as you normally would and let your curls hang naturally. Natural curls can give you a great romantic look for your wedding day.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6749,"error":null,"output":"B"}
{"question_id":"hellaswag:00006408","item_index":259,"attempt":0,"prompt_hash":"f65d1598b7474456bf972fb8e313e5b1c0f2aba378937f0e4e7060a25f2de437","question":"[header] How to dye your hair with washable markers [title] Choose your color (s). [step] Crayola washable markers work well and come in a variety of different colors and shades, but any brand of marker will do as long as it is labeled \" washable \". Once you've chosen your color (or colors), you need to get the ink out.\n\nA. One solution is to have the tip of a finger hover over an ink pen. Write the ink color (s) on a paper plate in the color wheel.\nB. With just a little bit of force, you can break open the marker. [substeps] Using scissors, pry the stopper off the end so that the back of the marker is completely open.\nC. [substeps] Ink pens or a ink brush are preferred, as they provide easy picking up and flushing utensils and don't require water. Get the ink out of your hand first.\nD. Any marker that doesn't look how you want it to look can fall off and cause a serious stain, though. [substeps] You can purchase washable markers at any pharmacy or fabric store.","prompt":"[header] How to dye your hair with washable markers [title] Choose your color (s). [step] Crayola washable markers work well and come in a variety of different colors and shades, but any brand of marker will do as long as it is labeled \" washable \". Once you've chosen your color (or colors), you need to get the ink out.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6751,"error":null,"output":"B"}
{"question_id":"hellaswag:00009617","item_index":260,"attempt":0,"prompt_hash":"bf22daf69b9fe86254c5ebab0f2f8de2f3ba0615e04b20a856113ba3857ae7c5","question":"[header] How to ship a computer [title] Properly pack the computer and accessories to avoid damage during transit. [substeps] Disconnect the computer cables, accessories, and peripherals. This includes the monitor, keyboard and mouse.\n\nA. Place another sata connector on one monitor to prevent future damage later on. First, detach the other sata cable.\nB. Ensure that all cables are organized, untangled and enclosed in a small bag or box. Wrap the computer and peripherals in a suitable air-cellular cushioning material.\nC. If the motherboard does not have power connections, you will need to use a simple adapter instead. Power adapters, which attach to the computer and your components, are not advisable for shipping laptops due to mass transport and contamination that can damage the motherboard.\nD. [title] Don't forget that the motherboard is as large as it can be so make sure all components have a separate space. [substeps] Wipe down the surface of the motherboard, clear any cables, and make sure everything is evenly covered.","prompt":"[header] How to ship a computer [title] Properly pack the computer and accessories to avoid damage during transit. [substeps] Disconnect the computer cables, accessories, and peripherals. This includes the monitor, keyboard and mouse.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6752,"error":null,"output":"B"}
{"question_id":"hellaswag:00006746","item_index":261,"attempt":0,"prompt_hash":"90f670be0e86454297040d5d975edfc4e500f9ec9c50c6e896825bea9a693fb3","question":"[header] How to wear an off the shoulder dress [title] Go for a mini cut for a summertime option. [step] Off the shoulder dresses come in a mini cut, where they hit high at your thighs. This style is ideal if you want to wear the dress to a summer event or in a hot climate where you want to show off your legs.\n\nA. [substeps] A mini cut is also a good option if you want to show off a fun pair of sandals or flats with the dress. [title] Try a midi or maxi cut for a versatile option.\nB. [substeps] Consider getting a cut that's shorter than you normally would when you want to dress up. These cuts contain plenty of fabric, and can give you some coverage while still making you feel stylish.\nC. [substeps] Low cut dresses can be worn high on your legs, and you may want to opt for one of these close-fitting dresses. [title] Switch out muted colors for cool colors for a sophisticated look.\nD. [substeps] It's best to go for a shorter cut for the most formal look. You want low-cut dresses, as no one wants to go running up and having to do a yard of sprinting for what's a really hot weather option.","prompt":"[header] How to wear an off the shoulder dress [title] Go for a mini cut for a summertime option. [step] Off the shoulder dresses come in a mini cut, where they hit high at your thighs. This style is ideal if you want to wear the dress to a summer event or in a hot climate where you want to show off your legs.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6751,"error":null,"output":"A"}
{"question_id":"hellaswag:00000081","item_index":262,"attempt":0,"prompt_hash":"7f1ef240e84e2d199ce0a2dffce83bb669f202f8183d09d96b308e1295a761b3","question":"A young girl in a green leotard begins twirling a baton and doing tricks. the girl\n\nA. does this in the beginning while using her foot and one hand to move around her body.\nB. begins to do some of her routine and does a few hip spins.\nC. puts their hands together, and twirl together like a ballet dancer.\nD. begins to incorporate her arms and does a flip and continues to flip the baton over her hands and arms.","prompt":"A young girl in a green leotard begins twirling a baton and doing tricks. the girl","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6753,"error":null,"output":"D"}
{"question_id":"hellaswag:00008577","item_index":263,"attempt":0,"prompt_hash":"2c8d8247d4031975248b9e6ce21d1edb1c0e257c19630b2161fed4aecf845f17","question":"[header] How to perform an nsp (normal safety precaution) on the cadet l98a1 [title] Safety catch. [step] The safety catch is situated just above the trigger guard. If you can see the red ring on the left side of the rifle, then the safety is off.\n\nA. If you cannot, do not attempt to release any pin, bolts, or other necessary trigger items before setting the safety catch. This will minimize the amount of precision you'll need to apply for the nsp.\nB. If you can't see the red ring, the rifle is safe. [title] Press the safety catch and make sure there is not a magazine fitted, remembering to use your left hand and to keep your right hand on the trigger guard at all times.\nC. If you can see the wires or pulleys, then the safety is off. [title] If you cannot see the safety catch button (red circle), press the bottom left button and then press the bottom right button (black circle).\nD. If you cannot see the red ring, then the shoulder strap is on. [title] Place the back of your right arm over your left shoulder.","prompt":"[header] How to perform an nsp (normal safety precaution) on the cadet l98a1 [title] Safety catch. [step] The safety catch is situated just above the trigger guard. If you can see the red ring on the left side of the rifle, then the safety is off.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6749,"error":null,"output":"B"}
{"question_id":"hellaswag:00004410","item_index":264,"attempt":0,"prompt_hash":"6f8e44a76b6c2fa3e4a3aed0ddca1c4515cfe6a22ce344f2fefd97c48d45f509","question":"[header] How to talk so teens will listen [title] Remove distractions to make listening easier. [step] Getting your teen to listen is much easier when you aren't competing with their cell phone or the television. Eliminate these distractions and your message is more likely to actually be heard.\n\nA. [substeps] If you're in a class, ask a teacher to remind your teen to turn off all phones during class. You can still provide them with distractions if they just want to take their mind off of school.\nB. [substeps] Decide which (or all) distractions will help your teen hear more clearly. Let your teen hear the message from another part of the house, from a windowsill outside, or in rooms that are quiet.\nC. [substeps] When you're planning to talk, turn off the tv and ask your teen to silence their phone or leave it in another room. If your teen is the type to feel intimidated by face-to-face discussions, try chatting with them during tv commercials.\nD. [substeps] Research shows that some teens prefer to be ignored or...without a distraction. Keeping the screen fairly up will help correct that.","prompt":"[header] How to talk so teens will listen [title] Remove distractions to make listening easier. [step] Getting your teen to listen is much easier when you aren't competing with their cell phone or the television. Eliminate these distractions and your message is more likely to actually be heard.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6749,"error":null,"output":"C"}
{"question_id":"hellaswag:00002532","item_index":265,"attempt":0,"prompt_hash":"09b709ab36153806c6fc7a187ef868f76bb0abbda015e8eceed33ab430403b3a","question":"1 a guy is playing the saxophone. 2 a guy talks about playing the saxophone. 3 the guy\n\nA. plays the saxophone again.\nB. rests his hand on the desk.\nC. plays the drums and turn off the camera.\nD. is talking about the saxophone against his will.","prompt":"1 a guy is playing the saxophone. 2 a guy talks about playing the saxophone. 3 the guy","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6753,"error":null,"output":"A"}
{"question_id":"hellaswag:00006526","item_index":266,"attempt":0,"prompt_hash":"c0c8c2492bdf2f48b920a97179f787b80571c784fdc1621d705e050e601e3361","question":"[header] How to grow taller faster (kids ) [title] Eat healthily. [step] Food can give you the energy to get through the day, but it also helps your body grow. Getting the vitamins and minerals your body needs by eating healthy, regular meals and snacks can help you grow taller faster.\n\nA. Food also helps your brain process new information, and it will help your body stop the binging. [title] Aim for about 200 calories a day.\nB. [substeps] Eat fresh fruits and vegetables to get the most more nutrients. High on carbohydrates and fiber, they make your body grow faster.\nC. Also, good foods also can help you manage your weight. Limiting sugar or saturated fats, like those found in sugary or high-fat foods like low fat milk and yogurt, can help you get taller faster.\nD. [substeps] Have breakfast, lunch, dinner, and two healthy snacks every day. This makes sure that your body has enough energy to get you through your day and encourages it to grow.","prompt":"[header] How to grow taller faster (kids ) [title] Eat healthily. [step] Food can give you the energy to get through the day, but it also helps your body grow. Getting the vitamins and minerals your body needs by eating healthy, regular meals and snacks can help you grow taller faster.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6748,"error":null,"output":"D"}
{"question_id":"hellaswag:00004602","item_index":267,"attempt":0,"prompt_hash":"c37b95b60d4078f41b8eb454e6a7a3ef96c8376f666cb104c7610022327f5d91","question":"[header] How to raise your children to be helpers [title] Call them helpers when you ask for things. [step] Instead of asking for help, ask your child to \" be a helper. \" all people, children included, are more motivated when their identity is in play.\n\nA. When you call your child helpers, describe what they do and what they need to be helped for. [substeps] You could say, \" i need you to help dad during his lunch break at work.\nB. You can start doing this with your children as early as two years old. [substeps] You might say, \" jayden, can you be a helper and clean your bedroom before grandma comes over? \" or \" please be a helper and stay quiet while your sister naps.\nC. Call them when you need them for help or for monetary help. [substeps] For example, if you need help with something you don't know how to do, let your child know you're excited to help with this.\nD. If you ask your child for things they have access to, it encourages them to put more effort into making things happen. [substeps] To make sure they understand exactly what's expected of them, you could try saying, \" i'm looking for helpers who can be helpers.","prompt":"[header] How to raise your children to be helpers [title] Call them helpers when you ask for things. [step] Instead of asking for help, ask your child to \" be a helper. \" all people, children included, are more motivated when their identity is in play.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6749,"error":null,"output":"B"}
{"question_id":"hellaswag:00007887","item_index":268,"attempt":0,"prompt_hash":"2a3e66e5646cc36bbc4131d98e93c1763bf8bd7590a5e50bf59d1a3c42e29b13","question":"[header] How to make a nerdy girl gorgeous [title] If you have a very large budget-buy stuff from the top designers, like dior, marc jacobs, chanel, gucci, versace and so on. [step] Read vanity fair, harper's bazaar, elle and vogue for ideas and news about the latest trends. Use common sense however, and do not try to become someone you're not.\n\nA. That doesn't make you anything, at least not at first. Be confident, and be up-to-date-with trends! [title] Look after your hair.\nB. Calm down and don't act too girly. [title] If you don't want to spend money, ditch school.\nC. Think about what will make your looks look most flattering, and be honest when you're asking. If you're not sure, ask! [title] Don't be afraid to speak up.\nD. [title] If you have a medium budget-try hitting the mall. [step] First off you need a pair of nice, cute, well fitting (not suffocating but not loose) pair of jeans.","prompt":"[header] How to make a nerdy girl gorgeous [title] If you have a very large budget-buy stuff from the top designers, like dior, marc jacobs, chanel, gucci, versace and so on. [step] Read vanity fair, harper's bazaar, elle and vogue for ideas and news about the latest trends. Use common sense however, and do not try to become someone you're not.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6750,"error":null,"output":"D"}
{"question_id":"hellaswag:00001399","item_index":269,"attempt":0,"prompt_hash":"5f26defcf35e39602914338a1fdaf1d7fd04a0297abffea7b8edc35171316325","question":"A bit of oil is then placed in a skillet and canadian bacon and onions are being fried. Next, more ingredients are added with pasta and egg yolks. finally\n\nA. , the sauce is done and all of the ingredients are compiled and cheese is added to the dish as it is positioned on the plate.\nB. , turkey is scooped into a bowl and blended together with chocolate chips and turkey.\nC. , the chicken sandwich is powdered mix and served up and served in plastic bags.\nD. , various juices are squeezed and mixed into the pan then they are set on drying racks.","prompt":"A bit of oil is then placed in a skillet and canadian bacon and onions are being fried. Next, more ingredients are added with pasta and egg yolks. finally","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6753,"error":null,"output":"A"}
{"question_id":"hellaswag:00006467","item_index":270,"attempt":0,"prompt_hash":"2b0afd29ef3a8df3e5f08d4d30353da89260e4034e0cd9ed953378d4a02461ba","question":"[header] How to deal with a lying child [title] Recognize why younger children lie. [step] In early childhood development, children have not been impacted and guided by the social mores of right and wrong, good and bad. A fibbing child at this age is doing so to either appease an adult, exaggerate a story to impress someone, or because they have forgotten something.\n\nA. When they do lie, they develop a willingness to ask for a right and all answer. Some children may be punished for lying by following this step.\nB. [substeps] This is especially true with children who are used to doing the same and end up telling both sides of the story on their own. [title] Recognize the consequences of lying to conceal the truth.\nC. Although it will sap your child's self-esteem, it eliminates the temptation to lie. Also, the child is telling the truth but has the ability to get caught in a lie.\nD. The child may not even be aware that he has done anything wrong. [title] Accept that lying is a modifiable behavior.","prompt":"[header] How to deal with a lying child [title] Recognize why younger children lie. [step] In early childhood development, children have not been impacted and guided by the social mores of right and wrong, good and bad. A fibbing child at this age is doing so to either appease an adult, exaggerate a story to impress someone, or because they have forgotten something.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6748,"error":null,"output":"D"}
{"question_id":"hellaswag:00001753","item_index":271,"attempt":0,"prompt_hash":"fd64455cfe518b6291024dd9013d1f98fc7c36c3296196acfcbe8a7e69fd40eb","question":"The person makes a hole while another walks around and makes holes. The men sit around the holes with a pole and one grabs a fish. the men\n\nA. break out the fish and show it off.\nB. watch a fish swim around on a fish belly.\nC. continue fishing and showing off their fish to the camera.\nD. get back to fishing and one catches a huge wooden fish.","prompt":"The person makes a hole while another walks around and makes holes. The men sit around the holes with a pole and one grabs a fish. the men","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6752,"error":null,"output":"C"}
{"question_id":"hellaswag:00005188","item_index":272,"attempt":0,"prompt_hash":"6bc795f64bf866b1ffa40db812c170fd34342943ec5b7241b00f9c1294219856","question":"[header] How to do a french braid headband [title] Detangle your hair. [step] Use a paddle brush or wide-tooth comb to get rid of any tangles. Make sure that the hair is completely detangled before beginning the braid.\n\nA. Getting your hair wet can help you with the detangling and braiding process. However, this is not necessary to complete the braid.\nB. You can also use a paddle comb to give your hair a very tight, somewhat twisted appearance. [title] Choose a large hair piece you can cut into four pieces.\nC. [substeps] Start with the side or bottom of your hair that's slightly parted. It should feel stiff now, so go ahead and braid it there.\nD. Depending on how messy your hair is you may need to use wax, oil, or mousse. Shampoo and dry your hair until it is well-hydrated.","prompt":"[header] How to do a french braid headband [title] Detangle your hair. [step] Use a paddle brush or wide-tooth comb to get rid of any tangles. Make sure that the hair is completely detangled before beginning the braid.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6751,"error":null,"output":"A"}
{"question_id":"hellaswag:00002495","item_index":273,"attempt":0,"prompt_hash":"a77564716869b6dbdf37a5b19e5327bb9b07306ba16e565068781214f8fb81c6","question":"A man is seen standing on a playground while lifting himself up and down on a set of bars. the man\n\nA. continues with this exercise and ends with him climbing back up and speaking to the camera.\nB. continues bending forward and swinging himself from side to side.\nC. then does several res crunches and finally turns around to the camera and smiles.\nD. increases his jump and run as well as doing a few tricks.","prompt":"A man is seen standing on a playground while lifting himself up and down on a set of bars. the man","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6750,"error":null,"output":"B"}
{"question_id":"hellaswag:00001904","item_index":274,"attempt":0,"prompt_hash":"e065606cf9e42700b407c477e5719c203c2ca4cb6224d319ce7625209690d73d","question":"A man on a horse chases after a cow with a ripe. A crowd around him watches. he\n\nA. picks the cow up and throws it onto a nearby patch of grass.\nB. clamps the cow down with a chain, breaking it into pieces.\nC. catches the cow with the rope.\nD. reaches the cow and grabs her mane, pulling her upright and marking her with a bite of a cow bar.","prompt":"A man on a horse chases after a cow with a ripe. A crowd around him watches. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6751,"error":null,"output":"C"}
{"question_id":"hellaswag:00006120","item_index":275,"attempt":0,"prompt_hash":"548d960ca44c22e1ae0a81fa7f8f9ca5a71232fac125d9d6daf0d4891c2d1e5c","question":"[header] How to find the perfect jeans for you [title] Wear tapered ankle jeans if you've got an apple shape. [step] If your upper body is curvier than your lower body, opt for tapered ankle or skinny jeans. These styles accentuate thin legs.\n\nA. If your upper body is slim, stick with straight knee jeans. [substeps] This thin look is not to be argued against, as trousers, tops, and t-shirts have to be slim and also accentuate your calves.\nB. If you have thin legs, opt for slim jeans. [substeps] Tapered ankle jeans can highlight your softer features, like your ribcage.\nC. [substeps] As another option, you can pair skinny jeans with a flowy top to create balance. Wear low-rise jeans if you're slimmer and want to create curve, and wear high-rise jeans if you're a curvier apple shape.\nD. [substeps] If you have thin legs, ankle or skinny jeans will not accentuate those assets. [title] Wear low-rise cropped jeans if you're curvy.","prompt":"[header] How to find the perfect jeans for you [title] Wear tapered ankle jeans if you've got an apple shape. [step] If your upper body is curvier than your lower body, opt for tapered ankle or skinny jeans. These styles accentuate thin legs.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6751,"error":null,"output":"C"}
{"question_id":"hellaswag:00004522","item_index":276,"attempt":0,"prompt_hash":"95923f6905212dc70e46bac00d302810f4086801879ce2ac5f925b7998f77dee","question":"[header] How to train your dog with silent signals [title] Choose a command to teach. [step] Before you begin your training session, choose one command to focus on with your dog so that he will not be overwhelmed. You may want to start with something simple, such as \" sit.\n\nA. \" a quiet click should help the dog put his ears back and relax. You can then give the command to sit by clapping your hands.\nB. \" instead of saying \" sit, \" teach your dog to sit if he is not already sitting. If he is, have him sit in a quiet area, such as a special spot in your home.\nC. \" or you may want to pick a specific command for your dog. Your dog will most likely respond to the command if you choose a simple one.\nD. \" [substeps] Commands such as sit, stay, heel, come, and lie down are considered basic obedience commands. These are important commands to teach your dog so that he will be well behaved.","prompt":"[header] How to train your dog with silent signals [title] Choose a command to teach. [step] Before you begin your training session, choose one command to focus on with your dog so that he will not be overwhelmed. You may want to start with something simple, such as \" sit.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6752,"error":null,"output":"D"}
{"question_id":"hellaswag:00002424","item_index":277,"attempt":0,"prompt_hash":"d2d79d40f6958e953807642165f6108fc0297c1506a3573b79b032fd8817276a","question":"People play games of badminton on indoor courts. the players\n\nA. have a long volley until the play in the foreground misses and the birdie lands at her feet.\nB. bring the stones, then ping the birdie back to the team on the other side of the court.\nC. hit the badminton, hitting the other both in the head and back.\nD. tap the wood with their racket repeatedly in the outdoor game area.","prompt":"People play games of badminton on indoor courts. the players","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6749,"error":null,"output":"A"}
{"question_id":"hellaswag:00003684","item_index":278,"attempt":0,"prompt_hash":"51a6afdee96429084b6a705c4d48f281eed54b2a3444ba24209936e36fa5e39b","question":"[header] How to lay off an employee [title] Think about who you need to lay off. [step] Make sure that the people that remain will be able to cover the work of those who are being laid off. Layoffs, unlike firings, are typically about the company performing poorly and needing to let part of the workforce go.\n\nA. Layoffs tend to be about keeping the workforce satisfied and working towards change on a cutting edge, but that's not the case very often. [title] Consider canceling certain lines of work.\nB. With layoffs, employees will be able to claim unemployment benefits. Firings, on the other hand, are usually about the poor performance of the employee.\nC. If you are thinking about building an employee k-5 team, then take a look at others working with the company to determine if they are capable of pitching on to perform well in everyday situations. Those that are being laid off will need to acquire willing employees.\nD. [substeps] Ideally, your most overlooked employees should be allowed to remain, but this isn't always possible. For instance, keep one employee off the payroll for the time being, and opt for another boss when you have the opportunity.","prompt":"[header] How to lay off an employee [title] Think about who you need to lay off. [step] Make sure that the people that remain will be able to cover the work of those who are being laid off. Layoffs, unlike firings, are typically about the company performing poorly and needing to let part of the workforce go.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6987,"error":null,"output":"B"}
{"question_id":"hellaswag:00009363","item_index":279,"attempt":0,"prompt_hash":"afcbcaa9f52bcaa173672948a923853dfd1083060960ed9381ca0e7d910c1e51","question":"[header] How to choose a hairstyle for spring [title] Look up spring styles online. [step] If you want to keep with modern trends, try using a search engine to find what's trending this spring. Type something like \" spring hair trends \" into a search engine and see what comes up.\n\nA. [substeps] You can find happy modification and many other inspiration websites for spring hairstyles. [title] Get a style that fits the season.\nB. This can narrow down your search to a wide range of colors. [substeps] You could start with a pretty, cute hairstyle in a spring that uses a wide variety of colors.\nC. You can also find websites (such as alerts.com) that focus on spring trends. [substeps] For example, goodreads.com is a good place to start your search to see what's currently on the market.\nD. You can browse styles online and see if you find anything you like. If you find a style you would like to try, make an appointment with your stylist.","prompt":"[header] How to choose a hairstyle for spring [title] Look up spring styles online. [step] If you want to keep with modern trends, try using a search engine to find what's trending this spring. Type something like \" spring hair trends \" into a search engine and see what comes up.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6985,"error":null,"output":"D"}
{"question_id":"hellaswag:00008406","item_index":280,"attempt":0,"prompt_hash":"dc0ca59e0bb12795edc6cd7e95584eb7ff87803b3a6c67f096bf80b3354dc9cb","question":"[header] How to research adoption on the internet [title] Check national websites. [step] The federal government has adoption resources available, primarily through the u.s. children's bureau's adoptuskids program. On these websites, you can find detailed information about adopting a child.\n\nA. The most common sites to find a child are the, , and. [title] Take a dna sample from a running puppy.\nB. [substeps] The adoptuskids website at adoptuskids.org is a good place to start if you want to research adoption on the internet. This website has national as well as state-specific information about the adoption process.\nC. You can also go to the national adoption database, which is probably one of the more useful resources for finding adoption professionals. [substeps] In oregon, you can search for laws requiring adoption from the department of human services on your behalf, even if these laws do not require you to apply for an agency's approval.\nD. This database will help you learn information about your child, from who lives in the home, where the child lives, and the proposed adoption program. [substeps] The national adoption hotline is a free service designed to work with all adoptions in your area.","prompt":"[header] How to research adoption on the internet [title] Check national websites. [step] The federal government has adoption resources available, primarily through the u.s. children's bureau's adoptuskids program. On these websites, you can find detailed information about adopting a child.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6987,"error":null,"output":"B"}
{"question_id":"hellaswag:00010010","item_index":281,"attempt":0,"prompt_hash":"8a1a715d175bdeead4160e5970135925a15fbcf59c37d47bca6520cdf37cf162","question":"[header] How to prevent others from buying stuff with your amazon echo [title] Open the application. [step] Click on the application in your smartphone or tablet to open it up. Once it is open, look for the \" menu \" tab in the upper left corner.\n\nA. Tap the \" menu \" tab to open up the options. [title] Tap \" settings \" in the \" menu \" tab.\nB. Click \" accessories \" to open up the application. [title] Click the \" cost \" button beside the card.\nC. Under the other three options, click \" settings. \" from here, select the \" server settings \" tab to open it.\nD. From here you can add and display the contact information. [substeps] You can also choose how you want to share information, such as your username, email address, and password.","prompt":"[header] How to prevent others from buying stuff with your amazon echo [title] Open the application. [step] Click on the application in your smartphone or tablet to open it up. Once it is open, look for the \" menu \" tab in the upper left corner.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6985,"error":null,"output":"A"}
{"question_id":"hellaswag:00008623","item_index":282,"attempt":0,"prompt_hash":"b832a0a3754217044d9758fbdf6599019f07ded4962f96d8b79969fa0b9bbe13","question":"[header] How to write a memo [title] Type \" memorandum \" at the top of the page. [step] State that this document is a memorandum at the outset. Label the page \" memorandum \" 1.5 inches (3.8 cm) from the top of the page.\n\nA. [title] Copy and paste the subject line in the upper left corner. [step] Use uppercase letters, bold letters, or capitals for the subject line.\nB. [substeps] You can easily get a free template online from http: // www.writewhippsmy.com/templates /. [title] Write a third column below the first column heading.\nC. Use one line per column, as noted earlier. [title] Place quotation marks at the top of the page.\nD. Put the word in bold on the first line. You can either center it on this line or left-align it.","prompt":"[header] How to write a memo [title] Type \" memorandum \" at the top of the page. [step] State that this document is a memorandum at the outset. Label the page \" memorandum \" 1.5 inches (3.8 cm) from the top of the page.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6986,"error":null,"output":"A"}
{"question_id":"hellaswag:00005230","item_index":283,"attempt":0,"prompt_hash":"76ac5b464d049791efc716a92d1c73a7dae607716268641a3c37afda04fa1d98","question":"[header] How to get your child to use a baby walker [title] Determine if your baby is ready for a walker. [step] There are differing opinions on exactly when a baby is ready to use a baby walker. There is no set age, since all babies develop at their own rate.\n\nA. Questions about how to start a child walker program should be considered for parents. [substeps] The early childhood walkers are older versions of the toddler walker.\nB. However, there are certain signs that indicate when the baby is ready to use a baby walker : [substeps] In order to use a baby walker, the baby should be able to sit up on their own and be able to crawl. Sitting up is necessary because the baby needs to sit up in the walker.\nC. Some children are ready with a baby walker when they are one year old and ready to have another baby. [title] Exercise caution when getting a baby walker.\nD. In general, babies between the ages of 4 and 6 months are ready for a walker. [title] Choose a stroller that has four handles.","prompt":"[header] How to get your child to use a baby walker [title] Determine if your baby is ready for a walker. [step] There are differing opinions on exactly when a baby is ready to use a baby walker. There is no set age, since all babies develop at their own rate.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6661,"error":null,"output":"B"}
{"question_id":"hellaswag:00006546","item_index":284,"attempt":0,"prompt_hash":"3d638420198d75bc6f072620f34cc606a0e35d6cfc83de6fc1f96afe9da91058","question":"[header] How to get a job fast [title] Look for relevant job postings online. [step] These days most jobs are posted online. Search a variety of websites for job listings.\n\nA. Generally, reputable job websites might be a good place to start. [substeps] If you can't find listings online, consider looking at local newspaper advertisements and job postings you've read online.\nB. Many companies and public institutions post their job openings right on their own websites. There are also many websites that collect a wide variety of job openings.\nC. Consider doing a general internet search, which can find what jobs are listed. Alternatively, if you just read up on something, it could be a great way to complete your online search.\nD. These websites will pull in job postings from sites like monster and indeed, nbc if it hasn't already done so. [substeps] Click on the \" apply online \" button to begin your search.","prompt":"[header] How to get a job fast [title] Look for relevant job postings online. [step] These days most jobs are posted online. Search a variety of websites for job listings.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6663,"error":null,"output":"B"}
{"question_id":"hellaswag:00007285","item_index":285,"attempt":0,"prompt_hash":"4ef3ba6ec02b74e526bd98113f6a2b750e01ea86b6f1ac961d20a3e40e88d0e0","question":"[header] How to be close to your crush [title] Find common interests. [step] One of the simplest ways to be closer is to find activities and interests that you and your crush both like. Talk with your crush about something they like.\n\nA. Alternatively, read a book you both like. If you both prefer at least some of the same books, try out reading together.\nB. Identify the interests that match your own, and spend more time on those activities. [substeps] Consider asking things like, \" what's it like being on the soccer team? I used to play soccer \" or, \" i noticed that you like to draw.\nC. You can do this together, or simply sit next to one another and reminisce about the topics you both like. Doing this can make the connection between the two of you more apparent, as you'll have the chance to show your crush that you really do like them too.\nD. Maybe your favorite movie, maybe your best friend likes chess or if one of you likes a book you think they might enjoy, keep that interest in mind. Find things you enjoy in that interest too.","prompt":"[header] How to be close to your crush [title] Find common interests. [step] One of the simplest ways to be closer is to find activities and interests that you and your crush both like. Talk with your crush about something they like.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6662,"error":null,"output":"C"}
{"question_id":"hellaswag:00007266","item_index":286,"attempt":0,"prompt_hash":"3677eaa8e3476ca5f8166228a873609aabfddcc13a365bb135c0bf53778aa07f","question":"[header] How to take care of your skin after using a tanning bed [title] Consider skipping a shower. [step] It's not because you need to worry about \" washing away \" your tan. The melanin production stimulated by uva light won't be stopped by a shower.\n\nA. Water can wash away that burning from the sun, so consider skipping a shower if you're concerned you'll itch or burn yourself. Your natural skin absorbs uv light better during the day when it is dehydrated.\nB. Simply skipping a shower can actually lead to darker, more noticeable skin. [substeps] A shower brings out the skin! Therefore, take a shower a couple of times daily just to see if that helps.\nC. Rather, studies show that showering and then applying moisturizer does not hydrate your skin as well as applying moisturizer alone. If you do shower, be sure to : [substeps] Take a cold or warm shower, not a hot one.\nD. You need to put on sunscreen as soon as you get out of the shower, so hot weather can block melanin production. [substeps] If you live in an environment with a lot of sun (such as a tropical climate), apply a sunscreen three to four hours before you plan to go on a tanning session.","prompt":"[header] How to take care of your skin after using a tanning bed [title] Consider skipping a shower. [step] It's not because you need to worry about \" washing away \" your tan. The melanin production stimulated by uva light won't be stopped by a shower.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6662,"error":null,"output":"C"}
{"question_id":"hellaswag:00008005","item_index":287,"attempt":0,"prompt_hash":"a0046786ccc4640098d89c2983acc3e748abc60b9d2789ce0452a9be0a94b69c","question":"[header] How to improve a toddler's speech [title] Get down to their level. [step] Squat or sit on the floor. This will get their attention.\n\nA. You will talk with your child instead of at her. Also, she will be able to see your mouth and get visual cues on how to say certain sounds.\nB. [title] If needed raise hands together in fists. [step] If you stretch your hands to fists and do this while speaking, your child will most likely also do the same.\nC. Try to just be as quiet as possible and only talk to them when they are calm. If you are talking for long enough, they will eventually hear your voice.\nD. Let them ask you to move closer to them. If possible, use a head-high perch.","prompt":"[header] How to improve a toddler's speech [title] Get down to their level. [step] Squat or sit on the floor. This will get their attention.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6662,"error":null,"output":"A"}
{"question_id":"hellaswag:00000449","item_index":288,"attempt":0,"prompt_hash":"105472222981cb617358547a461598ccfe290bbc7c0c878c1da5cf0f28b7de49","question":"A man is standing on a stage playing a violin. a man\n\nA. is standing behind him in a tuxedo.\nB. comes and stands on the stage next to him.\nC. in a position of power is walking on stage with the violin.\nD. is performing a song on a flute.","prompt":"A man is standing on a stage playing a violin. a man","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6010,"error":null,"output":"B"}
{"question_id":"hellaswag:00003622","item_index":289,"attempt":0,"prompt_hash":"62c3439eda88912174de350b3bafb08b849769e0f7d0a7cdbb3bbebe305218c1","question":"[header] How to make a christmas cheesecake [title] Preheat the oven to 180ºc , 160ºc fan forced (convection). [step] Line the cheesecake pan with parchment paper. [title] Arrange the fruit slices evenly across the base of the pan.\n\nA. [step] Place the pan in the oven and bake for 10 minutes or until the slices seem slightly browned. [title] Reduce the oven to 150ºc , 130ºc fan forced.\nB. [step] Make sure that the slices are touching evenly on all sides. [title] Place the cake mixture on the pre-greased plate.\nC. [step] Scatter the fruit slices with ice cream. Drizzle the topping (milk, jam, maple syrup, etc.\nD. [step] Use a rubber spatula to create a 9-inch circumference at the center. Place the remaining fruit slices over the top, too, but not so much that they block out air from the area between their halves.","prompt":"[header] How to make a christmas cheesecake [title] Preheat the oven to 180ºc , 160ºc fan forced (convection). [step] Line the cheesecake pan with parchment paper. [title] Arrange the fruit slices evenly across the base of the pan.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5997,"error":null,"output":"A"}
{"question_id":"hellaswag:00002168","item_index":290,"attempt":0,"prompt_hash":"f3db54bfc4b85397907282a4570ca91645506b5dc8bbd4dbb682d799753ab3b6","question":"There's a young boy and his mother who smile at the lady as she plays. Then they get off the elevator when they reach their level. the lady\n\nA. continues playing the harmonica as the elevator moves further and stops at other levels and more people walk in.\nB. makes a response and life resumes as they both go through the building and out the door.\nC. is brushing the cheeks of the boy as he lays on the ground.\nD. rings a bell for the boy to go up stairs.","prompt":"There's a young boy and his mother who smile at the lady as she plays. Then they get off the elevator when they reach their level. the lady","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6009,"error":null,"output":"A"}
{"question_id":"hellaswag:00008160","item_index":291,"attempt":0,"prompt_hash":"460a68ec7e42ce1ba36946216e61941d7ab2af940129918dccc14a3884095e64","question":"[header] How to ask a shy girl out [title] Try to make her laugh. [step] When getting to know a shy girl, it's important to make her feel comfortable around you. She will be more willing to open up to you if you make her feel comfortable.\n\nA. An easy way to do this is to make her laugh. Tell jokes and act goofy, but don't make fun of her.\nB. Here are some ways to keep her comfortable : [substeps] Tell her you like her. Sometimes the shy girl may have a difficult time coming to terms with her shyness.\nC. Here are some ways to do this : [substeps] Fill her ears with funny stories. Find some funny jokes about your own life.\nD. By approaching a shy girl and introducing yourself, you are showing her that you care about her. [substeps] To make a funny face with a shy girl is a great way to initiate a friendship.","prompt":"[header] How to ask a shy girl out [title] Try to make her laugh. [step] When getting to know a shy girl, it's important to make her feel comfortable around you. She will be more willing to open up to you if you make her feel comfortable.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5996,"error":null,"output":"A"}
{"question_id":"hellaswag:00002079","item_index":292,"attempt":0,"prompt_hash":"22eb97e2b83447273b626ddef864dc8480b55af3f5b54cc3e8c7376a9891f30c","question":"A close up of a poker table is shown followed by the camera panning around to other tables in the area as well as people playing. several clips\n\nA. are shown of people sitting at tables gambling as well as laughing to the camera.\nB. are shown of people discussing or speaking to the camera and playing beer pong.\nC. are shown of people playing in different locations while many watch on the sides.\nD. are shown of people playing and playing billiards using the shot puts as well as scoring goals followed by the people running around.","prompt":"A close up of a poker table is shown followed by the camera panning around to other tables in the area as well as people playing. several clips","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6008,"error":null,"output":"C"}
{"question_id":"hellaswag:00005192","item_index":293,"attempt":0,"prompt_hash":"f8db74597af35cf7d45c7160763d1b95224cd042e1078173b5b4e0a19198d7e9","question":"[header] How to know when your child is old enough to babysit [title] Ask your child if this is something they'd like to do. [step] Interest is a prerequisite for babysitting. An interested babysitter is much more likely to be attentive.\n\nA. In a school setting, you might ask if this is an option, but remember, you can always ask in social settings. [substeps] You might say, \" what's it like to babysit after school? How many hours per day do you usually babysit? \" [title] Confirm they'd be interested in the role.\nB. They may wish to spend a certain amount of time finding a suitable babysitter, or it may be as simple as asking them \" would you like to babysit this day? \" [substeps] Many parents leave this portion out. [title] Talk to your child's parents.\nC. If your child is not excited by the prospect of babysitting, it might not be the right time. [title] See if your child approaches you about it.\nD. They'll appreciate that you've taken the time to think about a task. [substeps] Think about if your kids are ready for doing this sort of work, or aren't much older.","prompt":"[header] How to know when your child is old enough to babysit [title] Ask your child if this is something they'd like to do. [step] Interest is a prerequisite for babysitting. An interested babysitter is much more likely to be attentive.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5995,"error":null,"output":"C"}
{"question_id":"hellaswag:00000410","item_index":294,"attempt":0,"prompt_hash":"b667a72637bafbd485bfbfc790588377cf3d9d1d878b25cbbf698e8bb0dd563a","question":"A person is seen fighting a bull and being trampled and impailed. the people\n\nA. carry him away and show more clips of people bull fighting.\nB. run along side of the bull and cower on the sides.\nC. are dragged back and forth as more videos are shown of the people having fun and running around the city fighting.\nD. push bulls around on each other while trampling them.","prompt":"A person is seen fighting a bull and being trampled and impailed. the people","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6007,"error":null,"output":"A"}
{"question_id":"hellaswag:00003933","item_index":295,"attempt":0,"prompt_hash":"3ec15de2fa77f845446a223af032e16f20b4a7145102262ce9ba80ebdb31e6da","question":"[header] How to buy an amethyst ring [title] Decide what hue of amethyst you wish to buy. [substeps] Iron is the agent that gives the amethyst its purple hue. The shade can vary from a pale reddish-violet to medium violet to deep blackish-purple.\n\nA. The shade of amethyst you purchase here is incredibly useful because it provides an violet glow. Stone hardness will range from 10 to 40 drp.\nB. Amethyst gets brighter as the stone is heated to achieve a darker hue. Expect to buy one ring of amethyst every six months.\nC. The color may also come in pastel or peach shades. [title] Measure the price of amethyst jewelry at the base of the stone.\nD. The most valuable amethysts are those that are deep purple with rose, blue, or red flashes. [title] Check the stone that you are considering buying in various types of lighting.","prompt":"[header] How to buy an amethyst ring [title] Decide what hue of amethyst you wish to buy. [substeps] Iron is the agent that gives the amethyst its purple hue. The shade can vary from a pale reddish-violet to medium violet to deep blackish-purple.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6006,"error":null,"output":"D"}
{"question_id":"hellaswag:00004730","item_index":296,"attempt":0,"prompt_hash":"69cbeb779d49396f45e879af9c31c7ba65253f50d45fd2ec99fbf14e204d07b0","question":"[header] How to play altissimo notes on clarinet [title] Understand that these notes are pitched squeaks and probably won't be the most relaxing sounds to your ears for a while. [step] As you get used to the fingerings and air intensity, you can shape the sound to create a warmer tone. [title] Finger a top space e. [step] Then lift your top finger.\n\nA. Then feel how the top space knuckle upon your finger brings air to the reed. You are now ready to play three notes without moving your finger at all.\nB. Then, lift the top and rub the top of your top finger around the bottom of your top lip. This action works on most popular types of notes, from oboe to symphonic scale.\nC. This will be your first altissimo note. Meet c! [title] Finger a top line f.\nD. Place the tip of the thumb on the top space e on the clarinet. This, in turn, mimics the lift of the fingers on the other keys.","prompt":"[header] How to play altissimo notes on clarinet [title] Understand that these notes are pitched squeaks and probably won't be the most relaxing sounds to your ears for a while. [step] As you get used to the fingerings and air intensity, you can shape the sound to create a warmer tone. [title] Finger a top space e. [step] Then lift your top finger.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6003,"error":null,"output":"C"}
{"question_id":"hellaswag:00006145","item_index":297,"attempt":0,"prompt_hash":"bc93a8a2f0a1ef36239bfbde824333ad9cf8f5e32c367323d427f8606ef4d195","question":"[header] How to cope with the death of a baby who died during labor and delivery [title] Say goodbye to the baby. [step] A stillbirth or death during delivery can be emotionally devastating. Many parents find it cathartic to hold the baby in order to create memories and find a sense of closure.\n\nA. [substeps] Seeing the baby, holding him or her, and giving the child a name can be very helpful to parents whose child died during labor. Some parents like to take photographs to help them remember.\nB. Think of this as the first time you know for sure that your baby is still alive, that his permanent home is there, and that your heart is still healing. [substeps] It may be daunting to decide what to say.\nC. Simply telling a baby goodbye reminds the baby that they were born not so young and that the world is now around them again. When a baby realizes it's time to come to terms with death, the baby is ready to learn how to cope.\nD. Talk to your loved one about your intentions, and how you would like to grieve together. A baby that passed on in your womb will not have the same emotional experience as a dead baby.","prompt":"[header] How to cope with the death of a baby who died during labor and delivery [title] Say goodbye to the baby. [step] A stillbirth or death during delivery can be emotionally devastating. Many parents find it cathartic to hold the baby in order to create memories and find a sense of closure.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6002,"error":null,"output":"A"}
{"question_id":"hellaswag:00000225","item_index":298,"attempt":0,"prompt_hash":"3693fffc647dd9fe94eac7dd0f59bed8c97ced351fb805253c3ef023a0b771da","question":"A man cross the street holding a small flag and extend his hand. A group of soldiers holds rifles and flags on front a black car. a veteran\n\nA. soldier carries a long shovel that blow leaves in the wind.\nB. soldier climbs the stairs of a house going to rifle shops.\nC. make the military salute.\nD. soldier in a yellow blue uniform is carry the flag of the fitters.","prompt":"A man cross the street holding a small flag and extend his hand. A group of soldiers holds rifles and flags on front a black car. a veteran","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6006,"error":null,"output":"C"}
{"question_id":"hellaswag:00007992","item_index":299,"attempt":0,"prompt_hash":"2d8bb1654a9e9e08c73e20e3eacb5a5223c05c7bc56efa63ae3dbd3abc98ffcd","question":"[header] How to officiate at a nondenominational funeral service [title] Talk to the family. [step] Call the family and arrange to meet them, preferably in their home. Meeting them in public is ok in a bind, but if that's the arrangement, try not to meet more than two to three people.\n\nA. [substeps] If the ceremony is inside a church, don't bother sitting outside on a park bench. If you're a minister or midwife, it's best to arrive by the usual time of worship.\nB. [substeps] Of course, only with a family member and one of their kids is okay. If no one at your funeral is willing to attend, that's fine.\nC. People aren't always happy with the small acts of an officiant. In some cases, the family can become drunk, agitated, or violent during or around a service.\nD. If a funeral home is taking care of other arrangements, the funeral directors may have space at the funeral home where you could meet. Invite the family to have other family members and close friends present if they wish, but be sensitive to their needs and to the space considerations.","prompt":"[header] How to officiate at a nondenominational funeral service [title] Talk to the family. [step] Call the family and arrange to meet them, preferably in their home. Meeting them in public is ok in a bind, but if that's the arrangement, try not to meet more than two to three people.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5994,"error":null,"output":"D"}
{"question_id":"hellaswag:00008109","item_index":300,"attempt":0,"prompt_hash":"8c17c48113af8ac8a50b65c8e0ac27efa80d2e3ff5b29cb7a14665c51691fdcd","question":"[header] How to increase breast milk production [title] Consume a minimum of 1 , 800 calories a day and drink at least 6 glasses of fluids while you are lactating. [step] If you're currently dieting, it could be decreasing your milk production. Unsurprisingly, what you eat has a big impact on the quality and quantity of the milk produced.\n\nA. Here are some general guidelines for you to remember about diet and breast milk : [substeps] Find excellent sources of calcium. These will help your little baby's little bones grow healthy and strong.\nB. [title] Eat 4 to 6 cups of milk per day. [step] If you're currently underweight, aim for 25 to 35 cups per day.\nC. A list of guidelines for how much you should consume might be printed on the back of your bottle. [substeps] For example, you may want to limit your milk (97 g) to about 2 glasses of water per day.\nD. [substeps] Women in the first month of pregnancy generally need between 80-400 calories a day. Men in the third month will eat within 15-30% of this extra calorie.","prompt":"[header] How to increase breast milk production [title] Consume a minimum of 1 , 800 calories a day and drink at least 6 glasses of fluids while you are lactating. [step] If you're currently dieting, it could be decreasing your milk production. Unsurprisingly, what you eat has a big impact on the quality and quantity of the milk produced.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5993,"error":null,"output":"A"}
{"question_id":"hellaswag:00006750","item_index":301,"attempt":0,"prompt_hash":"09f3f0968fe5654a2a8b356a1a4031ff4b20cf0cdb236de3c4708a06783d1f18","question":"[header] How to protect pets from mosquitoes [title] Apply a pet bug spray. [step] Pet stores do sell bug sprays formulated just for animals. If you are planning to be out for extended periods of time with your dog or other pet, find a spray that is appropriate for them, and apply it to their coat according to the directions on the bottle.\n\nA. [substeps] You can apply the insect repellant directly to pets, or you can use insect repellant solution. You can find bug repellant solution at most pet stores or online.\nB. [substeps] You may notice more flea and tick formulas than ones for mosquitoes. Look more closely at the bottle, as those sprays will sometimes guard against mosquitoes.\nC. [substeps] There are several different repellents available on the market. You may have to shop around a bit before coming up with one that appeals to you.\nD. [substeps] Check the bottle as well to make sure the chemicals in the spray do not hurt any pets. If your dog or other pet is traveling with you, consider getting a bug spray designed specifically for dogs.","prompt":"[header] How to protect pets from mosquitoes [title] Apply a pet bug spray. [step] Pet stores do sell bug sprays formulated just for animals. If you are planning to be out for extended periods of time with your dog or other pet, find a spray that is appropriate for them, and apply it to their coat according to the directions on the bottle.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5994,"error":null,"output":"C"}
{"question_id":"hellaswag:00003314","item_index":302,"attempt":0,"prompt_hash":"1d7464cb4491d801a03b5b7bdb8e17e143dd32c4c21bfe34655d2b97a33fc0f2","question":"[header] How to wear a dress in winter [title] Try tights and leggings. [step] Opaque tights can be worn with short, mid-length, and long dresses to keep your legs warm. Instead of tights, opt for leggings for those extra cold winter days and nights.\n\nA. Have a pair of tights for those freezing cold days. You can also wear or opt for leggings in a neutral color for a warm, chic look.\nB. Keep in mind that nude tights and leggings (black, grey, and cream) go well with any dress. [substeps] Keep it fun by wearing colored or patterned tights and leggings; for example, match a black dress with red or purple tights.\nC. [substeps] Cold weather cuts down on clothing and can get stinky hot as well if you're cold. Allow yourself to sweat for at least three hours in extreme temperatures.\nD. Also, look for tights made of a breathable material like cotton as they can help keep you warm. [substeps] You can find tights at almost any clothing retailer.","prompt":"[header] How to wear a dress in winter [title] Try tights and leggings. [step] Opaque tights can be worn with short, mid-length, and long dresses to keep your legs warm. Instead of tights, opt for leggings for those extra cold winter days and nights.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6001,"error":null,"output":"D"}
{"question_id":"hellaswag:00004672","item_index":303,"attempt":0,"prompt_hash":"62efdaf3921582fc1e899e5bbe8384d230f65286157cebe62c4a4d2a4f09af2a","question":"[header] How to become a destination wedding photographer [title] Take professional photography courses. [step] Before you jump into the specifics of becoming a destination wedding photographer, you must become familiar and comfortable with all the technical aspects of being a photographer. Enroll at a professional photography institute or take classes as your local community college.\n\nA. These courses will teach you about local photography, the role of the bride, and planning various wedding functions while providing you with additional studio experience. [substeps] One advantage of a wedding photography program is that you will be seen by photographers to be new to photography in the area.\nB. You should also take classes on digital photography, since these do not hold up well with the cover shots required for an entry-level position. [substeps] If you are not sure what level of photographer you'd like, your local university may be the perfect place to start.\nC. [substeps] Don't assume that you have to attend a top photography school. Learn the basics, regardless of where you go, and practice constantly to improve your skills.\nD. [substeps] Be familiar with the 100-200 subjects your subject will need to know in order to become a destination wedding photographer. You must know basic chemistry, photography skills, and how to load and fire an espresso.","prompt":"[header] How to become a destination wedding photographer [title] Take professional photography courses. [step] Before you jump into the specifics of becoming a destination wedding photographer, you must become familiar and comfortable with all the technical aspects of being a photographer. Enroll at a professional photography institute or take classes as your local community college.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6000,"error":null,"output":"C"}
{"question_id":"hellaswag:00000805","item_index":304,"attempt":0,"prompt_hash":"5d10d09f2c6a71b5fc749d521aa6761034706aa95d92e808cd85b61c2f00abce","question":"A man is shown working in an outdoor garden. he\n\nA. is using tools as he works.\nB. uses a large tool to bark out a few new leaves.\nC. uses trowels and strikes a tree.\nD. uses a sponge and a trowel to scrub down the side of a tree.","prompt":"A man is shown working in an outdoor garden. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6003,"error":null,"output":"A"}
{"question_id":"hellaswag:00000276","item_index":305,"attempt":0,"prompt_hash":"86cb272ebca40ebb6049c40a920e905dc7361d8276047dcf2d3aaad294a7b161","question":"There are several jockeys playing polo on the field in an event. There are also several spectators standing and watching the event. one of the polo players\n\nA. is trotting on the horse across the field.\nB. has a huge broken leg.\nC. falls and is caught by the crowd.\nD. is pulling off his jockey shorts.","prompt":"There are several jockeys playing polo on the field in an event. There are also several spectators standing and watching the event. one of the polo players","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6003,"error":null,"output":"A"}
{"question_id":"hellaswag:00003240","item_index":306,"attempt":0,"prompt_hash":"1c4efff3801853710ad9e27115d829089c119d1df266095f4d97075e78e74e9e","question":"A band of people are playing instruments. a man\n\nA. is standing up in a suit at the front playing a violin.\nB. has his wrist tattooed with music.\nC. taps his neck and arm drums.\nD. starts dancing on a stage.","prompt":"A band of people are playing instruments. a man","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6003,"error":null,"output":"D"}
{"question_id":"hellaswag:00001136","item_index":307,"attempt":0,"prompt_hash":"f56789785c1bda8279e936574592f6c65295ef20a320cbc0a960e0905a785549","question":"A man in gray talks to an usher and four other men comedown the stairs and go to their seats. a lady in pink shirt\n\nA. speaks to a man and a few younger adults face off on a stage.\nB. picks up a cue stick.\nC. walks up the stairs.\nD. gets the jet ski black carpet.","prompt":"A man in gray talks to an usher and four other men comedown the stairs and go to their seats. a lady in pink shirt","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6002,"error":null,"output":"C"}
{"question_id":"hellaswag:00009443","item_index":308,"attempt":0,"prompt_hash":"4633a42661dd525eb87cc2b45db00efe70a07ef3022c3c92afb5ceb0c08cc99e","question":"[header] How to deal with rats [title] Set out poison rat baits. [step] If you have an infrequent rat problem, or would rather not have to check traps on a daily (or weekly) basis, invest in poison rat bait. Simply set a small chunk of the poison bait in areas where you've seen rats, including the attic or basement of your home.\n\nA. Avoid releasing it further back into the rat's home. See warnings below for making sure to have the bait out before, during and after feeding.\nB. Hold the rat securely up above your head, then turn on the floor lights or shut the window blinds. Drop a small pouch over your shoulder, and you have an easy kill for rats.\nC. This will prevent rats from seeing the bait. You can also put the bait out during other times, like when you see rats on their way out of the house or when you're out at lunch.\nD. The poison is designed to smell appealing to rats, who will die once they've ingested the bait. [substeps] You can buy a bucket of poison rat baits at a local hardware store for about $25 usd.","prompt":"[header] How to deal with rats [title] Set out poison rat baits. [step] If you have an infrequent rat problem, or would rather not have to check traps on a daily (or weekly) basis, invest in poison rat bait. Simply set a small chunk of the poison bait in areas where you've seen rats, including the attic or basement of your home.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5999,"error":null,"output":"D"}
{"question_id":"hellaswag:00005035","item_index":309,"attempt":0,"prompt_hash":"7632cdc6d311909f612f7cc809587e5cf25eefdffb43395a9bb556c282959ed8","question":"[header] How to fall out of love with your best friend [title] Stay away from your best friend for a while. [step] Be civil and thoughtful in how you go about this. You don't want to completely reject the healthy relationship that you've established between each other, but you need to take care of your feelings.\n\nA. [substeps] Tell your best friend that you aren't interested. On any occasion after inviting them out for dinner, you can mention that you're going to watch a movie and want to see what the other person has done.\nB. Don't make active efforts to see them on a daily basis. [substeps] There are countless ways to set up safe and respectful boundaries between you and this person.\nC. It is what relationships are made out of. [title] Unfollow your best friend.\nD. Trying to give back without being too overbearing, especially in the long run, is not productive or effective. [substeps] Avoid overwhelming your friends by telling them outright that you're going to take them out on dates.","prompt":"[header] How to fall out of love with your best friend [title] Stay away from your best friend for a while. [step] Be civil and thoughtful in how you go about this. You don't want to completely reject the healthy relationship that you've established between each other, but you need to take care of your feelings.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6003,"error":null,"output":"B"}
{"question_id":"hellaswag:00009299","item_index":310,"attempt":0,"prompt_hash":"0b5ae684e2df42479ce606785d1747ad840ac5e1254845b6f700b43d1326efac","question":"[header] How to develop personality [title] Write down five personalities traits about yourself. [step] Think about at least five or more personality traits that you have. Write them down, and take a few sentences to describe why you think this personality trait applies to you.\n\nA. Come up with at least five traits that fit. [substeps] You can note that the hallmark traits of a personality trait include a positive attitude toward others, an optimistic mindset, and a knack for listening to others, and an ability to show respect for others.\nB. How do you think you would like these traits to be about, or think about them in concrete terms? You can either put the traits into a bowl and name them, or label and subtract traits by adding. [substeps] For example, write down: \" i'm an energetic person i love to jump up and down.\nC. [substeps] Make sure at least one of them is a positive, good personality trait that you see in yourself. Also, list one personality trait that is negative or bothering you in some way.\nD. [substeps] For example, if you don't like to draw, think about your personality when you write down why. For example, you may like how blue eyes look, but are less bold with merlot, or vice versa.","prompt":"[header] How to develop personality [title] Write down five personalities traits about yourself. [step] Think about at least five or more personality traits that you have. Write them down, and take a few sentences to describe why you think this personality trait applies to you.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5406,"error":null,"output":"C"}
{"question_id":"hellaswag:00005217","item_index":311,"attempt":0,"prompt_hash":"5da39460dae17902169553b55674c8cccf4c8fd37c790d0137b8c374241589c5","question":"[header] How to teach kids to learn from failure [title] Redefine failure as a \" learning opportunity. [step] \" change the meaning of the word failure in your family's vocabulary. Describe such situations as learning opportunities instead.\n\nA. You could teach them that the world and your home are similar, but that they had bad days instead of lessons. [substeps] Try mixing a positive perspective with a positive one for small children.\nB. This helps you use setbacks as teachable moments. Over time, your child will stop viewing failure negatively and look for the lesson.\nC. For example, \" five times i told you something i didn't mean (and i wasn't able to do it correctly), you didn't back me up. \" this might be the case, but do not overlook failures like this.\nD. Remind your family that failure and successes aren't caused by bad behavior, but rather by lessons they're learned from. Praise young children for teaching you lessons instead of lectures or disagreements in a school setting.","prompt":"[header] How to teach kids to learn from failure [title] Redefine failure as a \" learning opportunity. [step] \" change the meaning of the word failure in your family's vocabulary. Describe such situations as learning opportunities instead.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5409,"error":null,"output":"B"}
{"question_id":"hellaswag:00006863","item_index":312,"attempt":0,"prompt_hash":"2c31e9ec13661715097ecdb147a7371575d340dbe318ec667217ff5e7c2e418b","question":"[header] How to apply foil eyeshadow [title] Gather your materials. [step] Before you begin, you should have all your materials ready. You should have all your regular makeup and eyeshadow supplies.\n\nA. You may also want to pack a concealer brush and makeup remover. Keep everything else ready to go in your purse and backpack.\nB. The prep process will take some time so be sure to start with the preparation first. [substeps] The preparation is to apply eyeliner with a dark eyeshadow brush.\nC. In addition to this, you'll need some supplies specifically used for foil eyeshadow. [substeps] You will need pigmented eyeshadow in a color of your choice.\nD. Your basics will be : [substeps] Eyeshadow palette: an ultra-bright eyeshadow brush small fluffy spoon eyeshadow applicator [title] Select the right color palette. [step] The right palette is the body of the folded eyeshadow.","prompt":"[header] How to apply foil eyeshadow [title] Gather your materials. [step] Before you begin, you should have all your materials ready. You should have all your regular makeup and eyeshadow supplies.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5408,"error":null,"output":"C"}
{"question_id":"hellaswag:00005146","item_index":313,"attempt":0,"prompt_hash":"69a82fe593fbc97729a715118a26b54db6e39020c1c9eaab24ee75f1762972e0","question":"[header] How to make canape of filo wrapped asparagus [title] Pre-heat your oven to 220 ° c (428 ° f) or gas mark 7. [title] Blanch the asparagus. [step] Sprinkle salt into a saucepan of boiling water.\n\nA. Stir the asparagus filo into the boiling water. [substeps] Blanching asparagus ensures that they are crunchy.\nB. Add lentils and onion and cook until they melt. [substeps] Blanching is also a good way to cook the asparagus.\nC. [substeps] Put the bowl containing the asparagus into the boiling water in the preheated oven. While it boils, simmer the asparagus.\nD. Add the asparagus spears and boil for about 30 seconds (or until the water boils again). Use the slotted spoon to remove the spears from the water.","prompt":"[header] How to make canape of filo wrapped asparagus [title] Pre-heat your oven to 220 ° c (428 ° f) or gas mark 7. [title] Blanch the asparagus. [step] Sprinkle salt into a saucepan of boiling water.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5408,"error":null,"output":"D"}
{"question_id":"hellaswag:00007844","item_index":314,"attempt":0,"prompt_hash":"f4dd972959f6fb7786b564a635994c370530d2110bb9858b2a4b5d585e4bb95a","question":"[header] How to bid on government contracts [title] Search the federal business opportunities website (fbo). [step] Go to. Federal agencies use the site to list their contracting opportunities.\n\nA. If you have an internet browser but have an internet connection enabled then you can perform an internet search for \" government contracts. \" [title] Search the federal trade commission (fbo).\nB. All federal solicitations worth at least $25 , 000 are published on this website. [substeps] Be careful of private websites that charge money to access their listings.\nC. If you are in business with another business, you can use the search engine to find specific business opportunities and book a bid. [title] Type the location of your bid into the search box at the top left corner of the web page.\nD. You can also find the latest information by clicking on. [substeps] The fbo app is available for windows xp.","prompt":"[header] How to bid on government contracts [title] Search the federal business opportunities website (fbo). [step] Go to. Federal agencies use the site to list their contracting opportunities.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5408,"error":null,"output":"B"}
{"question_id":"hellaswag:00001953","item_index":315,"attempt":0,"prompt_hash":"c2b5fcb47c26d503fde54d9a1b4bdbe14c08172c2e4094ddeb2061df64d2f746","question":"A man sits on the sidewalk pouring water on his hands and washes his face. the camera\n\nA. is distracted by words that the man is using on his face.\nB. pans in to a man wearing welding equipment and shows the man's face.\nC. moves and we see other men sitting and standing on the sidewalk near their merchandise.\nD. shows him backpacking in dirt and splashing water all over a mountain.","prompt":"A man sits on the sidewalk pouring water on his hands and washes his face. the camera","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5473,"error":null,"output":"C"}
{"question_id":"hellaswag:00008870","item_index":316,"attempt":0,"prompt_hash":"dd22d3274adb1b0f190e2646c6715a063382836db983c06c407508a903848b73","question":"[header] How to get over a cheating boyfriend [title] Own your pain. [step] Being in denial about what you're feeling will only draw out the healing process. Betrayal is painful, so give yourself permission to mourn however you need to.\n\nA. It's ok to grieve, but simply do not let your hurt come out and make it tough. Doing so will only sap your joy from your relationship and only more sadness will come.\nB. Unfortunately, acknowledging how you are feeling can be the only way to begin healing. It is impossible to suppress your feelings in the moment.\nC. Holding yourself to absolution will only make what happened worse. [substeps] Changing your habit of ignoring painful things is one thing, but suffering through a cheating partner can do damage to you emotionally.\nD. [substeps] Curl up in bed for a day or so. Cry your eyes out until you can't cry anymore.","prompt":"[header] How to get over a cheating boyfriend [title] Own your pain. [step] Being in denial about what you're feeling will only draw out the healing process. Betrayal is painful, so give yourself permission to mourn however you need to.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5472,"error":null,"output":"D"}
{"question_id":"hellaswag:00001565","item_index":317,"attempt":0,"prompt_hash":"9b2971bf66333844fb7cc017c30383ec1eabed974f78a0b2636501937bd4dc23","question":"The camera angles down to just the two women's hips with the hula hoops as in the beginning and back up to mostly just seeing michele's face. michele\n\nA. then gets the hula hoops and continuously tries to show all the different color hoops as in her daily routine, she keeps trying to demonstrate the numerous colored hoops that she uses in her routines.\nB. shows her left arm and hula hoops and demonstrates the use by her left hand.\nC. now runs and jumps into a jumprope where she fails to successfully jump in the jump rope twice, both times looking embarrassed.\nD. is now animated and jumping up and spinning around while waving and throwing her legs in the air.","prompt":"The camera angles down to just the two women's hips with the hula hoops as in the beginning and back up to mostly just seeing michele's face. michele","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5472,"error":null,"output":"D"}
{"question_id":"hellaswag:00003952","item_index":318,"attempt":0,"prompt_hash":"76de7adca74e7aee8d7ffde541a21684b83a7662d91c7926d65603becd60b7ad","question":"[header] How to get thick hair [title] Avoid chemical based shampoo, conditioner and styling products. [step] Most hair products contain chemicals that are effective at cleaning your hair but may cause much damage. Some people aren't affected by these chemicals, but if you're worried about thinning hair, it's best to avoid them.\n\nA. Look for the ingredients listed below or refrain from adding an ingredient. Unsaturated is easier to digest and more even as you become thicker, while unsaturated will produce a longer drying time.\nB. Adding too much styling products while your hair is already thick will only result in too much damage. Consider the following products : [substeps] Blow dryers.\nC. [substeps] If you can't remember when you last used a shampoo and conditioner, make sure to ask a friend or family member for advice. While whatever type of shampoo and conditioner you use doesn't have to be bad, read some good tips.\nD. [substeps] Sodium laurel and laureth sulfates are cleaning agents found in most shampoos. They strip the hair of natural oils and cause hair to weaken and fall out over time.","prompt":"[header] How to get thick hair [title] Avoid chemical based shampoo, conditioner and styling products. [step] Most hair products contain chemicals that are effective at cleaning your hair but may cause much damage. Some people aren't affected by these chemicals, but if you're worried about thinning hair, it's best to avoid them.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5471,"error":null,"output":"D"}
{"question_id":"hellaswag:00004194","item_index":319,"attempt":0,"prompt_hash":"2366a7435e65a840b84873ec9ec267fc2c7134bc36bc2537a3deca409334a6f9","question":"[header] How to clean your back [title] Take warm showers instead of hot showers. [step] Hot showers can dry out your skin and strip it of its natural oils. Taking warm showers will help you keep your back moisturized.\n\nA. [title] Put body wash on a loofah and scrub your upper back with it. [step] Hold the loofah in your left hand and reach over your right shoulder to scrub the upper right section of your back.\nB. To keep it moisturized, keep it covered at all times when you shower or bathe. [substeps] As such, you may want to use an extra protective covering over your back during days of exercise or hot weather, such as a dress or sweater.\nC. [title] Use at least lukewarm water. [step] Lukewarm water also removes oils, dead skin cells, and debris from your spine, leaving it supple and fresh.\nD. As a bonus, hot water can also help prevent acne, a dermatologist should prescribe lukewarm water that's at least 70 ° f (10 ° c). Warm showers should be just about one part of your regular routine.","prompt":"[header] How to clean your back [title] Take warm showers instead of hot showers. [step] Hot showers can dry out your skin and strip it of its natural oils. Taking warm showers will help you keep your back moisturized.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5471,"error":null,"output":"A"}
{"question_id":"hellaswag:00002802","item_index":320,"attempt":0,"prompt_hash":"e7b0583da9240f710de5034a5c2c955962b8cc165fa6909c6ccdd406459fa156","question":"The man holds the ball against his neck and twists his back and throws it. The man grabs the ball and takes a big step behind him and throws the ball. the man\n\nA. watch the ball and discuss back and fourth.\nB. throws the ball between his legs.\nC. twists his body a few times as he holds the ball against his neck.\nD. almost knocks the woman over.","prompt":"The man holds the ball against his neck and twists his back and throws it. The man grabs the ball and takes a big step behind him and throws the ball. the man","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6135,"error":null,"output":"B"}
{"question_id":"hellaswag:00009132","item_index":321,"attempt":0,"prompt_hash":"8bc690293cb562a80effb112efdeb5f55f28fe7527bb7030829e2b62847c636b","question":"[header] How to buy a treadmill for your home [title] Keep in mind your financial limitations. [title] Think about which use you will give to the treadmill. [step] For example, if you use the treadmill just for walking, a basic model would be suitable for your needs.\n\nA. If you use the treadmill a lot, you should go for a high durability treadmill. [title] Learn a treadmill's controller.\nB. A more complex model is better for activities that require only walking. The less complex the model, the more gear you will need to share with your neighbors.\nC. If you think you may someday only use it for sports, a summer treadmill is a great choice. [title] Get the right type of pedal length.\nD. They can help you with push-ups, squats and more should be fine too. [substeps] Decide what type of model you will be using as well as how much you will charge.","prompt":"[header] How to buy a treadmill for your home [title] Keep in mind your financial limitations. [title] Think about which use you will give to the treadmill. [step] For example, if you use the treadmill just for walking, a basic model would be suitable for your needs.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6117,"error":null,"output":"A"}
{"question_id":"hellaswag:00008970","item_index":322,"attempt":0,"prompt_hash":"b5947959d0857710b0cdf5188e9b819a96ec396533df533c35d94750e4d4d5c0","question":"[header] How to make a wedding ribbon wand [title] Calculate your budget for the wand. [step] How much money are you willing to spend on a wand per guest? This will help you to choose your materials wisely. [title] Choose the size of the rod.\n\nA. [step] For a wand that has a heavy base, it will cost you plenty of money to make it. For a wand that has smaller base, you may need just one wand, or you may require more to complete a marriage.\nB. [step] Choosing the length of the rod will affect everything from dowel size to ribbon length. [substeps] Hardware stores carry unfinished wood dowels in 36 \" or 48 \" lengths.\nC. [step] It can be a wide blade of wood or it can be a one-piece cloth with a slot into the front that allows you to fold it and use it as a dowel rod. [title] Pick a cut size that fits your décor.\nD. [step] If you're creating some artistic flair, but only for the birthday party, make sure the rod size is reasonable. You can easily make a wand at home with enough room to do other household household chores, such as washing the dishes, sweeping the floors, and mowing the lawn.","prompt":"[header] How to make a wedding ribbon wand [title] Calculate your budget for the wand. [step] How much money are you willing to spend on a wand per guest? This will help you to choose your materials wisely. [title] Choose the size of the rod.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6118,"error":null,"output":"B"}
{"question_id":"hellaswag:00007219","item_index":323,"attempt":0,"prompt_hash":"ce08190d9fc9a452a065329ba208226078d371741ec283eeba8d20a923feaa9f","question":"[header] How to rag paint a wall [title] Prepare the room where you will be painting by removing all furniture (if possible) and other obstacles. [step] It's important in the process of knowing how to rag paint a wall to have a clear work area. [title] Cover the floor and any large furniture (like couches) that cannot be removed from the room with plastic drop cloths.\n\nA. [title] Start by lining up your work area. [step] Find a solid, level surface like wood, concrete, or asphalt or underneath some tarp.\nB. [step] These are the spots you are painting that would likely not be visible on a regular wall of your studio, or a family room. [title] Place other drop cloths over large furniture (like chairs).\nC. [title] Mark the area you'll be painting on with painter's tape. [step] This will help you to mark any curved edges later on on.\nD. [title] Tape edges of the areas you do not wish to be painted to protect the areas from paint splatters and brush edges. [title] Pour paint for the base color into a paint tray.","prompt":"[header] How to rag paint a wall [title] Prepare the room where you will be painting by removing all furniture (if possible) and other obstacles. [step] It's important in the process of knowing how to rag paint a wall to have a clear work area. [title] Cover the floor and any large furniture (like couches) that cannot be removed from the room with plastic drop cloths.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6118,"error":null,"output":"D"}
{"question_id":"hellaswag:00005676","item_index":324,"attempt":0,"prompt_hash":"a64090a0f86aa112820d70d4be1eff8e6f52dbf05a2d53863be07e06df17d19d","question":"[header] How to put a twitter badge on a website [title] Open 2 internet browser tabs on your computer. [step] In the first tab, login to your website's admin area. If you do not know html code, or you do not have access to the website's admin area, seek the help of your website's web developer.\n\nA. [title] Check for a link to an existing website. [step] The link will be located in the banner or text field on your admin area.\nB. [title] Type \" twitter \" in the top right corner of your web browser. [step] While this is probably not the best line of defense, it is still a good option for the website's layout.\nC. [title] Access twitter's admin page. [step] At the top part of the page are three vertical links.\nD. [title] Go to twitter.com in the second tab. [step] Sign into the twitter account that you want to post to your website.","prompt":"[header] How to put a twitter badge on a website [title] Open 2 internet browser tabs on your computer. [step] In the first tab, login to your website's admin area. If you do not know html code, or you do not have access to the website's admin area, seek the help of your website's web developer.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6118,"error":null,"output":"D"}
{"question_id":"hellaswag:00003265","item_index":325,"attempt":0,"prompt_hash":"b6dfb3fc6dd085cd6f315328db95e9cc6ec799d1193683c7dc828cb2b345140e","question":"[header] How to find excellent day care [title] Begin as soon as possible. [step] Many child care centers have long waiting lists, so it is never too early to start looking. If you are pregnant, start looking for child care for after your baby is born.\n\nA. If you are married, start looking right away. Find an attendant or friend at your local hospital to help you get the paperwork started early.\nB. [title] Determine whether you are eligible for child care subsidies. [step] Often there is a waiting period for these funds, and they are only accepted by certain child care providers.\nC. You should also look into outpatient child care after labor, as you may save a lot of money to simplify your baby's routine as well as make care more affordable. [substeps] If you decide to try outpatient, you should consider pursuing a lifestyle with a plan of life to meet your baby's needs.\nD. While this may be ideal, you need to start looking around only when you are currently in labor. If you are planning to go maternity, start looking right now.","prompt":"[header] How to find excellent day care [title] Begin as soon as possible. [step] Many child care centers have long waiting lists, so it is never too early to start looking. If you are pregnant, start looking for child care for after your baby is born.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6118,"error":null,"output":"B"}
{"question_id":"hellaswag:00000533","item_index":326,"attempt":0,"prompt_hash":"4c4ff47e231049c062b4dc632b6d44ad4e225513a0716cfc6d848567ba6ebde0","question":"An introduction comes onto the screen for a video about skate boarding tricks. Several tricks are shown while someone narrates the tricks. a man\n\nA. is shown standing on a slack line as details of his trick are shown.\nB. talks while playing a front row home on his skateboard as even more tricks are shown and narrates.\nC. is shown on the screen giving details about the tricks and offering pointers and tips.\nD. walks along a track with a helmet on.","prompt":"An introduction comes onto the screen for a video about skate boarding tricks. Several tricks are shown while someone narrates the tricks. a man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6124,"error":null,"output":"C"}
{"question_id":"hellaswag:00008974","item_index":327,"attempt":0,"prompt_hash":"4722083984f51e3fb1696aa382091e43eb096f99e11090406cefff0dbf611795","question":"[header] How to iron on a patch [title] Figure out what type of patch you have. [step] Some patches come with glue on the back, and others simply have a cloth backing. Take a close look at your patch and decide whether you may need additional materials.\n\nA. [substeps] If your patch's backing is rotten, use a glue stick to remove the patch. If you've got an iron, use it to remove gunk that can build up.\nB. [title] Try to get a patch with a slightly bigger border. [step] Some stains go away over time, so this method is especially useful if your patch is intended for a horizontal application.\nC. [substeps] Patches sold with glue on them can be packed and cropped. Depending on the size of your patch, you may want to use about three sheets of 1/4-inch (0.3 cm) fabric, or just three standard strips.\nD. [substeps] Decorative embroidered fabric patches are usually thick, stiff, and have what looks like plastic glue on one side. These can be used to cover ripped or discolored fabric.","prompt":"[header] How to iron on a patch [title] Figure out what type of patch you have. [step] Some patches come with glue on the back, and others simply have a cloth backing. Take a close look at your patch and decide whether you may need additional materials.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6112,"error":null,"output":"D"}
{"question_id":"hellaswag:00006934","item_index":328,"attempt":0,"prompt_hash":"78ac8a4995d51d156564ac54f620e283f2ae894ddb6b669432883d6bf0b3a217","question":"[header] How to evaluate your depression management plan [title] Consider if your goals are being met. [step] Think about what brought you to confront your depression in the first place and what you had hoped to gain from managing your depression. If you see a therapist, you may have created goals together early in treatment.\n\nA. If it has not been awhile since you made contact with an agency or someone who has taken steps to help you, the treatment may be a phase. Look for areas where you could improve : [substeps] You've been able to identify your strengths and highlight them.\nB. If you feel better earlier than you initially, you're more likely to move on. [substeps] For instance, are you being motivated enough to work harder and to exercise more than you normally would? [title] Celebrate and respect your dreams.\nC. Go over those goals and see how you are doing. Ask yourself what you want to improve and how you will improve it.\nD. Don't assume your goals were met because of your relapse, but rather because you didn't get professional help. [substeps] Being able to identify your goals out loud or in writing can help you keep your head on straight and realize they are worthy of reflecting your perceptions.","prompt":"[header] How to evaluate your depression management plan [title] Consider if your goals are being met. [step] Think about what brought you to confront your depression in the first place and what you had hoped to gain from managing your depression. If you see a therapist, you may have created goals together early in treatment.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6121,"error":null,"output":"C"}
{"question_id":"hellaswag:00008671","item_index":329,"attempt":0,"prompt_hash":"6c5596ccfcfb087bafb100bb5cd5f2e21639e8b26a3be5bc9bfcb1fea66490f1","question":"[header] How to manage cognitive dysfunction in senior dogs [title] Give your dog medication. [step] There is one medication that veterinarians use to treat cognitive dysfunction. This medication is called deprenyl (anipryl), and it has been shown to help with symptoms and increase brain concentration in dogs.\n\nA. [substeps] Deprenyl includes anti-anxiety drugs. You can take two to three simple doses daily and your dog will start feeling less anxious and irritable within one to two weeks of having any medication taken.\nB. [substeps] However, it does not work with all dogs. Some dogs will see no to minimal improvement.\nC. Most dogs will be able to take this medication for 60 minutes at a time without worsening symptoms. [substeps] Deprenyl is very difficult to manage for an adult dog.\nD. Your dog can use this medication alone, or with its friends and family. [substeps] Probenbenectate (napec) is another medication used to treat cognitive dysfunction but it does not work for older dogs.","prompt":"[header] How to manage cognitive dysfunction in senior dogs [title] Give your dog medication. [step] There is one medication that veterinarians use to treat cognitive dysfunction. This medication is called deprenyl (anipryl), and it has been shown to help with symptoms and increase brain concentration in dogs.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6121,"error":null,"output":"B"}
{"question_id":"hellaswag:00001618","item_index":330,"attempt":0,"prompt_hash":"665d9ef15360085ea9d44363333e76e46c9db2f1057e718b3876c71eeadc65fe","question":"A man is outside smoking a cigarette with his friend. they\n\nA. drive down a lane at the beach looking at the sand.\nB. both begin dancing together in front of a crowd.\nC. are just sitting out relaxing blowing smoke.\nD. are playing bagpipes together.","prompt":"A man is outside smoking a cigarette with his friend. they","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6123,"error":null,"output":"B"}
{"question_id":"hellaswag:00005306","item_index":331,"attempt":0,"prompt_hash":"f7857e536725818e6bd33b91ff6255db865e2ef068e9080f3cc3a0b0a264c4c0","question":"[header] How to say i love you in swahili [title] Say \" ninakupenda \" or \" nakupenda. [step] \" these are the two ways you can say \" i love you \" in swahili. \" ni-\" means \" i, \" and the single word represents the entire sentence in swahili.\n\nA. This is similar to saying: \" the past is not dead, \" or, \" the future is on its way. \" [substeps] Try pronouncing the phrase almost as you would a spanish pidgin.\nB. If both two people say the same phrase the same: \" is that's her? \" right away you are getting frustrated. [title] Try to understand \" beautiful \" terms for \" cute.\nC. After the phrase you say it, you'll find that your girlfriend thinks it's \" likely \". Examples: \" my friend chuck will drive me crazy, but he believes that i am the coolest, the most creative, and express in my smsoaking, etc.\nD. \"-kupenda \" means love in swahili. [substeps] Colloquial swahili speakers will drop the initial prefix \" ni-\" and simply say \" nakupenda.","prompt":"[header] How to say i love you in swahili [title] Say \" ninakupenda \" or \" nakupenda. [step] \" these are the two ways you can say \" i love you \" in swahili. \" ni-\" means \" i, \" and the single word represents the entire sentence in swahili.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6117,"error":null,"output":"D"}
{"question_id":"hellaswag:00002438","item_index":332,"attempt":0,"prompt_hash":"c48a2b611703b0292546d5f8a2ff0b1099bebc94f769c8a8637d313ba43fde97","question":"People are running down a street. Cars are driving on the street behind them. a woman in shorts\n\nA. is running down the street.\nB. is in the street and puts her arms up.\nC. runs around on the street.\nD. is setting up a video in a gym.","prompt":"People are running down a street. Cars are driving on the street behind them. a woman in shorts","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6119,"error":null,"output":"A"}
{"question_id":"hellaswag:00000714","item_index":333,"attempt":0,"prompt_hash":"6c9b26cc47f440e5a165b0b6a0afdf8e056668fc5b570ee80c4f05276b9962cd","question":"A man is running down the street wearing stilts. he\n\nA. turns around and runs the other direction.\nB. stops in front of a house and picks up a large piece of wood.\nC. takes off on a skateboard in a fast manner.\nD. takes a huge leap through the air.","prompt":"A man is running down the street wearing stilts. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6121,"error":null,"output":"A"}
{"question_id":"hellaswag:00008221","item_index":334,"attempt":0,"prompt_hash":"b374d0f28762b44b30d01d08637f3c9ef54f8b26f1086bd2974f0b8440bd970f","question":"[header] How to attain effortless beauty [title] Don't sweat it. [step] Find a physical activity you really enjoy. It does not mean you need to run, do cardio, or do sit-ups.\n\nA. This can mean a yoga class, purchase a dvd for a dance routine, or even go bike riding. Sweat is a good thing, working out not only makes you look better, but it also helps you feel better.\nB. Just find some exercise you enjoy doing regularly. [substeps] The opposite is often the case with beauty.\nC. In fact, being active will do a lot of the work for you and it will make you feel better physically. [substeps] Try going for a nature walk with your dog or running to catch up with a friend.\nD. So instead, just stay comfortable and try to do the things you love. Just being yourself and doing it so you are not obsessing over the results in bed or worrying about your looks in the morning.","prompt":"[header] How to attain effortless beauty [title] Don't sweat it. [step] Find a physical activity you really enjoy. It does not mean you need to run, do cardio, or do sit-ups.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6121,"error":null,"output":"C"}
{"question_id":"hellaswag:00005405","item_index":335,"attempt":0,"prompt_hash":"b1dcac8cd50ddfbb2d06cdb45f0dae1adda12d9cb88ee11e1d44a8d2c7e56a7f","question":"[header] How to increase your fertility [title] Take care during the preconception period to prepare your body for conception and pregnancy. [step] Fertility is obviously essential when getting pregnant, so let's discuss some simple ways to improve your fertility foundation. [title] Determine your ovulation date.\n\nA. [step] This information can help you know when you get pregnant and are able to begin treatment. If you are healthy and have not had a day of menstruation in a while, discuss going on a first date with your doctor to ensure that your body can continue to produce estrogen and female hormones.\nB. [step] The best ways to find out when (and if) you ovulate are basal body temperature charting, using ovulation predictor kits, and observing cervical mucus changes. Using more than one method gives you a more accurate determination of your ovulation date.\nC. [step] Generally, ovulation dates are long. Ovulation is often found at the beginning or end of a woman's menstrual cycle.\nD. [step] Women can get pregnant during birth control but the egg contains estrogen. Ovulation is a common issue with women during fertility.","prompt":"[header] How to increase your fertility [title] Take care during the preconception period to prepare your body for conception and pregnancy. [step] Fertility is obviously essential when getting pregnant, so let's discuss some simple ways to improve your fertility foundation. [title] Determine your ovulation date.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6120,"error":null,"output":"B"}
{"question_id":"hellaswag:00005337","item_index":336,"attempt":0,"prompt_hash":"8dc938edc3e6dc4b0fdc22e5dcd2ef9a4e6f9997421f7fba64f30f3405b6075a","question":"[header] How to give yourself a quick manicure [title] Give yourself a manicure in a well-ventilated room. [step] Nail polishes and nail polish removers contain many chemicals that give out nasty odours, as well as toxins/vapors that are dangerous to breathe in. Once you find your area, spread some newspapers or old magazines just in case of spills.\n\nA. Wear some rubber or vinyl gloves when handling/cleaning your nails. [title] Fill a spray bottle with nail polish remover.\nB. [title] Combine the base base layer with some polish remover. [step] The base layer is the area that consists of the cuticles and are the lining of your nails.\nC. [title] Soak your nails/hands in a bowl of warm water. [step] This is done to soften the nails and cuticles.\nD. [substeps] Keeping these places well kept may help you avoid any mistakes (let alone accidents) on your hands or face. [title] Minimize the frequency of polish spills.","prompt":"[header] How to give yourself a quick manicure [title] Give yourself a manicure in a well-ventilated room. [step] Nail polishes and nail polish removers contain many chemicals that give out nasty odours, as well as toxins/vapors that are dangerous to breathe in. Once you find your area, spread some newspapers or old magazines just in case of spills.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6120,"error":null,"output":"C"}
{"question_id":"hellaswag:00009605","item_index":337,"attempt":0,"prompt_hash":"7ffc5b235f7f7fd3fa9e9382f350543918950bae877f4a5f3569b3d05d46b6db","question":"[header] How to choose a perfume [title] Decide on a price point. [step] Perfume is a luxury item and its cost can vary hugely between products. Some perfumes are under 20 dollars while others cost hundreds or even thousands of dollars.\n\nA. It is best to stick with one brand and limit your choices in a variety of different brands or styles. [substeps] The typical sized perfume pair is $5, but you can buy larger, less ornate ones as well.\nB. Before you go shopping, decide how much money you want to spend on a perfume. [substeps] If the perfume is a gift, consider both your personal finances and your relationship with the recipient of the gift.\nC. Before making a decision, make sure you are sure that your choice of an item would actually cost you hundreds or thousands of dollars. [substeps] You can measure the quality of the bottle or your fragrance.\nD. Keep this in mind when deciding where to shop for great smelling perfumes. [substeps] Many perfume stores and online retailers offer personalized packages or deals for fragrances.","prompt":"[header] How to choose a perfume [title] Decide on a price point. [step] Perfume is a luxury item and its cost can vary hugely between products. Some perfumes are under 20 dollars while others cost hundreds or even thousands of dollars.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6119,"error":null,"output":"B"}
{"question_id":"hellaswag:00005835","item_index":338,"attempt":0,"prompt_hash":"0da0c7b9c6488db739dc84b68b7869bfffdc6e5c1dfd059ab5db7fcc59f2be52","question":"[header] How to have fun with your kids on a budget [title] Give everyone a job * they will enjoy so no one feels like they are working. [substeps] Younger kids can mix cold dishes, salads and batters. Older kids can help you prepare new cooked recipes and peel and slice vegetables.\n\nA. They can bake you desserts and wash your dishes. Younger kids can arrange up-to-do chores for their peers, such as when they cook together, and you can have them jump-rope the whole time.\nB. [title] You should have fun and be as fun as possible while really enjoying your child. [title] Find a job at a best furniture store such as walmart, that's great and them have full clients.\nC. [title] Do activities involving sweets (like baking cookies) are always hits. [title] Let your child pick out a new fresh fruit or vegetable at the grocery store once a week.\nD. [title] Get a small lawn pass and give the kids access to all parts of the lawn. [title] Ask your parents to introduce the kids into each other's yards.","prompt":"[header] How to have fun with your kids on a budget [title] Give everyone a job * they will enjoy so no one feels like they are working. [substeps] Younger kids can mix cold dishes, salads and batters. Older kids can help you prepare new cooked recipes and peel and slice vegetables.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6118,"error":null,"output":"C"}
{"question_id":"hellaswag:00000153","item_index":339,"attempt":0,"prompt_hash":"ac980d3ed79acf01908c3a528044d0d7e08072cccf49415cb9885a0e947e8efd","question":"A group of people are sitting in a sauna fully clothed. One guy stands up with a violin. The guy begins playing the violin and dancing. the guy\n\nA. kicks his shoe off while dancing.\nB. demonstrates how to cut the strings with the violin.\nC. plays behind a camera.\nD. finishes and gets up.","prompt":"A group of people are sitting in a sauna fully clothed. One guy stands up with a violin. The guy begins playing the violin and dancing. the guy","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6119,"error":null,"output":"B"}
{"question_id":"hellaswag:00009537","item_index":340,"attempt":0,"prompt_hash":"593d949720f65510dfdce8bffee41c78380a1dc706922deb0b36c7891a41df4e","question":"[header] How to retrieve a telstra puk code [title] Call the telstra service line on 125 111 (a flat 25c call charge ) [title] You'll hear this message. [step] \" welcome to telstra. So i can direct your call to the right place, in just a few words, please tell me the reason for your call \" [title] Say \" puk code \".\n\nA. [step] \" type \" in a line line that has a free code function. [title] After you use the code, you'll soon hear code ringtone.\nB. [title] When prompted, wait until s/he has answered the call. [title] State the code in the provided box, by example 1/159.\nC. [step] Meaning \" dial to the right of \" from here, \" to indicate where you are calling from and who your calling from. [title] Wish to call via the voice mail app.\nD. [step] You'll need to say it super clearly into the microphone. [title] It will ask you to say or enter your phone number.","prompt":"[header] How to retrieve a telstra puk code [title] Call the telstra service line on 125 111 (a flat 25c call charge ) [title] You'll hear this message. [step] \" welcome to telstra. So i can direct your call to the right place, in just a few words, please tell me the reason for your call \" [title] Say \" puk code \".","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6120,"error":null,"output":"D"}
{"question_id":"hellaswag:00003225","item_index":341,"attempt":0,"prompt_hash":"edc50373aeb73d3bc8aaf11584cfe01edd32079f640172c3e30a175fc1f7ae16","question":"People are gathered as two men arm wrestle. the competitors\n\nA. go down on one knee to wrestle.\nB. are shown running around, quickly beating each other.\nC. are shown, and the start rowing.\nD. make all kind of faces as they grimace.","prompt":"People are gathered as two men arm wrestle. the competitors","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6118,"error":null,"output":"A"}
{"question_id":"hellaswag:00007921","item_index":342,"attempt":0,"prompt_hash":"1a548d9e89e77c92fd3fbe01a75f1126252f0320e357060746ab9086dfa0293a","question":"[header] How to drive someone in labor to the hospital [title] Plan the route to the hospital. [step] Test drive different ways to get to the hospital. This way you'll know which way is the fastest, the best way to go if there's heavy traffic, or alternate ways if there is construction.\n\nA. Be aware that traffic will be different depending on the time of day. You can pick the best route when the woman goes into labor.\nB. Include time in your route exactly as you scheduled. Try driving at least half an hour each way to get there if you weren't in labor before you arrived at the hospital.\nC. [substeps] If the person is in labor, be sure to have a quiet and relaxed environment. Instead of making noises or driving the same route all day, sit in silence.\nD. Practice your route before you leave to ensure you won't run into traffic. [substeps] Pass the time by watching the sunrise, which should cue you into good traffic patterns.","prompt":"[header] How to drive someone in labor to the hospital [title] Plan the route to the hospital. [step] Test drive different ways to get to the hospital. This way you'll know which way is the fastest, the best way to go if there's heavy traffic, or alternate ways if there is construction.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6744,"error":null,"output":"A"}
{"question_id":"hellaswag:00009632","item_index":343,"attempt":0,"prompt_hash":"c290e8084b19f09f52ced6a3bdd200db55c88ed5519654ef6ff2c08f6547f4f7","question":"[header] How to make your own crystals [title] Combine the ingredients in the container. [step] Add about a half cup (120ml) of epsom salt (magnesium sulfide) with equal parts hot water. [substeps] This will create a saturated solution.\n\nA. The less hot your solution is, the better. Keep in mind that epsom salt floats and can discolor.\nB. You can add later until you've applied more of the entire solution. Cast iron is the cheapest of all items on the periodic table.\nC. It may also improve the ph of the water. Add two tablespoons (29.5 ml) of ammonia (tsp) and vinegar (xhella vevalva) for an acidic solution.\nD. A solution is considered saturated when no more solids can be dissolved into the liquid. Add a few drops of food coloring to the mixture if you want your crystals to have some color.","prompt":"[header] How to make your own crystals [title] Combine the ingredients in the container. [step] Add about a half cup (120ml) of epsom salt (magnesium sulfide) with equal parts hot water. [substeps] This will create a saturated solution.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6741,"error":null,"output":"D"}
{"question_id":"hellaswag:00001233","item_index":344,"attempt":0,"prompt_hash":"1c0e69a761694c2b881f4558eb88b9cd82556e2631877c5fc5614f19e7a3f325","question":"The man in black cap is talking to the camera. The man opened the door and got surprised then he closed the door. a male athlete run\n\nA. the parallel bars with a barbel in his hand.\nB. running and jump field.\nC. and do a high jump over the pole.\nD. chased by a crowd.","prompt":"The man in black cap is talking to the camera. The man opened the door and got surprised then he closed the door. a male athlete run","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6742,"error":null,"output":"C"}
{"question_id":"hellaswag:00007984","item_index":345,"attempt":0,"prompt_hash":"167c5bdf9fc501d6ffefefbb1c70192137bca86d128d5321432b68d4ec85f3ec","question":"[header] How to clean a couch [title] Determine the type of fabric your couch is made out of. [step] Sofas can be made of anything from cotton to leather, and knowing exactly what fabric you're working with is important for knowing how to clean it. Check the tag on the bottom of your sofa to determine the type of cloth.\n\nA. [substeps] For some commercial sofas, you'll need to consult with a presales person or the same person who made them. For one, there may be rough patches of upholstery on the back and side of the sofa.\nB. [substeps] Fabric with lug nuts usually is not the material you'll want to use. If you have no idea, you can check the tag on a cheap stainless steel couch you don't know if it's made of leather or woven.\nC. The tag should have detailing on it indicating how old the material is, and bring it out to the vet if you're worried about your sofa's current style of covering. [title] Store vinyl and other materials in a clean bag or zip-close bag.\nD. The tag should give a label of \" w, \" \" s, \" \" ws, \" \" x, \" or \" o. \" [substeps] A tag with the letter \" w \" or \" ws \" means that your couch is washable with a water-based soap.","prompt":"[header] How to clean a couch [title] Determine the type of fabric your couch is made out of. [step] Sofas can be made of anything from cotton to leather, and knowing exactly what fabric you're working with is important for knowing how to clean it. Check the tag on the bottom of your sofa to determine the type of cloth.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6743,"error":null,"output":"D"}
{"question_id":"hellaswag:00005816","item_index":346,"attempt":0,"prompt_hash":"22eae792d7cef4e64178219112d9f96d5cdac1b9d91d36858417ca8bc59bda51","question":"[header] How to chat on badoo [title] Open google play store on your android phone. [step] Just click on the google play store icon found in your device's home screen, or in its app drawer. [title] Search for badoo.\n\nA. [step] Type in \" badoo \" into the search field found in the app drawer, or tap the magnifying glass at the bottom of the app to begin. Hit the magnifying glass again to begin your search.\nB. [step] On the google play store screen, the search option will be on the top right corner. Tap on the magnifying-glass icon, and then type \" badoo \" into the search bar.\nC. [step] Tap on the magnifying glass icon in the upper left corner of the screen and enter \" google play. \" the app will expand to display multiple categories.\nD. [step] Tap the magnifying glass icon at the upper-right corner of the play store icon. This will provide a list of all available phone numbers, access to all of the available apps in the google play store, and the names of the available available contacts.","prompt":"[header] How to chat on badoo [title] Open google play store on your android phone. [step] Just click on the google play store icon found in your device's home screen, or in its app drawer. [title] Search for badoo.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6741,"error":null,"output":"B"}
{"question_id":"hellaswag:00008203","item_index":347,"attempt":0,"prompt_hash":"1e937b027d3e86d3d187429cc1e1c7d890db227c51a1cab123fb2510cb11a893","question":"[header] How to train a cocker spaniel [title] Select a crate for your cocker spaniel. [step] An important aspect of training your cocker spaniel is crate training. When done properly, your dog will see his crate as a place of refuge and relaxation, rather than a place of punishment.\n\nA. Selecting a crate for your cocker spaniel will give you room to explore your dog's new home before, during, and after you begin training him. [substeps] Crates need to be large enough for your dog to stand up, lie down, and move around comfortably in.\nB. Crates, which are available at your local pet store, come in different sizes and materials, such as plastic, fabric, and metal. [substeps] If your dog is a puppy, consider renting a crate from your local animal shelter, since he will eventually outgrow it.\nC. You can select a crate that is big enough for your dog to comfortably stay in, but not deep enough for him to physically dig, enter or escape. [substeps] The crate should have a cylinder, padded lid and a secure bottom frame.\nD. [substeps] Crates are dog-friendly, providing your dog with a safe and secured space from outside stimuli that will diminish his fear and anxiety. Crate training is a positive way to bond with your dog as a new pet.","prompt":"[header] How to train a cocker spaniel [title] Select a crate for your cocker spaniel. [step] An important aspect of training your cocker spaniel is crate training. When done properly, your dog will see his crate as a place of refuge and relaxation, rather than a place of punishment.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6703,"error":null,"output":"B"}
{"question_id":"hellaswag:00006527","item_index":348,"attempt":0,"prompt_hash":"05e81d9e05136ee6d83467110abaf53dd83bc33941721b1b3ae96e6587f90cc0","question":"[header] How to get rid of tattoo scarring and blowouts [title] Add background shading to the tattoo. [step] Ask an experienced tattoo artist to add some additional shading to the tattoo in order to cover up the scarring or blowout. Typically, a blowout is visible on the outermost edge of a tattoo.\n\nA. [substeps] Give the tattoo an outline before getting rid of the first layer of shading. Over time, your tattoo will become stronger and more defined.\nB. To cover this up, you can expand on or add to the tattoo. Alternatively, you could add some background shading to cover the blowout.\nC. The sharp edges of good shading will make your tattoo look more striking. [substeps] You can also add a second, finer shading to overlap the existing artist's shading.\nD. However, you may decide to add some other shading to cover up the tattoo. [substeps] Refine your tattoo if you decide you want to go the extra mile.","prompt":"[header] How to get rid of tattoo scarring and blowouts [title] Add background shading to the tattoo. [step] Ask an experienced tattoo artist to add some additional shading to the tattoo in order to cover up the scarring or blowout. Typically, a blowout is visible on the outermost edge of a tattoo.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6704,"error":null,"output":"B"}
{"question_id":"hellaswag:00005166","item_index":349,"attempt":0,"prompt_hash":"75e296017316d2a28fe6c96d9996a9d3dc3b8a7f97f12d91e902f5d06bf6d07a","question":"[header] How to cope with the death of a cousin [title] Embrace your feelings. [step] Grieve. Many people wrongly assume that ignoring their feelings or \" toughing it out \" will help the pain go away more quickly.\n\nA. This is a common myth associated with grief. Do not be afraid to cry or scream or whatever you think helps you react to the loss.\nB. It is not! Feeling sad, sad, thankful, or upset all relate to past pain. Embrace your emotions and let them manifest in their own way.\nC. Forgiving someone takes a lot of work and effort. You may believe that surviving the death will protect you at least temporarily, but in reality, you are only letting it get you down.\nD. But, remember that this is only an illusion and trying not to let your own grief destroy you will only make the process of going through this difficult time more painful. [substeps] Try to sort out your feelings and deal with them instead of trying to break the news first.","prompt":"[header] How to cope with the death of a cousin [title] Embrace your feelings. [step] Grieve. Many people wrongly assume that ignoring their feelings or \" toughing it out \" will help the pain go away more quickly.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6704,"error":null,"output":"A"}
{"question_id":"hellaswag:00009788","item_index":350,"attempt":0,"prompt_hash":"2ac540b81ae51fc1ab7e61e111b345e200c3207e68b3d3dac776dfa9e05fd430","question":"[header] How to make a baby's breath crown [title] Measure the circumference of your head. [step] Using a thin floral wire, measure the circumference of your head. Wrap the wire around the top of your head where you would like to crown to sit.\n\nA. [substeps] For more precise measurements, use a ruler. [title] Cut off the shorter ends of the wire.\nB. [substeps] If you'll be making a baby's breath crown, you can make the crown as wide or as narrow as you want, but for a younger child, you'll have to measure the size of their head. The rounder your child's head, the larger your crown will need to be.\nC. Then double the amount of wire and cut it using pliers. You want to double the wire in order to ensure that you have a sturdy base for the crown.\nD. Then, place one end of the wire over the center part of your crown at 1/8 of an inch from your hairline. You should see that you have a roughly average head circumference.","prompt":"[header] How to make a baby's breath crown [title] Measure the circumference of your head. [step] Using a thin floral wire, measure the circumference of your head. Wrap the wire around the top of your head where you would like to crown to sit.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6704,"error":null,"output":"B"}
{"question_id":"hellaswag:00005465","item_index":351,"attempt":0,"prompt_hash":"f48324d784eb69d943765e1355cc7af75e1537d577bba1c62e8b6b4b1f136728","question":"[header] How to invest in rental real estate [title] Assess your ability to manage the property. [step] Being a landlord is not passive income or a part-time job. You will have many responsibilities when it comes to managing the property.\n\nA. The following common roles are : [substeps] Owner. You provide legal or financial protection in the event that the tenant dies.\nB. Paying the bills in a timely fashion will set you apart from other renters and the people who will be visiting the property. Working in a management role will set your business apart from the regular complacent tenants on leases, offer the tenants a taste of how property management options might work, and create a better view of the property's worth.\nC. You should ensure that you are able to manage the entire area of the property, even if it is one of limited value. Instead of being the sole owner, you will be responsible for paying rent to tenants.\nD. This includes selecting and communicating with tenants, repairing the property when necessary, and dealing with a large variety of day-to-day issues. Make sure that you aren't already juggling too much with work and family before deciding to become a landlord.","prompt":"[header] How to invest in rental real estate [title] Assess your ability to manage the property. [step] Being a landlord is not passive income or a part-time job. You will have many responsibilities when it comes to managing the property.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6704,"error":null,"output":"D"}
{"question_id":"hellaswag:00007421","item_index":352,"attempt":0,"prompt_hash":"028daad11a947e287a20fff856bf389ecf6228822acda23ad5661d8b42cce5ae","question":"[header] How to pluck facial hair [title] Clean your skin. [step] It is important to start with clean skin because dirt and bacteria can get into the pores where you have plucked the hair and cause breakouts. Additionally, you will probably be touching your face with your hands, so it's good to start with a clean palette.\n\nA. [title] Clean your head and the rest of your face by standing in front of a mirror and gently running your hand across your face. [step] This will help scrub any dirt or bacteria off of your face and make you clean quicker.\nB. [title] Use an antiseptic exfoliator on the skin you want to pluck. [step] Cut a clean strip of cotton and rub it on a small piece of skin that can be plucked from under your chin.\nC. [substeps] Take a cotton swab and dampen it with some water. Then, gently dip the cotton swab into your cleaning solution.\nD. [substeps] Use a mild facial cleanser and lukewarm running water. Begin by splashing a bit of water on your face so it is wet.","prompt":"[header] How to pluck facial hair [title] Clean your skin. [step] It is important to start with clean skin because dirt and bacteria can get into the pores where you have plucked the hair and cause breakouts. Additionally, you will probably be touching your face with your hands, so it's good to start with a clean palette.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4318,"error":null,"output":"D"}
{"question_id":"hellaswag:00003636","item_index":353,"attempt":0,"prompt_hash":"c52df750fcf4c38c27d1e77254044256a7306d7ede2e6475c83361edd9abe7b8","question":"[header] How to change the password to a saved wireless network on a windows vista pc [title] Open the wireless networking control panel options button. [step] Single click the internet button from the bottom right corner of your windows taskbar and click networking and sharing center. [substeps] The network and sharing center is also able to be accessed through the control panel.\n\nA. If you don't have access to this screen, go to start] properties] network-] sharing. [title] Look under network posts and links for \" network properties \" and click on \" network & sharing center \"; this will be the blue button below the network and sharing center.\nB. Click network adapter options from the options above to view the network management pane of the network and sharing center section. The network and sharing center window will appear under network and sharing center, and control panel.\nC. [title] Select the \" properties \" option from the list of network objects in the options pane. [title] Click \" manage wireless networks \" tab.\nD. Open control panel, type network in the search box at the top of that dialog box and click \" network and sharing center \". [title] Look for a bar on the left side that displays several options and click the \" manage wireless networks \" option from that bar.","prompt":"[header] How to change the password to a saved wireless network on a windows vista pc [title] Open the wireless networking control panel options button. [step] Single click the internet button from the bottom right corner of your windows taskbar and click networking and sharing center. [substeps] The network and sharing center is also able to be accessed through the control panel.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4318,"error":null,"output":"D"}
{"question_id":"hellaswag:00007084","item_index":354,"attempt":0,"prompt_hash":"0be7538aabad01f39abb129a14fd548a7b4e00f7f09ab93f0feda5b425a4f98d","question":"[header] How to confront others about their poor hygiene [title] Set a goal for the conversation. [step] Making sure that you have a particular result in mind will help you plan for the difficult conversation. Know the specific changes that you want to see in your friend or colleague's behavior and be clear about why those changes are important.\n\nA. [substeps] For example, in order to have a conversation that will lead to a happy and pleasant reunion, write the goal down on a notepad. This will keep you on track and prevent you from running out the door and rudely telling your friend how excited and well you are.\nB. [substeps] Consider aspects of the person's behavior that annoy you. If they are a supervisor and you are unsure about what they are doing right, get involved.\nC. [substeps] Think about what you hope to say to your friend or colleague, or use a standard psychology approach. For example, you might want to talk about how your friend makes a huge mess and how they need to fix it right now.\nD. [substeps] Avoid examining his or her personality; keep your goals focused on behaviors. Write your goal down so that you can keep it as a reference as you plan the conversation.","prompt":"[header] How to confront others about their poor hygiene [title] Set a goal for the conversation. [step] Making sure that you have a particular result in mind will help you plan for the difficult conversation. Know the specific changes that you want to see in your friend or colleague's behavior and be clear about why those changes are important.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4321,"error":null,"output":"D"}
{"question_id":"hellaswag:00007060","item_index":355,"attempt":0,"prompt_hash":"f865611c87a57e52ee549245156b52ea1ada8f8b74c556808e27a7c32eea2b53","question":"[header] How to clean mold from leather [title] Apply a layer of petroleum jelly. [step] Test the effect of the petroleum jelly on a small area that won't be noticeable. Apply the petroleum jelly on the moldy areas creating only a thin layer.\n\nA. [substeps] The jelly can also be rub on with a clean cloth or simply applied onto painted leather. Avoid rubbing petroleum jelly on leather, as it will also cause the paintwork to flake off.\nB. If the mold works quickly, then the petroleum jelly will remain on the entire jacket. [title] Press on mold with cotton balls to make gentle circular motions.\nC. [substeps] Avoid applying petroleum jelly straight to your leather. This may damage the fabric or cause mildew.\nD. You may also use a suede cleaner but read the label of the cleaning product to ensure that it works on mold and mildew. [substeps] Nubuck can easily change colour so make sure the cleaning product you use has been tested before applying.","prompt":"[header] How to clean mold from leather [title] Apply a layer of petroleum jelly. [step] Test the effect of the petroleum jelly on a small area that won't be noticeable. Apply the petroleum jelly on the moldy areas creating only a thin layer.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4317,"error":null,"output":"D"}
{"question_id":"hellaswag:00005327","item_index":356,"attempt":0,"prompt_hash":"7de1b8f1efacd2764adfd20688efefc9f30fc547410c177229ab13718a2359f2","question":"[header] How to remove a ceiling tile [title] Cover the floor with a protective material if desired. [step] Gypsum ceiling tiles tend to crumble around the edges, especially as they get older. If you want to keep this debris from getting onto your floors, lay down something to protect them, such as sheets of newspaper.\n\nA. Usually, it won't come down on hard surfaces; instead, it will come down along the floor near the tiles. This can protect the surface but will also prevent tiles from sticking.\nB. [substeps] If you are installing a ceiling tile over a concrete floor, make sure it is not 3 centimeters high, as it will be more difficult to get over the edge. [title] Consider using strips of kitchen aluminum sheet that are slightly larger than the ceiling tiles to prevent dust and build-up from building up.\nC. A tarp is ideal if you are removing large amounts of ceiling tile. [title] Position a chair or stepladder underneath the tile.\nD. [title] Consider painting your ceiling white, if desired. [step] Many painters use this color in addition to primer to prevent blemishes and cuts in the tile.","prompt":"[header] How to remove a ceiling tile [title] Cover the floor with a protective material if desired. [step] Gypsum ceiling tiles tend to crumble around the edges, especially as they get older. If you want to keep this debris from getting onto your floors, lay down something to protect them, such as sheets of newspaper.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4318,"error":null,"output":"C"}
{"question_id":"hellaswag:00008236","item_index":357,"attempt":0,"prompt_hash":"68eae4621db3882dca7ef97bc4b25de908e9922e8d65b95b28f39bd7193e7598","question":"[header] How to be safe during a cruise ship fire [title] Participate fully in any muster drill. [step] This is an exercise designed for passengers to practice moving safely and quickly during an emergency to prevent injury and save lives. Passengers learn how to use life vests and about escape routes on their ships.\n\nA. Participate as much as possible in any muster drills. [title] Stay calm and mind your own safety.\nB. Muster drills are required by international law and for good reason. While they may seem like a nuisance during a vacation they are done for your own safety and well-being.\nC. [substeps] Some obstacles, such as fire, are difficult to get out of, but not impossible to overcome. Passengers are trained in how to navigate them and to recognize when and how an obstacle may close.\nD. [title] Know that basic tasks are as follows : [substeps] Wear good glasses. A wide brimmed hat is usually good.","prompt":"[header] How to be safe during a cruise ship fire [title] Participate fully in any muster drill. [step] This is an exercise designed for passengers to practice moving safely and quickly during an emergency to prevent injury and save lives. Passengers learn how to use life vests and about escape routes on their ships.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4320,"error":null,"output":"B"}
{"question_id":"hellaswag:00005550","item_index":358,"attempt":0,"prompt_hash":"b043ecb4f82a9fae371c22da9e3246e6003b314994fcf79bb5b0208b57f974bb","question":"[header] How to increase oxygen levels in your home [title] Open your windows. [step] You may spend a lot of time indoors, especially in the winter. Open a window in your home to allow fresh, oxygenated air inside.\n\nA. If you really like minimizing the possibility of a cold, try turning on some air conditioning or open a window occasionally. [substeps] If you are abroad, place your home in a cooler climate to increase your oxygen level.\nB. Standing in your home for extended periods of time can get your heart pumping and increase the level of oxygen in your blood stream. [substeps] Outside, for example, you may want to open your windows often in the morning so you can get some fresh air.\nC. If you live in a colder area, you may consider opening your windows manually every half an hour. If you live in a cooler area, use the ac unit that comes with your windows to regulate your air intake.\nD. If it's warm enough, open two windows on opposite sides of your home to increase air circulation. Try to open your windows for a few minutes 3 times a day, even in winter.","prompt":"[header] How to increase oxygen levels in your home [title] Open your windows. [step] You may spend a lot of time indoors, especially in the winter. Open a window in your home to allow fresh, oxygenated air inside.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4308,"error":null,"output":"D"}
{"question_id":"hellaswag:00006050","item_index":359,"attempt":0,"prompt_hash":"941ed98b9ad8afe9d4ef8cfa8a35329425e515aba6c1697f0fd776bcf2ab8bc9","question":"[header] How to use a breastfeeding nipple shield [title] Use a nipple shield if the baby is having difficulties in latching on. [step] The breastfeeding nipple shield was designed initially for an infant who is having difficulties in latching, or the act of grasping and sucking the mother's nipples and lower areola area. It is important that the baby latches on to achieve maximum breastfeeding potential without friction or other complications.\n\nA. While it may help the mother to wash her breasts thoroughly with warm water and soap, it is not mandatory. If using a nipples shield, consider removing the shield as well.\nB. [substeps] Make sure to consult your pediatrician's instructions for using a breast shield. [title] Apply the assistance of a family member or friend to brace the baby.\nC. [substeps] Place the nipple shield firmly over the top of the mother's breasts. Hook the back nipple shield around a combination of breasts.\nD. [title] Wear a nipple shield if you have flat or inverted nipples. [step] The breastfeeding nipple shield is can also be used if you have flat or inverted nipples.","prompt":"[header] How to use a breastfeeding nipple shield [title] Use a nipple shield if the baby is having difficulties in latching on. [step] The breastfeeding nipple shield was designed initially for an infant who is having difficulties in latching, or the act of grasping and sucking the mother's nipples and lower areola area. It is important that the baby latches on to achieve maximum breastfeeding potential without friction or other complications.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4308,"error":null,"output":"D"}
{"question_id":"hellaswag:00002658","item_index":360,"attempt":0,"prompt_hash":"08c32726f2c8f6aa3713fa54fdea5b6cb0ada9b4903a74bfd386bc431831b143","question":"A person plays a violin in front of a huge crowd of people. the people\n\nA. cheer and clap the musician.\nB. give him a standing ovation at the end.\nC. watch the performer play while clapping.\nD. in the audience applaud as the person play.","prompt":"A person plays a violin in front of a huge crowd of people. the people","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4310,"error":null,"output":"B"}
{"question_id":"hellaswag:00000552","item_index":361,"attempt":0,"prompt_hash":"2db8b056bca828ccbcdd3dcc2f5118eb98e722ec61139e87b2965100304d9031","question":"The two shapes the mix from the bowl into lumps and places them on a baking sheet. The two place the baking sheet in an oven. The two girls hug each other. the two girls\n\nA. place a brown paper bag over an oven.\nB. flat ironed the baked sheets.\nC. sit in front of the oven licking their mixing spoons.\nD. grab plates and dish wash and put them in the dishwasher and return to flipping the cards.","prompt":"The two shapes the mix from the bowl into lumps and places them on a baking sheet. The two place the baking sheet in an oven. The two girls hug each other. the two girls","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4311,"error":null,"output":"A"}
{"question_id":"hellaswag:00001909","item_index":362,"attempt":0,"prompt_hash":"5cb9b25dd9a1528977162558134f66ac409bf4c75cbd00a1e69d8f9d7070904e","question":"A man chugs an entire mug of beer. The friends raises his arms and cheers. the man\n\nA. walks to a stall and holds his crotch while he empties the beer.\nB. misses a wide shot, knocking over the beer.\nC. sets down the mug.\nD. steps away from the bar.","prompt":"A man chugs an entire mug of beer. The friends raises his arms and cheers. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4309,"error":null,"output":"C"}
{"question_id":"hellaswag:00005665","item_index":363,"attempt":0,"prompt_hash":"0d3121e68d3886bfb1757da0a0379ba3405bb7216d77fad78951964883fe8755","question":"[header] How to lighten or brighten dark hair with lemon juice [title] Cut and juice your lemons. [step] Slice your lemon in half. If you have a hand juicer, you can use it to speed up this step, otherwise simply squeezing the lemon by hand works fine.\n\nA. [title] Mix a tablespoon of lemon juice with two tablespoons of honey. [step] For a baby's first facial, your honey is necessary for quick lightening.\nB. Over a cup or other container, squeeze the lemon until all of the juice is thoroughly drained. You may accidentally get a few seeds in your cup, but that's ok.\nC. [title] Place the lemon halves in a bowl. [step] Use a large bowl to measure out 4 cups of lemon juice, while you're in the shower.\nD. Simply cut your lemon in half, then arrange the halves halves in a bowl. Be careful with the cut lemons; some meringues have a tough texture, and this could ruin your taste rating.","prompt":"[header] How to lighten or brighten dark hair with lemon juice [title] Cut and juice your lemons. [step] Slice your lemon in half. If you have a hand juicer, you can use it to speed up this step, otherwise simply squeezing the lemon by hand works fine.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4308,"error":null,"output":"B"}
{"question_id":"hellaswag:00008018","item_index":364,"attempt":0,"prompt_hash":"ff856eb24ee3b2ef0a543812c075fadb26347f2abd09bb0745812212f8048f79","question":"[header] How to apply base makeup [title] Use concealer. [step] You will wreck your look if you've got dark circles under your eyes or have blemishes throughout your face. Don't feel badly about such circles or blemishes, though.\n\nA. This is a natural part of makeup, and you can use that to your advantage. If you must draw your brows in, that will be where most of the concealer and powder will come in.\nB. You can always cover them up if need be. You can also apply some false lashes-it will look better, even if you only have the eyelashes.\nC. You can cover them up with concealer. Apply the concealer in an upside-down pyramid shape.\nD. [substeps] Use some concealer up off of your eyelids so you can wear concealer successfully without using too much. Apply concealer a little higher up on your eyelid than the same way.","prompt":"[header] How to apply base makeup [title] Use concealer. [step] You will wreck your look if you've got dark circles under your eyes or have blemishes throughout your face. Don't feel badly about such circles or blemishes, though.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4308,"error":null,"output":"C"}
{"question_id":"hellaswag:00005258","item_index":365,"attempt":0,"prompt_hash":"557330cf796cd309b38b6e58cc0a4c413895f894024771c7158bc32ef000fc0a","question":"[header] How to tie a sarong [title] Fold the sarong diagonally. [step] Fold the materially diagonally in half to get a triangle shape. [title] Wrap the sarong around your waist.\n\nA. [step] Fold towards your waist where it forms a t. [title] Pull the two sides together across the triangle.\nB. [step] First, start near one top part and move across the rest of the fabric/sarong for the final shape. [title] Fold the sarong to a rectangle.\nC. [title] Gather the two ends of the sarong and tie a knot on the side. [step] Do a second knot to secure, then fluff out the ends of the material.\nD. [step] Any sections that meet at the waist should be close together; otherwise, the sarong may fall off one inch (2.54 cm). [title] Loop the fehnb (or front & back folds) together, making sure that the edges are aligned without overlapping.","prompt":"[header] How to tie a sarong [title] Fold the sarong diagonally. [step] Fold the materially diagonally in half to get a triangle shape. [title] Wrap the sarong around your waist.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4311,"error":null,"output":"C"}
{"question_id":"hellaswag:00004493","item_index":366,"attempt":0,"prompt_hash":"318466e684a40ff951cbea628ee7fa06530d0e7c3484c47c7205eaf05aab57ed","question":"[header] How to earn your rabbit's trust [title] Spend time together alone. [step] Rabbits can easily be startled or overwhelmed by the noise and movement of multiple people. Similarly, other pets are likely to terrify a pet rabbit until they're used to their presence.\n\nA. This is why it's important to get your animal used to being with you while you're apart from other people. With your rabbit comfortable in your presence, it will be easy to get it used to the people around it.\nB. Start slowly by spending time alone with your rabbit, and let them acclimate to you slowly. [substeps] Some rabbits are more territorial than others.\nC. Plus, if your rabbit doesn't want to be near this person, they likely won't be a good pet either. [substeps] Rabbits are curious animals and will just learn to associate the presence of a person with pleasing them.\nD. In order to start earning your rabbit's trust, join your rabbit in the room where it lives without any other humans or animals present. [title] Open the rabbit's hutch at floor level.","prompt":"[header] How to earn your rabbit's trust [title] Spend time together alone. [step] Rabbits can easily be startled or overwhelmed by the noise and movement of multiple people. Similarly, other pets are likely to terrify a pet rabbit until they're used to their presence.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4307,"error":null,"output":"B"}
{"question_id":"hellaswag:00002836","item_index":367,"attempt":0,"prompt_hash":"8e234fdfb4e8b9e40927285dd22825ec1818fdd877e037bd1b24509bb0385525","question":"A man in a black shirt smiles at the camera. He pulls out a harmonica and begins to play it. he\n\nA. uses his hand to accentuate the notes.\nB. blows the smoke around a little then pauses.\nC. mows his lawn with it continuously.\nD. moves the sticks to adjust the string with his hands.","prompt":"A man in a black shirt smiles at the camera. He pulls out a harmonica and begins to play it. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4308,"error":null,"output":"A"}
{"question_id":"hellaswag:00003810","item_index":368,"attempt":0,"prompt_hash":"cd55713a06ba504cc85bfd3a11688a3fbf692b7e772d3f33f21b0397b06ddddf","question":"[header] How to start dreads with short hair [title] Make small circular motions with a soft bristled brush. [step] Brush small, inch sized circles in a clockwise motion until the hair starts to form into balls. This should only take about a minute or two.\n\nA. [title] Gather all the loose hair you will want to twist into dreads. [step] It's best that you only do this once.\nB. Be sure that you brush each small circle evenly; the circular motion of the brush can create tangles. [substeps] Repeat this process on the other side of your head.\nC. Once the balls start to form, move on to the next step. [title] Separate the first scalp oval section into groups by you.\nD. Once a ball of hair forms, move onto another section of the hair to continue making dreadlocks throughout the hair. [substeps] The brushing method works best for coarse hair that's 3/4 \" to 2.5 \" (1.905 cm-6.35 cm) long.","prompt":"[header] How to start dreads with short hair [title] Make small circular motions with a soft bristled brush. [step] Brush small, inch sized circles in a clockwise motion until the hair starts to form into balls. This should only take about a minute or two.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4309,"error":null,"output":"B"}
{"question_id":"hellaswag:00003557","item_index":369,"attempt":0,"prompt_hash":"1275c19ccfc11463fd47b67b65475e0a4962e5810ce3b53b220b3a08508675e8","question":"[header] How to manage dental disease in senior dogs [title] Take your dog to the vet for dental checkups. [step] You should take your dog for dental checkups two more times each year. During oral checkups, your vet can examine the state of your aging dog's teeth to make sure they stay healthy.\n\nA. [substeps] The vet may be able to see a problem that you can't see. They can do x-rays and other imaging tests to look deeper into the gums for problems.\nB. [substeps] If your aging dog has dental disease, you should take your dog to the vet once a year to get regular checkups. [title] Prepare daily to receive your dog's dental recommendations.\nC. Additionally, the vet may recommend a whitening treatment if plaque builds up on your dog's teeth. [title] Discuss benefits with your veterinarian.\nD. They can also check for other dental disease, such as your dog's thyroid disorder, gastroesophageal reflux disease, tuberculosis, or felv syndrome. [substeps] Your vet can do professional dental checkups if your dog is under a certain level of care.","prompt":"[header] How to manage dental disease in senior dogs [title] Take your dog to the vet for dental checkups. [step] You should take your dog for dental checkups two more times each year. During oral checkups, your vet can examine the state of your aging dog's teeth to make sure they stay healthy.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4308,"error":null,"output":"C"}
{"question_id":"hellaswag:00009794","item_index":370,"attempt":0,"prompt_hash":"09929ffa131a6d69e5f416cc3ba0eea7a57569e0ecc740861143d7d0512b24af","question":"[header] How to make mexican pizza [title] Preheat the oven and grease the pans. [step] To get the tortillas crispy and ready for the pizza, it's important to get your oven to the right temperature. Preheat it to 375 degrees fahrenheit (approximately 190 degrees celsius).\n\nA. Get your baking sheets ready by lightly greasing them with non-stick cooking spray. [substeps] Pay close attention to your oven to be sure that it's at the right temperature.\nB. [substeps] The best temperature is specified above the stove top, as well as in the oven itself. Call a pizza artist if you aren't sure if your oven has the right temperature.\nC. Check the temperature on the oven's thermometer to see how well it's cooking. The inside should be browned, and the oven should beep or flash when it reaches 155 degrees f (66 degrees c).\nD. [substeps] If you have a cooler oven, you may want to preheat it to 360 degrees fahrenheit. This will give the tamales the best crisp, but if you'd rather only use a bowl or all your tortillas, you can store them at room temperature.","prompt":"[header] How to make mexican pizza [title] Preheat the oven and grease the pans. [step] To get the tortillas crispy and ready for the pizza, it's important to get your oven to the right temperature. Preheat it to 375 degrees fahrenheit (approximately 190 degrees celsius).","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4308,"error":null,"output":"A"}