{"question_id":"hellaswag:00004149","item_index":0,"attempt":0,"prompt_hash":"320beff94af9487788934b2bba79555bafa090c5b0b72dfeab0e1aaa3422e817","question":"[header] How to survive financial hardship in a marriage [title] Remember your wedding vows ,' for better, for worse, for richer, for poorer'. [step] When we make these vows to each other it probably doesn't enter our minds what could happen down the road, but nonetheless we made a promise that we had the best intentions of keeping. During difficult financial times those vows become far more prominent and necessary.\n\nA. [title] Remember not to shut doors at all hours. [step] Shut off all lights at any time, especially those that serve to light up your wedding day.\nB. Stick with it not because of the promise you made but because of the person you made it to. [title] Remember that as with everything in life, it will turn around again.\nC. We finally remember that our intentions weren't necessarily good and neither were your intentions. A discharge of our vows is a second step towards firbility.\nD. [substeps] Write them down for future reference (or on a reference sheet). The more you understand the meaning of financial hardship, the greater your chances at survival.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to survive financial hardship in a marriage [title] Remember your wedding vows ,' for better, for worse, for richer, for poorer'. [step] When we make these vows to each other it probably doesn't enter our minds what could happen down the road, but nonetheless we made a promise that we had the best intentions of keeping. During difficult financial times those vows become far more prominent and necessary.\n\nA. [title] Remember not to shut doors at all hours. [step] Shut off all lights at any time, especially those that serve to light up your wedding day.\nB. Stick with it not because of the promise you made but because of the person you made it to. [title] Remember that as with everything in life, it will turn around again.\nC. We finally remember that our intentions weren't necessarily good and neither were your intentions. A discharge of our vows is a second step towards firbility.\nD. [substeps] Write them down for future reference (or on a reference sheet). The more you understand the meaning of financial hardship, the greater your chances at survival.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6210,"error":null,"output":"B"}
{"question_id":"hellaswag:00009152","item_index":1,"attempt":0,"prompt_hash":"39119b83551d01d61029da07059a7497899aa1fa231e53fa51998e94585f7126","question":"[header] How to plan your wedding music [title] Sit down together and talk about what music matters to you. [step] Each of you will have different ideas and preferences and this is about getting the main ideas heard, to allow both of you to plan in full knowledge of what matters to one another. [substeps] Write down a list of the songs and/or music that you would like to form a part of your wedding.\n\nA. Make sure to decide ahead of time the location where you want to arrange your wedding music. You can discuss how many romantic themes you hope to include in each song and set up time for each venue.\nB. Ask your future spouse to do the same. Bring the lists along to your discussion together.\nC. Brainstorm a logical order together with your partner. Everyone should have their own wants and preferences and think about what exactly matters to them, in order to get included.\nD. Or you can look through the music previously choreographed and combine the music together to put into song form. [title] Decide how the music matters to you.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to plan your wedding music [title] Sit down together and talk about what music matters to you. [step] Each of you will have different ideas and preferences and this is about getting the main ideas heard, to allow both of you to plan in full knowledge of what matters to one another. [substeps] Write down a list of the songs and/or music that you would like to form a part of your wedding.\n\nA. Make sure to decide ahead of time the location where you want to arrange your wedding music. You can discuss how many romantic themes you hope to include in each song and set up time for each venue.\nB. Ask your future spouse to do the same. Bring the lists along to your discussion together.\nC. Brainstorm a logical order together with your partner. Everyone should have their own wants and preferences and think about what exactly matters to them, in order to get included.\nD. Or you can look through the music previously choreographed and combine the music together to put into song form. [title] Decide how the music matters to you.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1657,"error":null,"output":"B"}
{"question_id":"hellaswag:00009645","item_index":2,"attempt":0,"prompt_hash":"b4e5f821a16bff378ec78d28ed045f1b7e9537c088e03b0648fc5e4df03456e1","question":"[header] How to dress in knee length skirts [title] Pick a length that flatters your height. [step] Not everyone has the same body type, and there are some ways you can make a knee-length skirt look better for yours. [substeps] Short women or girls look really good in knee-length skirts that fall right below the knees.\n\nA. When it comes to skirts that hit at your hip, go around the waist to accentuate it. Knee-length skirts are more flattering when they have structure and a twist.\nB. Not everyone likes the hemlines of a waist-length skirt, and you don't want the skirt to stop too low on their legs, or they'll look overly babyish. If you're going out to dinner, go for a skirt that reaches the mid-thigh.\nC. It's not true that short women can't wear knee-length skirts. If you're very tall, you will look great in a skirt that falls right at the mid-calf point.\nD. Medium petite women or girls that are fairly tall don't care if their feet are bare, and may prefer wearing knee-length skirts with ankle-length tights. For a more feminine look, opt for a length that hits at the knee and curves over your thighs.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to dress in knee length skirts [title] Pick a length that flatters your height. [step] Not everyone has the same body type, and there are some ways you can make a knee-length skirt look better for yours. [substeps] Short women or girls look really good in knee-length skirts that fall right below the knees.\n\nA. When it comes to skirts that hit at your hip, go around the waist to accentuate it. Knee-length skirts are more flattering when they have structure and a twist.\nB. Not everyone likes the hemlines of a waist-length skirt, and you don't want the skirt to stop too low on their legs, or they'll look overly babyish. If you're going out to dinner, go for a skirt that reaches the mid-thigh.\nC. It's not true that short women can't wear knee-length skirts. If you're very tall, you will look great in a skirt that falls right at the mid-calf point.\nD. Medium petite women or girls that are fairly tall don't care if their feet are bare, and may prefer wearing knee-length skirts with ankle-length tights. For a more feminine look, opt for a length that hits at the knee and curves over your thighs.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1970,"error":null,"output":"C"}
{"question_id":"hellaswag:00009316","item_index":3,"attempt":0,"prompt_hash":"40e43f92db9885124504dd65bf08aeec46c13b04e0a1652689bd793a9dd2e6e5","question":"[header] How to look like ashley barnum from' perfect ' [title] Get the look. [step] Ashley has blonde hair that ranges from light brown to almost white-blonde near her face. It's described as being' surfer girl' hair, so you should try using the product' surfer hair' by garnier.\n\nA. She also has super-shiny and glossy hair, and it is revealed in the book that she rinses her hair with white vinegar so that it becomes shinier. Now this actually works, so consider it: you will need 1 part white vinegar mixed with three parts cold water.\nB. For kit kerr, you need to mix emo' surfer' with ash blonde, but make sure you show your true color. Ashy hair works best, but you can try out the' pure girl' look for makeup and powdering.\nC. If you can't find this extremely attractive look, use straighten-up shampoo and conditioner. If you don't like the appearance of her hair, have a good waxing or body hair section done, with a paddle-like shape and wavy roots, to create this look.\nD. Hazel eyes that are never quite big are her signature. Look for a hair dye made for fair to medium hair.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to look like ashley barnum from' perfect ' [title] Get the look. [step] Ashley has blonde hair that ranges from light brown to almost white-blonde near her face. It's described as being' surfer girl' hair, so you should try using the product' surfer hair' by garnier.\n\nA. She also has super-shiny and glossy hair, and it is revealed in the book that she rinses her hair with white vinegar so that it becomes shinier. Now this actually works, so consider it: you will need 1 part white vinegar mixed with three parts cold water.\nB. For kit kerr, you need to mix emo' surfer' with ash blonde, but make sure you show your true color. Ashy hair works best, but you can try out the' pure girl' look for makeup and powdering.\nC. If you can't find this extremely attractive look, use straighten-up shampoo and conditioner. If you don't like the appearance of her hair, have a good waxing or body hair section done, with a paddle-like shape and wavy roots, to create this look.\nD. Hazel eyes that are never quite big are her signature. Look for a hair dye made for fair to medium hair.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1909,"error":null,"output":"C"}
{"question_id":"hellaswag:00009851","item_index":4,"attempt":0,"prompt_hash":"cc7ee9946aefd1b0d722a0ef1f8eed7a57d73b674f356f48296e3f001b85473e","question":"[header] How to set a digital watch [title] Contact the manufacturer or look up a manual for complicated watches. [step] If your watch is especially complicated and you don't have instructions, look up a digital manual online. Digital manuals can be found with an online keyword search for your watch brand and model.\n\nA. [substeps] Some watches turn off automatically during the day. If yours is in the morning, switch it to the middle of the night.\nB. Your watch may take you a while, or it may take a little while to get to a level where you won't experience any problem. [substeps] Digital watches come with online cameras and camera boxes.\nC. [substeps] If a digital manual is impossible to find, look up and call the customer service number for your watch. Model and brand information are usually written or engraved on the back of the watch or somewhere on the watch face.\nD. They are usually $15-$40. They cover the exact specs of your watch and include instructions to change them.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to set a digital watch [title] Contact the manufacturer or look up a manual for complicated watches. [step] If your watch is especially complicated and you don't have instructions, look up a digital manual online. Digital manuals can be found with an online keyword search for your watch brand and model.\n\nA. [substeps] Some watches turn off automatically during the day. If yours is in the morning, switch it to the middle of the night.\nB. Your watch may take you a while, or it may take a little while to get to a level where you won't experience any problem. [substeps] Digital watches come with online cameras and camera boxes.\nC. [substeps] If a digital manual is impossible to find, look up and call the customer service number for your watch. Model and brand information are usually written or engraved on the back of the watch or somewhere on the watch face.\nD. They are usually $15-$40. They cover the exact specs of your watch and include instructions to change them.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1656,"error":null,"output":"C"}
{"question_id":"hellaswag:00009474","item_index":5,"attempt":0,"prompt_hash":"57db1a1fe5a4b7b99bde19982d17ab1f2374ceafba7a4e5b4f680c61452f9776","question":"[header] How to make a birthday mug cake [title] Microwave the butter in a large, microwave-safe mug. [step] Melt it for about 30-45 seconds until it turns into a liquid. [title] Add the flour, egg, sugar, milk, vanilla, and baking powder.\n\nA. [step] Whisk gently with a fork until the ingredients have fully incorporated. [title] Pour in the sprinkles.\nB. [step] Stir through to form smooth, well-blended, creamy cake batter. [title] Fill a 16 x 16 inch (33.3 x 27.6 cm) mug pan halfway with baking powder and cook for about 10 minutes.\nC. [step] Stir with a mini whisk or flat spatula. [title] Pour the confectioners' sugar into the melted butter mixture.\nD. [step] Stir on low speed for about 1 hour. [title] Pour the flour, milk, and baking powder into the mug.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make a birthday mug cake [title] Microwave the butter in a large, microwave-safe mug. [step] Melt it for about 30-45 seconds until it turns into a liquid. [title] Add the flour, egg, sugar, milk, vanilla, and baking powder.\n\nA. [step] Whisk gently with a fork until the ingredients have fully incorporated. [title] Pour in the sprinkles.\nB. [step] Stir through to form smooth, well-blended, creamy cake batter. [title] Fill a 16 x 16 inch (33.3 x 27.6 cm) mug pan halfway with baking powder and cook for about 10 minutes.\nC. [step] Stir with a mini whisk or flat spatula. [title] Pour the confectioners' sugar into the melted butter mixture.\nD. [step] Stir on low speed for about 1 hour. [title] Pour the flour, milk, and baking powder into the mug.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1599,"error":null,"output":"A"}
{"question_id":"hellaswag:00004552","item_index":6,"attempt":0,"prompt_hash":"bde2607a16d85b8adf478b067001be20498e70fc18daeebb4961e64e341903ee","question":"[header] How to shrink your financial footprint [title] Look at this project in the same way as a weight loss diet. [step] Most people know that the best way to lose weight is to burn more calories than you take in; financial solvency comes when you spend less than you bring in. [title] Start with the low-hanging fruit.\n\nA. [step] You may think that fruit can help eat healthier, but fruit can actually contribute to weight gain for many reasons. Getting large fruit will help you take a step back and balance your diet and your calorie intake.\nB. [step] Eliminate the expenses that add no value to your life, such as late fees, unnecessary or duplicate purchases, excess interest on debt. [title] Spend your money on the things and experiences that truly enhance your quality of life.\nC. [step] Since the trees are always chock full of fruit, when you first start shopping online, you'll find plenty of low-hanging fruit. Use an online food auction tool such as retailmenot and purchase your fruit.\nD. [step] Even if you don't plan to cook it for the next six months, you'll need to add some weight-loss to the steer. When done properly, berries and nuts should be left whole and fresh, in small quantities to keep you consistently in good shape.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to shrink your financial footprint [title] Look at this project in the same way as a weight loss diet. [step] Most people know that the best way to lose weight is to burn more calories than you take in; financial solvency comes when you spend less than you bring in. [title] Start with the low-hanging fruit.\n\nA. [step] You may think that fruit can help eat healthier, but fruit can actually contribute to weight gain for many reasons. Getting large fruit will help you take a step back and balance your diet and your calorie intake.\nB. [step] Eliminate the expenses that add no value to your life, such as late fees, unnecessary or duplicate purchases, excess interest on debt. [title] Spend your money on the things and experiences that truly enhance your quality of life.\nC. [step] Since the trees are always chock full of fruit, when you first start shopping online, you'll find plenty of low-hanging fruit. Use an online food auction tool such as retailmenot and purchase your fruit.\nD. [step] Even if you don't plan to cook it for the next six months, you'll need to add some weight-loss to the steer. When done properly, berries and nuts should be left whole and fresh, in small quantities to keep you consistently in good shape.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1964,"error":null,"output":"B"}
{"question_id":"hellaswag:00006378","item_index":7,"attempt":0,"prompt_hash":"37ae720bf9d561fe94f7bcc946a6edc404f196317969a5fb467489ee806bdcdb","question":"[header] How to make pinwheel sandwiches [title] Make thin bread slices. [step] Begin with a loaf of your favourite sandwich bread. Cut thin slices lengthwise across the loaf.\n\nA. Now roll each slice of bread inside of itself and press or grip the slices to make them thin. [title] Cut long strips of bread lengthwise.\nB. [title] Preheat your oven to 350 ° f (177 ° c). [step] Make sure to put a barrier (like foil) between the bread and the oven.\nC. This will give the bread a distinctly boxy shape. Alternatively, slice thin slices sideways, like layers.\nD. The thinner you make the slices, the easier it will be to roll them into wheels. As you cut, make an effort to keep the slices at a consistent thickness.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make pinwheel sandwiches [title] Make thin bread slices. [step] Begin with a loaf of your favourite sandwich bread. Cut thin slices lengthwise across the loaf.\n\nA. Now roll each slice of bread inside of itself and press or grip the slices to make them thin. [title] Cut long strips of bread lengthwise.\nB. [title] Preheat your oven to 350 ° f (177 ° c). [step] Make sure to put a barrier (like foil) between the bread and the oven.\nC. This will give the bread a distinctly boxy shape. Alternatively, slice thin slices sideways, like layers.\nD. The thinner you make the slices, the easier it will be to roll them into wheels. As you cut, make an effort to keep the slices at a consistent thickness.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1287,"error":null,"output":"D"}
{"question_id":"hellaswag:00008838","item_index":8,"attempt":0,"prompt_hash":"bca062f88e667c96c5a18b7452c6b0f0153d9e557dad9d42e58854fc5cf48d0b","question":"[header] How to wash short hair [title] Pay attention to your scalp. [step] Identify the kind of scalp you have. The type of scalp you possess affects the types of shampoo and conditioner products you should be purchasing for healthy short hair.\n\nA. Look closely at the bottle before you buy. Whether the product is hydrating, clarifying, or moisturizing, its main properties should be listed right on the bottle for you.\nB. [substeps] Note whether your hair tends to be oily or dry. Oily hair is a much higher risk for hair cancer than dry hair.\nC. As part of your hair care regimen, you'll need to care for your scalp. There are many different types of hair, and choosing a hairstyle and style that reflects the kinds of hair you have will give you total control over your hair quality.\nD. Depending on your hair type, you may have none/none of the following symptoms, so pay attention. [substeps] Oily hair forms oily or puffiness when it is blended with other healthy hair products.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to wash short hair [title] Pay attention to your scalp. [step] Identify the kind of scalp you have. The type of scalp you possess affects the types of shampoo and conditioner products you should be purchasing for healthy short hair.\n\nA. Look closely at the bottle before you buy. Whether the product is hydrating, clarifying, or moisturizing, its main properties should be listed right on the bottle for you.\nB. [substeps] Note whether your hair tends to be oily or dry. Oily hair is a much higher risk for hair cancer than dry hair.\nC. As part of your hair care regimen, you'll need to care for your scalp. There are many different types of hair, and choosing a hairstyle and style that reflects the kinds of hair you have will give you total control over your hair quality.\nD. Depending on your hair type, you may have none/none of the following symptoms, so pay attention. [substeps] Oily hair forms oily or puffiness when it is blended with other healthy hair products.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1631,"error":null,"output":"A"}
{"question_id":"hellaswag:00001958","item_index":9,"attempt":0,"prompt_hash":"3fedceffa36456179ebcc8ac3a9060eadb7ab2e95a18e4cebded4a4ceb2d01f2","question":"Two men wearing hats are seated on a couch. one of the men\n\nA. throws the other one a ball.\nB. stands up and grabs the back of the other man's shirt.\nC. puts on a pair of stilts and the other mans a helmet.\nD. is playing a guitar.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nTwo men wearing hats are seated on a couch. one of the men\n\nA. throws the other one a ball.\nB. stands up and grabs the back of the other man's shirt.\nC. puts on a pair of stilts and the other mans a helmet.\nD. is playing a guitar.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":724,"error":null,"output":"A"}
{"question_id":"hellaswag:00002097","item_index":10,"attempt":0,"prompt_hash":"911126a1c78a943bb25d3b658e6538e5c7aa47eed9ceab8b598a5675a5227fda","question":"A man walks up to a barbell and bends over, lifting at the knees. He lifts the barbell up to his chest, rests for a moment, then lifts it over his head. he\n\nA. drops the barbell to the floor when he lowers it, holding it tightly as he bends over, then into the crowd before dropping it so far it lands on the floor.\nB. drops the barbell as another person hits it.\nC. throws the barbell across the gym to land on the white mat below him.\nD. holds the pose for a few seconds before dropping it to the ground.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA man walks up to a barbell and bends over, lifting at the knees. He lifts the barbell up to his chest, rests for a moment, then lifts it over his head. he\n\nA. drops the barbell to the floor when he lowers it, holding it tightly as he bends over, then into the crowd before dropping it so far it lands on the floor.\nB. drops the barbell as another person hits it.\nC. throws the barbell across the gym to land on the white mat below him.\nD. holds the pose for a few seconds before dropping it to the ground.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1083,"error":null,"output":"D"}
{"question_id":"hellaswag:00002308","item_index":11,"attempt":0,"prompt_hash":"29efa3ca3b47acdf016e434c58db012c9dd462ad91283ea3540c2b9a6a957d03","question":"A man is knelling down on snow covered ice. The man points to a whole in the ice. the man\n\nA. then digs through some snow on the ground.\nB. drills some holes in the snow.\nC. then retrieves a line from the line in the ice exposing a caught fish.\nD. crawls across the ice.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA man is knelling down on snow covered ice. The man points to a whole in the ice. the man\n\nA. then digs through some snow on the ground.\nB. drills some holes in the snow.\nC. then retrieves a line from the line in the ice exposing a caught fish.\nD. crawls across the ice.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":715,"error":null,"output":"C"}
{"question_id":"hellaswag:00009888","item_index":12,"attempt":0,"prompt_hash":"dc5c17c105901d375c0f59cdfd6e09dbd890e99345b1e4b8c02b35426ae43a05","question":"[header] How to cut your own hair (men ) [title] Wash your hair before you cut it. [step] Your hair will be easier to work with if it's clean and snarl-free. Get your hair wet under running water and use shampoo and conditioner, rinsing both out after you've worked them through your hair.\n\nA. [substeps] If you have no other hair options available, rinse your hair under running water for a minute to get out any tangles. [title] Use hair conditioner after showering.\nB. [title] Soak your hair in apple cider vinegar for 20 to 30 minutes. [step] Mix a 1/2 cup (120 ml) of apple cider vinegar in 1 cup (226 ml) of water.\nC. [title] Detangle your hair with a comb. [step] Run the comb through your hair and work out any knots with the comb or with your fingers.\nD. [title] Rinse and dry your hair with a towel. [step] After you're finished working out your hair, take a good, hot shower.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to cut your own hair (men ) [title] Wash your hair before you cut it. [step] Your hair will be easier to work with if it's clean and snarl-free. Get your hair wet under running water and use shampoo and conditioner, rinsing both out after you've worked them through your hair.\n\nA. [substeps] If you have no other hair options available, rinse your hair under running water for a minute to get out any tangles. [title] Use hair conditioner after showering.\nB. [title] Soak your hair in apple cider vinegar for 20 to 30 minutes. [step] Mix a 1/2 cup (120 ml) of apple cider vinegar in 1 cup (226 ml) of water.\nC. [title] Detangle your hair with a comb. [step] Run the comb through your hair and work out any knots with the comb or with your fingers.\nD. [title] Rinse and dry your hair with a towel. [step] After you're finished working out your hair, take a good, hot shower.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1722,"error":null,"output":"A"}
{"question_id":"hellaswag:00000895","item_index":13,"attempt":0,"prompt_hash":"81ab8ccd73912b6095f8e4fbb856f670600b42be004fcb65c41a146269f9eac8","question":"There's an open land area with greenery around and grassy surface area. there\n\nA. is a tennis court and a tennis court on the side.\nB. is a man wearing white shirt and jungle gym shorts exercising on the green surface.\nC. 's a lawn trimmer lying on the ground near some over grown patches of grass.\nD. are no trees, but there is a large green grassy area.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nThere's an open land area with greenery around and grassy surface area. there\n\nA. is a tennis court and a tennis court on the side.\nB. is a man wearing white shirt and jungle gym shorts exercising on the green surface.\nC. 's a lawn trimmer lying on the ground near some over grown patches of grass.\nD. are no trees, but there is a large green grassy area.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":877,"error":null,"output":"D"}
{"question_id":"hellaswag:00009257","item_index":14,"attempt":0,"prompt_hash":"7c63b78868a24d6ac5e5f737bd34659db026240fd07fbbfeef70da46361ce033","question":"[header] How to remove turbidity [title] Remove organic debris. [step] Use a pool net to fish out any large, visible debris in the water. Organic debris commonly includes leaves, branches, and dead insects.\n\nA. This debris will cause the turbidity to fade, and bacteria will be unable to grow. Not all organic matter is inherently easy to clean.\nB. [substeps] Organic debris can decay and add sediment to the water. The dirt on the debris can also cause cloudiness.\nC. If you live in a hot area, dry out your net in a garage, shed, or other combination of locations. By removing this debris, you can keep your home clean.\nD. Collecting it in your garden is one way to remove it. [substeps] Collect as much debris as possible and throw it away.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to remove turbidity [title] Remove organic debris. [step] Use a pool net to fish out any large, visible debris in the water. Organic debris commonly includes leaves, branches, and dead insects.\n\nA. This debris will cause the turbidity to fade, and bacteria will be unable to grow. Not all organic matter is inherently easy to clean.\nB. [substeps] Organic debris can decay and add sediment to the water. The dirt on the debris can also cause cloudiness.\nC. If you live in a hot area, dry out your net in a garage, shed, or other combination of locations. By removing this debris, you can keep your home clean.\nD. Collecting it in your garden is one way to remove it. [substeps] Collect as much debris as possible and throw it away.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1244,"error":null,"output":"A"}
{"question_id":"hellaswag:00007362","item_index":15,"attempt":0,"prompt_hash":"8138bcf223ee09ad9c7ee4151eaeaa888e70d396e899ec4992c9cf0288129c67","question":"[header] How to get a sexy back [title] Test your posture. [step] This test will help gauge how good your standing posture is : [substeps] Stand with your heels 2 to 4 inches away from a wall, and with your head, shoulder blades, and buttocks touching the wall. Reach one hand back and, placing your palm flat against he wall, slide your hand behind your lower back curve.\n\nA. At the same time, stretch your arms straight out in front of you. Pause for a few seconds and then slowly lift your arms out to look around.\nB. You will feel a slight stretch in your back. If the curve in your back is correct, you are actually standing.\nC. The space between your lower back and the wall should measure roughly one hand's thickness. If there's too much space, tightening your abs will help flatten the curve in your back.\nD. If you feel any resistance, grab the back of the leg at the top of the hip bone. If you feel a strong pressure, increase the resistance.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to get a sexy back [title] Test your posture. [step] This test will help gauge how good your standing posture is : [substeps] Stand with your heels 2 to 4 inches away from a wall, and with your head, shoulder blades, and buttocks touching the wall. Reach one hand back and, placing your palm flat against he wall, slide your hand behind your lower back curve.\n\nA. At the same time, stretch your arms straight out in front of you. Pause for a few seconds and then slowly lift your arms out to look around.\nB. You will feel a slight stretch in your back. If the curve in your back is correct, you are actually standing.\nC. The space between your lower back and the wall should measure roughly one hand's thickness. If there's too much space, tightening your abs will help flatten the curve in your back.\nD. If you feel any resistance, grab the back of the leg at the top of the hip bone. If you feel a strong pressure, increase the resistance.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1648,"error":null,"output":"C"}
{"question_id":"hellaswag:00001705","item_index":16,"attempt":0,"prompt_hash":"07c493df4bf19dd05c0d7533b3348dfbe9e1beebe2442aa06a1dd136a3d8eb8a","question":"Now two brown men who were crouched down in front of the adults playing instrument stand up and they do a dance that include both of them holding onto long bladed knives while everyone around them continues to watch. a white skin man\n\nA. enters and does his knife routine with one of the brown men as the other one leaves, and they continue switching out and taking turns with this double knife dance with only two of them dancing at a time.\nB. in some years walks past in one piece and then is seen walking next to a small girl behind the other and then backs away posing for pictures.\nC. wearing white pants pulls on a long braided beard on the right with a brush, the man wearing green shirt begins to move fast to the left of the man wearing brown pants starts to move rapid till all of the males are in place.\nD. wearing a blue sweater wearing a black shirt begins to roll the violin into a ball while other gray pucks stand nearby and watch him.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nNow two brown men who were crouched down in front of the adults playing instrument stand up and they do a dance that include both of them holding onto long bladed knives while everyone around them continues to watch. a white skin man\n\nA. enters and does his knife routine with one of the brown men as the other one leaves, and they continue switching out and taking turns with this double knife dance with only two of them dancing at a time.\nB. in some years walks past in one piece and then is seen walking next to a small girl behind the other and then backs away posing for pictures.\nC. wearing white pants pulls on a long braided beard on the right with a brush, the man wearing green shirt begins to move fast to the left of the man wearing brown pants starts to move rapid till all of the males are in place.\nD. wearing a blue sweater wearing a black shirt begins to roll the violin into a ball while other gray pucks stand nearby and watch him.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1604,"error":null,"output":"A"}
{"question_id":"hellaswag:00008956","item_index":17,"attempt":0,"prompt_hash":"91e6b6e39583c297948da6d14bdd9798579b87d24061a83a6d300e0abe55a8e4","question":"[header] How to use a bulb syringe on a baby [title] Consult your pediatrician. [step] You should always consult your pediatrician before administering any type of medication or using any kind of medical device on your baby. Your doctor will be able to diagnose your baby, and provide an adequate treatment regime.\n\nA. [substeps] The most common treatment for sad is a breathalyzer reading. However, most medications used in birth control cannot be used on babies with sad.\nB. [substeps] Your baby should not have symptoms for at least a month and a half. This will allow your baby to adjust to medications.\nC. They may also recommend that you keep your baby still for a couple of days after treatment to give blood cells time to fight infections. [title] Introduce your baby to their pediatrician as soon as possible.\nD. If your baby is severely congested, you doctor might prescribe nose drops, and syringe your baby's nose during the appointment in order to provide immediate relief. Your doctor can also show you how to correctly use the bulb syringe.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to use a bulb syringe on a baby [title] Consult your pediatrician. [step] You should always consult your pediatrician before administering any type of medication or using any kind of medical device on your baby. Your doctor will be able to diagnose your baby, and provide an adequate treatment regime.\n\nA. [substeps] The most common treatment for sad is a breathalyzer reading. However, most medications used in birth control cannot be used on babies with sad.\nB. [substeps] Your baby should not have symptoms for at least a month and a half. This will allow your baby to adjust to medications.\nC. They may also recommend that you keep your baby still for a couple of days after treatment to give blood cells time to fight infections. [title] Introduce your baby to their pediatrician as soon as possible.\nD. If your baby is severely congested, you doctor might prescribe nose drops, and syringe your baby's nose during the appointment in order to provide immediate relief. Your doctor can also show you how to correctly use the bulb syringe.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1720,"error":null,"output":"D"}
{"question_id":"hellaswag:00002388","item_index":18,"attempt":0,"prompt_hash":"8a03e4f373cc223e1aac88d7870f713478bb6b6b41883f58f2248e692531b5b6","question":"A bottle of moisturizer is shown. A woman on a beach applies it to her skin. she\n\nA. then spits it out into the ground.\nB. rubs it into her skin.\nC. then shows how to lather it up.\nD. rubs the moisturizer into her skin with several measured movements.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA bottle of moisturizer is shown. A woman on a beach applies it to her skin. she\n\nA. then spits it out into the ground.\nB. rubs it into her skin.\nC. then shows how to lather it up.\nD. rubs the moisturizer into her skin with several measured movements.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1124,"error":null,"output":"D. rubs the moisturizer into her skin with several measured movements."}
{"question_id":"hellaswag:00004698","item_index":19,"attempt":0,"prompt_hash":"0c2124322a6ea14398463ebb292d0b266621e2a143fb28558efade444cd1941d","question":"[header] How to create homemade natural skincare products [title] Remove makeup and wash your face with mild soap and warm water. [step] Pat dry with a clean towel. [title] Cut a piece of ripe papaya and remove the seeds and the pulp (which you can save for later as a snack).\n\nA. [title] Mix together half a cup (177 ml) of honey, two tablespoons of powdered sugar and two tablespoons of aloe vera gel. [step] Wipe your face with a cloth and then dab a little honey onto your face, according to the directions on the package.\nB. [title] Peel and chop 1 teaspoon of papaya into cubes. [step] Then spread the cubes on a low sheet (7 x 2 cm) of wax paper.\nC. [title] Cover your face with a paste of semi-ripe papaya and olive oil. [step] Rub it on with your fingers for about 20 minutes.\nD. [title] Rub the papaya slice over your face and skin, concentrating on any wrinkles or lines. [title] Allow papaya to dry for up to 20 minutes on the skin and then rinse with warm water.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to create homemade natural skincare products [title] Remove makeup and wash your face with mild soap and warm water. [step] Pat dry with a clean towel. [title] Cut a piece of ripe papaya and remove the seeds and the pulp (which you can save for later as a snack).\n\nA. [title] Mix together half a cup (177 ml) of honey, two tablespoons of powdered sugar and two tablespoons of aloe vera gel. [step] Wipe your face with a cloth and then dab a little honey onto your face, according to the directions on the package.\nB. [title] Peel and chop 1 teaspoon of papaya into cubes. [step] Then spread the cubes on a low sheet (7 x 2 cm) of wax paper.\nC. [title] Cover your face with a paste of semi-ripe papaya and olive oil. [step] Rub it on with your fingers for about 20 minutes.\nD. [title] Rub the papaya slice over your face and skin, concentrating on any wrinkles or lines. [title] Allow papaya to dry for up to 20 minutes on the skin and then rinse with warm water.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1877,"error":null,"output":"C"}
{"question_id":"hellaswag:00002257","item_index":20,"attempt":0,"prompt_hash":"8f84ee6c87df375f2aa6fee4db514faf8cdb206797ef2631cbf8ddbf0e3ceda3","question":"The lady lifts her leg up behind her and moves it to the front and rests it on her leg. the lady\n\nA. demonstrates the stance to properly do crunches and belly dancing steps.\nB. lifts her other leg in front of her and then the two begin to play the game of table tennis.\nC. stance her leg forward.\nD. stands in front of the camera facing the mirror.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nThe lady lifts her leg up behind her and moves it to the front and rests it on her leg. the lady\n\nA. demonstrates the stance to properly do crunches and belly dancing steps.\nB. lifts her other leg in front of her and then the two begin to play the game of table tennis.\nC. stance her leg forward.\nD. stands in front of the camera facing the mirror.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":790,"error":null,"output":"A"}
{"question_id":"hellaswag:00000054","item_index":21,"attempt":0,"prompt_hash":"c52d7f40bd201a07181d72a01839cec125dcdd44677b6693ba3ceedf34dde801","question":"A man walks onto the stage to join a band. A man plays up front of a band with a harmonica. the keyboardist\n\nA. plays the hip hop and dance moves with the band.\nB. plays a spirited solo during the song.\nC. plays his guitar while moving his body around in front of the band.\nD. sings with the band, twirling his violin as he walks.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA man walks onto the stage to join a band. A man plays up front of a band with a harmonica. the keyboardist\n\nA. plays the hip hop and dance moves with the band.\nB. plays a spirited solo during the song.\nC. plays his guitar while moving his body around in front of the band.\nD. sings with the band, twirling his violin as he walks.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1044,"error":null,"output":"B. plays a spirited solo during the song."}
{"question_id":"hellaswag:00005236","item_index":22,"attempt":0,"prompt_hash":"4eb671786cdacc34f7c513bff93eac891dcb1a1e1c025d1f0d869098b620e343","question":"[header] How to play tight end on a football team [title] Know where to block. [step] In some plays, where the ball is being run by the running back, or the tight end is just not receiving, they will usually be lined up on the offensive line, blocking. Depending on the play, the tight end will block a few different people, but most of the time, it is the defensive end.\n\nA. To play tighter end, you must block the ball until it is swinging towards your team. That means you will need to block it before it runs off the goal line, then move forward for that shot.\nB. To get this, block the ball so that it cannot go from either side of you, and block it through the middle (i.e. on the side that the ball is being run from).\nC. [title] Know when, and when not to be aggressive. [step] On blocking plays, your job is to keep the pocket open for the quarterback to get rid of the ball, or for the runner to get through the gap.\nD. Once you see someone coming from any distance, block it. This will force the dribbling back on their side, and cause them to dribble side to side and catch the ball.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to play tight end on a football team [title] Know where to block. [step] In some plays, where the ball is being run by the running back, or the tight end is just not receiving, they will usually be lined up on the offensive line, blocking. Depending on the play, the tight end will block a few different people, but most of the time, it is the defensive end.\n\nA. To play tighter end, you must block the ball until it is swinging towards your team. That means you will need to block it before it runs off the goal line, then move forward for that shot.\nB. To get this, block the ball so that it cannot go from either side of you, and block it through the middle (i.e. on the side that the ball is being run from).\nC. [title] Know when, and when not to be aggressive. [step] On blocking plays, your job is to keep the pocket open for the quarterback to get rid of the ball, or for the runner to get through the gap.\nD. Once you see someone coming from any distance, block it. This will force the dribbling back on their side, and cause them to dribble side to side and catch the ball.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1914,"error":null,"output":"B"}
{"question_id":"hellaswag:00001141","item_index":23,"attempt":0,"prompt_hash":"f83eb613c1f2aa8e760b9f653550a500062ed0564b271c82f6e0ce00ecd9342f","question":"A man is pushing a lawn mower through the grass. a child\n\nA. shows against the backdrop a pink help wanted ad.\nB. is standing next to him pushing a toy lawn mower.\nC. follows behind the man, covering the yard.\nD. is kneeling down next to him handling the lawn mower.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA man is pushing a lawn mower through the grass. a child\n\nA. shows against the backdrop a pink help wanted ad.\nB. is standing next to him pushing a toy lawn mower.\nC. follows behind the man, covering the yard.\nD. is kneeling down next to him handling the lawn mower.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":755,"error":null,"output":"B"}
{"question_id":"hellaswag:00002580","item_index":24,"attempt":0,"prompt_hash":"8eecb5bb77a2b1aa4a0c1003fb0cce49dafbdca755b07f254518e06073059f0e","question":"A person lays carpet down over hardwood floors using carpet laying tools and adhesive. a person on hands and knees\n\nA. hammers nails into a long wooden strip on the floor next to floorboards.\nB. places carpet down over carpet.\nC. rubs the rug along the back of it.\nD. attaches carpet to black carpet using carpet scraper and adhesive.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA person lays carpet down over hardwood floors using carpet laying tools and adhesive. a person on hands and knees\n\nA. hammers nails into a long wooden strip on the floor next to floorboards.\nB. places carpet down over carpet.\nC. rubs the rug along the back of it.\nD. attaches carpet to black carpet using carpet scraper and adhesive.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":709,"error":null,"output":"B"}
{"question_id":"hellaswag:00005202","item_index":25,"attempt":0,"prompt_hash":"cb18ed79b5c420e2cb90f28d02e099fee9743400239529a807dc61ab54880313","question":"[header] How to use cognitive behavioral therapy [title] Practice mindfulness to observe your thoughts and feelings. [step] You may believe that a bad situation leads to negative feelings. The cbt approach challenges this by stating that it is the thoughts we have that lead us to have those emotions.\n\nA. You get a rewarding response from a therapist if you observe calm within yourself and then ask questions. Focus on your thought when you start feeling angry, frustrated, embarrassed, or uncomfortable.\nB. A situation gives rise to a thought which in turn leads to a feeling or action. [substeps] Here is an example of how a positive outcome is linked to thought: you went to the gym and exercised.\nC. The focus is on recognizing the negative thoughts, which are the thoughts we choose to walk into situations to cope and not let them interfere with our life. [substeps] As a basic process develops over the course of becoming successful, practice mindfulness more frequently.\nD. Example: you believe you have been rude to a friend because you upset him. Instead of criticizing him, say to yourself, \" thank you for the explanation and for the apology.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to use cognitive behavioral therapy [title] Practice mindfulness to observe your thoughts and feelings. [step] You may believe that a bad situation leads to negative feelings. The cbt approach challenges this by stating that it is the thoughts we have that lead us to have those emotions.\n\nA. You get a rewarding response from a therapist if you observe calm within yourself and then ask questions. Focus on your thought when you start feeling angry, frustrated, embarrassed, or uncomfortable.\nB. A situation gives rise to a thought which in turn leads to a feeling or action. [substeps] Here is an example of how a positive outcome is linked to thought: you went to the gym and exercised.\nC. The focus is on recognizing the negative thoughts, which are the thoughts we choose to walk into situations to cope and not let them interfere with our life. [substeps] As a basic process develops over the course of becoming successful, practice mindfulness more frequently.\nD. Example: you believe you have been rude to a friend because you upset him. Instead of criticizing him, say to yourself, \" thank you for the explanation and for the apology.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1762,"error":null,"output":"C"}
{"question_id":"hellaswag:00001511","item_index":26,"attempt":0,"prompt_hash":"69f014e4610d433ff867683795d78b7ff1510db76b764e8a06eecb3907002389","question":"Trees blow in the breeze as a man mows the lawn. the sun\n\nA. peeks from behind the clouds and lights the scene up.\nB. drops below the trees as people cheer the man on.\nC. begins to set as a small parking lot full of cars is shown.\nD. begins to set on a day in june with numerous hollywood stars.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nTrees blow in the breeze as a man mows the lawn. the sun\n\nA. peeks from behind the clouds and lights the scene up.\nB. drops below the trees as people cheer the man on.\nC. begins to set as a small parking lot full of cars is shown.\nD. begins to set on a day in june with numerous hollywood stars.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":777,"error":null,"output":"A"}
{"question_id":"hellaswag:00009221","item_index":27,"attempt":0,"prompt_hash":"d18cc9289816b472937264b85c6d58a601f38d260c2476eb366615945c0f0383","question":"[header] How to make haystacks [title] Cover two large baking sheets with parchment paper or wax paper, and set them aside. [step] If you'd like to make bird nests instead, lightly grease the wells of a large muffin or cupcake tin instead. [title] Add the chocolate chips or butterscotch chips into a small saucepan.\n\nA. [step] Use an electric mixer with a paddle attachment to beat the ingredients until evenly combined. [title] Add the 3-4 liters of baking soda.\nB. [step] Simply add as much as necessary to the mixture, stirring occasionally. Do not exceed 2 tablespoons (30 ml) of milk for every 20 pounds (1.2 kg) of flour or eu/metric flour you use.\nC. [step] For traditional haystacks, use the butterscotch chips. If you don't like butterscotch, use chocolate chips instead.\nD. [step] Heat the ingredients on low for 30 to 60 seconds, stirring frequently. [title] Pour the flour, stirring constantly, into a big mixing bowl.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make haystacks [title] Cover two large baking sheets with parchment paper or wax paper, and set them aside. [step] If you'd like to make bird nests instead, lightly grease the wells of a large muffin or cupcake tin instead. [title] Add the chocolate chips or butterscotch chips into a small saucepan.\n\nA. [step] Use an electric mixer with a paddle attachment to beat the ingredients until evenly combined. [title] Add the 3-4 liters of baking soda.\nB. [step] Simply add as much as necessary to the mixture, stirring occasionally. Do not exceed 2 tablespoons (30 ml) of milk for every 20 pounds (1.2 kg) of flour or eu/metric flour you use.\nC. [step] For traditional haystacks, use the butterscotch chips. If you don't like butterscotch, use chocolate chips instead.\nD. [step] Heat the ingredients on low for 30 to 60 seconds, stirring frequently. [title] Pour the flour, stirring constantly, into a big mixing bowl.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1777,"error":null,"output":"C"}
{"question_id":"hellaswag:00009249","item_index":28,"attempt":0,"prompt_hash":"f001b1cfe271c2338edacc5f3b0117e857b13a15c7c10de5e6a41c963cf8a71c","question":"[header] How to clean burnt copper [title] Bring water and cleaning agent to a boil. [step] Fill a pot or pan with water. Add a few squirts of mild dish detergent and/or a generous splash of vinegar.\n\nA. Bring this mixture to a boil over medium heat. [substeps] If you have a store-bought solvent for your copper, such as hydrogen peroxide, you don't need to add it to the water.\nB. If you're cleaning anything other than a copper pot or pan (or cleaning the outside of a smaller pot or pan), place the item in the water. Set the burner to medium heat and bring to a boil.\nC. The water shouldn't be too cold, though. Warm or cold water doesn't work very well on copper.\nD. Turn your burner to high and bring the water to a boil over high heat. You'll need to keep the water at a simmer for about 30 minutes.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to clean burnt copper [title] Bring water and cleaning agent to a boil. [step] Fill a pot or pan with water. Add a few squirts of mild dish detergent and/or a generous splash of vinegar.\n\nA. Bring this mixture to a boil over medium heat. [substeps] If you have a store-bought solvent for your copper, such as hydrogen peroxide, you don't need to add it to the water.\nB. If you're cleaning anything other than a copper pot or pan (or cleaning the outside of a smaller pot or pan), place the item in the water. Set the burner to medium heat and bring to a boil.\nC. The water shouldn't be too cold, though. Warm or cold water doesn't work very well on copper.\nD. Turn your burner to high and bring the water to a boil over high heat. You'll need to keep the water at a simmer for about 30 minutes.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1616,"error":null,"output":"B"}
{"question_id":"hellaswag:00006819","item_index":29,"attempt":0,"prompt_hash":"8b67a144571d2c23adad8072b158e6b47c48fb6a9c42c43de20f9f13e6267649","question":"[header] How to set up an email address book [title] Log into your gmail account. [title] Click on the red gmail link, then click on the \" contacts \" button. [step] This should take you to your contacts list.\n\nA. [title] Select the \" password if prompted \" item from the drop-down menu. [step] You will be directed to a page where you can access password verification software or a keypad to enter the password manually.\nB. [title] Click on the name of the contact you are looking to set up the account with. [step] You will be asked for your account number.\nC. [title] Click on \" new contact \" at the left side. [title] Enter your new contact's details.\nD. [title] Click on the email address you'd like to set up your account for. [step] Copy this email address without erasing the address.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to set up an email address book [title] Log into your gmail account. [title] Click on the red gmail link, then click on the \" contacts \" button. [step] This should take you to your contacts list.\n\nA. [title] Select the \" password if prompted \" item from the drop-down menu. [step] You will be directed to a page where you can access password verification software or a keypad to enter the password manually.\nB. [title] Click on the name of the contact you are looking to set up the account with. [step] You will be asked for your account number.\nC. [title] Click on \" new contact \" at the left side. [title] Enter your new contact's details.\nD. [title] Click on the email address you'd like to set up your account for. [step] Copy this email address without erasing the address.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1581,"error":null,"output":"C"}
{"question_id":"hellaswag:00007139","item_index":30,"attempt":0,"prompt_hash":"6255ed5551c94d96de22920d8bee137abccb146863ab981a5bdb093cc075b5d5","question":"[header] How to be a virtuous woman [title] Understand that true beauty lies within. [step] True beauty is virtuous because true beauty develops out of good character and caring about the world. Before you consider looking virtuous, you need to look internally so that you actually become virtuous.\n\nA. Every human being is virtuous in his or her own way; up or down the line, on all levels. [substeps] True beauty isn't just in appearance or physical form.\nB. This means that if you are born really modest or didn't spend much money on such an internal appearance then you could develop a virtuous attitude. [substeps] Being virtuous means that you can be virtuous by turning all the negative feelings into positive ones, which leads to greater moral righteousness.\nC. [substeps] True beauty lies within what the fabric of your life actually is. An indication of your true nature will differ depending on your inner and outer self-image.\nD. [substeps] True inner beauty and virtue go hand-in-hand. If you practice the life of a virtuous woman, you will also be a beautiful woman.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to be a virtuous woman [title] Understand that true beauty lies within. [step] True beauty is virtuous because true beauty develops out of good character and caring about the world. Before you consider looking virtuous, you need to look internally so that you actually become virtuous.\n\nA. Every human being is virtuous in his or her own way; up or down the line, on all levels. [substeps] True beauty isn't just in appearance or physical form.\nB. This means that if you are born really modest or didn't spend much money on such an internal appearance then you could develop a virtuous attitude. [substeps] Being virtuous means that you can be virtuous by turning all the negative feelings into positive ones, which leads to greater moral righteousness.\nC. [substeps] True beauty lies within what the fabric of your life actually is. An indication of your true nature will differ depending on your inner and outer self-image.\nD. [substeps] True inner beauty and virtue go hand-in-hand. If you practice the life of a virtuous woman, you will also be a beautiful woman.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1770,"error":null,"output":"D"}
{"question_id":"hellaswag:00006798","item_index":31,"attempt":0,"prompt_hash":"c9e5af035dd6c97f58d28653c4ac72ddc034f6312c4b10fda7c78656e9bfd78d","question":"[header] How to remove weeds from a patio [title] Pull weeds from damp soil. [step] There are many product you can apply to weeds to kill them temporarily, but there is still no more effective and long-lasting way to get rid of weeds than to remove them by hand, as tedious as that can be. In order to remove weeds from your patio, start with wet soil.\n\nA. [substeps] Either weed your patio right after a heavy rain, or use a garden hose to moisten the soil in the cracks of your patio. [title] Use tools to get under the roots.\nB. Brush wet soil to remove the weeds and the clinging foliage from the cracks and crevices. Wipe the area down with a damp cloth or paper towel and throw away the cloth and whatever it's used for.\nC. Wet soil will flatten out the weeds. Just be careful it is not too dry.\nD. Place your shovel or trowel in the soil and make sure it is in the shape you want to achieve. Then shake off as much water as possible.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to remove weeds from a patio [title] Pull weeds from damp soil. [step] There are many product you can apply to weeds to kill them temporarily, but there is still no more effective and long-lasting way to get rid of weeds than to remove them by hand, as tedious as that can be. In order to remove weeds from your patio, start with wet soil.\n\nA. [substeps] Either weed your patio right after a heavy rain, or use a garden hose to moisten the soil in the cracks of your patio. [title] Use tools to get under the roots.\nB. Brush wet soil to remove the weeds and the clinging foliage from the cracks and crevices. Wipe the area down with a damp cloth or paper towel and throw away the cloth and whatever it's used for.\nC. Wet soil will flatten out the weeds. Just be careful it is not too dry.\nD. Place your shovel or trowel in the soil and make sure it is in the shape you want to achieve. Then shake off as much water as possible.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1739,"error":null,"output":"A"}
{"question_id":"hellaswag:00008849","item_index":32,"attempt":0,"prompt_hash":"1d4679028572cbc76ef453988287d76b824c4047025624f0415470c5ccde3724","question":"[header] How to keep your children and teens safe online [title] Place computers with internet access in a central location in the home. [step] When computers are in a central location, they can be easily monitored, but if placed in private places such as bedrooms, children can quickly close inappropriate websites when they hear their parents coming. [title] Get rid of the webcam.\n\nA. [step] It might seem like a last resort because it will easily block access to your online box. It might also cause your child to constantly check out your internet identity and send unwanted notifications.\nB. [step] Make sure all the webcam knobs are securely locked. [title] Disable video and picture uploads.\nC. [step] Webcams can be a great way to communicate with your friends and family, but leaving a teen unsupervised with a webcam can lead to your child's strip show debut. [title] Decide what online activities are age appropriate.\nD. [step] If a webcam is easily accessed in your home, don't allow your children and teens to use it. Inappropriate viewing may result in your children being caught.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to keep your children and teens safe online [title] Place computers with internet access in a central location in the home. [step] When computers are in a central location, they can be easily monitored, but if placed in private places such as bedrooms, children can quickly close inappropriate websites when they hear their parents coming. [title] Get rid of the webcam.\n\nA. [step] It might seem like a last resort because it will easily block access to your online box. It might also cause your child to constantly check out your internet identity and send unwanted notifications.\nB. [step] Make sure all the webcam knobs are securely locked. [title] Disable video and picture uploads.\nC. [step] Webcams can be a great way to communicate with your friends and family, but leaving a teen unsupervised with a webcam can lead to your child's strip show debut. [title] Decide what online activities are age appropriate.\nD. [step] If a webcam is easily accessed in your home, don't allow your children and teens to use it. Inappropriate viewing may result in your children being caught.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1709,"error":null,"output":"D"}
{"question_id":"hellaswag:00005341","item_index":33,"attempt":0,"prompt_hash":"bd56a1ae0de82ffce7eda464f946a02eb58899210e5abcd74914d0fe67b52250","question":"[header] How to fix knee pain [title] Lose weight if you're overweight. [step] People who are overweight, particularly the obese, experience more knee pain from increased compression on the joints. The increased pressure wears out the knees with time (arthritis) and also leads to more irritation and injury to ligaments and tendons.\n\nA. If you're overweight, stop eating. As with exercise, eating more often will add extra weight to your knees.\nB. Focus on eating a balanced diet that has protein in it and minerals from meat, vegetables, fruits, lean meats, and lean sources of fat. [substeps] This can cause lower cortisol levels and assist with menstruation.\nC. In addition, obese people are more likely to develop flat feet and fallen arches, which promotes \" knock knees \" and puts more strain on the outer (lateral) part of the knee joints. As such, losing weight takes pressure off your knee joints and reduces the likelihood of pain.\nD. Your doctor may suggest that you eat more lean protein (which can nourish your bones) to help maintain your strength. [substeps] Ask your doctor if any options for when you're working out have been tried.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to fix knee pain [title] Lose weight if you're overweight. [step] People who are overweight, particularly the obese, experience more knee pain from increased compression on the joints. The increased pressure wears out the knees with time (arthritis) and also leads to more irritation and injury to ligaments and tendons.\n\nA. If you're overweight, stop eating. As with exercise, eating more often will add extra weight to your knees.\nB. Focus on eating a balanced diet that has protein in it and minerals from meat, vegetables, fruits, lean meats, and lean sources of fat. [substeps] This can cause lower cortisol levels and assist with menstruation.\nC. In addition, obese people are more likely to develop flat feet and fallen arches, which promotes \" knock knees \" and puts more strain on the outer (lateral) part of the knee joints. As such, losing weight takes pressure off your knee joints and reduces the likelihood of pain.\nD. Your doctor may suggest that you eat more lean protein (which can nourish your bones) to help maintain your strength. [substeps] Ask your doctor if any options for when you're working out have been tried.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1820,"error":null,"output":"C"}
{"question_id":"hellaswag:00001697","item_index":34,"attempt":0,"prompt_hash":"0235c6dd8bb8184bc8306bacd12ea7070e201a1eac1319e434a513f3e4afd4b8","question":"The man throws frisbees for the dog to catch. The man performs special tricks with the dog using the frisbee. the man\n\nA. spins the dog in his hands and catches the frisbees and throw with the dog.\nB. jumps on the dog with the frisbee.\nC. lifts the dog triumphantly over his head and carries him on his shoulder.\nD. jumps on the dog on his back.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nThe man throws frisbees for the dog to catch. The man performs special tricks with the dog using the frisbee. the man\n\nA. spins the dog in his hands and catches the frisbees and throw with the dog.\nB. jumps on the dog with the frisbee.\nC. lifts the dog triumphantly over his head and carries him on his shoulder.\nD. jumps on the dog on his back.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":848,"error":null,"output":"C"}
{"question_id":"hellaswag:00007000","item_index":35,"attempt":0,"prompt_hash":"6363fc3bd3838ecadf22ecf060883c9f24e2f240fb0012ca42bdcc8e968b6160","question":"[header] How to attend an indonesian traditional wedding [title] Wear formal dress. [step] Females can wear almost anything proper, but pick the dress that is not revealing too much skin. Revealing your cleavage would be considered rude.\n\nA. However, showing cleavage would be very embarrassing if other people, including your boyfriend or significant other, could see it, and would likely embarrass you. [substeps] As always, always have a reservation if there is no other option.\nB. For males, it is a suggestion to wear batik and formal long pants. T-shirt and anything jeans is a no.\nC. Also in the event of a recent makeover, you should wear skirts with a modest neckline without looking trashy. This will make your outfit formal, yet still appropriate.\nD. You don't have to wear anything covering your breasts, as long as it's not showing as you are wearing your lingerie. Opt for evening wear, as that is usually considered more formal.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to attend an indonesian traditional wedding [title] Wear formal dress. [step] Females can wear almost anything proper, but pick the dress that is not revealing too much skin. Revealing your cleavage would be considered rude.\n\nA. However, showing cleavage would be very embarrassing if other people, including your boyfriend or significant other, could see it, and would likely embarrass you. [substeps] As always, always have a reservation if there is no other option.\nB. For males, it is a suggestion to wear batik and formal long pants. T-shirt and anything jeans is a no.\nC. Also in the event of a recent makeover, you should wear skirts with a modest neckline without looking trashy. This will make your outfit formal, yet still appropriate.\nD. You don't have to wear anything covering your breasts, as long as it's not showing as you are wearing your lingerie. Opt for evening wear, as that is usually considered more formal.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1610,"error":null,"output":"D"}
{"question_id":"hellaswag:00009574","item_index":36,"attempt":0,"prompt_hash":"d38aa58b59922f5bd5cc952136a05d23829befa5f2ffdeeb7864efaad0e61656","question":"[header] How to encourage parallel play [title] Recognize stages of play. [step] Young children go through several distinct stages when learning how to play and interact with others. Although kids go through these stages in order, they still use each type of play as they grow.\n\nA. So, a preschooler may engage in parallel play for a while before going off to play by themselves or join a group game. These are the stages you can expect to see in young children : [substeps] Independent or solitary play: this is when babies and one year olds play by themselves.\nB. Once your child reaches an understanding of the role, they can begin to learn how to interact with others. [substeps] Certain types of games tend to be timed more so because players are sometimes inclined to separate the children until the physical move involves them.\nC. [substeps] Whether your child plays soccer or basketball, they will be aware of the movements but will not only get familiar playing these games on their own, but each time someone engages a child in a game of that age group. People and children learn faster and more quickly when they get the chance to become themselves at children's ages.\nD. There are four stages that children play on. They are exploration, play, and response.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to encourage parallel play [title] Recognize stages of play. [step] Young children go through several distinct stages when learning how to play and interact with others. Although kids go through these stages in order, they still use each type of play as they grow.\n\nA. So, a preschooler may engage in parallel play for a while before going off to play by themselves or join a group game. These are the stages you can expect to see in young children : [substeps] Independent or solitary play: this is when babies and one year olds play by themselves.\nB. Once your child reaches an understanding of the role, they can begin to learn how to interact with others. [substeps] Certain types of games tend to be timed more so because players are sometimes inclined to separate the children until the physical move involves them.\nC. [substeps] Whether your child plays soccer or basketball, they will be aware of the movements but will not only get familiar playing these games on their own, but each time someone engages a child in a game of that age group. People and children learn faster and more quickly when they get the chance to become themselves at children's ages.\nD. There are four stages that children play on. They are exploration, play, and response.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1886,"error":null,"output":"B"}
{"question_id":"hellaswag:00007396","item_index":37,"attempt":0,"prompt_hash":"c4a10706518aa553efc92ca5250a900342d8c293ba8d2f227fc2d86c48453ffa","question":"[header] How to make legs bigger (for women ) [title] Eat more food. [step] If you're on a diet, you're going to have trouble building leg muscle. In fact, you won't build muscle unless you take in plenty of calories to feed muscle growth in your legs.\n\nA. This doesn't mean you should go wild and eat everything you want, but when your aim is to get bigger legs, getting more calories is essential. Eating 3 regular meals plus snacks filled with high quality foods will also help you gain some healthy weight, adding shape and definition to your legs.\nB. For women, try to eat 200-500 calories a day. For men, choose to eat about 150-200 meals per day.\nC. A diet plan, or a general eating plan, lists a specific amount of calories that you should eat every day. [substeps] For example, eat a piece of cheese, whole wheat pita chips, or whole wheat dry toast.\nD. You'll also need more time between workouts, so if you're planning on running days or doing bike rides, you should eat something extra indulgent. [substeps] Try protein shakes, pita chips, oatmeal, yogurt and vegetables.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make legs bigger (for women ) [title] Eat more food. [step] If you're on a diet, you're going to have trouble building leg muscle. In fact, you won't build muscle unless you take in plenty of calories to feed muscle growth in your legs.\n\nA. This doesn't mean you should go wild and eat everything you want, but when your aim is to get bigger legs, getting more calories is essential. Eating 3 regular meals plus snacks filled with high quality foods will also help you gain some healthy weight, adding shape and definition to your legs.\nB. For women, try to eat 200-500 calories a day. For men, choose to eat about 150-200 meals per day.\nC. A diet plan, or a general eating plan, lists a specific amount of calories that you should eat every day. [substeps] For example, eat a piece of cheese, whole wheat pita chips, or whole wheat dry toast.\nD. You'll also need more time between workouts, so if you're planning on running days or doing bike rides, you should eat something extra indulgent. [substeps] Try protein shakes, pita chips, oatmeal, yogurt and vegetables.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1895,"error":null,"output":"A"}
{"question_id":"hellaswag:00007532","item_index":38,"attempt":0,"prompt_hash":"3c4ceab1e6ba5ce5865e2a81929bd7e0f607b1fca0b22563806f7338e19103fa","question":"[header] How to have smooth and silky hair [title] Wash your hair a few times a week instead of every day. [step] In order for your hair to stay soft and smooth, it needs the natural conditioning of sebum, the oil that your scalp produces to keep your hair strong. If you wash your hair every day, you rinse away this nourishing oil, leaving your strands feeling dry and coarse instead of smooth and silky.\n\nA. [substeps] When you shampoo and condition your hair, work it through in small, multi-strand sections with your fingers. Use a texturizing spray each time.\nB. [substeps] Instead of shampoo, use conditioner. You can also use a loofah for this purpose.\nC. If you skip washing your hair, you won't be able to maintain that smooth and silky hair. [title] Follow what proper shampoo and conditioner you apply to your hair.\nD. [substeps] Find the washing routine that's right for you. Some people do it 2 or 3 times a week, while others wash every other day if their hair is especially oily.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to have smooth and silky hair [title] Wash your hair a few times a week instead of every day. [step] In order for your hair to stay soft and smooth, it needs the natural conditioning of sebum, the oil that your scalp produces to keep your hair strong. If you wash your hair every day, you rinse away this nourishing oil, leaving your strands feeling dry and coarse instead of smooth and silky.\n\nA. [substeps] When you shampoo and condition your hair, work it through in small, multi-strand sections with your fingers. Use a texturizing spray each time.\nB. [substeps] Instead of shampoo, use conditioner. You can also use a loofah for this purpose.\nC. If you skip washing your hair, you won't be able to maintain that smooth and silky hair. [title] Follow what proper shampoo and conditioner you apply to your hair.\nD. [substeps] Find the washing routine that's right for you. Some people do it 2 or 3 times a week, while others wash every other day if their hair is especially oily.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1761,"error":null,"output":"D"}
{"question_id":"hellaswag:00006075","item_index":39,"attempt":0,"prompt_hash":"a7512e5292d37b0e076645366af00f67c4f5d29dad439701cdee279a415f57d6","question":"[header] How to lace boots [title] Select a boot lace. [step] Most likely you have a pair of boots that go at least as high as your ankles. Make sure that you get boot laces that are long enough for your pair of boots.\n\nA. Most people who wear boots to work outdo boot laced or tailored boots. [substeps] Opt for a higher quality, more robust boot lace or something that is made for thicker boot laces, if possible.\nB. [substeps] Ask a sales clerk for advice or read the packaging to determine the right boot lace length. If you are getting new laces, measure the laces that came with your boots.\nC. Pre-laced boots will not hold up properly if you want your boots to be as stretchy and tight as you want them to be. Take a pair of pre-laced boots, or go wide-trip and pick up two or three boots.\nD. Remember, when lace is placed in the boot, it is more likely that it will slip up to make the fit that it needs for it to work. [substeps] Boots with a higher heel tend to be the easiest to lace, in this scenario, as boot laces will thread up the cuff of the boot instead of criss crossing, which cause more discomfort.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to lace boots [title] Select a boot lace. [step] Most likely you have a pair of boots that go at least as high as your ankles. Make sure that you get boot laces that are long enough for your pair of boots.\n\nA. Most people who wear boots to work outdo boot laced or tailored boots. [substeps] Opt for a higher quality, more robust boot lace or something that is made for thicker boot laces, if possible.\nB. [substeps] Ask a sales clerk for advice or read the packaging to determine the right boot lace length. If you are getting new laces, measure the laces that came with your boots.\nC. Pre-laced boots will not hold up properly if you want your boots to be as stretchy and tight as you want them to be. Take a pair of pre-laced boots, or go wide-trip and pick up two or three boots.\nD. Remember, when lace is placed in the boot, it is more likely that it will slip up to make the fit that it needs for it to work. [substeps] Boots with a higher heel tend to be the easiest to lace, in this scenario, as boot laces will thread up the cuff of the boot instead of criss crossing, which cause more discomfort.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1893,"error":null,"output":"B"}
{"question_id":"hellaswag:00003338","item_index":40,"attempt":0,"prompt_hash":"5598ea8490c8d02b1cec11c6c9258507b4919f8fb6502aacec8dfe70e0b684f6","question":"[header] How to choose makeup primer [title] Choose a hydrating makeup primer if you have dry skin. [step] If your skin begs for lotion after a shower, you definitely need to get a makeup primer for dry skin. A good, nourishing primer can take the place of your lotion, so you can hydrate your skin and prep it for makeup in one step.\n\nA. Look for primers with words like \" hydrating, \" \" soothing, \" or \" replenishing. \" [title] Pick a mattifying primer if you have oily skin.\nB. [title] Choose a neutral primer if you have combination skin. [step] If you have a combination skin, consider what it will look like after you apply blush, eye shadow, and lipstick.\nC. [substeps] If your oily skin is dry, hydrating your skin is not the best idea either. A good, hydrating primer is especially important to get into when wet.\nD. [title] Try a primer if you have oily skin. [step] Oily skin often resembles porcelain--oily, dry skin with a generally yellowish appearance.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to choose makeup primer [title] Choose a hydrating makeup primer if you have dry skin. [step] If your skin begs for lotion after a shower, you definitely need to get a makeup primer for dry skin. A good, nourishing primer can take the place of your lotion, so you can hydrate your skin and prep it for makeup in one step.\n\nA. Look for primers with words like \" hydrating, \" \" soothing, \" or \" replenishing. \" [title] Pick a mattifying primer if you have oily skin.\nB. [title] Choose a neutral primer if you have combination skin. [step] If you have a combination skin, consider what it will look like after you apply blush, eye shadow, and lipstick.\nC. [substeps] If your oily skin is dry, hydrating your skin is not the best idea either. A good, hydrating primer is especially important to get into when wet.\nD. [title] Try a primer if you have oily skin. [step] Oily skin often resembles porcelain--oily, dry skin with a generally yellowish appearance.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1779,"error":null,"output":"A"}
{"question_id":"hellaswag:00007216","item_index":41,"attempt":0,"prompt_hash":"16ab38789f492e2ca0172e2edce8bc81f93aab96118e7232bc9b9e022aba0159","question":"[header] How to sun dry fruit [title] Make a drying tray with a plastic screen or using cheesecloth stretched tightly on a frame. [step] Crisscross string on the back of the frame and staple it in the corners to prevent the cheesecloth from sagging. [title] Harvest the fruit just before it becomes over-ripe.\n\nA. [step] Be sure to spray it lightly with an insecticide or another fertilizing agent. [title] Dry the fruit between the hours of 11 pm and 3 pm.\nB. [step] The peel will naturally begin to ripen before you peel it, so save the fruit to use when frying later. [title] Heat water in a large pot and turn the heat to low.\nC. [step] Select fruit that is free from blemishes. [title] Allow the fruit to cool if it is warm from growing in the sun.\nD. [step] Otherwise, leave the fruit and eat it later. [title] Use a pole to stretch strips of the fruit skin-side down if necessary.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to sun dry fruit [title] Make a drying tray with a plastic screen or using cheesecloth stretched tightly on a frame. [step] Crisscross string on the back of the frame and staple it in the corners to prevent the cheesecloth from sagging. [title] Harvest the fruit just before it becomes over-ripe.\n\nA. [step] Be sure to spray it lightly with an insecticide or another fertilizing agent. [title] Dry the fruit between the hours of 11 pm and 3 pm.\nB. [step] The peel will naturally begin to ripen before you peel it, so save the fruit to use when frying later. [title] Heat water in a large pot and turn the heat to low.\nC. [step] Select fruit that is free from blemishes. [title] Allow the fruit to cool if it is warm from growing in the sun.\nD. [step] Otherwise, leave the fruit and eat it later. [title] Use a pole to stretch strips of the fruit skin-side down if necessary.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1685,"error":null,"output":"C"}
{"question_id":"hellaswag:00006737","item_index":42,"attempt":0,"prompt_hash":"f71a40e26ad110a903448bd3175b6e3464d49d21e42f299fc9df6d45e3f5196e","question":"[header] How to clean your phone screen [title] Get a microfiber cloth. [step] This extra-soft, lint-free cloth cleans glass and plastic without scratching the surface. You can buy one at stores that sell computers, phones, or cameras.\n\nA. If you wear eyeglasses, you might have got a free microfiber cloth when you bought your last pair. [substeps] The next best substitute is a soft , 100% cotton cloth or t-shirt.\nB. Microfiber cloths have soft, lint-free bristles. [substeps] Make sure that you use the microfiber cloth; otherwise you can scratch the glass and damage the screen.\nC. Plastic, however, does contain allergens that can rub off and damage the screen. If you have to clean your phone with a microfiber cloth, you should also soak it in a small amount of water.\nD. [substeps] Get a disposable t-shirt, wool/cotton paper and a commercial plastic grocery or photo-store microfiber cloth. Wet the paper and wipe the areas thoroughly.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to clean your phone screen [title] Get a microfiber cloth. [step] This extra-soft, lint-free cloth cleans glass and plastic without scratching the surface. You can buy one at stores that sell computers, phones, or cameras.\n\nA. If you wear eyeglasses, you might have got a free microfiber cloth when you bought your last pair. [substeps] The next best substitute is a soft , 100% cotton cloth or t-shirt.\nB. Microfiber cloths have soft, lint-free bristles. [substeps] Make sure that you use the microfiber cloth; otherwise you can scratch the glass and damage the screen.\nC. Plastic, however, does contain allergens that can rub off and damage the screen. If you have to clean your phone with a microfiber cloth, you should also soak it in a small amount of water.\nD. [substeps] Get a disposable t-shirt, wool/cotton paper and a commercial plastic grocery or photo-store microfiber cloth. Wet the paper and wipe the areas thoroughly.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1710,"error":null,"output":"A"}
{"question_id":"hellaswag:00005878","item_index":43,"attempt":0,"prompt_hash":"a84ff18e06bbfb4f26a5c8671e2101c7399a13133d92125e8708bfeca8da956c","question":"[header] How to keep weeds out of your vegetable garden [title] Cut with a sharp hoe. [step] A sharpened hoe blade allows you to sever the weeds without having to bend over or crouch. Swing the blade against the weed near the base, then leave the weed to decompose.\n\nA. Most gardens come with a tool box or bucket to hold your tools and a butter knife. [substeps] Most rows with looser weeds are growing due to long rows of weeds.\nB. Keep a hoe with thick bristles, protecting from the sun with your hands to stop weed growth. [substeps] Use the hoe to pull weeds along your garden for this purpose.\nC. [substeps] Splitting dandelion roots will produce tuberculosis, which can form naturally. Keep in mind that rooting weeds to promote well-draining soil changes the soil.\nD. If vegetables are already growing, a thin-bladed \" onion hoe \" may be easiest to maneuver without damaging the useful plants. [substeps] If the weed already has visible seed pods or seed heads, pull these off before you cut, and discard them in a trash can covered or far away from your garden.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to keep weeds out of your vegetable garden [title] Cut with a sharp hoe. [step] A sharpened hoe blade allows you to sever the weeds without having to bend over or crouch. Swing the blade against the weed near the base, then leave the weed to decompose.\n\nA. Most gardens come with a tool box or bucket to hold your tools and a butter knife. [substeps] Most rows with looser weeds are growing due to long rows of weeds.\nB. Keep a hoe with thick bristles, protecting from the sun with your hands to stop weed growth. [substeps] Use the hoe to pull weeds along your garden for this purpose.\nC. [substeps] Splitting dandelion roots will produce tuberculosis, which can form naturally. Keep in mind that rooting weeds to promote well-draining soil changes the soil.\nD. If vegetables are already growing, a thin-bladed \" onion hoe \" may be easiest to maneuver without damaging the useful plants. [substeps] If the weed already has visible seed pods or seed heads, pull these off before you cut, and discard them in a trash can covered or far away from your garden.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1850,"error":null,"output":"B"}
{"question_id":"hellaswag:00003581","item_index":44,"attempt":0,"prompt_hash":"d4872be87401cbd90721c5030ddfb922456f1320c33a8cadda46cdc3190a8f3f","question":"[header] How to choose the best hair products for fine hair [title] Wash and condition fine hair with a hair thickening shampoo and conditioner. [step] Look for products that claim to add thickness, volume or body to fine or thin hair. [title] Choose styling products that are specifically designed for fine hair.\n\nA. [step] A lot of these don't end up evenly distributed throughout coarse hair. Picking out products that say \" heat protection \" on the package will give you a great deal of easier hair to work with.\nB. [step] Adding volume will help you to grow more effectively without looking boyish. [substeps] Ask your hair stylist about different styling products so you can discuss the products with them.\nC. [step] At the very least, be sure to use a hair thinning shampoo and conditioner that will not cause your hair to break down as you work with your hair. [substeps] Products with ingredients such as curl enhancing or deep conditioning can help create weak-over roots if you've already tried several products.\nD. [step] There are many fine hair products on the market, including mousses, gels, root lifts, sprays and serums. Look for tag words such as \" volumizing, \" \" thickening, \" \" bodifying \" and \" texturizing.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to choose the best hair products for fine hair [title] Wash and condition fine hair with a hair thickening shampoo and conditioner. [step] Look for products that claim to add thickness, volume or body to fine or thin hair. [title] Choose styling products that are specifically designed for fine hair.\n\nA. [step] A lot of these don't end up evenly distributed throughout coarse hair. Picking out products that say \" heat protection \" on the package will give you a great deal of easier hair to work with.\nB. [step] Adding volume will help you to grow more effectively without looking boyish. [substeps] Ask your hair stylist about different styling products so you can discuss the products with them.\nC. [step] At the very least, be sure to use a hair thinning shampoo and conditioner that will not cause your hair to break down as you work with your hair. [substeps] Products with ingredients such as curl enhancing or deep conditioning can help create weak-over roots if you've already tried several products.\nD. [step] There are many fine hair products on the market, including mousses, gels, root lifts, sprays and serums. Look for tag words such as \" volumizing, \" \" thickening, \" \" bodifying \" and \" texturizing.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1966,"error":null,"output":"D"}
{"question_id":"hellaswag:00007533","item_index":45,"attempt":0,"prompt_hash":"cde1a294e8f2010576650692aebaa6bc5a10cf09c32564d14d577566e981b3e0","question":"[header] How to make loose face powder into compact at home [title] Find a suitable container. [step] Preferably use an old, cleaned out compact or a new one but you could also use a pillbox, flat plastic container, and so on. The main thing is that the item be clean and dry.\n\nA. Any clothes are good as long as the item is dry. A sponge will do, but you could also consider using a cloth or your fingers instead.\nB. Also you don't want to mess with the bottom of the container and smell bad. [title] Blend the powder and powder in a blender until the good ingredients are blended together.\nC. Clean and dry your face everyday, from mid-morning to mid-afternoon. Wear cool colours but don't let the temperature get to top 25 ° f (135.3 ° c).\nD. [title] Spread paper towel down over the workspace. [step] It is probable that some powder will spill and this makes it easier to clean up after.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make loose face powder into compact at home [title] Find a suitable container. [step] Preferably use an old, cleaned out compact or a new one but you could also use a pillbox, flat plastic container, and so on. The main thing is that the item be clean and dry.\n\nA. Any clothes are good as long as the item is dry. A sponge will do, but you could also consider using a cloth or your fingers instead.\nB. Also you don't want to mess with the bottom of the container and smell bad. [title] Blend the powder and powder in a blender until the good ingredients are blended together.\nC. Clean and dry your face everyday, from mid-morning to mid-afternoon. Wear cool colours but don't let the temperature get to top 25 ° f (135.3 ° c).\nD. [title] Spread paper towel down over the workspace. [step] It is probable that some powder will spill and this makes it easier to clean up after.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1707,"error":null,"output":"B"}
{"question_id":"hellaswag:00001801","item_index":46,"attempt":0,"prompt_hash":"37fc0de333d0be01f2be27a153d3f210b059cb85d4a311e3472d3f40248b4864","question":"Men is standing in stage playing frums. blonde woman wearnig a white dress\n\nA. and men is talking maging kickball behind her.\nB. is playing violin walking around the stage.\nC. in stage.\nD. and start partying with people.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nMen is standing in stage playing frums. blonde woman wearnig a white dress\n\nA. and men is talking maging kickball behind her.\nB. is playing violin walking around the stage.\nC. in stage.\nD. and start partying with people.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":842,"error":null,"output":"B. is playing violin walking around the stage."}
{"question_id":"hellaswag:00004891","item_index":47,"attempt":0,"prompt_hash":"d30e09d1b561b9ab9eb6d0f9b809a41c67179535ea35e3ba7443ad8e1ab67ace","question":"[header] How to apply a bright eyeliner look [title] Start by using moisturizer and any acne or blemish creams. [step] Blend them into your skin with your fingers, using circular motions. [title] Apply a bb cream.\n\nA. [step] Bb cream is a fairly mild acne cleanser that can be applied to everyday skin and chosen whenever you want to be mysterious. Simply put a little bb cream on any blemishes you want to create a mysterious indent.\nB. [step] This has a lighter coverage, so it won't look as chalky. The eyes are the attention drawer in this look, so everything else needs to fall behind and blend.\nC. [step] Bb cream is a rich product that has tiny, white circles and darker coverage than bb cream. Additionally, bb cream produces a stronger, fuller shine.\nD. [step] Use your fingers to blend the cream into your skin, making your skin look moisturized. Use a bright, non-distracting and minimal eyeliner.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to apply a bright eyeliner look [title] Start by using moisturizer and any acne or blemish creams. [step] Blend them into your skin with your fingers, using circular motions. [title] Apply a bb cream.\n\nA. [step] Bb cream is a fairly mild acne cleanser that can be applied to everyday skin and chosen whenever you want to be mysterious. Simply put a little bb cream on any blemishes you want to create a mysterious indent.\nB. [step] This has a lighter coverage, so it won't look as chalky. The eyes are the attention drawer in this look, so everything else needs to fall behind and blend.\nC. [step] Bb cream is a rich product that has tiny, white circles and darker coverage than bb cream. Additionally, bb cream produces a stronger, fuller shine.\nD. [step] Use your fingers to blend the cream into your skin, making your skin look moisturized. Use a bright, non-distracting and minimal eyeliner.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1673,"error":null,"output":"D."}
{"question_id":"hellaswag:00006032","item_index":48,"attempt":0,"prompt_hash":"59b6b0e7a6b9df8371f7c92a6156a0a5ad5d6bc8fbe7f1455fecd1de6e89ab4c","question":"[header] How to become a judge [title] Obtain a bachelor's degree from a 4-year university. [step] To prepare yourself for getting into a top-notch law school, attend the best university possible. Ivy league schools such as harvard, yale and princeton will prepare you well for law school.\n\nA. Applying to law schools may require a track record of at least a bachelor's degree. That means still have a high school diploma; without a legal degree you would not be an a student in law.\nB. Completing college and showing a high 4.0 gpa can also help make you successful at law school. [substeps] High schools tend to produce job listings by the time you graduate law school, so you'll need to try on several to find one that meets your personality.\nC. You don't have to choose a liberal arts college--focus on the strongest law in your area for you and your associates. It will also help you train your students, widen your focus and you will likely attend clinics, seminars and training sessions.\nD. However, prestigious law schools often accept students from much smaller undergraduate institutions. Making sure that you participate in extracurricular activities like debate, and preparing well for the lsat, are the most important things to remember while earning your bachelor's degree.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to become a judge [title] Obtain a bachelor's degree from a 4-year university. [step] To prepare yourself for getting into a top-notch law school, attend the best university possible. Ivy league schools such as harvard, yale and princeton will prepare you well for law school.\n\nA. Applying to law schools may require a track record of at least a bachelor's degree. That means still have a high school diploma; without a legal degree you would not be an a student in law.\nB. Completing college and showing a high 4.0 gpa can also help make you successful at law school. [substeps] High schools tend to produce job listings by the time you graduate law school, so you'll need to try on several to find one that meets your personality.\nC. You don't have to choose a liberal arts college--focus on the strongest law in your area for you and your associates. It will also help you train your students, widen your focus and you will likely attend clinics, seminars and training sessions.\nD. However, prestigious law schools often accept students from much smaller undergraduate institutions. Making sure that you participate in extracurricular activities like debate, and preparing well for the lsat, are the most important things to remember while earning your bachelor's degree.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1980,"error":null,"output":"D"}
{"question_id":"hellaswag:00003394","item_index":49,"attempt":0,"prompt_hash":"5c3bac93c6c530f21f7938fb46617fc9de6ca5d247f88f2520b62e4847908057","question":"[header] How to wear mirror sunglasses [title] Determine the best frame for your face. [step] Some shapes look fantastic on round faces, while others look best on heart-shaped faces, and so on. The sunnies that your best friend pulls off flawlessly might not work for you.\n\nA. In this case, finding and purchasing a perfect frame. [substeps] First, you need to consider what shape you want your face to be.\nB. Consider making sure that the frames you choose are smooth and even when they are closed and you won't have to worry about them taking the focus away from you. [title] Add yourself to the \" never mind the pair \" screen.\nC. Make your sunglass shopping experience easiest by coming armed with the knowledge about which shapes best flatter your face. [substeps] If you have a heart-shaped face, look for square, cat eye, or sport sunglasses.\nD. Instead, select a pair with a good frame. [substeps] If you're looking for a style that's somewhat blocky, go for a pair that will make a small area of your face flush.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to wear mirror sunglasses [title] Determine the best frame for your face. [step] Some shapes look fantastic on round faces, while others look best on heart-shaped faces, and so on. The sunnies that your best friend pulls off flawlessly might not work for you.\n\nA. In this case, finding and purchasing a perfect frame. [substeps] First, you need to consider what shape you want your face to be.\nB. Consider making sure that the frames you choose are smooth and even when they are closed and you won't have to worry about them taking the focus away from you. [title] Add yourself to the \" never mind the pair \" screen.\nC. Make your sunglass shopping experience easiest by coming armed with the knowledge about which shapes best flatter your face. [substeps] If you have a heart-shaped face, look for square, cat eye, or sport sunglasses.\nD. Instead, select a pair with a good frame. [substeps] If you're looking for a style that's somewhat blocky, go for a pair that will make a small area of your face flush.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1771,"error":null,"output":"C"}
{"question_id":"hellaswag:00005690","item_index":50,"attempt":0,"prompt_hash":"95cae9770b882830d63c1311439178e9cd499c222f1127a3062323922be3bacb","question":"[header] How to massage someone's back [title] Get a bed. [step] The first thing you will need is a nice, clean bed (firm is best). Fresh linens are essential.\n\nA. Many cats want to sleep under a bed, so go out and buy one, or buy the best mattress for your particular type of cat. The bottom of the mattress should be made small enough for the cat to burrow in and not get too heavy.\nB. [title] Ask the client to lie down on his/her stomach, with their shirt off. [title] Clean and disinfect your hands.\nC. Also make sure to find a good sink or tub. Also make sure the area where you will be performing the massage is organized.\nD. You don't want to dehydrate the person, eating at least two meals a day is recommended. A good idea for this is to have a curtain in the room.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to massage someone's back [title] Get a bed. [step] The first thing you will need is a nice, clean bed (firm is best). Fresh linens are essential.\n\nA. Many cats want to sleep under a bed, so go out and buy one, or buy the best mattress for your particular type of cat. The bottom of the mattress should be made small enough for the cat to burrow in and not get too heavy.\nB. [title] Ask the client to lie down on his/her stomach, with their shirt off. [title] Clean and disinfect your hands.\nC. Also make sure to find a good sink or tub. Also make sure the area where you will be performing the massage is organized.\nD. You don't want to dehydrate the person, eating at least two meals a day is recommended. A good idea for this is to have a curtain in the room.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1589,"error":null,"output":"B"}
{"question_id":"hellaswag:00004430","item_index":51,"attempt":0,"prompt_hash":"816984ac2ba4161505782f6b666102d754babaa3b24f838b539d16354f0e703f","question":"[header] How to dissolve a corporation [title] Vote to dissolve. [step] Read your by-laws or articles of incorporation to find the procedures for dissolving the corporation. Typically, you will need the board of directors or shareholders to vote in order to authorize the dissolution.\n\nA. [substeps] Because bankruptcy can be complicated, it is often best to draft your notice of dissolution in the presence of a judge. [title] File the papers to dissolve a corporation.\nB. [title] Identify the owners of the company and identify the debtor. [step] For instance, if the company owns bank accounts, you would vote to dissolve those accounts.\nC. You will also be given notice of the dissolution of the corporation. All articles of incorporation must address this waiver.\nD. [substeps] If your governing documents are silent about the dissolution process, then you will need to read your state's statute. You can find it either at a law library or by searching online.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to dissolve a corporation [title] Vote to dissolve. [step] Read your by-laws or articles of incorporation to find the procedures for dissolving the corporation. Typically, you will need the board of directors or shareholders to vote in order to authorize the dissolution.\n\nA. [substeps] Because bankruptcy can be complicated, it is often best to draft your notice of dissolution in the presence of a judge. [title] File the papers to dissolve a corporation.\nB. [title] Identify the owners of the company and identify the debtor. [step] For instance, if the company owns bank accounts, you would vote to dissolve those accounts.\nC. You will also be given notice of the dissolution of the corporation. All articles of incorporation must address this waiver.\nD. [substeps] If your governing documents are silent about the dissolution process, then you will need to read your state's statute. You can find it either at a law library or by searching online.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1646,"error":null,"output":"C"}
{"question_id":"hellaswag:00005729","item_index":52,"attempt":0,"prompt_hash":"9a6556ebdf18606e774ca6ecab245ece37d45d6134250c957f6461666ad42e46","question":"[header] How to make fish pie [title] Cut raw boneless, skinless salmon into bite-size chunks. [step] Then cut raw undyed smoked haddock fillets with the bones and skin removed into bite-size pieces. [title] Substitute cod in place of the salmon, if you prefer.\n\nA. [title] Remove the guts from 1 lb-3/4 of the haddock fillets. [step] Place each fillet in a bowl covered with hot running water and cover with plastic wrap or aluminum foil to speed up the cooking process.\nB. [title] Use any white fish, such as cod, alone in the recipe for a more uniform fish flavor. [title] Add the fish pieces into the 9x13 inch baking dish.\nC. [step] Cod is a strong, recognizable fish that has a belly fat that helps the fish hang more easily and may last longer. [substeps] Soak the fillets in cold water for 12-15 minutes.\nD. [step] If you want to give your delicious fish pie a savory sweetness, a boneless and skinless cod fillet are both ideal. [substeps] If you like a little bit of fish in your fish pie, substitute half of that instead.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make fish pie [title] Cut raw boneless, skinless salmon into bite-size chunks. [step] Then cut raw undyed smoked haddock fillets with the bones and skin removed into bite-size pieces. [title] Substitute cod in place of the salmon, if you prefer.\n\nA. [title] Remove the guts from 1 lb-3/4 of the haddock fillets. [step] Place each fillet in a bowl covered with hot running water and cover with plastic wrap or aluminum foil to speed up the cooking process.\nB. [title] Use any white fish, such as cod, alone in the recipe for a more uniform fish flavor. [title] Add the fish pieces into the 9x13 inch baking dish.\nC. [step] Cod is a strong, recognizable fish that has a belly fat that helps the fish hang more easily and may last longer. [substeps] Soak the fillets in cold water for 12-15 minutes.\nD. [step] If you want to give your delicious fish pie a savory sweetness, a boneless and skinless cod fillet are both ideal. [substeps] If you like a little bit of fish in your fish pie, substitute half of that instead.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1911,"error":null,"output":"B"}
{"question_id":"hellaswag:00005428","item_index":53,"attempt":0,"prompt_hash":"ce60ba00a38769ec31b6639fded4a795f60f80595a82483f303c7f9621961d6b","question":"[header] How to replace hardwood floor [title] Put on gloves, knee pads, and protective eye glasses. [step] Make single cuts straight across the flooring that are about 1 foot (30 cm) to 2 feet (61 cm) apart. Be sure your cuts are perpendicular to the direction of the wood.\n\nA. Make every cut across the entire floor in 4-5 even strokes. Repeat this process with all of the floors in your home.\nB. [title] Use a long piece of industrial door branch and exterior putty. [step] Take one end of the door branch and use the base strike direction to make chalk impressions on the floor.\nC. It's okay if you cannot see the cuts clearly, as you cannot let the wood paint on the surface that you work in come into direct contact with your skin. [substeps] Before filling in the cuts, make sure you and your co workers are wearing the appropriate kinds of eye shields and protective eye glasses.\nD. Start on 1 side of the room and work your way systematically to the other side, spacing each cut about 1 foot (30 cm) to 2 feet (61 cm) apart. [substeps] Always keep your hands away from the blade of a circular saw.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to replace hardwood floor [title] Put on gloves, knee pads, and protective eye glasses. [step] Make single cuts straight across the flooring that are about 1 foot (30 cm) to 2 feet (61 cm) apart. Be sure your cuts are perpendicular to the direction of the wood.\n\nA. Make every cut across the entire floor in 4-5 even strokes. Repeat this process with all of the floors in your home.\nB. [title] Use a long piece of industrial door branch and exterior putty. [step] Take one end of the door branch and use the base strike direction to make chalk impressions on the floor.\nC. It's okay if you cannot see the cuts clearly, as you cannot let the wood paint on the surface that you work in come into direct contact with your skin. [substeps] Before filling in the cuts, make sure you and your co workers are wearing the appropriate kinds of eye shields and protective eye glasses.\nD. Start on 1 side of the room and work your way systematically to the other side, spacing each cut about 1 foot (30 cm) to 2 feet (61 cm) apart. [substeps] Always keep your hands away from the blade of a circular saw.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1916,"error":null,"output":"A"}
{"question_id":"hellaswag:00001401","item_index":54,"attempt":0,"prompt_hash":"bd05b41b9573e62257ee5f6f53ee5d2813799a79d7a9a092787b0e7eceaf359c","question":"The white rabbit is in the cage. The rabbit is cleaning its hands by licking it. the rabbit\n\nA. acts ecstatic at being given a prize.\nB. picks its legs up.\nC. cleaned his back and then wend down to the first floor of his cage.\nD. stands up and walks back to brush its fur.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nThe white rabbit is in the cage. The rabbit is cleaning its hands by licking it. the rabbit\n\nA. acts ecstatic at being given a prize.\nB. picks its legs up.\nC. cleaned his back and then wend down to the first floor of his cage.\nD. stands up and walks back to brush its fur.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1047,"error":null,"output":"D. stands up and walks back to brush its fur."}
{"question_id":"hellaswag:00004550","item_index":55,"attempt":0,"prompt_hash":"edcf86b7cb345eb9ceb437f7c01b3a5034f4e4c9feebb75504bcd3f2f4496a57","question":"[header] How to treat upper respiratory infections in ferrets [title] Wait for your ferret to recover. [step] In most cases, upper respiratory infections disappear in a few days. Give your ferret time to rest and heal.\n\nA. Give it more time if it is sore. [substeps] For dwarf ferrets , 8 to 10 hours is usually good.\nB. The sooner you treat your ferret, the better. [substeps] Ferrets recovering from upper respiratory infections are a very vulnerable breed who may be susceptible to other health problems.\nC. Don't try to train or play with your ferret when it is suffering from an upper respiratory infection. [title] Wipe your ferret's nose before meals if it is runny.\nD. The ferret's immune system is strong and you'll want to avoid aggravating it. [substeps] Hamsters who exhibit upper respiratory infections in the first 2 to 4 weeks of their lives tend to recover faster.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to treat upper respiratory infections in ferrets [title] Wait for your ferret to recover. [step] In most cases, upper respiratory infections disappear in a few days. Give your ferret time to rest and heal.\n\nA. Give it more time if it is sore. [substeps] For dwarf ferrets , 8 to 10 hours is usually good.\nB. The sooner you treat your ferret, the better. [substeps] Ferrets recovering from upper respiratory infections are a very vulnerable breed who may be susceptible to other health problems.\nC. Don't try to train or play with your ferret when it is suffering from an upper respiratory infection. [title] Wipe your ferret's nose before meals if it is runny.\nD. The ferret's immune system is strong and you'll want to avoid aggravating it. [substeps] Hamsters who exhibit upper respiratory infections in the first 2 to 4 weeks of their lives tend to recover faster.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1676,"error":null,"output":"A"}
{"question_id":"hellaswag:00000736","item_index":56,"attempt":0,"prompt_hash":"69e5a115e24bf82eccb9c20677ab7c7ece5e77618b5f2c5d0e83b31e70126539","question":"Two people get into an truck. They drive onto the street. a man carrying a ladder\n\nA. goes into an emergency room.\nB. is pushing a lawn mower.\nC. sets it against a house.\nD. brings a box into the house.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nTwo people get into an truck. They drive onto the street. a man carrying a ladder\n\nA. goes into an emergency room.\nB. is pushing a lawn mower.\nC. sets it against a house.\nD. brings a box into the house.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":585,"error":null,"output":"B"}
{"question_id":"hellaswag:00007701","item_index":57,"attempt":0,"prompt_hash":"6644400476016104a4000cf557749e54d30eb2556ad0874751724fad0b9f07a4","question":"[header] How to dress for a curvaceous look [title] Wear a properly fitting bra. [step] This is so crucial, whether you're working with a large or small bust. If your bra is too small you'll be squishing your breasts, and if it's too large, you won't be supporting them.\n\nA. Add some support to the bra by running yourself through the motions. After the cardio, that's when good stuff comes in! [title] Avoid wearing support bras.\nB. [substeps] Always wear a bra that's snug around your stomach. This will make your breasts look proportionate rather than smaller as well.\nC. To really accentuate your bust you should wear a push-up bra, which will lift your breasts and create some intense cleavage. Girls with naturally small busts should choose push-up bras with extra padding as well, to increase the overall size.\nD. Instead, check your bra sizing for your size before fitting it. Follow these tips to fit your bra properly : [substeps] For a small bust, wear a strapless bra.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to dress for a curvaceous look [title] Wear a properly fitting bra. [step] This is so crucial, whether you're working with a large or small bust. If your bra is too small you'll be squishing your breasts, and if it's too large, you won't be supporting them.\n\nA. Add some support to the bra by running yourself through the motions. After the cardio, that's when good stuff comes in! [title] Avoid wearing support bras.\nB. [substeps] Always wear a bra that's snug around your stomach. This will make your breasts look proportionate rather than smaller as well.\nC. To really accentuate your bust you should wear a push-up bra, which will lift your breasts and create some intense cleavage. Girls with naturally small busts should choose push-up bras with extra padding as well, to increase the overall size.\nD. Instead, check your bra sizing for your size before fitting it. Follow these tips to fit your bra properly : [substeps] For a small bust, wear a strapless bra.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1712,"error":null,"output":"C"}
{"question_id":"hellaswag:00007112","item_index":58,"attempt":0,"prompt_hash":"bc08579d539f26e4a06f47fc1820f4205ef7040e61ee5347648ba4e30345629f","question":"[header] How to find the publisher of a book [title] Look for the publisher's name on the title page. [step] The name of the publisher should appear at the bottom of the title page, under the title and author of the book. The publisher's name will usually be listed under the logo for the publisher.\n\nA. [substeps] The title of the book may appear under a suffix such as \" isbn \" or \" publisher \", or is preceded by a comma. It is usually listed below a comma for publisher.\nB. [title] Check the copyright page for the publisher's name. [step] The copyright page may appear on the back of the title page or on the following page.\nC. The word \" publisher \" should also be in the name of the book. [substeps] The publisher should also be listed before the authors' names, at the beginning of each publication page.\nD. Try to find the author's last name when you add the publisher. [substeps] If you are unsure, check the publisher's website for more information.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to find the publisher of a book [title] Look for the publisher's name on the title page. [step] The name of the publisher should appear at the bottom of the title page, under the title and author of the book. The publisher's name will usually be listed under the logo for the publisher.\n\nA. [substeps] The title of the book may appear under a suffix such as \" isbn \" or \" publisher \", or is preceded by a comma. It is usually listed below a comma for publisher.\nB. [title] Check the copyright page for the publisher's name. [step] The copyright page may appear on the back of the title page or on the following page.\nC. The word \" publisher \" should also be in the name of the book. [substeps] The publisher should also be listed before the authors' names, at the beginning of each publication page.\nD. Try to find the author's last name when you add the publisher. [substeps] If you are unsure, check the publisher's website for more information.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1710,"error":null,"output":"B"}
{"question_id":"hellaswag:00002789","item_index":59,"attempt":0,"prompt_hash":"670631c5c26e38396a7a3b829caa31191de5123bfbd61f997405e8de0b853415","question":"She grabs a paper towel and wipes the sink. She turns on the water and wets the sink. She grabs another paper towel and wipes the sink. she\n\nA. grabs a glass and wet it under the sink.\nB. lathers up her hand and drains it.\nC. finishes wiping the sink and speaks.\nD. puts it under the faucet and rinses it off.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nShe grabs a paper towel and wipes the sink. She turns on the water and wets the sink. She grabs another paper towel and wipes the sink. she\n\nA. grabs a glass and wet it under the sink.\nB. lathers up her hand and drains it.\nC. finishes wiping the sink and speaks.\nD. puts it under the faucet and rinses it off.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1135,"error":null,"output":"C. finishes wiping the sink and speaks."}
{"question_id":"hellaswag:00001937","item_index":60,"attempt":0,"prompt_hash":"7de32af3c47c6e3ec63f741166af8295987528937b3bc0268916a4648ff5c00b","question":"A man stands in a driveway covered with snow talking and holding a shovel. then\n\nA. , the man puts snow over the car, put snow cover around the car and points to bare areas while talking.\nB. , the man shovel the snow on the driveway and put in a pile.\nC. , the man opens the driveway of a house and reveal snow covering a car.\nD. , the man passes in the driveway and continues cutting the snow.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA man stands in a driveway covered with snow talking and holding a shovel. then\n\nA. , the man puts snow over the car, put snow cover around the car and points to bare areas while talking.\nB. , the man shovel the snow on the driveway and put in a pile.\nC. , the man opens the driveway of a house and reveal snow covering a car.\nD. , the man passes in the driveway and continues cutting the snow.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":849,"error":null,"output":"B"}
{"question_id":"hellaswag:00005328","item_index":61,"attempt":0,"prompt_hash":"4e34f4263ebdfe1f34d07cd277e5cf4e794baaf121b3f673c18458ee3916ced7","question":"[header] How to rename a drive [title] Access window's disk management feature. [step] Open the control panel from the start menu and, if necessary, switch to the default view by clicking \" control panel home \" in the upper-left corner. [substeps] Select the \" performance and maintenance \" option and click the link to the \" administrative tools.\n\nA. \" if your system requirements for the drive have changed, you can formatted your drive to some perspective. You can preview the new processing settings in the \" performance \" section of the main control panel, and enjoy the free trial.\nB. \" look for a running driver firewall installation window, then find the \" run \" tab. Click \" run \" in the windows installation window to install the new driver.\nC. \" [title] Access the user user profile summary window and explore the user's profile profile or profile. [step] Before you delete a user's profile profile or profile, navigate through the user profiles or profiles section at the top of the window.\nD. \" double-click the \" computer management option \" icon and select \" disk management \" from the options listed in the panel on the left. [title] Locate the volume, drive or partition to be renamed from the list of available options.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to rename a drive [title] Access window's disk management feature. [step] Open the control panel from the start menu and, if necessary, switch to the default view by clicking \" control panel home \" in the upper-left corner. [substeps] Select the \" performance and maintenance \" option and click the link to the \" administrative tools.\n\nA. \" if your system requirements for the drive have changed, you can formatted your drive to some perspective. You can preview the new processing settings in the \" performance \" section of the main control panel, and enjoy the free trial.\nB. \" look for a running driver firewall installation window, then find the \" run \" tab. Click \" run \" in the windows installation window to install the new driver.\nC. \" [title] Access the user user profile summary window and explore the user's profile profile or profile. [step] Before you delete a user's profile profile or profile, navigate through the user profiles or profiles section at the top of the window.\nD. \" double-click the \" computer management option \" icon and select \" disk management \" from the options listed in the panel on the left. [title] Locate the volume, drive or partition to be renamed from the list of available options.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1867,"error":null,"output":"D"}
{"question_id":"hellaswag:00006820","item_index":62,"attempt":0,"prompt_hash":"6ef129935ec6c8d75a4a42675291323043c605b4994227ec6432469171524d00","question":"[header] How to look cute [title] Wear cute clothes. [step] Wearing cute clothes is the key to being cute. You don't have to change everything in your wardrobe to be cute--instead, just work on acquiring a few cute items that tie all of your clothes together in a cute way.\n\nA. Ditch the drab and wrinkled feel when looking for cute items, get creative. [substeps] Dresses, skirts, crop tops, and fishnet dresses are all cute clothing.\nB. [substeps] Wear a cute top with leggings or shorts or a cute dress. Leave your t-shirt off the equation if you have to wear a shirt with cute slogans (such as \" cool \" or \" happy.\nC. Here are some ways to wear cute clothes : [substeps] Wear skirts and dresses instead of pants or shorts when you can. Cardigans, knit sweaters and vests, boot-cut jeans, and t-shirts with cute patterns like hearts or polka-dots, are cute forms of clothing.\nD. If you constantly berate yourself, you're actually trying to look cute. You're just using this as a way to show off your cute traits, no matter what others think.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to look cute [title] Wear cute clothes. [step] Wearing cute clothes is the key to being cute. You don't have to change everything in your wardrobe to be cute--instead, just work on acquiring a few cute items that tie all of your clothes together in a cute way.\n\nA. Ditch the drab and wrinkled feel when looking for cute items, get creative. [substeps] Dresses, skirts, crop tops, and fishnet dresses are all cute clothing.\nB. [substeps] Wear a cute top with leggings or shorts or a cute dress. Leave your t-shirt off the equation if you have to wear a shirt with cute slogans (such as \" cool \" or \" happy.\nC. Here are some ways to wear cute clothes : [substeps] Wear skirts and dresses instead of pants or shorts when you can. Cardigans, knit sweaters and vests, boot-cut jeans, and t-shirts with cute patterns like hearts or polka-dots, are cute forms of clothing.\nD. If you constantly berate yourself, you're actually trying to look cute. You're just using this as a way to show off your cute traits, no matter what others think.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1894,"error":null,"output":"A"}
{"question_id":"hellaswag:00008861","item_index":63,"attempt":0,"prompt_hash":"6e14dbbabfc0f647c37895ab45379195dbaf0736522727c120588252742606fa","question":"[header] How to keep necklaces and chains from rusting [title] Put on lotion before you wear your chain. [step] Lotion, creams, and perfumes can all cause your chain to rust. Make sure that they are completely rubbed in and dry before you put on your necklace or chain.\n\nA. Oils can actually help your necklaces last longer. [substeps] Using a thin, natural oil is not recommended because it can break down your chain and cause a rust.\nB. [title] Put necklaces away before you shower. [step] Any water can cause your necklace or chain to rust.\nC. Take a shower before going out and stick to clean water. If you're wet, your chain could eventually wear out.\nD. [title] Make a metal chain knot to go behind the three loops on the first chain. [step] This will keep it from splintering or breaking.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to keep necklaces and chains from rusting [title] Put on lotion before you wear your chain. [step] Lotion, creams, and perfumes can all cause your chain to rust. Make sure that they are completely rubbed in and dry before you put on your necklace or chain.\n\nA. Oils can actually help your necklaces last longer. [substeps] Using a thin, natural oil is not recommended because it can break down your chain and cause a rust.\nB. [title] Put necklaces away before you shower. [step] Any water can cause your necklace or chain to rust.\nC. Take a shower before going out and stick to clean water. If you're wet, your chain could eventually wear out.\nD. [title] Make a metal chain knot to go behind the three loops on the first chain. [step] This will keep it from splintering or breaking.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1560,"error":null,"output":"C"}
{"question_id":"hellaswag:00000524","item_index":64,"attempt":0,"prompt_hash":"70bf98aaa29c83deb086630bb69aaff46a854ab8262b44d27e83c5cb2b19c8e3","question":"The man is back at the large light gray counter and in a sped up motion the man is shown with various different supplies as he assembles a very large truck cake that was previously shown. when the cake\n\nA. ready is ready the man takes a pain killer that it looks like is made of chocolate chips and the cals don't seem very happy as they collide into the other cakes that are going up and down the scale.\nB. is complete the chef lays down a blue sheet over the cake area and lays out a foot long cake piece that looks like it has duct tape on it.\nC. is done, the man begins wrapping and peeling the cake as the camera focuses on his movements some someone else is close to the kitchen counter which has a very pointed ceiling.\nD. is all done the camera pans the whole cake so the details can be seen and once again we see the smaller scale cake and the toy.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nThe man is back at the large light gray counter and in a sped up motion the man is shown with various different supplies as he assembles a very large truck cake that was previously shown. when the cake\n\nA. ready is ready the man takes a pain killer that it looks like is made of chocolate chips and the cals don't seem very happy as they collide into the other cakes that are going up and down the scale.\nB. is complete the chef lays down a blue sheet over the cake area and lays out a foot long cake piece that looks like it has duct tape on it.\nC. is done, the man begins wrapping and peeling the cake as the camera focuses on his movements some someone else is close to the kitchen counter which has a very pointed ceiling.\nD. is all done the camera pans the whole cake so the details can be seen and once again we see the smaller scale cake and the toy.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1581,"error":null,"output":"D"}
{"question_id":"hellaswag:00005212","item_index":65,"attempt":0,"prompt_hash":"22c9212faa8812bc39a1b073e52b87ae3c07ce1f8635f5e77ba80dbb42f0473e","question":"[header] How to make a panzerotti [title] Pour 1/4 cup of warm water into a large bowl. [step] Add your package of active dry yeast and a pinch of sugar. Set the mixture aside for five minutes.\n\nA. After that, pour 1/4 cup of boiling water over the dried panzerotti and stir the mixture with a whisk. [substeps] If you prefer, you can substitute corn meal or chicken barley for the water in the bowl.\nB. [substeps] You can use baking soda as a substitute if you don't like baking soda. [title] Whisk the water and the dry yeast together.\nC. [title] Remove the panzerotti from the mortar. [step] Using a spoon, keep mixing until the yeast forms a thick dough.\nD. [title] Add the olive oil and a sprinkle of salt and pepper to the bowl. [step] Add an additional cup of warm water.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make a panzerotti [title] Pour 1/4 cup of warm water into a large bowl. [step] Add your package of active dry yeast and a pinch of sugar. Set the mixture aside for five minutes.\n\nA. After that, pour 1/4 cup of boiling water over the dried panzerotti and stir the mixture with a whisk. [substeps] If you prefer, you can substitute corn meal or chicken barley for the water in the bowl.\nB. [substeps] You can use baking soda as a substitute if you don't like baking soda. [title] Whisk the water and the dry yeast together.\nC. [title] Remove the panzerotti from the mortar. [step] Using a spoon, keep mixing until the yeast forms a thick dough.\nD. [title] Add the olive oil and a sprinkle of salt and pepper to the bowl. [step] Add an additional cup of warm water.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1600,"error":null,"output":"A"}
{"question_id":"hellaswag:00000169","item_index":66,"attempt":0,"prompt_hash":"659726260d918de48115ae8fef1a0a6e9a67c73b2e6e2bad049baf72e6f2e556","question":"A man is shown standing in front of a blue background as many harmonicas are shown and he starts playing them. He is then shown speaking into the camera and going into detail with regards to the products features. he\n\nA. continues talking about the machine while smoking a cigarette in front of the camera.\nB. continues playing the stereo while detail is shown on the side as the scenes continue to pan in focus and ends with a blow by blow image.\nC. shows off how many he has in his coat and continues talking.\nD. is 6'6 \" and continues to play with the device throughout the video as well as cutting it back to a man playing polls.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA man is shown standing in front of a blue background as many harmonicas are shown and he starts playing them. He is then shown speaking into the camera and going into detail with regards to the products features. he\n\nA. continues talking about the machine while smoking a cigarette in front of the camera.\nB. continues playing the stereo while detail is shown on the side as the scenes continue to pan in focus and ends with a blow by blow image.\nC. shows off how many he has in his coat and continues talking.\nD. is 6'6 \" and continues to play with the device throughout the video as well as cutting it back to a man playing polls.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1149,"error":null,"output":"C"}
{"question_id":"hellaswag:00004303","item_index":67,"attempt":0,"prompt_hash":"2c09b9927cd94af13a651c95942ebcb97cebddfd61716b3cca6e355744d6856f","question":"[header] How to make blue eyes pop [title] Apply concealer under your eyes. [step] Blue eyes can easily get lost in dark undereye circles, and this is especially noticeable with a natural look. Apply concealer around and under your eyes to put the focus on them, not your circles! [substeps] Try using a salmon-colored concealer to hide the bluish tones in your skin.\n\nA. [title] Use a dark shade of eyeshadow as your shadow. [step] In the morning, put on a dark shade of eyeshadow on your darker circles.\nB. Let the concealer dry for at least a couple of minutes on your skin. [title] Clean under your eyes with an eyelid primer.\nC. This will help blend out the dark circles. You can also choose a bright color like pink for blues and red for green eyes.\nD. If you only need light coverage, dot the concealer along your undereye, then blend with your fingers or a makeup sponge. If you have very dark circles, apply a layer of concealer in an inverted triangle shape under your eye.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make blue eyes pop [title] Apply concealer under your eyes. [step] Blue eyes can easily get lost in dark undereye circles, and this is especially noticeable with a natural look. Apply concealer around and under your eyes to put the focus on them, not your circles! [substeps] Try using a salmon-colored concealer to hide the bluish tones in your skin.\n\nA. [title] Use a dark shade of eyeshadow as your shadow. [step] In the morning, put on a dark shade of eyeshadow on your darker circles.\nB. Let the concealer dry for at least a couple of minutes on your skin. [title] Clean under your eyes with an eyelid primer.\nC. This will help blend out the dark circles. You can also choose a bright color like pink for blues and red for green eyes.\nD. If you only need light coverage, dot the concealer along your undereye, then blend with your fingers or a makeup sponge. If you have very dark circles, apply a layer of concealer in an inverted triangle shape under your eye.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1703,"error":null,"output":"D"}
{"question_id":"hellaswag:00006912","item_index":68,"attempt":0,"prompt_hash":"421e606f17aed2f0f0bc48a66547d1c6798ce98969cb44abe7cd73972d6c5b36","question":"[header] How to prepare oily skin for makeup [title] Wash your face with a gentle cleanser. [step] They key to getting a healthy glow with makeup is making sure the palette-your face--underneath is clean. Use a gentle, ph-balanced cleanser to clear any dirt or excess oil off of your skin.\n\nA. [substeps] If you have oily skin, use a facial cleanser instead of a pore-cleansing product. Because pore-cleansing products can dry out your skin, avoid that part of your face.\nB. [substeps] You may want to consider using a gel cleanser, which is often gentle and doesn't have excess oils that could exacerbate your oily skin. Avoid using products that are heavy in chemicals, which may irritate your skin and increase oil production.\nC. Never use a cleanser with salicylic acid, but you should also avoid regular facial cleansers that will leave your skin looking oily. [substeps] Look for products that are 100% organic; these will not only prevent you from picking at dirt, but also have antimicrobial properties.\nD. Put some jojoba oil into a large bowl, and fill it with warm water and a little apple cider vinegar. Use your fingertips to gently pat away any excess sugar.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to prepare oily skin for makeup [title] Wash your face with a gentle cleanser. [step] They key to getting a healthy glow with makeup is making sure the palette-your face--underneath is clean. Use a gentle, ph-balanced cleanser to clear any dirt or excess oil off of your skin.\n\nA. [substeps] If you have oily skin, use a facial cleanser instead of a pore-cleansing product. Because pore-cleansing products can dry out your skin, avoid that part of your face.\nB. [substeps] You may want to consider using a gel cleanser, which is often gentle and doesn't have excess oils that could exacerbate your oily skin. Avoid using products that are heavy in chemicals, which may irritate your skin and increase oil production.\nC. Never use a cleanser with salicylic acid, but you should also avoid regular facial cleansers that will leave your skin looking oily. [substeps] Look for products that are 100% organic; these will not only prevent you from picking at dirt, but also have antimicrobial properties.\nD. Put some jojoba oil into a large bowl, and fill it with warm water and a little apple cider vinegar. Use your fingertips to gently pat away any excess sugar.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1912,"error":null,"output":"A"}
{"question_id":"hellaswag:00001108","item_index":69,"attempt":0,"prompt_hash":"9032e279430c5a37b451a7f019bf207b9dd003587e68230d5f907ce3d6c5ca5c","question":"A group of people are seen sitting on a dock while another group rides away in a boat. the people\n\nA. ride off a boat into the ocean while the camera captures them.\nB. talk back and fourth while one man sits on a pair of skis.\nC. ride all around the water on a raft while showing clips of people riding on the water.\nD. continue riding along the water with the boat bumping into one another.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA group of people are seen sitting on a dock while another group rides away in a boat. the people\n\nA. ride off a boat into the ocean while the camera captures them.\nB. talk back and fourth while one man sits on a pair of skis.\nC. ride all around the water on a raft while showing clips of people riding on the water.\nD. continue riding along the water with the boat bumping into one another.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1306,"error":null,"output":"D. continue riding along the water with the boat bumping into one another."}
{"question_id":"hellaswag:00007428","item_index":70,"attempt":0,"prompt_hash":"ca6f86ba88e2b6a6a08ee28f7114afd2c7136a64df2b63716bf34616b26a024a","question":"[header] How to have good mental health [title] Exercise your body. [step] When experiencing stress, your brain produces hormones that tell the body to prepare to respond to a threat. Serious stress can compromise your mental health and may cause physical symptoms, too.\n\nA. Experts recommend practicing a balanced exercise regimen every day, depending on your moods, physical environment, and overall health. [substeps] Find a place with around 60 to 70 minutes of moderate activity per day.\nB. One good way to manage stress is through exercise. [substeps] Exercise and physical activity can bring relief to tense muscles.\nC. To manage stress, try to increase your daily aerobic exercise to counteract it. [substeps] Exercising can include: walking, cycling, swimming, yoga, using a stationary bike, painting, or dancing.\nD. Getting regular exercise can prevent stress and improve your mental health. Exercise often expresses the opposite sex's desire to attach to someone or accommodate one another mentally.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to have good mental health [title] Exercise your body. [step] When experiencing stress, your brain produces hormones that tell the body to prepare to respond to a threat. Serious stress can compromise your mental health and may cause physical symptoms, too.\n\nA. Experts recommend practicing a balanced exercise regimen every day, depending on your moods, physical environment, and overall health. [substeps] Find a place with around 60 to 70 minutes of moderate activity per day.\nB. One good way to manage stress is through exercise. [substeps] Exercise and physical activity can bring relief to tense muscles.\nC. To manage stress, try to increase your daily aerobic exercise to counteract it. [substeps] Exercising can include: walking, cycling, swimming, yoga, using a stationary bike, painting, or dancing.\nD. Getting regular exercise can prevent stress and improve your mental health. Exercise often expresses the opposite sex's desire to attach to someone or accommodate one another mentally.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1757,"error":null,"output":"A"}
{"question_id":"hellaswag:00002173","item_index":71,"attempt":0,"prompt_hash":"c857369baeff9b72dce23969ce8fc8d87495b2645cd1bc2c491b0de108c8e193","question":"We see a man conducting an orchestra. we\n\nA. see a lady in a blue dress is playing in front of the other players.\nB. see people walking in the street with microphones in hand.\nC. see the pianist taking notes in the orchestra.\nD. switch to a little girl performing a baton routine.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nWe see a man conducting an orchestra. we\n\nA. see a lady in a blue dress is playing in front of the other players.\nB. see people walking in the street with microphones in hand.\nC. see the pianist taking notes in the orchestra.\nD. switch to a little girl performing a baton routine.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1052,"error":null,"output":"D. switch to a little girl performing a baton routine."}
{"question_id":"hellaswag:00008855","item_index":72,"attempt":0,"prompt_hash":"911447879bf360e256b1633eedc0b000528a51c700463b2fdcb01a55e4a73bd1","question":"[header] How to mediate family arguments [title] Think about everyone's perspective. [step] Before attempting to mediate disagreements, make sure you look at all sides of the argument. While you shouldn't take sides when attempting to mediate, you don't want anyone to think you're overlooking their point of view.\n\nA. Remember, however, that you have three options : [substeps] Persuade, not reason or move-on. Offer a solution that everyone agrees on and walk away.\nB. If you point out that your family member has a point, you can subtly nudge them towards a different viewpoint. [substeps] Try a self-deprecating comment like, \" thank you, but i disagree.\nC. If each side of the argument challenges the other, then you and the other person come into conflict as a team. [title] Decide who is right and who is wrong.\nD. Take some time to think about why everyone feels wronged or hurt in the situation. [substeps] You will have to resist the urge to judge here.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to mediate family arguments [title] Think about everyone's perspective. [step] Before attempting to mediate disagreements, make sure you look at all sides of the argument. While you shouldn't take sides when attempting to mediate, you don't want anyone to think you're overlooking their point of view.\n\nA. Remember, however, that you have three options : [substeps] Persuade, not reason or move-on. Offer a solution that everyone agrees on and walk away.\nB. If you point out that your family member has a point, you can subtly nudge them towards a different viewpoint. [substeps] Try a self-deprecating comment like, \" thank you, but i disagree.\nC. If each side of the argument challenges the other, then you and the other person come into conflict as a team. [title] Decide who is right and who is wrong.\nD. Take some time to think about why everyone feels wronged or hurt in the situation. [substeps] You will have to resist the urge to judge here.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1737,"error":null,"output":"A"}
{"question_id":"hellaswag:00002855","item_index":73,"attempt":0,"prompt_hash":"1d0cdf775c2b606fcdc01e023e4675fcdd81c7d380ea05b1b98936ffc4e89c03","question":"The man holds up a jar of peanut butter and a bottle of jelly after pulling out two pieces of bread while standing in front of a kitchen countertop. The man puts jelly and peanut butter on the bread. the man\n\nA. then places the peanut butter on top of bread slices.\nB. talks to the camera again while holding more peanut butter and using a spoon to size up the bread.\nC. turns and wipes peanut butter off the bread on a black paper towel and uses the towel to wipe side of the counter.\nD. bites into the sandwich, begins talking again, slams the sandwich down on a plate and walks off.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nThe man holds up a jar of peanut butter and a bottle of jelly after pulling out two pieces of bread while standing in front of a kitchen countertop. The man puts jelly and peanut butter on the bread. the man\n\nA. then places the peanut butter on top of bread slices.\nB. talks to the camera again while holding more peanut butter and using a spoon to size up the bread.\nC. turns and wipes peanut butter off the bread on a black paper towel and uses the towel to wipe side of the counter.\nD. bites into the sandwich, begins talking again, slams the sandwich down on a plate and walks off.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1157,"error":null,"output":"A"}
{"question_id":"hellaswag:00001357","item_index":74,"attempt":0,"prompt_hash":"52ff476ffcc0d8d9d205249749c97cc6dbd406870badbb3ae8e8042ccbbe23da","question":"The lady takes the pacifier from the baby and lets the baby eat ice cream. The baby makes a face and stare at the camera as she eats. the lady\n\nA. sits with the baby and your dad puts an ice cream cone in her hand.\nB. removes the pacifier from the baby and lets the baby have some ekg and then she goes back to eating her ice cream.\nC. holds the baby as she goes down the baby's face.\nD. puts the pacifier back in the babies mouth and eats her ice cream.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nThe lady takes the pacifier from the baby and lets the baby eat ice cream. The baby makes a face and stare at the camera as she eats. the lady\n\nA. sits with the baby and your dad puts an ice cream cone in her hand.\nB. removes the pacifier from the baby and lets the baby have some ekg and then she goes back to eating her ice cream.\nC. holds the baby as she goes down the baby's face.\nD. puts the pacifier back in the babies mouth and eats her ice cream.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1030,"error":null,"output":"A"}
{"question_id":"hellaswag:00001492","item_index":75,"attempt":0,"prompt_hash":"1633f2ca1d2f76d52368eb5974c9124965ffc17da178fff27e56defd86b3549a","question":"A pile of clothing is shown on a bench in a house. A sweater hangs from a rack. the sweater\n\nA. is then shown in various sizes.\nB. turns into a sweater.\nC. is drying with a towel several times.\nD. is shown from several angles as a vacuum is used to pick up hair and lint.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA pile of clothing is shown on a bench in a house. A sweater hangs from a rack. the sweater\n\nA. is then shown in various sizes.\nB. turns into a sweater.\nC. is drying with a towel several times.\nD. is shown from several angles as a vacuum is used to pick up hair and lint.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":711,"error":null,"output":"D"}
{"question_id":"hellaswag:00009753","item_index":76,"attempt":0,"prompt_hash":"8132311ccf07d55f616c9f95ac24fe72fa33ab9aaf2acbb18dabcffc5e49b5ba","question":"[header] How to clean bathroom grout [title] Steam clean the grout. [step] One effective and environmentally friendly way of cleaning grout is to attack it with a steam cleaner. Outfit your steam cleaner with a thin and pointed nozzle, and a brush attachment if it's available.\n\nA. Run your steam cleaner over the entire floor, making sure to scour the entire grout area. [substeps] However, if you cannot steam clean your grout by yourself, you can freeze grout at home and use the steam cleaner to scrub grout specific areas.\nB. Swab some hot water into the steam cleaner and then rinse the grout clean with warm water. Tap the brush attachment onto the grout drain, and then unscrew and fill a bucket with warm water.\nC. Aim the nozzle at the grout lines, apply continuous steam, and follow the grout lines with the steamer. [substeps] Even if the steam cleaner doesn't get the grout perfectly clean, it will clean it somewhat, and help loosen the rest of the dirt and debris to make it easier to clean with other products.\nD. Head over to your kitchen sink, and turn on the steam cleaner. You can also use a sponge or box brush to remove scum from your grout.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to clean bathroom grout [title] Steam clean the grout. [step] One effective and environmentally friendly way of cleaning grout is to attack it with a steam cleaner. Outfit your steam cleaner with a thin and pointed nozzle, and a brush attachment if it's available.\n\nA. Run your steam cleaner over the entire floor, making sure to scour the entire grout area. [substeps] However, if you cannot steam clean your grout by yourself, you can freeze grout at home and use the steam cleaner to scrub grout specific areas.\nB. Swab some hot water into the steam cleaner and then rinse the grout clean with warm water. Tap the brush attachment onto the grout drain, and then unscrew and fill a bucket with warm water.\nC. Aim the nozzle at the grout lines, apply continuous steam, and follow the grout lines with the steamer. [substeps] Even if the steam cleaner doesn't get the grout perfectly clean, it will clean it somewhat, and help loosen the rest of the dirt and debris to make it easier to clean with other products.\nD. Head over to your kitchen sink, and turn on the steam cleaner. You can also use a sponge or box brush to remove scum from your grout.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":2010,"error":null,"output":"C"}
{"question_id":"hellaswag:00000059","item_index":77,"attempt":0,"prompt_hash":"527f17a9870844b5b1f82e4d3cd21925ebc4af1f1ccb2d1bb4e42eafd336603e","question":"A logo shows that this video is presented by expert village. a man in a small blue boat\n\nA. is pushing a kress of water in a lake while instructing his boarder.\nB. discusses the topic of hand surfing in a hole and gives hints on how to perform this activity.\nC. , in the middle of a lake, is skiing.\nD. is in the water pushing a boat with a sail.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA logo shows that this video is presented by expert village. a man in a small blue boat\n\nA. is pushing a kress of water in a lake while instructing his boarder.\nB. discusses the topic of hand surfing in a hole and gives hints on how to perform this activity.\nC. , in the middle of a lake, is skiing.\nD. is in the water pushing a boat with a sail.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":885,"error":null,"output":"D"}
{"question_id":"hellaswag:00009874","item_index":78,"attempt":0,"prompt_hash":"4f3a888b5badb6d351a58463c95159ed2a7f59a069ee2fe41aa6640fc62b1ff8","question":"[header] How to measure temperature [title] Use the thermometer to take body temperature. [step] While glass thermometers are not ideal for taking body temperature, you can still use them instead of an electric thermometer. Place them in the appropriate area of the body and wait between 2 and 4 minutes before reading the temperature.\n\nA. [substeps] The ideal temperature to take is about 1 ° f (−32 ° c). If you were to place them in very hot water for 1-10 minutes, they wouldn't make a difference.\nB. [substeps] Reading the temperature on an electric thermometer costs less than the average person with a normal body temperature of 19 ° f (- 7.9 ° c). [title] Use a thermometer to measure the temperature of your arms and legs.\nC. Subtract your temperature from the temperature provided by a thermometer to determine your actual temperature. [substeps] As noted earlier, natural (and occasionally manufactured) thermometers are non-digital, so there's no need to visit an electronic thermometer in a digital format.\nD. [substeps] Young children may bite and break the thermometer, which is why it is usually safer to take temperature using a digital thermometer. For children over 5 and adults, place the metal end under the tongue.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to measure temperature [title] Use the thermometer to take body temperature. [step] While glass thermometers are not ideal for taking body temperature, you can still use them instead of an electric thermometer. Place them in the appropriate area of the body and wait between 2 and 4 minutes before reading the temperature.\n\nA. [substeps] The ideal temperature to take is about 1 ° f (−32 ° c). If you were to place them in very hot water for 1-10 minutes, they wouldn't make a difference.\nB. [substeps] Reading the temperature on an electric thermometer costs less than the average person with a normal body temperature of 19 ° f (- 7.9 ° c). [title] Use a thermometer to measure the temperature of your arms and legs.\nC. Subtract your temperature from the temperature provided by a thermometer to determine your actual temperature. [substeps] As noted earlier, natural (and occasionally manufactured) thermometers are non-digital, so there's no need to visit an electronic thermometer in a digital format.\nD. [substeps] Young children may bite and break the thermometer, which is why it is usually safer to take temperature using a digital thermometer. For children over 5 and adults, place the metal end under the tongue.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":2025,"error":null,"output":"A"}
{"question_id":"hellaswag:00008932","item_index":79,"attempt":0,"prompt_hash":"f98e9fd12e90c2f6f7082495a26c9e512095fd30fc6d92de7a60cafbf4367b1d","question":"[header] How to straighten hair without the use of an iron [title] Begin by shampooing, conditioning and towel-drying hair. [title] Apply a heat protectant product to your hair, paying special attention to the ends. [title] Place a quarter-size dollop of straightening balm in your palm.\n\nA. [title] Squeeze dry shampoo into the palm of your hand. [title] Squirt a few tablespoons of conditioner into your palm.\nB. [title] Let sit for 1-3 minutes while rolling towel on your head. [title] After allowing product to sit on your hair for a few minutes, using the palm of your hand instead of a towel, brush your hair to get rid of any hair sticking out.\nC. [step] Rub your palms together to distribute the product over your hands, then massage it evenly through hair. [title] Comb through your hair with your fingers while gently blow-drying it on a low setting.\nD. [title] Brush your hair evenly, from root to tip. [title] Rub gently the smoothing balm in a direction opposite of your hair's conditioner.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to straighten hair without the use of an iron [title] Begin by shampooing, conditioning and towel-drying hair. [title] Apply a heat protectant product to your hair, paying special attention to the ends. [title] Place a quarter-size dollop of straightening balm in your palm.\n\nA. [title] Squeeze dry shampoo into the palm of your hand. [title] Squirt a few tablespoons of conditioner into your palm.\nB. [title] Let sit for 1-3 minutes while rolling towel on your head. [title] After allowing product to sit on your hair for a few minutes, using the palm of your hand instead of a towel, brush your hair to get rid of any hair sticking out.\nC. [step] Rub your palms together to distribute the product over your hands, then massage it evenly through hair. [title] Comb through your hair with your fingers while gently blow-drying it on a low setting.\nD. [title] Brush your hair evenly, from root to tip. [title] Rub gently the smoothing balm in a direction opposite of your hair's conditioner.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1864,"error":null,"output":"C"}
{"question_id":"hellaswag:00002412","item_index":80,"attempt":0,"prompt_hash":"c455a3c0403202df117486c5cf950a637653be92f227a90881c0114f3c68e4fc","question":"A person is seen hammering down a floor following by cutting up carpet. The person glues down the carpet and laying weights down to level it out. the person\n\nA. presses down a button on a piece of wood.\nB. continues working and leads into a man pushing carpet up toward the ceiling.\nC. continues gluing down the carpet and cutting out the sides.\nD. shows off the floor and ends by covering it with tiles.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA person is seen hammering down a floor following by cutting up carpet. The person glues down the carpet and laying weights down to level it out. the person\n\nA. presses down a button on a piece of wood.\nB. continues working and leads into a man pushing carpet up toward the ceiling.\nC. continues gluing down the carpet and cutting out the sides.\nD. shows off the floor and ends by covering it with tiles.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1249,"error":null,"output":"C. continues gluing down the carpet and cutting out the sides."}
{"question_id":"hellaswag:00006869","item_index":81,"attempt":0,"prompt_hash":"b62a8b9a2d688a4fb457d6869a0f2c9ae9092cf34eed2b9a8f8028242afec8c4","question":"[header] How to identify a gifted child [title] Pay attention to your child's memory. [step] Gifted children tend to have a greater memory than average children. Oftentimes, you may notice memory in unexpected, somewhat subtle ways.\n\nA. However, this makes it easy to evaluate your child's memory. [substeps] Your child may seem strange when they're young.\nB. The child may forget to mention certain things (depending on the age of the child) later on. Look out for any telltale but sudden changes in memory.\nC. [substeps] Gifted children may seem distracted during their activities. They may forget something when they're studying or trying something new.\nD. Keep an eye out for signs of a superior memory. [substeps] Children may remember facts better than others.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to identify a gifted child [title] Pay attention to your child's memory. [step] Gifted children tend to have a greater memory than average children. Oftentimes, you may notice memory in unexpected, somewhat subtle ways.\n\nA. However, this makes it easy to evaluate your child's memory. [substeps] Your child may seem strange when they're young.\nB. The child may forget to mention certain things (depending on the age of the child) later on. Look out for any telltale but sudden changes in memory.\nC. [substeps] Gifted children may seem distracted during their activities. They may forget something when they're studying or trying something new.\nD. Keep an eye out for signs of a superior memory. [substeps] Children may remember facts better than others.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1432,"error":null,"output":"D"}
{"question_id":"hellaswag:00006080","item_index":82,"attempt":0,"prompt_hash":"44f3e698c15444ba9abbd1257b00d9fc59f183b873d587b2a879783526631040","question":"[header] How to make money in forex [title] Know how currencies are traded in the forex market. [step] The forex market is a global exchange of currencies and currency-backed financial instruments (contracts to buy or sell currencies at a later date). Participants include everyone from the largest banks and financial institutions to individual investors.\n\nA. Understand that there are 14 currencies :. In the form of mixed currencies or the same value , 0 , 2, and 5 , 1 , 1, and 2.\nB. This provides a solid foundation for investing in a conventional economy. In comparison, conventional currencies can be traded in the futures exchange for a couple of hundred dollars (as described before).\nC. Currencies are traded directly for other currencies in the market. As a result, currencies are priced in terms of other currencies, like euros per us dollar or japanese yen per british pound sterling.\nD. Part of materialism, especially here in western countries, is how customers approach the currencies in the lower east and western countries. This gives investors a sense of the most basic link between upper and lower countries, such as the european economy of america (190).","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make money in forex [title] Know how currencies are traded in the forex market. [step] The forex market is a global exchange of currencies and currency-backed financial instruments (contracts to buy or sell currencies at a later date). Participants include everyone from the largest banks and financial institutions to individual investors.\n\nA. Understand that there are 14 currencies :. In the form of mixed currencies or the same value , 0 , 2, and 5 , 1 , 1, and 2.\nB. This provides a solid foundation for investing in a conventional economy. In comparison, conventional currencies can be traded in the futures exchange for a couple of hundred dollars (as described before).\nC. Currencies are traded directly for other currencies in the market. As a result, currencies are priced in terms of other currencies, like euros per us dollar or japanese yen per british pound sterling.\nD. Part of materialism, especially here in western countries, is how customers approach the currencies in the lower east and western countries. This gives investors a sense of the most basic link between upper and lower countries, such as the european economy of america (190).","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1887,"error":null,"output":"C"}
{"question_id":"hellaswag:00002961","item_index":83,"attempt":0,"prompt_hash":"cde937b52d003732a3f621bbc02a324af72c75af2532c0ca681f54bd83ce865c","question":"One of the men starts to chase after a bull on his horse. He gets off of his horse and brings the bull down and starts walking back to his horse. he\n\nA. helps the bull up off of the ground and gives it a trophy, then the bull changes his tactics and starts chasing another bull off of the land while another woman walks around the area.\nB. then ends up dismounting and waving to the camera.\nC. picks the bull up off the ground and tackles him and beatin the bull to the ground.\nD. gets back on the horse as the bull just lays on the ground defeated.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nOne of the men starts to chase after a bull on his horse. He gets off of his horse and brings the bull down and starts walking back to his horse. he\n\nA. helps the bull up off of the ground and gives it a trophy, then the bull changes his tactics and starts chasing another bull off of the land while another woman walks around the area.\nB. then ends up dismounting and waving to the camera.\nC. picks the bull up off the ground and tackles him and beatin the bull to the ground.\nD. gets back on the horse as the bull just lays on the ground defeated.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1122,"error":null,"output":"C"}
{"question_id":"hellaswag:00009787","item_index":84,"attempt":0,"prompt_hash":"463957cf136c9f4291a7ae1bd5188afe6848cda3ad87385f4e986630d809f844","question":"[header] How to maintain african hair [title] Shampoo hair once a week. [step] African hair tends to be naturally dry with minimal oil, which is why it isn't necessary to use shampoo more than once a week. Shampooing your hair several times a week or even every day strips the oil that the hair needs from the hair shaft so that it becomes drier, frizzier and more prone to breaking.\n\nA. As african hair grows, it will be stronger and more muscular as it ages. Use lukewarm water whenever you wash your hair.\nB. [title] Exfoliate african hair once a week. [step] Since african hair tends to be drier every day, it is particularly important to exfoliate to the parts of your scalp that got the oil.\nC. [substeps] African hair does not often get overly oily like caucasian or asian hair does when it is not washed for several days. Do not wash tangled hair, as this will make the knots even worse.\nD. [substeps] Use a small amount of dry shampoo to start. Use your finger or a comb to gently exfoliate your hair several times a week, focusing on the scalp and neck.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to maintain african hair [title] Shampoo hair once a week. [step] African hair tends to be naturally dry with minimal oil, which is why it isn't necessary to use shampoo more than once a week. Shampooing your hair several times a week or even every day strips the oil that the hair needs from the hair shaft so that it becomes drier, frizzier and more prone to breaking.\n\nA. As african hair grows, it will be stronger and more muscular as it ages. Use lukewarm water whenever you wash your hair.\nB. [title] Exfoliate african hair once a week. [step] Since african hair tends to be drier every day, it is particularly important to exfoliate to the parts of your scalp that got the oil.\nC. [substeps] African hair does not often get overly oily like caucasian or asian hair does when it is not washed for several days. Do not wash tangled hair, as this will make the knots even worse.\nD. [substeps] Use a small amount of dry shampoo to start. Use your finger or a comb to gently exfoliate your hair several times a week, focusing on the scalp and neck.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1902,"error":null,"output":"D"}
{"question_id":"hellaswag:00009863","item_index":85,"attempt":0,"prompt_hash":"67e72b402de34a122bf18440aff06ca376b4f689547bb08a670537ed2bc31c1c","question":"[header] How to remove interior walls [title] First, determine if the wall you want to remove is a load bearing wall, meaning it supports the floor above. [step] Go into your basement (or crawlspace) and look at the alignment of floor joists. These are large pieces of lumber going across the house from side to side supporting the floor/roof and are generally 16 \" apart.\n\nA. If one side is facing out of the house, they'll likely be 16 \" apart. And if one side is facing in, these will be 24 \" apart.\nB. Remember the alignment-they generally go from side to side on most modern homes across the width of the house. [title] If the wall you want to remove is at a right angle to the floor joists-stop.\nC. The pallets facing your home and building plans typically occupy much less space in a single wall frame. [title] Use a small trowel to scrub smooth surfaces with the grain.\nD. Also, examine the front corners of the front wall. This is a great place to check on a load bearing wall.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to remove interior walls [title] First, determine if the wall you want to remove is a load bearing wall, meaning it supports the floor above. [step] Go into your basement (or crawlspace) and look at the alignment of floor joists. These are large pieces of lumber going across the house from side to side supporting the floor/roof and are generally 16 \" apart.\n\nA. If one side is facing out of the house, they'll likely be 16 \" apart. And if one side is facing in, these will be 24 \" apart.\nB. Remember the alignment-they generally go from side to side on most modern homes across the width of the house. [title] If the wall you want to remove is at a right angle to the floor joists-stop.\nC. The pallets facing your home and building plans typically occupy much less space in a single wall frame. [title] Use a small trowel to scrub smooth surfaces with the grain.\nD. Also, examine the front corners of the front wall. This is a great place to check on a load bearing wall.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1736,"error":null,"output":"B"}
{"question_id":"hellaswag:00006697","item_index":86,"attempt":0,"prompt_hash":"1f15239ad87497e546cd94519e5438edef76957145ddf1a5ce8ef9e118da5a8f","question":"[header] How to get a dark beard [title] Select a color that is slightly lighter than your natural color. [step] When you are selecting a color for your beard dye, choose a color that is slightly lighter than your natural color. A darker color can appear very bold and may look unnatural.\n\nA. Instead, try a color that is a few shades lighter. You can always darken the beard later if you desire.\nB. If you want a darker beard, go for a color that is half a shade lighter than your natural color. [substeps] If you cannot find a color suitable for your gender, choose one that is more subtle.\nC. [title] Speak to a stylist if you want to try a beard that you have already cut. [step] Having a stylist cut your beard specifically is very important because, if you're going to use a beard dye, it will be necessary for the dye to match the color you'll be using.\nD. [substeps] For example, if you want a blond beard, a darker shade of beard dye could be quite striking. Opt for a color that is a shade like red or orange.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to get a dark beard [title] Select a color that is slightly lighter than your natural color. [step] When you are selecting a color for your beard dye, choose a color that is slightly lighter than your natural color. A darker color can appear very bold and may look unnatural.\n\nA. Instead, try a color that is a few shades lighter. You can always darken the beard later if you desire.\nB. If you want a darker beard, go for a color that is half a shade lighter than your natural color. [substeps] If you cannot find a color suitable for your gender, choose one that is more subtle.\nC. [title] Speak to a stylist if you want to try a beard that you have already cut. [step] Having a stylist cut your beard specifically is very important because, if you're going to use a beard dye, it will be necessary for the dye to match the color you'll be using.\nD. [substeps] For example, if you want a blond beard, a darker shade of beard dye could be quite striking. Opt for a color that is a shade like red or orange.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1861,"error":null,"output":"A"}
{"question_id":"hellaswag:00001240","item_index":87,"attempt":0,"prompt_hash":"ba8839d25b573b4e752a948ca89e9eeac29cd47343b6af5741d232722dba35a3","question":"He then begins to rub the frame with a blue rag. As he does this he explains his methods. using the same rag he\n\nA. puts white foam around the edges of the frame to make a square frame.\nB. rubs over the railings of a wall.\nC. then shuffles the wax off of the first bar.\nD. moves to the other pane to clean the frame.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nHe then begins to rub the frame with a blue rag. As he does this he explains his methods. using the same rag he\n\nA. puts white foam around the edges of the frame to make a square frame.\nB. rubs over the railings of a wall.\nC. then shuffles the wax off of the first bar.\nD. moves to the other pane to clean the frame.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":853,"error":null,"output":"D"}
{"question_id":"hellaswag:00006915","item_index":88,"attempt":0,"prompt_hash":"2cacc0782dc513ded3a2f066f3527973afeadfcd09a24957743e6a162c99c727","question":"[header] How to apply aftershave splash [title] Rinse your face with cold water immediately after shaving. [step] Before you even consider applying aftershave, you want to make sure your face is completely free of shaving cream or leftover hairs. Fill your cupped hands with cold water and rub it thoroughly over the entire area that you shaved.\n\nA. [substeps] Warm water will slow down the process. You should also add a splash of aftershave spray to your water to keep your shaving chemicals from absorbing into your skin.\nB. Your freshly shaved face should be completely covered and clear of hard-to-see hairs. [title] Store shaving cream in the refrigerator.\nC. The benefit of cold water is that it closes the pores of your skin which is best for aftershave application. Be gentle as your rub your face so you don't chafe or irritate the skin.\nD. By rinsing thoroughly, you can ensure that each piece of stubble is covered completely. [substeps] If you use a soap, do so in a quiet, secure place.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to apply aftershave splash [title] Rinse your face with cold water immediately after shaving. [step] Before you even consider applying aftershave, you want to make sure your face is completely free of shaving cream or leftover hairs. Fill your cupped hands with cold water and rub it thoroughly over the entire area that you shaved.\n\nA. [substeps] Warm water will slow down the process. You should also add a splash of aftershave spray to your water to keep your shaving chemicals from absorbing into your skin.\nB. Your freshly shaved face should be completely covered and clear of hard-to-see hairs. [title] Store shaving cream in the refrigerator.\nC. The benefit of cold water is that it closes the pores of your skin which is best for aftershave application. Be gentle as your rub your face so you don't chafe or irritate the skin.\nD. By rinsing thoroughly, you can ensure that each piece of stubble is covered completely. [substeps] If you use a soap, do so in a quiet, secure place.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1689,"error":null,"output":"C"}
{"question_id":"hellaswag:00002998","item_index":89,"attempt":0,"prompt_hash":"0436fbda0e46bd9bf3c43460e69a004f2a9ab80c0e04a63109cada63dd1df570","question":"He is now outside of a car and is demonstrating how to use the frosted edge of a car window to sharpen the knife. He is back in the garage and explains and demonstrates that using a knife to sharpen another knife. he\n\nA. is then shown sharpening a attachment to the phone by pressure.\nB. then shows how to shirt the knife with a tissue to get the cleaned and sharpened metal against the clear surface.\nC. then asks for people to like the video and to subscribe and thanks everyone from watching.\nD. then demonstrates how to snow melt the ice into the car window, then use a brand new folding knife not bothering to put any excess coated tissue paper in it to get the wet.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nHe is now outside of a car and is demonstrating how to use the frosted edge of a car window to sharpen the knife. He is back in the garage and explains and demonstrates that using a knife to sharpen another knife. he\n\nA. is then shown sharpening a attachment to the phone by pressure.\nB. then shows how to shirt the knife with a tissue to get the cleaned and sharpened metal against the clear surface.\nC. then asks for people to like the video and to subscribe and thanks everyone from watching.\nD. then demonstrates how to snow melt the ice into the car window, then use a brand new folding knife not bothering to put any excess coated tissue paper in it to get the wet.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":2006,"error":null,"output":"B. then shows how to shirt the knife with a tissue to get the cleaned and sharpened metal against the clear surface."}
{"question_id":"hellaswag:00009472","item_index":90,"attempt":0,"prompt_hash":"fb6c6dfd7a0e5573855afce104634eac84f8234913a87c6f5cf9fd2b1c1ebf86","question":"[header] How to make hair straight with a curling iron [title] Search for a curling iron that has plates or a clamp. [step] It is much easier to straighten your hair with a curling iron that has a larger barrel, at least 1 to 2 inches in diameter. A larger barrel will allow you to run the curling iron through sections of your hair more easily and much more quickly than a curling iron with a smaller barrel.\n\nA. [substeps] Do a quick internet search for curling irons that have a different barrel, and have lower settings for best results. [title] Opt for shoulder-width twists instead of deep twists.\nB. [substeps] The clamp will keep the curling iron separate from the hair you are curling. [title] Set the curling iron to a low heat setting.\nC. [substeps] If you only have access to a curling iron with a smaller barrel ,. 75 to 1 inch in diameter, you can still use it to straighten your hair.\nD. You will also have less tangles and curls as you use a curling iron. [substeps] Curling irons come in two types: tube irons and half-barrel irons.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make hair straight with a curling iron [title] Search for a curling iron that has plates or a clamp. [step] It is much easier to straighten your hair with a curling iron that has a larger barrel, at least 1 to 2 inches in diameter. A larger barrel will allow you to run the curling iron through sections of your hair more easily and much more quickly than a curling iron with a smaller barrel.\n\nA. [substeps] Do a quick internet search for curling irons that have a different barrel, and have lower settings for best results. [title] Opt for shoulder-width twists instead of deep twists.\nB. [substeps] The clamp will keep the curling iron separate from the hair you are curling. [title] Set the curling iron to a low heat setting.\nC. [substeps] If you only have access to a curling iron with a smaller barrel ,. 75 to 1 inch in diameter, you can still use it to straighten your hair.\nD. You will also have less tangles and curls as you use a curling iron. [substeps] Curling irons come in two types: tube irons and half-barrel irons.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1941,"error":null,"output":"C"}
{"question_id":"hellaswag:00002395","item_index":91,"attempt":0,"prompt_hash":"da415b386e0538060aa3ae228130e2a86e675cde5bfaefd4739083fa4a8ad899","question":"An athletic woman is seen standing before a track and running into a pit. she\n\nA. gets up and walks back while others watch her.\nB. throws a disc off into the distance and is shown again in slow motion.\nC. stands up and begins falling down afterwards.\nD. jumps off stilts into the sand.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nAn athletic woman is seen standing before a track and running into a pit. she\n\nA. gets up and walks back while others watch her.\nB. throws a disc off into the distance and is shown again in slow motion.\nC. stands up and begins falling down afterwards.\nD. jumps off stilts into the sand.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1027,"error":null,"output":"D. jumps off stilts into the sand."}
{"question_id":"hellaswag:00009653","item_index":92,"attempt":0,"prompt_hash":"268d5c47389f6074156705fe2dc533da175386e7cb8358bd7a105f6c5e8d374b","question":"[header] How to react when your child ignores you [title] Take some deep breaths. [step] When your child ignores you, you might feel intensely annoyed, angry and/or stressed. If you feel this way, take some deep breaths in through the nose and exhale slowly through your mouth until you feel calm again and ready to address the situation.\n\nA. This feeling of anger and pent up physical tension can be extremely hurtful and can be very harmful. Stop yourself or the child and count to ten to calm yourself and ease your anxieties.\nB. [title] Walk away if you need a timeout. [step] Sometimes, deep breaths may not be enough.\nC. [substeps] If you feel anxious, take a few deep breaths and count to five. If your child acts impatient and uncommunicative, take some deep breaths through your nose and exhale slowly through your mouth.\nD. [substeps] If you are having trouble breathing while your child yells at you or insults you or is barely able to understand your words, take some deep breaths and ask your child for help. [title] Address your concerns by talking to your child.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to react when your child ignores you [title] Take some deep breaths. [step] When your child ignores you, you might feel intensely annoyed, angry and/or stressed. If you feel this way, take some deep breaths in through the nose and exhale slowly through your mouth until you feel calm again and ready to address the situation.\n\nA. This feeling of anger and pent up physical tension can be extremely hurtful and can be very harmful. Stop yourself or the child and count to ten to calm yourself and ease your anxieties.\nB. [title] Walk away if you need a timeout. [step] Sometimes, deep breaths may not be enough.\nC. [substeps] If you feel anxious, take a few deep breaths and count to five. If your child acts impatient and uncommunicative, take some deep breaths through your nose and exhale slowly through your mouth.\nD. [substeps] If you are having trouble breathing while your child yells at you or insults you or is barely able to understand your words, take some deep breaths and ask your child for help. [title] Address your concerns by talking to your child.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1973,"error":null,"output":"A"}
{"question_id":"hellaswag:00009378","item_index":93,"attempt":0,"prompt_hash":"a6ab9c8144dc005030a5c1736fa5aff31e2e47c68b208b35f20c97e00abc1e65","question":"[header] How to leave your hairdresser [title] Give your stylist a second chance. [step] Be specific about what you want done at the beginning of the appointment. If you still don't like it, explain to your stylist what it is that you want and what she/he is not doing.\n\nA. Your stylist will understand all you are asking of her/his and will be able to provide professional advice. It is a good idea to pick another stylist when you are willing to walk away with a two day haircut so it's not as stressful.\nB. If you want a certain look bring pictures from a magazine. Most hair stylists are cool about adjusting their work to meet customer specifications.\nC. As a result, she/he will be able to help you decide on a hairstyle that you will not tolerate. That being said, let your stylist know what style you feel you would like in order to help ensure it is what you are after.\nD. [title] Be accepting of your stylist's suggestions. [step] Also, remember that when your hairdresser uses you as a personal stylist, it is your responsibility to accept their suggestions.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to leave your hairdresser [title] Give your stylist a second chance. [step] Be specific about what you want done at the beginning of the appointment. If you still don't like it, explain to your stylist what it is that you want and what she/he is not doing.\n\nA. Your stylist will understand all you are asking of her/his and will be able to provide professional advice. It is a good idea to pick another stylist when you are willing to walk away with a two day haircut so it's not as stressful.\nB. If you want a certain look bring pictures from a magazine. Most hair stylists are cool about adjusting their work to meet customer specifications.\nC. As a result, she/he will be able to help you decide on a hairstyle that you will not tolerate. That being said, let your stylist know what style you feel you would like in order to help ensure it is what you are after.\nD. [title] Be accepting of your stylist's suggestions. [step] Also, remember that when your hairdresser uses you as a personal stylist, it is your responsibility to accept their suggestions.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1945,"error":null,"output":"C"}
{"question_id":"hellaswag:00000466","item_index":94,"attempt":0,"prompt_hash":"394a1c5bb06744efcf1b2a5a6bae2b6fd8951d06a590ac688b8dffbcc2e2024a","question":"Several people are in a manmade race track. Bmx bikers are going over the hills along the path. they\n\nA. spin in the air and do stunts as they ride.\nB. are lined up, racing to break people to get to the finish line.\nC. race through the streets of a city where the streets are paved.\nD. race down the hills, making fast jumps into the sand.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nSeveral people are in a manmade race track. Bmx bikers are going over the hills along the path. they\n\nA. spin in the air and do stunts as they ride.\nB. are lined up, racing to break people to get to the finish line.\nC. race through the streets of a city where the streets are paved.\nD. race down the hills, making fast jumps into the sand.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":960,"error":null,"output":"A"}
{"question_id":"hellaswag:00009180","item_index":95,"attempt":0,"prompt_hash":"080043bd442c2c17404ca86e6a834de2c0b6e75e78279cb691973e95c5ab279e","question":"[header] How to act around your grandpa [title] Have respect. [step] It is crucial to show respect to all people, but particularly to anyone in your family, and most particularly to family members who are older than yourself. Your grandfather is very wise compared to you, and has had a great deal of life experience.\n\nA. Do not expect your grandfather to follow you around like a longlost god. Learning how to respect yourself does not mean being rude or disrespected; doing so shows that you care about yourself and wants to help others.\nB. Demonstrating love to and giving respect in return can make him a useful grandparent and a great grandfather. [title] Make peace with your grandpa's decisions.\nC. This means he probably knows a lot more than you do about a great many things. Therefore you should listen to what he has to say, and take it very seriously.\nD. You want to be kind and considerate of anyone you meet so you can be a good leader in society. [substeps] Respect your grandfather's age, but also respect that your mother might be older than him and this is so you can be an effective leader in your grandchildren.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to act around your grandpa [title] Have respect. [step] It is crucial to show respect to all people, but particularly to anyone in your family, and most particularly to family members who are older than yourself. Your grandfather is very wise compared to you, and has had a great deal of life experience.\n\nA. Do not expect your grandfather to follow you around like a longlost god. Learning how to respect yourself does not mean being rude or disrespected; doing so shows that you care about yourself and wants to help others.\nB. Demonstrating love to and giving respect in return can make him a useful grandparent and a great grandfather. [title] Make peace with your grandpa's decisions.\nC. This means he probably knows a lot more than you do about a great many things. Therefore you should listen to what he has to say, and take it very seriously.\nD. You want to be kind and considerate of anyone you meet so you can be a good leader in society. [substeps] Respect your grandfather's age, but also respect that your mother might be older than him and this is so you can be an effective leader in your grandchildren.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1845,"error":null,"output":"C"}
{"question_id":"hellaswag:00008253","item_index":96,"attempt":0,"prompt_hash":"04dfbad08a8b976bf6612d19f7047e3031fa078dc30d587d15e6aaa4a05f5ac2","question":"[header] How to help your boss see how a coworker is undermining others [title] Determine if the coworker is hypercompetitive. [step] Sometimes it is hard to tell the difference between someone who is hypercompetitive and someone who is sabotaging others at work. The two are not mutually exclusive.\n\nA. [substeps] One person may be full of energy and expect others to help them succeed. Another person may be full of stresses and insecurities.\nB. There is a great deal of distance between micro-promotion and super-promotion. Balance the two in some way to see what your coworkers are doing better in regards to efficiency and performance in between assignments.\nC. [substeps] Someone who is hypercompetitive tries to beat out everyone, however someone who is actively undermining others wants to see coworkers fail. They usually do not care about the outcome of the project.\nD. But there are a few common examples of hypercompetitive employees in company careers. [substeps] Those who have high standards above the other employees in which they compete: those with a high level of skill and experience in an area that defines them (or person) as superior.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to help your boss see how a coworker is undermining others [title] Determine if the coworker is hypercompetitive. [step] Sometimes it is hard to tell the difference between someone who is hypercompetitive and someone who is sabotaging others at work. The two are not mutually exclusive.\n\nA. [substeps] One person may be full of energy and expect others to help them succeed. Another person may be full of stresses and insecurities.\nB. There is a great deal of distance between micro-promotion and super-promotion. Balance the two in some way to see what your coworkers are doing better in regards to efficiency and performance in between assignments.\nC. [substeps] Someone who is hypercompetitive tries to beat out everyone, however someone who is actively undermining others wants to see coworkers fail. They usually do not care about the outcome of the project.\nD. But there are a few common examples of hypercompetitive employees in company careers. [substeps] Those who have high standards above the other employees in which they compete: those with a high level of skill and experience in an area that defines them (or person) as superior.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1944,"error":null,"output":"C"}
{"question_id":"hellaswag:00007677","item_index":97,"attempt":0,"prompt_hash":"eade7de883f742d7d8d309c6a8e8676c44a558b3610e19ae263fb92b1b0b6103","question":"[header] How to clean a sauna [title] Scrub the benches, walls, and backrests after use. [step] Dip a hand brush into a bucket of water and scrub down all the areas where you, or any other people, sat during the steaming. 30 seconds of scrubbing should be enough for regular cleaning.\n\nA. [substeps] Place towels under the handles and lower backrests to protect them from splashing water. Running the shower removes all the steam and its residue, leaving behind a warmer, healthier environment in which to work out or go to bed.\nB. Rinse the wood down with clean water when you are finished. [substeps] Keep your brush and bucket near the door to remind you to clean as you leave the sauna.\nC. [substeps] Make sure your benches and backrests are completely clean and free of dirt or debris. Be careful-some benches and backrests may crumble in a few powerful kicks.\nD. If you have a caddy for cleaning, use that instead of a squat bucket. [substeps] Aim for soft and easy motion cleansers when you clean your sauna.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to clean a sauna [title] Scrub the benches, walls, and backrests after use. [step] Dip a hand brush into a bucket of water and scrub down all the areas where you, or any other people, sat during the steaming. 30 seconds of scrubbing should be enough for regular cleaning.\n\nA. [substeps] Place towels under the handles and lower backrests to protect them from splashing water. Running the shower removes all the steam and its residue, leaving behind a warmer, healthier environment in which to work out or go to bed.\nB. Rinse the wood down with clean water when you are finished. [substeps] Keep your brush and bucket near the door to remind you to clean as you leave the sauna.\nC. [substeps] Make sure your benches and backrests are completely clean and free of dirt or debris. Be careful-some benches and backrests may crumble in a few powerful kicks.\nD. If you have a caddy for cleaning, use that instead of a squat bucket. [substeps] Aim for soft and easy motion cleansers when you clean your sauna.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1887,"error":null,"output":"A"}
{"question_id":"hellaswag:00007440","item_index":98,"attempt":0,"prompt_hash":"4d51a5d191d155d317e4dcb2c840de7c3328c79d3bbf54a0cb6e5291e2f968a9","question":"[header] How to ground yourself [title] Set up your workspace in an area without rugs or carpet. [step] This helps greatly reduce the incidence of electric shock. If working on a bare floor is not an option, consider applying a light coating of anti-static spray to the rug or carpet prior to handling electronics.\n\nA. [substeps] If you wish to cover your workspace with a rug, consider using a carbon fiber carpet to cover the floor. Avoid using mats or or other expensive materials that will corrode your ground floor.\nB. [title] Use electric shock gloves if your workspace is bare. [step] Even for freshly ground ground material, wearing gloves will allow you to avoid stepping foot on the ground without discomfort.\nC. [title] Keep pets away from your workspace. [step] Pets with hair such as dogs, cats, and ferrets can increase the risk for electric shock if they come into contact with you or your electronics.\nD. [substeps] Do not attempt to lay down x-rated electronics on a living room floor as this can cause burns to the tile or carpet. [title] Lay a floor mat or rug on a flat ground.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to ground yourself [title] Set up your workspace in an area without rugs or carpet. [step] This helps greatly reduce the incidence of electric shock. If working on a bare floor is not an option, consider applying a light coating of anti-static spray to the rug or carpet prior to handling electronics.\n\nA. [substeps] If you wish to cover your workspace with a rug, consider using a carbon fiber carpet to cover the floor. Avoid using mats or or other expensive materials that will corrode your ground floor.\nB. [title] Use electric shock gloves if your workspace is bare. [step] Even for freshly ground ground material, wearing gloves will allow you to avoid stepping foot on the ground without discomfort.\nC. [title] Keep pets away from your workspace. [step] Pets with hair such as dogs, cats, and ferrets can increase the risk for electric shock if they come into contact with you or your electronics.\nD. [substeps] Do not attempt to lay down x-rated electronics on a living room floor as this can cause burns to the tile or carpet. [title] Lay a floor mat or rug on a flat ground.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1804,"error":null,"output":"A"}
{"question_id":"hellaswag:00007848","item_index":99,"attempt":0,"prompt_hash":"4c2a9fbb0ccfaa14809a4ee11cbddf5179e1d0e749094c77a36e22c555c559e0","question":"[header] How to prevent spider mites on houseplants [title] Keep your houseplants hydrated. [step] Watering your houseplants as needed on a regular basis will prevent spider mite infestation, since spider mites only thrive on houseplants that are dry. [substeps] Use cold water instead of lukewarm water to water your houseplants.\n\nA. It is recommended to be sure that every day you water your houseplants, you may see spider mites appear on the \" float \" on the water bottles. In some cases, spider mites may appear as large clusters, rather than a single cluster.\nB. If you do not have a tank, place plants in small jars or bowls that can drain into the tank regularly. [title] Plant your plants in terracotta or porcelain pots.\nC. This may prevent weak, poorly hydrated houseplants from being able to survive an infestation. [title] Defrost your plants every six to eight weeks.\nD. Spider mites will not infest your plants if the temperature is too cold. [title] Remove dust from your houseplants once per week.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to prevent spider mites on houseplants [title] Keep your houseplants hydrated. [step] Watering your houseplants as needed on a regular basis will prevent spider mite infestation, since spider mites only thrive on houseplants that are dry. [substeps] Use cold water instead of lukewarm water to water your houseplants.\n\nA. It is recommended to be sure that every day you water your houseplants, you may see spider mites appear on the \" float \" on the water bottles. In some cases, spider mites may appear as large clusters, rather than a single cluster.\nB. If you do not have a tank, place plants in small jars or bowls that can drain into the tank regularly. [title] Plant your plants in terracotta or porcelain pots.\nC. This may prevent weak, poorly hydrated houseplants from being able to survive an infestation. [title] Defrost your plants every six to eight weeks.\nD. Spider mites will not infest your plants if the temperature is too cold. [title] Remove dust from your houseplants once per week.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1785,"error":null,"output":"C"}
{"question_id":"hellaswag:00006781","item_index":100,"attempt":0,"prompt_hash":"262b369c52ca5a7cadf86a9b000c7c8dedaff330d49f1590cdd8ca69969e2b12","question":"[header] How to style air dried hair [title] Use a moisturizing shampoo and conditioner to prevent frizz. [step] Frizz is a very common problem with air-dried hair. Combat frizz by starting in the shower with moisturizing products that smooth the hair cuticle.\n\nA. This will help reduce frizz and condition the hair. [substeps] When you use a moisturizing shampoo and conditioner, get into the habit of rubbing your hair with the moisturizing product more frequently.\nB. [substeps] Hot water is helpful for combating frizz by setting the temperature to a cool setting, like room temperature. It's also good for drying out or straightening your hair.\nC. Opt for shampoos and conditioners that contain hydrating ingredients like shea butter, argan oil, soy proteins, and moroccan oil. Try to avoid shampoos and conditioners that list sulfates or alcohol as ingredients, since these can dry out hair.\nD. You can find a wide variety of moisturizing products in your local supermarket and, if you are planning to keep your hair easily combed throughout the week, search for products with sunscreen. [title] Dry your hair with a clean towel.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to style air dried hair [title] Use a moisturizing shampoo and conditioner to prevent frizz. [step] Frizz is a very common problem with air-dried hair. Combat frizz by starting in the shower with moisturizing products that smooth the hair cuticle.\n\nA. This will help reduce frizz and condition the hair. [substeps] When you use a moisturizing shampoo and conditioner, get into the habit of rubbing your hair with the moisturizing product more frequently.\nB. [substeps] Hot water is helpful for combating frizz by setting the temperature to a cool setting, like room temperature. It's also good for drying out or straightening your hair.\nC. Opt for shampoos and conditioners that contain hydrating ingredients like shea butter, argan oil, soy proteins, and moroccan oil. Try to avoid shampoos and conditioners that list sulfates or alcohol as ingredients, since these can dry out hair.\nD. You can find a wide variety of moisturizing products in your local supermarket and, if you are planning to keep your hair easily combed throughout the week, search for products with sunscreen. [title] Dry your hair with a clean towel.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1947,"error":null,"output":"A"}
{"question_id":"hellaswag:00001829","item_index":101,"attempt":0,"prompt_hash":"deea8ff9ebd07e957e5bdccd693ee93d2bf3d57cb2c83e46534a4ee7126e13be","question":"A man is shown mowing his lawn on a hazy day. He goes back and forth and looks into the camera and seems to be struggling. he\n\nA. goes back and forth making sure to stay in a straight line.\nB. continues his mowing and looks into the camera leaves but joins another man when he's done.\nC. turns and smiles to the camera as he continues to mow his lawn.\nD. closes the mower and wanders back and forth.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA man is shown mowing his lawn on a hazy day. He goes back and forth and looks into the camera and seems to be struggling. he\n\nA. goes back and forth making sure to stay in a straight line.\nB. continues his mowing and looks into the camera leaves but joins another man when he's done.\nC. turns and smiles to the camera as he continues to mow his lawn.\nD. closes the mower and wanders back and forth.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1008,"error":null,"output":"A"}
{"question_id":"hellaswag:00006749","item_index":102,"attempt":0,"prompt_hash":"1ad16650e360406612cbfe0c9f17c4936646d98d82ac2afe684b27f65ec0d359","question":"[header] How to wear a diamond ring [title] Wear traditional engagement and wedding rings on your left ring finger. [step] Traditionally, an engagement ring is worn on your left-hand ring finger at the time of your engagement. Your wedding band is added onto that finger when you get married.\n\nA. This is so that it's not an outdated tradition for the band to go on your ring finger. As soon as you get married, put your ring on your pinky finger and loosely secure the ring with the other two fingers.\nB. This style does not have a traditional or traditional pattern. [substeps] If you wear a western ring and wear a western ring, the band should have a ranch ring on it.\nC. [substeps] Most people wear their wedding band on their finger below the engagement ring, symbolizing that it's closer to their heart. When wearing your engagement and wedding rings together, slide your wedding band onto your finger first, then your engagement ring after that.\nD. Your ring is married on your ring finger when you get married. [substeps] Popular designer engagement and wedding rings come prearranged so as to make accepting your engagement ring easier.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to wear a diamond ring [title] Wear traditional engagement and wedding rings on your left ring finger. [step] Traditionally, an engagement ring is worn on your left-hand ring finger at the time of your engagement. Your wedding band is added onto that finger when you get married.\n\nA. This is so that it's not an outdated tradition for the band to go on your ring finger. As soon as you get married, put your ring on your pinky finger and loosely secure the ring with the other two fingers.\nB. This style does not have a traditional or traditional pattern. [substeps] If you wear a western ring and wear a western ring, the band should have a ranch ring on it.\nC. [substeps] Most people wear their wedding band on their finger below the engagement ring, symbolizing that it's closer to their heart. When wearing your engagement and wedding rings together, slide your wedding band onto your finger first, then your engagement ring after that.\nD. Your ring is married on your ring finger when you get married. [substeps] Popular designer engagement and wedding rings come prearranged so as to make accepting your engagement ring easier.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1897,"error":null,"output":"C"}
{"question_id":"hellaswag:00009234","item_index":103,"attempt":0,"prompt_hash":"d98342af39ee7119288e501350f49f3c76dc05a40addb2717b365b9bb4d9bbb8","question":"[header] How to treat a yeast infection [title] Check for symptoms. [step] There are a number of physical signs that may indicate a yeast infection. The most common symptoms include : [substeps] Itching, soreness, and overall discomfort in the vaginal area.\n\nA. Abnormalities in the appearance of your vagina. Infection or sores on the vaginal walls.\nB. Itching around the lining of the vagina which changes frequently over the course of the day. Bloody vaginal discharge around the tampon.\nC. You may also notice: dry, inflamed, pus-like discharge visible drainage from the vaginal opening. Diarrhea, vaginal constipation, and a sense of heaviness (possibly due to a blockage due to a yeast infection).\nD. Pain or burning during urination or sex. Thick (like cottage cheese), white discharge in the vagina.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to treat a yeast infection [title] Check for symptoms. [step] There are a number of physical signs that may indicate a yeast infection. The most common symptoms include : [substeps] Itching, soreness, and overall discomfort in the vaginal area.\n\nA. Abnormalities in the appearance of your vagina. Infection or sores on the vaginal walls.\nB. Itching around the lining of the vagina which changes frequently over the course of the day. Bloody vaginal discharge around the tampon.\nC. You may also notice: dry, inflamed, pus-like discharge visible drainage from the vaginal opening. Diarrhea, vaginal constipation, and a sense of heaviness (possibly due to a blockage due to a yeast infection).\nD. Pain or burning during urination or sex. Thick (like cottage cheese), white discharge in the vagina.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":2430,"error":null,"output":"D. Pain or burning during urination or sex. Thick (like cottage cheese), white discharge in the vagina."}
{"question_id":"hellaswag:00009089","item_index":104,"attempt":0,"prompt_hash":"a8c309ddd2c0c5018990f96737d2e7a53c37e751a301e4b9cf28e2b13281fa56","question":"[header] How to do sunglass nails [title] File and clean your nails. [step] Use a file to shape your nails into the desired look. Wash your hands to get rid of the gritty nail particles created by the file.\n\nA. Work in a circular motion and file out any rough or rough spots on your nails. [substeps] File your nails to the point that you can see them through the file.\nB. Dry your hands thoroughly on a clean towel. Soak a cotton ball in polish remover and wipe each nail with it.\nC. [title] Apply a single coat of topcoat. [step] This will give the polish a plump and fluffy finish.\nD. Polish a cuticle cuticle file or a regular nail file. [substeps] If you prefer a cuticle-free nail polish, then you may skip the cuticle file.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to do sunglass nails [title] File and clean your nails. [step] Use a file to shape your nails into the desired look. Wash your hands to get rid of the gritty nail particles created by the file.\n\nA. Work in a circular motion and file out any rough or rough spots on your nails. [substeps] File your nails to the point that you can see them through the file.\nB. Dry your hands thoroughly on a clean towel. Soak a cotton ball in polish remover and wipe each nail with it.\nC. [title] Apply a single coat of topcoat. [step] This will give the polish a plump and fluffy finish.\nD. Polish a cuticle cuticle file or a regular nail file. [substeps] If you prefer a cuticle-free nail polish, then you may skip the cuticle file.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1422,"error":null,"output":"A"}
{"question_id":"hellaswag:00004434","item_index":105,"attempt":0,"prompt_hash":"595d1baf62563131ce253481913d8b8558f2b02e524fc553f249791fb5dcd892","question":"[header] How to give yourself a scalp massage [title] Find a relaxing place to sit. [step] Choose a comfortable chair with a solid back that can support your weight. Put your phone on silent and minimize background noises.\n\nA. Still, focus on the feel of the hands of the therapist. [substeps] For an even better sense of what it's like to have someone massage your scalp, park somewhere quiet and relaxing.\nB. [substeps] Traditionally, massages are performed in the morning, while other times of day, the practitioner will deliver the massage. One advantage of a scalp massage is that you won't have to worry about the weather.\nC. You may even want to dim the lights a bit. [substeps] If you find music relaxing, put something on, but set it to low volume.\nD. [substeps] If you're hard of hearing, try focusing on a small object nearby. If you're into music, play quietly or turn on your favorite music.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to give yourself a scalp massage [title] Find a relaxing place to sit. [step] Choose a comfortable chair with a solid back that can support your weight. Put your phone on silent and minimize background noises.\n\nA. Still, focus on the feel of the hands of the therapist. [substeps] For an even better sense of what it's like to have someone massage your scalp, park somewhere quiet and relaxing.\nB. [substeps] Traditionally, massages are performed in the morning, while other times of day, the practitioner will deliver the massage. One advantage of a scalp massage is that you won't have to worry about the weather.\nC. You may even want to dim the lights a bit. [substeps] If you find music relaxing, put something on, but set it to low volume.\nD. [substeps] If you're hard of hearing, try focusing on a small object nearby. If you're into music, play quietly or turn on your favorite music.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1719,"error":null,"output":"C"}
{"question_id":"hellaswag:00009346","item_index":106,"attempt":0,"prompt_hash":"c3451647042b340422a824e8d2badc4c4d28a0e744827609e3ea1c07a48f10be","question":"[header] How to write a complaint letter to your principal [title] Think about your options. [step] Consider if writing a letter to your principal is the best option to solve your specific problem. For some issues, there might be more effective ways to fix what's wrong.\n\nA. For example, if you lost your coat on the playground, you should check in the office's lost and found rather than addressing the issue in a letter to your principal. [substeps] Before deciding to write a letter to your principal, ask yourself a few questions.\nB. If your principal isn't the culprit, you may come up with alternative means of effective problem resolution that could lead to an acceptable resolution. [substeps] Although your principal might be fine with certain methods, sometimes there might be unethical methods, which could be more acceptable to your principal.\nC. For others, writing a letter is better, but ultimately, its only working if you can correctly discuss what's bugging you and your problem with a supervisor who knows about the problem. [substeps] For instance, if your boss used to tell you that your grades didn't improve when you went to college, your letter might be better received.\nD. For example, writing a letter to your teacher may seem like a reasonable solution in the moment, but it's likely to result in the student being expelled anyway. People typically refuse to cooperate in reporting misconduct, so it might be more wise to write a letter.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to write a complaint letter to your principal [title] Think about your options. [step] Consider if writing a letter to your principal is the best option to solve your specific problem. For some issues, there might be more effective ways to fix what's wrong.\n\nA. For example, if you lost your coat on the playground, you should check in the office's lost and found rather than addressing the issue in a letter to your principal. [substeps] Before deciding to write a letter to your principal, ask yourself a few questions.\nB. If your principal isn't the culprit, you may come up with alternative means of effective problem resolution that could lead to an acceptable resolution. [substeps] Although your principal might be fine with certain methods, sometimes there might be unethical methods, which could be more acceptable to your principal.\nC. For others, writing a letter is better, but ultimately, its only working if you can correctly discuss what's bugging you and your problem with a supervisor who knows about the problem. [substeps] For instance, if your boss used to tell you that your grades didn't improve when you went to college, your letter might be better received.\nD. For example, writing a letter to your teacher may seem like a reasonable solution in the moment, but it's likely to result in the student being expelled anyway. People typically refuse to cooperate in reporting misconduct, so it might be more wise to write a letter.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":2364,"error":null,"output":"C"}
{"question_id":"hellaswag:00002555","item_index":107,"attempt":0,"prompt_hash":"20e56cec3e43f35283a364bf5b70f315bbeafd991631713dc3e4237d47802fac","question":"A person is moving the lawn. The person steps over the orange cord. the person\n\nA. uses the trowel to hit the yellow cord.\nB. moves the mower over the brick walkway.\nC. moves the hedge a second time.\nD. moves the middle of the hedge.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA person is moving the lawn. The person steps over the orange cord. the person\n\nA. uses the trowel to hit the yellow cord.\nB. moves the mower over the brick walkway.\nC. moves the hedge a second time.\nD. moves the middle of the hedge.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":813,"error":null,"output":"C"}
{"question_id":"hellaswag:00006207","item_index":108,"attempt":0,"prompt_hash":"5d9d1d2510982e7f6b949177e15d10875fadd67941d06cd34823eee42b289821","question":"[header] How to put on a loincloth [title] Use leather for a comfortable and durable loincloth. [step] Leather is a common material used in most loincloths. Try using leather for a more durable and traditional loincloth.\n\nA. They provide great protection against the elements and are more and more comfortable. They are long and stick to the lower body, though they can also be added with feathers.\nB. Cloth looks appropriate compared to other fabrics and also hides some of the body's flaws so you look a bit like that particular man. [substeps] Leather, on average, is a breathable, breathable material (especially when it's slept on).\nC. Cooler areas are probably best, as they do not provide as much material and can be more expensive. However, the lightcloth should be fairly breathable, as it will prevent excessive sweating.\nD. However, leather can be quite warm and is not a very breathable material. Certain leathers can be bought in bulk at various fabric or textile supply stores.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to put on a loincloth [title] Use leather for a comfortable and durable loincloth. [step] Leather is a common material used in most loincloths. Try using leather for a more durable and traditional loincloth.\n\nA. They provide great protection against the elements and are more and more comfortable. They are long and stick to the lower body, though they can also be added with feathers.\nB. Cloth looks appropriate compared to other fabrics and also hides some of the body's flaws so you look a bit like that particular man. [substeps] Leather, on average, is a breathable, breathable material (especially when it's slept on).\nC. Cooler areas are probably best, as they do not provide as much material and can be more expensive. However, the lightcloth should be fairly breathable, as it will prevent excessive sweating.\nD. However, leather can be quite warm and is not a very breathable material. Certain leathers can be bought in bulk at various fabric or textile supply stores.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1709,"error":null,"output":"A"}
{"question_id":"hellaswag:00000945","item_index":109,"attempt":0,"prompt_hash":"49960f3ca76c5a0bc1e04359072fd7f7099fdd1907c41b1df3701b7f75d0806f","question":"A girl is jumping and swinging on balance bars. A woman in a black shirt is standing under her watching. she\n\nA. flips over the bar and lands on her back.\nB. helps her jump off and land on a mat.\nC. then jumps and swings with her legs.\nD. gives the young girl a high five.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA girl is jumping and swinging on balance bars. A woman in a black shirt is standing under her watching. she\n\nA. flips over the bar and lands on her back.\nB. helps her jump off and land on a mat.\nC. then jumps and swings with her legs.\nD. gives the young girl a high five.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":836,"error":null,"output":"C"}
{"question_id":"hellaswag:00000514","item_index":110,"attempt":0,"prompt_hash":"6870a7a182c119d08995a6927ef08be42e55220b7b10e0173d284c106c73e1a9","question":"A man is shown riding down a snowy embankment. He is using snow skis to ski down a snow covered hill. he\n\nA. gets to the bottom of the hill, where he drops off his board.\nB. moves quickly, doing a flip as he goes.\nC. goes over a winter covered hill several times.\nD. uses the rope to hold him up while continuing to ride.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA man is shown riding down a snowy embankment. He is using snow skis to ski down a snow covered hill. he\n\nA. gets to the bottom of the hill, where he drops off his board.\nB. moves quickly, doing a flip as he goes.\nC. goes over a winter covered hill several times.\nD. uses the rope to hold him up while continuing to ride.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":728,"error":null,"output":"A"}
{"question_id":"hellaswag:00003924","item_index":111,"attempt":0,"prompt_hash":"d6a4ff66797dc0b72f0ab1e1020a57b0bc1bf7d0df25dcf06b0b4a00813ae70b","question":"[header] How to cut a girl's hair [title] Start with washed hair. [step] The hair should be clean and damp when cutting. After the hair is washed, towel dry it to remove excess water, and then brush out any knots.\n\nA. [substeps] You can cut your hair straight or down. Your preference will be whatever looks best for you.\nB. [substeps] If you've already washed your hair, the gentle cycle will give you no qualms. If you feel your hair is too wet and needs to be gentle, consider letting it air dry.\nC. [substeps] Brush from the ends, and work your way up to the root to work out knots. This will prevent damage to the hair and will put less strain on the scalp.\nD. Then apply conditioner and finish by misting the hair with a diffuser to encourage the roots to move. [substeps] Avoid using shampoo or conditioner that can stain wet hair.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to cut a girl's hair [title] Start with washed hair. [step] The hair should be clean and damp when cutting. After the hair is washed, towel dry it to remove excess water, and then brush out any knots.\n\nA. [substeps] You can cut your hair straight or down. Your preference will be whatever looks best for you.\nB. [substeps] If you've already washed your hair, the gentle cycle will give you no qualms. If you feel your hair is too wet and needs to be gentle, consider letting it air dry.\nC. [substeps] Brush from the ends, and work your way up to the root to work out knots. This will prevent damage to the hair and will put less strain on the scalp.\nD. Then apply conditioner and finish by misting the hair with a diffuser to encourage the roots to move. [substeps] Avoid using shampoo or conditioner that can stain wet hair.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1693,"error":null,"output":"C"}
{"question_id":"hellaswag:00003476","item_index":112,"attempt":0,"prompt_hash":"e3d8bcc0fe5339054960e40af193d7c4f61699c9d6bc879c72024c990416d28e","question":"[header] How to deal with an unexpected menstruation [title] Carry sanitary products, if you are expecting your period. [step] Have a change of underwear. Wear a thin, comfy panty liner if you are due on or, even if the super thin ones will help protect your clothes for a little while, especially if your flow is light.\n\nA. Remember, all you really have to do is stop the blood making mess of you and your clothes, it's not a huge issue. [title] Use toilet paper, napkins, or tissues as sanitary protection.\nB. Launder your pants if you are going to work. Wipe your vagina up with mentheliac or fertile birth control oil if you have one.\nC. Always remember not to use disposable underwear; if you are late, you can clean yourself up with a soft, sanitary bag or a tampon. Even if you miss an important moment on your period, you will still be going clean.\nD. Cotton should be your safest item of clothing. If you are hoping for a light, breathable look, and have gotten on a pair of jeans and a shirt or dress, do so.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to deal with an unexpected menstruation [title] Carry sanitary products, if you are expecting your period. [step] Have a change of underwear. Wear a thin, comfy panty liner if you are due on or, even if the super thin ones will help protect your clothes for a little while, especially if your flow is light.\n\nA. Remember, all you really have to do is stop the blood making mess of you and your clothes, it's not a huge issue. [title] Use toilet paper, napkins, or tissues as sanitary protection.\nB. Launder your pants if you are going to work. Wipe your vagina up with mentheliac or fertile birth control oil if you have one.\nC. Always remember not to use disposable underwear; if you are late, you can clean yourself up with a soft, sanitary bag or a tampon. Even if you miss an important moment on your period, you will still be going clean.\nD. Cotton should be your safest item of clothing. If you are hoping for a light, breathable look, and have gotten on a pair of jeans and a shirt or dress, do so.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1964,"error":null,"output":"C"}
{"question_id":"hellaswag:00007666","item_index":113,"attempt":0,"prompt_hash":"9759e1288c8d501e490ecbb9eea6a558a1b725ea22ab6b02db605b5e108f71de","question":"[header] How to clean asphalt [title] Remove loose residue from the asphalt surface. [step] Use the oblong push broom to sweep away leaves, grass cuttings, loose dirt and other residue that is lightly resting on the surface. Removing all loose dirt and grime will make the task of actually cleaning the asphalt much easier.\n\nA. [substeps] Don't forget to throw away any leaf litter. When you are cleaning asphalt, you'll need to put it in a plastic bag to protect it from getting snagged on any dirt, grass and other debris that is usually on the asphalt surface.\nB. [title] Soak the asphalt with a steady stream of water. [step] Use the garden hose to apply the water over the surface.\nC. You may need to use a hose or a rake for this. Do not use a brush or any motor as this does not work as well or as efficiently as a brush or rake.\nD. [substeps] Be sure to be as precise and thorough as possible so you don't scratch the surface of the asphalt. [title] Use a mild solvent like hydrometer to wipe off any crusted-on flakes and debris.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to clean asphalt [title] Remove loose residue from the asphalt surface. [step] Use the oblong push broom to sweep away leaves, grass cuttings, loose dirt and other residue that is lightly resting on the surface. Removing all loose dirt and grime will make the task of actually cleaning the asphalt much easier.\n\nA. [substeps] Don't forget to throw away any leaf litter. When you are cleaning asphalt, you'll need to put it in a plastic bag to protect it from getting snagged on any dirt, grass and other debris that is usually on the asphalt surface.\nB. [title] Soak the asphalt with a steady stream of water. [step] Use the garden hose to apply the water over the surface.\nC. You may need to use a hose or a rake for this. Do not use a brush or any motor as this does not work as well or as efficiently as a brush or rake.\nD. [substeps] Be sure to be as precise and thorough as possible so you don't scratch the surface of the asphalt. [title] Use a mild solvent like hydrometer to wipe off any crusted-on flakes and debris.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":2000,"error":null,"output":"A"}
{"question_id":"hellaswag:00007408","item_index":114,"attempt":0,"prompt_hash":"2d1598c3451f019bfb630b5a7844101771c3249a6cec745a6ba96419c93ecbb3","question":"[header] How to use tiki torches [title] Place torches 6-8 feet (1.8-2.4 m) (1.83-2.44 m) apart. [step] Use a measuring tape to space your torches in your yard. This will help prevent accidents if you accidentally knock a torch over.\n\nA. Use separate torches for different targets (e.g. cooking liquids), as well as single torches for irregularities.\nB. [title] Avoid placing tiki torches near overhangs or other flammable materials. [step] Plant tiki torches away from trees, porches, awnings, or hanging party decorations.\nC. Torches should ideally be at least 3 feet (1.9 m) apart, so you want 7-12 feet (4.7-4.9 m) of space between each torch. [title] Equip each torch with a few feet of wire, weltle, or beam.\nD. Make sure that you have at least an extra meter of space in your yard between torches and the ground. [substeps] If you have more than six torches, place them two meters (1.1 m) apart.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to use tiki torches [title] Place torches 6-8 feet (1.8-2.4 m) (1.83-2.44 m) apart. [step] Use a measuring tape to space your torches in your yard. This will help prevent accidents if you accidentally knock a torch over.\n\nA. Use separate torches for different targets (e.g. cooking liquids), as well as single torches for irregularities.\nB. [title] Avoid placing tiki torches near overhangs or other flammable materials. [step] Plant tiki torches away from trees, porches, awnings, or hanging party decorations.\nC. Torches should ideally be at least 3 feet (1.9 m) apart, so you want 7-12 feet (4.7-4.9 m) of space between each torch. [title] Equip each torch with a few feet of wire, weltle, or beam.\nD. Make sure that you have at least an extra meter of space in your yard between torches and the ground. [substeps] If you have more than six torches, place them two meters (1.1 m) apart.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":2000,"error":null,"output":"C"}
{"question_id":"hellaswag:00009424","item_index":115,"attempt":0,"prompt_hash":"c27f2e7345654a2257b9008399ab6431aa659bbf5a015e68ff1694f8e95c2a96","question":"[header] How to buy your daughter's first bra [title] Remind your daughter that breasts come in different shapes. [step] Talk to your daughter about the kind of bra she would like. If she is confused, let her know what might best work with her size and shape.\n\nA. Give her cues about what different styles would look good together. [substeps] For example, you might say, \" look at this guide; how would you feel about a large bra? \" also be sure to consider the basic shapes and subshapes of your breasts.\nB. Let your daughter know that every body is different, which may also help her better accept her changing shape. [substeps] Explain to your daughter that every person's breasts are different and that she may even need something different than you.\nC. As a result, she can better preform a good corset based on her skin's needs. [substeps] You can say something like, \" your top should have a pump in it so your breasts don't get wet if you let your body sweat for a long time.\nD. [substeps] Make sure to ask your daughter what she would like the shape of. You might say, \" i love larger breasts, but i don't think that could work with the size of my chest.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to buy your daughter's first bra [title] Remind your daughter that breasts come in different shapes. [step] Talk to your daughter about the kind of bra she would like. If she is confused, let her know what might best work with her size and shape.\n\nA. Give her cues about what different styles would look good together. [substeps] For example, you might say, \" look at this guide; how would you feel about a large bra? \" also be sure to consider the basic shapes and subshapes of your breasts.\nB. Let your daughter know that every body is different, which may also help her better accept her changing shape. [substeps] Explain to your daughter that every person's breasts are different and that she may even need something different than you.\nC. As a result, she can better preform a good corset based on her skin's needs. [substeps] You can say something like, \" your top should have a pump in it so your breasts don't get wet if you let your body sweat for a long time.\nD. [substeps] Make sure to ask your daughter what she would like the shape of. You might say, \" i love larger breasts, but i don't think that could work with the size of my chest.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":2091,"error":null,"output":"B"}
{"question_id":"hellaswag:00004858","item_index":116,"attempt":0,"prompt_hash":"9f8d9aeba756e22b3a3d81c2c581fe9b76566ef0e77b0bd56216f1c2783000d2","question":"[header] How to tell if you have body odor [title] Smell yourself. [step] Your first line of defense against troublesome body odor is a keen sense of smell. Take a whiff of potential problem areas, including your underarms, feet and genitals.\n\nA. Pop something soft, like a soft quilt, into your nose for a few seconds. Then, gently dust your face with a washcloth and let it soak into your skin.\nB. Try to incorporate all facial fluids into your body. By applying some colognes or moisturizing agents while showering and leaving the bathroom after showering in order to help reduce odor, you can help avoid body odor.\nC. Place a clean cotton ball in your ear canal and smell around your body for other smell-sensitive areas. The best spot to smell is your armpit.\nD. While your own body odor is typically tough to distinguish, you may be able to pick up on it if it's strong enough. [substeps] See if you can detect any salty, musty or pungent notes.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to tell if you have body odor [title] Smell yourself. [step] Your first line of defense against troublesome body odor is a keen sense of smell. Take a whiff of potential problem areas, including your underarms, feet and genitals.\n\nA. Pop something soft, like a soft quilt, into your nose for a few seconds. Then, gently dust your face with a washcloth and let it soak into your skin.\nB. Try to incorporate all facial fluids into your body. By applying some colognes or moisturizing agents while showering and leaving the bathroom after showering in order to help reduce odor, you can help avoid body odor.\nC. Place a clean cotton ball in your ear canal and smell around your body for other smell-sensitive areas. The best spot to smell is your armpit.\nD. While your own body odor is typically tough to distinguish, you may be able to pick up on it if it's strong enough. [substeps] See if you can detect any salty, musty or pungent notes.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1784,"error":null,"output":"C"}
{"question_id":"hellaswag:00006403","item_index":117,"attempt":0,"prompt_hash":"7ab60cb0bf717bebf88986f6fb4d778e410e20e36a303b95024e304767eafac9","question":"[header] How to make citrus gravy [title] Take the meat out of the roasting pan. [step] To make a delicious gravy, you need the drippings from a turkey or chicken. After it's finished roasting, take the pan out of the oven and remove the meat so you can mix the gravy in the pan.\n\nA. Season the gravy in your favorite spices. [substeps] For a little sweeter gravy, you can season it with ginger or garlic oil, onion powder, olive oil, or your favorite spices.\nB. [substeps] Use a grater to wash your ribs and remove the fat from the meat. Set the meat aside in a clean dish.\nC. [substeps] If you're not ready to eat the turkey or chicken right away, leave it on the cutting board. Place a sheet of foil over it, though, to keep it warm.\nD. Using a wooden fork, mix all the liquids together until you have a runny gravy that's like a broth; it should be thick but clean. [substeps] It's important to stir all the liquids well into the gravy.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make citrus gravy [title] Take the meat out of the roasting pan. [step] To make a delicious gravy, you need the drippings from a turkey or chicken. After it's finished roasting, take the pan out of the oven and remove the meat so you can mix the gravy in the pan.\n\nA. Season the gravy in your favorite spices. [substeps] For a little sweeter gravy, you can season it with ginger or garlic oil, onion powder, olive oil, or your favorite spices.\nB. [substeps] Use a grater to wash your ribs and remove the fat from the meat. Set the meat aside in a clean dish.\nC. [substeps] If you're not ready to eat the turkey or chicken right away, leave it on the cutting board. Place a sheet of foil over it, though, to keep it warm.\nD. Using a wooden fork, mix all the liquids together until you have a runny gravy that's like a broth; it should be thick but clean. [substeps] It's important to stir all the liquids well into the gravy.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1882,"error":null,"output":"D"}
{"question_id":"hellaswag:00004635","item_index":118,"attempt":0,"prompt_hash":"ca8b2c2a4630d910b92649f16e3b8d56f656ff44fba5fce960fa62321befbcc4","question":"[header] How to drape your lehenga dupatta [title] Make narrow pleats in your lehenga dupatta. [step] Fold your dupatta lengthwise to create narrow pleats. Then, position and pin your dupatta onto your right shoulder.\n\nA. [title] Fold the dupatta from front to back, creating a square fold. [step] Use your end result to create a square square on your right shoulder.\nB. Keep the pleats on the right side of your shoulder. [title] Pin the dupatta to your navel.\nC. [title] Cross your right ankle over your left foot. [step] Using your free hand, cross one leg over your left ankle.\nD. Lay your pleated lehenga dupatta over your right shoulder so that most of the excess material hangs down in back. The material in front should hang down to your hip.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to drape your lehenga dupatta [title] Make narrow pleats in your lehenga dupatta. [step] Fold your dupatta lengthwise to create narrow pleats. Then, position and pin your dupatta onto your right shoulder.\n\nA. [title] Fold the dupatta from front to back, creating a square fold. [step] Use your end result to create a square square on your right shoulder.\nB. Keep the pleats on the right side of your shoulder. [title] Pin the dupatta to your navel.\nC. [title] Cross your right ankle over your left foot. [step] Using your free hand, cross one leg over your left ankle.\nD. Lay your pleated lehenga dupatta over your right shoulder so that most of the excess material hangs down in back. The material in front should hang down to your hip.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1686,"error":null,"output":"D"}
{"question_id":"hellaswag:00007630","item_index":119,"attempt":0,"prompt_hash":"8dd8cb0bd6408f3e91108d257e1c2c5c234a8b185577d8ccb2a9bbd26e17f306","question":"[header] How to manage curly hair [title] Go medium or long. [step] If you have curly hair, and go too short, you could end up with hair that has too much poof or looks misshapen. The key to managing curly hair can simply be the right cut.\n\nA. At minimum, choose scissors with a quality brush, low-grit paper and fine-grit paper. A ½-inch regular line is most effective.\nB. That doesn't mean you shouldn't ever try a short cut. It means you should do it with caution.\nC. If you want soft hair, don't let it guide your cutting. [title] Separate damp or wavy hair into mini-ponytails.\nD. Try to find a person with hair who is both natural and curly with your haircut in mind. Do not cut just a few strands at the end.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to manage curly hair [title] Go medium or long. [step] If you have curly hair, and go too short, you could end up with hair that has too much poof or looks misshapen. The key to managing curly hair can simply be the right cut.\n\nA. At minimum, choose scissors with a quality brush, low-grit paper and fine-grit paper. A ½-inch regular line is most effective.\nB. That doesn't mean you shouldn't ever try a short cut. It means you should do it with caution.\nC. If you want soft hair, don't let it guide your cutting. [title] Separate damp or wavy hair into mini-ponytails.\nD. Try to find a person with hair who is both natural and curly with your haircut in mind. Do not cut just a few strands at the end.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1688,"error":null,"output":"B"}
{"question_id":"hellaswag:00004295","item_index":120,"attempt":0,"prompt_hash":"39e22f397ddc02d7ffcc3351f46b401aaa244162065037fc210db8ca856b7078","question":"[header] How to use a chronograph watch [title] Pick a chronograph watch that is easy to read. [step] Chronograph watches have many small features that can be difficult to see. Choose a watch with a clear face that showcases each sub-dial clearly.\n\nA. [title] Determine whether you wish to use the watch for pleasure or for pain. [step] Using the watch for pleasure allows you to watch different things at different times without losing track of the entire event.\nB. It can also feature a compass and can be used to make a series of predetermined measurements. [title] Cut two rectangles out of the outside of the watch.\nC. Make sure that the hands on your watch contrast the color of the watch face, making them easy to see. [substeps] Be sure to try on a watch before buying it to make sure that you can see the numbers on your wrist without having to bring it up to your face.\nD. [substeps] If you have trouble distinguishing chronograph watches from their counterparts, ask a friend to take the watch with you. Alternatively, purchase a watch as a business object.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to use a chronograph watch [title] Pick a chronograph watch that is easy to read. [step] Chronograph watches have many small features that can be difficult to see. Choose a watch with a clear face that showcases each sub-dial clearly.\n\nA. [title] Determine whether you wish to use the watch for pleasure or for pain. [step] Using the watch for pleasure allows you to watch different things at different times without losing track of the entire event.\nB. It can also feature a compass and can be used to make a series of predetermined measurements. [title] Cut two rectangles out of the outside of the watch.\nC. Make sure that the hands on your watch contrast the color of the watch face, making them easy to see. [substeps] Be sure to try on a watch before buying it to make sure that you can see the numbers on your wrist without having to bring it up to your face.\nD. [substeps] If you have trouble distinguishing chronograph watches from their counterparts, ask a friend to take the watch with you. Alternatively, purchase a watch as a business object.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1822,"error":null,"output":"A"}
{"question_id":"hellaswag:00002983","item_index":121,"attempt":0,"prompt_hash":"78ded153a603359d017eaf2b4b9299e56db02f4bb3ff1d6a809126f24cbc4b1f","question":"Bright pink french cancan candy wrap is shown. the materials, tools and and how to make it\n\nA. are shown in first person perspective.\nB. are demonstrated step by step.\nC. are all shown.\nD. appear shiny are displayed on a white counter.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nBright pink french cancan candy wrap is shown. the materials, tools and and how to make it\n\nA. are shown in first person perspective.\nB. are demonstrated step by step.\nC. are all shown.\nD. appear shiny are displayed on a white counter.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":680,"error":null,"output":"B"}
{"question_id":"hellaswag:00009184","item_index":122,"attempt":0,"prompt_hash":"445abb0a4ef6e08652f882173deefeb4861068e248613ebd1e458c965be5c2c7","question":"[header] How to encourage good behaviour in children [title] Tell your kids what you are planning. [step] If they still don't understand spankings and time-out is is still in the picture. Even if they are a little old.\n\nA. Tell them what you are doing when they are old enough. When they understand how punishment works, they will sense the adult is helping them to appreciate and soften the emotional impact of the punishment.\nB. So take some time to explain to them the main purpose of time out. [title] Remind them to fully understand the importance of occasional spankings.\nC. Even if they have little brain power, tell them that it is a good idea to learn to work out attention-management strategies. As for careers, chances are your kids are already interested in something else.\nD. [title] Tell them that they need to change their behaviour. [step] Lay down the law so when you get to the next step they know that something is going to change.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to encourage good behaviour in children [title] Tell your kids what you are planning. [step] If they still don't understand spankings and time-out is is still in the picture. Even if they are a little old.\n\nA. Tell them what you are doing when they are old enough. When they understand how punishment works, they will sense the adult is helping them to appreciate and soften the emotional impact of the punishment.\nB. So take some time to explain to them the main purpose of time out. [title] Remind them to fully understand the importance of occasional spankings.\nC. Even if they have little brain power, tell them that it is a good idea to learn to work out attention-management strategies. As for careers, chances are your kids are already interested in something else.\nD. [title] Tell them that they need to change their behaviour. [step] Lay down the law so when you get to the next step they know that something is going to change.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1809,"error":null,"output":"A."}
{"question_id":"hellaswag:00004447","item_index":123,"attempt":0,"prompt_hash":"aa1825f7a7626e604c214639805fef5fb31791aa3c5ec90cc2030ee0970c131c","question":"[header] How to prevent insomnia [title] Create a relaxing bedtime ritual. [step] It's important to have a relaxing ritual to engage in before bed. Having an activity you regularly engage in before bed can help signal to your mind and body that it's time to sleep.\n\nA. Relaxation techniques before bed can also help the brain wind down. [substeps] Deep breathing can help aid in sleep.\nB. [substeps] This will also help you create a bedtime ritual. This is important, because reading, listening to music, reading a book, writing in the dark, or doing something you enjoy will all disrupt your sleep.\nC. It might be something simple like going to the bathroom in the morning, reading a book in the afternoon, or taking a nap. [substeps] No matter what time of day you have to get out of bed, take a few minutes to commit it to your activity and behavior habits.\nD. [substeps] This might take some trial and error. Don't get overwhelmed by the idea of relaxing before bed.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to prevent insomnia [title] Create a relaxing bedtime ritual. [step] It's important to have a relaxing ritual to engage in before bed. Having an activity you regularly engage in before bed can help signal to your mind and body that it's time to sleep.\n\nA. Relaxation techniques before bed can also help the brain wind down. [substeps] Deep breathing can help aid in sleep.\nB. [substeps] This will also help you create a bedtime ritual. This is important, because reading, listening to music, reading a book, writing in the dark, or doing something you enjoy will all disrupt your sleep.\nC. It might be something simple like going to the bathroom in the morning, reading a book in the afternoon, or taking a nap. [substeps] No matter what time of day you have to get out of bed, take a few minutes to commit it to your activity and behavior habits.\nD. [substeps] This might take some trial and error. Don't get overwhelmed by the idea of relaxing before bed.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1844,"error":null,"output":"B"}
{"question_id":"hellaswag:00005725","item_index":124,"attempt":0,"prompt_hash":"5c2a03f353dde9f8fd6be946bbe97477ce5c405fe7fd771bf4c90be1e67c7ece","question":"[header] How to be beautiful, just like lauren conrad [title] Gain self-confidence. [step] This may sound a bit clichéd, but confidence is the most beautiful quality in a person and lauren conrad has lots of it. You can't be afraid to be yourself or wear what you want.\n\nA. She makes sure that you get what you want, and that you have an confidence that can grow. [substeps] The key to having an amazing self-confidence is losing it.\nB. Confidence is most important for lauren to have. [title] Picture herself as a gorgeous goddess who is beautiful, no matter what.\nC. The key step to being as beautiful as her is to be as confident as her. To be confident, you have to love who you are and don't beat yourself up.\nD. Dress up, wear a dress, and be nice with whomever you see. But remember that they're still you.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to be beautiful, just like lauren conrad [title] Gain self-confidence. [step] This may sound a bit clichéd, but confidence is the most beautiful quality in a person and lauren conrad has lots of it. You can't be afraid to be yourself or wear what you want.\n\nA. She makes sure that you get what you want, and that you have an confidence that can grow. [substeps] The key to having an amazing self-confidence is losing it.\nB. Confidence is most important for lauren to have. [title] Picture herself as a gorgeous goddess who is beautiful, no matter what.\nC. The key step to being as beautiful as her is to be as confident as her. To be confident, you have to love who you are and don't beat yourself up.\nD. Dress up, wear a dress, and be nice with whomever you see. But remember that they're still you.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1722,"error":null,"output":"C"}
{"question_id":"hellaswag:00003718","item_index":125,"attempt":0,"prompt_hash":"0cf1fd05c90b94c686b1c89666eaed681784c61654c5e8c4c7888bc8322db111","question":"[header] How to dress for an outdoor wedding [title] Wear cool outfits for spring and summer weddings. [step] Selecting an outfit that you won't sweat through is key. However, be sure to keep your choice classy and suitable for the occasion.\n\nA. If you'll be spending a lot of time outdoors, try on clothing that you want to dress down for the occasion. [title] Make a list of basic ingredients and supplies.\nB. You don't want to come on too strong-pressed dress is better for a fall wedding, although you could still wear some fun shoes, if summer comes around instead. [substeps] Run a few errands for your wedding before you plan on wearing a suit to the reception.\nC. [substeps] You can often wear sleeveless, strapless, and halter wedding dresses for an outdoor summer wedding, and may choose to wear a matching jacket at night when the sun goes down. You should, on the other hand, feel appropriately formal in dress pants and a button-down shirt with a suit jacket over it.\nD. Keep your clothing as neutral as possible, and don't go overboard with accessories. [substeps] For example, try on cool colors for chilly outdoor weddings.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to dress for an outdoor wedding [title] Wear cool outfits for spring and summer weddings. [step] Selecting an outfit that you won't sweat through is key. However, be sure to keep your choice classy and suitable for the occasion.\n\nA. If you'll be spending a lot of time outdoors, try on clothing that you want to dress down for the occasion. [title] Make a list of basic ingredients and supplies.\nB. You don't want to come on too strong-pressed dress is better for a fall wedding, although you could still wear some fun shoes, if summer comes around instead. [substeps] Run a few errands for your wedding before you plan on wearing a suit to the reception.\nC. [substeps] You can often wear sleeveless, strapless, and halter wedding dresses for an outdoor summer wedding, and may choose to wear a matching jacket at night when the sun goes down. You should, on the other hand, feel appropriately formal in dress pants and a button-down shirt with a suit jacket over it.\nD. Keep your clothing as neutral as possible, and don't go overboard with accessories. [substeps] For example, try on cool colors for chilly outdoor weddings.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1986,"error":null,"output":"C"}
{"question_id":"hellaswag:00007670","item_index":126,"attempt":0,"prompt_hash":"e9b0a726c09070f2448d80091fdb111ca66653d8b27b686c832919595ede76c9","question":"[header] How to remove car battery terminals [title] Open the hood of the car and use the support bar to prop it open. [title] Locate the battery within the engine compartment. [step] If you're unsure about which part is the battery, then check your owner's manual.\n\nA. [title] Slide the body of the battery off of the alternator. [step] Then disconnect the alternator with a pair of pliers.\nB. Most vehicles have bottom drain hoses, but hoses in your engine should also be labelled with this information. To make it easier, just tug the bottom of the hose hose out of the compartment.\nC. Some vehicles have their battery in the trunk, under or behind an access panel. [title] Check the positive post to be sure that the cover is on.\nD. [title] Review the tab on the left sidewall of the car and see if there is a loose screw or another resistant locking mechanism. [step] Sometimes the battery actually lies in the contact area.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to remove car battery terminals [title] Open the hood of the car and use the support bar to prop it open. [title] Locate the battery within the engine compartment. [step] If you're unsure about which part is the battery, then check your owner's manual.\n\nA. [title] Slide the body of the battery off of the alternator. [step] Then disconnect the alternator with a pair of pliers.\nB. Most vehicles have bottom drain hoses, but hoses in your engine should also be labelled with this information. To make it easier, just tug the bottom of the hose hose out of the compartment.\nC. Some vehicles have their battery in the trunk, under or behind an access panel. [title] Check the positive post to be sure that the cover is on.\nD. [title] Review the tab on the left sidewall of the car and see if there is a loose screw or another resistant locking mechanism. [step] Sometimes the battery actually lies in the contact area.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1795,"error":null,"output":"C"}
{"question_id":"hellaswag:00004049","item_index":127,"attempt":0,"prompt_hash":"a1395989e20730f21e023420d37e43b2faa6b9d2759a58929ccbc6735fc33ddb","question":"[header] How to brew japanese sencha green tea [title] Pour enough boiling water into an empty teapot to fill everyone's cup. [step] The teapot should not contain any teas leaves at this point. [title] Pour the water into each cup, filling it to the desired level (usually about 80% to 90% full).\n\nA. [step] Do not add the teabags yet. [substeps] It is best to start from the bottom and increase the steeping time until the tea reaches about the size of a small serving, about 16 to 23 teabags.\nB. [step] This will cool the water from boiling temperature to the desired 80 ° c (176 ° f). This is key for green tea--the water should not be boiling.\nC. [step] Place 1-2 full tea bags over each cup. [title] On top of each tea bag, collect boiled shignum moss.\nD. [step] Do not put too much water in, as you do not want to churn the tea too much. The amount of water you add depends on how full the cup should be; however, about 30 milliliters (230 liters) of water will be fine.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to brew japanese sencha green tea [title] Pour enough boiling water into an empty teapot to fill everyone's cup. [step] The teapot should not contain any teas leaves at this point. [title] Pour the water into each cup, filling it to the desired level (usually about 80% to 90% full).\n\nA. [step] Do not add the teabags yet. [substeps] It is best to start from the bottom and increase the steeping time until the tea reaches about the size of a small serving, about 16 to 23 teabags.\nB. [step] This will cool the water from boiling temperature to the desired 80 ° c (176 ° f). This is key for green tea--the water should not be boiling.\nC. [step] Place 1-2 full tea bags over each cup. [title] On top of each tea bag, collect boiled shignum moss.\nD. [step] Do not put too much water in, as you do not want to churn the tea too much. The amount of water you add depends on how full the cup should be; however, about 30 milliliters (230 liters) of water will be fine.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1997,"error":null,"output":"A"}
{"question_id":"hellaswag:00007885","item_index":128,"attempt":0,"prompt_hash":"795c412b5bb6bdf39c26ea54e94f14f64488a63571d128b390a5ace47c0d57b2","question":"[header] How to measure a saddle [title] Learn what the bars are. [step] When you're looking at new saddles, one of the things you'll need to pay attention to is the bars. The bars are the weight-distributing foundation for the saddle; the part of the saddle that rests on the horse and holds you up.\n\nA. There are two' bars' that distribute weight evenly on either side of the spine. If your saddle is well-fitting, the horse's back will be in contact with the entire length of the bars.\nB. Most new saddles have bars on their backs that hang to the sides. [substeps] If your horse's saddle only has bars, flip the bars over so that the back is facing away from you.\nC. The bars are located at the top of the horse and at either end of the saddle, which are the sides that you'll almost touch. [substeps] The bars range from two-hundred-200.\nD. While the bars are peerequisite for the saddle, they're also useful if you want an added comfort. [substeps] For a standard saddle, half of the weight on the bar weights toward the seat rather than the rear.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to measure a saddle [title] Learn what the bars are. [step] When you're looking at new saddles, one of the things you'll need to pay attention to is the bars. The bars are the weight-distributing foundation for the saddle; the part of the saddle that rests on the horse and holds you up.\n\nA. There are two' bars' that distribute weight evenly on either side of the spine. If your saddle is well-fitting, the horse's back will be in contact with the entire length of the bars.\nB. Most new saddles have bars on their backs that hang to the sides. [substeps] If your horse's saddle only has bars, flip the bars over so that the back is facing away from you.\nC. The bars are located at the top of the horse and at either end of the saddle, which are the sides that you'll almost touch. [substeps] The bars range from two-hundred-200.\nD. While the bars are peerequisite for the saddle, they're also useful if you want an added comfort. [substeps] For a standard saddle, half of the weight on the bar weights toward the seat rather than the rear.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":2037,"error":null,"output":"A"}
{"question_id":"hellaswag:00003027","item_index":129,"attempt":0,"prompt_hash":"544e29e4f56c0edc569c747cd82de5334139197825b2ce4ab29f6a9ecf08ce5c","question":"A person is seen sitting on the snow around a hole and loading up a pole. he\n\nA. throws the pole down on to the snow next to his.\nB. jumps down next to the hole and moves off.\nC. spins the pole around several times and begins skating around a site.\nD. looks at a machine and continues adjusting his pole.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA person is seen sitting on the snow around a hole and loading up a pole. he\n\nA. throws the pole down on to the snow next to his.\nB. jumps down next to the hole and moves off.\nC. spins the pole around several times and begins skating around a site.\nD. looks at a machine and continues adjusting his pole.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":829,"error":null,"output":"C"}
{"question_id":"hellaswag:00008594","item_index":130,"attempt":0,"prompt_hash":"fda434e1fe662d4ad55274cbdbdec806ae254ad98e438dafcf817f6c06801866","question":"[header] How to cook yucca [title] Trim the yucca flowers. [step] Take 1 full cluster of yucca flowers from the top of a yucca plant and cut the stems off of each flower. Use scissors to snip away the green base from each flower.\n\nA. Place the yucca flowers on a cutting board and cut a strip from each flower. [substeps] The strips should be about 1 inch (2.5 cm) in diameter and should be cut so that they can fit on your cutting board.\nB. Make sure you cut each flower off of the root end so there are none left behind. [substeps] It can be daunting to cut the flowers if they still have their roots.\nC. Pull the long central threads (the pistils and stamens) out and discard them. [title] Rinse and soak the flowers for 10 minutes.\nD. Remove any blooms or leaves from each flower, as well as any roots. [substeps] Be careful when trimming the flowers in order to still have usable stems and leaves.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to cook yucca [title] Trim the yucca flowers. [step] Take 1 full cluster of yucca flowers from the top of a yucca plant and cut the stems off of each flower. Use scissors to snip away the green base from each flower.\n\nA. Place the yucca flowers on a cutting board and cut a strip from each flower. [substeps] The strips should be about 1 inch (2.5 cm) in diameter and should be cut so that they can fit on your cutting board.\nB. Make sure you cut each flower off of the root end so there are none left behind. [substeps] It can be daunting to cut the flowers if they still have their roots.\nC. Pull the long central threads (the pistils and stamens) out and discard them. [title] Rinse and soak the flowers for 10 minutes.\nD. Remove any blooms or leaves from each flower, as well as any roots. [substeps] Be careful when trimming the flowers in order to still have usable stems and leaves.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1803,"error":null,"output":"C"}
{"question_id":"hellaswag:00009733","item_index":131,"attempt":0,"prompt_hash":"104e96098b925ee7f0f3af04258e7b33a22f02e76b72649897cf3dec4d0bc33f","question":"[header] How to become a football player after 20 [title] Be motivated to play the sport and play it well. [step] Motivation can drive you a long way and keep you focused. If you really want this to happen, you have to focus on one main thing and that is football.\n\nA. Focus on this and don't let the other one get you down. Whatever you be doing, do so with confidence and persistence.\nB. [title] Ensure that you are fit and in shape. [step] Being in good shape and fit are essential for a football player.\nC. Be putting yourself in the shoes of others who are passionate about this sport. [substeps] It can be physically demanding to be a pro football player, a lot of extra work, it's not necessary to give up.\nD. Don't be afraid to make a point and demonstrate motivation to the field and the people playing. [title] Gather or talk with coaches or other team members.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to become a football player after 20 [title] Be motivated to play the sport and play it well. [step] Motivation can drive you a long way and keep you focused. If you really want this to happen, you have to focus on one main thing and that is football.\n\nA. Focus on this and don't let the other one get you down. Whatever you be doing, do so with confidence and persistence.\nB. [title] Ensure that you are fit and in shape. [step] Being in good shape and fit are essential for a football player.\nC. Be putting yourself in the shoes of others who are passionate about this sport. [substeps] It can be physically demanding to be a pro football player, a lot of extra work, it's not necessary to give up.\nD. Don't be afraid to make a point and demonstrate motivation to the field and the people playing. [title] Gather or talk with coaches or other team members.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1740,"error":null,"output":"A"}
{"question_id":"hellaswag:00003809","item_index":132,"attempt":0,"prompt_hash":"51574d47d093a08774643155c131ab7814f81c781f27b1b8765bce8d7067a139","question":"[header] How to make your legs super soft and super sexy [title] Get in the shower. [step] Give yourself a couple of minutes to rinse off and relax before you break out the razor. You want your skin to have a bit of time to warm and soften up, otherwise you'll risk irritating it.\n\nA. [title] Exfoliate your legs with a body scrub. [step] You can easily make your own sugar scrub with a simple mix of olive oil, sugar, and lemon.\nB. Shave slowly and careful not to press against your skin (with slightly larger strokes). [title] Use a bar of toothpaste to lightly brush your legs.\nC. [substeps] Allow your body to cool before jumping in the shower. People are trained to regulate their body temperature since that's what they are doing doing when they shave.\nD. [title] Begin shaving your thighs and the rest of your body. [step] Start with your legs and move down to your calves, the soles of your feet, and between your knees.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make your legs super soft and super sexy [title] Get in the shower. [step] Give yourself a couple of minutes to rinse off and relax before you break out the razor. You want your skin to have a bit of time to warm and soften up, otherwise you'll risk irritating it.\n\nA. [title] Exfoliate your legs with a body scrub. [step] You can easily make your own sugar scrub with a simple mix of olive oil, sugar, and lemon.\nB. Shave slowly and careful not to press against your skin (with slightly larger strokes). [title] Use a bar of toothpaste to lightly brush your legs.\nC. [substeps] Allow your body to cool before jumping in the shower. People are trained to regulate their body temperature since that's what they are doing doing when they shave.\nD. [title] Begin shaving your thighs and the rest of your body. [step] Start with your legs and move down to your calves, the soles of your feet, and between your knees.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1836,"error":null,"output":"A"}
{"question_id":"hellaswag:00003776","item_index":133,"attempt":0,"prompt_hash":"86cd1f5db8905a570b7588313f05142e115c78b783f74a033ed1bd207d9583a7","question":"[header] How to remove a book from a bookshelf on goodreads [title] Open up and log in to goodreads. [title] Go to \" my books \". [title] Go to the shelf that the book is on.\n\nA. [title] Press the \" help \" button and then select \" read only. [step] \" [title] Now go to the shelf that the book belongs to.\nB. [title] Decide if you want to remove the entire book from this shelf or just browse the page. [step] If you want to remove the entire book from the shelf, click the \" remove all \" button beside it.\nC. [step] If you don't know what shelf it's on, then use the search option in the upper right corner underneath your profile picture. [title] Locate the book on the shelf and click its x at the far right of the item.\nD. [title] Make a note that we are removing the book; make one, and go to \" my books \". [step] Scroll down until it says \" it looks like there's a book \".","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to remove a book from a bookshelf on goodreads [title] Open up and log in to goodreads. [title] Go to \" my books \". [title] Go to the shelf that the book is on.\n\nA. [title] Press the \" help \" button and then select \" read only. [step] \" [title] Now go to the shelf that the book belongs to.\nB. [title] Decide if you want to remove the entire book from this shelf or just browse the page. [step] If you want to remove the entire book from the shelf, click the \" remove all \" button beside it.\nC. [step] If you don't know what shelf it's on, then use the search option in the upper right corner underneath your profile picture. [title] Locate the book on the shelf and click its x at the far right of the item.\nD. [title] Make a note that we are removing the book; make one, and go to \" my books \". [step] Scroll down until it says \" it looks like there's a book \".","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1888,"error":null,"output":"C"}
{"question_id":"hellaswag:00007375","item_index":134,"attempt":0,"prompt_hash":"1864a3f9bedc086943f931131ab8606fd78927c4bc87e58dff0590e552c1ddc1","question":"[header] How to make a mini bakery (kids ) [title] Think of a name for your bakery, and create a simple banner for it. [step] Be sure it gives people an idea of the pastries being sold. A short, unique name, such as, will attract attention.\n\nA. Avoid using an overly popular or long name, as this will not capture interest. [substeps] For example, if you're selling vanilla pastries, a name along the lines of vanilla delight would suffice.\nB. You can place it in front of a farm if you plan to sell it as a bakery. For as the name becomes popular, the banner might pull customers to you.\nC. [title] Pick out the bakery's website to advertise. [step] It's important to get it out in the public's light.\nD. [title] Add cardboard to the front of the bakery. [step] Making a long banner with a business name and number at the top will take attention away from the bakery.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make a mini bakery (kids ) [title] Think of a name for your bakery, and create a simple banner for it. [step] Be sure it gives people an idea of the pastries being sold. A short, unique name, such as, will attract attention.\n\nA. Avoid using an overly popular or long name, as this will not capture interest. [substeps] For example, if you're selling vanilla pastries, a name along the lines of vanilla delight would suffice.\nB. You can place it in front of a farm if you plan to sell it as a bakery. For as the name becomes popular, the banner might pull customers to you.\nC. [title] Pick out the bakery's website to advertise. [step] It's important to get it out in the public's light.\nD. [title] Add cardboard to the front of the bakery. [step] Making a long banner with a business name and number at the top will take attention away from the bakery.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1794,"error":null,"output":"A"}
{"question_id":"hellaswag:00003853","item_index":135,"attempt":0,"prompt_hash":"01b6893f5fc512f351f5ab9f38a9b22f2580c3e1da4a561edcb1537010a36e32","question":"[header] How to train shihpoos [title] Start crate training early. [step] Since shihpoos can be stubborn, you should start training your shihpoo as soon as you bring him home. If you wait too long, his stubbornness could add a frustrating challenge to your training.\n\nA. Start training then at mid-teens or early twenties. [substeps] Note, however, that you should not stay alone in a crate for long period of time, as it will cause too much stress on your dog's muscles and bones.\nB. [substeps] The crate is not dog training and is not a place to socialize your shihpoo. The shihpoo does not like to be handled, so try making a routine out of your crate and expecting it to change some.\nC. [substeps] If you crate train your shihpoo, he may have the notion that he cannot get out of the crate. He may be very excited to get out and explore his new home.\nD. [substeps] Crate training will help keep your shihpoo out of trouble when you are not home. It will also increase the likelihood of him going to the bathroom outside, rather than in his crate or elsewhere in your home.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to train shihpoos [title] Start crate training early. [step] Since shihpoos can be stubborn, you should start training your shihpoo as soon as you bring him home. If you wait too long, his stubbornness could add a frustrating challenge to your training.\n\nA. Start training then at mid-teens or early twenties. [substeps] Note, however, that you should not stay alone in a crate for long period of time, as it will cause too much stress on your dog's muscles and bones.\nB. [substeps] The crate is not dog training and is not a place to socialize your shihpoo. The shihpoo does not like to be handled, so try making a routine out of your crate and expecting it to change some.\nC. [substeps] If you crate train your shihpoo, he may have the notion that he cannot get out of the crate. He may be very excited to get out and explore his new home.\nD. [substeps] Crate training will help keep your shihpoo out of trouble when you are not home. It will also increase the likelihood of him going to the bathroom outside, rather than in his crate or elsewhere in your home.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1940,"error":null,"output":"D"}
{"question_id":"hellaswag:00003868","item_index":136,"attempt":0,"prompt_hash":"316281ac292f1a607aa1fdce767cf9b066160b65f7b93be3b9e484762c39b81c","question":"[header] How to unjam a vcr [title] Check to make sure it's plugged in. [step] If its still jammed, go to step 2 [title] Unplug the vcr. [title] Unscrew any screw that is holding the top on the unit.\n\nA. [title] Take the cover off [title] Stop and look at what you see. [title] Check to see if anything is inside that shouldn't be inside.\nB. [step] This is called a ductwork screw. Pop it out of the way and pull the top back.\nC. [step] Squeeze hard on the screw to release it. [title] Scoop up the film to find it.\nD. [step] You want to slide it off and set it aside if it can't be detached from the unit. [substeps] Lift or twist the unit away from the plug.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to unjam a vcr [title] Check to make sure it's plugged in. [step] If its still jammed, go to step 2 [title] Unplug the vcr. [title] Unscrew any screw that is holding the top on the unit.\n\nA. [title] Take the cover off [title] Stop and look at what you see. [title] Check to see if anything is inside that shouldn't be inside.\nB. [step] This is called a ductwork screw. Pop it out of the way and pull the top back.\nC. [step] Squeeze hard on the screw to release it. [title] Scoop up the film to find it.\nD. [step] You want to slide it off and set it aside if it can't be detached from the unit. [substeps] Lift or twist the unit away from the plug.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1695,"error":null,"output":"B."}
{"question_id":"hellaswag:00004452","item_index":137,"attempt":0,"prompt_hash":"2ab00f2aec1f4a3963490dd576f925d61bc8e2ed2215b4f76a27cd8b69d03a5c","question":"[header] How to rip jeggings [title] Put your jeggings on. [step] It's hard to estimate exactly where distressing lines will fall if you're not wearing the jeans. Prior to distressing, slip the jeans on so you can pinpoint the spots where you want your jeans to rip.\n\nA. [substeps] Get into the habit of searching around for these lines whenever you're ready to rip. Ask a coworker to give you a hand.\nB. [title] Mark the areas you want to distress with pen or chalk. [step] While you're wearing the jeggings, take a piece of chalk or a pen.\nC. [substeps] With jeggings on, you will have to remove them one at a time. You will have to pull them off one by one.\nD. Help your jeggings get wet in order to wash them efficiently. [substeps] Rub some food dye on your jeans before you begin distressing them.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to rip jeggings [title] Put your jeggings on. [step] It's hard to estimate exactly where distressing lines will fall if you're not wearing the jeans. Prior to distressing, slip the jeans on so you can pinpoint the spots where you want your jeans to rip.\n\nA. [substeps] Get into the habit of searching around for these lines whenever you're ready to rip. Ask a coworker to give you a hand.\nB. [title] Mark the areas you want to distress with pen or chalk. [step] While you're wearing the jeggings, take a piece of chalk or a pen.\nC. [substeps] With jeggings on, you will have to remove them one at a time. You will have to pull them off one by one.\nD. Help your jeggings get wet in order to wash them efficiently. [substeps] Rub some food dye on your jeans before you begin distressing them.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1742,"error":null,"output":"B"}
{"question_id":"hellaswag:00008320","item_index":138,"attempt":0,"prompt_hash":"7cbbd0585c1f6f101bd48fabf96a94d65d383fe2f055ca39423872f5da11d8ce","question":"[header] How to recognize a fake ralph lauren [title] Check the logo. [step] Ralph lauren shirts often contain a small stitched on graphic of a polo player riding a horse. On a fake ralph lauren, the mallet the rider holds will be more circular than rectangular.\n\nA. Look closely at the logo to make sure that it is authentic. [substeps] For example, fake ralph lauren t-shirts from dr.\nB. The tail will be difficult to see and the horse's right back leg will be bent. On a genuine product, the horse's tail is clearly visible and the right back leg is straight.\nC. [substeps] Purchase a real ralph lauren print collar from a thrift or sporting goods store. This will give you the best possible appearance, but also make sure that it isn't too thick.\nD. Read and check the logo to make sure it is a replica collar. [title] Determine if the polo is a bonnet.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to recognize a fake ralph lauren [title] Check the logo. [step] Ralph lauren shirts often contain a small stitched on graphic of a polo player riding a horse. On a fake ralph lauren, the mallet the rider holds will be more circular than rectangular.\n\nA. Look closely at the logo to make sure that it is authentic. [substeps] For example, fake ralph lauren t-shirts from dr.\nB. The tail will be difficult to see and the horse's right back leg will be bent. On a genuine product, the horse's tail is clearly visible and the right back leg is straight.\nC. [substeps] Purchase a real ralph lauren print collar from a thrift or sporting goods store. This will give you the best possible appearance, but also make sure that it isn't too thick.\nD. Read and check the logo to make sure it is a replica collar. [title] Determine if the polo is a bonnet.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1773,"error":null,"output":"A"}
{"question_id":"hellaswag:00001622","item_index":139,"attempt":0,"prompt_hash":"bf8350e4cb4610662929499b844a0f9ec5f93ddab67bf527355584b2d578c09b","question":"A herb garden appears with a woman standing next to it in a large garden next to a wheelbarrow filled with mulch. the woman\n\nA. moves the mulch across the ground in the wheelbarrow, falling backwards on attempts.\nB. then begins to mulch close to the wheelbarrow with mulching tool in her hand and while waving her arms in the air.\nC. begins to talk to the camera while gesturing to the flowerbed and the mulch, before eventually picking up a handful of the mulch.\nD. takes some of the mulch away and starts bagging it in the wheelbarrow.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA herb garden appears with a woman standing next to it in a large garden next to a wheelbarrow filled with mulch. the woman\n\nA. moves the mulch across the ground in the wheelbarrow, falling backwards on attempts.\nB. then begins to mulch close to the wheelbarrow with mulching tool in her hand and while waving her arms in the air.\nC. begins to talk to the camera while gesturing to the flowerbed and the mulch, before eventually picking up a handful of the mulch.\nD. takes some of the mulch away and starts bagging it in the wheelbarrow.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1201,"error":null,"output":"B"}
{"question_id":"hellaswag:00008086","item_index":140,"attempt":0,"prompt_hash":"c34000fb68da90591bc36496e7dbca9f16dcf5264fcc08b1aa588065efce59b4","question":"[header] How to buy a house with friends [title] Determine what the agreement will cover. [step] A partnership agreement is a legal contract entered into when a group of individuals go in together on a joint venture. In this situation, the joint venture is buying a home.\n\nA. The partnership agreement will dictate how you and your friends will be required to interact when it comes to purchasing and living in the home. While you and your friends can agree to almost anything, if you fail to address certain issues, state laws (usually called uniform partnership acts) will fill in the blanks for you.\nB. [substeps] A joint venture allows a large portion of the company's remaining revenue to be entered into as the term of the partnership. A joint venture is usually formed when everyone contributes to the plan, with a \" positive \" outcome at the end and usually lasts several months after the joint venture expires.\nC. [substeps] Before making a partner's agreement, you should analyze the potential risks involved in buying the property. A limited liability company may only buy a share of the property and a sole proprietorship, the two companies can only purchase the property when they are certain it is in the best interest of the joint venture to make use of the full share.\nD. A partnership agreement will provide the joint venture with terms like budgetary and money-management, as well as an agreement to allow the parties to lease the house to others. In addition, a partnership agreement requires the parties to be friends of one another.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to buy a house with friends [title] Determine what the agreement will cover. [step] A partnership agreement is a legal contract entered into when a group of individuals go in together on a joint venture. In this situation, the joint venture is buying a home.\n\nA. The partnership agreement will dictate how you and your friends will be required to interact when it comes to purchasing and living in the home. While you and your friends can agree to almost anything, if you fail to address certain issues, state laws (usually called uniform partnership acts) will fill in the blanks for you.\nB. [substeps] A joint venture allows a large portion of the company's remaining revenue to be entered into as the term of the partnership. A joint venture is usually formed when everyone contributes to the plan, with a \" positive \" outcome at the end and usually lasts several months after the joint venture expires.\nC. [substeps] Before making a partner's agreement, you should analyze the potential risks involved in buying the property. A limited liability company may only buy a share of the property and a sole proprietorship, the two companies can only purchase the property when they are certain it is in the best interest of the joint venture to make use of the full share.\nD. A partnership agreement will provide the joint venture with terms like budgetary and money-management, as well as an agreement to allow the parties to lease the house to others. In addition, a partnership agreement requires the parties to be friends of one another.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":2414,"error":null,"output":"A"}
{"question_id":"hellaswag:00008650","item_index":141,"attempt":0,"prompt_hash":"27f8d0193b9d6d06a6443894c29f29f27795070b2532e7501861b4dcb08ab2e7","question":"[header] How to make hot sauce [title] Choose your peppers. [step] Pepper selection is important. This will determine how mild or hot your sauce will be.\n\nA. [title] Write down the recommended amount of pepper for each serving. [step] Use a pencil to write a proper number.\nB. Here is a small list of peppers and how they taste : [substeps] Jalapeños: these peppers are ideal for making mild sauces. They have a strong flavor with less kick than other peppers.\nC. Best known peppers include jalapeño, tarragon, and argon. The peppers that cook best in your sauce include : [substeps] Swiss chard beets [title] Dig in.\nD. Some peppers would be sour, some savory, and some sweet, depending on the difficulty of the recipe. [substeps] If possible, pick only somewhat sweet peppers.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make hot sauce [title] Choose your peppers. [step] Pepper selection is important. This will determine how mild or hot your sauce will be.\n\nA. [title] Write down the recommended amount of pepper for each serving. [step] Use a pencil to write a proper number.\nB. Here is a small list of peppers and how they taste : [substeps] Jalapeños: these peppers are ideal for making mild sauces. They have a strong flavor with less kick than other peppers.\nC. Best known peppers include jalapeño, tarragon, and argon. The peppers that cook best in your sauce include : [substeps] Swiss chard beets [title] Dig in.\nD. Some peppers would be sour, some savory, and some sweet, depending on the difficulty of the recipe. [substeps] If possible, pick only somewhat sweet peppers.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1667,"error":null,"output":"A"}
{"question_id":"hellaswag:00000511","item_index":142,"attempt":0,"prompt_hash":"a0e90a23f0a4fba95fa6d84d8718a207d942f6dbabc6a0461c5431acf85dae91","question":"As he begins, a small rectangle appears in the upper right corner of the video and shows the strings as well as the note he is playing. the guy\n\nA. takes a pause and then begins playing a song and the lyrics to the song appears at the bottom as well as the notes above them.\nB. stops playing and begins drumming on the car while looking back at the camera.\nC. ends the video and starts playing drums in the room until the video ends.\nD. seems to have more than one set of notes after the move and is very worked up about his performance.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nAs he begins, a small rectangle appears in the upper right corner of the video and shows the strings as well as the note he is playing. the guy\n\nA. takes a pause and then begins playing a song and the lyrics to the song appears at the bottom as well as the notes above them.\nB. stops playing and begins drumming on the car while looking back at the camera.\nC. ends the video and starts playing drums in the room until the video ends.\nD. seems to have more than one set of notes after the move and is very worked up about his performance.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1110,"error":null,"output":"A"}
{"question_id":"hellaswag:00008188","item_index":143,"attempt":0,"prompt_hash":"87e0cc4a81780ded19c897694c758c612ebfd7084903c02ec2cccd24a8379711","question":"[header] How to plan your honeymoon in bora bora [title] Plan a beach trip. [step] If you're honeymooning in bora bora, you'll want to pencil in lots of time at the beach. The most popular beach in bora bora is matira beach, where the water is crystal blue and the boardwalk is full of eateries, resorts, and souvenir shops.\n\nA. Stop by a boardwalk and find a beach friendly bar or restaurant where you can lounge. [substeps] Choose as many chairs as you can afford to lounge in.\nB. However, many wedding destinations don't have beaches in bora bora, so you're more likely to get a hawaiian vacationer near you. As with many tropical destinations, location determines what you want.\nC. Look into lesser-known beaches if you want to avoid traffic. [substeps] If you or your partner love aquatic life, plan a trip to bora bora's lagoonarium, a natural outdoor aquarium.\nD. However, there are also various beaches around bora regional and most of them are landscape-friendly and low-maintenance. Only plan a beach trip if you're already in bora bora.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to plan your honeymoon in bora bora [title] Plan a beach trip. [step] If you're honeymooning in bora bora, you'll want to pencil in lots of time at the beach. The most popular beach in bora bora is matira beach, where the water is crystal blue and the boardwalk is full of eateries, resorts, and souvenir shops.\n\nA. Stop by a boardwalk and find a beach friendly bar or restaurant where you can lounge. [substeps] Choose as many chairs as you can afford to lounge in.\nB. However, many wedding destinations don't have beaches in bora bora, so you're more likely to get a hawaiian vacationer near you. As with many tropical destinations, location determines what you want.\nC. Look into lesser-known beaches if you want to avoid traffic. [substeps] If you or your partner love aquatic life, plan a trip to bora bora's lagoonarium, a natural outdoor aquarium.\nD. However, there are also various beaches around bora regional and most of them are landscape-friendly and low-maintenance. Only plan a beach trip if you're already in bora bora.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":2050,"error":null,"output":"A"}
{"question_id":"hellaswag:00000943","item_index":144,"attempt":0,"prompt_hash":"49ddb938f7182f6b422098f61405602193364931234583a95350b6eedb7df5a5","question":"A person is seen wiping off the face of a baby with a rag while the baby sticks it's tongue out. the woman\n\nA. continues plastering around the babies face and the baby sticks his tongue out.\nB. continues cleaning the baby and is shown smiling to the camera and waving.\nC. continues wiping the baby and letting him out with her tongue.\nD. then shaves a face while the child still shows the sister wiping it's face and ends by looking around the camera.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA person is seen wiping off the face of a baby with a rag while the baby sticks it's tongue out. the woman\n\nA. continues plastering around the babies face and the baby sticks his tongue out.\nB. continues cleaning the baby and is shown smiling to the camera and waving.\nC. continues wiping the baby and letting him out with her tongue.\nD. then shaves a face while the child still shows the sister wiping it's face and ends by looking around the camera.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1028,"error":null,"output":"B"}
{"question_id":"hellaswag:00003370","item_index":145,"attempt":0,"prompt_hash":"274d76a8fb8391affed93df77e14faccf42895da0476def82f2cd3faa7597894","question":"[header] How to remove hard water stains from glass [title] Use an acidic cleaner for best results. [step] Combine salt and water and use the mixture to remove the mineral buildup. The salt acts as a scouring powder and can cut through the stain.\n\nA. Apply the mixture to the area and use a clean cloth to rub the stain in a circular motion. Rinse the glass thoroughly when finished to remove all salt water residue.\nB. [title] Dab the stain with a paper towel to remove ink, but do not rub the stain. [step] The acidic solution works well on hard water stains due to it soaking into the fabric and removing the ink from the glass.\nC. If you notice persistent deposits after use, you may want to try an acidic cleaner to get rid of them. A natural acidic cleaner will be less likely to damage glass and may be less expensive.\nD. While you're cleaning, rub or file the stained area with a soft touch and rinse with water. [substeps] Any acidic cleaner should be fine regardless of whether you're using salicylic acid, cetaphil or natural household bleach.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to remove hard water stains from glass [title] Use an acidic cleaner for best results. [step] Combine salt and water and use the mixture to remove the mineral buildup. The salt acts as a scouring powder and can cut through the stain.\n\nA. Apply the mixture to the area and use a clean cloth to rub the stain in a circular motion. Rinse the glass thoroughly when finished to remove all salt water residue.\nB. [title] Dab the stain with a paper towel to remove ink, but do not rub the stain. [step] The acidic solution works well on hard water stains due to it soaking into the fabric and removing the ink from the glass.\nC. If you notice persistent deposits after use, you may want to try an acidic cleaner to get rid of them. A natural acidic cleaner will be less likely to damage glass and may be less expensive.\nD. While you're cleaning, rub or file the stained area with a soft touch and rinse with water. [substeps] Any acidic cleaner should be fine regardless of whether you're using salicylic acid, cetaphil or natural household bleach.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1882,"error":null,"output":"A"}
{"question_id":"hellaswag:00002676","item_index":146,"attempt":0,"prompt_hash":"421a0858e49999250f0ceda27f413114525298cf65aa1008b94890bbb7c88ae2","question":"She continues talking to the camera followed by her playing the violin. She takes her hair down and continues talking to the camera and showing various violin moves while singing. she\n\nA. makes secret routines before walking away.\nB. stopping to play a couple of notes and ending by bowing to the camera.\nC. is crying again, and we can see her holding her hands up.\nD. once again puts her hair up and continues with her playing.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nShe continues talking to the camera followed by her playing the violin. She takes her hair down and continues talking to the camera and showing various violin moves while singing. she\n\nA. makes secret routines before walking away.\nB. stopping to play a couple of notes and ending by bowing to the camera.\nC. is crying again, and we can see her holding her hands up.\nD. once again puts her hair up and continues with her playing.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1487,"error":null,"output":"B. stopping to play a couple of notes and ending by bowing to the camera."}
{"question_id":"hellaswag:00009571","item_index":147,"attempt":0,"prompt_hash":"db19bf810c38f976c761912cb4e3900e031d45caa6e7f8f705f525a9e9bfc94b","question":"[header] How to use the internet to promote your business [title] Create a website. [step] A visible online business presence requires having a website that users can visit for information. Luckily, you don't have to be an experienced web developer to create a website for your business.\n\nA. You just need to use a fairly high profile name and an attractive ad to give people an idea of who you are. [substeps] You can create a website to build both an external and internal presence.\nB. On your website, simply create the following : [substeps] Profile. The key points of your website here are the basic rules regarding the search algorithms and operating systems that make your website successful in a number of ways.\nC. [substeps] Many services (such as wordpress and wix) make it easy for you to set up a business website using simple, guided steps. Google has partnered with local business services through the get your business online (gybo) program to help businesses create websites and promote themselves online in other ways.\nD. A good website design will require you to control the search results and decide which people to leave out, like little pink elephants. You should also have an informative website that describes the services you offer, the first and last steps you plan on exploring, and your business philosophy, as well as its plans and goals.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to use the internet to promote your business [title] Create a website. [step] A visible online business presence requires having a website that users can visit for information. Luckily, you don't have to be an experienced web developer to create a website for your business.\n\nA. You just need to use a fairly high profile name and an attractive ad to give people an idea of who you are. [substeps] You can create a website to build both an external and internal presence.\nB. On your website, simply create the following : [substeps] Profile. The key points of your website here are the basic rules regarding the search algorithms and operating systems that make your website successful in a number of ways.\nC. [substeps] Many services (such as wordpress and wix) make it easy for you to set up a business website using simple, guided steps. Google has partnered with local business services through the get your business online (gybo) program to help businesses create websites and promote themselves online in other ways.\nD. A good website design will require you to control the search results and decide which people to leave out, like little pink elephants. You should also have an informative website that describes the services you offer, the first and last steps you plan on exploring, and your business philosophy, as well as its plans and goals.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":2122,"error":null,"output":"C"}
{"question_id":"hellaswag:00008172","item_index":148,"attempt":0,"prompt_hash":"67d71655b33e8b3c59ab9237ba7b24d9786c5a9ad10becaf9dc5febf5bcc5a71","question":"[header] How to wear a maxi dress [title] Recognize your body shape. [step] Every woman's body is different and the fit of clothing can enhance your shape. Knowing parts of your body you'd like to highlight can help you find the most flattering maxi dress for your shape.\n\nA. [substeps] For example, if you are a 6-foot-4 petite figure, you might want a maxi dress that slihugs your body but pushes more stretch and fat while retaining greater volume. Or, if you are petite and tall, you might want a structured maxi dress that sags a little less behind your hips.\nB. [substeps] Figure out your body shape, and allow your body and its structure to be reflected in a dress. You can look in magazines to find pages focusing on your body type or try online resources like this: http: // www.tamenta.com/looking _ curvy.\nC. Keep in mind you may be petite if you are under 5'4 \" or plus size if you wear larger than a size 16. Research different styles to decide on what you like and might look great on you.\nD. The key is looking into the photographer's face when you choose your dress. Knowing your body shape can help you accentuate the correct parts of your body to balance out your look.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to wear a maxi dress [title] Recognize your body shape. [step] Every woman's body is different and the fit of clothing can enhance your shape. Knowing parts of your body you'd like to highlight can help you find the most flattering maxi dress for your shape.\n\nA. [substeps] For example, if you are a 6-foot-4 petite figure, you might want a maxi dress that slihugs your body but pushes more stretch and fat while retaining greater volume. Or, if you are petite and tall, you might want a structured maxi dress that sags a little less behind your hips.\nB. [substeps] Figure out your body shape, and allow your body and its structure to be reflected in a dress. You can look in magazines to find pages focusing on your body type or try online resources like this: http: // www.tamenta.com/looking _ curvy.\nC. Keep in mind you may be petite if you are under 5'4 \" or plus size if you wear larger than a size 16. Research different styles to decide on what you like and might look great on you.\nD. The key is looking into the photographer's face when you choose your dress. Knowing your body shape can help you accentuate the correct parts of your body to balance out your look.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":2126,"error":null,"output":"A"}
{"question_id":"hellaswag:00007893","item_index":149,"attempt":0,"prompt_hash":"51fd0f8ed7486fa6bdc39cc4fa378529926de19e2582fa49e5734752192da3b5","question":"[header] How to get rid of oil on your face [title] Wash your face with a mild cleanser. [step] Try dove, neutrogena, cetaphil, cerave or basics, as these products will not cause dryness. Wash your face one to two times per day.\n\nA. [title] Use your hand to pat dry your face. [step] Start off with this if you have oily skin, just to avoid the formation of blemishes.\nB. If oil persists, use an over-the-counter medicated acne wash that contains benzoyl peroxide, sulfur, resorcinol or salicylic acid. [substeps] Use warm water to wet your face.\nC. Use a non-comedogenic cleanser to ensure it is safe to use on your face. A liquid cleanser such as dove will not keep your face clean and create oils on your face.\nD. [substeps] Do this before bed, as the oil will sit on your skin more easily than a cleanser. Also make sure you moisturize your skin afterwards.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to get rid of oil on your face [title] Wash your face with a mild cleanser. [step] Try dove, neutrogena, cetaphil, cerave or basics, as these products will not cause dryness. Wash your face one to two times per day.\n\nA. [title] Use your hand to pat dry your face. [step] Start off with this if you have oily skin, just to avoid the formation of blemishes.\nB. If oil persists, use an over-the-counter medicated acne wash that contains benzoyl peroxide, sulfur, resorcinol or salicylic acid. [substeps] Use warm water to wet your face.\nC. Use a non-comedogenic cleanser to ensure it is safe to use on your face. A liquid cleanser such as dove will not keep your face clean and create oils on your face.\nD. [substeps] Do this before bed, as the oil will sit on your skin more easily than a cleanser. Also make sure you moisturize your skin afterwards.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1842,"error":null,"output":"C"}
{"question_id":"hellaswag:00001455","item_index":150,"attempt":0,"prompt_hash":"607237586c63cc13f686eaa7ed813720c4feeac2453021758d0ca7e5bb177050","question":"The videos transitions into a countdown of various people getting hurt on the streets from bull attacks. many different people\n\nA. are shown injured individuals and circles around them in the streets while the shots of them getting hurt are shown.\nB. get hurt, but the video returns with only two men injured.\nC. get hurt by bull attacks but each one impacts a few bystanders.\nD. are seen running down streets while a bull flips them over and hurt them.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nThe videos transitions into a countdown of various people getting hurt on the streets from bull attacks. many different people\n\nA. are shown injured individuals and circles around them in the streets while the shots of them getting hurt are shown.\nB. get hurt, but the video returns with only two men injured.\nC. get hurt by bull attacks but each one impacts a few bystanders.\nD. are seen running down streets while a bull flips them over and hurt them.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":988,"error":null,"output":"A"}
{"question_id":"hellaswag:00000702","item_index":151,"attempt":0,"prompt_hash":"eb93249afbb3bccb16e82dba368a7eea3c4a3e3423aef0caee8d56e98610046e","question":"When the rabbit stops licking its right side it briefly faces forward and its left back foot scratches its left ear then he licks his left paw. The rabbit then lifts its right paw and scratches his right ear and then licks that paw. the rabbit\n\nA. is now standing on its front feet licking its right paw.\nB. then begins to take its front paws and starts rubbing it's face and ears and licking its front paws over and over.\nC. then proceeds to get up off the floor to get into a shower.\nD. finishes licking its right paw and starts brushing its claws dripping with cream.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nWhen the rabbit stops licking its right side it briefly faces forward and its left back foot scratches its left ear then he licks his left paw. The rabbit then lifts its right paw and scratches his right ear and then licks that paw. the rabbit\n\nA. is now standing on its front feet licking its right paw.\nB. then begins to take its front paws and starts rubbing it's face and ears and licking its front paws over and over.\nC. then proceeds to get up off the floor to get into a shower.\nD. finishes licking its right paw and starts brushing its claws dripping with cream.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":2076,"error":null,"output":"B. then begins to take its front paaws and starts rubbing its face and ears and licking its front paaws over and over."}
{"question_id":"hellaswag:00006704","item_index":152,"attempt":0,"prompt_hash":"bfe7869fabc8e44d347357558fd0b03c9dee06a9dc2de8d9930aeb891ae50c13","question":"[header] How to dye hair blue [title] Start with a clarifying shampoo. [step] Using a clarifying shampoo can help to remove buildup from your hair and make it easier for you to dye it. It can also help to remove leftover dye from the last time you dyed your hair.\n\nA. Look for clarifying shampoo that contains aluminum sulfates and which are made especially for hair that isn't dyed. Remember that green hair dye is very dark blue, so it's really best to use a shampoo made from other green products.\nB. [substeps] Clarifying shampoos are also generally safe for use as their names imply. [title] Rinse your hair with cool water.\nC. [substeps] Look for clarifying shampoos at beauty supply stores like lowe's and beauty sugar's. They can generally be found in various colors.\nD. You can find a clarifying shampoo in beauty supply store and some drug stores. [substeps] Follow the instructions on the package for the clarifying shampoo.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to dye hair blue [title] Start with a clarifying shampoo. [step] Using a clarifying shampoo can help to remove buildup from your hair and make it easier for you to dye it. It can also help to remove leftover dye from the last time you dyed your hair.\n\nA. Look for clarifying shampoo that contains aluminum sulfates and which are made especially for hair that isn't dyed. Remember that green hair dye is very dark blue, so it's really best to use a shampoo made from other green products.\nB. [substeps] Clarifying shampoos are also generally safe for use as their names imply. [title] Rinse your hair with cool water.\nC. [substeps] Look for clarifying shampoos at beauty supply stores like lowe's and beauty sugar's. They can generally be found in various colors.\nD. You can find a clarifying shampoo in beauty supply store and some drug stores. [substeps] Follow the instructions on the package for the clarifying shampoo.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1869,"error":null,"output":"C"}
{"question_id":"hellaswag:00001137","item_index":153,"attempt":0,"prompt_hash":"fb45242b99023cdd4bdab9ed9141292db7d771237beafc7a37c23e491c9ced3c","question":"A man is standing holding a knife over a table. the man\n\nA. proceeds to place wax on a piece of paper.\nB. demonstrates other techniques for wounding a patient.\nC. points to several parts of the knife.\nD. then takes a sharpener and cuts two slices of lemon.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA man is standing holding a knife over a table. the man\n\nA. proceeds to place wax on a piece of paper.\nB. demonstrates other techniques for wounding a patient.\nC. points to several parts of the knife.\nD. then takes a sharpener and cuts two slices of lemon.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1076,"error":null,"output":"C. points to several parts of the knife."}
{"question_id":"hellaswag:00007563","item_index":154,"attempt":0,"prompt_hash":"8ee2eede8e14eec2dcd9e5c51c881e7512755f224e637503d457f71bd94bb747","question":"[header] How to create an e map file with e map editor [title] From the main screen of the e-map editor click on the \" add map \" button to create a new map [title] Click on the \" rename \" button to change the file name of the map. [title] Type a new name to identify the map. [step] For example \" office \".\n\nA. You will see a blue \" new \" button. [title] Select the location where you want to place a folder and click \" save \" to save the map to the map.\nB. [title] Click on the map \" office \" to import a floor plan in bmp, gif or jpg format. [title] Click on the \" load map \" button.\nC. Click on \" extensions \" and then choose the name you want to use from the drop down menu in the drop down menu on the bottom. [title] Right click on the map you want to create and select \" design map.\nD. If there are plans you'd like to change, then type \" echo \" to add a new name to the existing map [title] Open the map editor. [title] Go to the archive path and make sure you aren't over/over-named ;.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to create an e map file with e map editor [title] From the main screen of the e-map editor click on the \" add map \" button to create a new map [title] Click on the \" rename \" button to change the file name of the map. [title] Type a new name to identify the map. [step] For example \" office \".\n\nA. You will see a blue \" new \" button. [title] Select the location where you want to place a folder and click \" save \" to save the map to the map.\nB. [title] Click on the map \" office \" to import a floor plan in bmp, gif or jpg format. [title] Click on the \" load map \" button.\nC. Click on \" extensions \" and then choose the name you want to use from the drop down menu in the drop down menu on the bottom. [title] Right click on the map you want to create and select \" design map.\nD. If there are plans you'd like to change, then type \" echo \" to add a new name to the existing map [title] Open the map editor. [title] Go to the archive path and make sure you aren't over/over-named ;.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":2026,"error":null,"output":"A"}
{"question_id":"hellaswag:00006150","item_index":155,"attempt":0,"prompt_hash":"4dee5868106d7e9923410bde28944541ab74aa56a192605ce1b47e8c823bf89e","question":"[header] How to become a laser hair removal technician [title] Know your state requirements for laser hair removal technicians. [step] There are no federal laws governing laser hair removal, so each state has their own regulations. Make sure that you know what those regulations are and find a school which will give you a certification recognized by the state where you intend to work.\n\nA. [substeps] Some states, beginning with arizona, will require a certain number of training hours, around 40 hours of class time. About 24 hours of hands on training could also be required.\nB. [substeps] Any professional laser hair removal clinic or hair delivery service will publish their certification documents on their website. The processing label will tell you what the blue sticker will say on it.\nC. Make sure that you find a school that is accredited by your state's state university. [substeps] If you decide to work on your own, start looking into enrollment at major universities in your area.\nD. [substeps] You must have at least 2 years of experience working with laser hair surgeons. In some states you don't possess additional qualifications, but you can still follow the federal law.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to become a laser hair removal technician [title] Know your state requirements for laser hair removal technicians. [step] There are no federal laws governing laser hair removal, so each state has their own regulations. Make sure that you know what those regulations are and find a school which will give you a certification recognized by the state where you intend to work.\n\nA. [substeps] Some states, beginning with arizona, will require a certain number of training hours, around 40 hours of class time. About 24 hours of hands on training could also be required.\nB. [substeps] Any professional laser hair removal clinic or hair delivery service will publish their certification documents on their website. The processing label will tell you what the blue sticker will say on it.\nC. Make sure that you find a school that is accredited by your state's state university. [substeps] If you decide to work on your own, start looking into enrollment at major universities in your area.\nD. [substeps] You must have at least 2 years of experience working with laser hair surgeons. In some states you don't possess additional qualifications, but you can still follow the federal law.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1965,"error":null,"output":"A"}
{"question_id":"hellaswag:00009767","item_index":156,"attempt":0,"prompt_hash":"161ae4861b2dc4f38ef90cc94ea7bfec23949332e313bf1f308e78ac10d81fb0","question":"[header] How to get good looking hair (milk conditioning ) [title] Decide which products you'd like to use for your conditioner. [step] Milk conditioning can be used with just milk, but you can also use other items in your pantry for a natural hair mask to get beautiful, shiny hair. [substeps] For regular milk conditioning, whole milk works best.\n\nA. You can use lowfat milk for fine hair, and cream for very coarse hair. Put it in a small spray bottle and spray over your hair, waiting 30 minutes before washing it out with shampoo.\nB. It helps the hair stand up, with no lumps, and helps the hair hair bind together better. Some excellent choices are honey, soy milk, and honey oil.\nC. Substitute , 1:2% and okay quality milk has 3 %. Any extra cream, milk substitute, and different oils can work well too.\nD. Whole milk is the nastiest-you can get unlimited (which means one cup per banana and six bottles of milk makes for a total smoothie). It also lowers your cholesterol and leads to less fat.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to get good looking hair (milk conditioning ) [title] Decide which products you'd like to use for your conditioner. [step] Milk conditioning can be used with just milk, but you can also use other items in your pantry for a natural hair mask to get beautiful, shiny hair. [substeps] For regular milk conditioning, whole milk works best.\n\nA. You can use lowfat milk for fine hair, and cream for very coarse hair. Put it in a small spray bottle and spray over your hair, waiting 30 minutes before washing it out with shampoo.\nB. It helps the hair stand up, with no lumps, and helps the hair hair bind together better. Some excellent choices are honey, soy milk, and honey oil.\nC. Substitute , 1:2% and okay quality milk has 3 %. Any extra cream, milk substitute, and different oils can work well too.\nD. Whole milk is the nastiest-you can get unlimited (which means one cup per banana and six bottles of milk makes for a total smoothie). It also lowers your cholesterol and leads to less fat.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1879,"error":null,"output":"C"}
{"question_id":"hellaswag:00006793","item_index":157,"attempt":0,"prompt_hash":"82d27f6626502c60f27f8f5af1c8f6ae74b309aba7767b961f9a523c5e8602cc","question":"[header] How to give yourself a manicure [title] Get your supplies together. [step] To give yourself a gorgeous manicure, make sure you have all the right supplies. You might have to spend a bit up front, but next time you want to paint your nails, you'll already be covered.\n\nA. Go to your local painting store to get the following supplies : [substeps] Remover cotton ball clay/sand to make your nails nice and smooth a cotton ball nail balm wipe alcohol cream nail polish remover [title] Prepare your nails. [step] There are different kinds of nail care products available for wanting a manicure.\nB. Get two different kinds of supplies : [substeps] 3 bags of ink or a small jar (you won't need this all of the time) a nail polish palette [title] Place your water bottle on your bathtub. [step] To get your nails in the habit of being wet, take a small (preferably very small) bottle.\nC. Put your cuticles into the collection (the nail brush) and chop them up. The cuticles are your foundation and cover any crevices and cracks.\nD. Get the following products : [substeps] Nail polish remover cotton balls or cotton swabs cuticle trimmer nail buffer nail trimmers nail file cuticle or hand cream nail polish base coat top coat [title] Set up your workspace. [step] Nail polish and remover can damage many surfaces such as cloth, wood finish and plastic.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to give yourself a manicure [title] Get your supplies together. [step] To give yourself a gorgeous manicure, make sure you have all the right supplies. You might have to spend a bit up front, but next time you want to paint your nails, you'll already be covered.\n\nA. Go to your local painting store to get the following supplies : [substeps] Remover cotton ball clay/sand to make your nails nice and smooth a cotton ball nail balm wipe alcohol cream nail polish remover [title] Prepare your nails. [step] There are different kinds of nail care products available for wanting a manicure.\nB. Get two different kinds of supplies : [substeps] 3 bags of ink or a small jar (you won't need this all of the time) a nail polish palette [title] Place your water bottle on your bathtub. [step] To get your nails in the habit of being wet, take a small (preferably very small) bottle.\nC. Put your cuticles into the collection (the nail brush) and chop them up. The cuticles are your foundation and cover any crevices and cracks.\nD. Get the following products : [substeps] Nail polish remover cotton balls or cotton swabs cuticle trimmer nail buffer nail trimmers nail file cuticle or hand cream nail polish base coat top coat [title] Set up your workspace. [step] Nail polish and remover can damage many surfaces such as cloth, wood finish and plastic.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":2534,"error":null,"output":"B"}
{"question_id":"hellaswag:00005951","item_index":158,"attempt":0,"prompt_hash":"7799beec65e182b6c2f2441ffd1051747c3caec76aa5f63bf0abf56d474d55bb","question":"[header] How to offer help and support to a widowed mother [title] Contact her after the passing of the loved one. [step] In times of crisis people gather to provide support to one another. When someone has experienced a profound loss, it is important to show her she is not alone.\n\nA. Sitting with the person and actively listening to her tell you what happened and acknowledging her feelings can be extremely helpful. You are there to comfort and reassure her, which will include listening, talking and calming the situation in any way.\nB. Even people in financial straits can get support simply by sharing personal thoughts. Do not hesitate to reach out for support and advice.\nC. [substeps] Tell the mother of the loved one who has passed that she loves and supports your relationship with her and is involved in reaching the person's thoughts and setting up a support system in case of an emergency. When you do visit, also provide support if the bereaved person is declining or he or she is feeling really down.\nD. Call the following numbers : [substeps] Call 1-800-273-2159 to speak with someone you know. Reach the national center for older women's survivors.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to offer help and support to a widowed mother [title] Contact her after the passing of the loved one. [step] In times of crisis people gather to provide support to one another. When someone has experienced a profound loss, it is important to show her she is not alone.\n\nA. Sitting with the person and actively listening to her tell you what happened and acknowledging her feelings can be extremely helpful. You are there to comfort and reassure her, which will include listening, talking and calming the situation in any way.\nB. Even people in financial straits can get support simply by sharing personal thoughts. Do not hesitate to reach out for support and advice.\nC. [substeps] Tell the mother of the loved one who has passed that she loves and supports your relationship with her and is involved in reaching the person's thoughts and setting up a support system in case of an emergency. When you do visit, also provide support if the bereaved person is declining or he or she is feeling really down.\nD. Call the following numbers : [substeps] Call 1-800-273-2159 to speak with someone you know. Reach the national center for older women's survivors.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":2036,"error":null,"output":"A"}
{"question_id":"hellaswag:00009671","item_index":159,"attempt":0,"prompt_hash":"497f34f16ee8c556dace5cb5832cfe7fa97ff6dfa89b84850126e3bc0f9351bb","question":"[header] How to clean your longchamp le pliage bag [title] Don't wash it in the washing machine. [step] The nylon material of the le pliage tends to \" bubble \" when handled improperly. Do not put them in the washing machine! They need to be hand-washed, gently.\n\nA. [title] Lay the bag on the clean surface. [step] Wet both sides and the inside of the bag with water.\nB. Dry off the bag thoroughly with a soft dry cloth. [title] If you do notice that the nylon material is touching the strap when holding it in place or you're worried that it might fall off, take it out of the washing machine.\nC. Always hand wash one bag at a time with the exception of the synthetic inner bag, which is waterless. Wash your bag in the washing machine, not on it's surface.\nD. [substeps] Setting the temperature to \" soft cycle \" is more likely to be effective than washing the water or cleaning the bag as usual. [title] Make sure you wash it at the proper temperature.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to clean your longchamp le pliage bag [title] Don't wash it in the washing machine. [step] The nylon material of the le pliage tends to \" bubble \" when handled improperly. Do not put them in the washing machine! They need to be hand-washed, gently.\n\nA. [title] Lay the bag on the clean surface. [step] Wet both sides and the inside of the bag with water.\nB. Dry off the bag thoroughly with a soft dry cloth. [title] If you do notice that the nylon material is touching the strap when holding it in place or you're worried that it might fall off, take it out of the washing machine.\nC. Always hand wash one bag at a time with the exception of the synthetic inner bag, which is waterless. Wash your bag in the washing machine, not on it's surface.\nD. [substeps] Setting the temperature to \" soft cycle \" is more likely to be effective than washing the water or cleaning the bag as usual. [title] Make sure you wash it at the proper temperature.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1870,"error":null,"output":"A"}
{"question_id":"hellaswag:00001775","item_index":160,"attempt":0,"prompt_hash":"57c6d1768a9f5f2057e01d09782a7917d387149b900fdb265c4d6b425f79729d","question":"A male chef appears in a kitchen standing behind an island with several knives on a block of wood with sharpening utensils on it. the man\n\nA. then demonstrates the proper way to sharpen the knives, pierces one of the knives using the pointed knife.\nB. then grabs the knife and starts gliding it over the block very gently and smoothly before hitting it over a silver steel stick.\nC. then begins rubbing a steel plate into a wooden wooden horizontally lay surface and starts to sharpen the knife on it using several knives.\nD. now pours water on the knife and then knives.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA male chef appears in a kitchen standing behind an island with several knives on a block of wood with sharpening utensils on it. the man\n\nA. then demonstrates the proper way to sharpen the knives, pierces one of the knives using the pointed knife.\nB. then grabs the knife and starts gliding it over the block very gently and smoothly before hitting it over a silver steel stick.\nC. then begins rubbing a steel plate into a wooden wooden horizontally lay surface and starts to sharpen the knife on it using several knives.\nD. now pours water on the knife and then knives.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1238,"error":null,"output":"B"}
{"question_id":"hellaswag:00000331","item_index":161,"attempt":0,"prompt_hash":"6028e7f70210d9a81dfa36aa198e318e94409b6f5a85f86f92eabb5d4219e6f2","question":"A girl is getting her nose pierced. the nose\n\nA. and other parts of her face stop.\nB. piercing drill is in her lip and she is getting a piercing.\nC. is cleaned before the piercing.\nD. is connected to a diver causing the piercing to go through her head.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA girl is getting her nose pierced. the nose\n\nA. and other parts of her face stop.\nB. piercing drill is in her lip and she is getting a piercing.\nC. is cleaned before the piercing.\nD. is connected to a diver causing the piercing to go through her head.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":750,"error":null,"output":"C"}
{"question_id":"hellaswag:00000390","item_index":162,"attempt":0,"prompt_hash":"03be3662a85481ffb9348e4d7887d20e5b1cd6a81edc46a12552d064cbbcac04","question":"They are enjoying themselves as they rhythmically play their instruments and dance along to the tune. the player on the extreme right\n\nA. is showing off his abdomen while dancers on both sides of him get distracted.\nB. is playing a longer harmonica while the others are playing a smaller harmonica.\nC. is also enjoying herself dancing with another male and she seems very into what she's doing.\nD. starts to struggle as he tries to get up, but they continue to soothe him with their hand movements.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nThey are enjoying themselves as they rhythmically play their instruments and dance along to the tune. the player on the extreme right\n\nA. is showing off his abdomen while dancers on both sides of him get distracted.\nB. is playing a longer harmonica while the others are playing a smaller harmonica.\nC. is also enjoying herself dancing with another male and she seems very into what she's doing.\nD. starts to struggle as he tries to get up, but they continue to soothe him with their hand movements.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1006,"error":null,"output":"A"}
{"question_id":"hellaswag:00008340","item_index":163,"attempt":0,"prompt_hash":"b92532b8983e3a01e0666ff4b7e9de9868473c9b8f328a12531dd635ff2a6427","question":"[header] How to clean your hair with beer [title] De-carbonate your beer. [step] Pour a bottle or can of beer into a jug or bowl and let it sit overnight or throughout the day. The more air surface your beer is exposed to, the faster it will oxidize and become flat.\n\nA. Rinse it off in the morning, and then use it to clean your hair again. [substeps] Keep in mind that beer often contains chlorine, which should be avoided while dealing with your hair.\nB. If you drink your beer through coffee or tea, avoid standing in the tap during the entire day. Pour it directly into the coffee or tea cooker.\nC. [substeps] Beer becomes flat by releasing the carbon dioxide it creates. It's important that the carbon dioxide is gone from the beer when using it to treat hair as carbon dioxide combined with water can create hard water or water with a high mineral content.\nD. [substeps] The best time to de-carbonate your beer is before noon. This will ensure that the beer is not going to harden before a full day passes without exposure to sunlight.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to clean your hair with beer [title] De-carbonate your beer. [step] Pour a bottle or can of beer into a jug or bowl and let it sit overnight or throughout the day. The more air surface your beer is exposed to, the faster it will oxidize and become flat.\n\nA. Rinse it off in the morning, and then use it to clean your hair again. [substeps] Keep in mind that beer often contains chlorine, which should be avoided while dealing with your hair.\nB. If you drink your beer through coffee or tea, avoid standing in the tap during the entire day. Pour it directly into the coffee or tea cooker.\nC. [substeps] Beer becomes flat by releasing the carbon dioxide it creates. It's important that the carbon dioxide is gone from the beer when using it to treat hair as carbon dioxide combined with water can create hard water or water with a high mineral content.\nD. [substeps] The best time to de-carbonate your beer is before noon. This will ensure that the beer is not going to harden before a full day passes without exposure to sunlight.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1842,"error":null,"output":"A"}
{"question_id":"hellaswag:00007021","item_index":164,"attempt":0,"prompt_hash":"bd7fea85e7df248653cae7b27e092512b0233a554184b62661689b606c9993a9","question":"[header] How to respond after a kiss [title] Enjoy the moment slowly, without worrying about next steps. [step] A good kiss is a moment out of time, so savor it. Don't feel like you must immediately move forward or have something to do next.\n\nA. Let your feelings shine through and let anyone who needs a hug and a kiss next to you know that you're still there for them. These may include your family, friends, co-workers, and coworkers.\nB. When your new partner initiates the kiss, don't interrupt or rush things-just watch. [substeps] You'll have good moments ahead of you if you're having trouble remaining in the moment.\nC. You just have to be present with the moment. You don't need to think about other people, your future, to embrace it.\nD. Just enjoy the kiss! More often than not, the two of you will naturally find something to talk about, do, or just keep on kissing--so let the moment just happen. [substeps] In general, the best advice is just to go slow.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to respond after a kiss [title] Enjoy the moment slowly, without worrying about next steps. [step] A good kiss is a moment out of time, so savor it. Don't feel like you must immediately move forward or have something to do next.\n\nA. Let your feelings shine through and let anyone who needs a hug and a kiss next to you know that you're still there for them. These may include your family, friends, co-workers, and coworkers.\nB. When your new partner initiates the kiss, don't interrupt or rush things-just watch. [substeps] You'll have good moments ahead of you if you're having trouble remaining in the moment.\nC. You just have to be present with the moment. You don't need to think about other people, your future, to embrace it.\nD. Just enjoy the kiss! More often than not, the two of you will naturally find something to talk about, do, or just keep on kissing--so let the moment just happen. [substeps] In general, the best advice is just to go slow.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1825,"error":null,"output":"D"}
{"question_id":"hellaswag:00001825","item_index":165,"attempt":0,"prompt_hash":"131e57cf7ad6dee1557e38f557eb87472e0c4bfdff0805fe7e8ed204f4a3b566","question":"A woman in a black shirt is playing a violin. she\n\nA. stands up and starts dancing.\nB. puts the violin down in her lap.\nC. is pointing at the notes on the violin.\nD. starts to play the violin.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA woman in a black shirt is playing a violin. she\n\nA. stands up and starts dancing.\nB. puts the violin down in her lap.\nC. is pointing at the notes on the violin.\nD. starts to play the violin.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":838,"error":null,"output":"D. starts to play the violin."}
{"question_id":"hellaswag:00007923","item_index":166,"attempt":0,"prompt_hash":"0c75e30b1b6753375e7846207661465d06feb232e9309e29a4e809e61aaa78c5","question":"[header] How to make a watermelon smoothie [title] Cut out enough watermelon to fill 2 cups (304 grams). [step] You can do this by cutting a watermelon into cubes and then removing the rind. You can also cut a watermelon in half, and then scoop out the flesh using a melon scoop or coffee scoop.\n\nA. [title] Put the watermelon into a blender. [step] For a more refreshing smoothie, consider adding 2 cups (400 grams) of strawberries and/or 1 tablespoon of lime juice.\nB. You will need 2-3 watermelon halves (140 milliliters) in buckets or a jug. [title] Stuff the watermelon with the contents.\nC. [substeps] Make sure that the watermelon is fresh. If you don't have access to the sugar, you can use frozen watermelon chunks.\nD. [title] Place the watermelon slices in the refrigerator to chill for at least 2 hours. [step] Next, peel the watermelon and remove the rind.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make a watermelon smoothie [title] Cut out enough watermelon to fill 2 cups (304 grams). [step] You can do this by cutting a watermelon into cubes and then removing the rind. You can also cut a watermelon in half, and then scoop out the flesh using a melon scoop or coffee scoop.\n\nA. [title] Put the watermelon into a blender. [step] For a more refreshing smoothie, consider adding 2 cups (400 grams) of strawberries and/or 1 tablespoon of lime juice.\nB. You will need 2-3 watermelon halves (140 milliliters) in buckets or a jug. [title] Stuff the watermelon with the contents.\nC. [substeps] Make sure that the watermelon is fresh. If you don't have access to the sugar, you can use frozen watermelon chunks.\nD. [title] Place the watermelon slices in the refrigerator to chill for at least 2 hours. [step] Next, peel the watermelon and remove the rind.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1917,"error":null,"output":"A"}
{"question_id":"hellaswag:00003417","item_index":167,"attempt":0,"prompt_hash":"f567970580a5329940a74c90884e6c4e16509a0340cc43aa8fa89c1221844e4e","question":"[header] How to make a hair clip with a scarf attached [title] Assemble the items needed. [step] These are listed below under \" things you'll need \". [substeps] For the scarf, use a smallish long sized one of a very pliable and gauze-like fabric.\n\nA. [title] Open the barrette or hair clip. [title] Lay one end of the scarf so that it overlaps the fastener end (opening end) of the barrette or hair clip.\nB. This piece needs to be about five inches and tall as to be used for braiding and using any elastic or solid size strands. For more information or instructions on using the scarf: check out: how to make a hair clip with a fabric made from cast iron, nylon, or mesh about 3 inches wide.\nC. For the scarf length piece, use a small piece of thin satin tape, fabric scissors, tape or masking tape. For different wigs, different kinds of wigs are used.\nD. For the long piece of cloth, use a medium sized one of a lighter color or a darker color. For the scarf: cut off a piece of the cloth that's approximately as wide as the scarf, made to about 6 1/2 \" by 10 1/2 \".","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make a hair clip with a scarf attached [title] Assemble the items needed. [step] These are listed below under \" things you'll need \". [substeps] For the scarf, use a smallish long sized one of a very pliable and gauze-like fabric.\n\nA. [title] Open the barrette or hair clip. [title] Lay one end of the scarf so that it overlaps the fastener end (opening end) of the barrette or hair clip.\nB. This piece needs to be about five inches and tall as to be used for braiding and using any elastic or solid size strands. For more information or instructions on using the scarf: check out: how to make a hair clip with a fabric made from cast iron, nylon, or mesh about 3 inches wide.\nC. For the scarf length piece, use a small piece of thin satin tape, fabric scissors, tape or masking tape. For different wigs, different kinds of wigs are used.\nD. For the long piece of cloth, use a medium sized one of a lighter color or a darker color. For the scarf: cut off a piece of the cloth that's approximately as wide as the scarf, made to about 6 1/2 \" by 10 1/2 \".","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":2182,"error":null,"output":"D"}
{"question_id":"hellaswag:00003642","item_index":168,"attempt":0,"prompt_hash":"f7b5728ecf2485040aaa679c4ca6a7e54b758911e4ff7e168d0d3cd3052f452b","question":"[header] How to buy glasses that fit [title] Have an eye examination. [step] To order glasses that fit from an optometrist, you'll need to have an up-to-date eye examination on file with their office. This usually means an exam that was performed within the last year.\n\nA. The optometrist will likely ask you about your prescription. Know when you register your prescription, and what tools or equipment you use to obtain the correct lenses.\nB. If you haven't had an exam in the last 12 months, you'll need to schedule one. [substeps] Prepare for an eye exam by scheduling it with your eye doctor.\nC. [substeps] Most optometrists will not take x-rays of sensitive areas, such as the eye canals. They may require one to one imaging or vision examinations to make sure that glasses fit properly.\nD. Tell them you want to see glasses that get altered over time and that you're looking for glasses that are rigid, while also asking how the glasses fit you. [substeps] These scans can usually be performed at a store that sells prescription glasses.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to buy glasses that fit [title] Have an eye examination. [step] To order glasses that fit from an optometrist, you'll need to have an up-to-date eye examination on file with their office. This usually means an exam that was performed within the last year.\n\nA. The optometrist will likely ask you about your prescription. Know when you register your prescription, and what tools or equipment you use to obtain the correct lenses.\nB. If you haven't had an exam in the last 12 months, you'll need to schedule one. [substeps] Prepare for an eye exam by scheduling it with your eye doctor.\nC. [substeps] Most optometrists will not take x-rays of sensitive areas, such as the eye canals. They may require one to one imaging or vision examinations to make sure that glasses fit properly.\nD. Tell them you want to see glasses that get altered over time and that you're looking for glasses that are rigid, while also asking how the glasses fit you. [substeps] These scans can usually be performed at a store that sells prescription glasses.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1917,"error":null,"output":"B"}
{"question_id":"hellaswag:00006496","item_index":169,"attempt":0,"prompt_hash":"3cfd83a052aaabca8a161cc64c1e73f77b4f43b98b5654354a004114d92413fc","question":"[header] How to do cow nail art [title] File your nails to give your nails a round shape. [step] This step is completely optional however it will make your nails look cuter. [title] With the clear nail polish apply a base coat and allow to dry.\n\nA. [title] If desired use another color on your nails if they are not already. [title] Using a cuticle pusher with a water based brush one, double strike all over each nail.\nB. [title] Allow to dry, roll your cuticles between your hands and once dry apply an topcoat to your nails. [step] Test this to see if it suits you.\nC. [step] This will prevent your nails from yellowing. [title] Apply the main colour (white) onto your nails and allow to dry.\nD. [title] Carefully comb through your nails with a nail file. [step] You want to cover each nail evenly with your base coat.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to do cow nail art [title] File your nails to give your nails a round shape. [step] This step is completely optional however it will make your nails look cuter. [title] With the clear nail polish apply a base coat and allow to dry.\n\nA. [title] If desired use another color on your nails if they are not already. [title] Using a cuticle pusher with a water based brush one, double strike all over each nail.\nB. [title] Allow to dry, roll your cuticles between your hands and once dry apply an topcoat to your nails. [step] Test this to see if it suits you.\nC. [step] This will prevent your nails from yellowing. [title] Apply the main colour (white) onto your nails and allow to dry.\nD. [title] Carefully comb through your nails with a nail file. [step] You want to cover each nail evenly with your base coat.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1768,"error":null,"output":"D."}
{"question_id":"hellaswag:00007192","item_index":170,"attempt":0,"prompt_hash":"d3c7ad3d2d31e1e645dab7cddce4475e9c9485557fa013672d9c5d8120cb062b","question":"[header] How to become a hair color specialist [title] Determine your state's requirements. [step] How long you need to be in cosmetology school depends mainly on how many hours are required by your individual u.s. state. You should determine how many hours of training and practice that you will need in order to qualify for a cosmetology certificate.\n\nA. Once you have this information, you should start looking for a job, possibly a job search program. [substeps] Each state has a growing number of jobs available to those aspiring to become a beauty technician.\nB. [substeps] For example, montana requires 2000 hours, while new mexico requires 1600 hours. If you move to another state, you will need to transfer your license.\nC. [substeps] To begin your program with the juice certification or the curls certification, you need to complete at least six months of practice before graduating. You'll need at least 40 hours of practice and at least 5 years of supervised instruction prior to going to the certification exam.\nD. [substeps] The educational requirements in cosmetology schools vary widely from state to state. Some schools might require more than one degree to gain certification.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to become a hair color specialist [title] Determine your state's requirements. [step] How long you need to be in cosmetology school depends mainly on how many hours are required by your individual u.s. state. You should determine how many hours of training and practice that you will need in order to qualify for a cosmetology certificate.\n\nA. Once you have this information, you should start looking for a job, possibly a job search program. [substeps] Each state has a growing number of jobs available to those aspiring to become a beauty technician.\nB. [substeps] For example, montana requires 2000 hours, while new mexico requires 1600 hours. If you move to another state, you will need to transfer your license.\nC. [substeps] To begin your program with the juice certification or the curls certification, you need to complete at least six months of practice before graduating. You'll need at least 40 hours of practice and at least 5 years of supervised instruction prior to going to the certification exam.\nD. [substeps] The educational requirements in cosmetology schools vary widely from state to state. Some schools might require more than one degree to gain certification.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1994,"error":null,"output":"C"}
{"question_id":"hellaswag:00001605","item_index":171,"attempt":0,"prompt_hash":"7632c8ce0a1e3f25c1fe373e9be21757fec9860569c1057fe0695ff25f7f6b26","question":"A video recap of a beach soccer game begins with an argentinian goal off a deep pass. The brazilians follow up with a goal on a penalty kick, taken by an exuberant keeper. the argentinians\n\nA. applaud as they take their bows and take their bows.\nB. then race back and begin flashing briefly with the distance scored.\nC. then restart the game in normal speed.\nD. steal a pass halfway across the sand field and take it down for a goal in the tight corner.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA video recap of a beach soccer game begins with an argentinian goal off a deep pass. The brazilians follow up with a goal on a penalty kick, taken by an exuberant keeper. the argentinians\n\nA. applaud as they take their bows and take their bows.\nB. then race back and begin flashing briefly with the distance scored.\nC. then restart the game in normal speed.\nD. steal a pass halfway across the sand field and take it down for a goal in the tight corner.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1327,"error":null,"output":"C. then restart the game in normal speed."}
{"question_id":"hellaswag:00003098","item_index":172,"attempt":0,"prompt_hash":"ba70450615d15a0e1f86e04da3d0f97aa53df1753b11515078ca5d4a293e97ca","question":"Blonde woman is running in a race track and make a jump in an elastic bed. In the background people are running around the court. women\n\nA. women are standing on front curling canoes.\nB. are bungee jumping in fancy style indoors.\nC. are walking with water sunglasses on.\nD. are kneeling down in the racetrack and runs and in the end makes a long jump.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nBlonde woman is running in a race track and make a jump in an elastic bed. In the background people are running around the court. women\n\nA. women are standing on front curling canoes.\nB. are bungee jumping in fancy style indoors.\nC. are walking with water sunglasses on.\nD. are kneeling down in the racetrack and runs and in the end makes a long jump.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1512,"error":null,"output":"D. are kneeing down in the race track and runs and in the end makes a long jump."}
{"question_id":"hellaswag:00006849","item_index":173,"attempt":0,"prompt_hash":"934469017c86a837bc0fa26574c48a28db82fc4cffc0d95f282eebf7757cdf80","question":"[header] How to prevent hair tourniquet syndrome in babies [title] Understand what causes hair tourniquet syndrome. [step] Hair tourniquet syndrome is most common in the first four months of your baby's life. This is because it is during this timeframe that women may inadvertently lose more hair following pregnancy.\n\nA. This is related to the changes the body is undergoing as it returns to its normal state following pregnancy. [substeps] A baby can get hair tourniquet syndrome from repetitive motion in a confined space where there is hair present.\nB. Women who lose more hair during pregnancy may permanently acquire more hair in the long run, as many of the more affected women in your baby's life are potential mothers. [title] Be aware of the risks and benefits of hair tourniquet syndrome.\nC. In the fetus, blood vessels move more towards the cervix than do testosterone cells. Testosterone cell walls during this time period block oxygen from entering the uterus, causing excess hair to clog the cervix.\nD. These women are pregnant by the time they start the second trimester. They may experience significant hair loss over the first four to eight months following.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to prevent hair tourniquet syndrome in babies [title] Understand what causes hair tourniquet syndrome. [step] Hair tourniquet syndrome is most common in the first four months of your baby's life. This is because it is during this timeframe that women may inadvertently lose more hair following pregnancy.\n\nA. This is related to the changes the body is undergoing as it returns to its normal state following pregnancy. [substeps] A baby can get hair tourniquet syndrome from repetitive motion in a confined space where there is hair present.\nB. Women who lose more hair during pregnancy may permanently acquire more hair in the long run, as many of the more affected women in your baby's life are potential mothers. [title] Be aware of the risks and benefits of hair tourniquet syndrome.\nC. In the fetus, blood vessels move more towards the cervix than do testosterone cells. Testosterone cell walls during this time period block oxygen from entering the uterus, causing excess hair to clog the cervix.\nD. These women are pregnant by the time they start the second trimester. They may experience significant hair loss over the first four to eight months following.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1915,"error":null,"output":"A"}
{"question_id":"hellaswag:00004075","item_index":174,"attempt":0,"prompt_hash":"8d3a446369c75bcee9171c9e592e7ffc776303aef211d4a24ffd10f73260ddaa","question":"[header] How to know what to expect on a newborn's skin [title] Note your newborn's skin tone. [step] At birth, a newborn's skin may be reddish or pinkish. However, the baby's hands and feet may be bluish (acrocyanosis) because blood and oxygen are not yet circulating fully to the extremities.\n\nA. As the newborn's circulatory system opens, this bluish color will subside. [substeps] If your newborn's skin is bluish all over (cyanosis), however, let your physician know right away.\nB. This means they will not produce much oxygen. During your newborn's first few months, keep in mind that the inside of their diaper should be pink in appearance, and the eyes blue.\nC. Therefore, it's important to measure the temperature and tone of your newborn's skin before attempting to formulate a plan to replicate the actual pregnancy. [substeps] You'll first need to check the mother's temperature.\nD. A newborn's skin color is linked to their normal appearance, so the newborn's hands and feet may appear lighter than normal at birth. [substeps] The \" white \" spot is located just behind the eyeball of the newborn.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to know what to expect on a newborn's skin [title] Note your newborn's skin tone. [step] At birth, a newborn's skin may be reddish or pinkish. However, the baby's hands and feet may be bluish (acrocyanosis) because blood and oxygen are not yet circulating fully to the extremities.\n\nA. As the newborn's circulatory system opens, this bluish color will subside. [substeps] If your newborn's skin is bluish all over (cyanosis), however, let your physician know right away.\nB. This means they will not produce much oxygen. During your newborn's first few months, keep in mind that the inside of their diaper should be pink in appearance, and the eyes blue.\nC. Therefore, it's important to measure the temperature and tone of your newborn's skin before attempting to formulate a plan to replicate the actual pregnancy. [substeps] You'll first need to check the mother's temperature.\nD. A newborn's skin color is linked to their normal appearance, so the newborn's hands and feet may appear lighter than normal at birth. [substeps] The \" white \" spot is located just behind the eyeball of the newborn.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1996,"error":null,"output":"A"}
{"question_id":"hellaswag:00005874","item_index":175,"attempt":0,"prompt_hash":"9828bdfb260b9784a05d20bf81f8a1b51f9e87e13506686ba37cef506e22ead3","question":"[header] How to care for straight hair [title] Shampoo and condition medium-to-thick straight hair 2-3 times per week. [step] Naturally straight hair that has a medium or thick texture can be shampooed 2-3 times per week, or every other day if you're very active. [substeps] Use a regular or hydrating shampoo, or shampoo for color-treated hair if you use color in your hair.\n\nA. For curly hair, use a clarifying shampoo that includes sulfates. If you're under the age of 20 and have hair that's already dyed blonde, skip this step.\nB. [title] Comb and condition your hair once or twice a week. [step] Thinning hair can cause hair to age a lot faster, so regular regular brushing can help your hair grow out a bit more.\nC. [title] Shampoo naturally fine, straight hair every day if it's oily. [step] A drawback of naturally straight, fine hair is that it can become oily quickly.\nD. Using a treatment formulated specifically for curly hair will make the shampoo more hydrating. If you have curly hair, try shampooing a few times per week in the morning or evening.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to care for straight hair [title] Shampoo and condition medium-to-thick straight hair 2-3 times per week. [step] Naturally straight hair that has a medium or thick texture can be shampooed 2-3 times per week, or every other day if you're very active. [substeps] Use a regular or hydrating shampoo, or shampoo for color-treated hair if you use color in your hair.\n\nA. For curly hair, use a clarifying shampoo that includes sulfates. If you're under the age of 20 and have hair that's already dyed blonde, skip this step.\nB. [title] Comb and condition your hair once or twice a week. [step] Thinning hair can cause hair to age a lot faster, so regular regular brushing can help your hair grow out a bit more.\nC. [title] Shampoo naturally fine, straight hair every day if it's oily. [step] A drawback of naturally straight, fine hair is that it can become oily quickly.\nD. Using a treatment formulated specifically for curly hair will make the shampoo more hydrating. If you have curly hair, try shampooing a few times per week in the morning or evening.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":2034,"error":null,"output":"A"}
{"question_id":"hellaswag:00000122","item_index":176,"attempt":0,"prompt_hash":"df719ff3e699875f782d6e6c0459ea1bb5e3b887ca17b13dedd315f5d13c41ef","question":"A group of people are in a house. a man\n\nA. is holding cored soap in his hand as he washes with a bottle.\nB. is mopping the floor with a mop.\nC. is shown wearing skis as he talks about areas he will like to ski on.\nD. uses a paintball gun on his child.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA group of people are in a house. a man\n\nA. is holding cored soap in his hand as he washes with a bottle.\nB. is mopping the floor with a mop.\nC. is shown wearing skis as he talks about areas he will like to ski on.\nD. uses a paintball gun on his child.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":841,"error":null,"output":"A"}
{"question_id":"hellaswag:00000236","item_index":177,"attempt":0,"prompt_hash":"40bedc25dcad0b75cc83f656ac214fb30b79f011f0790fa041e3603aba8c2513","question":"A girl stands in a bathroom talking to the camera as a man films her. the lady\n\nA. wipes her eye makeup off and washes her face.\nB. then grabs pink lip stick and takes the girl's lip stick.\nC. finally gets a space.\nD. hoses the girl down and makes sure she does a great job.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA girl stands in a bathroom talking to the camera as a man films her. the lady\n\nA. wipes her eye makeup off and washes her face.\nB. then grabs pink lip stick and takes the girl's lip stick.\nC. finally gets a space.\nD. hoses the girl down and makes sure she does a great job.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":778,"error":null,"output":"C"}
{"question_id":"hellaswag:00007684","item_index":178,"attempt":0,"prompt_hash":"ab0dc9f8e30558010ef11ea7581fdda6aaaaf70e4c28513a0064358df0f1b1ed","question":"[header] How to make a \" draw my life \" video [title] Start a document either on physical pieces of paper or digitally on your computer to store all of your information. [step] This document will include any and all notable events in your life, so you will need plenty of paper if you are using physical paper. [title] Contact your family.\n\nA. [step] Talk to them and ask for their support. Always try to have a \" family meeting.\nB. [step] If you have any siblings who aren't related to any of your other relatives, call ahead and let them know you are creating the video. [substeps] Don't make it too obvious with the press as this can drive off the edge of your decision to create your video.\nC. [step] Chances are, your family will have the most intimate knowledge of your life and can help describe who you are. Your parents, in particular, will remember what you were like as an infant and a young child much better than you can.\nD. [step] Tell them your purpose for creating this video, and allow them to pitch in on what you are drawing. It's best to involve them beforehand or ask them to sing in and take the lead as they draw! [title] Make a silhouette of the subject.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make a \" draw my life \" video [title] Start a document either on physical pieces of paper or digitally on your computer to store all of your information. [step] This document will include any and all notable events in your life, so you will need plenty of paper if you are using physical paper. [title] Contact your family.\n\nA. [step] Talk to them and ask for their support. Always try to have a \" family meeting.\nB. [step] If you have any siblings who aren't related to any of your other relatives, call ahead and let them know you are creating the video. [substeps] Don't make it too obvious with the press as this can drive off the edge of your decision to create your video.\nC. [step] Chances are, your family will have the most intimate knowledge of your life and can help describe who you are. Your parents, in particular, will remember what you were like as an infant and a young child much better than you can.\nD. [step] Tell them your purpose for creating this video, and allow them to pitch in on what you are drawing. It's best to involve them beforehand or ask them to sing in and take the lead as they draw! [title] Make a silhouette of the subject.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":2066,"error":null,"output":"C"}
{"question_id":"hellaswag:00005585","item_index":179,"attempt":0,"prompt_hash":"48b676cebe42e6cf50b41d9fae5296b8f2161750f40aee9c2d1c615999229762","question":"[header] How to straighten curly and thick hair [title] Wash your hair with shampoo and conditioner to rinse away the dirt. [step] Use a gentle shampoo and conditioner as flat-ironing hair may cause damage to your hair. Shampooing and conditioning will remove unwanted oil and product build-up from your hair.\n\nA. [title] Scrape out dry, damaged, split ends using a comb or a rat tail comb. [step] Coarse, split ends can be tight on a rat tail comb.\nB. [substeps] If you're going to wash your hair with soap, be sure to pay special attention to your hair's natural oil content. The bottle should say \" baby shampoo \" and \" natural \" on the label.\nC. [substeps] You can skip this step if your hair has been bleached recently. [title] Apply a serum or oil to your hair, starting at the roots and working downward.\nD. [substeps] Look for shampoo and conditioner products that are sulfate-free and jam packed with glycerin. Glycerin helps reduce frizz as it penetrates hair follicles, hydrating hair from the inside out.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to straighten curly and thick hair [title] Wash your hair with shampoo and conditioner to rinse away the dirt. [step] Use a gentle shampoo and conditioner as flat-ironing hair may cause damage to your hair. Shampooing and conditioning will remove unwanted oil and product build-up from your hair.\n\nA. [title] Scrape out dry, damaged, split ends using a comb or a rat tail comb. [step] Coarse, split ends can be tight on a rat tail comb.\nB. [substeps] If you're going to wash your hair with soap, be sure to pay special attention to your hair's natural oil content. The bottle should say \" baby shampoo \" and \" natural \" on the label.\nC. [substeps] You can skip this step if your hair has been bleached recently. [title] Apply a serum or oil to your hair, starting at the roots and working downward.\nD. [substeps] Look for shampoo and conditioner products that are sulfate-free and jam packed with glycerin. Glycerin helps reduce frizz as it penetrates hair follicles, hydrating hair from the inside out.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1983,"error":null,"output":"A"}
{"question_id":"hellaswag:00006191","item_index":180,"attempt":0,"prompt_hash":"4550832889f154e7655fe509745f62bc9105420ab3ff508303d598381df98881","question":"[header] How to make a makeup artist portfolio [title] Present your makeup artist portfolio in a professional manner. [step] Purchase a professional portfolio, either purchased from an art supply store or ordered online for customization, in which you can display photos of your work. A standard black artist portfolio with your name on the front works best.\n\nA. [title] Collect any photographs you have of your artist, including photographs taken by the artist's assistants. [title] Include a list of your work, contact information, feedback and word of mouth.\nB. [title] Catch the attention of the client with your first photo. [step] Your first picture should be your best work and the style of make up you are most passionate about.\nC. Wrap photos of your work in decorative paper to appeal to the eye, then hide your portfolio under a cover sheet to avoid exposed artwork. [substeps] If possible, limit your portfolio to three or four photographs.\nD. [substeps] When presenting your portfolio professionally, you are replying to follow up questions regarding your work. It's up to you what can be covered and the subject matter discussed for you to have an idea of what to cover.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make a makeup artist portfolio [title] Present your makeup artist portfolio in a professional manner. [step] Purchase a professional portfolio, either purchased from an art supply store or ordered online for customization, in which you can display photos of your work. A standard black artist portfolio with your name on the front works best.\n\nA. [title] Collect any photographs you have of your artist, including photographs taken by the artist's assistants. [title] Include a list of your work, contact information, feedback and word of mouth.\nB. [title] Catch the attention of the client with your first photo. [step] Your first picture should be your best work and the style of make up you are most passionate about.\nC. Wrap photos of your work in decorative paper to appeal to the eye, then hide your portfolio under a cover sheet to avoid exposed artwork. [substeps] If possible, limit your portfolio to three or four photographs.\nD. [substeps] When presenting your portfolio professionally, you are replying to follow up questions regarding your work. It's up to you what can be covered and the subject matter discussed for you to have an idea of what to cover.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1890,"error":null,"output":"A"}
{"question_id":"hellaswag:00004013","item_index":181,"attempt":0,"prompt_hash":"2cbbc64ad7953bcf26d512606b4811035731525f814c5471549d993250265cbe","question":"[header] How to react if a toddler swallows a coin [title] Remain calm. [step] It can be scary to learn your toddler swallowed a foreign object. However, the calmer you are, the better you'll be able to deal with the situation.\n\nA. Don't panic-take a deep breath, count to five, and try not to give into your emotions. [substeps] Describe the experience calmly and explain what happened.\nB. Even if you think the child is choking, it is best to remain composed. [substeps] If you have the urge to call for help, do so.\nC. Angry, upset, and anxious toddlers will often develop cravings for unknown foods or dangerous objects, which will often lead to eating difficult food (such as foods like candy) or unsafe foods. [substeps] Write down a list of the things your toddler eats and, if necessary, the people who eat them.\nD. In the aftermath, take a few deep breaths and then proceed to care for your child. [substeps] You also want to remain calm to keep your toddler calm.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to react if a toddler swallows a coin [title] Remain calm. [step] It can be scary to learn your toddler swallowed a foreign object. However, the calmer you are, the better you'll be able to deal with the situation.\n\nA. Don't panic-take a deep breath, count to five, and try not to give into your emotions. [substeps] Describe the experience calmly and explain what happened.\nB. Even if you think the child is choking, it is best to remain composed. [substeps] If you have the urge to call for help, do so.\nC. Angry, upset, and anxious toddlers will often develop cravings for unknown foods or dangerous objects, which will often lead to eating difficult food (such as foods like candy) or unsafe foods. [substeps] Write down a list of the things your toddler eats and, if necessary, the people who eat them.\nD. In the aftermath, take a few deep breaths and then proceed to care for your child. [substeps] You also want to remain calm to keep your toddler calm.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1900,"error":null,"output":"A"}
{"question_id":"hellaswag:00007386","item_index":182,"attempt":0,"prompt_hash":"b606034293754b6dab47ba5c07b1fad117047ca750f68caaf46ac44a1ea8a1d4","question":"[header] How to get married [title] Plan ahead to pop the question. [step] Your (hopefully) spouse-to-be should be surprised, delighted, and/or put a bit off-balance by your proposition. It is the sort of romantic moment most people live for, so be sure to get it right by planning it out early.\n\nA. You will be surprised by their pleasure at the prospect of marriage, so don't be surprised when they have asked all the questions and no qualms. [substeps] Figure out your wishes, then talk it out with them beforehand.\nB. Put things off until you get married and you will be far more likely to visit the venue and get married on time. Also, try planning in advance to avoid the awkwardness of waiting until the last minute.\nC. Think about the appropriate place, time, and words to say. Try to think of things your lover particularly enjoys-favorite restaurants, activities, and/or music-but for whatever reason does not get to indulge in very often.\nD. Even someone you're keen to go on date with may get laid off. , try planning earlier, for instance, unless your spouse/to-be has a work-related medical issue that's contributing to his/her inability to get married.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to get married [title] Plan ahead to pop the question. [step] Your (hopefully) spouse-to-be should be surprised, delighted, and/or put a bit off-balance by your proposition. It is the sort of romantic moment most people live for, so be sure to get it right by planning it out early.\n\nA. You will be surprised by their pleasure at the prospect of marriage, so don't be surprised when they have asked all the questions and no qualms. [substeps] Figure out your wishes, then talk it out with them beforehand.\nB. Put things off until you get married and you will be far more likely to visit the venue and get married on time. Also, try planning in advance to avoid the awkwardness of waiting until the last minute.\nC. Think about the appropriate place, time, and words to say. Try to think of things your lover particularly enjoys-favorite restaurants, activities, and/or music-but for whatever reason does not get to indulge in very often.\nD. Even someone you're keen to go on date with may get laid off. , try planning earlier, for instance, unless your spouse/to-be has a work-related medical issue that's contributing to his/her inability to get married.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":2056,"error":null,"output":"C"}
{"question_id":"hellaswag:00005459","item_index":183,"attempt":0,"prompt_hash":"94b211fa3bc2a4a0e4859f528b82daf6eca8805e09b8314b3a3997d9411072a7","question":"[header] How to wash victoria's secret bras [title] Buy a delicate laundry soap or fabric softener. [step] Bras have a thin layer of a foam called the contour. The contour's job is to camouflage the main shape of the nipple and breast.\n\nA. These fibers are a great way to show off your breasts if you choose to wash any new bras when they are the same size as your breasts. [substeps] For example, you could purchase a bra so that you have 40% strength but 180% softness.\nB. At victoria's secret, the material used for the contour is very delicate and is very sensitive to certain products. Buy a delicate laundry soap or fabric softener to wash your bra in.\nC. When placed properly, this layer hides the lower portion of the breast, while leaving the outside smooth and exposed. [substeps] If your underarm line is pretty slim, you may prefer to use a looser fabric such as cotton to help conceal more of your underarm area.\nD. Use an organic-grade detergent to wash your bra. Wash only as much as you plan to wear.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to wash victoria's secret bras [title] Buy a delicate laundry soap or fabric softener. [step] Bras have a thin layer of a foam called the contour. The contour's job is to camouflage the main shape of the nipple and breast.\n\nA. These fibers are a great way to show off your breasts if you choose to wash any new bras when they are the same size as your breasts. [substeps] For example, you could purchase a bra so that you have 40% strength but 180% softness.\nB. At victoria's secret, the material used for the contour is very delicate and is very sensitive to certain products. Buy a delicate laundry soap or fabric softener to wash your bra in.\nC. When placed properly, this layer hides the lower portion of the breast, while leaving the outside smooth and exposed. [substeps] If your underarm line is pretty slim, you may prefer to use a looser fabric such as cotton to help conceal more of your underarm area.\nD. Use an organic-grade detergent to wash your bra. Wash only as much as you plan to wear.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1963,"error":null,"output":"B"}
{"question_id":"hellaswag:00009359","item_index":184,"attempt":0,"prompt_hash":"fa8c95c1149aefdda8138dd4880880b1777c7da7297f91582040d6e7be1fcf38","question":"[header] How to remove spray tan [title] Try a home remedy of baking soda and lemon. [step] Mix lemon juice with baking soda until it forms a paste. [substeps] In the shower, scrub the paste on to your skin.\n\nA. You can use a loofah or other shower sponge to scrub it on. You can also just use your hand, since the baking soda will act as an exfoliant.\nB. Leave it on for a couple of minutes before rinsing it off. You will notice the sebum evaporating and your skin will feel refreshed and fresh again.\nC. The paste will rub the spray tan away, leaving your skin with only the spray tan. Follow up with conditioner to promote intense hydration.\nD. For the best results, place the paste in a clean bottle, crack open the cap, and squirt the soap directly in your spray stains. Leave the paste in the fridge until you are comfortable with the way it looks on your skin.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to remove spray tan [title] Try a home remedy of baking soda and lemon. [step] Mix lemon juice with baking soda until it forms a paste. [substeps] In the shower, scrub the paste on to your skin.\n\nA. You can use a loofah or other shower sponge to scrub it on. You can also just use your hand, since the baking soda will act as an exfoliant.\nB. Leave it on for a couple of minutes before rinsing it off. You will notice the sebum evaporating and your skin will feel refreshed and fresh again.\nC. The paste will rub the spray tan away, leaving your skin with only the spray tan. Follow up with conditioner to promote intense hydration.\nD. For the best results, place the paste in a clean bottle, crack open the cap, and squirt the soap directly in your spray stains. Leave the paste in the fridge until you are comfortable with the way it looks on your skin.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1732,"error":null,"output":"C"}
{"question_id":"hellaswag:00009225","item_index":274,"attempt":0,"prompt_hash":"964b730ee2eded6d70eddd25feab686b4355033e93e5eb84c20163c0a7051b93","question":"[header] How to pay for ivf treatment [title] Inform yourself on state laws. [step] In the us, fifteen states have laws that require insurance coverage for infertility treatments. For information on each states' respective laws, see.\n\nA. If you live in any of the following states, you can pay for ivf at least in part with insurance : [substeps] Arkansas california connecticut hawaii illinois louisiana maryland massachusetts montana new jersey new york ohio rhode island texas west virginia [title] Contact your insurance company. [step] If you carry an insurance policy and are considering ivf, call or visit your insurance company as soon as possible.\nB. [title] Check the number of infertility clinics in your area. [step] The number of infertility clinics in a given area vary greatly.\nC. [title] Know what contact information for your donor. [step] Even though you might be making ivf treatments for your child, such as money that you no longer can give to your donor, the process is still in place.\nD. [substeps] You should be aware of the laws. Generally, you need to be covered if you have eggs, are over thirty, do not have other sexually transmitted diseases, have another custodial sex drive, or have already been diagnosed with infertility.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to pay for ivf treatment [title] Inform yourself on state laws. [step] In the us, fifteen states have laws that require insurance coverage for infertility treatments. For information on each states' respective laws, see.\n\nA. If you live in any of the following states, you can pay for ivf at least in part with insurance : [substeps] Arkansas california connecticut hawaii illinois louisiana maryland massachusetts montana new jersey new york ohio rhode island texas west virginia [title] Contact your insurance company. [step] If you carry an insurance policy and are considering ivf, call or visit your insurance company as soon as possible.\nB. [title] Check the number of infertility clinics in your area. [step] The number of infertility clinics in a given area vary greatly.\nC. [title] Know what contact information for your donor. [step] Even though you might be making ivf treatments for your child, such as money that you no longer can give to your donor, the process is still in place.\nD. [substeps] You should be aware of the laws. Generally, you need to be covered if you have eggs, are over thirty, do not have other sexually transmitted diseases, have another custodial sex drive, or have already been diagnosed with infertility.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4478,"error":null,"output":"A"}
{"question_id":"hellaswag:00003690","item_index":275,"attempt":0,"prompt_hash":"4924d591dda79ea4c60843f1a766bfe5e7c350b30b6ff264ffeaa9a33f76794e","question":"[header] How to do a circle dance in a wedding [title] Plan the wedding reception in an area where a circle dance can be performed with no obstacles in the way. [step] A large room with lots of open floor space or an outside venue would work well. Guests must feel comfortable dancing without obstructions in the way.\n\nA. [title] Understand that you and your partner will only perform this type of dancing in a small space. [step] This may be within the hall of a small restaurant or the changing rooms of a larger club.\nB. [title] Have a date set up with your guest list. [step] The result to this approach is the engagement party.\nC. The venue also does not need to be crowded because it is not your wedding day and the reception already is a few days away. [title] Have a large table set up (where the guests can make their own food and drinks easily) in the reception area.\nD. [title] Plan ahead for appropriate music to be played or performed at your wedding reception. [step] A circle dance should be performed to lively and exuberant music.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to do a circle dance in a wedding [title] Plan the wedding reception in an area where a circle dance can be performed with no obstacles in the way. [step] A large room with lots of open floor space or an outside venue would work well. Guests must feel comfortable dancing without obstructions in the way.\n\nA. [title] Understand that you and your partner will only perform this type of dancing in a small space. [step] This may be within the hall of a small restaurant or the changing rooms of a larger club.\nB. [title] Have a date set up with your guest list. [step] The result to this approach is the engagement party.\nC. The venue also does not need to be crowded because it is not your wedding day and the reception already is a few days away. [title] Have a large table set up (where the guests can make their own food and drinks easily) in the reception area.\nD. [title] Plan ahead for appropriate music to be played or performed at your wedding reception. [step] A circle dance should be performed to lively and exuberant music.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1557,"error":null,"output":"D"}
{"question_id":"hellaswag:00007361","item_index":276,"attempt":0,"prompt_hash":"9b3d73631eb8a6f90885fc8ffcc72d190bdb00a5063b91a003ca22f1649b9b3f","question":"[header] How to make rainbow bracelets [title] Select rubber bands. [step] Choose a rubber band for each color of the rainbow, e.g., blue, red, green, etc. Rubber bands.\n\nA. Gold gold gold bands make for large bracelets, while silver gold bands provide a nice rainbow bracelet. You can purchase rubber bands from sephora or look at the bug, ball and snake stores for enthusiasts.\nB. If you can't find all the colors, use markers to color plain rubber bands the colors that you are missing. [title] Put the rubber bands on your wrist.\nC. [substeps] If you're making rainbow bracelets for a child, you can choose colored rubber bands as a gift. How you use the bands will depend on whether you want to draw the pattern or use a contrasting color (e.g., blue wouldn't work well with black).\nD. Choose as many different colors as possible, since these rubber bands will be used to hold on concert tickets. They also create the movement that the artist will do while walking.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make rainbow bracelets [title] Select rubber bands. [step] Choose a rubber band for each color of the rainbow, e.g., blue, red, green, etc. Rubber bands.\n\nA. Gold gold gold bands make for large bracelets, while silver gold bands provide a nice rainbow bracelet. You can purchase rubber bands from sephora or look at the bug, ball and snake stores for enthusiasts.\nB. If you can't find all the colors, use markers to color plain rubber bands the colors that you are missing. [title] Put the rubber bands on your wrist.\nC. [substeps] If you're making rainbow bracelets for a child, you can choose colored rubber bands as a gift. How you use the bands will depend on whether you want to draw the pattern or use a contrasting color (e.g., blue wouldn't work well with black).\nD. Choose as many different colors as possible, since these rubber bands will be used to hold on concert tickets. They also create the movement that the artist will do while walking.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1563,"error":null,"output":"B"}
{"question_id":"hellaswag:00001527","item_index":277,"attempt":0,"prompt_hash":"67fa856b38ebe6a5d7207c0eec8ac7b3f7270db399a7cf6d001628da76847b6f","question":"A man is seen standing in a field with his eye closed and standing still. The man then begins moving his arms and legs around his body. he\n\nA. then uses his hands to move the objects around and then cuts it all to pieces.\nB. is seen in several angles swinging around himself and others as well.\nC. continues moving his body around with his eyes closed.\nD. moves himself all around the field and ends by brushing off the dirt with his hand.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA man is seen standing in a field with his eye closed and standing still. The man then begins moving his arms and legs around his body. he\n\nA. then uses his hands to move the objects around and then cuts it all to pieces.\nB. is seen in several angles swinging around himself and others as well.\nC. continues moving his body around with his eyes closed.\nD. moves himself all around the field and ends by brushing off the dirt with his hand.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":864,"error":null,"output":"D"}
{"question_id":"hellaswag:00002599","item_index":278,"attempt":0,"prompt_hash":"e030f56dcbddf276d17f34f518126717ce641bae7a98c5b4f2390601677f2208","question":"A kid is in the yard putting the ball on the side of the fence. An adult came and start to cut grass with the kid. the man\n\nA. moved the goal net on the side and continue to cut the grass using the grass cutter, then he move the trampoline and move cut the grass as the boy is playing on the side.\nB. pushed the kid and grabbed the grass.\nC. went back and looked at the kid and take pictures of the girl.\nD. walked up to the kid and started to take the leaves off the fence.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA kid is in the yard putting the ball on the side of the fence. An adult came and start to cut grass with the kid. the man\n\nA. moved the goal net on the side and continue to cut the grass using the grass cutter, then he move the trampoline and move cut the grass as the boy is playing on the side.\nB. pushed the kid and grabbed the grass.\nC. went back and looked at the kid and take pictures of the girl.\nD. walked up to the kid and started to take the leaves off the fence.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":898,"error":null,"output":"D"}
{"question_id":"hellaswag:00008629","item_index":279,"attempt":0,"prompt_hash":"1b5d237551a36e2752c1588c1624f649d679678cf416c623c105c900ef8720f7","question":"[header] How to prepare for a settlement conference [title] Decide what you want. [step] Settlement is voluntary, which means no one can force you to accept a settlement you disagree with. However, before going to the conference, you need to know your goals.\n\nA. You want to know them at an appropriate time and place. [substeps] You can find settlement conferences in the following ways: look for settlement conferences that do not include the word \" settlement \" in the list of conference participants-the usual \" settlement \" or \" offer \" word from the list of attendees.\nB. These will depend on the type of case. [substeps] In a divorce dispute, for example, you'll need to determine child custody, visitation, child support, and spousal maintenance (alimony).\nC. If you disagree with the other side, then your agreement could lead to bad treatment. [substeps] You should take into account how much legal costs you are entitled to receive for your settlement.\nD. Before going to the conference, talk to your doctor and other healthcare providers about what you want to do, and what their rationale is. [substeps] Often, settlement conferences are for families or people willing to pay for health care.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to prepare for a settlement conference [title] Decide what you want. [step] Settlement is voluntary, which means no one can force you to accept a settlement you disagree with. However, before going to the conference, you need to know your goals.\n\nA. You want to know them at an appropriate time and place. [substeps] You can find settlement conferences in the following ways: look for settlement conferences that do not include the word \" settlement \" in the list of conference participants-the usual \" settlement \" or \" offer \" word from the list of attendees.\nB. These will depend on the type of case. [substeps] In a divorce dispute, for example, you'll need to determine child custody, visitation, child support, and spousal maintenance (alimony).\nC. If you disagree with the other side, then your agreement could lead to bad treatment. [substeps] You should take into account how much legal costs you are entitled to receive for your settlement.\nD. Before going to the conference, talk to your doctor and other healthcare providers about what you want to do, and what their rationale is. [substeps] Often, settlement conferences are for families or people willing to pay for health care.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1725,"error":null,"output":"A"}
{"question_id":"hellaswag:00006762","item_index":280,"attempt":0,"prompt_hash":"45564c9e434f13f93012d597a5c9d19df0dc2bd5a85367fda8ddf0cf9db80c8d","question":"[header] How to blunt a sword or knife [title] Consider the type of blade you're blunting. [step] Thinner blades are more delicate and will require gentler tools to effectively blunt them. Thicker blades, on the other hand, may be dulled with heavier tools without as much worry about damage.\n\nA. Consider if you are going to be using a heavy blade when you're pulling it out of its sheath. Thinner blades may blunt faster than thicker ones.\nB. [title] Select smaller, heavier, and less flammable blades. [step] Heavier blades are easier to handle, making for harsher sawing.\nC. [title] Consider blades that have heavy prongs, or are shaped like knives. [step] Compared to curved blades, light-point knives have thinner heads and smaller prongs.\nD. [substeps] Generally speaking, you should use manual techniques for long, flimsy, and thin blades. Sanding machines work best for the larger ones.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to blunt a sword or knife [title] Consider the type of blade you're blunting. [step] Thinner blades are more delicate and will require gentler tools to effectively blunt them. Thicker blades, on the other hand, may be dulled with heavier tools without as much worry about damage.\n\nA. Consider if you are going to be using a heavy blade when you're pulling it out of its sheath. Thinner blades may blunt faster than thicker ones.\nB. [title] Select smaller, heavier, and less flammable blades. [step] Heavier blades are easier to handle, making for harsher sawing.\nC. [title] Consider blades that have heavy prongs, or are shaped like knives. [step] Compared to curved blades, light-point knives have thinner heads and smaller prongs.\nD. [substeps] Generally speaking, you should use manual techniques for long, flimsy, and thin blades. Sanding machines work best for the larger ones.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1623,"error":null,"output":"A"}
{"question_id":"hellaswag:00002336","item_index":281,"attempt":0,"prompt_hash":"8a2b3e0584adec97353bd26aea1ce3c83dd5e8871e22766d8ab59a686890311d","question":"He cleans the knife on a towel and shows the residue left behind. He explains the difference between a honing steel and a sharpening steel. a black screen\n\nA. appears with blue and white drawings of swords and arrows on it and the word \" swords lacrosse \" on it.\nB. appears on black, then black again.\nC. appears and fades into a white screen with online culinary schools, websites and a copyright to a company.\nD. flashes and the new recipe appears.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nHe cleans the knife on a towel and shows the residue left behind. He explains the difference between a honing steel and a sharpening steel. a black screen\n\nA. appears with blue and white drawings of swords and arrows on it and the word \" swords lacrosse \" on it.\nB. appears on black, then black again.\nC. appears and fades into a white screen with online culinary schools, websites and a copyright to a company.\nD. flashes and the new recipe appears.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":925,"error":null,"output":"B"}
{"question_id":"hellaswag:00003482","item_index":282,"attempt":0,"prompt_hash":"74e8f08df1334e807819fbf2b7c18ae7bf55fc241aabbf11a9ffabe833bbe123","question":"[header] How to apply pencil eyeliner [title] Consider what type of pencil eyeliner to use. [step] There are many different types of eyeliner that come in a pencil form, each of which has a different consistency and texture, application, and effect. [substeps] Powder-based pencils, sometimes called kohl, offer less intense color.\n\nA. Your normal eyeliner, like 1 ounce (30 ml), may not stand out. Almost any eyeliner should bring your eyes together.\nB. These millimeters are typically all about one-eighth strength, whereas pencil pencils come in a multi-color. Thick kohl pencils are thicker than pencil pencils and usually have a shorter difference in length than pencil pencils.\nC. They're also a good option if you want to smudge the eyeliner for a \" smoky eye \" effect. Gel-or cream-based pencils offer simple and excellent application.\nD. Kohl is very liquid or gel-like and is prone to lint buildup and streaking. It tends to get weighed down during use.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to apply pencil eyeliner [title] Consider what type of pencil eyeliner to use. [step] There are many different types of eyeliner that come in a pencil form, each of which has a different consistency and texture, application, and effect. [substeps] Powder-based pencils, sometimes called kohl, offer less intense color.\n\nA. Your normal eyeliner, like 1 ounce (30 ml), may not stand out. Almost any eyeliner should bring your eyes together.\nB. These millimeters are typically all about one-eighth strength, whereas pencil pencils come in a multi-color. Thick kohl pencils are thicker than pencil pencils and usually have a shorter difference in length than pencil pencils.\nC. They're also a good option if you want to smudge the eyeliner for a \" smoky eye \" effect. Gel-or cream-based pencils offer simple and excellent application.\nD. Kohl is very liquid or gel-like and is prone to lint buildup and streaking. It tends to get weighed down during use.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1540,"error":null,"output":"B"}
{"question_id":"hellaswag:00006955","item_index":283,"attempt":0,"prompt_hash":"40b608aca063d95198acb5a9779f0983d6c125eae430da86bede08bee9917078","question":"[header] How to keep latchkey kids safe [title] Enlist the help of a neighbor who lives close by to keep an eye out on your child and who can serve as an emergency contact for you and your child. [title] Make sure your child locks the door after letting himself in. [title] Instruct your child to limit sharing the fact that he is home alone when school is out.\n\nA. [step] It can be dangerous for your child to tell others that he is unsupervised after school. [title] Establish a regular time for your child to call you to tell you he is home.\nB. [step] This ensures that your child knows that he can rely on you to be in charge of his school. [title] Have your child lock up as soon as school is out so he knows that you will be in charge of his school the next day.\nC. [title] Also know that you'll need to make sure your family members know about everything. [title] Rule out a bad or suitable explanation of why your child is home alone.\nD. [step] Describe this fact loudly so that children can hear it. If possible, have your child lock the front door, even if homework is early in the day.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to keep latchkey kids safe [title] Enlist the help of a neighbor who lives close by to keep an eye out on your child and who can serve as an emergency contact for you and your child. [title] Make sure your child locks the door after letting himself in. [title] Instruct your child to limit sharing the fact that he is home alone when school is out.\n\nA. [step] It can be dangerous for your child to tell others that he is unsupervised after school. [title] Establish a regular time for your child to call you to tell you he is home.\nB. [step] This ensures that your child knows that he can rely on you to be in charge of his school. [title] Have your child lock up as soon as school is out so he knows that you will be in charge of his school the next day.\nC. [title] Also know that you'll need to make sure your family members know about everything. [title] Rule out a bad or suitable explanation of why your child is home alone.\nD. [step] Describe this fact loudly so that children can hear it. If possible, have your child lock the front door, even if homework is early in the day.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1784,"error":null,"output":"D"}
{"question_id":"hellaswag:00007189","item_index":284,"attempt":0,"prompt_hash":"798010d27a91daec182db1656b9a793d0760738307866e9c8ab7d6f7d7386fb0","question":"[header] How to deal with mental illness linked to childhood bullying [title] Talk to a professional. [step] The first step in dealing with any type of mental illness is to speak with a professional about what you are going through. If you aren't already doing so, you should seek the help of a professional.\n\nA. Professionals like psychologists, psychiatrists, and therapists are trained to help people deal with mental illness. Their experience and training can help you manage your mental illness better.\nB. Children with mental illness often seek mental health care more often than adults. [title] Read on the child's situation.\nC. Talk to your doctor or a mental health professional to get specific information, and then take some time to talk in private. [substeps] Professional support can be a great help.\nD. They may be able to bring you resources to help you treat the psychological illnesses associated with childhood bullying as well as child trauma, neglect, or other symptoms related to mental trauma. Speak to friends and family and ask what they would recommend after you have a conversation with them.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to deal with mental illness linked to childhood bullying [title] Talk to a professional. [step] The first step in dealing with any type of mental illness is to speak with a professional about what you are going through. If you aren't already doing so, you should seek the help of a professional.\n\nA. Professionals like psychologists, psychiatrists, and therapists are trained to help people deal with mental illness. Their experience and training can help you manage your mental illness better.\nB. Children with mental illness often seek mental health care more often than adults. [title] Read on the child's situation.\nC. Talk to your doctor or a mental health professional to get specific information, and then take some time to talk in private. [substeps] Professional support can be a great help.\nD. They may be able to bring you resources to help you treat the psychological illnesses associated with childhood bullying as well as child trauma, neglect, or other symptoms related to mental trauma. Speak to friends and family and ask what they would recommend after you have a conversation with them.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1603,"error":null,"output":"A"}
{"question_id":"hellaswag:00004402","item_index":285,"attempt":0,"prompt_hash":"75093b7dc436a604f763ae2c5c15c087e663c70a859f8b4286ae777468d8e869","question":"[header] How to make brightly colored eyelashes [title] Brush clear mascara onto your eyelashes. [step] You will be tapping the eyeshadow onto your lashes while the mascara is still wet, so it might be a good idea to do just one eye for now. If you do both eyes now, the mascara will dry out too fast, and the eyeshadow won't stick.\n\nA. Don't worry; you won't need to keep doing it. Let it dry on both before moving on to the next step.\nB. But if you do both eyes in the same shade, you will have plenty of options, and there are different whole loads of options as many as you can pick from. You could decide on extensions in either color, or go one color at a time.\nC. [title] Sweep a soft, eyeshadow brush across the eyeshadow color you want to use. [step] Be sure to get a good amount on your brush, so that it sticks to the wet mascara.\nD. Swipe a pea-sized amount of eyeshadow on the skin under your cheekbones. Place your forefinger in the center and gently tap it downward.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to make brightly colored eyelashes [title] Brush clear mascara onto your eyelashes. [step] You will be tapping the eyeshadow onto your lashes while the mascara is still wet, so it might be a good idea to do just one eye for now. If you do both eyes now, the mascara will dry out too fast, and the eyeshadow won't stick.\n\nA. Don't worry; you won't need to keep doing it. Let it dry on both before moving on to the next step.\nB. But if you do both eyes in the same shade, you will have plenty of options, and there are different whole loads of options as many as you can pick from. You could decide on extensions in either color, or go one color at a time.\nC. [title] Sweep a soft, eyeshadow brush across the eyeshadow color you want to use. [step] Be sure to get a good amount on your brush, so that it sticks to the wet mascara.\nD. Swipe a pea-sized amount of eyeshadow on the skin under your cheekbones. Place your forefinger in the center and gently tap it downward.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1744,"error":null,"output":"C"}
{"question_id":"hellaswag:00006689","item_index":286,"attempt":0,"prompt_hash":"e66eca44572bbc181179ac925d248a02843a5e138858afef326484cec5914513","question":"[header] How to back up a computer [title] Find an appropriate storage device. [step] You will need a device which is capable of storing all of the data you need to back up. This should be at least twice the size of the hard drive you are trying to back up.\n\nA. [substeps] Vcs are common hard drives on laptops, desktops, and flash drives, but are are also very vulnerable to crashes if they are not backed up. If the hard drive is in jeopardy, choose a specific usb drive (known as the icloud drive) or newer hard drive (known as a flash drive).\nB. [title] Remove any cable on your usb device or computer. [step] If you are purchasing a new computer, you will need an inbuilt media storage device or wifi portable device.\nC. An external hard drive is the best option, and these are easy to find. [substeps] It is also possible to create a partition, if you wish to use your current computer as a backup.\nD. The device should also have an internet connection and all of the antivirus software on board. [title] Go to your device's hardware store.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to back up a computer [title] Find an appropriate storage device. [step] You will need a device which is capable of storing all of the data you need to back up. This should be at least twice the size of the hard drive you are trying to back up.\n\nA. [substeps] Vcs are common hard drives on laptops, desktops, and flash drives, but are are also very vulnerable to crashes if they are not backed up. If the hard drive is in jeopardy, choose a specific usb drive (known as the icloud drive) or newer hard drive (known as a flash drive).\nB. [title] Remove any cable on your usb device or computer. [step] If you are purchasing a new computer, you will need an inbuilt media storage device or wifi portable device.\nC. An external hard drive is the best option, and these are easy to find. [substeps] It is also possible to create a partition, if you wish to use your current computer as a backup.\nD. The device should also have an internet connection and all of the antivirus software on board. [title] Go to your device's hardware store.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":1645,"error":null,"output":"C"}
{"question_id":"hellaswag:00003293","item_index":287,"attempt":0,"prompt_hash":"8bccc7839e90da7de13495b5e5b8850b2ab76daf6d154b42d7ed5ab0bbc9e03f","question":"[header] How to know if a versace bag is real [title] Enter in the clg code. [step] All versace bags come with a built-in authenticity code known as the certilogo of clg. This code is 12-digits long and usually located on the label or hang-tag of the bag.\n\nA. It can help you to know whether the bag is real if looking inside the bag. [title] Locate the button on the side of your bag.\nB. The code will indicate whether or not the bag is authentic. [substeps] For example, it might look like this: sity 127 lb.\nC. Once you've located the code you can then go to the certilogo website and enter it in to verify authenticity. [substeps] Be aware that the clg code information must be kept with any bag that is returned.\nD. It is the most common code used to verify authenticity. Upon opening the bag, the word \" clg \" will appear next to the name of the bag.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to know if a versace bag is real [title] Enter in the clg code. [step] All versace bags come with a built-in authenticity code known as the certilogo of clg. This code is 12-digits long and usually located on the label or hang-tag of the bag.\n\nA. It can help you to know whether the bag is real if looking inside the bag. [title] Locate the button on the side of your bag.\nB. The code will indicate whether or not the bag is authentic. [substeps] For example, it might look like this: sity 127 lb.\nC. Once you've located the code you can then go to the certilogo website and enter it in to verify authenticity. [substeps] Be aware that the clg code information must be kept with any bag that is returned.\nD. It is the most common code used to verify authenticity. Upon opening the bag, the word \" clg \" will appear next to the name of the bag.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1626,"error":null,"output":"B"}
{"question_id":"hellaswag:00000474","item_index":288,"attempt":0,"prompt_hash":"25b1c79f2774ebbeb78ce381b315554cfe5ce6ab2a440c399b70c02272fd7fe1","question":"A man is seen skating around a city street while a car followed by the side. the man\n\nA. continues to walk around the area while the camera follows.\nB. skates past several scores shown in the original skaters but weaving in and out of cars and people.\nC. rides past cars at a quick pace and performs several tricks and flips.\nD. walks all the way to the end where he jumps off and is seen again sitting on the street.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\nA man is seen skating around a city street while a car followed by the side. the man\n\nA. continues to walk around the area while the camera follows.\nB. skates past several scores shown in the original skaters but weaving in and out of cars and people.\nC. rides past cars at a quick pace and performs several tricks and flips.\nD. walks all the way to the end where he jumps off and is seen again sitting on the street.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1598,"error":null,"output":"B. skate past several scores shown in the original skaters but weaving in and out of cars and people."}
{"question_id":"hellaswag:00005474","item_index":289,"attempt":0,"prompt_hash":"4ccdc7b17bbcede6bbe440576b34f863fe1842cbd3b7c97e13aa31d7b48970ef","question":"[header] How to talk a friend out of doing something [title] Invite your friend over for a private conversation. [step] It is important that your conversation not be interrupted. Think of somewhere that your friend will feel comfortable, and plan to meet there when you both have plenty of time to talk.\n\nA. Cutting the conversation short will not be an effective way to change your friend's mind. [substeps] For example, if your friend enjoys coffee, you could meet at a coffee shop on a saturday morning when you are both free.\nB. This could be a restaurant your friend usually frequents, an outdoor café that makes s'mores at home, or an area where your friend often works. [substeps] This will help give you some time to talk to your friend without interruption.\nC. [title] Make sure the planned conversation is public. [step] If your friend hangs out with another person, make sure the speaker is in a public space, like an auditorium.\nD. [title] Make your decision to meet at a time when you both get to spend quality time together. [step] Eating supper together, going bowling, getting coffee, taking the dog for a walk, etc.","prompt":"Answer the question. Reply with only the letter of the correct choice.\n\n[header] How to talk a friend out of doing something [title] Invite your friend over for a private conversation. [step] It is important that your conversation not be interrupted. Think of somewhere that your friend will feel comfortable, and plan to meet there when you both have plenty of time to talk.\n\nA. Cutting the conversation short will not be an effective way to change your friend's mind. [substeps] For example, if your friend enjoys coffee, you could meet at a coffee shop on a saturday morning when you are both free.\nB. This could be a restaurant your friend usually frequents, an outdoor café that makes s'mores at home, or an area where your friend often works. [substeps] This will help give you some time to talk to your friend without interruption.\nC. [title] Make sure the planned conversation is public. [step] If your friend hangs out with another person, make sure the speaker is in a public space, like an auditorium.\nD. [title] Make your decision to meet at a time when you both get to spend quality time together. [step] Eating supper together, going bowling, getting coffee, taking the dog for a walk, etc.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":1728,"error":null,"output":"B"}