{"question_id":"hellaswag:00001623","item_index":0,"attempt":0,"prompt_hash":"a061d3a6bb7b4ffbcc443367a8a6124f6dd0645410b770b0d7defc0509e78ac4","question":"A man has climbed a large ladder outside. he\n\nA. is using trimmers to cut and trim large trees.\nB. ropes and lures a cow into the open.\nC. is using it to pull himself up onto a platform where a car is parked.\nD. is life saving at the bottom of the ladder.","prompt":"A man has climbed a large ladder outside. he","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4403,"error":null,"output":"C"}
{"question_id":"hellaswag:00006547","item_index":1,"attempt":0,"prompt_hash":"e498ae6053dd184b2b87ec0576d8969b7c1dd1fccd7d555f78102e73d9994768","question":"[header] How to buy organic soil [title] Do your research. [step] Truly organic soil can be hard to come by since the market for organic potting media has only begun increasing in the past few years. Moreover, organic standards have also become stricter as the demand for organic goods has grown, so a soil that was once deemed organic may no longer fulfill those requirements legally.\n\nA. To survive, the best option is to research the quality of organic soil you buy and see how they relate with your home or ecosystem. [substeps] Make sure that your soil is organic.\nB. Determining what is organic will help you decide whether it would be a good idea to buy the soil you need. [substeps] There are two main kinds of organic soil: organic and non-organic.\nC. Before shopping for an organic soil, inform yourself so that you know what to expect. Traces of chemicals can be common, especially in soils containing certain types of manure, but if the amount is low enough it could still qualify as organic.\nD. [title] Buy organic potting soil instead of commercial ones. [step] Changing to commercial potting soil will reduce the amount of organic overgrowth in your field and reduce the amount of organic manure produced by your cultivars' buckers.","prompt":"[header] How to buy organic soil [title] Do your research. [step] Truly organic soil can be hard to come by since the market for organic potting media has only begun increasing in the past few years. Moreover, organic standards have also become stricter as the demand for organic goods has grown, so a soil that was once deemed organic may no longer fulfill those requirements legally.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4401,"error":null,"output":"C"}
{"question_id":"hellaswag:00008485","item_index":2,"attempt":0,"prompt_hash":"ef5ba648c703d363b959e1c7f05d1cc77ad11996d7ff44f9b177056030d6868e","question":"[header] How to burp a sleeping baby [title] Hold your baby and burp them. [step] This technique is good for babies who sleep on their stomach or who like to cuddle when they sleep. [substeps] Slowly move your baby next to your body so you do not wake them up.\n\nA. Allow your baby's head or chin to rest on your shoulder and cup their bottom to support them so they don't slip as you hold them. Place your other hand on their back and gently pat it to help them burp.\nB. You can also lower your baby's head and neck down into a deep belly position, which will help you burp your baby more easily. Gently nudge your baby at first and again until they are used to this position and begin to adjust to the feel of their neck and pillow.\nC. Hold your baby close with your hand and record the amount of air they intake and how much you burp each time before you burp them. This way, you can do it for several days to see how comfortable they will be with your hand.\nD. Then gently roll them over so they are lying on their back. Hold and burp your baby together until they become comfortable.","prompt":"[header] How to burp a sleeping baby [title] Hold your baby and burp them. [step] This technique is good for babies who sleep on their stomach or who like to cuddle when they sleep. [substeps] Slowly move your baby next to your body so you do not wake them up.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4400,"error":null,"output":"A"}
{"question_id":"hellaswag:00005294","item_index":3,"attempt":0,"prompt_hash":"7f5b2215e32097798fb7f2031326564298d8394f32986e5ec961c6a3e4132064","question":"[header] How to use a highlighter stick [title] Choose an ivory or cream highlighter shade if you have fair skin. [step] If you have a pale complexion, cream and ivory highlighters with a pearlescent or icy-silver tinge work really well. While these shades tend to look ghostly and unnatural on other complexions, they add a natural-looking glow to fair skin.\n\nA. [substeps] Look for highlighter shades that include words like \" moonbeam, \" \" ice, \" and \" crystalline. \" avoid shades that are darker than cream and ivory.\nB. [title] Try a metallic highlighter shade to go with your outfit. [step] If you have a pale complexion, you may prefer something metallic.\nC. [substeps] Ivory highlighters are quite pricey. They cost anywhere from $14-20.\nD. They make blending difficult to maintain, so get creative and test out different color highlighters. [substeps] For darker skin colors, choose a similarly color highlighter.","prompt":"[header] How to use a highlighter stick [title] Choose an ivory or cream highlighter shade if you have fair skin. [step] If you have a pale complexion, cream and ivory highlighters with a pearlescent or icy-silver tinge work really well. While these shades tend to look ghostly and unnatural on other complexions, they add a natural-looking glow to fair skin.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4403,"error":null,"output":"B"}
{"question_id":"hellaswag:00010031","item_index":4,"attempt":0,"prompt_hash":"34999dc38706771e516f9808c7423802a3cd646ccbe79eff2c3eaf930558eec7","question":"[header] How to rent a limousine [title] Decide when you'll rent the limo. [step] Early summer is the peak of the season for hiring limousines. You will probably be able to secure a better price if you hire a limo outside the months of may and june.\n\nA. You're also likely to get a better rate on weekdays. [substeps] Sunday through thursday are typically the least expensive days, while fridays are about 20% less than saturdays.\nB. If you're looking for one in july or august, you might want to be the driver. [title] Decide whether to ask friends, relatives, or strangers.\nC. [substeps] Many limousine offers large limousine packages for lower prices. However, the driver hiring a limousine for the funeral may not be the first limo you hire.\nD. A limousine is usually on the streets or across city traffic and less hectic between the ages of september and july. Saturday taxis are typically used if you live in an area where it is hot.","prompt":"[header] How to rent a limousine [title] Decide when you'll rent the limo. [step] Early summer is the peak of the season for hiring limousines. You will probably be able to secure a better price if you hire a limo outside the months of may and june.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4399,"error":null,"output":"A"}
{"question_id":"hellaswag:00005104","item_index":5,"attempt":0,"prompt_hash":"ea6b5cd4a03e3b2e9fcfc8b2d910a38c80330e030c595b7be00939133ad7c226","question":"[header] How to dip dye dark hair [title] Protect yourself and work area from bleach stains. [step] Cover your counter with newspaper. Wrap a dyeing cape or an old towel around your shoulders and put on a pair of plastic gloves.\n\nA. Lastly, make sure that you have all of your supplies laid out and ready to go. [title] Mix the bleach according to the instructions on the package.\nB. [substeps] Deep dye dark hair requires heavy amounts of bleach. For deep dye, i recommend having a shirt with dark sleeves.\nC. Wear a sturdy t-shirt and apron to protect yourself while you work. Put bleach bowls or a bucket into your sink so that you can come to an additional solution that's the same color as the dye.\nD. Bleach is caustic and it affects your hair and eyes. [substeps] You can also wear a bandanna around your neck.","prompt":"[header] How to dip dye dark hair [title] Protect yourself and work area from bleach stains. [step] Cover your counter with newspaper. Wrap a dyeing cape or an old towel around your shoulders and put on a pair of plastic gloves.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4404,"error":null,"output":"A"}
{"question_id":"hellaswag:00005596","item_index":6,"attempt":0,"prompt_hash":"fb6cc22c3d0a51ad95ed274f91eabcc74442507726c72314032aa3890bc3d5d8","question":"[header] How to hire syrian refugees [title] Find your local resettlement office. [step] Refugees admitted into a new country are eligible to work immediately. Government programs assist refugees in finding employment as part of their resettlement programs.\n\nA. If you identify yourself as a refugee, you must work as a refugee in the country. [title] Complete any required occupations (such as a position, career, or order) before applying for a job.\nB. [substeps] Use the job search tools at the resource website: https: // www.resettlement-cepus.gov/workspace/guild-acanees. Cfm look on your city or town's website for a list of resettlement department locations.\nC. [substeps] In the united states, you can find your local resettlement office by visiting https: // www.acf.hhs.gov/orr/state-programs-annual-overview. Click on your state on the map, or select your state from the drop-down menu below the map to get a list of local offices and contact information.\nD. The national guard service reacts positively to refugees seeking work as part of their resettlement program, as they provide a much more productive working environment. [substeps] After visiting your local resettlement office, locate the website by typing \" resettlement program \" and your city and state into a search engine.","prompt":"[header] How to hire syrian refugees [title] Find your local resettlement office. [step] Refugees admitted into a new country are eligible to work immediately. Government programs assist refugees in finding employment as part of their resettlement programs.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4627,"error":null,"output":"C"}
{"question_id":"hellaswag:00001187","item_index":7,"attempt":0,"prompt_hash":"7d9da675c4e03c5bdba27c9ae0069474735b35ec4d796375593b176ededf5282","question":"The woman stands still for a short while in her blue/white outfit. Then she lifts the heavy yellow weight and someone who's watching her shows peremptory by yelling. she\n\nA. picks it back up and lifts it again, guzzling the last of the shiraz.\nB. drops the weight and the face of female trainer starts into the screen.\nC. holds it up for a good second and then drops it back down to the ground.\nD. slowly lifts the weight up she long the overweight weight over her head, then drops the weight to the ground.","prompt":"The woman stands still for a short while in her blue/white outfit. Then she lifts the heavy yellow weight and someone who's watching her shows peremptory by yelling. she","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4406,"error":null,"output":"D"}
{"question_id":"hellaswag:00009657","item_index":8,"attempt":0,"prompt_hash":"db239160640e14a6c79794a3049f085ee004658c5b97690bab5e6d38cf34a65c","question":"[header] How to tell your parents you're flunking out [title] Help around the house. [step] Offer to do small things like cleaning the dishes, setting the table, washing the car...they'll be suspicious at first, but chill. If you flash a smile at them, they'll freak out even more.\n\nA. [title] If your parents see you're kind or decent but have homework the night before, ask them if you could join them at their house around 11, or before their curfew. [step] This shows them that you can be organised and learn to be a good citizen.\nB. When you let your parents know you can handle any all the chores, they'll get a sense that you've this all figured out. [substeps] Send them a little pep talk in the morning.\nC. If you make a mess while doing dishes, congratulate yourself for being able to do it without them noticing. [substeps] Try putting a small potted plant, like a cranberry, right on the table.\nD. Remember: known teenage trouble-makers are not supposed to smile. [title] Be nicer to them, so they can see that you're at least trying to be different.","prompt":"[header] How to tell your parents you're flunking out [title] Help around the house. [step] Offer to do small things like cleaning the dishes, setting the table, washing the car...they'll be suspicious at first, but chill. If you flash a smile at them, they'll freak out even more.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4403,"error":null,"output":"D"}
{"question_id":"hellaswag:00002826","item_index":9,"attempt":0,"prompt_hash":"bafdaba8bb19bbecd4ac881c7ab5ba9eb0cd390c3f1ed22de2528b3a55f5d1f7","question":"A black screen appears again with red and white words that say \"other similar exercises on the same idea: focus on the low point of the implement trajectory. A boy is now standing on the concrete circle and he's practicing his swings with a shirt. a man\n\nA. is now indoors and he's practicing his swings with a golf club.\nB. wearing a neatly pressed blue shirt picks up the pace and continues to swing at the swing.\nC. is then shown teaching the learned to swing the tried and fail swing and he is shown farther back behind in the weightlifting attempt, swinging the lot of his arms, doing more over and over.\nD. shows him how to properly demonstrates and how to properly get the swing to move.","prompt":"A black screen appears again with red and white words that say \"other similar exercises on the same idea: focus on the low point of the implement trajectory. A boy is now standing on the concrete circle and he's practicing his swings with a shirt. a man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4406,"error":null,"output":"A"}
{"question_id":"hellaswag:00004414","item_index":10,"attempt":0,"prompt_hash":"2b939b93aeff4c0d75bd7c15e43d0fd238f4e71f47783f19cd9c04a544d12d63","question":"[header] How to bottle feed babies [title] Gather your equipment. [step] Before you are able to bottle feed you will need to purchase bottles, nipples, a bottle brush, and a sterilizer. A sterilizer is optional if you have a dishwasher that has a steam setting.\n\nA. If your dishwasher does not have a steam setting, then you will need to purchase a pair of sterilized cotton balls and a clean syringe. Also remember that you will be using this brush to administer weaning.\nB. There are many types of bottles and nipples you can choose from. Some babies take to only one type of nipple and may you go through many nipples trying to figure out which one they like and latch on too.\nC. If not, your access to the sterilizer should be limited, or restricted. [substeps] Sterilizers can be purchased for \" non-food \" equipment, but also for household implements like a toilet.\nD. [title] Purchase a blanket or cloth diaper. [step] If you are concerned about babies' bodies getting wet, you can sterilize a blanket or cloth diaper for a younger baby.","prompt":"[header] How to bottle feed babies [title] Gather your equipment. [step] Before you are able to bottle feed you will need to purchase bottles, nipples, a bottle brush, and a sterilizer. A sterilizer is optional if you have a dishwasher that has a steam setting.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4403,"error":null,"output":"B"}
{"question_id":"hellaswag:00008541","item_index":11,"attempt":0,"prompt_hash":"4247ea85e3f27674344d297c87b56d5d4215562929169ed21a95f585ac9ac696","question":"[header] How to prevent heartworm disease in cats [title] Give your cat preventative medication. [step] Like most diseases, prevention is the safest and most cost effective measure. Cats can be given once a month preventative medication that kills the heartworms.\n\nA. These preventative medications are available from your veterinarian in the form of a prescription. [substeps] In year round warm weather regions, your cat should be given this medication year round.\nB. [substeps] In the united states, heartworm preventative medications like prednisolone are available on prescription. The fda recommends that cats drink about 8-10 ounces of ibuprofen per day, as needed, three days a week.\nC. In severe cases, the cat's immune system will try to flush the medication from their system. [substeps] Dosage is less drastic if your cat has it.\nD. It's usually given only once your cat is in the care of a vet, though it can be given again and again. [substeps] The veterinarian will assess your cat's conditions, although it may take several weeks to figure out if it needs treatment.","prompt":"[header] How to prevent heartworm disease in cats [title] Give your cat preventative medication. [step] Like most diseases, prevention is the safest and most cost effective measure. Cats can be given once a month preventative medication that kills the heartworms.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4401,"error":null,"output":"A"}
{"question_id":"hellaswag:00000389","item_index":12,"attempt":0,"prompt_hash":"2802a406f49ace6d0605bf3f9f59d4cdca59564884c0ecff25938170517ef9dc","question":"A young man is seen playing drums in front of an audience. other people\n\nA. are seen speaking to the camera as well as running into the audience.\nB. are watching as he continues to play.\nC. are walking by back and fourth.\nD. are shown playing guitar while the man plays more drums.","prompt":"A young man is seen playing drums in front of an audience. other people","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4406,"error":null,"output":"B"}
{"question_id":"hellaswag:00004765","item_index":13,"attempt":0,"prompt_hash":"c1f73ca53d031c671c3f02351b6f2091296479a1d66a4b5d0d555f9ba422b9b5","question":"[header] How to generate employee referrals [title] Build a happy corporate culture. [step] Have good core values. Advertise on social media to ensure that external candidates get to know about your company.\n\nA. You should aim to offer quality, friendly jobs for your employees. This will ensure that the new hires are unlikely to suffer severance at any point.\nB. Voters should become familiar with your employees and your current leadership styles. They may be hesitant to hire you because they are looking for employees already in corporate roles.\nC. [title] Pay attention to current employees' experience. [step] Make your current employees satisfied and they will be your company's effective ambassadors.\nD. Review the company website to find out more about its values so that you can motivate them into voting the right way. Networking is another big motivating factor for your company.","prompt":"[header] How to generate employee referrals [title] Build a happy corporate culture. [step] Have good core values. Advertise on social media to ensure that external candidates get to know about your company.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":8089,"error":null,"output":"C"}
{"question_id":"hellaswag:00008268","item_index":14,"attempt":0,"prompt_hash":"d96daca0443407a49cc8433ca53d6063eb27b5d1660107d5d1573cd6b2033816","question":"[header] How to discreetly breastfeed your baby on the go [title] Wear nursing clothing. [step] Nursing tops can be purchased at maternity stores and most major clothing retailers. These tops have hidden openings that allow you to easily feed your baby.\n\nA. The lining on these dresses is designed to allow you to access the breast once your baby has grown accustomed to you. To remove the lining, simply slide it off and then gently remove it.\nB. [substeps] Wear one that covers the breasts, but does not cover them completely. Many maternity tops are available in mesh or sheer fabrics.\nC. You can get a variety of items such as tops, bras, and dresses. [substeps] Nursing bras that are made like sports bras are usually better.\nD. When your baby is traveling, these have tiny baby items that you can remove and show your baby. [substeps] You can also move to a place where you will not be bothered by baby items.","prompt":"[header] How to discreetly breastfeed your baby on the go [title] Wear nursing clothing. [step] Nursing tops can be purchased at maternity stores and most major clothing retailers. These tops have hidden openings that allow you to easily feed your baby.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4398,"error":null,"output":"A"}
{"question_id":"hellaswag:00009500","item_index":15,"attempt":0,"prompt_hash":"be08f7ca6e2ca7618a328af315455e0ebdf540c7a7393c132b3aff01c63231cf","question":"[header] How to track your gas mileage with fuelfrog [title] Go to fuelfrog to sign up for a free account. [title] Go to the next page. [step] It will take you to your fuelfrog dashboard.\n\nA. [title] Near your dashboard is a button that says \" change fuel. \" click this button and enter your current fuel mileage.\nB. This is where you can enter in the information on the website. It works best if you can enter two or three fill-ups.\nC. [title] Go ahead and pay the upfront fee. [step] Don't worry, it's slightly higher than the online fees.\nD. [title] Click on \" new step \" at the top right side of the dashboard. [title] Look where the steps on the dashboard have appeared.","prompt":"[header] How to track your gas mileage with fuelfrog [title] Go to fuelfrog to sign up for a free account. [title] Go to the next page. [step] It will take you to your fuelfrog dashboard.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4405,"error":null,"output":"A"}
{"question_id":"hellaswag:00005925","item_index":16,"attempt":0,"prompt_hash":"d3796a64af8859706a3722af2817e7b791d141401d96cf0c82c40a3627bfd010","question":"[header] How to expunge an arrest [title] Determine what options for expungement are available in your state. [step] Not all states allow criminal records to be expunged, and those that do may only allow it in specific instances. [substeps] You must follow the expungement rules and procedures of the state where you were arrested, even if you are a resident of another state.\n\nA. Even if you have already been expunged, there is still a possibility that you have been convicted of a felony and you have not had the conviction expunged. [title] Familiarize yourself with your rights as a convicted felon.\nB. Some states allow for \" sealing \" rather than \" expungement \" specifically. If a criminal record is sealed, it's hidden from the general public but still exists on your record as a whole.\nC. Many states do not allow you to expunge a misdemeanor or offense. You also must not expunge a dui if the person you arrested had a dui conviction within the past two years.\nD. The expungement rules in california, alaska, hawaii, and the u.s. vary greatly depending on your location in which state you live.","prompt":"[header] How to expunge an arrest [title] Determine what options for expungement are available in your state. [step] Not all states allow criminal records to be expunged, and those that do may only allow it in specific instances. [substeps] You must follow the expungement rules and procedures of the state where you were arrested, even if you are a resident of another state.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4402,"error":null,"output":"A"}
{"question_id":"hellaswag:00008561","item_index":17,"attempt":0,"prompt_hash":"e26e2a766c285c2a33c54644c87c56866a55beaa6cebbd83a3b8e684bd7ca909","question":"[header] How to feed a family of 4 on a budget [title] Start with the basics : [step] Home grown vegetables and herbs. You don't have to grow a wide range but some staples such as parsley, chives, rosemary, cherry tomatoes, lettuce, spring onions etc all help. To buy herbs can cost $2-$3 a bunch and that is more than it costs to buy the seedlings.\n\nA. In addition, you can buy these products at shops, websites etc as well. And check your local grocery store for whole foods.\nB. Put them on your windowsill, in pots, on the desk. [title] Try to keep breakfast to a basic level but vary it at least twice a week.\nC. [title] Plant the herbs in their first hole with a shovel. [step] Be sure to fertilize the soil first before planting.\nD. Choose the quantity that will satisfy the family. If you have a large garden at home you may want a smaller garden and a medium sized garden.","prompt":"[header] How to feed a family of 4 on a budget [title] Start with the basics : [step] Home grown vegetables and herbs. You don't have to grow a wide range but some staples such as parsley, chives, rosemary, cherry tomatoes, lettuce, spring onions etc all help. To buy herbs can cost $2-$3 a bunch and that is more than it costs to buy the seedlings.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4403,"error":null,"output":"D"}
{"question_id":"hellaswag:00005927","item_index":18,"attempt":0,"prompt_hash":"1df107b97c9d91467e225a16226efb5f094ca766835f2ecc1ad6e0addba9dd81","question":"[header] How to prevent ascarids in dogs [title] Find the proper deworming medication for your dog. [step] Feel free to ask your vet for suggestions. There are a huge number of effective worming products available on the market.\n\nA. Key to worming success is checking the package to see exactly which parasites the product is effective against. You are looking for effectiveness against ascarids species, including toxocara and toxascaris.\nB. [substeps] Over the counter deworming treatments alone are not likely to reduce the symptoms of ascarids in dogs. If you have tried other types of methods for treating these various ailments, you should try serious or potentially dangerous treatment options first.\nC. In order to make sure that your dog is getting enough pain relief, you might ask your vet for recommendations. Stay within the recommended dosage as most medications are specially formulated for dogs.\nD. The best advice will come from a vet who can specifically recommend an appropriate medication for your dog. [substeps] You also might want to visit your veterinarian for recommendations on the correct dosages.","prompt":"[header] How to prevent ascarids in dogs [title] Find the proper deworming medication for your dog. [step] Feel free to ask your vet for suggestions. There are a huge number of effective worming products available on the market.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4403,"error":null,"output":"C"}
{"question_id":"hellaswag:00002608","item_index":19,"attempt":0,"prompt_hash":"52a4db1d6a44cf5490d8aa9d300fe7ac236dce4fd59324fc697ce332df45c9d1","question":"An african american female takes off running and begins to attempt a high jump but fails when the bar drops and walks off. Another female shows up and tries her third attempt and the same thing happens. after\n\nA. , another female with a large assortment of tattoos tries and she clears the bar and comes in first place on her first attempt and a replay is shown.\nB. , the girl makes several unsuccessful attempts and finally has her third attempt.\nC. over 400 attempts high jumps her young girl stops and begins to walk off the bar.\nD. , a second female throws six balls of green paint and heads towards shots as the males walk away.","prompt":"An african american female takes off running and begins to attempt a high jump but fails when the bar drops and walks off. Another female shows up and tries her third attempt and the same thing happens. after","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4403,"error":null,"output":"A"}
{"question_id":"hellaswag:00001739","item_index":20,"attempt":0,"prompt_hash":"ef82b6c3a9b9fb0b2d581a05218f567353a12d488c560f9ad3e22b5e17532db0","question":"Some people are standing by an ice cream truck talking about getting some ice cream. They walk up to the truck window and get their ice cream. as they\n\nA. 're down, they have a selfie stick.\nB. eat the ice cream, they talk among each other.\nC. 're eating they begin to play shuffleboard for fun.\nD. are walking away, a dog with ice cream on its front legs gets through.","prompt":"Some people are standing by an ice cream truck talking about getting some ice cream. They walk up to the truck window and get their ice cream. as they","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4404,"error":null,"output":"B"}
{"question_id":"hellaswag:00008369","item_index":21,"attempt":0,"prompt_hash":"2f229537ea5048b99295bbd4760634c50eac95b086794e58576cbef10a3be031","question":"[header] How to get children to save money [title] Start with coupons. [step] Beginning as early as 3-5 years of age, children can start to learn the rudiments of saving money and using coupons is a great way to do that. They can earn coupons for things like helping, for being cooperative, for clearing their plate-the possibilities are endless.\n\nA. [title] Make sure when they are hungry try to teach them how to save. [step] If they are hungry, offer to help them.\nB. If you are going to make a habit of saving money every week, try : [substeps] Introducing coupons or savings accounts at home. Checking your checks for money each month to find out if you have enough.\nC. They can then save those coupons and exchange them for fun things to do. This is a great first lesson in saving.\nD. However, it is important that you start with the most affordable means available to give children money. [substeps] Purchase coupons from your local grocery or supermarket.","prompt":"[header] How to get children to save money [title] Start with coupons. [step] Beginning as early as 3-5 years of age, children can start to learn the rudiments of saving money and using coupons is a great way to do that. They can earn coupons for things like helping, for being cooperative, for clearing their plate-the possibilities are endless.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4402,"error":null,"output":"C"}
{"question_id":"hellaswag:00005184","item_index":22,"attempt":0,"prompt_hash":"07667799c1f877396c20901c6f8bf607ff213e8dacf7abec0659fd95caeb0062","question":"[header] How to get rid of wasps naturally [title] Set up a big bucket of water. [step] Put it close to the wasp nest on the ground. Obtain a cloth bag with a tie at the end and a large rock.\n\nA. The rock needs to be large enough for a shoebox. [substeps] Cut the rock to its maximum size.\nB. Water the rock and cover your bucket up with small stones, pebbles, gravel, and other rocks and create a tent. [substeps] You can also use paper or plastic cup holders to hold water.\nC. There is a large hole in the bag, so your target wasps will not fly any farther away. [title] Pick up the rock close to the nest.\nD. [substeps] This cloth bag should be able to be tied tightly closed and should be big enough to hold the wasp nest. If you do not have a suitable cloth bag, you should pick another method to get rid of the wasps.","prompt":"[header] How to get rid of wasps naturally [title] Set up a big bucket of water. [step] Put it close to the wasp nest on the ground. Obtain a cloth bag with a tie at the end and a large rock.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4399,"error":null,"output":"D"}
{"question_id":"hellaswag:00000295","item_index":23,"attempt":0,"prompt_hash":"9300c0d5a59b47d2c0968ff812ded8e0f737be7a5f5590cde45d6bf977e9e2a1","question":"A man is seen skiing behind a boat. he\n\nA. skis all around part of the water road while people distract him.\nB. pushes himself on the water while the boat speeds up.\nC. holds on tight as he is pulled through the water.\nD. is seen giving instructions as well as a dog.","prompt":"A man is seen skiing behind a boat. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4404,"error":null,"output":"C"}
{"question_id":"hellaswag:00005526","item_index":24,"attempt":0,"prompt_hash":"aa73830c909aa1f2679e263152a77259cd024efc390810d18ed9e5aad5f8ff25","question":"[header] How to make english muffins or bread [title] Make a sourdough starter. [step] Combine 2 1/2 cups of flour, one packet of yeast, and 2 cups of warm (90-110f) water in at least a 8 cup bowl. Mix with a spoon until it is a smooth batter.\n\nA. Spread 1 cup extra-sweet and/or spring-loaded yeast flour into the dough. Use your fingers to form a dough ball and set it aside, laying it out on clean, cool, and level surface.\nB. [substeps] Make sure the batter isn't too thick, as it could clog the muffin pans on your way home. [title] Stir the yeast and condensed milk until the mixture is sticky.\nC. This will sit and \" cook \" for at least 4 hours during which time it will increase in volume a lot (hence the big bowl). Cover with some plastic wrap or a cloth and leave in a warm place (room temperature is fine).\nD. [substeps] You can choose your type of sourdough starter mix, as well. You can use a clay mix, a matte one, or something else.","prompt":"[header] How to make english muffins or bread [title] Make a sourdough starter. [step] Combine 2 1/2 cups of flour, one packet of yeast, and 2 cups of warm (90-110f) water in at least a 8 cup bowl. Mix with a spoon until it is a smooth batter.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4629,"error":null,"output":"C"}
{"question_id":"hellaswag:00006056","item_index":25,"attempt":0,"prompt_hash":"6c02f28bccd26025258bb17c554cd003f643c273d15f785094eb3e8e439cc3ad","question":"[header] How to pamper yourself after a bad day (girls ) [title] Try to finish your homework at school. [step] This is so that you don't have to worry about it later. Finish any other work as soon as you get home.\n\nA. If you choose to do your work at home, you are limiting your time to relax. If it is simply impossible to finish your homework at school, do it at home--it may limit your time to pamper yourself, but at least you won't stress yourself out even more.\nB. [title] Pick up a pillow or a book and put it in front of you with your feet propped on them, crossing your legs. [step] Put your head on it and let it stay there for a while.\nC. [title] If you must finish other tasks at school, don't fold your clothes and stuff until it's past your bedtime. [step] If you must finish homework at school before bed, do it in your own bed as opposed to your usual one.\nD. Don't complete other activities while feeling bad. The sooner you finish, the easier it will be to recover from a bad day.","prompt":"[header] How to pamper yourself after a bad day (girls ) [title] Try to finish your homework at school. [step] This is so that you don't have to worry about it later. Finish any other work as soon as you get home.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":8089,"error":null,"output":"A"}
{"question_id":"hellaswag:00009765","item_index":26,"attempt":0,"prompt_hash":"7e42356ddba63527bdda3095c3a823fe4293557790569bb6038590ca46c222c6","question":"[header] How to apply hippie makeup [title] Sweep a neutral eyeshadow across your upper eyelids. [step] For this look, you want to look like the prettiest, most natural version of yourself. Pick an eyeshadow color that's close to your natural skin color and use a brush or your fingers to apply it all the way up to the crease.\n\nA. As you apply it, go slow and avoid blending it into your skin. Instead, apply a small amount at a time and smooth your strokes to blend the eyeshadow into your skin.\nB. [substeps] By scrunching your lid lid up into your forefinger, you can get the final crease even more defined. [title] Blend your eyelid shadow into the crease.\nC. [substeps] If you prefer, you could also use an eyeshadow with golden tones, but avoid anything too glittery so you'll look as natural as possible. [title] Line your eyes with a brown pencil.\nD. Use a thin gray or rose eyeshadow around the crease, too. [substeps] You can also use eye shadows to hide dark spots.","prompt":"[header] How to apply hippie makeup [title] Sweep a neutral eyeshadow across your upper eyelids. [step] For this look, you want to look like the prettiest, most natural version of yourself. Pick an eyeshadow color that's close to your natural skin color and use a brush or your fingers to apply it all the way up to the crease.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":8088,"error":null,"output":"C"}
{"question_id":"hellaswag:00007782","item_index":27,"attempt":0,"prompt_hash":"9cacdca059cd04e3aec5d4a59309966978121b136c1f11baa956956bf44ad0cf","question":"[header] How to make your lips pink [title] Use a lip brightening scrub. [step] Dull, colorless lips are usually the result of dead skin buildup. Many people regularly exfoliate their face, but forget their lips.\n\nA. To prevent your lips from pulling away from your face, consider using an exfoliating scrub on them before putting on lip liner. It's the easiest lip scrub and water to use, but it won't completely cover your lips.\nB. Scrubbing your lips not only removes the dead skin to reveal new skin beneath, it increases circulation to bring more color to your lips. Try a homemade brightening scrub using the following ingredients : [substeps] 1 teaspoon sugar 1 teaspoon honey 1 teaspoon lemon juice a few drops of rose water [title] Try an exfoliating brush.\nC. Consider using some of this astringent to increase the quality of your lips. [substeps] Note that this process is not required for your lips.\nD. Once your lips are pink, you can rock it with a bit of lip gloss or lip gloss. [substeps] Rub the area around your lips with clean fingers or a clean toothbrush to get the result that you want.","prompt":"[header] How to make your lips pink [title] Use a lip brightening scrub. [step] Dull, colorless lips are usually the result of dead skin buildup. Many people regularly exfoliate their face, but forget their lips.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4398,"error":null,"output":"B"}
{"question_id":"hellaswag:00002711","item_index":28,"attempt":0,"prompt_hash":"2f391237edee8fff3f6baa3335c2f8c7f828bb085c819fe84d4458ce604b5670","question":"A view of a vehicle loaded with skateboards and skateboarding equipment. Another view of the skateboards lined up on the street in the upright position. spectators\n\nA. are seen crawling around a skateboard on the ground.\nB. sit at a sidewalk in the background.\nC. watch from the street curb while a group of skateboarders take turns skating down hill and making a sharp spin stop at a designated spot.\nD. zoom in on the objects as the top of the vehicle lifts up.","prompt":"A view of a vehicle loaded with skateboards and skateboarding equipment. Another view of the skateboards lined up on the street in the upright position. spectators","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4404,"error":null,"output":"B"}
{"question_id":"hellaswag:00001970","item_index":29,"attempt":0,"prompt_hash":"825a16f8c99042b6ac5a59adfe132509f52e2bcad4e6f6e62835f4eb65818bbd","question":"Two men plays foosball in an arena with two referees standing near by watching the fame. we\n\nA. see the gopro before the contestants are judged.\nB. see the bleacher an spectators.\nC. return to the man and woman, still playing foosball on the bench.\nD. see a replay of the player.","prompt":"Two men plays foosball in an arena with two referees standing near by watching the fame. we","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4625,"error":null,"output":"D"}
{"question_id":"hellaswag:00007348","item_index":30,"attempt":0,"prompt_hash":"f0b6c95d31da2956c838a731a821cfefb004d92723947d8b3286ec46d84df195","question":"[header] How to get in touch with your inner pisces [title] Ditch the order. [step] If you have a job or calling in life that requires a lot of order for you, it can be stifling. You need to whip up a little chaos.\n\nA. Start with a messy desk or brighter clothes. Perhaps this will surprise others but you must let the real you shine through.\nB. [substeps] Your server needs to be sufficiently chaotic for them to stand up for themselves. If they can't get it (or are being too lazy), they're easy targets.\nC. Don't just' run' everywhere. Show up at the office and lie low, dim the lights a bit, make faces and drop the secret to the boss.\nD. [substeps] Think of some way to make things easier for yourself by doing only what needs to be done. If it's your job, you can try working at it.","prompt":"[header] How to get in touch with your inner pisces [title] Ditch the order. [step] If you have a job or calling in life that requires a lot of order for you, it can be stifling. You need to whip up a little chaos.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":8086,"error":null,"output":"D"}
{"question_id":"hellaswag:00000376","item_index":31,"attempt":0,"prompt_hash":"3190def975b7a7c03a7ad78f21cce8c77b47457e30c035fa2c3b8455fb9fb3c8","question":"Two little kids are eating an ice cream cone at mcdonalds. They are licking and so happy to have ice cream. the little girl\n\nA. opens her mouth and a drop of eat the ice cream all over her face.\nB. cut off an ice cream cone to get some cream near the table.\nC. is playing peek a boo with someone and both of their faces have ice cream on them.\nD. gives a being an ice cream cone.","prompt":"Two little kids are eating an ice cream cone at mcdonalds. They are licking and so happy to have ice cream. the little girl","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4405,"error":null,"output":"C"}
{"question_id":"hellaswag:00007201","item_index":32,"attempt":0,"prompt_hash":"27230495ba2b78419d31ec6480e0945e1c5932b46c099c4352acc2702f41f6b1","question":"[header] How to do a plaited wraparound bun [title] Brush your hair until there are no knots. [step] This works best with fresh hair. [title] Separate the hair into three sections.\n\nA. [step] Repeat this a few times before releasing it. [title] Part the hair to the right of the part on the left side of your head.\nB. [step] Have one section down the middle, one to the left side and one to the right side. Tie these three sections up with a hair tie.\nC. [step] Take the first section, and brush the other hair to separate it into smaller sections. [substeps] Try to wet your hair very well.\nD. [step] Use your fingers or a comb to part each hair horizontally from left to right, and tie the ends tightly. [title] Split the right section into a half-up, half-down ponytail.","prompt":"[header] How to do a plaited wraparound bun [title] Brush your hair until there are no knots. [step] This works best with fresh hair. [title] Separate the hair into three sections.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6264,"error":null,"output":"B"}
{"question_id":"hellaswag:00010004","item_index":33,"attempt":0,"prompt_hash":"9619ae419da4234de57369a4d3d9f9b774aa37441c4715a576791848d599411f","question":"[header] How to get a nice tan [title] Consider simply lying out in direct sunlight. [step] Tanning naturally is safer than tanning beds, and it will give you the most natural results. [title] Find a way to pass the time.\n\nA. [step] Sleep in any position that makes you happy, and gently pull your head up to wash your face. A t-shirt with your favorite cartoon characters is best, but tanning beds can also help you wake up soon after being in the sun.\nB. [step] No matter what type of tanning session you can do, it might be helpful to leave your house altogether to aim for longer sessions. To avoid looking like a freelancer you can usually do one of the following.\nC. [step] You could use this opportunity to watch a game show, listen to music, or read a book. One of the best habits in tanning, is to find a way to relax.\nD. [step] Sleeping is good because it causes the time to pass faster, while reading a book blocks some of the sunlight while on your back. Carefully time a nap, say an hour to an hour and a half, and set an alarm to wake up after that time has passed.","prompt":"[header] How to get a nice tan [title] Consider simply lying out in direct sunlight. [step] Tanning naturally is safer than tanning beds, and it will give you the most natural results. [title] Find a way to pass the time.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6257,"error":null,"output":"C"}
{"question_id":"hellaswag:00002956","item_index":34,"attempt":0,"prompt_hash":"9bf49400aaad16ef4b86c4a4aad95f7ec211168dad3ee21f5a40b870b9cd7b54","question":"A man practices a long jump inside of an indoor gym using a interior sandbox for landing and a drawn race track isle. another man\n\nA. practices hurling himself backward over a pole onto a gym mat inside of the gym.\nB. presents it but it isn't a race and he starts a slow one.\nC. is shown practicing long jump over obstacles and metal gears while another sets aside to perform higher jumps in preparation.\nD. runs and throws himself into the sand pit to land upright in a long jump.","prompt":"A man practices a long jump inside of an indoor gym using a interior sandbox for landing and a drawn race track isle. another man","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6262,"error":null,"output":"D"}
{"question_id":"hellaswag:00005901","item_index":35,"attempt":0,"prompt_hash":"a73038e58cea8709e56d7a29463fd922bca3144a432049e2623e4ec0bf6bc23b","question":"[header] How to wear wedge sneakers [title] Wear skinny jeans. [step] Wedge sneakers are large and a little clunky, but they can still look stylish as long as the rest of the leg looks slender and long. To that end, when wearing jeans, you should opt for a sleek pair of skinny jeans that hug your leg over classic styles like straight-leg or boot-cut.\n\nA. [substeps] Slimmer jeans are a good choice for any occasion because they lift more leg than converse or corduroys. Classic cut jeans work well for both meet-and-greets and nautical casual.\nB. [substeps] To keep the length of your leg, wear thick-soled sneakers over nikes or anything equally boxy. [substeps] To keep the grip tight, try pairing wedge sneakers with high heeled slippers or ankle boots.\nC. Create an even narrower line by tucking the jeans into the sneakers. [title] Throw on a pair of leggings.\nD. The wider the leg all around, the better-height you'll be tall. [substeps] Wedge sneakers come in a number of different shapes and sizes.","prompt":"[header] How to wear wedge sneakers [title] Wear skinny jeans. [step] Wedge sneakers are large and a little clunky, but they can still look stylish as long as the rest of the leg looks slender and long. To that end, when wearing jeans, you should opt for a sleek pair of skinny jeans that hug your leg over classic styles like straight-leg or boot-cut.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6257,"error":null,"output":"C"}
{"question_id":"hellaswag:00004415","item_index":36,"attempt":0,"prompt_hash":"5768fd581e0c4a60558c68408e19bc0f1f0fe24e7fb573451c11946398dff45c","question":"[header] How to improve digestive health with fermented foods [title] Eat fermented beans. [step] Studies have linked fermented beans like tempeh (fermented soybeans) with improvements in bowel and menopausal health. Tempeh may also protect your heart against disease and protect you from some cancers.\n\nA. Fermented beans may also reduce inflammation of the esophagus. [title] Introduce instant fermented vinegar in your diet.\nB. Fermented beans contain beneficial bacteria that heal liver and kidney problems. However, they are not regulated and should not be ingested with fermented beverages.\nC. [substeps] Fermented beans, such as tempeh, are also known as purnate beans. Women should eat four to eight grams of fermented beans per day.\nD. More research is needed to determine a daily recommended intake, but you should start by just including tempeh in your diet. [substeps] Tempeh is also a complete protein which means it has all the amino acids your body needs to make more protein.","prompt":"[header] How to improve digestive health with fermented foods [title] Eat fermented beans. [step] Studies have linked fermented beans like tempeh (fermented soybeans) with improvements in bowel and menopausal health. Tempeh may also protect your heart against disease and protect you from some cancers.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6256,"error":null,"output":"D"}
{"question_id":"hellaswag:00000519","item_index":37,"attempt":0,"prompt_hash":"4aca31d840fdc5cacb62fcd80690f2f0ddab25a810fe36fd2a4fb2e19510771d","question":"The man spins the blindfolded woman around while she's holding the bat in her hand. but\n\nA. then she starts swinging the bat and the man moves out of the way.\nB. instead of swinging her bat, the woman swings on the man's head.\nC. the woman has other bat and it hits the man right in the face.\nD. instead of using the bat to hit the man, he steps on the man's foot and she lets go that spins the girl around so she hits the man.","prompt":"The man spins the blindfolded woman around while she's holding the bat in her hand. but","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6259,"error":null,"output":"A"}
{"question_id":"hellaswag:00008738","item_index":38,"attempt":0,"prompt_hash":"11c10044709569bfb90a689ffafd9895fff6d3e8088326fd1f006df681b7aaf0","question":"[header] How to prove police misconduct [title] Arrange for release on bail if you've been detained. [step] It is a lot easier to plan your defense on the outside than it is behind bars. If you were appointed a criminal defense attorney, work with them to obtain your release.\n\nA. [substeps] Keep in mind you often get a bad rap on the side of the dollar based on the circumstances of your case. For example, you might have been charged with a felony that involved reckless driving.\nB. [substeps] It's possible to be released on your own recognizance. This means you simply sign a paper agreeing to show up for all your court dates and don't have to pay any money.\nC. [substeps] If you're arrested for physical assault, you may be able to negotiate an agreement with the judge to release you from the charges. Keep in mind that many courts now don't allow unsupervised arrest.\nD. At at this stage, you can begin negotiations with the prosecutor and/or bail bondsman. [title] Determine the minimum bail amount required for you.","prompt":"[header] How to prove police misconduct [title] Arrange for release on bail if you've been detained. [step] It is a lot easier to plan your defense on the outside than it is behind bars. If you were appointed a criminal defense attorney, work with them to obtain your release.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6256,"error":null,"output":"B"}
{"question_id":"hellaswag:00004231","item_index":39,"attempt":0,"prompt_hash":"4750051f5518d1a1169687d3d5b427798d7bec8e681937e8cc219200340ea7a1","question":"[header] How to tell if your baby has colic [title] Prepare yourself for intense bouts of crying. [step] A colicky baby wails loudly, and the cries often start without warning and come in waves. You will likely find yourself unable to comfort your baby during these crying fits.\n\nA. As you move toward your baby's enrichment for the first time, you will likely be concerned that he or she is sick or injured. If you ignore the baby's attempts to calm yourself, you might inadvertently injure him or her.\nB. It's always best to buy a bottle or syringe and prepare yourself beforehand for large (and potentially deadly) bouts of crying. [substeps] During any infant's initial crying session, you need to be prepared to take measures to accommodate their screaming.\nC. When these episodes begin, make an effort to contact the parents and speak soothingly to them. Make sure to always keep an ear handy during these bouts of crying.\nD. [title] Watch your baby's body language. [step] Many babies suffering from colic stretch their legs out or pull their feet up under their bodies, as though trying to ease a cramp.","prompt":"[header] How to tell if your baby has colic [title] Prepare yourself for intense bouts of crying. [step] A colicky baby wails loudly, and the cries often start without warning and come in waves. You will likely find yourself unable to comfort your baby during these crying fits.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6254,"error":null,"output":"D"}
{"question_id":"hellaswag:00001221","item_index":40,"attempt":0,"prompt_hash":"316b707f150520d477a18860dc057309df4ffd56e0cb33ba7aadfaa1d051cc2b","question":"A person is shown in various shots close up as well as leaves and him stretching. The man is then seen running through a park while looking off into the distance. the man\n\nA. runs shirtless along an empty trail and jumps into a pole vault.\nB. continues running and ends with text across the screen.\nC. then jumps into water and uses a scraper to clear all the grass from the sides.\nD. gets up and climbs back on after falling down on the ground.","prompt":"A person is shown in various shots close up as well as leaves and him stretching. The man is then seen running through a park while looking off into the distance. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6258,"error":null,"output":"B"}
{"question_id":"hellaswag:00003645","item_index":41,"attempt":0,"prompt_hash":"fd5f651572646854c2161c362501974b54576c24d149990243b870a8d656b66a","question":"[header] How to stop your child from wetting the bed [title] Wait until your child is ready. [step] Your child may have mastered staying dry during the day, but that does not mean that he or she will be ready to stay dry at night. For most children, it's fine to keep them in diapers (or pull-up style disposable underwear) until they start waking up dry most mornings.\n\nA. [substeps] Understand that developmentally, each child is different. Some children can stay dry at night during the toddler years; others are still having accidents at age six or beyond.\nB. [substeps] Schedule an estimated amount of time for each child you keep dry. It would be a good idea to see an attendant or masseuse to help them learn to stay dry.\nC. Wait until it's at least bright enough to light the room, if possible. [title] Light up your child's room where you will see them before bed.\nD. [title] Try changing diapers only in the evening. [step] Instead of changing your children at night, try changing them at the same time each day.","prompt":"[header] How to stop your child from wetting the bed [title] Wait until your child is ready. [step] Your child may have mastered staying dry during the day, but that does not mean that he or she will be ready to stay dry at night. For most children, it's fine to keep them in diapers (or pull-up style disposable underwear) until they start waking up dry most mornings.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6256,"error":null,"output":"D"}
{"question_id":"hellaswag:00001680","item_index":42,"attempt":0,"prompt_hash":"871ec854bd121aaebd597eaf6febbab64de1bc1dabcca399a3f0ddeeb91f8a5a","question":"A woman is standing in front of a green background. she\n\nA. then has her hair brushed, twisting and styling it, combing it, giving all of it a feel.\nB. is playing the violin with a long bow.\nC. is throwing a ball back and forth with a stick.\nD. then starts removing wallpaper from a wall.","prompt":"A woman is standing in front of a green background. she","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6258,"error":null,"output":"C"}
{"question_id":"hellaswag:00005909","item_index":43,"attempt":0,"prompt_hash":"49464992f741af7f9d1af7e4d75db89722345427b65fea5abd3320977c3336bc","question":"[header] How to make short legs look longer [title] Wear clothes that fit. [step] Fitted pants and tops that seem custom made for your body will make you look long and lean. If you're petite, shop in the petite section of stores to boost your chances of finding clothes that fit properly.\n\nA. [substeps] Baggy and ill-fitting clothes will only make you look frumpy, and will do nothing to lengthen your legs. Avoid buying clothes that don't fit you perfectly.\nB. This is a fine location for height-conscious people to look, but avoid gaudy clothes that show interest in the female genitalia. [title] Accentuate the things that help you look.\nC. [substeps] For instance, if you weigh 13, you should accentuate your body's inner thighs by wearing items high enough so that your thighs are above your hips. If you don't particularly want to highlight your curves, pick short skirts or dresses that fall nicely and that hug your waist without drapes.\nD. Your wardrobe should also remind you that a bit of your figure is confined in all different ways and only when clothing accentuates or constricts these curves you can look the part. [substeps] Wear vertical stripes to make your legs look shorter.","prompt":"[header] How to make short legs look longer [title] Wear clothes that fit. [step] Fitted pants and tops that seem custom made for your body will make you look long and lean. If you're petite, shop in the petite section of stores to boost your chances of finding clothes that fit properly.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6256,"error":null,"output":"A"}
{"question_id":"hellaswag:00002464","item_index":44,"attempt":0,"prompt_hash":"2de77e02198b39c0613484f4a54b8710d310df21f250c0ab43caa0a8222ad5a3","question":"A person is see handing an officer an ice cream cone while a woman watches on the side. the woman\n\nA. then speaks to the officers while others watch on the side.\nB. then throws vanilla ice cream to the dog only to have the dog run away.\nC. then eats the ice cream while watching the officer behind them.\nD. is then seen pushing snow into the ice cream cart while a dog walks along the side.","prompt":"A person is see handing an officer an ice cream cone while a woman watches on the side. the woman","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6257,"error":null,"output":"A"}
{"question_id":"hellaswag:00006004","item_index":45,"attempt":0,"prompt_hash":"8adfb16016991c7d3631b32536593a2cb69ae70d914d6b3637ada9f5b87daa87","question":"[header] How to get rid of tiny ants [title] Find the ants' entry areas. [step] Ants come into your home for two main reasons: to find food to bring back to the colony, and to seek shelter from cold weather conditions. Ants could enter your home from many different areas, so be sure to check around window cracks, door entryways, lighting fixtures, and other possible entrances.\n\nA. This is especially important if the ants you're dealing with are migrating from outside. [substeps] If you can't find where ants are immigrating from, check a nearby garage, attic, chimney, or other exposed area.\nB. If you see a trail of ants in your home, follow them to see where they are entering from. [substeps] If you find specific entry points where ants are coming into your home, focus your repellant and deterrent efforts on these areas.\nC. [substeps] Give extra attention to the entrance to your home or apartment when searching for food. If you frequently use doors in the dining room, backyard, nursery, or hallways, these are probably your best choices.\nD. [substeps] See if your house or home has cement tiles, concrete, or metal flooring, especially. Small ants prefer these materials to hard surfaces.","prompt":"[header] How to get rid of tiny ants [title] Find the ants' entry areas. [step] Ants come into your home for two main reasons: to find food to bring back to the colony, and to seek shelter from cold weather conditions. Ants could enter your home from many different areas, so be sure to check around window cracks, door entryways, lighting fixtures, and other possible entrances.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6258,"error":null,"output":"B"}
{"question_id":"hellaswag:00005573","item_index":46,"attempt":0,"prompt_hash":"230a3616cc5005b6b3c8bb41724d24760588e19b1a24c93e1750a57195ced66d","question":"[header] How to convince parents to find time for themselves [title] Start the conversation. [step] The very first and basic step is you need to initiate talking about the matter. Keep it light.\n\nA. Something like, \" i really want to talk about my new job. Have you thought about that? \" would be great times to talk.\nB. No more than three minutes are a good number. Ideally, you want to start the conversation with something dull and boring-like anything that would be seen on television, such as doctors or actors.\nC. You have to bring that comfort level so that they feel like they can open up to you. This comes through sharing and asking them about what is happening in their life.\nD. When you're chatting, start your message with an' i', rather than' you', and keep it on the point that's coming up. The more positive you are as a person, the more you'll grow to understand that your parents may want to encourage your learning.","prompt":"[header] How to convince parents to find time for themselves [title] Start the conversation. [step] The very first and basic step is you need to initiate talking about the matter. Keep it light.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6258,"error":null,"output":"C"}
{"question_id":"hellaswag:00007956","item_index":47,"attempt":0,"prompt_hash":"2b8c16cd25d94ada578b96f97df15230e771ce4b04e3fae7f659ef91dad7c5fd","question":"[header] How to buy a used car off craigslist [title] Decide on a make and model. [substeps] Perform research on different car brands to determine which fits your needs. [title] Visit the \" cars & trucks \" section of craigslist.\n\nA. [step] While all areas have a public market with a huge number of used cars available to purchase, there may be discounts for older cars. [title] Contact different companies to request discounts or to learn what loans are available.\nB. [title] Make sure you have opened a large enough account to make the purchase. [title] Review each information page of the site.\nC. [substeps] Select \" by owner \" within this section if you wish to omit the postings of used car dealers. [title] Browse your city's used car postings for your make and model.\nD. [title] Determine what type of car you are looking for. [substeps] Is the car for sale, used, new or recently owned? [title] Feel for the driver's floor mounts to make sure you are looking for a safe, sturdy, easy to move car.","prompt":"[header] How to buy a used car off craigslist [title] Decide on a make and model. [substeps] Perform research on different car brands to determine which fits your needs. [title] Visit the \" cars & trucks \" section of craigslist.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6255,"error":null,"output":"C"}
{"question_id":"hellaswag:00001195","item_index":48,"attempt":0,"prompt_hash":"755fc16ecfd2d00e8340f90faec5a6a38a2133ae739e4370fe259c2c16f9e6a7","question":"Text appears across the white screen. two people\n\nA. are shown on the internet, with an etcetera, the man holding an ipad and a key in hand.\nB. hold up knives and begin fencing with each other.\nC. then become wet and the image becomes murky and grainy.\nD. are shown in a kitchen.","prompt":"Text appears across the white screen. two people","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6255,"error":null,"output":"D"}
{"question_id":"hellaswag:00002441","item_index":49,"attempt":0,"prompt_hash":"96dd11ae7aa3b4ebb2a6fad193f9731f13301d60cf862c0bf5574bc4f11ddcc0","question":"A close up of cats claws are seen with a woman speaking to the camera. the woman\n\nA. dices the cats claws while speaking to the camera.\nB. holds a pair of clippers and begins cutting the cats claws.\nC. continues speaking to the camera and walks around the cat on a table with the cat on the table.\nD. then grabs the claws and explains while another girl commentates and a pet tattoo is shown.","prompt":"A close up of cats claws are seen with a woman speaking to the camera. the woman","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6256,"error":null,"output":"B"}
{"question_id":"hellaswag:00008726","item_index":50,"attempt":0,"prompt_hash":"0d2760fa5139d4bf7c541bc248fd1025fd29f07720c4d4ee29cf8f67fe5329a2","question":"[header] How to ensure the safety of homemade cosmetic preparations [title] Use a spotless, clean workspace. [step] Before even proceeding with making a product, clear away a suitable workspace and ensure that it has been thoroughly cleaned. [title] Keep all equipment used clean.\n\nA. [step] Bring any patients that were on your ebay or supplied healthcare facilities and or facilities that are not mobile during the creation process. [title] Record the author's full legal name.\nB. [step] Any associated defects of the product can be brought to the cleaners' attention, so don't assume a timely in-house cleaner will be around. Also, clean the workspace thoroughly for each day, and when finished you should be ready to start.\nC. [step] Every single time that you make a new preparation, be sure to thoroughly wash and dry the equipment that is being used. [substeps] It is preferable to keep equipment used for the preparation of handmade cosmetics for this purpose alone, rather than sharing the equipment with other household tasks such as cooking or cleaning.\nD. [step] To ensure that the product you choose to use will cover your shelf, wash and dry it with hand soap or oven cleaner. Or, set the equipment aside for now.","prompt":"[header] How to ensure the safety of homemade cosmetic preparations [title] Use a spotless, clean workspace. [step] Before even proceeding with making a product, clear away a suitable workspace and ensure that it has been thoroughly cleaned. [title] Keep all equipment used clean.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6254,"error":null,"output":"C"}
{"question_id":"hellaswag:00001810","item_index":51,"attempt":0,"prompt_hash":"b3adb761c4135f5c1b0b64ed3180ce66ff8b0cb649a2222add767e45829be787","question":"A woman shows a red edge grip while talking. Then, she takes a knife and put it on the cleft of the edge grip and continues talking. after\n\nA. , the woman pass the knife on the edge grip from front to back while explaining.\nB. , she writes down instructions the point turn of the knife to adjust the pommel.\nC. , the woman puts out the nut around.\nD. , the woman uses a measuring tape to trim the crotch of the lady.","prompt":"A woman shows a red edge grip while talking. Then, she takes a knife and put it on the cleft of the edge grip and continues talking. after","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6255,"error":null,"output":"A"}
{"question_id":"hellaswag:00006153","item_index":52,"attempt":0,"prompt_hash":"4710427b3e85df8860cd366b676ac0575e854a2f2b0a96a8d2ac8725f4fdd956","question":"[header] How to feel comfortable in high heels [title] Get your feet measured and sized. [step] It is important to know the exact size of your feet when purchasing shoes. Once at a shoe store, you can ask a sales associate to assist you in proper foot measurement so that you can find shoes that fit you correctly.\n\nA. [substeps] They may be able to give you specific measurements to get a good idea of how big your feet are. However, this is a little under the bill.\nB. You can also ask the sales associate if you have concerns about the shoes having a leading sole or not allowing the heels to leave the floor. [substeps] Don't trust a sales associate to indicate exactly how large your feet are.\nC. [substeps] Wearing shoes that are even just a half size off can make for a very painful outing. Your poor tootsies will be in pain and your leg muscles will be very sore.\nD. Once at a shoe store, walk around in the store with your feet measured and trimmed and let a salesperson make a pre-fitting decision. You may ask sales associate to record all of your measurements to make sure you know which shoe you should wear.","prompt":"[header] How to feel comfortable in high heels [title] Get your feet measured and sized. [step] It is important to know the exact size of your feet when purchasing shoes. Once at a shoe store, you can ask a sales associate to assist you in proper foot measurement so that you can find shoes that fit you correctly.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6256,"error":null,"output":"B"}
{"question_id":"hellaswag:00009042","item_index":53,"attempt":0,"prompt_hash":"83bcf88812145ff640359f83e16f33195c33d2f31828f3573a4db7dbbb606259","question":"[header] How to prevent keloid and hypertrophic scars [title] Identify the risk factors for keloid scar formation. [step] Keloid scars are more commonly found on the arms, upper chest, and upper back. They grow beyond the borders of the original abrasion and appear as red, raised nodules.\n\nA. Keloid scars will usually take on the appearance of a raised scar when self-sewn onto the body, from which the scars are more visible. If your scars are recent, older scars may appear as they are and you want to avoid them.\nB. These nodules are easily seen in unsupported black circles and from afar; however, discoloration and dark splotches are particularly common. Even though blacks and whites are sometimes called keloid scars, the scars are especially common in areas that have chemical dermatitis.\nC. They may appear yellow, purple, or greenish. Keloid scars are temporary, but should rarely return.\nD. Keloids are the result of an overgrowth of dense fibrous tissue that usually develops after healing of a skin injury. You can also develop a keloid scar from a cut, a piercing, or acne.","prompt":"[header] How to prevent keloid and hypertrophic scars [title] Identify the risk factors for keloid scar formation. [step] Keloid scars are more commonly found on the arms, upper chest, and upper back. They grow beyond the borders of the original abrasion and appear as red, raised nodules.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6256,"error":null,"output":"D"}
{"question_id":"hellaswag:00006309","item_index":54,"attempt":0,"prompt_hash":"cf26db14cadba40742d0eaa98604de7bae2c609eaa394b5fd166b0dfd2bcd285","question":"[header] How to clean playtex sippy cups [title] Make sure to wash new sippy cups before using them. [step] Before giving a new playtex sippy cup to your child, wash it to remove any dust or dirt that may have come from the packaging. Even sippy cups that appear clean can contain germs and bacteria; washing the cup before use gives you a clean and fresh start.\n\nA. [substeps] Remove all the cleaner that came with the new toys. Use vinegar or distilled water to clean the cup.\nB. [substeps] Different ways to wash the cups will be covered in the following steps. [title] Separate all of the parts of the cup.\nC. [title] Remove the barbell and loosen the screws holding the small cup. [step] How you do this may depend on your settings, so consult your settings on your small cup.\nD. [title] Clean the sippy cup outside before using it. [step] Before taking your new playtex cup outside to wash it, take your child outdoors to cleanse it.","prompt":"[header] How to clean playtex sippy cups [title] Make sure to wash new sippy cups before using them. [step] Before giving a new playtex sippy cup to your child, wash it to remove any dust or dirt that may have come from the packaging. Even sippy cups that appear clean can contain germs and bacteria; washing the cup before use gives you a clean and fresh start.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6256,"error":null,"output":"B"}
{"question_id":"hellaswag:00004575","item_index":55,"attempt":0,"prompt_hash":"1092f014df3be6bfdc432c7ba5558be6b6dd4a7612afc4a8fe60ad1d5d0230a1","question":"[header] How to push back your cuticles [title] Prepare a bowl of warm water. [step] It shouldn't be too hot, or too cold. Soaking your nails in warm water makes the cuticles soft and pliable.\n\nA. Let them soak for several minutes before moving on to the next step. [substeps] You can also put them in ice water to help soften them slightly.\nB. When you push them back, they'll lift easily from your nails instead of sticking. You can also push them back after taking a warm shower, while your hands are still warm and moist.\nC. This will help to loosen them, while making them easier to push. [title] Run a thin fingernail under water and tap it against your cuticles.\nD. Therefore, you may want to use an epsom salt bath before attempting this method; it is not recommended for calluses. [title] Apply lukewarm or warm water to the cuticles.","prompt":"[header] How to push back your cuticles [title] Prepare a bowl of warm water. [step] It shouldn't be too hot, or too cold. Soaking your nails in warm water makes the cuticles soft and pliable.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6747,"error":null,"output":"B"}
{"question_id":"hellaswag:00005325","item_index":56,"attempt":0,"prompt_hash":"caa2d1939699c2b7c5e4e0b673f7e44f1d23f3388834769e83e798ae54fe5b60","question":"[header] How to make a natural insecticide [title] Look for holes in the leaves. [step] There are different types of insects that can be problematic in the garden, including ones like caterpillars and beetles that chew holes in leaves. These types of bugs can cause a great deal of damage to plants, so you should take steps to get rid of them.\n\nA. [substeps] Along with using insecticide to treat caterpillars and beetles in the garden, you can also remove them by hand. Pick the bugs off the plant with your fingers and drop them into a bucket filled with soapy water.\nB. To locate the holes, feel around the edges of the leaves and pull them up and down. There should be tiny holes where the insects attach themselves, usually located at the base.\nC. [substeps] Whether the leaves are crispy or look like they're teat-infested, insects like caterpillars are more likely to chew. Watch out for tiny vents in the leaves.\nD. [substeps] Holes in the leaves will also often be associated with insects that grow inside plants. You will want to examine the small holes near the front and sides of the leaves as well.","prompt":"[header] How to make a natural insecticide [title] Look for holes in the leaves. [step] There are different types of insects that can be problematic in the garden, including ones like caterpillars and beetles that chew holes in leaves. These types of bugs can cause a great deal of damage to plants, so you should take steps to get rid of them.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6748,"error":null,"output":"A"}
{"question_id":"hellaswag:00004454","item_index":57,"attempt":0,"prompt_hash":"4f4cdd0f722a6a6d290679cafa8c7d0c13751257e9ea438d4ab5b6e13ef78fb5","question":"[header] How to use a broiler [title] Find the broiler. [step] Older gas ovens have a drawer at the bottom of the oven that contains the broiling unit. This compartment is commonly called the \" broiler drawer.\n\nA. \" broilers have a smaller compartment in the oven, and this is also called a \" broiler drawer. \" [substeps] When you're using a stove top broiler, the drawer is easy to find.\nB. \" if you cannot find a broiler drawer, you can also find in your refrigerator. The broiler drawer also contains a knob for the broiler unit.\nC. \" if your oven doesn't have a broiler drawer, the broiler is inside the main oven compartment. In this case, the broiling unit in will be attached to the top of the oven interior.\nD. \" some broilers have compartment f. Check the pan drawer, as older gas ovens have compartment 6.","prompt":"[header] How to use a broiler [title] Find the broiler. [step] Older gas ovens have a drawer at the bottom of the oven that contains the broiling unit. This compartment is commonly called the \" broiler drawer.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6528,"error":null,"output":"C"}
{"question_id":"hellaswag:00002503","item_index":58,"attempt":0,"prompt_hash":"b1e0bc1ef3a84239b76825ee432f19e0d6ffe6135dba559391bfbf955cc57c37","question":"A female weight lifter bends at the knees. She lifts a barbell to her chest. she\n\nA. then lifts it over her head before dropping it heavily to the ground.\nB. lowers the barbell and stands, then sways.\nC. lifts it over her head.\nD. then lifts it over her head to her body.","prompt":"A female weight lifter bends at the knees. She lifts a barbell to her chest. she","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6526,"error":null,"output":"C"}
{"question_id":"hellaswag:00001710","item_index":59,"attempt":0,"prompt_hash":"b5a799f1fc32fc923f0662254ef173fc8583e33965cb2dea49bf78e8dd737294","question":"Another car is parked and the cat comes to a halt again before perusing the rest of the sidewalk and walking over the puddles. finally\n\nA. , it scoops up the ice off the sidewalk and returns to putting in more everywhere until it reaches the snow.\nB. , the cats grab their bags and leave.\nC. , the walk ends and the cat begins walking up the steps and looking at the person who was walking him.\nD. they arrive at an empty table where a man is talking and the cat walks findings first on a dog and dog hin.","prompt":"Another car is parked and the cat comes to a halt again before perusing the rest of the sidewalk and walking over the puddles. finally","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":10089,"error":null,"output":"C"}
{"question_id":"hellaswag:00007794","item_index":60,"attempt":0,"prompt_hash":"d7288b422a083455011833506dde9fa8f1ece9616cdcb567b0ca35f90b7f2066","question":"[header] How to make a homemade diaper [title] Use a shirt that is 100% cotton. [step] Cotton is more absorbent than most synthetic fibers, so it makes a better material to use for a cloth diaper. [substeps] Use a short sleeve or three-quarter-length sleeve for best results.\n\nA. A three-quarter length sleeve may make it easier to pin the diaper on larger babies and toddlers, but may be more material than you need for smaller babies. Choose a size based on the size of your child.\nB. You don't need to wash the shirt before wearing it. If you do wash your shirt before putting it on, then don't roll up the sleeves so you don't have to lift the shirt down when pulling it on.\nC. Look for a cotton with only one color on it. Darker colors will make the fabric more prone to wrinkling.\nD. [title] Measure the size of your chosen cloth diaper from head to ankle. [step] Purchase this measurement at a home improvement store.","prompt":"[header] How to make a homemade diaper [title] Use a shirt that is 100% cotton. [step] Cotton is more absorbent than most synthetic fibers, so it makes a better material to use for a cloth diaper. [substeps] Use a short sleeve or three-quarter-length sleeve for best results.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6629,"error":null,"output":"A"}
{"question_id":"hellaswag:00001540","item_index":61,"attempt":0,"prompt_hash":"8bf60d7a21d2df703e0c1372204ef2719a37c91876656226a9792648310d77e0","question":"The boy in the red hat and the blue coat is being pulled in a red tube by a rope over some icy roads. a man in a black hat\n\nA. has his turn with the tube.\nB. is talking to the camera.\nC. walks in front of the camera and camera man talks to the boy in the helmet.\nD. is getting thrown down a side slide.","prompt":"The boy in the red hat and the blue coat is being pulled in a red tube by a rope over some icy roads. a man in a black hat","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6628,"error":null,"output":"C"}
{"question_id":"hellaswag:00004867","item_index":62,"attempt":0,"prompt_hash":"2f5fd2ab50b90ab44e58c64120bcd36f34c11fba5cd23b2101e5761720131e6a","question":"[header] How to adjust to hair loss and baldness [title] Try to come to terms with your loss. [step] Many (probably all) men panic when they first see a thinning patch in their hair. Hair symbolizes vitality, sex appeal, and youth.\n\nA. Even an occasional trim can help curb your worries and allow your hair to return to its natural shape. [title] Keep a diary to keep track of your thinning condition.\nB. After your initial attempts, however, slowly and persistently accept your hair as your new warm-up, whether just by touching it or by talking to yourself in the mirror. Beauty can't be \" shared \" with no one but you.\nC. Studies show this is especially true of men with disease at their first shave. This is commonly used as a welcoming reminder of youthful vigor and maturity, due to the hair loss as a result of the shave.\nD. Men fear going bald because they fear that they are losing these things. However, the truth is that it's all in the mind.","prompt":"[header] How to adjust to hair loss and baldness [title] Try to come to terms with your loss. [step] Many (probably all) men panic when they first see a thinning patch in their hair. Hair symbolizes vitality, sex appeal, and youth.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6627,"error":null,"output":"D"}
{"question_id":"hellaswag:00000045","item_index":63,"attempt":0,"prompt_hash":"abfd5e7512d28cef592f7312d034773f943d803b8b28c08c7126a08ed937056c","question":"The representative from the company is talking about the services they offer and how those dogs participate in dog shows. a groomer\n\nA. is seen using shears to creatively trim the dog's hair and even dye it in colorful shades of red, green, yellow and blue.\nB. is presenting cookies to the room with a loaf of bread.\nC. shows crowd opinion for use as their garden service representative.\nD. speaks to an older man about snow cleaning dogs and showing how it is flectious.","prompt":"The representative from the company is talking about the services they offer and how those dogs participate in dog shows. a groomer","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6627,"error":null,"output":"A"}
{"question_id":"hellaswag:00000228","item_index":64,"attempt":0,"prompt_hash":"66158e36b9746736114df66122b40734429c838a7563e35587fd58332e5397ac","question":"A graphic shows on the screen for a website about basketball drills. Some young men on an indoor basketball court are shown doing some of the basketball drills. an older man\n\nA. in a blue shirt watches him.\nB. watch on the sidelines as the young men on the court make some best strikes with sticks.\nC. does roanoke court drills with the young man before him.\nD. begins explaining the drills while more footage is shown of the players demonstrating them.","prompt":"A graphic shows on the screen for a website about basketball drills. Some young men on an indoor basketball court are shown doing some of the basketball drills. an older man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6502,"error":null,"output":"D"}
{"question_id":"hellaswag:00006188","item_index":65,"attempt":0,"prompt_hash":"6e5b7bc73afcb750aaa254a404a73add92378cedea85c125a02147240a093dd1","question":"[header] How to write a letter from santa [title] Address the letter to the specific child. [step] If you want the child to believe the letter is really from, you should address it to that child. Mention the child's age in the letter as well.\n\nA. Mention their name in the body of the letter at least twice in addition to addressing it to the specific child. [substeps] Thread details into the letter that show santa knows the child.\nB. This will help the child feel needed and supported, thereby allowing them to properly grasp the letter's purpose. [substeps] For example, the letter might say \" congratulations on santa's birthday, jack.\nC. This will help the child to believe that santa claus and katniss are not real entities and that the letter you are sending will fit their individual needs. [substeps] Explain to your child that this letter would give them great benefits and reduce feelings of helplessness and self-doubt.\nD. You should address the letter by putting the child's father's name in the first line, followed by a period. [substeps] Address the letter to: santa claus-this is santa's father.","prompt":"[header] How to write a letter from santa [title] Address the letter to the specific child. [step] If you want the child to believe the letter is really from, you should address it to that child. Mention the child's age in the letter as well.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6494,"error":null,"output":"A"}
{"question_id":"hellaswag:00008446","item_index":66,"attempt":0,"prompt_hash":"47943728b4e7a76ccb20640ac994514ba3375b213d091960589958e49deb2857","question":"[header] How to ensure a child feels loved [title] Understand that especially in the early years how important it is to play and spend time with the young child. [step] Consistent attention and affection is the key to happiness. You can never spoil a child with too much.\n\nA. Don't forget that this is his or her moment. You can have at least your own play time, as long as you're genuinely being attentive and gentle.\nB. [title] Look at your child to see what his or her needs are. [step] Try to cuddle and read to them every night.\nC. [substeps] Make sure that the child is always quiet and out of the crib. If they are lonely, they may wish to talk to a teacher and invite him/her over, but this will only make them feel old and put-off.\nD. Spend time with the child and see them play with toys or home-grown kits. You should learn the child's wants and needs and suggest arrangements to accommodate these needs.","prompt":"[header] How to ensure a child feels loved [title] Understand that especially in the early years how important it is to play and spend time with the young child. [step] Consistent attention and affection is the key to happiness. You can never spoil a child with too much.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6493,"error":null,"output":"B"}
{"question_id":"hellaswag:00005245","item_index":67,"attempt":0,"prompt_hash":"add60205fe2d3f490966fcb1ef674af72925b7d49bbf446d00efaf940d44808f","question":"[header] How to get silky hair if you are a guy [title] Choose a gentle shampoo. [step] To keep your hair silky, you need to keep it clean, but not too clean. The natural oils in your hair are what helps to keep it strong and healthy, but those oils can also trap dirt and become greasy, making you hair dirty.\n\nA. Also, don't use sulfates on your hair either-they are too harsh, which can damage your hair more than it already is. [title] Pick an anti-frizz shampoo that has anti-frizz ingredients.\nB. You need to strike a balance between too much cleaning and not enough to have silky hair, meaning that a gentle shampoo is an essential part of your cleaning routine. [substeps] Try to use shampoo made with natural ingredients, and avoid shampoo with strong artificial fragrances.\nC. [substeps] Be sure to use only a mild shampoo if your hair is very thick or coarse, and never use product stronger than a light shampoo. If your hair is very thick, opt for a more moisturizing conditioner.\nD. [title] Use warm or cool water in your shower. [step] Although not a necessary step, it's a good idea to use warm water to prevent your hair from becoming dry or greasy.","prompt":"[header] How to get silky hair if you are a guy [title] Choose a gentle shampoo. [step] To keep your hair silky, you need to keep it clean, but not too clean. The natural oils in your hair are what helps to keep it strong and healthy, but those oils can also trap dirt and become greasy, making you hair dirty.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6494,"error":null,"output":"B"}
{"question_id":"hellaswag:00008430","item_index":68,"attempt":0,"prompt_hash":"64ff5db52719f742e01b6d6abb06e260b9415587ee17d3fd64a83ef8055bc540","question":"[header] How to shrink jeans [title] Wash your jeans in hot water. [step] Put your jeans in your washing machine and run them through a standard washing cycle. Set the machine to the hottest water level possible.\n\nA. [substeps] Avoid using delicate or hand-wash settings. Instead, use a permanent press or heavy duty setting.\nB. [substeps] Be aware that jeans can shrink as a result of wearing them. Use caution when washing jeans that are hot running this cycle.\nC. Ensure you are using a light cycle, not a heavy one. Fill the washing machine's water tank with only one cup (250 ml) of water, typically for heavy jeans.\nD. [substeps] Since jeans are water-resistant, this process is not recommended. Hot water will only shrink jeans, and jeans can shrink between 120 and 120 ° f (93 to 93 ° c).","prompt":"[header] How to shrink jeans [title] Wash your jeans in hot water. [step] Put your jeans in your washing machine and run them through a standard washing cycle. Set the machine to the hottest water level possible.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6500,"error":null,"output":"A"}
{"question_id":"hellaswag:00009664","item_index":69,"attempt":0,"prompt_hash":"d84aa1e79febb907cb4cb96d1399b2455ad5159fcb92ab6ea066b5320138eaea","question":"[header] How to arrange the best hen party [title] Talk to the bride-to-be. [step] She might just want a soothing time at a spa, or maybe something a bit more exciting! She will tell you the people to contact. Remember, she will invite people whom you may not know, so really don't take it upon yourself to complete this.\n\nA. [title] Take some time to decide on a name for the hen, and make a list of people to call out. [step] Remember she might get nervous about addressing all of her guests.\nB. The sooner you do this the better-2 months before the hen party will be okay. [title] Get a rough calculation of the number of people that will probably be joining the hen party.\nC. Decide what the next options are, and then always ask if anyone will come with you. [substeps] It can help if you have some extra funds, and if so, what the host will bring and take, as well as money to cover the cost of catering.\nD. A good way to save yourself time is to call up the bride and formally invite her. [title] Pack a new list of toiletries (see next step for more details).","prompt":"[header] How to arrange the best hen party [title] Talk to the bride-to-be. [step] She might just want a soothing time at a spa, or maybe something a bit more exciting! She will tell you the people to contact. Remember, she will invite people whom you may not know, so really don't take it upon yourself to complete this.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6492,"error":null,"output":"B"}
{"question_id":"hellaswag:00006543","item_index":70,"attempt":0,"prompt_hash":"9d8982351aeaa6ab0419d41a5fd91d18cf88a69a37af85d1b4f1c4e27f963eff","question":"[header] How to become a successful freelance writer at freelancing platforms [title] Make yourself look good. [step] An attractive profile and a resume is the key to success. The first thing you should do to get your job, is to create an attractive profile with a resume.\n\nA. You can do this without going below the heading which is ideal. [substeps] Research the job descriptions online before entering into an interview to find the best fit for you.\nB. Think about what your current skills are as you go up and down the career ladder, maybe even cite them on your resume. Employers may ask for your phone, computer and email, or even google your name.\nC. It should not be too long, it should contain your key strengths in an attractive and easy to read format. Don't list any irrelevant information, for example, your marital status or any other unnecessary personal information because it could turn off your potential client.\nD. Start by posting your resume in a \" subscript \" position so you get a good idea of what your resume will be. Having your resume written out and then putting it into writing might boost your skills on that particular job.","prompt":"[header] How to become a successful freelance writer at freelancing platforms [title] Make yourself look good. [step] An attractive profile and a resume is the key to success. The first thing you should do to get your job, is to create an attractive profile with a resume.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6493,"error":null,"output":"C"}
{"question_id":"hellaswag:00002128","item_index":71,"attempt":0,"prompt_hash":"a37fbc4fd9f3e6178d4b337ff12301ece9a9f24a017ee5c12a7eaeb578d2f839","question":"We see a basketball title screen. A man is coaching as a teen makes basketball shots. the man\n\nA. pans to a gym filled with athletes.\nB. is demonstrating the technique as other group of youth play in front of him.\nC. switches to his other fingers moving his hands.\nD. takes the ball and demonstrates a hip move the the boy tries.","prompt":"We see a basketball title screen. A man is coaching as a teen makes basketball shots. the man","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6496,"error":null,"output":"B"}
{"question_id":"hellaswag:00007410","item_index":72,"attempt":0,"prompt_hash":"e879e1c8d307fe939a65963f99480344795901f93504f827474bf5718f4dd134","question":"[header] How to develop good sleeping habits in children [title] Make sure you know how much sleep your child needs. [step] How much sleep people need varies between individuals, as well as by age. Young children may need up to twelve hours, whereas older school-aged children may only need nine.\n\nA. Children over age six and under need four to eight hours of sleep a night, while teenagers over age seven need eight to nine hours of sleep. [substeps] In one study , 13 children seven and up (and over the age of two) had over 8 hours of sleep a night.\nB. Most adults need between 20-40 hours. [substeps] Tell your child that you think your child needs sleep.\nC. If your child is having trouble sleeping, they may be overtired by the time they get to bed, or they're going to bed too early for their waking time. Adjust your child's sleep and wake times accordingly, and consider limiting naps in the day.\nD. The sleep you provide your child must be for him or her. [substeps] Start weighing your child at 12 or 13 weeks for measuring their maximum needed sleep.","prompt":"[header] How to develop good sleeping habits in children [title] Make sure you know how much sleep your child needs. [step] How much sleep people need varies between individuals, as well as by age. Young children may need up to twelve hours, whereas older school-aged children may only need nine.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6494,"error":null,"output":"C"}
{"question_id":"hellaswag:00005009","item_index":73,"attempt":0,"prompt_hash":"c29102fdb1e400e48439d8a0a4c248aea4bc7d75d18d2ec9e5b082b24fc9ea7e","question":"[header] How to treat internal hemorrhoids [title] Drink plenty of water. [step] Water is essential to prevent constipation, which is a major cause of hemorrhoids. Make sure that you are drinking around eight 8-ounce (1.9l) glasses of water every day.\n\nA. If you are active or if you still feel thirsty, then drink some more. [substeps] You can also drink other types of beverages to get more fluids, such as juice, herbal tea, and sparkling water.\nB. This will help your body fight the bacteria that are pushing hemorrhoids and prevent further damage to your digestive system. [substeps] The vast majority of people who take eight 8-ounce (1.9l) glasses a day are men over 35.\nC. Any less may irritate your abdominal area and worsen the problem. [substeps] Drinking water will moisten your cervix and prevent more painful hemorrhoids.\nD. [title] Use acupressure to relieve hemorrhoids. [step] Acupressure is similar to massage but uses the downward pressure in the butt, not your buttock.","prompt":"[header] How to treat internal hemorrhoids [title] Drink plenty of water. [step] Water is essential to prevent constipation, which is a major cause of hemorrhoids. Make sure that you are drinking around eight 8-ounce (1.9l) glasses of water every day.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6493,"error":null,"output":"A"}
{"question_id":"hellaswag:00008654","item_index":74,"attempt":0,"prompt_hash":"e74640fe927bc3e1969ed5ce6af3d4383971494442a0e0747b736dddd513f026","question":"[header] How to care for your feet and toenails [title] Make sure to wash your feet daily. [step] Feet tend to get dirtier and sweatier than other parts of the body, so it's important to have good sanitary habits. If you take showers in the morning and not at night, at least make feet-washing a part of your nightly routine when you get home.\n\nA. [title] If you want to give your feet a break from everyday life, get your toenails waxed. [step] They don't have to be pristine to be waxed.\nB. [substeps] Have a toothbrush and cotton wipes handy for use with your feet and toenails. [title] Make sure your toenails are always clean.\nC. [substeps] If you have psittacosis, you can go easy on those feet. Take a shower once daily-once a day-during winter, and once a day while in the spring or summer if your weather allows.\nD. Aside from being a mark of good hygiene, doing this also stops you from carrying dirt into your bed. [substeps] Wash them with soap and warm water.","prompt":"[header] How to care for your feet and toenails [title] Make sure to wash your feet daily. [step] Feet tend to get dirtier and sweatier than other parts of the body, so it's important to have good sanitary habits. If you take showers in the morning and not at night, at least make feet-washing a part of your nightly routine when you get home.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6493,"error":null,"output":"D"}
{"question_id":"hellaswag:00002546","item_index":75,"attempt":0,"prompt_hash":"fb2dd8ce1402c7327e974081b2e2e38dadbdbd44f8a906f638594ae3566478e3","question":"The video shows a demonstration of dyson vacuum cleaner and how well it can clean particles from the floor. there\n\nA. are various graphics on the screen that include names and font and video of the machine.\nB. 's rice grain, flour and other food particles scattered on the floor.\nC. are various solutions presented to the total vacuum product with her comment describing each implement as purchasing safety precautions onthank you very much for using it.\nD. is one person that shows how to remove the particles from carpet.","prompt":"The video shows a demonstration of dyson vacuum cleaner and how well it can clean particles from the floor. there","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6495,"error":null,"output":"B"}
{"question_id":"hellaswag:00007634","item_index":76,"attempt":0,"prompt_hash":"8bdd345c491d18ed6a25291e4c6f0c532a90add7720c359fbfa44e904cd4336e","question":"[header] How to prevent bed wetting [title] Realize that bedwetting is normal. [step] Although it may be inconvenient for you and embarrassing for your child, bedwetting is normal behavior in children. You can rest easy knowing that in many cases bedwetting will come to an end as your child grows older.\n\nA. Until then you can patiently and calmly work with with your child until she has outgrown the ages most commonly associated with bedwetting. [substeps] Bedwetting is common in children of up to around five years of age.\nB. In fact, bedwetting every two to three months is normal. [title] Talk to your child if they are having trouble sleeping.\nC. But when your children are older, you'll experience a more nasty stop to bedwetting. [substeps] Another reason bedwetting is normal is because a child ages.\nD. [substeps] Make sure that you understand how to accept bedwetting as an inevitable part of your child's development, and identify the possible consequences. Bedwetting will actually be a lifelong point of your child's development.","prompt":"[header] How to prevent bed wetting [title] Realize that bedwetting is normal. [step] Although it may be inconvenient for you and embarrassing for your child, bedwetting is normal behavior in children. You can rest easy knowing that in many cases bedwetting will come to an end as your child grows older.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6492,"error":null,"output":"A"}
{"question_id":"hellaswag:00002705","item_index":77,"attempt":0,"prompt_hash":"2bb08fd4d959f66ba5ff802ee4a9729d9ce103bcde14a0d772cd74170a7c2589","question":"A very large, muscular man is shown in a gym with a weight bar as piano music plays in the background. the video\n\nA. begins with the weight on the floor and the man getting ready to lift it.\nB. shows clips of athletes throwing various objects onto the other side.\nC. continues explaining his boxing talents as he shows how to apply specially made boxing gloves.\nD. , the run, and end with a karate photo shot of the previous man.","prompt":"A very large, muscular man is shown in a gym with a weight bar as piano music plays in the background. the video","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6496,"error":null,"output":"A"}
{"question_id":"hellaswag:00007307","item_index":78,"attempt":0,"prompt_hash":"eab903899f8007515388c869aae7be1820d03521e3e97eef8913e176422f128e","question":"[header] How to look like kourtney kardashian [title] Start with deep cleaning. [step] Kardashian believes in deep cleaning her face every day. Pick something that's right for your skin type, and use it before you go to sleep every night.\n\nA. [substeps] Cleansers come in a variety of types, for everything from oily to dry skin. The bottle will say what type of skin it's appropriate for.\nB. Kourtney prefers regular cleanings from her make up artist, makeup artist or groomer. It's a good idea to use at least twice daily.\nC. For example, use a rounded head brush in a shower bottle with plenty of suds for oily skin, or focus on cleansers made specifically for oily skin. However, aim for cleansers that contain spf, and don't leave residue behind.\nD. [title] When you wake up, put on your makeup. [step] Consider getting some so you know what you're doing.","prompt":"[header] How to look like kourtney kardashian [title] Start with deep cleaning. [step] Kardashian believes in deep cleaning her face every day. Pick something that's right for your skin type, and use it before you go to sleep every night.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6493,"error":null,"output":"A"}
{"question_id":"hellaswag:00009774","item_index":79,"attempt":0,"prompt_hash":"2ad282d6dbb9e34092a96d555fb5a2e3de50204bb4a69cefd064f870e4e735da","question":"[header] How to read a book to a baby or infant [title] Read at a slow pace. [step] As you read, make sure to go slow. In addition, enunciate words carefully and correctly.\n\nA. This ensures that the book tells a complete story. [substeps] Avoid over-stressing the child.\nB. The more you slow down, the more likely the child will be to guess what you're reading. [substeps] Read at any fast pace you have.\nC. This way, your child-depending on age-can better follow along. In addition, reading slowly will make your time together more relaxed and enjoyable for both of you.\nD. Your baby may think, \" whoa, yeah, now you don't understand, \" or \" wow! This is so fast! \" this is because you are planning to read it before you leave the library. [substeps] If your baby gets fussy, they're probably hungry.","prompt":"[header] How to read a book to a baby or infant [title] Read at a slow pace. [step] As you read, make sure to go slow. In addition, enunciate words carefully and correctly.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6496,"error":null,"output":"C"}
{"question_id":"hellaswag:00008105","item_index":80,"attempt":0,"prompt_hash":"0a9858a159abbf63fc3e152a50c3481e375862af1924372bb4f8d08e0053ed47","question":"[header] How to be the girl everyone wants to be [title] Get organized and stay that way. [step] Being perfectly put together is a quality many people will envy. Start by cleaning your room to perfection, and making sure there is a place for everything and everything has its place.\n\nA. Also be sure to get rid of things you know you don't need. Minimizing your \" stuff \" will help you find things quicker and easier.\nB. You will be assured that you haven't forgotten anything in your room. Be careful not to take things haphazardly.\nC. Organized girls would have laptops, notebooks, liquids, and sharpies. You might find that all you have to work with is an empty desk or a messy binder, just keep it organized and don't have yourself responsible for all the work right now.\nD. If you feel your room is cluttered or disorganized, don't attempt to change it. Cleaning is an impossible process, so you want to avoid this, and to prepare yourself for it physically.","prompt":"[header] How to be the girl everyone wants to be [title] Get organized and stay that way. [step] Being perfectly put together is a quality many people will envy. Start by cleaning your room to perfection, and making sure there is a place for everything and everything has its place.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6495,"error":null,"output":"A"}
{"question_id":"hellaswag:00009162","item_index":81,"attempt":0,"prompt_hash":"985f5d7937db74bc4fbddb9cbd9e12d35e90f91ea4598647958a554fa634b598","question":"[header] How to get cheekbones [title] Choose your basic blush. [step] Blush is a great way to wake up your face. But if it's applied incorrectly, it could turn into a makeup disaster.\n\nA. It's important to pick a blush that works with your skin tone and skin type. [substeps] For dry skin, try a cream based blush.\nB. To prevent this from happening, think about contouring your cheekbones first to disguise them. [substeps] Contouring can look translucent at first, especially using a little blush.\nC. Careful where you put your blush; on the nose, eyes, and mouth. [substeps] Blush will blend nicely with your skin colour.\nD. Different types of blushes look good with different complexion shapes, from subtle shades to bright hues like red or crimson. [substeps] When choosing blush, the most important thing is choosing the right shade for your face shape.","prompt":"[header] How to get cheekbones [title] Choose your basic blush. [step] Blush is a great way to wake up your face. But if it's applied incorrectly, it could turn into a makeup disaster.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6496,"error":null,"output":"A"}
{"question_id":"hellaswag:00004469","item_index":82,"attempt":0,"prompt_hash":"c469d86a7371546c09809defba408ca108acf96fa899ef527fd433f48e774b7e","question":"[header] How to remove smell from an old leather bag [title] Try a simple clean first. [step] If this clean doesn't shift the odor, you can try one of the other suggested methods after. [substeps] Wipe the leather bag inside and out with a clean, dry, soft cloth.\n\nA. This will pick up dust, loose debris and even some mold or mildew. Wipe the leather bag down with a damp cloth.\nB. Pour down a little dishwashing liquid directly over the odour. Washing the cleaning cloth in warm water along with it will prevent it from drying out or settling into a moldy head of the bag.\nC. Using gloves to protect your hands from the chemicals, take the stained leather cloth and place it on the soiled bag. Let it sit for 10 minutes, then wipe from top to bottom.\nD. [substeps] Set a low-ph or neutral or ph-free vinegar solution into the bag after cleaning. Mix it with a little water and warm water and press a few drops of the solution to the leather.","prompt":"[header] How to remove smell from an old leather bag [title] Try a simple clean first. [step] If this clean doesn't shift the odor, you can try one of the other suggested methods after. [substeps] Wipe the leather bag inside and out with a clean, dry, soft cloth.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6494,"error":null,"output":"A"}
{"question_id":"hellaswag:00000551","item_index":83,"attempt":0,"prompt_hash":"e84fd31744e57fdd391235a3d0fe0e09b1e761940c8521ec563d9a0aeeb99fb6","question":"The two are shown mixing the bowl in turn. The two shapes the mix from the bowl into lumps and places them on a baking sheet. the two\n\nA. mix the cookies over the oven with the arm of the baking sheet out and hand shake an the hands of the person holding the bowl.\nB. are shown once again in several more sitka footage.\nC. place the baking sheet in an oven.\nD. are then seen blowing them onto a cake and on the edges on baking sheets.","prompt":"The two are shown mixing the bowl in turn. The two shapes the mix from the bowl into lumps and places them on a baking sheet. the two","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6496,"error":null,"output":"C"}
{"question_id":"hellaswag:00003337","item_index":84,"attempt":0,"prompt_hash":"52077968f59a2d0470512612753bddd3d181a2d5bad1ee28c4730fd0c450a3b5","question":"[header] How to read a foot reflexology chart [title] Familiarize yourself with the basic foot reflexology chart. [step] Starting out, learn about the basic areas on a foot reflexology chart. This outlines the major organs locations on the foot.\n\nA. This will help you to determine the exact direction of movements you should make during another stage of your experimental activities. [substeps] Start with the sole of your foot and work your way up to the pads and toes of your feet.\nB. By learning these, you will be able to identify the general areas where you will need to focus. [substeps] You cannot use a chart to accurately analyze the chart.\nC. [substeps] The right foot is associated with the right part of the body and left foot is associated with the left side of the body. The stomach, for example, is primarily located on the left side of the body so massaging and applying pressure to the left foot can treat stomach ailments.\nD. [substeps] The first circle of the chart should detail the movement of the heel and sides of the foot during exercise, or the movement of the feet as a whole. The second circle should also describe the movement of the toes.","prompt":"[header] How to read a foot reflexology chart [title] Familiarize yourself with the basic foot reflexology chart. [step] Starting out, learn about the basic areas on a foot reflexology chart. This outlines the major organs locations on the foot.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6495,"error":null,"output":"C"}
{"question_id":"hellaswag:00004212","item_index":85,"attempt":0,"prompt_hash":"3a84d77c19fb96f94964eb1bce7b4ee395f0041ac53f7f01c292635dca3b17bc","question":"[header] How to clean mold in shower [title] Make a mold-killing solution. [step] Pour ½ cup (120 ml) of vinegar into 1 quart (1 liter) of warm water. Use distilled white vinegar.\n\nA. The solution will evaporate because the mold is gone. Stir with a spoon to mix in the vinegar.\nB. Then mix in ¼ cup (59 ml) of borax. Mix the ingredients together in a pitcher until they are well-combined and the borax is completely dissolved.\nC. Stir to combine, then pour it into the mold-killing solution. Using a small can opener, cut and mash the mold-killing solution.\nD. After the vinegar forms, get down onto the floor and dunk your face into the mixture. Bring the solution up high until you see bubbles that bubble up to the surface.","prompt":"[header] How to clean mold in shower [title] Make a mold-killing solution. [step] Pour ½ cup (120 ml) of vinegar into 1 quart (1 liter) of warm water. Use distilled white vinegar.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6495,"error":null,"output":"B"}
{"question_id":"hellaswag:00003765","item_index":86,"attempt":0,"prompt_hash":"cef439ba0218f58f43106c7748d0085452a43311bd4540f1392f72a0bb8ea2b3","question":"[header] How to make deep fried chicken [title] Marinate the chicken. [step] Combine 1 us-quart (950 ml) (. 9 liters) buttermilk , 4 teaspoons salt, and one teaspoon of pepper into a large bowl. Stir the ingredients to combine them.\n\nA. [substeps] Make sure to check the marinade instructions on the package. [title] Pour the marinade in a shallow pan over medium heat.\nB. Cover the bowl and refrigerate it for four to six hours or until the marinade is absorbed. [substeps] In addition to curing the chicken, marinade it as well.\nC. [title] Simmer the chicken in the marinade. [step] Place the marinade in a deep skillet and bring the chicken to a simmer.\nD. Next, place all of the chicken pieces into the marinade, pressing them into the liquid to completely coat them. [substeps] Cover the bowl with plastic wrap and place it in the refrigerator for 4-24 hours.","prompt":"[header] How to make deep fried chicken [title] Marinate the chicken. [step] Combine 1 us-quart (950 ml) (. 9 liters) buttermilk , 4 teaspoons salt, and one teaspoon of pepper into a large bowl. Stir the ingredients to combine them.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6492,"error":null,"output":"D"}
{"question_id":"hellaswag:00006599","item_index":87,"attempt":0,"prompt_hash":"806bbce6af21626322a079040c49ae65b64ecc4dab9ceb649a9228476f3f67ab","question":"[header] How to support a spouse who has cancer [title] Get past the initial shock together. [step] This is the dreaded \" getting your affairs in order. \" while it's unpleasant to think about, and it may even seem selfish to you, both of you should be prepared in case the worst does happen.\n\nA. Think of it this way: even if your spouse survives the cancer, you will both feel more comfortable knowing that you have put your personal affairs in order. [substeps] Prepare or update your will and/or trust.\nB. Our first instinct will probably be to rush this hurdle, but once the reality is confirmed and the danger has passed, the conversation and reunion will be forced on both of you. [substeps] This may even begin as a casual way to show your mate how much you care for him or her.\nC. [substeps] Coming to terms with the diagnosis for cancer can be a complicated process, and requires patience and commitment. You can say to your partner, \" dr.\nD. [substeps] Do not blame your spouse for any difficult times. Surround yourself with people who love and respect you, and embrace shared interests, passions, and talents you potentially may not possess.","prompt":"[header] How to support a spouse who has cancer [title] Get past the initial shock together. [step] This is the dreaded \" getting your affairs in order. \" while it's unpleasant to think about, and it may even seem selfish to you, both of you should be prepared in case the worst does happen.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6711,"error":null,"output":"A"}
{"question_id":"hellaswag:00009829","item_index":88,"attempt":0,"prompt_hash":"c94c7e0c47477674366ba715f1a68dad274dc2dcc3cac1798f7184ca42bd3773","question":"[header] How to keep a single ferret happy [title] Set aside play time every day. [step] Ferrets need a lot of attention, and they should receive a considerable amount of out of cage time every day. Make sure you have at least an hour to play one-on-one with your ferret on a daily basis, as too much cage time can lead to stress and illness.\n\nA. [substeps] Make sure every night is play time for your ferret. Provide enough play time for both you and your ferret each day, as you may want to let each of them individually play for hours.\nB. [substeps] Ferrets are very intelligent and active, so you need to be engaged in play. This is especially important for single ferrets, who can get bored without a companion.\nC. [title] Trim the ferret's nails. [step] Ferrets rarely gnaw at nails, nor do ferrets chew on very long and thin nails.\nD. [title] Remove your ferret from the cage after meals and before bedtime. [step] This will reduce chances of your ferret crashing to the floor in his cage when he relaxes in his temporary den.","prompt":"[header] How to keep a single ferret happy [title] Set aside play time every day. [step] Ferrets need a lot of attention, and they should receive a considerable amount of out of cage time every day. Make sure you have at least an hour to play one-on-one with your ferret on a daily basis, as too much cage time can lead to stress and illness.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6712,"error":null,"output":"B"}
{"question_id":"hellaswag:00002367","item_index":89,"attempt":0,"prompt_hash":"5c2f6687e25a5fe93dbf71fd60669a72461732a476dddf47dd1b7a42b60c21c8","question":"A man is seen in several clips shooting a basket into a hoop. He is then shown attempting to play tennis when another man walks in. the man\n\nA. hit the ball in another other direction, but eventually shows the person in the hoop again.\nB. attempts several other poses while still shooting the ball to another man.\nC. teaches him how to play tennis and has him practice.\nD. pushes on the ball and causes it to flip back to him where he misses several times.","prompt":"A man is seen in several clips shooting a basket into a hoop. He is then shown attempting to play tennis when another man walks in. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6710,"error":null,"output":"C"}
{"question_id":"hellaswag:00007725","item_index":90,"attempt":0,"prompt_hash":"77108c03b3b24b5ba7efa3b4950f023942eb62000c18690c9c4bb99d1b168d73","question":"[header] How to control your appetite [title] Eat from small plates. [step] Many people finish all the food on their plates despite their best efforts. One way to control your appetite is to control your portions by eating from smaller plates and bowls.\n\nA. This will allow you to control the size of your plate. [substeps] You don't have to eat portions to correct these portion sizes.\nB. When you are eating in front of a mirror, try to make sure your jaw is not clenched so tightly that your jaw cracks. That can cause the swelling and ache you're experiencing.\nC. While you'll mentally feel satisfied that you're eating a full portion, you're more likely to consume less. [substeps] Try switching from a 12-inch plate (the average plate size in the us) to a 10-inch plate (the average plate size in france).\nD. Use bigger plate sizes, placing them at different lengths and equipping your plates with different cuts of meat or seafood. [substeps] Portion sizes are usually identical to portion sizes.","prompt":"[header] How to control your appetite [title] Eat from small plates. [step] Many people finish all the food on their plates despite their best efforts. One way to control your appetite is to control your portions by eating from smaller plates and bowls.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6709,"error":null,"output":"C"}
{"question_id":"hellaswag:00009466","item_index":91,"attempt":0,"prompt_hash":"f39b8eab96346e876c5ff46772be04ff516b751ecb1e80ba1db6f055c972d9c6","question":"[header] How to watch a movie [title] Play dvds through your player or computer to watch movies simply and easily. [step] If you have a physical copy of the movie, simply put it in to play it. You will need to appropriate device, of course.\n\nA. Almost all computers and laptops can handle dvds now, and many can handle blu-ray without a hiccup. Most video game systems, too, can play blu-ray/dvd.\nB. You can select not to screen movie files or to play the movies yourself. [title] Put in headphones, video games and even tea bags and coffee filters to get in order to watch the movie at night.\nC. [title] Never keep kids quiet on a video. [step] Video games read full movies, not drags all night.\nD. [title] Find a place where you can watch your files and docusate it online. [step] Place the location in your player's sd card as well to minimize that screen time.","prompt":"[header] How to watch a movie [title] Play dvds through your player or computer to watch movies simply and easily. [step] If you have a physical copy of the movie, simply put it in to play it. You will need to appropriate device, of course.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6453,"error":null,"output":"A"}
{"question_id":"hellaswag:00004771","item_index":92,"attempt":0,"prompt_hash":"bee8ae676b02ce50f4ce92a02cda6d225ce5febd96e1ae2420b9d3e18b678b22","question":"[header] How to balance your professional and personal life [title] Try to separate work and play. [step] In the age of online learning and telecommuting, you can easily spend your whole day at home and get everything done. Attending school or working from afar may even add more flexibility to your home life.\n\nA. However, if you're not capable of working from home, try to find ways to mix both busy times and outside distractions. [substeps] There's no need to be in your home all day.\nB. If in doubt, make the decision to balance your work and play in your home. Use one section of your home, such as your bedroom, for leisure, and one section of your home for extracurricular activities.\nC. However, a downside is that work or school can creep over into home and/or family activities. It can be hard to back away when work is always within reach.\nD. [substeps] Whether you're working from home, learning new skills, looking for new recipes, etc. , trying to balance work and school is much more difficult.","prompt":"[header] How to balance your professional and personal life [title] Try to separate work and play. [step] In the age of online learning and telecommuting, you can easily spend your whole day at home and get everything done. Attending school or working from afar may even add more flexibility to your home life.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6454,"error":null,"output":"B"}
{"question_id":"hellaswag:00006343","item_index":93,"attempt":0,"prompt_hash":"db5f4d35253e379045fecfbe509f4eb2e539eba28b352a507cbb7ab3512385df","question":"[header] How to become a financial investment advisor [title] Decide whether financial advising suits you. [step] Working as a financial adviser can be rewarding, but it is also challenging due to the amount of skills required and the long hours you have to work for. Think of the interest you have in finance and investment.\n\nA. Decide if you would find this career job fulfilling or if financial advising might be better suited for you. This is important because financial advice can be much easier said than done.\nB. Determine if financial advising suits you the most and where you would like to practice. Many financial advisors are responsible for their clients' financial backing.\nC. You will need to find the kind of high level position you need in order to make a better investment. [substeps] Companies are confident that they are climbing the corporate ladder.\nD. The more interest you have in financial matters, the more efficient you are likely to be as a financial investment adviser. [substeps] Being a successful financial advisor also depends greatly on your ability to work with people.","prompt":"[header] How to become a financial investment advisor [title] Decide whether financial advising suits you. [step] Working as a financial adviser can be rewarding, but it is also challenging due to the amount of skills required and the long hours you have to work for. Think of the interest you have in finance and investment.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6453,"error":null,"output":"D"}
{"question_id":"hellaswag:00004214","item_index":94,"attempt":0,"prompt_hash":"0835525d5b9c4e9f0d809a44234bc9280358a168716973235b9c4d9d2c887498","question":"[header] How to teach teachers [title] Build rapport between yourself and the teachers. [step] Invest in relationships with your teachers so they feel supported and encouraged. Teachers are more likely to examine their own methods and self-reflect on their teaching when they feel led by a supportive guide rather than a critic.\n\nA. [title] Ensure that the balance is kept between it being helpful and not being effective. [step] Even when a teacher uses the same techniques for teaching, they may think it is trying too hard so focusing on intolerances or grades over problems is not helpful at all.\nB. [substeps] Build communication between teachers by making frequent phone calls and attending meetings with your teachers. Write on the board whenever you have time.\nC. [substeps] Showing your teachers support with your methods may encourage others to follow yours. Talk to your teachers regularly to encourage individual members of their staff to do the same.\nD. [substeps] For example, you could ask your teachers to one-on-one meetings to talk about their hopes and anxieties for the coming year. Check in with your teachers throughout the year to ask if there are any areas of difficulty they want help with.","prompt":"[header] How to teach teachers [title] Build rapport between yourself and the teachers. [step] Invest in relationships with your teachers so they feel supported and encouraged. Teachers are more likely to examine their own methods and self-reflect on their teaching when they feel led by a supportive guide rather than a critic.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6454,"error":null,"output":"D"}
{"question_id":"hellaswag:00009508","item_index":95,"attempt":0,"prompt_hash":"3915bbcb46741233fcd93c1c6a30eb2c243fdb388e60fa9ae0289bd269538e48","question":"[header] How to stop picking your scabs [title] Disinfect the wound. [step] Open wounds and sores can develop infections. Always wash a new wound thoroughly with soap and water as soon as you obtain it.\n\nA. Do not use hydrogen peroxide, petroleum or other deodorizers. Do not use antibacterial or antiseptic cleaners, as these may irritate the wound.\nB. Then clean it up with an antiseptic wipe or a bit of neosporin and apply a bandage to protect it while it heals. You can also try using betadine or peroxide on the wound to clean and remove unwanted bacteria.\nC. [substeps] Spread antibiotic ointment onto the wound, cover the wound with a sterile dressing that may be applied to the wound if it requires it. If you get a scab, wait 15 minutes to let the ointment finish fully absorbing in to the wound.\nD. At least once every two weeks, take a shower or bath for seven to 10 days and splash your wound with warm water before applying bandages. Use soap with antibacterial properties and do not share it with others.","prompt":"[header] How to stop picking your scabs [title] Disinfect the wound. [step] Open wounds and sores can develop infections. Always wash a new wound thoroughly with soap and water as soon as you obtain it.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6454,"error":null,"output":"B"}
{"question_id":"hellaswag:00008842","item_index":96,"attempt":0,"prompt_hash":"c287d321e7abefc9c7acb6f11db50389c095964aed45297eee5979082ebaa3a5","question":"[header] How to teach reading [title] Teach the alphabet. [step] The first step in learning to read is recognizing the letters of the alphabet. Use a poster, chalkboard, or notebook to write or display the alphabet.\n\nA. Go over the letters with the student until he or she learns them all. Use the alphabet song to help the student to remember.\nB. [substeps] Name all letters in the alphabet for use in the classroom. Place them in contrasting colors, rather than all at the same time.\nC. Pay special attention to each letter in the alphabet as this will help you recognize everything you need to know. Some methods for learning the alphabet are : [substeps] Creative writing.\nD. This will help you to put them in context. List the letters first, then move on to rhyming them out loud.","prompt":"[header] How to teach reading [title] Teach the alphabet. [step] The first step in learning to read is recognizing the letters of the alphabet. Use a poster, chalkboard, or notebook to write or display the alphabet.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6305,"error":null,"output":"A"}
{"question_id":"hellaswag:00009464","item_index":97,"attempt":0,"prompt_hash":"ad1c382829ad5efd07d0c494e7fec32a09ac0f514a65dce72b4a5a4d1bdeb0d6","question":"[header] How to make a facial mask [title] Gather the ingredients. [step] This mask works to reduce the appearance of pores with clay, which pulls dead skin cells and other pore-clogging debris away from your skin as it dries on your face. Go to a health food stores and get the following supplies : [substeps] Bentonite clay, which draws out impurities.\n\nA. Bentonite clay produces more waste than bentonite clay. Organic soaps containing pore cleansing compounds.\nB. Baking soda (bentonite usually draws them out from within pores, but check the label to be sure). Salicylic acid (omega 3).\nC. Oatmeal, which leaves your skin feeling baby soft. Essential oil of your choice, such as peppermint or lemongrass, for an added sweet scent.\nD. You can typically make a mask online at http: // www.bentonite/nutrition-news-13410dc.html. It is dark-colored and has a whitish appearance.","prompt":"[header] How to make a facial mask [title] Gather the ingredients. [step] This mask works to reduce the appearance of pores with clay, which pulls dead skin cells and other pore-clogging debris away from your skin as it dries on your face. Go to a health food stores and get the following supplies : [substeps] Bentonite clay, which draws out impurities.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6298,"error":null,"output":"C"}
{"question_id":"hellaswag:00005026","item_index":98,"attempt":0,"prompt_hash":"03da44a37608c8ff59886df558dde791e65031bf01f38cbe91800453e42e12b7","question":"[header] How to paint shoes [title] Use leather or spray paint for leather or vinyl shoes. [step] There are acrylic paints designed to adhere to leather products, including shoes. You can buy them at your local craft store.\n\nA. [substeps] Cover your shoe with a piece of rubber cling film after pouring in the paint. Allow it to dry completely before walking barefoot on it.\nB. These paints go on with a brush for a smooth and lasting finish. Your other option is to use spray paint purchased from a hardware store.\nC. Some even come with extra spraypaint depending on the design, so that you can paint the insole, parts, or strap of the shoe. [title] Apply anhydrous adhesive to the insole.\nD. If you decide to use paint, you'll need to consider washing your shoes before use. This should ensure that the paint doesn't damage the leather.","prompt":"[header] How to paint shoes [title] Use leather or spray paint for leather or vinyl shoes. [step] There are acrylic paints designed to adhere to leather products, including shoes. You can buy them at your local craft store.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6303,"error":null,"output":"B"}
{"question_id":"hellaswag:00000889","item_index":99,"attempt":0,"prompt_hash":"e0fe0c6ea4970a212e81651002443c4b42937b887d3bd283e92f143f60e22016","question":"They spit it out into a sink. They take another drink out of a cup. They put their mouth on a towel. they\n\nA. take another drink and dump it out in the sink.\nB. rub their faces together.\nC. begin to shave their legs with their hands.\nD. carry on an arm wrestling match.","prompt":"They spit it out into a sink. They take another drink out of a cup. They put their mouth on a towel. they","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6303,"error":null,"output":"A"}
{"question_id":"hellaswag:00009088","item_index":100,"attempt":0,"prompt_hash":"556e3b0bff021d789e0bfb092d6883676e32fab39b2bd3e3eb3ae94da02e2638","question":"[header] How to identify rosemary [title] Measure the rosemary. [step] Upright forms of rosemary can grow to 5 feet (1.5 m). ( 1.5 m) in height.\n\nA. There are trailing forms of rosemary as well which grow low and spread on the ground. [title] Look for plants that resembles a short pine tree with a fat trunk and smaller, long, thin branches which grow straight up into the air from the trunk.\nB. [title] Use the length of a long broom to measure rosemary. [step] Measure and mark a length for the broom in your yard.\nC. The narrowest is about 1 foot (30. 5 m) in length, while deeper forms are only 3 feet (0.9 m).\nD. Take a measuring tape and hold it at waist level. Carefully measure from the tip of the sloping point of the leaf nearest to the ground.","prompt":"[header] How to identify rosemary [title] Measure the rosemary. [step] Upright forms of rosemary can grow to 5 feet (1.5 m). ( 1.5 m) in height.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6303,"error":null,"output":"A"}
{"question_id":"hellaswag:00005310","item_index":101,"attempt":0,"prompt_hash":"63854544df48becfa47ffe1b1be93907822fe32651fd67d1ace71e3d303847f1","question":"[header] How to get your adult children to move out [title] Assess the situation as objectively as possible. [step] As a parent, you might have mixed feelings about encouraging your child to move out. On one hand, you might enjoy the company, or you don't want them to struggle on their own, or you don't want to feel like you're \" kicking \" anyone out.\n\nA. But, when you identify the situation as objectively wrong and see the root cause, you can fix the problem. You can get help from a local community center.\nB. However, you should also consider that all you do to get your kids to move out is discipline them. [substeps] Think about why you are encouraging your adult children to move out.\nC. Alternately, maybe you don't plan to let them go. Some common reasons for this include : [substeps] You aren't ready to give your kids the responsibility of keeping them on their own.\nD. On the other hand, perhaps you sense that, and if you don't take action s/he might never become self-sufficient. It's important to sort through all of these feelings before you talk to your child.","prompt":"[header] How to get your adult children to move out [title] Assess the situation as objectively as possible. [step] As a parent, you might have mixed feelings about encouraging your child to move out. On one hand, you might enjoy the company, or you don't want them to struggle on their own, or you don't want to feel like you're \" kicking \" anyone out.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6296,"error":null,"output":"D"}
{"question_id":"hellaswag:00001701","item_index":102,"attempt":0,"prompt_hash":"7d6ac35a7b28fd286a97693b89220fd9ea376557b49e2115cb1770c7c56a4b2c","question":"Man is in a workshop and puts a glove in his hand and grab a weld machine. man\n\nA. start welding something in a metal table.\nB. is welding a dimensional surface an separating parts.\nC. is standing in front of a wall and is whipping the wall with the weld machine.\nD. is standing in a cubed in front of a laptop and is showing how to weld pants and large belt.","prompt":"Man is in a workshop and puts a glove in his hand and grab a weld machine. man","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6301,"error":null,"output":"C"}
{"question_id":"hellaswag:00008059","item_index":103,"attempt":0,"prompt_hash":"f18e4f654765263efcba749d0364d6ea87708b7faf6470f6a72abbcea5b97604","question":"[header] How to make an impossible bottle [title] Remove the plastic wrapper and toss. [title] Remove the cards. [title] Heat the sticker with a blow dryer.\n\nA. [title] Remove the plastic tabs near the top of the soda pop/muffin cup. [title] Find the candy tray that holds the soda pop bottles together.\nB. [title] Place the adhesive strip in the oven. [step] Remove plastic wrap and place the sticker in the oven.\nC. [title] Turn off the blow dryer and cool the game pieces. [title] With ice cream, cold dip the glass in water and spread with the exterior of the popsicle sticks.\nD. [step] It will come off easily without tearing. [title] Using a blow dryer again, or using a sharp hobby knife, slice open the bottom seam of the box so that it can be flattened.","prompt":"[header] How to make an impossible bottle [title] Remove the plastic wrapper and toss. [title] Remove the cards. [title] Heat the sticker with a blow dryer.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6301,"error":null,"output":"B"}
{"question_id":"hellaswag:00000822","item_index":104,"attempt":0,"prompt_hash":"7b925d6b18f2cf4562a4a84ec1819b6eeafdee6514f26b0427241bc3200e9671","question":"A man is outside with two dogs on leashes. He puts on a show with the trained dogs. he\n\nA. throws frisbees, and the small crowd claps for the performing dogs.\nB. is spraying water from a hose into the air in front of him.\nC. holds up a bag at his side and dogs inside.\nD. is performing with the dogs, catching frisbees and catching frisbees.","prompt":"A man is outside with two dogs on leashes. He puts on a show with the trained dogs. he","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6300,"error":null,"output":"B"}
{"question_id":"hellaswag:00006728","item_index":105,"attempt":0,"prompt_hash":"f86e40780bc803decd88c6416d4cd2d144722271484d0d79580ef6c5c332d15c","question":"[header] How to grow out bangs [title] Arm yourself with bobby pins and hair barrettes. [step] Cute hair accessories serve an important function when it comes to growing out bangs. Use bobby pins and barrettes to pin back your bangs and incorporate them into your styled hair.\n\nA. [substeps] If you don't own many bobby pins, place a few in front of your bangs, let it sit and then cover with another layer of hair. This will help keep your bangs looking fuller and small, without putting too much emphasis on length.\nB. [substeps] Natural hairstyles, like braids or buns are a great option when it comes to your bangs. Check that your hair is growing out in-between each one before you begin braiding.\nC. Clip back the rest of your hair so it stands up straight. [title] Make braids that are shorter than your bangs.\nD. With a little practice, no one will realize that you're dealing with the awkward grow-out stage. [substeps] Get bobby pins in color that matches your hair, so you'll be able to pin it back discreetly.","prompt":"[header] How to grow out bangs [title] Arm yourself with bobby pins and hair barrettes. [step] Cute hair accessories serve an important function when it comes to growing out bangs. Use bobby pins and barrettes to pin back your bangs and incorporate them into your styled hair.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6298,"error":null,"output":"D"}
{"question_id":"hellaswag:00007894","item_index":106,"attempt":0,"prompt_hash":"d9ce0c6f899e97d780ab02004389c47dedf19c2b37a94e8200512a610d72afba","question":"[header] How to treat botulism in a duck [title] Watch for the symptoms. [step] Botulism poisons the duck, causing what's sometimes known as \" limberneck disease. \" it causes paralysis, starting with the duck having trouble taking off or going under the water.\n\nA. The duck's legs will be paralyzed, and you may see the duck trying to move only with its wings. The duck's eyelids and neck will droop.\nB. Chlamydia doesn't have any symptoms, but is caused by consuming ylang-ylang and urinating a lot. It may contribute to seizures, nausea, urination, body odor, and more.\nC. The longer a duck is in the water the sicker it gets. These symptoms can include : [substeps] Undigested food in the guts.\nD. The duck will not return to normal the following morning. The symptoms first occur when the duck takes off from the water.","prompt":"[header] How to treat botulism in a duck [title] Watch for the symptoms. [step] Botulism poisons the duck, causing what's sometimes known as \" limberneck disease. \" it causes paralysis, starting with the duck having trouble taking off or going under the water.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6295,"error":null,"output":"A"}
{"question_id":"hellaswag:00004451","item_index":107,"attempt":0,"prompt_hash":"a7844846a69f1aeb8735ad3340f4432e77880e4d16a7bead76d052459ae30720","question":"[header] How to cope with panic attacks [title] Do some grounding exercises. [step] When in the middle of a panic attack, it's likely you will struggle to breathe naturally. The best way to work through a panic attack is to turn your attention to your breathing.\n\nA. You can simply take a few deep breaths in and out without feeling any pain. To perform a grounding exercise : [substeps] Reach under your desk and grab a book with a cover.\nB. Here are some things to start with : [substeps] Get on the floor. Spread your feet apart, so that you are almost fully upright on all fours.\nC. Focusing on your breath and learning to deepen it will help you relax and work through the panic attack. Breath awareness can end a panic attack and decrease their frequency overall.\nD. Do these exercises for several minutes at a time : [substeps] Breathe in slowly, hold your breath for five seconds, then exhale. Breathe out slowly through your nose four times.","prompt":"[header] How to cope with panic attacks [title] Do some grounding exercises. [step] When in the middle of a panic attack, it's likely you will struggle to breathe naturally. The best way to work through a panic attack is to turn your attention to your breathing.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6294,"error":null,"output":"D"}
{"question_id":"hellaswag:00006245","item_index":108,"attempt":0,"prompt_hash":"4f255bf89a2891b9508197a2b0ea62d9aff634111f5548420cc5edfa3e7a5eaf","question":"[header] How to tell the difference between prokaryotes and eukaryotes [title] Obtain a specimen slide. [step] Specimen slides of prokaryotes and eukaryotes can be obtained from biological supply companies. [substeps] If you are in school, ask your science teacher if they have access to slides.\n\nA. [title] Place your specimen slide on the microscope stage (the platform that holds the slides). [step] Some microscopes will have stage clips that allow you to secure the slide in place to prevent movement during focusing and viewing.\nB. During a science study or activity that occurs in the classroom, we will be asked what type of microscope, computer, cd/dvd or digital camera is used. If not, ask the librarian where there is one.\nC. If they don't, they'll let you get one in a different, more requested process. The specimen slides will be engraved on chalkboard with measurements, with images, and in acrylic epoxy.\nD. If they do, you may not want to share the slide as you do not want someone to see your prokaryotes sample, but keep a journal of anything you find they share. If you go, you can carry the specimen (s) and record these onto another slide.","prompt":"[header] How to tell the difference between prokaryotes and eukaryotes [title] Obtain a specimen slide. [step] Specimen slides of prokaryotes and eukaryotes can be obtained from biological supply companies. [substeps] If you are in school, ask your science teacher if they have access to slides.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6295,"error":null,"output":"A"}
{"question_id":"hellaswag:00009181","item_index":109,"attempt":0,"prompt_hash":"d4c719b603779cb510923a139b1d5c9b02e393f40751ffc041529c323f0aaaae","question":"[header] How to make a vest [title] Take your measurements. [step] You'll need to measure your bust and the dimensions needed for the armholes of the vest. [substeps] To measure your bust, simply wrap a tape measure around the widest part of your chest.\n\nA. Keep the tape measure parallel to the ground as you wrap it around your torso. Measure the armhole depth by measuring from the top of your shoulder to the bottom of your armpit.\nB. Using the tape measure, draw a perfect circle around your bust. Then, use the tape measure and cut around that circular area.\nC. At a right angle, about 8 inches (20. 3 cm) between the bust measurement and the measurement of the shoulder area, find the number of feet you'd need for each armhole.\nD. This measure can be found on your chest, right below the belly button. Once you've taken the tape measure over the widest part of your chest, subtract about 3.5 inches (118 cm) from that measurement.","prompt":"[header] How to make a vest [title] Take your measurements. [step] You'll need to measure your bust and the dimensions needed for the armholes of the vest. [substeps] To measure your bust, simply wrap a tape measure around the widest part of your chest.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6294,"error":null,"output":"A"}
{"question_id":"hellaswag:00009237","item_index":110,"attempt":0,"prompt_hash":"21263fd93517d43843619a39276d7b69e04675f214eef1f7e0db4f160ff2192d","question":"[header] How to solo travel to chile [title] Decide how you will get to chile. [step] Start by figuring out how you will get to chile. If you live in north america, you may decide to fly into a major city in the country, such as santiago, and then take public transit to get around or fly within chile.\n\nA. If you live in a rural area or you do not plan to travel for a long time, you may opt to take a bus. Using a bus or subway to get around chile is a good route to travel.\nB. If you live in australia, you will need to negotiate a road trip, such as a train or bus, to get around chile. [substeps] Plan out the route in advance of arrival so that you can do the same to chile in the future.\nC. If you live in a smaller area, or if you live near a large tourist attraction, you may decide to fly in the opposite direction. [substeps] Skiing, mountain climbing, and bicycle riding are examples of solo travel.\nD. If you live in south america, it may be possible to take a bus or a train into chile, depending on your budget and your timeframe for the trip. [substeps] There is no special visa required for you to travel into chile and no fee to cross the border into chile.","prompt":"[header] How to solo travel to chile [title] Decide how you will get to chile. [step] Start by figuring out how you will get to chile. If you live in north america, you may decide to fly into a major city in the country, such as santiago, and then take public transit to get around or fly within chile.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6295,"error":null,"output":"D"}
{"question_id":"hellaswag:00003674","item_index":111,"attempt":0,"prompt_hash":"5d5c73836b3a9d40eaf385763dd4f1331dab6e9a87c02a9713f13ab2f224b358","question":"[header] How to manually recover music from your ipod [title] Make sure itunes is installed on your computer. [title] Open the itunes folder (located in your \" my music \" folder) and delete everything except the \" itunes music \" folder. [step] This will prevent itunes from erasing any songs on your ipod that are not listed in itunes.\n\nA. [title] Go to the \" music \" folder in itunes. [title] Select \" music \" if prompted, or \" import songs.\nB. If they are, copy them and install them into your itunes library. [title] Get itunes version 4.0.6.\nC. [title] Connect your ipod to your computer and open itunes. [title] When the message asking if you want to erase everything on your ipod and synchronize with the new library, click \" cancel \".\nD. [title] Click the \" ipod \" tab under \" my music. [step] \" this will open the itunes sync window.","prompt":"[header] How to manually recover music from your ipod [title] Make sure itunes is installed on your computer. [title] Open the itunes folder (located in your \" my music \" folder) and delete everything except the \" itunes music \" folder. [step] This will prevent itunes from erasing any songs on your ipod that are not listed in itunes.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6296,"error":null,"output":"C"}
{"question_id":"hellaswag:00000861","item_index":112,"attempt":0,"prompt_hash":"322ebc6a9f9b7edc6b8ffcea14308a77291ff784456de0d2c1945668fe6f78dd","question":"A man is seen standing ready outside a track and begins running towards a bar. the man\n\nA. then throws a long pole across the distance in slow motion and watches himself jump into the sand.\nB. places his hands on the bar continuing to perform several tricks.\nC. then jumps over the bar and is shown again in slow motion.\nD. then takes a giant leap into the air and begins performing a gymnastic routine.","prompt":"A man is seen standing ready outside a track and begins running towards a bar. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6298,"error":null,"output":"C"}
{"question_id":"hellaswag:00004398","item_index":113,"attempt":0,"prompt_hash":"3455e6e522c1d75be3d20daf642c7c333eefc58cc74eab28f54ec9ae9ef66bed","question":"[header] How to make an all natural facial at home [title] Pull back you hair and wear a comfortable t-shirt. [step] You're going to be making and using a scrub, a steam, and a mask. It's best to make sure your hair is out of your face so it doesn't get in your way.\n\nA. [substeps] If you don't want to be bothered, use some water. You don't want your skin to get oily.\nB. The steam technique is very important because the smell can cause blackheads and other blemishes. [substeps] To create an all natural facial, wash your hair the day before.\nC. Put on a comfy t-shirt that you don't mind splattering with different facial products. [title] Use a gentle cleanser to wash your face.\nD. [substeps] For a less dangerous facial to use at home, you can substitute a chapstick. Use this on the inside flap of your lip.","prompt":"[header] How to make an all natural facial at home [title] Pull back you hair and wear a comfortable t-shirt. [step] You're going to be making and using a scrub, a steam, and a mask. It's best to make sure your hair is out of your face so it doesn't get in your way.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6297,"error":null,"output":"C"}
{"question_id":"hellaswag:00007043","item_index":114,"attempt":0,"prompt_hash":"84776f088a13bd67f3a56c5d9c94f59f0abd90f68fe17fd0a97fd50017be7642","question":"[header] How to identify pearls in vintage jewelry [title] Look for rings around the drill hole. [step] A fake pearl will not have any rings or or ridges around a drill hole. Put the pearls under a light and use a magnifying glass to inspect the drill hole.\n\nA. If the holes are smooth, then the pearls are likely fake. [title] Examine for variations in size and color.\nB. Be careful not to squeeze your fingers too much or else you may break the actual pearl. [substeps] Your fake pearl will have rings, ridges, engraving under it.\nC. You may also see splashes of flesh. Pearls on a drilling ground are usually collected from spills, not from hitting the drill with the jewelry hammer.\nD. You can also check for tiny pearls by scanning the graduated tip of the drill. This should reveal a tiny pearl attached to a thread.","prompt":"[header] How to identify pearls in vintage jewelry [title] Look for rings around the drill hole. [step] A fake pearl will not have any rings or or ridges around a drill hole. Put the pearls under a light and use a magnifying glass to inspect the drill hole.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6295,"error":null,"output":"A"}
{"question_id":"hellaswag:00002291","item_index":115,"attempt":0,"prompt_hash":"9eaa167aa8e581f320a6050adef8537b9383de91e96061e4558458818b1fa255","question":"Woman is walking in a green grassy backyard. Man is standing in a bacyard in front of a fence with a machine in front of him. woman\n\nA. is in a field doing acrobatics on the soil.\nB. is standing in front of a white house holding the machine and is showing how to use it.\nC. is in the backyard and a man is behind her using a lawn mower to cut the grass.\nD. is standing in a house standing next to a pile of cement and weedwacklers.","prompt":"Woman is walking in a green grassy backyard. Man is standing in a bacyard in front of a fence with a machine in front of him. woman","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6297,"error":null,"output":"C"}
{"question_id":"hellaswag:00000827","item_index":116,"attempt":0,"prompt_hash":"8bb19542b84b12a3e3b6568a804d7d163cb6ce021291cbe6100cea86711ffa4a","question":"Man is on the 10 m trampoline and make a hand stand to make the jump to the pool. Woman is impressed siting by the pool and have her hands on her face. presentator\n\nA. is talking about the argument between mike tyson of swansboro and smith of egg plants.\nB. is sitting thng a round table.\nC. is want to talk about the important part of this ad invest cyou and one person makes the jump to the pool.\nD. is interviewing the man.","prompt":"Man is on the 10 m trampoline and make a hand stand to make the jump to the pool. Woman is impressed siting by the pool and have her hands on her face. presentator","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6296,"error":null,"output":"D"}
{"question_id":"hellaswag:00003421","item_index":117,"attempt":0,"prompt_hash":"f6707b5219d3995b5db98a906d5dc9f36faac01b4dd6e7f3f886a8fe00d000cb","question":"[header] How to tell if your child has a learning disability [title] Talk to your child's teacher. [step] Most children with learning disabilities perform inconsistently in math, reading and writing. [substeps] Find out if your child is performing at the expected level, or if he or she is below the level of other classmates.\n\nA. You can call your child's teacher just to check his or her level. These talks often span several hours or more and require answering questions across multiple lines.\nB. [title] Notice whether your child can stay organized or if he has trouble managing his time and completing assignments or homework. [title] Pay attention to your child's ability to remember information.\nC. [title] Observe your child's social interactions with others. [step] In addition to social interaction, eye contact is often the main indicator that something is wrong with your child.\nD. Also see what the typical learning length for at-home disabled students is. If the maximum attainable grade for your child has been reached, you can get your child certified as impaired instead of needing treatments or professionals to correct it.","prompt":"[header] How to tell if your child has a learning disability [title] Talk to your child's teacher. [step] Most children with learning disabilities perform inconsistently in math, reading and writing. [substeps] Find out if your child is performing at the expected level, or if he or she is below the level of other classmates.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6297,"error":null,"output":"B"}
{"question_id":"hellaswag:00006766","item_index":118,"attempt":0,"prompt_hash":"d2843247aa1f945f2bacd9e9a21a5f695e1d1a0c94b7e290777a37360d11b9d0","question":"[header] How to throw a bake off party [title] Plan on who you will invite. [title] Plan a date. [title] Send out invitations a few weeks in advance so they will be prepared and maybe not plan anything else for that day.\n\nA. [title] Remind them how far in advance the party will be. [title] Pack some more food for that day.\nB. [title] Give each guest special wishes and wish them well. [step] Some examples are: pets, family, favorite foods.\nC. [title] Have a basic idea of how many people you will invite. [title] Make sure you have excellent lighting and have a schedule.\nD. [title] Plan on what you will bake. [step] The host will usually participate in events of a party.","prompt":"[header] How to throw a bake off party [title] Plan on who you will invite. [title] Plan a date. [title] Send out invitations a few weeks in advance so they will be prepared and maybe not plan anything else for that day.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6800,"error":null,"output":"C"}
{"question_id":"hellaswag:00002044","item_index":119,"attempt":0,"prompt_hash":"4bcdc9483f1fc2cc5af39bfddd13385b4b03f5271d548a8b6e8e12ffac2ccc65","question":"The man grabs his fishing pole and starts fishing in the hole in the ice. The man appears to have caught something and kneels down to pull his fishing line by hand. the man\n\nA. releases the fishing line and drops the fishing pole.\nB. grabs his pole and pulls several fish out of the hole.\nC. stops fishing and emerges to speak to the camera.\nD. pulls a fish out of the water and poses with it.","prompt":"The man grabs his fishing pole and starts fishing in the hole in the ice. The man appears to have caught something and kneels down to pull his fishing line by hand. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6093,"error":null,"output":"D"}
{"question_id":"hellaswag:00005960","item_index":120,"attempt":0,"prompt_hash":"d9ea06e7d5835127d6028ab747918035ea637eadb40a9c30e63ff5e319bfad55","question":"[header] How to install a car seat [title] Separate the carrier from the base. [step] Most rear-facing seats consist of two parts: the base and the infant carrier. You will install the base, the part that remains in the car.\n\nA. [substeps] Check to make sure the base remains inside the vehicle, and on the exterior of the vehicle, but not in any of the rear-facing seats. Lift the base out of the backseat and place it on a table in front of you.\nB. Consider moving the base out of the vehicle once you have finished installing the seat. This piece will sit in the trunk and provide your infant with the seat.\nC. Follow the manufacturer's instructions for removing the carrier from the base. Usually, there is simply a lever or button to activate.\nD. [title] Test for bias with the index finger of one hand on the infant carrier's base. [step] You will use your other thumb to separate the base and the upper corner.","prompt":"[header] How to install a car seat [title] Separate the carrier from the base. [step] Most rear-facing seats consist of two parts: the base and the infant carrier. You will install the base, the part that remains in the car.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6092,"error":null,"output":"C"}
{"question_id":"hellaswag:00002834","item_index":121,"attempt":0,"prompt_hash":"d50545f24bb4993cdc2fdd6e571cdeee12fa2b079e20d51296252299842f92e8","question":"A man is talking to the camera from a parking lot. He runs down the street while continuing to talk. he\n\nA. is then shown instructing a young girl in lingerie on how to do push ups.\nB. shows off a rollerblade in the parking lot.\nC. runs onto several city streets, not stopping.\nD. gets a red bike and discusses its features.","prompt":"A man is talking to the camera from a parking lot. He runs down the street while continuing to talk. he","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6092,"error":null,"output":"A"}
{"question_id":"hellaswag:00000877","item_index":122,"attempt":0,"prompt_hash":"ec7ad35802febd51b8b8ddbc5a1615e74911a874df1966ba32b4a2086de45bb8","question":"A person is seen passing a basketball all around a court to others and ending by shooting a basket. the group\n\nA. continues to play back and fourth on their basketball around the court.\nB. continues playing basketball while looking forward and stopping to speak.\nC. continues with other youth playing the game and taking pictures to show around.\nD. continues to pass the ball around to one another performing drills while the coach instructs.","prompt":"A person is seen passing a basketball all around a court to others and ending by shooting a basket. the group","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6090,"error":null,"output":"D"}
{"question_id":"hellaswag:00002687","item_index":123,"attempt":0,"prompt_hash":"687d297d8948081bf5435cf8022824bde6fa53798fc5d9a804810a421fcfede5","question":"We see a lady from china run and perform a high jump and see her score and we see her manager and a slow motion. An different lady from uzbekistan performs a high jump and we see her score. the first lady\n\nA. jumps hard in fifteen seconds and scores, we see the closing screen.\nB. runs off the mat and trampolines and shakes her pom poms right and left.\nC. stands in front of the stage and smiles and says something.\nD. performs again and knocks the bar over and looks disappointed.","prompt":"We see a lady from china run and perform a high jump and see her score and we see her manager and a slow motion. An different lady from uzbekistan performs a high jump and we see her score. the first lady","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6104,"error":null,"output":"D"}
{"question_id":"hellaswag:00009729","item_index":124,"attempt":0,"prompt_hash":"83cd059d8cd5547d258e38435c278e5a5562d90c5b7bfd76146aaaba699a7fd2","question":"[header] How to get your dog to take his medicine [title] Have your dog sit. [step] If the dog is small then work at a comfortable height for you, by putting the dog on a table. Lay down a blanket or towel so the dog feels more secure.\n\nA. You can also wrap your dog in a newspaper or a cloth. [substeps] You can also place the dog's bed or couch in a baby carrier.\nB. Sit on the edge of the curb or on the ground or some other simple step place with support. Take a small dose of vitamin c or caffeine, then switch to a new one each day, about a week later.\nC. For medium to large-sized dogs, work with the dog on the floor but back him into the corner of a room so he can't try to run away from you. [substeps] If you can, have another family member or friend hold the dog steady by the shoulders.\nD. You don't want the dog to be in a wheelchair and pinned so your pet cannot stand. Place another blanket or towels on top of the newborn puppy so that the room is cold and just comfortable.","prompt":"[header] How to get your dog to take his medicine [title] Have your dog sit. [step] If the dog is small then work at a comfortable height for you, by putting the dog on a table. Lay down a blanket or towel so the dog feels more secure.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6104,"error":null,"output":"C"}
{"question_id":"hellaswag:00000233","item_index":125,"attempt":0,"prompt_hash":"1730b95c21dad377601d52fc397e7d3a145906767ca1afe3c1070c37687f448d","question":"A man professionally paints several pieces of furniture using a spray painter in a paint shop while wearing overalls, a denim shirt and a paint mask. the man\n\nA. begins by painting a wooden armchair metallic gray with a gold spray paint can.\nB. then paints a floor with buckets of paint and he uses several buckets to paint the room.\nC. gently glides the paint brush along more of the furniture making white lines on the fine rectangular piece of furniture.\nD. drops the paint painting and talks to the camera for a while holding a paintbrush and showing off various brushes and sketches.","prompt":"A man professionally paints several pieces of furniture using a spray painter in a paint shop while wearing overalls, a denim shirt and a paint mask. the man","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6104,"error":null,"output":"B"}
{"question_id":"hellaswag:00001064","item_index":126,"attempt":0,"prompt_hash":"7a519511a5d25f6df8a18932788802e21f285f42d0bd849d891a7981497e53d4","question":"The woman is spraying water on the small dog with a hose. She is now blow drying the dog. the dog\n\nA. is still in the shower with the left hand holding the silver hose and is move the right hand to rinse the top part of the dog.\nB. is placed in the kennel next to a woman's feet.\nC. starts moving in the clothes.\nD. is doing tricks trying to stay calm.","prompt":"The woman is spraying water on the small dog with a hose. She is now blow drying the dog. the dog","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6103,"error":null,"output":"A"}
{"question_id":"hellaswag:00009648","item_index":127,"attempt":0,"prompt_hash":"0d54f8c505b1f05ea942efc318c8d569b92e54272a6c04e087abe432f0cec5d8","question":"[header] How to talk to a guy who doesn't like you anymore [title] Ask yourself if it's really necessary. [step] If he doesn't like you anymore, accept the fact that he probably doesn't want to shoot the breeze. Save yourself from any pain or embarrassment.\n\nA. Chances are you won't hurt his feelings; it's your job to be patient. But sometimes you have to say something to him (like a pull-over, tell him you're not interested in him anymore) and figure out which way to proceed.\nB. Take a step back and think about whatever it is you want to talk to him about. [substeps] If it's something that you really need to discuss, go ahead and do so.\nC. If he told you that you have a new girlfriend and you think you have nothing to worry about, then you've made a bad decision. He's not interested in you any more.\nD. [substeps] If he is interested, then the best answer is no. This will give you some time to grieve over the rejection.","prompt":"[header] How to talk to a guy who doesn't like you anymore [title] Ask yourself if it's really necessary. [step] If he doesn't like you anymore, accept the fact that he probably doesn't want to shoot the breeze. Save yourself from any pain or embarrassment.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6103,"error":null,"output":"B"}
{"question_id":"hellaswag:00008947","item_index":128,"attempt":0,"prompt_hash":"1573eeee510f3cfdee58152cf5e7a6e4a22050e2775113e3ba62cbb52ac3a306","question":"[header] How to look like a ganguro girl [title] Dye your hair : [step] Either blonde, orange, silver, gray, gold or white, the choice is yours. You may also like to add complimentary highlights or pink hair extensions. If you don't want to dye your hair, you could always buy a wig.\n\nA. Dyed hair in japan is known as chapatsu hair. Also, it is popular to not wash your hair often.\nB. You can ask your parents for one and see what happens. [title] After all that, do what you want.\nC. [substeps] Model the look onto a photo or on a fashion magazine. Style your hair in a plunging neckline or pull it back into a ponytail or pigtails, then tie it with a satin ribbon or even a loose half-up.\nD. [title] Although the sex on this list isn't very identifiable, you may want to try various hairstyles. [step] For example, if you're a brunette, consider colour-blocking your hair hair into washable pink, gold, red, beige or white blonde.","prompt":"[header] How to look like a ganguro girl [title] Dye your hair : [step] Either blonde, orange, silver, gray, gold or white, the choice is yours. You may also like to add complimentary highlights or pink hair extensions. If you don't want to dye your hair, you could always buy a wig.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5786,"error":null,"output":"B"}
{"question_id":"hellaswag:00001157","item_index":129,"attempt":0,"prompt_hash":"bcfc17e9bf3337b706280f58a9a5ae9e90a49697976c722f86e4e58efedd6116","question":"A man demonstrates how to spray varnish a chest of drawers using a spray nozzle varnish hose. A man sprays a chest of drawers with varnish while standing barefoot outdoors. the man\n\nA. tables the dresser with a bottle of oil, and then use the brush to smooth out the wax on the dresser surface.\nB. then takes his shirt off and applies a tattoo to a dog upside down.\nC. covers the entire chest from the very top to the bottom with the spray varnish which is coming from a long hose attached to a metal pump.\nD. dries the floor with the hose.","prompt":"A man demonstrates how to spray varnish a chest of drawers using a spray nozzle varnish hose. A man sprays a chest of drawers with varnish while standing barefoot outdoors. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5792,"error":null,"output":"C"}
{"question_id":"hellaswag:00001911","item_index":130,"attempt":0,"prompt_hash":"dae9510cddaf2d69ed6bd0c0d3caa8f77fa5dbdb1485e18dda24826d23ee93ec","question":"A man is standing in front of a building playing the bag pipes. people\n\nA. are standing around watching him.\nB. are playing golf on a court behind him.\nC. are dancing in front of the building.\nD. are watching from the side as the man continues to play.","prompt":"A man is standing in front of a building playing the bag pipes. people","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5791,"error":null,"output":"A"}
{"question_id":"hellaswag:00003194","item_index":131,"attempt":0,"prompt_hash":"7bd0e99725dd83999d0f3ac496e6475c4aaf0915d4800acc8aeabc1d08cbb01b","question":"A large red slack line is connected to two trees out in a park. a strong young man\n\nA. is lifting heavy weights over his head.\nB. is shown on the other side of the tree pulling the line to ensure that is the tightest it can be around the trunk of the trees.\nC. takes a stunt high, letting the rope go and goes flying.\nD. combs the slack line one handed at a time along the trees in a park and cuts system of starts with three never ending lines.","prompt":"A large red slack line is connected to two trees out in a park. a strong young man","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5789,"error":null,"output":"A"}
{"question_id":"hellaswag:00006043","item_index":132,"attempt":0,"prompt_hash":"dd6e7afdb5e7a18974a76dcf485ee663bbae3ebf01b153e4d197db70be26e70a","question":"[header] How to make basic fondant for candies [title] Stir the sugar and water in a saucepan. [step] Set on the back part of the range, until the sugar is melted, then draw the saucepan to a hotter part of the range, and stir until the boiling point is reached. [title] Add the cream of tartar or acid and, with the hand or a cloth wet repeatedly in cold water.\n\nA. [step] Wash down the sides of the saucepan, to remove any grains of sugar that have been thrown there. [title] Cover the saucepan and let boil rapidly three or four minutes.\nB. [step] Pour more cream into the saucepan, and continue stirring. This helps prevent the cream from clumping up into solids.\nC. [step] Stir for a minute, then fill a deep pot with water. Bring this to a boil, then add half of the sugar.\nD. [step] ( there are many basic fondant recipes so look into them). [title] Turn on the low flame and select white flour for a smooth, non-starchy consistency.","prompt":"[header] How to make basic fondant for candies [title] Stir the sugar and water in a saucepan. [step] Set on the back part of the range, until the sugar is melted, then draw the saucepan to a hotter part of the range, and stir until the boiling point is reached. [title] Add the cream of tartar or acid and, with the hand or a cloth wet repeatedly in cold water.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5785,"error":null,"output":"A"}
{"question_id":"hellaswag:00004213","item_index":133,"attempt":0,"prompt_hash":"fd9396d19e425a4ce7708e7774256b24d37574b2b7a0c4f7f5bd6a6a8cee65d7","question":"[header] How to defend yourself in an online data breach lawsuit [title] Read the complaint. [step] A lawsuit is started when the plaintiff files a complaint and serves it on you (the defendant). A complaint contains the allegations against you, the relief sought, and other technical information telling the court that the lawsuit is permissible (e.g., standing).\n\nA. If you are served with a complaint, read it carefully and plan your next move. [substeps] The complaint will likely allege that you or your company in some way acted negligently or breached some fiduciary duty you owed to the plaintiff.\nB. If you have information being flamored in court that you want to undermine, then you should read the complaint. [substeps] The complaint might be regarding the company that provided the data the plaintiff claims was harmful.\nC. You should find the complaint and its claim form. [substeps] If you don't have access to a copy of the complaint, then you don't have much to prove.\nD. The plaintiff shows that you have breached the only protection your personal property would have provided before you are injured. To prove this, the complaint contains a strong argument that is alleging you were not protected in an online breach.","prompt":"[header] How to defend yourself in an online data breach lawsuit [title] Read the complaint. [step] A lawsuit is started when the plaintiff files a complaint and serves it on you (the defendant). A complaint contains the allegations against you, the relief sought, and other technical information telling the court that the lawsuit is permissible (e.g., standing).","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5785,"error":null,"output":"A"}
{"question_id":"hellaswag:00000215","item_index":134,"attempt":0,"prompt_hash":"be592a135ab947bc737923c2135ed1b3df20f084ecdc2301a3130eb1855e70f5","question":"A girl is shown washing and scrubbing her face several times while speaking into the camera. she\n\nA. then rides a bike down the hill and gets her hair dried.\nB. takes the lip stick out of her mouth while in the bathroom and begins brushing her teeth while still speaking into the camera.\nC. gets her mascara out and then applies it to her face.\nD. finishes by turning on the faucet and rinsing water through her face.","prompt":"A girl is shown washing and scrubbing her face several times while speaking into the camera. she","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5787,"error":null,"output":"D"}
{"question_id":"hellaswag:00006204","item_index":135,"attempt":0,"prompt_hash":"963b721c8bb485e695a85e0184b6ccf30ca0f4f0c171cc914ea8d3434630f039","question":"[header] How to enter receipts for walmart's savings center via the walmart website [title] Visit the walmart savings catcher website and log in to your walmart online account. [step] Have your store receipt ready with you. [title] Click the box that says \" tc \" below the words \" enter new receipt \" on the right side of the page.\n\nA. [title] Enter the following information in the box : [substeps] Your store receipt. This should be listed under both your store and walmart account numbers in line to the right.\nB. [title] Enter your walmart account details and click the \" next \" button on the upper right of the page. [title] Use the \" remember i use my business name? [step] Your store will be on the left side of the page.\nC. [title] Enter your walmart credit or debit card information. [step] When you first register for walmart, you might just need to use a credit or debit card.\nD. [step] The tc number is the complete 20 digit barcode number of the receipt. [title] Type the barcode number of the receipt in.","prompt":"[header] How to enter receipts for walmart's savings center via the walmart website [title] Visit the walmart savings catcher website and log in to your walmart online account. [step] Have your store receipt ready with you. [title] Click the box that says \" tc \" below the words \" enter new receipt \" on the right side of the page.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5786,"error":null,"output":"C"}
{"question_id":"hellaswag:00009783","item_index":136,"attempt":0,"prompt_hash":"47c0ff5a4d0721818ecc9297dd1a9844187b934fac5b124443010a64c298c886","question":"[header] How to be stylish in a hoodie [title] Choose a plain hoodie in a dark, neutral color. [step] Hoodies are typically considered to be casual garments, but an easy way to instantly dress them up and add some style is by choosing a dark color like black or dark gray. Dark, plain colors will automatically upgrade the hoodie's appearance and create a sleeker look.\n\nA. [substeps] A hoodie should, at minimum, sit between 1 and 3 inches (2.5 and 5 cm) across. [title] Choose a hoodie with a solid color.\nB. If you pair it with dark denim and basic trainers, you can look both polished and casual. [substeps] Try pairing a dark gray hoodie with black skinny jeans.\nC. [substeps] Some brands color up popular hoodies for a more sleek look. Consider including a backless hoodie in a brighter color for adding a casual vibe.\nD. Pick a plain hoodie to work with, or you can always choose a plain one in a neutral color like tan or dark gray. [substeps] Make sure that you are a medium size person, from the waist to feet.","prompt":"[header] How to be stylish in a hoodie [title] Choose a plain hoodie in a dark, neutral color. [step] Hoodies are typically considered to be casual garments, but an easy way to instantly dress them up and add some style is by choosing a dark color like black or dark gray. Dark, plain colors will automatically upgrade the hoodie's appearance and create a sleeker look.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5785,"error":null,"output":"B"}
{"question_id":"hellaswag:00002364","item_index":137,"attempt":0,"prompt_hash":"47b297d556eaa28d5e54eaa8ab7445ba7a55f456c719ba450ad8c8c5a1e8be17","question":"A girl is leaning forward and clipping her hair into place. She continues clipping it in several different areas. she\n\nA. cuts it to a sexy finish.\nB. then washes and rinses her face.\nC. clips and braids her hair followed by putting an eye-shadow over the eye areas.\nD. combs it through with a roller brush.","prompt":"A girl is leaning forward and clipping her hair into place. She continues clipping it in several different areas. she","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5787,"error":null,"output":"B"}
{"question_id":"hellaswag:00007639","item_index":138,"attempt":0,"prompt_hash":"6eb44c90b85a9638d38eb26b56c2446f686a83e994c522009cb4f31553d122be","question":"[header] How to stop making excuses for toxic parents [title] Take a step back from your parents. [step] Get some space so you can gain perspective on your parent's behavior and your relationship with them. If you're around toxic or abusive behavior all the time, you may get used to it.\n\nA. Give yourself a few minutes to breathe before jumping into a more draining conversation. [substeps] Take a moment to sort the information into fresh, concrete terms.\nB. [substeps] Asking people to keep your distance can help you to find a second perspective. If you don't know what to say, ask questions.\nC. Taking a break can help you get a clearer-eyed look at what's wrong. [substeps] If you still live with your parents, creating some distance can be tough.\nD. Try to put yourself in their shoes, and see how you can make a change in yourself. [substeps] Just as many parents try their hand at parenting, they don't always resolve conflicts tactfully and maturely.","prompt":"[header] How to stop making excuses for toxic parents [title] Take a step back from your parents. [step] Get some space so you can gain perspective on your parent's behavior and your relationship with them. If you're around toxic or abusive behavior all the time, you may get used to it.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5783,"error":null,"output":"C"}
{"question_id":"hellaswag:00007343","item_index":139,"attempt":0,"prompt_hash":"2edafe9a1993923db0b0bc75bc750fe098a6e0813bbada349ebfaefdf22e533d","question":"[header] How to grow valerian [title] Purchase valerian seedlings or starts from a garden shop. [step] They will usually be in the herb section of the store. Valerian seeds can be finicky, so this is the best way to ensure you will have young plants for your garden.\n\nA. [title] Cut between 20 and 30 inches (60 and 59 cm) of valerian root. [step] Do not cut larger ones than four or five feet long.\nB. [title] Consider harvesting \" daughter \" or volunteer sections of the root system from older plants. [step] If valerian grows naturally in your area, you can usually find a number of young plants sprouting in the spring.\nC. [substeps] Try looking for seeds that are difficult to care for in the herb section of the store. Some seeds may be tightly-packed, so check the soil to find a few.\nD. You can obtain valerian seeds online or sell them for a price somewhere between $20 and $40. [title] Prepare two large pots with a wire lid.","prompt":"[header] How to grow valerian [title] Purchase valerian seedlings or starts from a garden shop. [step] They will usually be in the herb section of the store. Valerian seeds can be finicky, so this is the best way to ensure you will have young plants for your garden.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5783,"error":null,"output":"B"}
{"question_id":"hellaswag:00009280","item_index":140,"attempt":0,"prompt_hash":"5be5426c9efabb709a664cbb3ce097e1e83255368646a6b8dfc21547b620e3d8","question":"[header] How to protect your watch [title] Wear any bracelets or chains on your other wrist. [step] Wrist jewelry has the potential to scratch either the sides or the face of your watch. Only fabric or thin leather bracelets would be acceptable along with a watch.\n\nA. There are some very simple homemade jewelry choices, with necklaces, neck bands, rings, etc. , used to protect your watch from scratches and corrosion.\nB. If you have friends or family who also have watches, then they can help protect you from scratching. [title] Protect your watch more than you wear it.\nC. Avoid metal on the same wrist, no matter what. Friendship bracelets, knitted and crocheted bracelets are not harmful either.\nD. Avoid wearing silver or gold bracelets on your other wrist. Lighter wrist settings will wear away the scratched effect of your watch.","prompt":"[header] How to protect your watch [title] Wear any bracelets or chains on your other wrist. [step] Wrist jewelry has the potential to scratch either the sides or the face of your watch. Only fabric or thin leather bracelets would be acceptable along with a watch.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5782,"error":null,"output":"B"}
{"question_id":"hellaswag:00007209","item_index":141,"attempt":0,"prompt_hash":"7248078c541aa4042c311852fb4807c10721d4e5d9397f11ff6b547f2270c58f","question":"[header] How to make an automatic chicken feeder [title] Purchase a cheap large bin. [step] Aim to pick up a big bin without wheels. Because the feed holes will be placed near the bottom, a wheeled bin might get in the way of the feed holes you'll want to make.\n\nA. Larger bins will be better for making the feeder. [substeps] A gallon-sized bin fits best; larger bins will make it easier to hold food.\nB. Most feeders are sold with wheels on only some wheels. If yours doesn't have wheels, you'll need a jar with a handle on the left side that also contains wheels.\nC. [substeps] Avoid buying a bin that you see will get stuck. If you can get one without wheels, then use a wheel to make it easier for you to dump your hay in the bin.\nD. A 30 gallon trash bin should be enough to hold 150 pounds of chicken feed. [substeps] While you're at a hardware store, pick up six pipe elbows and six pipe end caps.","prompt":"[header] How to make an automatic chicken feeder [title] Purchase a cheap large bin. [step] Aim to pick up a big bin without wheels. Because the feed holes will be placed near the bottom, a wheeled bin might get in the way of the feed holes you'll want to make.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5784,"error":null,"output":"D"}
{"question_id":"hellaswag:00001114","item_index":142,"attempt":0,"prompt_hash":"b167dcf0d603a7764411e7e7be09bdeb30ebdf90f1a29ae70ae54160d71b2d46","question":"A blurry black and white video plays of a woman opening glass doors as the video clears up and she enters the business and she sits down on a chair. the woman\n\nA. then sparkles the tennis ball with a toothpick.\nB. talks about a pinata and then she takes off her gloves and brushes her hair.\nC. is then shown sitting down with a male hairdresser behind her fussing with her hair, measuring it, washing it, and rinsing it.\nD. slowly food her beef and garlic in a white skillet.","prompt":"A blurry black and white video plays of a woman opening glass doors as the video clears up and she enters the business and she sits down on a chair. the woman","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5785,"error":null,"output":"B"}
{"question_id":"hellaswag:00005923","item_index":143,"attempt":0,"prompt_hash":"e80469e189ddc162b14f3164e89cda094be72dc5f84bd7e407fd93de274f5225","question":"[header] How to make peppermint tea [title] Pick the peppermint. [step] For each serving of tea you plan on making, pick 7-10 peppermint leaves from the stem of the plant. Try to pick leaves that are green and unblemished.\n\nA. These produce a more pleasant and robust flavor. [substeps] Look for colored peppermint leaves in your local garden store.\nB. If you bought a bunch of pre-cut peppermint stems, choose the healthiest looking leaves to pick off. [title] Rinse the peppermint leaves.\nC. Choose resinous or black peppermint leaves, as those tend to produce the best pollen. [title] Fill 2 clean tea bags with water.\nD. Look for basil that has a light green color instead of yellowish or brown leaves. Basil has good healing properties because it kills very fast.","prompt":"[header] How to make peppermint tea [title] Pick the peppermint. [step] For each serving of tea you plan on making, pick 7-10 peppermint leaves from the stem of the plant. Try to pick leaves that are green and unblemished.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5784,"error":null,"output":"B"}
{"question_id":"hellaswag:00000893","item_index":144,"attempt":0,"prompt_hash":"80eae0ef0dda2eb7bcd12ed85ea0b957cab274a0e266026a2b6136a455ccc775","question":"We see kids with feet on skateboards. we\n\nA. see the kids riding down the street from a skater pov.\nB. see a boy rollerblading.\nC. see a woman holding part of a hat.\nD. see a kitchen and a doctor.","prompt":"We see kids with feet on skateboards. we","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5786,"error":null,"output":"B"}
{"question_id":"hellaswag:00007069","item_index":145,"attempt":0,"prompt_hash":"f616cbfce84820caf809ed70ea6d82884d28fc82069e565a3090fac748858b82","question":"[header] How to make new year's resolutions [title] Be clear about what you want. [step] For example, if you want to take better care of yourself this year, what does that look like? Perhaps you are ready to quit smoking, to take on a consistent exercise regimen, or to eat healthier. [title] Understand why you want it.\n\nA. [step] Why is it important to you to take better care of yourself? Perhaps you want to set a good example for your children or grandchildren. Maybe you're concerned about aging or hereditary diseases, and you're taking preventative measures.\nB. [step] You're entitled to some particular things, and your resolution is to earn them. By this point, you may not have time for more important goals.\nC. [step] You have spent a lot of time visualizing changes your life would make, and so you will want to focus on achieving those goals. The important thing is to be clear why you want something more than what you are asking of yourself.\nD. [step] Before committing to change, be clear with yourself why you want to change your life. [substeps] What does this idea or concept mean? Is your goal a drastic change to your attitude or lifestyle? If yes, then not yet.","prompt":"[header] How to make new year's resolutions [title] Be clear about what you want. [step] For example, if you want to take better care of yourself this year, what does that look like? Perhaps you are ready to quit smoking, to take on a consistent exercise regimen, or to eat healthier. [title] Understand why you want it.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5786,"error":null,"output":"A"}
{"question_id":"hellaswag:00008959","item_index":146,"attempt":0,"prompt_hash":"8531fff83109d707f91f17eeab8b04bfb249c11f82d6f53387a9eefc83ab5d76","question":"[header] How to roll fondant [title] Work on a clean, flat surface. [step] In many cases this will be your counter top. A marble or wooden chopping board works well if your counter is irregular in shape or has trenches in it from grouting.\n\nA. An alternative is a writing utensil such as a spatula to work with; this will give you a broad, flat surface to work on and also provides you a moist surface to work on. [title] Choose a cutting board for your fondant.\nB. Irregularities in your counter can make your fondant irregular when rolling. [substeps] Depending on the amount of fondant you're working with, you might only need a small area to work in.\nC. The roughness of the stone can make the method easier. [title] Chop your fondant dough into fits about an inch smaller than your pie filling.\nD. [substeps] Use a 220 grit pitcotta grinder for a smooth bottom. Add buttercream or sealant to the bottom and in the corners to give your workbench a nicer, finishing finish.","prompt":"[header] How to roll fondant [title] Work on a clean, flat surface. [step] In many cases this will be your counter top. A marble or wooden chopping board works well if your counter is irregular in shape or has trenches in it from grouting.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5786,"error":null,"output":"B"}
{"question_id":"hellaswag:00007756","item_index":147,"attempt":0,"prompt_hash":"67a1826fc297357d4bfc029c6ff983b249289315767cf319de28eb90c047a822","question":"[header] How to join a clan in clash of clans [title] Tap on the menu button on the left side of the screen to access chat. [title] Tap on the \" global \" tab. [title] Compose a message explaining that you want to join a clan, and post your message.\n\nA. [title] Add a new member or tell the target the type of clan you'd like to join. [step] You'll also be informed about other clans nearby and added members that you might have brought in recently.\nB. [title] Tap on the \" log in \" button at the bottom right corner of the screen to log into your account, if you are a current member of clash of clans. [title] Tap on the \" all \" button, which is located to the left of the contact info screen.\nC. [step] All users logged in to clash of clans will have the ability to view your message. In most cases, there will be other players looking for new clan members.\nD. [title] Send the message using the \" in game \" option. [title] Tap on the chat bubble next to \" message forwarding.","prompt":"[header] How to join a clan in clash of clans [title] Tap on the menu button on the left side of the screen to access chat. [title] Tap on the \" global \" tab. [title] Compose a message explaining that you want to join a clan, and post your message.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5785,"error":null,"output":"C"}
{"question_id":"hellaswag:00008196","item_index":148,"attempt":0,"prompt_hash":"6f373b01d3a20f81b111869b77c74e5898e68a4a3d9759fecec13c9488f1b97e","question":"[header] How to read pedigrees [title] Figure out the orientation of the pedigree. [step] Pedigrees can be laid out in a variety of orientations. Assess whether the chart is oriented either vertically or horizontally on the page.\n\nA. Look at the orientation of the words to understand which direction it should be held. [title] Trace lines in the pedigree.\nB. [substeps] Pedigrees are generally placed at different heights, from 1 degree to 12 degrees. If the chart is tilted upwards on one height, then it tells you how high it is.\nC. As your child grows and begins acquiring pedigrees, the chart you will need will be oriented horizontally and vertically. [title] Learn what each pedigree means.\nD. You can also ask your book store employee to help you look for the chart. [substeps] Each pedigree may have a specific orientation, often with many different aspects, that the reader is most familiar with.","prompt":"[header] How to read pedigrees [title] Figure out the orientation of the pedigree. [step] Pedigrees can be laid out in a variety of orientations. Assess whether the chart is oriented either vertically or horizontally on the page.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5784,"error":null,"output":"A"}
{"question_id":"hellaswag:00009063","item_index":149,"attempt":0,"prompt_hash":"71775593777ed1e1c10e142bbe346611678308737c62f5f20fcb2b61a78b73c2","question":"[header] How to discourage skunks in your garden [title] Discourage skunks in your garden by eliminating their access to the food source. [step] The best approach is to fence your garden. Put cages around the plants and cover the berries with netting.\n\nA. Closing or blocking off the plants is crucial to repel skunks away from food sources that it is unlikely it will find. [title] Place charcoal in the center of the garden.\nB. You can also put electric fencing on the plants, separating them. For particularly hungry skunks, the first thing you should do is put a fence around the garden.\nC. Also, install mulch to keep the plants protected, although it does not prevent skunks. It is too cold to mulch, so do not disturb your plants until it has cooled down.\nD. [title] Choose species of plants that repel skunks. [substeps] Plant squash around the perimeter of your garden.","prompt":"[header] How to discourage skunks in your garden [title] Discourage skunks in your garden by eliminating their access to the food source. [step] The best approach is to fence your garden. Put cages around the plants and cover the berries with netting.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5785,"error":null,"output":"D"}
{"question_id":"hellaswag:00007355","item_index":150,"attempt":0,"prompt_hash":"b948c07e26404af71a4b55787aa4e56f6d8163af64e388205886012ba470a518","question":"[header] How to teach practical life skills to your children [title] Teach your child to brush their teeth. [step] Purchase a toddler-sized toothbrush with soft bristles for gentle brushing. Initially, you can help your toddler brush their teeth, but you should still be responsible for the cleaning.\n\nA. A small child won't be thorough. Most children are capable of brushing alone between the ages of 6 and 8.\nB. Use soft bristles as to minimize rubbing or breakage. [substeps] Be sure to use soft brushes, as toddlers can scratch their teeth with fingernail.\nC. If you are teaching your child to brush their teeth, start by washing your hands before, during, and after brushing. [title] Encourage your toddler to wash their hands before and after brushing.\nD. Provide your toddler with the toothbrush only during diaper-time. Whenever your toddler has to use the toothbrush, you should also treat the bristles with baby wipe to remove teeth if the bristles are dirty.","prompt":"[header] How to teach practical life skills to your children [title] Teach your child to brush their teeth. [step] Purchase a toddler-sized toothbrush with soft bristles for gentle brushing. Initially, you can help your toddler brush their teeth, but you should still be responsible for the cleaning.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5632,"error":null,"output":"C"}
{"question_id":"hellaswag:00009631","item_index":151,"attempt":0,"prompt_hash":"662d62d215862a857d2443df62b47bcf3c268cecd3a8d22b2efd288bb0b9a8ab","question":"[header] How to make clove cigarettes or kretek [title] Gather your looseleaf tobacco and whole cloves. [step] Depending on how much clove you like, you can do a clove: tobacco ratio of anywhere from 1:3 to 2:3. Any more than that and it will not burn properly.\n\nA. [title] Buy one clove of whole cloves. [step] Grosgrain is the best one to start with, since the garlic potassium contains magnesium, so if you only use one pod, you will not produce any flavor! [title] Get everyone together in an underground cellar or room.\nB. [title] Cut your cloves at the stem. [step] The main difference between old split and new clip cloves is that they are of the same type.\nC. [title] Start to prepare the cloves for inclusion. [step] A clove is actually a dried flower.\nD. [title] Peel the skin off the cloves. [step] The peel is best done in a pan so that you can heat it up more evenly.","prompt":"[header] How to make clove cigarettes or kretek [title] Gather your looseleaf tobacco and whole cloves. [step] Depending on how much clove you like, you can do a clove: tobacco ratio of anywhere from 1:3 to 2:3. Any more than that and it will not burn properly.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5633,"error":null,"output":"C"}
{"question_id":"hellaswag:00008025","item_index":152,"attempt":0,"prompt_hash":"0db3b3f8bd6dccd773d9af5ab696263d85503ce5366d350d68168ef9d8da2773","question":"[header] How to wash coconut oil out of hair [title] Cut through the oil with the citric acid in lemon juice. [step] Squeeze the juice of 2 fresh lemons into a bowl, then mix in 8 fluid ounces (240 ml) of water. Pour this mixture over your hair and scalp, wait a few minutes, and then rinse it out with warm water.\n\nA. The excess coconut oil should wash out with it. [substeps] You can also add a few tablespoons of honey to the mixture to soften your hair.\nB. [substeps] Lemon juice will keep you moisturized, also. [title] Use coconut oil for the practical purposes of the treatment, and not the body.\nC. [title] Take a hair super or hair wax and massage lemon oil. [step] This may be safer for those with very curly hair but most people with thicker hair prefer to use vitamin e oil.\nD. [substeps] You can wash both coconut oil and lemon oil two ways, so just make sure to use two separated bowls. [title] Hang up your coconut oil-covered hair.","prompt":"[header] How to wash coconut oil out of hair [title] Cut through the oil with the citric acid in lemon juice. [step] Squeeze the juice of 2 fresh lemons into a bowl, then mix in 8 fluid ounces (240 ml) of water. Pour this mixture over your hair and scalp, wait a few minutes, and then rinse it out with warm water.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5634,"error":null,"output":"A"}
{"question_id":"hellaswag:00001035","item_index":153,"attempt":0,"prompt_hash":"0cd2ef87554f04f049f7055cad0d7a0288c45b05a925fa8eac4ad8a9ac2f0cfa","question":"A woman is seen speaking to the camera and leads into her holing up a jar of honey. she\n\nA. whips off the cream and one lens, takes the lens off and puts them back on.\nB. sprinkles some into a bowl, then stirs it together with honey.\nC. continues speaking and leads into her rubbing honey on her face.\nD. then opens the jar and pours the honey around the skillet while on top.","prompt":"A woman is seen speaking to the camera and leads into her holing up a jar of honey. she","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5632,"error":null,"output":"C"}
{"question_id":"hellaswag:00006084","item_index":154,"attempt":0,"prompt_hash":"7cbf297e79430656622743c6f6880b5b7c7e017cf7ca09ddc7a8ed423635db1d","question":"[header] How to choose a horse trainer [title] Ask for a referral from other horse owners you know. [step] One of the best ways to get recommendations for horse trainers is by asking people you already know in the equestrian world. Ask your friends and family members or any other acquaintances who have horses what trainers they use and who they would recommend to you.\n\nA. [substeps] Talk to trainers about different trainers you might use and what they think of the trainers and what they typically charge for their services. You want to make sure that the trainer will deal with your horse in an unbiased and honest manner, and that you trust their service.\nB. [substeps] If you are a member of any horse showing or sporting association, you can also ask the association for recommendation for horse trainers. You may also want to ask about horse trainers at your local feed store.\nC. If there is a horse show you really like and it's giving you great reviews, there might be a horse trainer you would be interested in. This will give you important information about how the trainer treats and interacts with your other horses and's trainers.\nD. [substeps] Don't just request a trainer, but call the trainer as well. If you don't know much about horses, start with one good friend, relative, or family member who is familiar with horses.","prompt":"[header] How to choose a horse trainer [title] Ask for a referral from other horse owners you know. [step] One of the best ways to get recommendations for horse trainers is by asking people you already know in the equestrian world. Ask your friends and family members or any other acquaintances who have horses what trainers they use and who they would recommend to you.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5633,"error":null,"output":"B"}
{"question_id":"hellaswag:00000507","item_index":155,"attempt":0,"prompt_hash":"dde4d798425bbfe7ef0e326dd29ff5f28a0b6a4366b19ed7b8cf922088f3a6f2","question":"An african american male wearing a blue muscle shirt and under armour sweat shorts is standing on an elliptical talking. although he\n\nA. is talking he is also being held by being shirtless, and is showing off the number 678 alongside the elliptical opposite him that continues to display the number 779.\nB. is talking to the camera and showing the body specifications of the machine it's powered down and you can see how it is slower.\nC. is the only one in the gym, there are two people standing behind him at the reception desk.\nD. is wearing a tank top underneath he has a racquetball racketball racket and a black background.","prompt":"An african american male wearing a blue muscle shirt and under armour sweat shorts is standing on an elliptical talking. although he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5694,"error":null,"output":"C"}
{"question_id":"hellaswag:00007881","item_index":156,"attempt":0,"prompt_hash":"81e8d6505cde0ac274d3818c9d23006d7121c1b99bb0f16479021961c6d8b3d5","question":"[header] How to make shoes last longer [title] Keep your shoes in the box for long-term storage. [step] Your shoebox was made specifically to store shoes and is the optimal storage area for longer-term storage. Storing your shoes in the box protects the shoe from the dangers of sunlight, dust, and extreme temperatures.\n\nA. You also reduce the amount of shock a shoe experiences by preventing crud from forming. [substeps] You can easily repurpose used shoes.\nB. When using a plastic shoe box, make sure it is completely dry to prevent mildew from growing. [substeps] Examples of shoes that you'd keep in long-term storage include dress shoes or heels.\nC. [substeps] Place the box away from windows and doors leading to your house or work area. That way, they're hidden whenever you return home from the office.\nD. [substeps] A shoebox is good storage for shoes even in temperatures of 360 degrees fahrenheit (15. 8 degrees celsius) or higher.","prompt":"[header] How to make shoes last longer [title] Keep your shoes in the box for long-term storage. [step] Your shoebox was made specifically to store shoes and is the optimal storage area for longer-term storage. Storing your shoes in the box protects the shoe from the dangers of sunlight, dust, and extreme temperatures.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5694,"error":null,"output":"B"}
{"question_id":"hellaswag:00009383","item_index":157,"attempt":0,"prompt_hash":"432becf980baa800582fe89e9c96f7bf69884df5fb751e540a41917b737dfb56","question":"[header] How to outlet pass [title] Jump to snag rebounded balls. [step] Attack the ball at the highest point possible. Jump and meet the ball as high as you can in its arc off the rebound.\n\nA. The ball must turn rather than just bounce off it. [substeps] Make contact with the ball with a little and a little outside force.\nB. This can be done verbally over your head : [substeps] Start the shot with your arm bent around the ball before your arm swings back. Your forearm should be facing forward, and your leg bent at 90 degrees.\nC. [substeps] You also have the choice of attacking with your feet instead of your hands. Both of these options allow you to reach the ball more quickly, which can help dramatically reduce the strain on your wrists.\nD. The higher you jump, the better your chances of snatching the ball ahead of someone else. [substeps] Pay attention to your peripheral vision as you snag the ball.","prompt":"[header] How to outlet pass [title] Jump to snag rebounded balls. [step] Attack the ball at the highest point possible. Jump and meet the ball as high as you can in its arc off the rebound.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5694,"error":null,"output":"D"}
{"question_id":"hellaswag:00009698","item_index":158,"attempt":0,"prompt_hash":"1ad09777ebd170e23a9f6625cb4dac758070b484042e250cae05d391b2c19a7c","question":"[header] How to create high shine lips [title] Prepare the lips. [step] Your lips must be in excellent condition for this to work, so don't skip this step. [substeps] Rub your lips with either lip salve or a small dot of facial scrub.\n\nA. Gently brush the lips with a soft toothbrush. This will lift off any pieces of dry or dead skin and will leave the lips feeling soft and smooth.\nB. Perhaps your choice of powder option for exfoliating. In the end, the finest powder is the one which provides the best results.\nC. This will take care of any blemishes or roughness. There are several different lip balm ingredients available for lip.\nD. Put two to three drops of your favorite lip moisturizer in your mouth and rub it into your lips. You can also create a cream or powder blend if you don't mind your lips looking a little greasy.","prompt":"[header] How to create high shine lips [title] Prepare the lips. [step] Your lips must be in excellent condition for this to work, so don't skip this step. [substeps] Rub your lips with either lip salve or a small dot of facial scrub.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5693,"error":null,"output":"A"}
{"question_id":"hellaswag:00006420","item_index":159,"attempt":0,"prompt_hash":"e3f0449d2629eb426de9e88ca410bc4774120cbe18670a6d13c9fecd45d13cbd","question":"[header] How to avoid blogging scams [title] Use social networking sites to get connected with your potential customers. [step] Nowadays, people are spending more time online and in this regard, social networking sites are their preference. Therefore, you can use platforms like facebook, twitter, linkedin, etc.\n\nA. To get information about your web visitors and friends. [substeps] This will help eliminate bad news filters on your site and reach a wider audience.\nB. On which you can connect with your potential customers for chatting. [substeps] Use social networking sites like gmail, instagram, snapchat, etc.\nC. For a reliable scam, not 10% of your customers are in on the scam. [substeps] For example, hackers can scam addresses, in some cases called sms addresses.\nD. To build positive image of your brand in front of your consumers. By posting about your services on your facebook page or tweeting about your company, you can build a long term relationship with your current customers and attract new customers.","prompt":"[header] How to avoid blogging scams [title] Use social networking sites to get connected with your potential customers. [step] Nowadays, people are spending more time online and in this regard, social networking sites are their preference. Therefore, you can use platforms like facebook, twitter, linkedin, etc.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5694,"error":null,"output":"D"}
{"question_id":"hellaswag:00006999","item_index":160,"attempt":0,"prompt_hash":"1dc63a92c98aecf9ea054c8229c2b0a5a47d26d0704f13d6b8cb3b0514229cdb","question":"[header] How to make money selling at festivals [title] Scout the show ahead of time. [step] Before vending a show, take time to scout it out. Go and see what people are selling.\n\nA. Figure out how heavy the merchandise is and what the brochure clearly states and how many people are present at the show. You can also focus more on making money on the show itself as opposed to the band.\nB. If you enjoy reading and listening to music, go ahead and get tickets to those shows. If you're trying to be successful at selling at festivals, you'll need to find a place to hang out at.\nC. Check out the quality of work, evaluating whether your products are of similar quality and whether or not they fit the general feel of the show. For instance, edgy, experimental art is not likely to fit in at a county craft festival.\nD. Find out what the stores are selling, what the artists are doing at the show, and whether/when you can start making flyers or postcards. [substeps] Follow the conventions of festivals in your area and use this knowledge to make a flyer or postcards that will attract smaller crowds.","prompt":"[header] How to make money selling at festivals [title] Scout the show ahead of time. [step] Before vending a show, take time to scout it out. Go and see what people are selling.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6028,"error":null,"output":"B"}
{"question_id":"hellaswag:00006217","item_index":161,"attempt":0,"prompt_hash":"c4be29443a146bdd2b3f33559e880d50c3e8dd9b9d904b6ad4d8d3563a106760","question":"[header] How to make your pavers level [title] Excavate. [step] Before you can install pavers you need to excavate the area 6 to 8 inches (15. 2 to 20. 3 cm). [title] Use a level.\n\nA. [step] A level means that you start at the center point in the soil. When you excavate the site, you dig around the desired level.\nB. [step] Use a level to drill at the center of the raised paver. Drill slowly to prevent the paver from tipping over.\nC. [step] Take a large square rod and run it over the excavated area to ensure that it is flat and level. If you feel it rising and falling you will need to readjust the soil surface to make it even.\nD. [step] You can do this using a straightedge on the side of the tool bar, as long as it is big enough to hold the grit of your work and flush against the sharpened wood. Screw the level onto the top, or use a drawer, or use a vise, instead.","prompt":"[header] How to make your pavers level [title] Excavate. [step] Before you can install pavers you need to excavate the area 6 to 8 inches (15. 2 to 20. 3 cm). [title] Use a level.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6029,"error":null,"output":"C"}
{"question_id":"hellaswag:00003837","item_index":162,"attempt":0,"prompt_hash":"6c94cf69f8be50c6eea5cc5f3461ad4cb7735196903fefe6e26892d219b13088","question":"[header] How to dress smart casual (females ) [title] Think about what bottoms to wear. [step] For a smart casual outfit, you must balance flairs of professionalism and fashion. Consider wearing bottoms that are comfortable but sharp looking; they need to be able to pull through a professional yet casual look.\n\nA. Choose simple, cool colors, mostly leopard print or solid green, to really add to this look. [substeps] Think of a top that fits snugly across your torso.\nB. [substeps] Avoid shorts with plunging or sloping fronts. Slip off the top of the shorts as well when you're dressing to keep your knee-length, breezy bottoms from showing through when you top them off.\nC. For example, a nice pair of slacks, darker-colored conservative skirts, and even a dark wash pair of jeans can all be appropriate bottoms for a smart casual outfit. Most times, your bottoms can act as a base from which you create your ensemble outfit.\nD. Suit and tie is optional, but try to avoid that to an extreme by wearing something flowing and simple. You don't need to pull it off for fun, but you should avoid tight tops, tight necklines, skirts, or clothes with class prints.","prompt":"[header] How to dress smart casual (females ) [title] Think about what bottoms to wear. [step] For a smart casual outfit, you must balance flairs of professionalism and fashion. Consider wearing bottoms that are comfortable but sharp looking; they need to be able to pull through a professional yet casual look.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6028,"error":null,"output":"C"}
{"question_id":"hellaswag:00008082","item_index":163,"attempt":0,"prompt_hash":"6580a0262994e550950e33f3ab95bcea9d22071ed821fdc6e453d9a0eaf16897","question":"[header] How to adjust the time on a rolex replica [title] Locate and release the crown. [step] On the right side of a rolex there is a small, circular dial called the \" crown \" that allows you to control the mechanism of the watch. Unscrew the crown and pull it out from the watch base.\n\nA. [substeps] The crown is orange with a large white dot. It is usually located under the hand spindle.\nB. As you remove the crown it will look like it has been pushed into a slot underneath a ring within the watch. Use your finger to gently release the crown and prepare your watch for adjustment.\nC. Once the crown is extended, it puts the watch in position to be wound to change the time. [substeps] The crown dial is found in the same position on all rolex models.\nD. While not completely unscrewed, this feature will help you determine the type of rolex replica to purchase. If you do not know, consult the handgrip manual to be certain.","prompt":"[header] How to adjust the time on a rolex replica [title] Locate and release the crown. [step] On the right side of a rolex there is a small, circular dial called the \" crown \" that allows you to control the mechanism of the watch. Unscrew the crown and pull it out from the watch base.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6028,"error":null,"output":"C"}
{"question_id":"hellaswag:00005634","item_index":164,"attempt":0,"prompt_hash":"e26b805418635327c8b3fc11eb7494fc35a3f740c119f99ba35a7dbd9050f522","question":"[header] How to buy get well gifts [title] Research local and online floral shops. [step] Floral arrangements are a classic get well gift and enhance energy and decrease depression. They can be found at the hospital, supermarket, flower store, flower markets, or online flower stores, like ftd and even amazon.com.\n\nA. [substeps] Also, at first glance, there is no standard set of flower arrangements. In 2010, the david foster boys made a pattern of clusters of flowers arranged in rows at flower shops.\nB. Decide which company you would like to buy the arrangement from. [substeps] By purchasing flowers online, the flowers come directly from the grower and are usually freshly arranged so you do not have to worry about wilting.\nC. You should also shop at specialty stores like target, starbucks, and walmart. [title] Search for a bouquet in a florist.\nD. Search online to browse expert, local designs from independent floral and shopping blogs. [substeps] You could shop at local hobby stores or antique stores.","prompt":"[header] How to buy get well gifts [title] Research local and online floral shops. [step] Floral arrangements are a classic get well gift and enhance energy and decrease depression. They can be found at the hospital, supermarket, flower store, flower markets, or online flower stores, like ftd and even amazon.com.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6027,"error":null,"output":"B"}
{"question_id":"hellaswag:00005425","item_index":165,"attempt":0,"prompt_hash":"8267c18e85a0d67f92f81972b3cfedea917a0fe00a6a7729f59d650cd35534ed","question":"[header] How to start walking for exercise [title] Find a good place to walk. [step] Generally, the locations for walking have a flat terrain, straight path, smooth surface and minimal traffic. The convenient choice would be the neighborhood around your block, but if the road is too steep, curvy or just not what you're looking for, you might want to consider other areas around your town.\n\nA. [substeps] Bicycle routes are typically also good options. If you're a beginner, the chances are good you'll find a local trail that's easy to follow.\nB. [substeps] Ensure you are wearing appropriate footwear as walking puts a bit of pressure on your feet which can cause pain. Also, be sure you are wearing the correct footwear for the weather.\nC. [substeps] If possible, find a public park. You might even want to stop in an actual parking lot in your neighborhood.\nD. [substeps] You want a place that has a lot of traffic, not an area where people might be stopping. This can take a bit of time and effort, so find a safe and distraction free area to walk.","prompt":"[header] How to start walking for exercise [title] Find a good place to walk. [step] Generally, the locations for walking have a flat terrain, straight path, smooth surface and minimal traffic. The convenient choice would be the neighborhood around your block, but if the road is too steep, curvy or just not what you're looking for, you might want to consider other areas around your town.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6033,"error":null,"output":"B"}
{"question_id":"hellaswag:00007090","item_index":166,"attempt":0,"prompt_hash":"380a61da6c00dabcc4c86c7197fa30ef04cfa6141a700db42837672aab03c70c","question":"[header] How to make vaseline [title] Combine the beeswax and olive oil. [step] Add 1 ounce (28 g) of beeswax and ½ cup (118 ml) of olive oil to a small sauce pan. There's no need to stir the ingredients because they won't combine until the beeswax begins to melt.\n\nA. [substeps] When you cook the beeswax, it will begin to bubble. Even if you'd rather not use olive oil, you can substitute almond or coconut oil to help prevent burning.\nB. [substeps] It's usually easier to use beeswax pellets instead of a block. They're easier to measure and usually melt more quickly.\nC. [substeps] If you prefer, you can substitute hot tamagotchi sauce for the olive oil. Just cook until the beeswax becomes melted before you add it to the sauce pan.\nD. [substeps] You can find vaseline at grocery stores or online. [title] Turn off the burner and stir in the beeswax.","prompt":"[header] How to make vaseline [title] Combine the beeswax and olive oil. [step] Add 1 ounce (28 g) of beeswax and ½ cup (118 ml) of olive oil to a small sauce pan. There's no need to stir the ingredients because they won't combine until the beeswax begins to melt.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6033,"error":null,"output":"B"}
{"question_id":"hellaswag:00003991","item_index":167,"attempt":0,"prompt_hash":"653dcc8a665d42e32d4caead16962c42e5db79ce1bf49ee0734b888efe3ec42a","question":"[header] How to succeed in college with young children [title] Set a daily routine. [step] Having a set routine helps create some form of normalcy, a sense of security, and it establishes everyday expectations. Planning set times for dinner, story time, bath time, bed time, etc.\n\nA. Not only does this help your child adjust to your new lifestyle but it helps you succeed in college. [substeps] Consider writing out a family schedule for all to see.\nB. Helps alert your child about what is anticipated, and provides encouragement and reward for your child that you are there for them and their needs. [substeps] Try to set daily relaxation hours, but only during particularly difficult situations (such as family outings).\nC. Can go a long way in creating a sense of consistent routine. [substeps] Set small daily moments for smaller sections of life, like leaving work early or taking the kids for a walk after school.\nD. Becomes easier when you establish a routine in which everyone can relax. [substeps] This will act as a buffer for your child from feelings of anxiety or embarrassment in the daytime.","prompt":"[header] How to succeed in college with young children [title] Set a daily routine. [step] Having a set routine helps create some form of normalcy, a sense of security, and it establishes everyday expectations. Planning set times for dinner, story time, bath time, bed time, etc.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6027,"error":null,"output":"A"}
{"question_id":"hellaswag:00009809","item_index":168,"attempt":0,"prompt_hash":"8aa34144e5f3e5112079264af53f4f235d369eff108aa23496e310e653f7ec54","question":"[header] How to close an internet pop up [title] Look for a small \" x \" at the top right corner of the pop-up. [step] Some ads do a good job of hiding close buttons and links in busy images, so you may not notice it at first. [substeps] It may be more difficult to find a close button on devices with smaller screens.\n\nA. If you see a message that says \" don't show more alerts for this webpage \" (or something similar), place a check into the box provided. This should stop pop-ups from recurring.\nB. [title] Click on a button you want to close. [step] You should see this button at the bottom right corner of the pop-up.\nC. [title] Click on the button that contains a circular handle with some links. [step] It is located at the bottom left corner of the pop-up.\nD. Don't magnify huge displays. [title] Check to see if other people's pop-ups move.","prompt":"[header] How to close an internet pop up [title] Look for a small \" x \" at the top right corner of the pop-up. [step] Some ads do a good job of hiding close buttons and links in busy images, so you may not notice it at first. [substeps] It may be more difficult to find a close button on devices with smaller screens.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6032,"error":null,"output":"B"}
{"question_id":"hellaswag:00004168","item_index":169,"attempt":0,"prompt_hash":"e67a3ec2d3cfbf64e92e87663152d425f0a88a9fa4b6126fc4850fa63afbcc11","question":"[header] How to make an oatmeal bath [title] Grind the oatmeal into a fine powder. [step] Place about one cup of raw, unflavored oats into a blender, food processor or coffee grinder. No need to get too fancy-the kind of normal packaged oatmeal you'd find at the supermarket will work just fine.\n\nA. Blend the oats until they reach a very fine, powdered texture. This will help them dissolve more easily in the bath.\nB. [title] Add water and a splash of powdered milk. [step] While you can add milk to make it chunky, it's not required in this step.\nC. [substeps] Plain oats are roughly equal in size to the size of the oats themselves. You might find it much easier if you use an instant-read grinder, but a multigrain or hard-soled grind is best.\nD. Don't grind the oats too hard. Once you have the oats in there can be a bit of a pitty grind, but the important thing is to get them into the correct hardness at the right temperature.","prompt":"[header] How to make an oatmeal bath [title] Grind the oatmeal into a fine powder. [step] Place about one cup of raw, unflavored oats into a blender, food processor or coffee grinder. No need to get too fancy-the kind of normal packaged oatmeal you'd find at the supermarket will work just fine.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6027,"error":null,"output":"A"}
{"question_id":"hellaswag:00003665","item_index":170,"attempt":0,"prompt_hash":"ef7fe59cb48b7a994f609930b2c38c0a0604bcd0e45c0e4701e9b42c83016edf","question":"[header] How to choose a skin moisturizer [title] Observe your skin conditions. [step] Dry skin is often rough, with red patches in certain areas. Pores are small and the skin may feel tight.\n\nA. Find the location on your face that has rough, dry patches, or areas that are pitted, lightly rubbed with a clean cloth. If you find your skin looking rough when you buy products, it could indicate a dry regimen.\nB. People with dry skin also tend to have flakiness on the face and body. [substeps] Dry skin may become more irritated from cold weather, harsh soaps, prolonged time on your feet, and after water activities like swimming.\nC. This can be a sign of problem skin, or a possible environmental problem. Look for pimples under your eyes if you have sensitive skin.\nD. Make a list of all of the types of oily skin you have. List any acne or eye problems you have.","prompt":"[header] How to choose a skin moisturizer [title] Observe your skin conditions. [step] Dry skin is often rough, with red patches in certain areas. Pores are small and the skin may feel tight.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6027,"error":null,"output":"B"}
{"question_id":"hellaswag:00003323","item_index":171,"attempt":0,"prompt_hash":"6bddc3ae169f19e5bc211748d39eedd67880dd25ceff75370100091780d8b36d","question":"[header] How to become a meteorologist [title] Take the right courses in high school. [step] If you want to become a meteorologist, start preparing in high school. Take advanced math and science courses.\n\nA. Being a meteorologist is good enough! Not only does that give you skills to put in, it is also beneficial to go the extra mile. [substeps] Many meteorology courses are in english, chemistry, and math.\nB. You might think you'll want to go into intermediate first, such as biology, but as the aries, you'll need to learn other subjects for your career. Take elective courses while still in high school.\nC. Many schools offer ap courses that can transfer into college credit. [substeps] Take calculus, physics, chemistry, and earth science.\nD. Most meteorlogists are required to take at least one semester of high school in order to qualify as a meteorologist. [substeps] Getting this degree is just as important as getting your college education.","prompt":"[header] How to become a meteorologist [title] Take the right courses in high school. [step] If you want to become a meteorologist, start preparing in high school. Take advanced math and science courses.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6031,"error":null,"output":"C"}
{"question_id":"hellaswag:00000848","item_index":172,"attempt":0,"prompt_hash":"542f1e809981de2009a566d75334a7ba82e2209ed12d8351ac83be6340c786b4","question":"A man in a black shirt is talking on a basketball court. a boy\n\nA. is walking down the court and starts doing several martial arts moves.\nB. in the stands shuffle to the front of the court and stands over him.\nC. in a red shirt is practicing dribbling and hitting the ball with a stick.\nD. is playing basketball on the court.","prompt":"A man in a black shirt is talking on a basketball court. a boy","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6031,"error":null,"output":"D"}
{"question_id":"hellaswag:00008981","item_index":173,"attempt":0,"prompt_hash":"9159f8d9625bdb709b7d60a2ce74177bf49e86b8e6f732208d9ff78adee4ebd8","question":"[header] How to make a hair protectant spray [title] Pour the water into a spray bottle. [step] You can mix your hair protectant spray in the bottle that you plan to keep it in. Start by filling the bottle with 6 ounces (177 ml) of distilled or filtered water.\n\nA. Add 6 ounces (100 ml) of pure distilled or filtered water. You'll need this water in every chemical form that you use and make sure that the bottle is filled with the ingredients that you'll need.\nB. [substeps] Make sure that the water is at least 80% water or the spray will not achieve the same benefits as it would with distilled water. If your hair feels too wet and the curls are weak, consider running a wet hair dryer over it.\nC. Because the water is denser than the oil, it's important to add it first so they mix properly. [substeps] You can use a plastic or glass spray bottle, but make sure that it holds 7 ounces (207 ml) or more.\nD. [substeps] Make sure to use distilled water or filtered water. The carbonation in the water makes it more dangerous to use for hair products.","prompt":"[header] How to make a hair protectant spray [title] Pour the water into a spray bottle. [step] You can mix your hair protectant spray in the bottle that you plan to keep it in. Start by filling the bottle with 6 ounces (177 ml) of distilled or filtered water.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6027,"error":null,"output":"C"}
{"question_id":"hellaswag:00003192","item_index":174,"attempt":0,"prompt_hash":"6691848f62e6f4fd43b3a1819c53b8ae6a502948f4ca858d29faf822bda4eb00","question":"A woman is seen scooping up a spoonful of ice cream and taking a bite with a baby in front of her. the woman\n\nA. continues squeezing milk into her spoon and handing the baby back to the woman, after which she bites her ice cream the same way she did the baby.\nB. continues to tease the baby with the ice cream giving him little bites here and there as well as taking bites for herself and laughing.\nC. then mimics the same bite and finishes by taking the small bite to let go.\nD. walks around a city street with shots of shops and bars.","prompt":"A woman is seen scooping up a spoonful of ice cream and taking a bite with a baby in front of her. the woman","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6030,"error":null,"output":"B"}
{"question_id":"hellaswag:00000930","item_index":175,"attempt":0,"prompt_hash":"2bd69171468335ac6dfaefbec6ec116d7c33f5efd8c8f1cc7de518be1b736eac","question":"A man is seen standing in front of a window speaking to the camera while wearing a headpiece. the man\n\nA. sits down near a table while speaking into the camera.\nB. continues speaking failing into a microphone while holding a pose.\nC. then brushes along the sides of the window and points back and fourth to the window while speaking to the camera.\nD. then stands on his hands and begins rubing his body into various positions.","prompt":"A man is seen standing in front of a window speaking to the camera while wearing a headpiece. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6028,"error":null,"output":"C"}
{"question_id":"hellaswag:00003834","item_index":176,"attempt":0,"prompt_hash":"fef9ccaeb018921dbe7e712b57f1da4089dd64de78dae19b5288c872b35dae8e","question":"[header] How to find a symbolic baby name [title] Pick family names. [step] Naming your child after your grandparents, uncles, and aunts is a way to make that person live on in the next generation. It ties your child's name into the heritage of your family.\n\nA. It is likely that a given child's name is one that is repeating those names from the 10 most foundational generations from your grandparents' robes. [substeps] You want the gender of the parents in your birth canal experience, to show how they differ from other children whose names are traditionally included in fitting names.\nB. Thus, this is a way to make your baby's name symbolic both of a person who you admire and your family more generally. [substeps] Sometimes family names have become outdated and would sound off-putting in the modern world.\nC. You can even use your personal personality to your advantage. [substeps] For example, if your grandmother has long been growing up in that house, it's probably a series of names.\nD. To name just 4 people, the birth name is \" elizabeth, jul-we love to refer to ourselves as \" twins \", \" about. \" you can also use it to make your baby name come later when the details of the child's name have arrived.","prompt":"[header] How to find a symbolic baby name [title] Pick family names. [step] Naming your child after your grandparents, uncles, and aunts is a way to make that person live on in the next generation. It ties your child's name into the heritage of your family.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6030,"error":null,"output":"B"}
{"question_id":"hellaswag:00009213","item_index":177,"attempt":0,"prompt_hash":"42e5a4c172c90937fa3c3703931cfc24e98db3753edb5dfaaab5b5504e4eaf4a","question":"[header] How to wear a brown suit [title] Choose a brown suit shade. [step] If you don't have a brown suit yet, you may be wondering what shade of brown would work best for you. If you have dark hair and/or a dark or ruddy complexion, a darker shade of brown will complement your features.\n\nA. Next, try on a pair of brown trousers and a black coat to wear for a casual feel. [substeps] The same goes for a nicer jacket or sweater; this is a hint that it is more appropriate for more formal work.\nB. To find the right shade of brown, go to a shop and try to match it to your skin tone. [substeps] Go for an olive-toned brown.\nC. That said, a light brown or tan suit is better for professional use, especially in warmer climates. [substeps] Dark brown suits are better if you like to incorporate color into your outfits.\nD. However, if that's all you prefer, you can wear any coloring of brown to your suit. [substeps] You should also avoid wearing brightly colored suits.","prompt":"[header] How to wear a brown suit [title] Choose a brown suit shade. [step] If you don't have a brown suit yet, you may be wondering what shade of brown would work best for you. If you have dark hair and/or a dark or ruddy complexion, a darker shade of brown will complement your features.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6030,"error":null,"output":"B"}
{"question_id":"hellaswag:00005566","item_index":178,"attempt":0,"prompt_hash":"75bf80977fed7f3034c8c1f36739338fcbe88a8be3916d753bb31213a6e61b84","question":"[header] How to make pork roast with garlic and rosemary [title] Prepare your grill. [step] Using your favorite method light the grill. Have a bed of coals on one side of the grill, if you are using charcoal briquettes.\n\nA. Keep in mind that we will be using what we use for this. Used coals will smoke better than burnt-out coals on an gas grill.\nB. The coals will become the butt of the roast. Use glass or aluminum pans to hold the heat in the pan to keep it from burning the meat on one side.\nC. [title] Close the grill and head to the kitchen to prepare the pork roast. [title] Add the garlic and rosemary to a food processor bowl.\nD. If not, you can place wooden skewers on the side opposite of the grill to line in the coals. [title] Pour your pork roast stock or water over your coals.","prompt":"[header] How to make pork roast with garlic and rosemary [title] Prepare your grill. [step] Using your favorite method light the grill. Have a bed of coals on one side of the grill, if you are using charcoal briquettes.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6030,"error":null,"output":"C"}
{"question_id":"hellaswag:00002979","item_index":179,"attempt":0,"prompt_hash":"703f90ee46f532074aa0878bcb84dad77317bf15d1ecf4eea1f0ad2e3ede4b45","question":"A man is talking in front of a camera. He shows off a tube of hair gel. he\n\nA. shakes some of it out and sprays things on his hair.\nB. spoon creates several bees to spread the gel in his hair.\nC. brushes and cuts his hair off, leaving bites in the process.\nD. then demonstrates how to use the gel by smoothing it into his hair.","prompt":"A man is talking in front of a camera. He shows off a tube of hair gel. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6028,"error":null,"output":"D"}
{"question_id":"hellaswag:00008121","item_index":180,"attempt":0,"prompt_hash":"d057efadde05e0e1af56cd2255f7b86825d69e11d7c350883d68fc1dfe044807","question":"[header] How to choose a christmas gift for a baby [title] Ask the parents what the baby needs. [step] They will likely have at least a few things that they'd appreciate for a gift, from clothing to toys and maybe even furniture. If you're not keen to ask them, be observant.\n\nA. Listen to anything the parents mention about wanting or notice whether the baby is missing certain needed items whenever you visit. [title] Consider giving a basket of goodies.\nB. They may point out things that they could use and offer \" helpful \" things, too. [title] Consider pet cookies, fruit popsicles, ice cream and candy.\nC. Ask anything that appears rational or simple to you. If you're not sure, be sure you ask more than your budget can handle.\nD. [title] Break down the ideas you like and find out who's going to give the gift. [step] Who? Is it one of the kids? If you have enough ideas and are willing to spend some time brainstorming what to spend the gift on, it can be your lucky day.","prompt":"[header] How to choose a christmas gift for a baby [title] Ask the parents what the baby needs. [step] They will likely have at least a few things that they'd appreciate for a gift, from clothing to toys and maybe even furniture. If you're not keen to ask them, be observant.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6031,"error":null,"output":"A"}
{"question_id":"hellaswag:00005267","item_index":181,"attempt":0,"prompt_hash":"e68dff722924a141f58645007db640412034aaf5a7d42613b7502c3f5a0b691e","question":"[header] How to make a nesting box for parakeets [title] Get wood panels. [step] The nest has to be made of wood. Parakeets like to chew and they will shred up a cardboard box quickly.\n\nA. Since your parakeet is small, it will be able to chew inside panels because of the lightweight insulation. The wood panel will also help your parakeet train its back (making sure to wear gloves and goggles when handling the nest).\nB. Go to the hardware store and pick up some wood paneling to build your cage. [substeps] Panels of plywood board will work best.\nC. If the size is too big, then whatever you use will simply be too much for your parakeet. If you have whole wood panels of the same size that you will use outside, put the top box on top of the second box.\nD. You will want to purchase in a box that is 4-5 inches wide by 8-12 inches deep. It needs to be tall enough to keep your parakeet on all 4 sides except for the bottom.","prompt":"[header] How to make a nesting box for parakeets [title] Get wood panels. [step] The nest has to be made of wood. Parakeets like to chew and they will shred up a cardboard box quickly.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6027,"error":null,"output":"B"}
{"question_id":"hellaswag:00009077","item_index":182,"attempt":0,"prompt_hash":"2480897be689f614f739e41d475506b174e3cd60a1d21a4a337ee4538e9569ed","question":"[header] How to be a successful muslim husband [title] Talk with your spouse. [step] Ask about their feelings, and share yours with as well. Be honest and open about your joys and fears.\n\nA. Encourage your spouse to express and discuss these as well. [substeps] If one or the other feels uncomfortable discussing their feelings, keep the focus on both of you.\nB. [title] Serve the blessings god provides to us. [title] Inspire your spouse to follow a range of virtues.\nC. If your husband seems offended, frustrated, or gives signs that he is upset, it may be a sign of a deeper problem. Be sure to ask him things like, \" is it okay to say things in arabic? \" and \" have you ever noticed when i...\".\nD. Listen to what they have to say, and ask follow-up questions when they are telling you something. [substeps] Make time for conversations even on busy days.","prompt":"[header] How to be a successful muslim husband [title] Talk with your spouse. [step] Ask about their feelings, and share yours with as well. Be honest and open about your joys and fears.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6709,"error":null,"output":"D"}
{"question_id":"hellaswag:00001742","item_index":183,"attempt":0,"prompt_hash":"a029eb0da55e5952d6031fe89a99f0fbcb2b60a03d8fb5a7a77c79c45cdc599e","question":"Woman is standing in frnot of a mirror with the wet hair and its separating it into parts. then\n\nA. with a brush dry the hair with an animal print hair dryer.\nB. it is braiding it back in different parts with a rope.\nC. it shows the older woman, she takes a comb and comb the hair of the older woman using a pin and combing her hair to dry.\nD. sveet sprayed it and start acl tome that woman put the hair styles on your head for a while.","prompt":"Woman is standing in frnot of a mirror with the wet hair and its separating it into parts. then","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6708,"error":null,"output":"A"}
{"question_id":"hellaswag:00002217","item_index":184,"attempt":0,"prompt_hash":"b62f0e5b26d766ff170d9e23343b660939d8dfe031b3256865d5cd00d158a631","question":"A little girl in green takes her turn and then more of her friends follow after. All three girls stand and then drop a bottle cap to move forward with their jumps. they\n\nA. skip their jumps up till all three jump into the air to finish.\nB. continue to do an arm wrestling match with others who give them several shots where they are successful.\nC. each do a flip that takes them to the base of their leap.\nD. play different variations of the game just having fun doing so and when its over one of the girls throws her hands in the air.","prompt":"A little girl in green takes her turn and then more of her friends follow after. All three girls stand and then drop a bottle cap to move forward with their jumps. they","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6708,"error":null,"output":"D"}
{"question_id":"hellaswag:00003245","item_index":185,"attempt":0,"prompt_hash":"73339633ada022b33bb0dea16e2b722ef49f6545bd37b22e3719cf58c16e87de","question":"[header] How to do a multicolored french manicure [title] Prepare your nails. [step] It is important to start with clean and unpainted nails. If you have any nail polish on already, remove it with a cotton pad and nail polish remover.\n\nA. Then, trim your nails if they are broken or uneven. Once you have done this, wash your hands and prepare to paint.\nB. If not, clear nail polish remover might work better. [substeps] Dry polish remover works beautifully; it takes advantage of the dryness that french manicers crave when coloring their nails.\nC. [substeps] If you are worried about the color or the effort it takes, you can always start with one of four natural colors: light pink, dark pink, and light green. [title] Layer on two coats of the pink polish.\nD. Wear an old t-shirt under the nail polish remover. You want to use a base coat so that your nails can absorb the base coat quickly.","prompt":"[header] How to do a multicolored french manicure [title] Prepare your nails. [step] It is important to start with clean and unpainted nails. If you have any nail polish on already, remove it with a cotton pad and nail polish remover.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6707,"error":null,"output":"A"}
{"question_id":"hellaswag:00000412","item_index":186,"attempt":0,"prompt_hash":"e54c71c76313d9d2c690335e0bb5cef4c0b6cbb2c1190d07a8ae794def8ad07f","question":"The small bowls of salt are arranged and limes are sliced in halves. The limes are juiced into a cup using a hand held press. the salt\n\nA. tubes are filled with limes and a plate of limes.\nB. is poured into the cup from the small bowls.\nC. is stirred and mixed with sugar until it is boiled in water held by a crank.\nD. and lemon juice are used to melt into a clear container.","prompt":"The small bowls of salt are arranged and limes are sliced in halves. The limes are juiced into a cup using a hand held press. the salt","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6707,"error":null,"output":"B"}
{"question_id":"hellaswag:00007613","item_index":187,"attempt":0,"prompt_hash":"cf58cd06dc82b47047637d41d6c24ca13b6b8808c4d2f6946dea6347f48aeaa0","question":"[header] How to retire in mexico [title] Consider if you need to hire a relocation consultant or specialist. [step] Moving between countries can be complicated. And there's no need to learn all the details on your own.\n\nA. If you are moving to mexico, you need to find your precise position, what the country and time can give you, and possibly costs points. In order to figure out how much you need to accumulate and eventually-in the fall or summer months-you'll need to find out what the country your relocated to is.\nB. Relocation consultants or specialists exist that can help you with all the details your personal situation requires. Do keep in mind, however, that hiring a consultant or specialist will add to your retirement expenses, but in the long-run, it may be money well spent.\nC. The problem is that while you may find the most uses in your home country, it won't offer the same kinds of benefits that a uk exit does. [substeps] Seminars are a great way to explore the different canadian destinations you visit and get some perspective while you arrange your accommodations.\nD. So in order to save money and be realistic about your level of experience, examine the following of an rc truck. [substeps] Adjust the values of your emissions levels based on your family's preferred use of the vehicle: license to drive, mileage, mileage, and interest rates for relocating.","prompt":"[header] How to retire in mexico [title] Consider if you need to hire a relocation consultant or specialist. [step] Moving between countries can be complicated. And there's no need to learn all the details on your own.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6467,"error":null,"output":"B"}
{"question_id":"hellaswag:00004349","item_index":188,"attempt":0,"prompt_hash":"2a18d884330ec281aaba240f1e94990aa1e18cd2c796427ab0293bafefbd108d","question":"[header] How to quickly get rid of laryngitis [title] Know the causes of laryngitis. [step] Laryngitis is typically caused by a viral infection such as a cold or bronchitis, and it tends to go away by itself in adults. [substeps] In children, however, laryngitis can cause complications that may lead to croup, a respiratory condition.\n\nA. In some cases, a bacterial or fungal infection causes laryngitis. Exposure to a chemical irritant can also lead to laryngitis.\nB. For instance, your child might develop a fever, painful abdominal cramps, or neurological problems. These complications often happen between the ages of eight-12 weeks and several months after infection.\nC. Stomach flu or meningitis can also cause laryngitis. A large number of immunodeficiency virus infections cause laryngitis.\nD. Children also may have abdominal constipation due to the spreading of laryngitis. Adults also experience heartburn due to inadequate vitamins and nutrients, and stomachaches due to food in the environment.","prompt":"[header] How to quickly get rid of laryngitis [title] Know the causes of laryngitis. [step] Laryngitis is typically caused by a viral infection such as a cold or bronchitis, and it tends to go away by itself in adults. [substeps] In children, however, laryngitis can cause complications that may lead to croup, a respiratory condition.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6467,"error":null,"output":"A"}
{"question_id":"hellaswag:00001277","item_index":189,"attempt":0,"prompt_hash":"16711ba76785a4bc6bd624ee5490b6b7d15003a239cba37a6584118bc6f3acf2","question":"On the inside, several cluttered walls are present and then a set of knifes become apparent. A woman in a room is then shown sitting behind an electrical wheel sharpening a knife. several more knifes\n\nA. are then shown as well as the techniques used to sharpen then.\nB. are shown sharpening with his help as well as her lifting and tossing the object.\nC. appear in the same positions as before.\nD. are shown with the same progress being made intermittently by people sharpening bronze knives.","prompt":"On the inside, several cluttered walls are present and then a set of knifes become apparent. A woman in a room is then shown sitting behind an electrical wheel sharpening a knife. several more knifes","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6468,"error":null,"output":"A"}
{"question_id":"hellaswag:00007322","item_index":190,"attempt":0,"prompt_hash":"a97f54b9f4ab860614d73f32624aa4c75e5063b29fc25d7d0542f309aefe6d09","question":"[header] How to organize your gmail inbox [title] Go to the sidebar where it shows the labels like \" inbox \", \" chats \" etc. [step] Etc. [title] Click underneath the little grey bar where it shows your custom labels, and press \" create label \" if you don't see it, try expanding the menu.\n\nA. [title] Create labels for important emails, notifications and obligations. [step] You can even change the placement of your gmail gmail id on a certain tab, which will give you a way to manually assign labels to certain destinations.\nB. [title] Select the tab called \" appearance \" to select the color ranges for your labels, such as \" standard \", \" all \", etc. [title] Click \" create labels \" or \" order', \" if you want your labels organized in a specific style.\nC. [title] Create labels specifying different people who email you frequently. [title] Skip this step if you know how to create a filter.\nD. [title] Scroll up the menu, and there is your custom labels sticker in the top right hand corner of it; click it. [title] Under the \" custom labels \", there will be a large, grey button that says \" name & email code.","prompt":"[header] How to organize your gmail inbox [title] Go to the sidebar where it shows the labels like \" inbox \", \" chats \" etc. [step] Etc. [title] Click underneath the little grey bar where it shows your custom labels, and press \" create label \" if you don't see it, try expanding the menu.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6467,"error":null,"output":"C"}
{"question_id":"hellaswag:00007625","item_index":191,"attempt":0,"prompt_hash":"acac9908df48765a75aa99dff78389eb76bd9f9a828000ffa43090ea4221eb72","question":"[header] How to rekey a lock [title] Get a rekeying kit. [step] Also called pinning kits, rekeying kits are available at most hardware stores and home centers, as well as online. [substeps] Kits are available for most lock brands and come equipped to do multiple locks, but frequently work for only locks of the same brand.\n\nA. [title] Locate the right lock to rekey. [step] Each lock type has its unique lock components and the lock functions are the same.\nB. If your car is more commonly locked than your home, you may want to stock your kit with pins and chains that come in slots that allow you to remove pins. [title] Remove the lock bolt that sits on the side that folds up to the wrench.\nC. Check their name on the back of your lock kit. If one brand isn't working, you'll need to search.\nD. Some kits include only lock pins, while others include special tools (cylinder follower, ring remover, plug follower) to remove the lock cylinder so you can replace the lock pins. You can order extra pins if you have more locks to re-key than the kit has pins for.","prompt":"[header] How to rekey a lock [title] Get a rekeying kit. [step] Also called pinning kits, rekeying kits are available at most hardware stores and home centers, as well as online. [substeps] Kits are available for most lock brands and come equipped to do multiple locks, but frequently work for only locks of the same brand.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6467,"error":null,"output":"D"}
{"question_id":"hellaswag:00000494","item_index":192,"attempt":0,"prompt_hash":"9edb624d92f2abc104a25a2a8a22448d7e656a7c54b7e1b6b9ac0f8b3ef8dbdb","question":"The man roll the barbel on the floor. the man\n\nA. uses the barbel on the wall.\nB. lifted the barbel up to his chest.\nC. lays the keys on a wire table.\nD. stands still, preparing for the cymbals.","prompt":"The man roll the barbel on the floor. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5719,"error":null,"output":"B"}
{"question_id":"hellaswag:00007491","item_index":193,"attempt":0,"prompt_hash":"6f8a900f7018936591b9297ec824189b22b85c18bff7ba924c01b22dc63240af","question":"[header] How to help your child manage stress [title] Offer your child help at all times. [step] A good way to reinforce your willingness to help your child manage stress offering unconditional love and support from you and other family members. Let your child know that you are available to talk and will help however you can.\n\nA. This may mean inviting your child out for sweets, or telling your child to come over for an afternoon to talk. [substeps] One way to reinforce your love and support is by sharing the affection, reassurance, and comfort you offer your child.\nB. [substeps] You might tell your child, \" i know you are struggling \" or \" i know you and i can help. \" another way to reinforce your role in the child's daily life is to offer unconditional love and support via other ways.\nC. [substeps] Let them know they can call your number or e-mail you anytime when they are going through a crisis. Advise them that you know they are at risk and need to help them manage their emotions and ensure they have you available for their needs as well.\nD. [substeps] Reinforce your love and willingness to help your child with stress support by repeating these reassuring statements often. Your consistency is important to can help your child more effectively handle a stressful situation.","prompt":"[header] How to help your child manage stress [title] Offer your child help at all times. [step] A good way to reinforce your willingness to help your child manage stress offering unconditional love and support from you and other family members. Let your child know that you are available to talk and will help however you can.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5714,"error":null,"output":"D"}
{"question_id":"hellaswag:00001960","item_index":194,"attempt":0,"prompt_hash":"c5fc0a0a054626b8241d117c21d17171a1321c61b04b510c3dd4428b906e5980","question":"A man and a woman are playing rock paper scissors. the man\n\nA. throws the scissors on the ground.\nB. pulls the woman's arm down.\nC. gets on his knee and pulls out a ring.\nD. stops and puts down the knife.","prompt":"A man and a woman are playing rock paper scissors. the man","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5718,"error":null,"output":"A"}
{"question_id":"hellaswag:00009713","item_index":195,"attempt":0,"prompt_hash":"8a0d58536ece0de381845d608ea3dd1081902b2fb6d15562ece1963b2335978d","question":"[header] How to help your asthma using home remedies [title] Drink ginger root tea. [step] Cut off about 1 inch of ginger root and peel the skin skin off using a vegetable peeler. Chop the ginger into small pieces and steep it in 1 cup of boiled water for 5 minutes.\n\nA. This will help to ease congestion in your respiratory system. [substeps] Ginger root takes about 7 to 9 days to settle and help to prevent shortness of breath.\nB. Drink the tea immediately after drinking the ginger root to reduce the dryness. Ginger root has anti-inflammatory properties, which can help with asthma symptoms.\nC. Drink the ginger tea 4 to 5 times a day. Ginger is a natural anti-inflammatory herb that has been used for centuries to treat respiratory problems.\nD. Once the ginger is boiled, strain the ginger root through a strainer and throw away the ginger root. [substeps] Ginger root contains nitrites, which are compounds that stimulate the body's production of acid.","prompt":"[header] How to help your asthma using home remedies [title] Drink ginger root tea. [step] Cut off about 1 inch of ginger root and peel the skin skin off using a vegetable peeler. Chop the ginger into small pieces and steep it in 1 cup of boiled water for 5 minutes.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5714,"error":null,"output":"C"}
{"question_id":"hellaswag:00000578","item_index":196,"attempt":0,"prompt_hash":"a8eebfa14b143dbaed69f6ae76c82d0707bcd7f87c0199e2f9a979e6effae84f","question":"A matador got knocked down by a bull and the bull injured him. Another matador helped him get away after he was gored in the neck. a matador in a gold costume\n\nA. storm opposite a bull.\nB. rides a bull and score and gets his helmet knocked off by a bull that attacked him.\nC. lifted the loser and threw him on the ground.\nD. waved a red banner at him, but he avoided the bull.","prompt":"A matador got knocked down by a bull and the bull injured him. Another matador helped him get away after he was gored in the neck. a matador in a gold costume","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5717,"error":null,"output":"B"}
{"question_id":"hellaswag:00001292","item_index":197,"attempt":0,"prompt_hash":"6f605e68e940313823b9b177b5ce755bb0e41223d8d9933c1dba5d22d71e1670","question":"We see a man in a diving suit under the water. We see a man talking to the camera. we\n\nA. see the man diving off the diving board.\nB. see an opening title screen.\nC. see a man underwater.\nD. see the hole with a giant weight.","prompt":"We see a man in a diving suit under the water. We see a man talking to the camera. we","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5716,"error":null,"output":"C"}
{"question_id":"hellaswag:00003743","item_index":198,"attempt":0,"prompt_hash":"333b1d56b1989929ad974400cd20fa07685d17360fd275af8984d21ce8ba488b","question":"[header] How to cross examine an expert witness [title] Read the expert report. [step] An expert must provide a written report of their findings, and the other side's lawyer must give it to you ahead of time. You should read it several times.\n\nA. Law experts never provide all the information they can get across in one paragraph. Additionally, you should write down key facts to rebut the expert report.\nB. A rule of thumb is to skim through the report before you make a possible counter-claim. Your opponent is entitled to be at least partially impartial-and potentially the most so.\nC. [substeps] Make sure you read everything that goes into the expert's report. It is helpful for expert witnesses to have pictures taken of how they felt when they were cross-examined.\nD. The expert report should form the basis of the expert's testimony in your case. [substeps] If the expert relied on tests or data, that information should be provided as well (not simply the summary provided in the expert report).","prompt":"[header] How to cross examine an expert witness [title] Read the expert report. [step] An expert must provide a written report of their findings, and the other side's lawyer must give it to you ahead of time. You should read it several times.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5713,"error":null,"output":"D"}
{"question_id":"hellaswag:00001977","item_index":199,"attempt":0,"prompt_hash":"2a5dc782be58c0d0b3e2f00c4367509875842805c67ff1551be30cfbb41aad58","question":"A woman sits on a riding lawnmower in the distance. the woman\n\nA. drives the lawnmower towards the camera.\nB. begins pushing the mower.\nC. uses her hands to mow.\nD. adjusts the camera, then begins walking.","prompt":"A woman sits on a riding lawnmower in the distance. the woman","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5716,"error":null,"output":"A"}
{"question_id":"hellaswag:00005711","item_index":200,"attempt":0,"prompt_hash":"c80148c930c1eac1beb62ef6161fcf6684b5c0b710d43072f47bcff279561b44","question":"[header] How to make your own nail transfers [title] Apply a few thick coats of clear nail polish to a section of the parchment paper. [step] Make it a thick layer--if it is not thick enough, the polish might crack when you apply your color. [title] Paint your design on top of the coated section of parchment paper.\n\nA. [step] Make sure that you apply enough in front of your design--that you can see the entire color though the polish. The design makes the color stand out more and it makes your transfer special.\nB. [title] Place your quicksilver nail polish face down on a flat surface and heat it gently using a small candle or butane lighter. [step] The surface should reflect most of the uv light that's released when you apply your polish.\nC. [step] The design can be anything you want, so be creative but ensure that it looks good. A simple design can often be the best.\nD. [step] If you are dyeing your nails to the tips of your finger, be sure to back off you nail polish for a moment to allow the polish to set properly. [title] Allow the ink to dry before adding the transfer.","prompt":"[header] How to make your own nail transfers [title] Apply a few thick coats of clear nail polish to a section of the parchment paper. [step] Make it a thick layer--if it is not thick enough, the polish might crack when you apply your color. [title] Paint your design on top of the coated section of parchment paper.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5712,"error":null,"output":"C"}
{"question_id":"hellaswag:00008380","item_index":201,"attempt":0,"prompt_hash":"2f6ae528200a0fc6aadb0470d5047f213e4058649418ab1e7b16dec620faecb5","question":"[header] How to help your kids feel safe [title] Brainstorm the plans you need. [step] Sit down with your kids and talk about what safety plans you need to have in place. This is a good way to help them feel safe because it involves them in the planning process, which may help them remember the plan better.\n\nA. Consider including things like : [substeps] Having a coach, too. Having someone to supervise your kids during games and practice.\nB. It can also help to discuss ideas with your kids and create pieces for them. [substeps] For example, be sure to give some flexibility about how many people can supervise your children to ensure they will be able to each do the same task at the same time.\nC. It also shows them that you're thinking about their safety. [substeps] Talk about plans to keep you and your kids safe from danger from other people.\nD. [substeps] Determine what needs to be in place in advance so that you can get them the safest foods, as well as the safest places to sleep when going to bed. List your kids' needs as well as your own needs so that they can be included in each plan.","prompt":"[header] How to help your kids feel safe [title] Brainstorm the plans you need. [step] Sit down with your kids and talk about what safety plans you need to have in place. This is a good way to help them feel safe because it involves them in the planning process, which may help them remember the plan better.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5711,"error":null,"output":"C"}
{"question_id":"hellaswag:00000733","item_index":202,"attempt":0,"prompt_hash":"fb5dccca68fd00391e6c481738e3665cd04e1305388a0bd39a7da2f6e0fd120c","question":"She points to vodka, jagermeister and food coloring. She then takes a shaker and pours and vodka and the jagermeister. then\n\nA. , she pours and each drink a glass of the liqueur.\nB. adds some ice in it and shakes it.\nC. , she takes a drink of vodka and pours it on the conch and presents it.\nD. she grabs a cup and places it on her lap as she greats from the glass.","prompt":"She points to vodka, jagermeister and food coloring. She then takes a shaker and pours and vodka and the jagermeister. then","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5715,"error":null,"output":"B"}
{"question_id":"hellaswag:00006784","item_index":203,"attempt":0,"prompt_hash":"167fb7d761d22261bd38a4e09bc2508d6a0b5840c6f9d4c5d4f47c057f972586","question":"[header] How to break in your doc martens [title] Understand that a brand-new pair out of the box is going to hurt your feet like crazy. [step] This means that you should avoid wearing them for a long period of time for the first several wears. That means no walks or dances when they're fresh out of the box! [title] Buy the right fit.\n\nA. [step] Always buy your doc martens one size larger so that you can wear them while still being able to walk. Otherwise your feet won't be able to handle them.\nB. [step] This part can't be changed, so make sure that the retailer helps you to make the correct choice. [title] Take out the laces and rub dr.\nC. [step] The best fit for your feet isn't just how comfortable they are, it's how they feel. In order to figure out a good fit, purchase the right shoes.\nD. [step] Some brand-new doc martens are more expensive than others. Knowing exactly how to find the right fit is one of the keys to getting your feet perfectly fitted.","prompt":"[header] How to break in your doc martens [title] Understand that a brand-new pair out of the box is going to hurt your feet like crazy. [step] This means that you should avoid wearing them for a long period of time for the first several wears. That means no walks or dances when they're fresh out of the box! [title] Buy the right fit.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5712,"error":null,"output":"A"}
{"question_id":"hellaswag:00009993","item_index":204,"attempt":0,"prompt_hash":"a41e7560460252f95596bbe29e110009dc810822b0bb23aaed934637c8393e4b","question":"[header] How to identify a hoof abscess in horses [title] Observe if your horse can put weight on the affected hoof. [step] Hoof abscesses occur when bacteria get trapped in the hoof. Pus then builds up, which increases pressure in the hoof because there isn't much room for the hoof to swell.\n\nA. This pressure becomes extremely painful to a horse, causing it to become lame. If your horse has a hoof abscess, it may not be able to put any weight on the affected hoof.\nB. They can appear as flat, elongated patches on the cow. The enlargement is usually the result of mother or foal hoof damage.\nC. This is why other small animals can instead put weight on an affected hoof. [title] Determine if the horse is not moving much.\nD. The swelling or swelling is not caused by this bacteria, but by a more subtle abscess, and it can interfere with the bandage should you ever see it. [substeps] It is best to refrain from closely examining the hoof for abscess site.","prompt":"[header] How to identify a hoof abscess in horses [title] Observe if your horse can put weight on the affected hoof. [step] Hoof abscesses occur when bacteria get trapped in the hoof. Pus then builds up, which increases pressure in the hoof because there isn't much room for the hoof to swell.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5709,"error":null,"output":"A"}
{"question_id":"hellaswag:00005674","item_index":205,"attempt":0,"prompt_hash":"88b7cfa5579ce2b92ff4277a295aab736484f43aabff9864d7f7ab59dca39317","question":"[header] How to identify and treat different types of diaper rash [title] Look for physical symptoms. [step] Diaper rash presents as inflamed, red and tender skin in the areas covered by a diaper. It often appears on the butt, thighs, and genitals.\n\nA. The rash will typically go away if you contact a medical professional. [substeps] Make sure that the rash isn't in sore places.\nB. Leave these areas alone-they can be revealed later. [substeps] If you see red, inflamed bumps, especially inside or outside the child's vagina, it's likely just a rash.\nC. It may appear at the end of a diaper or just below the skin folds at the bottom of the diaper. [substeps] The rash also occurs on the palms, feet, and buttocks, but it will form around time, often at night and can escalate at any time.\nD. Other symptoms include : [substeps] Warmth to the touch red rash. Pimples, blisters, or other sores [title] Notice changes in your baby's behavior.","prompt":"[header] How to identify and treat different types of diaper rash [title] Look for physical symptoms. [step] Diaper rash presents as inflamed, red and tender skin in the areas covered by a diaper. It often appears on the butt, thighs, and genitals.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5711,"error":null,"output":"D"}
{"question_id":"hellaswag:00001358","item_index":206,"attempt":0,"prompt_hash":"3760dee37b78239207151bfd10560fe996cfcc4a003f91c138c2723d595a0782","question":"Two people are seen walking to one another leads into them playing a volleyball match with others. The man continues taking over by pushing his team mate down and jumping on her back. he\n\nA. leads them into another game where the people hit the ball all over each other and continue playing with one another.\nB. swings her around and ends by throwing her down on the ground and laughing.\nC. then takes a path around to attack his opponent and others surround him and treat them very well.\nD. spikes the ball in the end and cheers with the woman.","prompt":"Two people are seen walking to one another leads into them playing a volleyball match with others. The man continues taking over by pushing his team mate down and jumping on her back. he","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5713,"error":null,"output":"B"}
{"question_id":"hellaswag:00004857","item_index":207,"attempt":0,"prompt_hash":"f4c49dca007b3145e6289db00fb210a80d20bc534498ce6827e79b0525f833b0","question":"[header] How to wear green pants [title] Go with chinos for versatility. [step] Chinos can be easily dressed up or down, making a statement color like green much easier to introduce into your wardrobe. Pair olive green chinos with a neutral-colored button-down shirt to create an office-ready look.\n\nA. You can also pair trousers with these button-down shirts, or pair a pair of khakis with the chinos. [substeps] For example, pairing bandeau top with green pants can look fantastic paired with a simple blouse or oxford shirt.\nB. [substeps] Even if you are trying to go green on occasion, the color of your pants will generally be the desired color. The shirt should have solid gray patterns to the outer edges, like a neutral neutral.\nC. Try a deep forest green pair with rugged boots and a plaid flannel button-down for a more casual style. [title] Wear a cargo style for a casual look.\nD. [substeps] Jeans look great with green pants-if you end up with baggy jeans, pair the chinos with the same of dress shoes or long-sleeved shirt. [title] Stick with a solid color in your wardrobe.","prompt":"[header] How to wear green pants [title] Go with chinos for versatility. [step] Chinos can be easily dressed up or down, making a statement color like green much easier to introduce into your wardrobe. Pair olive green chinos with a neutral-colored button-down shirt to create an office-ready look.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5711,"error":null,"output":"C"}
{"question_id":"hellaswag:00000722","item_index":208,"attempt":0,"prompt_hash":"4ad0f81c549f0fdc6caed88e892b0ba99141984083de2c1f613dd4b3530e2078","question":"A man wearing a green outfit is leading a camel. Two people are sitting on top of the camel. a man\n\nA. tries to ride the camel but the camels legs just keep kicking.\nB. stands behind them relaxing.\nC. is sitting in front of them talking.\nD. is walking next to the camel.","prompt":"A man wearing a green outfit is leading a camel. Two people are sitting on top of the camel. a man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5711,"error":null,"output":"D"}
{"question_id":"hellaswag:00004236","item_index":209,"attempt":0,"prompt_hash":"8db77c7ea7361e7934c626bae783e0cf2731ccccf353c40d2c90f2d7a5218084","question":"[header] How to style short hair (men ) [title] Apply putty to damp hair. [step] Place a small amount of styling putty in the palm of your hand, then rub your hands together to distribute it. Evenly work the product into your hair from end to root.\n\nA. Wait for the putty to sink into your hair. [substeps] It can take anywhere from two to three minutes for it to penetrate your hair.\nB. [substeps] Choose a matte putty instead of a glossy one. If your hair is notably fine, the putty may weigh it down too much.\nC. Leave the putty on your hair for 1 to 1 1/2 hours, or until hair is coated all over with product. Be sure to rinse thoroughly, as nuffing here will not leave your hair as neat as it was.\nD. [substeps] Do not use the putty for dandruff. Dandruff will only make problems worse.","prompt":"[header] How to style short hair (men ) [title] Apply putty to damp hair. [step] Place a small amount of styling putty in the palm of your hand, then rub your hands together to distribute it. Evenly work the product into your hair from end to root.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5712,"error":null,"output":"A"}
{"question_id":"hellaswag:00002378","item_index":210,"attempt":0,"prompt_hash":"f88edaabb4756427de34c8608a994c4519101cd848d090d1ab35a7e19113df03","question":"A man is talking inside a room. he\n\nA. stands up on a ladder to put padding on a hardwood floor.\nB. is holding a large axe to a piece of wood.\nC. is holding a violin and string in his hands.\nD. 's broiling a piece of metal on a stove.","prompt":"A man is talking inside a room. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5712,"error":null,"output":"C"}
{"question_id":"hellaswag:00002666","item_index":211,"attempt":0,"prompt_hash":"9ccb5085776bb7bbe369deb83e95347a2d581441be1c50f50256aea51d578100","question":"A view from the water begins and a few houses and a lot of trees are shown on land. a person underwater in snorkeling gear\n\nA. is now swimming around the bottom of the water and meets up with another snorkeler and they both go into a ship that is underwater as they both hold their flashlights and cameras to explore the ship and everything else at the bottom of the water.\nB. goes to ocean floor swimming air.\nC. appears on screen, busy with the snorkeling game and then shows up with a sandwich in his hand.\nD. shoes and goggles examines the shorefront of a big wave where some swimmers are as the land continues to rise up in the water.","prompt":"A view from the water begins and a few houses and a lot of trees are shown on land. a person underwater in snorkeling gear","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5711,"error":null,"output":"A"}
{"question_id":"hellaswag:00008282","item_index":212,"attempt":0,"prompt_hash":"3f2280d78b136c1227fc36ad588d53ef07756ff9dfd91fc2dac0c70d5c62b329","question":"[header] How to stay focused on your dreams (for teens ) [title] Write down your dream. [step] Be as specific as possible. Give reasons why you want to reach this particular goal and what you hope to get out of it.\n\nA. [title] Tell yourself what you believe will happen in the future. [step] For example, if you are trying to quit smoking, then writing down your goal may be too abstract; the idea is to stay focused on your goal.\nB. The goal should be specific and attainable. [substeps] For example, you could write: \" i hope to be a professor in biology two years from now.\nC. [substeps] If you have long-term plans for the future, make sure they are strong enough to overcome your short term goal. For example, say your dream is to become a successful businessman.\nD. The better you can describe what you hope to accomplish and why, the easier it will be to do what is necessary to get there. [substeps] Start a journal to write in every day as you achieve steps towards your larger goal.","prompt":"[header] How to stay focused on your dreams (for teens ) [title] Write down your dream. [step] Be as specific as possible. Give reasons why you want to reach this particular goal and what you hope to get out of it.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5712,"error":null,"output":"D"}
{"question_id":"hellaswag:00009957","item_index":213,"attempt":0,"prompt_hash":"5e9ab1f06c8a465ce2619ad4fc9228f13242712985fdcfbbbe3d0fa8de71f905","question":"[header] How to force a burp [title] Drink a carbonated beverage. [step] Soda, beer, sparkling water, champagne, or other carbonated beverages contain gases. This is what makes them bubbly.\n\nA. Carbonated water helps to ease abdominal pain and refills the esophagus. It also draws out extra electrolytes from the stomach, resulting in bigger, more draining meals.\nB. The carbonation in soda can make the burp hurt. [substeps] Other beverages require the body to compensate for the carbonation in a burp.\nC. Since soda contains multiple chemicals, soda is highly recommended to help lubricate your esophagus, so don't limit yourself to alcohol-free water because it increases the fizz and swelling associated with a burp. However, there are many different types of carbonated drinks and your taste buds should be adjusting to these as your food remains empty.\nD. If you drink this sort of beverage, then, the gases will build up in your stomach and more than likely trigger a burp, which might help to relieve your abdominal discomfort. Try it! [substeps] Don't just drink the beverage-chug it.","prompt":"[header] How to force a burp [title] Drink a carbonated beverage. [step] Soda, beer, sparkling water, champagne, or other carbonated beverages contain gases. This is what makes them bubbly.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5711,"error":null,"output":"D"}
{"question_id":"hellaswag:00000665","item_index":214,"attempt":0,"prompt_hash":"fef37d93f2360cb3f84f19618898717cbe6b5ff523d84791ba0df27010440f5e","question":"Two woman are outdoors at a zoo riding on the back of a camel. the woman\n\nA. are waving and a man behind the camera is asking them to wave, look and say hi at the camera.\nB. is riding, he has rope.\nC. stands in the middle of the camel while walking at a rapid pace as she jumps and flips herself on the camel.\nD. falls off the camel and has the camel carry her.","prompt":"Two woman are outdoors at a zoo riding on the back of a camel. the woman","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6270,"error":null,"output":"D"}
{"question_id":"hellaswag:00002475","item_index":215,"attempt":0,"prompt_hash":"b5423da86b2e85ce065be566c84f4f71d177db31bd1adbf1db3beb33a1a84d38","question":"A woman is playing with a dog. The dog is catching the frisbee over and over again. the dog\n\nA. is laying down by the woman holding the frisbee over his head.\nB. jumps for a ball.\nC. is chasing the frisbees and the woman receives wile ring from the dog.\nD. ran away with the frisbee but brought it back to the woman.","prompt":"A woman is playing with a dog. The dog is catching the frisbee over and over again. the dog","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6270,"error":null,"output":"D"}
{"question_id":"hellaswag:00007376","item_index":216,"attempt":0,"prompt_hash":"b5249fc76ff7f39cb34afd93fc2f98bc4a57c362b2fee7bc67a5f062d706593b","question":"[header] How to remove mywebsearch [title] Download anti-malware software you trust. [step] Mywebsearch is good at hiding from many anti-virus and anti-malware suites, but malwarebytes anti-malware can find and destroy all of its elements on both windows and mac. Point your web browser to malwarebytes.org and click \" download.\n\nA. \" [substeps] For example, you may want to free mywebsearch and try the safari extension, then you could download links to popular download options. [title] Run the malwarebytes.org homepage.\nB. \" [substeps] Generally, the content you download will be directly on the platform or marketplace the mywebsearch software is designed for. The type of web browsing software will also influence which software you use as well.\nC. \" [substeps] This will automatically download it onto your computer. Windows and macos users can also download malwarebytes.org onto their phones and tablets.\nD. \" [substeps] Windows users will then click \" download free \" to begin the download. When prompted to save the file, choose a location you'll remember, like the desktop, and click \" save.","prompt":"[header] How to remove mywebsearch [title] Download anti-malware software you trust. [step] Mywebsearch is good at hiding from many anti-virus and anti-malware suites, but malwarebytes anti-malware can find and destroy all of its elements on both windows and mac. Point your web browser to malwarebytes.org and click \" download.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6270,"error":null,"output":"D"}
{"question_id":"hellaswag:00003929","item_index":217,"attempt":0,"prompt_hash":"79cba76b6d984b54089e26a21740e6d743234cc17e10cf0d4e87f5be71292102","question":"[header] How to find the best hairstyle for you [title] Relax. [step] Most cuts look great on you! [title] For medium to long lengths, frame your face with bangs and layers. [title] Switch it up! [step] Part your hair in the centre or on the side.\n\nA. [title] Try braiding hair on your forehead. [step] To really bring out the part, this is known as volumizing and it helps out the hairline.\nB. [substeps] Never hide your face behind your hair. Your best cut lets you pull your hair back, clip it up, or wear it down.\nC. [title] Pin that side back! [title] For longer layers, roll-up your hair till you tuck it in. [title] Braid your hair so that it goes all the way around, but not so much that you don't always put a band (hairband! [step] ) in it.\nD. This can mean you set your brows in her direction, or keep them in a loose look. [title] Put put concealer on blemishes or added pigmentation.","prompt":"[header] How to find the best hairstyle for you [title] Relax. [step] Most cuts look great on you! [title] For medium to long lengths, frame your face with bangs and layers. [title] Switch it up! [step] Part your hair in the centre or on the side.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6271,"error":null,"output":"B"}
{"question_id":"hellaswag:00000976","item_index":218,"attempt":0,"prompt_hash":"5889e98fa308ede43f1c64c60d1b46fd44c77b335cea72f92200737030ce5c8d","question":"The person made many mid size hole in the ice and have fishing line hanging deep in the water underneath. the person walking to one of the hole\n\nA. holes and pulled a from a hole in ice and wiped it with a rag.\nB. holes and touch the hole with the rod.\nC. in the ice for the fish to head for the ice.\nD. and checks the fishing line to see if he caught nay fish.","prompt":"The person made many mid size hole in the ice and have fishing line hanging deep in the water underneath. the person walking to one of the hole","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6270,"error":null,"output":"D"}
{"question_id":"hellaswag:00005635","item_index":219,"attempt":0,"prompt_hash":"bc710f0f581879a2d9dce50c02642ee1629835e00aba3f16c006c0116252663b","question":"[header] How to plan an affordable beach wedding [title] Choose your location. [step] A beach wedding is typically free or fairly cheap. If isn't free, you will probably have to purchase a permit that could range from $30 to $300.\n\nA. Often people take days or weeks to arrive-some would agree at least two days or less. Different locations will leave you with favorable views on the beach.\nB. Once you've committed to beach wedding, plan the exact location. Some of the most affordable beach wedding destinations are in mexico, hawaii, and jamaica.\nC. Make sure you've paid out a fair rate of business. [substeps] Cheap beaches also offer younger couples, who can skip the fees.\nD. On that note, you should also consider where your wedding will be held. If you want a beach-themed wedding, the beach can obviously be your destination.","prompt":"[header] How to plan an affordable beach wedding [title] Choose your location. [step] A beach wedding is typically free or fairly cheap. If isn't free, you will probably have to purchase a permit that could range from $30 to $300.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6376,"error":null,"output":"D"}
{"question_id":"hellaswag:00003313","item_index":220,"attempt":0,"prompt_hash":"a97c839b1497f67860e41c2da691a2ddf55ee184cb4010d885b64188d39d9cf3","question":"[header] How to wet wrap hair [title] Set aside time to wrap your hair. [step] Wrapping can take a long time, so make sure you have enough time set aside to fully wrap your hair. The exact amount of time spent wrapping depends on the length of your hair.\n\nA. [substeps] For long hair, wrap your hair twice a day for about 2 weeks. For short hair, wrap up to 3 times a day.\nB. It can even be as long as a few hours, so make sure to plan for enough time to wet your hair thoroughly after wrapping to cover any tangled sections. [substeps] If you're on a tight budget or decide to wrap your hair yourself, hold off.\nC. Some experts recommend wrapping for 3 to 4 hours, while others recommend 5 to 7 hours. Make sure you've developed an understanding of the length of your hair before spending time wrapping it.\nD. The whole process, including the drying, could take three or more hours. [title] Wash and condition your hair as normal.","prompt":"[header] How to wet wrap hair [title] Set aside time to wrap your hair. [step] Wrapping can take a long time, so make sure you have enough time set aside to fully wrap your hair. The exact amount of time spent wrapping depends on the length of your hair.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6375,"error":null,"output":"C"}
{"question_id":"hellaswag:00000471","item_index":221,"attempt":0,"prompt_hash":"021177a08b7d640ecd830ff52526fbdcca16e3eb3d3b5a1f017fd9ce0f070a58","question":"A view of the grass is shown, then a close up of a hand, and finally a man is shown riding on a riding lawn mower in a very large grassy area surrounded by a lot of trees. the man\n\nA. continues to surf and seems to have a great self control as he continues to do various jumps and tricks along the grass.\nB. begins mowing the large green grass with the other camera man following close behind.\nC. waves to the camera, gives a peace sign, then drinks from a can.\nD. is also shown riding around with a small yellow and white dog doing what appears to be a form of exercise.","prompt":"A view of the grass is shown, then a close up of a hand, and finally a man is shown riding on a riding lawn mower in a very large grassy area surrounded by a lot of trees. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6376,"error":null,"output":"C"}
{"question_id":"hellaswag:00004904","item_index":222,"attempt":0,"prompt_hash":"a850c1940efb29ebb175eee34620f84a91c30a1a012b7dff103c3c054b1af092","question":"[header] How to tell the difference between a frog and a toad [title] Look at the hind legs. [step] You can tell the difference between a frog and a toad by the length of its legs. If you can get close enough, get a good look at the back legs specifically.\n\nA. [substeps] A frog's hind legs are very long, as frogs hop more frequently than toads. A frog's hind legs will be bigger than its head and body.\nB. They should look like smooth squares with the 4 legs. If there are any obvious differences between your frog and a toad, there are likely two frog species in your area that are similar to each other.\nC. If your frog has a wide rear end, it will be larger than a toad and have webbed feet, meaning it doesn't have a \" guard \" around its legs. [title] Look at a toad's chest.\nD. Toads' front legs extend from the back legs and these feature subsides, which the fin/hind leg is connected with. [substeps] If you can see them while looking directly at the water, you might be able to tell a toad's back legs as the hind legs are connected.","prompt":"[header] How to tell the difference between a frog and a toad [title] Look at the hind legs. [step] You can tell the difference between a frog and a toad by the length of its legs. If you can get close enough, get a good look at the back legs specifically.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6376,"error":null,"output":"A"}
{"question_id":"hellaswag:00005597","item_index":223,"attempt":0,"prompt_hash":"104dba108e1f09b2c70d357de8fe3606cf73d872185c40728d0f7c0945f9f5ea","question":"[header] How to make a finger loop bracelet [title] Gather your supplies. [step] For this you will need 5 strands of string, each about 2.5 feet (0.8 m) long but they all must be the same length. [title] Fold all of the strings in half.\n\nA. [step] Lay the long strings outside in a corner and the long ones inside out. This will help you do the favors together.\nB. [step] Draw an x in the middle of each strand. [title] Thread the string through a knot in the previous one.\nC. [step] You should fold them all in half separately and then later gather them together to avoid having them twisted together. [title] Once all the strings are gathered together, tie a regular knot at the top of all the strings.\nD. [step] The lengths you need can be anywhere from 10 yards (2.4 m) and 6 feet (1.8 m) long. [title] Connect the ends to separate the long end and the longer end to separately connect the shorter end to the shorter one.","prompt":"[header] How to make a finger loop bracelet [title] Gather your supplies. [step] For this you will need 5 strands of string, each about 2.5 feet (0.8 m) long but they all must be the same length. [title] Fold all of the strings in half.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6376,"error":null,"output":"C"}
{"question_id":"hellaswag:00004688","item_index":224,"attempt":0,"prompt_hash":"87b5992e897f28331d469eac96824f90cc4cb799fb0127d168da9c956d0d5d31","question":"[header] How to clean shiitake mushrooms [title] Wipe away visible dirt. [step] If the mushrooms are still caked with dirt or other debris, gently remove it by hand. Use a clean cloth, dry paper towel, or soft-bristled brush if you prefer.\n\nA. Remove the mushrooms and discard the dirt. [substeps] You can wash the mushrooms in a large pot or bucket instead of just throwing them into the wash machine.\nB. If you don't want to discard your mushrooms, you can contact the recycling center for assistance. [substeps] If you keep your mushrooms at home, you may need to clean them in the sink or in a bucket of soapy water on a day to day basis.\nC. Don't worry too much about getting every bit of dirt. [substeps] There are kitchen brushes designed to help clean delicate foods, such as shiitake mushrooms.\nD. [title] Place the mushrooms on a non-damp paper towel and allow them to dry for about 10 to 15 minutes. [step] If a wet paper towel becomes saturated with moisture, set the mushrooms aside to dry for a day or two.","prompt":"[header] How to clean shiitake mushrooms [title] Wipe away visible dirt. [step] If the mushrooms are still caked with dirt or other debris, gently remove it by hand. Use a clean cloth, dry paper towel, or soft-bristled brush if you prefer.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6372,"error":null,"output":"D"}
{"question_id":"hellaswag:00008761","item_index":225,"attempt":0,"prompt_hash":"62b3bf0df383a05bf2a31b0623c731dc42d9810d024527151dac5d1e5b4cdb6b","question":"[header] How to build your baby's motor skills [title] Stroke the back of your child's hands. [step] You could stroke your child's hands with your own hand, or with an object you want them to become familiar with and grasp. For instance, if you want to encourage motor skills that could help your child grip a rattle, you might stroke the back of their hands gently with a rattle.\n\nA. If your child becomes proficient at counting, you could have them count down while you stroke them. [substeps] Did you teach your child how to focus on numbers? Consider lesson ideas like \" 4 , 4 , 5-0.\nB. When they open their fingers, place it in their palm. [substeps] Your baby might be able to grasp the rattle already, but they probably won't be able to hold it for long.\nC. [substeps] Generally, babies are thrilled by the feeling of their hands touching the shiny surface. They'll be able to grasp any object quickly without fear of injury.\nD. You may also pretend to grapple with inanimate objects by keeping the hands separated. During the stroke, you can make your child grip the objects firmly in order to become familiar with their altered positions.","prompt":"[header] How to build your baby's motor skills [title] Stroke the back of your child's hands. [step] You could stroke your child's hands with your own hand, or with an object you want them to become familiar with and grasp. For instance, if you want to encourage motor skills that could help your child grip a rattle, you might stroke the back of their hands gently with a rattle.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6373,"error":null,"output":"B"}
{"question_id":"hellaswag:00003801","item_index":226,"attempt":0,"prompt_hash":"3f77b33c1c125a5ee6b990d76b4fa63f49ec70a45f13584e30deba2d96e90cfe","question":"[header] How to make a nightlight [title] Take 4 square pieces of cardboard box, each with a length of about 6 centimeter (2.4 in). [step] If you want to make a bigger nightlight, use a card that is bigger. [title] Take 4 kitchen rolls, or any hollow cylindrical object.\n\nA. [step] They can be as small as 15 centimeter (3.5 in), and so long as they are wide enough to fit inside the box. [title] Place the plastic cover over the rolls of cardboard.\nB. [step] Roll about 3 times around the box. [title] Write your name on the cardboard box.\nC. [step] You will need the sort of big box you get at a dollar store. [title] Take one 12-inch (30.\nD. [step] The rolls must be about twice as tall as the length of the card. [title] Make 2 slots on 2 sides of the roll, in such a way that the cardboard box card can fit into them, in order to make a square.","prompt":"[header] How to make a nightlight [title] Take 4 square pieces of cardboard box, each with a length of about 6 centimeter (2.4 in). [step] If you want to make a bigger nightlight, use a card that is bigger. [title] Take 4 kitchen rolls, or any hollow cylindrical object.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6373,"error":null,"output":"D"}
{"question_id":"hellaswag:00007632","item_index":227,"attempt":0,"prompt_hash":"6ea301eb05d2d59978131368931c21dc868b64729d711013fe4b243e82265217","question":"[header] How to do doll like makeup [title] Apply foundation. [step] Use your finger, a blending sponge, or a brush to apply liquid foundation evenly all over the skin of your face in light, circular dabbing motions. [substeps] Use foundation that matches your skin tone, or is just one shade lighter.\n\nA. [title] Apply one layer of eyeshadow. [step] Use a large makeup brush to apply one layer of eyeshadow.\nB. [title] Blend it in with a makeup sponge. [step] Use a blending sponge to blend off the excess foundation.\nC. Allow foundation to dry before applying the fake makeup. Apply blush to your cheekbones, ears, nose and mouth.\nD. Feel free to apply foundation makeup a little thicker than you would normally, as the doll look features a porcelain or plastic face that is unlike natural skin. Add concealer to any blemishes, red patches, or dark circles under the eyes so that your face color is completely even.","prompt":"[header] How to do doll like makeup [title] Apply foundation. [step] Use your finger, a blending sponge, or a brush to apply liquid foundation evenly all over the skin of your face in light, circular dabbing motions. [substeps] Use foundation that matches your skin tone, or is just one shade lighter.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6372,"error":null,"output":"A"}
{"question_id":"hellaswag:00003695","item_index":228,"attempt":0,"prompt_hash":"672fb1fce17d0eb1e9b74bc29a00282b5fc5dd1d46e5ab1d23a8b89f6d7d366c","question":"[header] How to install winamp [title] Download winamp from its official website. [title] Click the yellow button that says \" free download \". [title] Click on free download or go pro (go pro is about 20 us dollars if you want it).\n\nA. [title] Once you downloaded the setup, find it and double click it. [step] Winamp installer will pop up.\nB. [title] If you already know how to use winamp, then click install now. [step] If you already do, click begin, go start] change software panel then click run-ok.\nC. [title] Once you click on free download select \" continue \". [title] Double click the operating system icon at the top of the download window.\nD. [title] Click on install and then click on continue. [step] When the download is finished install winamp once install is complete download will be shown below and it will ask you what you want.","prompt":"[header] How to install winamp [title] Download winamp from its official website. [title] Click the yellow button that says \" free download \". [title] Click on free download or go pro (go pro is about 20 us dollars if you want it).","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6372,"error":null,"output":"A"}
{"question_id":"hellaswag:00001836","item_index":229,"attempt":0,"prompt_hash":"360c6b144c49a6985cfa3aa273d05205bb16ee5b3a39fe36a9a27a880f3ca969","question":"The people begin to add ornments to the christmas tree. A boy pops bubble wrap on the floor. A boy rides around on a toy. a kid\n\nA. cuts post clipped christmas tree branches off of the christmas tree.\nB. lifts up a weight above his head and drops it, then a boy enters lifting it above his neck.\nC. lays under the tree putting things on it.\nD. uses a ukulele on the christmas tree.","prompt":"The people begin to add ornments to the christmas tree. A boy pops bubble wrap on the floor. A boy rides around on a toy. a kid","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6377,"error":null,"output":"C"}
{"question_id":"hellaswag:00003485","item_index":230,"attempt":0,"prompt_hash":"e28cba0e02c5ec6d3ece473312954be2b95bd772f3c77d875f0d1ef1b292b36d","question":"[header] How to choose the right credit card [title] Determine your credit score. [step] Most of the best cards require good to excellent credit. Your fico score (the term refers to the company that created the formula for determining the score) shows lenders how much of a credit risk you are.\n\nA. The score goes from 350 to 850, and the higher your score is, the better. If you live in the united states, you are entitled by law to one free credit report per year from each of the three major credit monitoring companies, experian, equifax, and transunion.\nB. However, you also want to know your credit score. This is not always an exact science, but in fact you must know your credit score.\nC. Thus, paying off your minimum credit scores is a low-cost option. However, that statement is quite unlikely to be the same as paying off your current credit card debt.\nD. Lenders need to be truly concerned about your credit score. Given that their credit score is not generally very high, credit card companies prefer to evaluate both your score and the quality of the credit.","prompt":"[header] How to choose the right credit card [title] Determine your credit score. [step] Most of the best cards require good to excellent credit. Your fico score (the term refers to the company that created the formula for determining the score) shows lenders how much of a credit risk you are.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6372,"error":null,"output":"A"}
{"question_id":"hellaswag:00000835","item_index":231,"attempt":0,"prompt_hash":"590e7ad2fb2a589e6343a679803055758dcc412a95040be76313cab7ff7eb096","question":"They wave again to the camera. The group are shown in their raft as they sail along the river. they\n\nA. cut through deep ramps into rapids.\nB. are shown taking a break in the river.\nC. ride their boards slowly through the rough waters.\nD. are able to get out the water and sit on the raft.","prompt":"They wave again to the camera. The group are shown in their raft as they sail along the river. they","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6375,"error":null,"output":"B"}
{"question_id":"hellaswag:00007624","item_index":232,"attempt":0,"prompt_hash":"e2a943d389983b7f6741cbd8a1bdf66d5a7b58a912ce003e2bcd2579e7650539","question":"[header] How to hold your syrian hamster [title] Let your hamster get used to his environment. [step] If you have just brought your syrian hamster home, he is going to need some time to adjust to his new surroundings. Let him explore his cage for a day or two before you start trying to hold him.\n\nA. You can place him outside in your yard or in a different area. Your hamster may be very excited at the prospect of eating something that him and his owner did not buy.\nB. [substeps] You may decide to allow him to sleep in your crib until he warms up. You may then take him on brief walks every day.\nC. Then, take him outside to a dog park and take him for walks around the neighborhood. [substeps] Make sure to bring a leash and harness as well, as this will help prevent your syrian hamster from bolting away when let out of his cage.\nD. During this time, he will mark his territory by washing his face and grooming a lot (' scenting'). [substeps] Do not mistake scenting for nervousness or anxiety.","prompt":"[header] How to hold your syrian hamster [title] Let your hamster get used to his environment. [step] If you have just brought your syrian hamster home, he is going to need some time to adjust to his new surroundings. Let him explore his cage for a day or two before you start trying to hold him.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6372,"error":null,"output":"C"}
{"question_id":"hellaswag:00001699","item_index":233,"attempt":0,"prompt_hash":"8cdc2bd03c5abafdd6016c3994589897bc32ff59f9724e21783f045a9121c24a","question":"A group of young women and men play a game of volleyball on the beach while wearing swimsuits and trunks. a group of woman and men\n\nA. continue to run after each other while moving their arms and legs.\nB. wears bikinis on a beach.\nC. are playing volleyball, on the beach on an overcast day with all players approaching the ball to hit it.\nD. enjoy a surf surf toss in the ocean, and watch a man of asian descent and two women of asian descent turn around on the seated it.","prompt":"A group of young women and men play a game of volleyball on the beach while wearing swimsuits and trunks. a group of woman and men","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6375,"error":null,"output":"A"}
{"question_id":"hellaswag:00005626","item_index":234,"attempt":0,"prompt_hash":"fd901a939dd17c4a0a377e80603bc8836fa3cc38b13ddb92ae6859e4c28cf232","question":"[header] How to get a yellow belt in gkr karate [title] Know the process. [step] When you start karate you are a 10th kyu white belt. This means you are a students who is 10 grades away from teaching a black belt.\n\nA. If you start the skater start, you can't learn a new style or use techniques. [title] Learn both the black belt and the flag (which are the two most important rule for str.\nB. You don't have to wear a karate uniform at this stage, but if you do you wear a plain white belt. This way the sensei's can see that you are quite new to karate and might need some extra help.\nC. Without this, the seminars on the topics of strikes, punches, kick and evasion techniques would be out the door. Trying to get a yellow belt in shinbok darko (a style class designed for speed and strength) will be impossible and the weaknesses of this class will be an obstacle.\nD. You have to choose the university of district you want to attend so as to avoid attention of your robe, it is important that you check with your state's civil and guard status to see if they have any laws regarding the system. As well as the kinds of red belts that become available, based on different situations or roles, all initiates require a master's degree in karate.","prompt":"[header] How to get a yellow belt in gkr karate [title] Know the process. [step] When you start karate you are a 10th kyu white belt. This means you are a students who is 10 grades away from teaching a black belt.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6372,"error":null,"output":"B"}
{"question_id":"hellaswag:00004166","item_index":235,"attempt":0,"prompt_hash":"8d0a4b18068d8ae7d470591401c172a8ae4b15ae79dc6c4f152d821bccf97f61","question":"[header] How to write a residential lease [title] Check with an attorney. [step] If you are writing a new residential lease form from scratch, it may be helpful to work with an attorney. Some lease provisions will be required by the laws of your state, while some provisions that are required in one state may be illegal in another.\n\nA. Working with an experienced real estate attorney will help you make sure that your lease will be enforceable in your state. [substeps] Typically, if one provision isn't legal, then that provision won't be enforced but the rest of the lease will be.\nB. [substeps] To find an attorney, you can visit your state bar association. Call up and ask for a referral, or run your research.\nC. [substeps] You should read and understand your state's laws carefully before making a request for a residential lease. Federal law prohibits qualified attorneys from drafting residential leases.\nD. Some states only require these provisions in certain areas. If you're unable to find a lawyer familiar with property law, discuss your options with your local or state bar association.","prompt":"[header] How to write a residential lease [title] Check with an attorney. [step] If you are writing a new residential lease form from scratch, it may be helpful to work with an attorney. Some lease provisions will be required by the laws of your state, while some provisions that are required in one state may be illegal in another.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6372,"error":null,"output":"A"}
{"question_id":"hellaswag:00004277","item_index":236,"attempt":0,"prompt_hash":"c63be7f80ce10e399dffa3b0fcf08ab50c944a078f0bda787ec01df5232b9494","question":"[header] How to make slow cooker shredded chicken [title] Place the ingredients in the slow cooker. [step] Add the chicken, liquid, and a dash of salt and pepper to the slow cooker. It's best to use boneless, skinless chicken thighs or breasts, but you can really use any cut.\n\nA. Chicken thighs can be frozen, canned, or caught in a pan. [substeps] Don't worry if you can't find any cut chicken thighs, you can still use them in whatever recipe you like.\nB. You can scale this recipe up, too, as long as you keep the ratio of chicken to liquid the same. [substeps] You can also add any seasoning you like, such as a dash of garlic powder or a sprinkle of seasoned salt.\nC. [substeps] Eggs: crack 4 to 6 eggs into the slow cooker, followed by another dollop of salt and pepper. Clean the colander, going all the way up the inside and legs.\nD. If you can't find boneless or skinless chicken thighs, you can also substitute boneless chicken legs. [substeps] Double check that you add the full grain cooking liquid and that all of the chicken you're cooking in the slow cooker includes whole wheat or broth.","prompt":"[header] How to make slow cooker shredded chicken [title] Place the ingredients in the slow cooker. [step] Add the chicken, liquid, and a dash of salt and pepper to the slow cooker. It's best to use boneless, skinless chicken thighs or breasts, but you can really use any cut.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6374,"error":null,"output":"B"}
{"question_id":"hellaswag:00000263","item_index":237,"attempt":0,"prompt_hash":"e09cc377e9a073e1f33ce058420ddbd9b240d2152ad9d76652582e491236c9b9","question":"A violinist, lindsay stirling is performing live with some other participants on a stage in front of an audience. she\n\nA. begins by playing the violin while the other performers dance in a choreographed fashion.\nB. is giving brief commentary on her performance.\nC. sings her song right into the microphone followed by dancing gracefully and holding her body up and facing the audience as she finishes.\nD. is blowing large amounts of smoke while on a set of drums and a guitar as some people watch her on a stage.","prompt":"A violinist, lindsay stirling is performing live with some other participants on a stage in front of an audience. she","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6374,"error":null,"output":"A"}
{"question_id":"hellaswag:00004746","item_index":238,"attempt":0,"prompt_hash":"ba630b9f80144443e8a6322b2df4ec30dff207f035fdf1c4c8366d1b09352984","question":"[header] How to scotchgard shoes [title] Take your shoes to a well-ventilated area. [step] Scotchgard fabric protector is high in fumes, which you want to avoid breathing for too long. Going outside to spray your shoes is the best option if the weather is good.\n\nA. [substeps] Starchy material thalights your shoes easily. You can buy it at any major retailer that sells school supplies, such as walmart.\nB. [title] Moisten the sole of your shoe. [step] You don't have to do this; just wet it.\nC. Otherwise, go into a garage or open the windows in the room. [substeps] Keep your kids and pets away while you spray your shoes.\nD. [substeps] Do not spray your shoes with water or the bleaching could damage them. [title] Lay your shoes out on a towel.","prompt":"[header] How to scotchgard shoes [title] Take your shoes to a well-ventilated area. [step] Scotchgard fabric protector is high in fumes, which you want to avoid breathing for too long. Going outside to spray your shoes is the best option if the weather is good.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6371,"error":null,"output":"C"}
{"question_id":"hellaswag:00004653","item_index":239,"attempt":0,"prompt_hash":"70b30ce24002f80d29be6a6377a80b8abf50e8eb8583af9fada7ef175454a2ed","question":"[header] How to make a non alcoholic mimosa [title] Opt for sparkling white grape juice to mimic champagne. [step] If you want to go for the classic mimosa taste, pick up some sparkling white grape juice at a grocery store. Sparkling white grape juice has a similar taste to champagne and will provide the traditional taste of a mimosa.\n\nA. [substeps] If you like to drink on the rocks, you can take advantage of the bubbles that the carbonated champagne provides. [title] Whip up sparkling white champagne for a traditional, classic mimosa.\nB. You can also consider adding a little orange flavor to it to satisfy these tastes. [title] Use a floating \" float \" type of champagne to add flavor to a classic mimosa.\nC. [title] Add creme de cacao to your mimosa. [step] Vintage country mimosas are made from white grape juice.\nD. [title] Try ginger ale for a strong taste. [step] You can get a little creative when making your mimosa.","prompt":"[header] How to make a non alcoholic mimosa [title] Opt for sparkling white grape juice to mimic champagne. [step] If you want to go for the classic mimosa taste, pick up some sparkling white grape juice at a grocery store. Sparkling white grape juice has a similar taste to champagne and will provide the traditional taste of a mimosa.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6373,"error":null,"output":"D"}
{"question_id":"hellaswag:00002404","item_index":240,"attempt":0,"prompt_hash":"9e52cf128358ae95efc72fe3434509879b3ce7e8c22bbb4df08fc491d9d48292","question":"A woman speaks to the camera while turning her face back and fourth. She holds up a product and wipes it all along her face. she\n\nA. tosses it to a woman next to her.\nB. then puts the product into her mouth while still speaking.\nC. holds up a toothbrush while still speaking to the camera.\nD. washes her faces and wipes it down with a rag still speaking to the camera.","prompt":"A woman speaks to the camera while turning her face back and fourth. She holds up a product and wipes it all along her face. she","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6374,"error":null,"output":"C"}
{"question_id":"hellaswag:00000099","item_index":241,"attempt":0,"prompt_hash":"f32b00da8752f93c5b59345f3ce930e152c73718529e076b416aeb8e8e8edae2","question":"A man is seen speaking to the camera while a shot of a person playing basketball shows behind. the man\n\nA. continues going around the basket while the camera is sitting in the back.\nB. continues to speak and leads into more clips of him playing tennis.\nC. pulls the ball up on the court while still speaking to the camera.\nD. continues speaking as more shots are shown of the boy playing basketball.","prompt":"A man is seen speaking to the camera while a shot of a person playing basketball shows behind. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6375,"error":null,"output":"D"}
{"question_id":"hellaswag:00000516","item_index":242,"attempt":0,"prompt_hash":"95993261f75c868124534758afb2470be9f4eecfe9c3612e087c8ca55d8dd5ad","question":"A large white truck with its hood open backs up. Marc man speaks to the camera by a vintage red car. he\n\nA. wipes some car cleaner on the driver's side door and polishes it.\nB. proceeds to wipe down the shiny black car with sponge.\nC. is painting the inside windshield purple.\nD. vacuums back onto his and someone else's hose.","prompt":"A large white truck with its hood open backs up. Marc man speaks to the camera by a vintage red car. he","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6376,"error":null,"output":"B"}
{"question_id":"hellaswag:00003887","item_index":243,"attempt":0,"prompt_hash":"76ed5ba04db96f96e311fca3dd0f97a0a8a097fdbd5da9230e0c2ae599ea598d","question":"[header] How to pack and move your bird [title] Visit your veterinarian. [step] Before you make any arrangements to move with your bird, you will need to have it checked out by a veterinarian. This will help you determine if it is healthy enough to travel.\n\nA. Make an appointment at your local animal shelter or pet supply store. Bring your bird to be looked at closely by a vet certified by the avian national certification commission for the certification of birds.\nB. It can also help to determine if it needs any blood thinners. You will also need to get a detailed medical history.\nC. They may also give it a medical exam to rule out other illnesses. [substeps] Your doctor will recommend other causes of illness, such as infection, malaria, diseases, conditions that may affect her ability to talk, or the different ways in which her breath smells.\nD. Your vet will evaluate your bird and likely provide you with tips to help it stay healthy during the move. [substeps] Your doctor may suggest that you begin giving your bird immunity and digestion-boosting supplements a month before you move.","prompt":"[header] How to pack and move your bird [title] Visit your veterinarian. [step] Before you make any arrangements to move with your bird, you will need to have it checked out by a veterinarian. This will help you determine if it is healthy enough to travel.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6376,"error":null,"output":"D"}
{"question_id":"hellaswag:00005478","item_index":244,"attempt":0,"prompt_hash":"e1bf20151919f0811ae06f86e398442dc64ed89aabfa3326f2dbfa3b6b1762e0","question":"[header] How to apply root concealer [title] Choose the right shade. [step] The range of shades available varies from brand to brand, but most of them offer around five options. These are typically light blonde, dark blonde, warm brown, brown and dark brown.\n\nA. [substeps] A good rule of thumb is to choose just a color that matches your natural hair color. Start with a slightly darker shade (like blonde) and work your way up to lighter colors such as light brown, dark auburn or purples.\nB. Find shades that are similar to your hair color and other skin tones to work on your complexion. Generally, black and warm brown shades work best for sun kissed skin.\nC. Get the product that most closely matches your needs. [title] Dry and style your hair as usual.\nD. Avoid applying your favorite shade and consider using more or less depending on your skin tone. If your skin is very tanned, then you might be able to find a shade that compliments your skin tone.","prompt":"[header] How to apply root concealer [title] Choose the right shade. [step] The range of shades available varies from brand to brand, but most of them offer around five options. These are typically light blonde, dark blonde, warm brown, brown and dark brown.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6374,"error":null,"output":"D"}
{"question_id":"hellaswag:00009292","item_index":245,"attempt":0,"prompt_hash":"783a3372c51fa1c23a288720b8f57d517099a9e3d7514a62e434d73050c3488b","question":"[header] How to deadhead petunias [title] Check what type of petunias you are growing. [step] Find the seeds or the plant markers. If they are new types of petunias, like wave or tidal wave, they don't need to be deadheaded.\n\nA. However, if they are in pairs, they may not be deadheaded until they produce more than one flower. Look for shriveled, wilted stems, or other signs of mold.\nB. Opt for a variety where the flowers center stem outward. [substeps] How old the flowers are does not matter, as these plants grow year round, and do not require deadheading.\nC. [substeps] Many new petunias have been engineered to be low-maintenance. They will fill out without being deadheaded.\nD. Try the following plants if you want to eliminate the ones that are still blooming in mid-spring. [substeps] Semi-flowering petunias are in one of three categories: non-flowering species, which are small purple flowers with leaf marks and leaves arranged together.","prompt":"[header] How to deadhead petunias [title] Check what type of petunias you are growing. [step] Find the seeds or the plant markers. If they are new types of petunias, like wave or tidal wave, they don't need to be deadheaded.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6372,"error":null,"output":"C"}
{"question_id":"hellaswag:00007328","item_index":246,"attempt":0,"prompt_hash":"317b260af935582959cd963ba6f2c9a6000d13550b52dcb7199f7529d5bcf3ea","question":"[header] How to stop taking zoloft [title] Consider why you want to discontinue taking zoloft. [step] You should generally continue taking zoloft if the medication has effectively controlled your depression or disorder. However, there are good reasons to discontinue or change your medication under the supervision of a doctor.\n\nA. [substeps] People often leave a prescription for a different drug for a different reason. If you are taking zoloft to control the effects of your depression, you will need to consult a doctor.\nB. [substeps] For example, you might try other anti-depressants or ssris. Ssris may lead to weight gain or obesity.\nC. To support your reasons for discontinuing zoloft : [substeps] You've not taken zoloft for a long time because of its high potential for unwanted changes in mood. You've been diagnosed with hyperglycemia.\nD. These reasons include : [substeps] If you experience severe or continuous side effects. If your depression or disorder is not under control with zoloft.","prompt":"[header] How to stop taking zoloft [title] Consider why you want to discontinue taking zoloft. [step] You should generally continue taking zoloft if the medication has effectively controlled your depression or disorder. However, there are good reasons to discontinue or change your medication under the supervision of a doctor.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5491,"error":null,"output":"D"}
{"question_id":"hellaswag:00008641","item_index":247,"attempt":0,"prompt_hash":"559883e0cbbf9810eeec82bc41f6ea962ecbc7f607834ca73313ec9771e43495","question":"[header] How to deal with sex problems in marriage [title] Choose a time when you are both alone and not distracted. [step] Having a conversation about sexual issues can be difficult and awkward, especially if you are both feeling the lack of sexual chemistry in the bedroom. Don't spring the conversation on your partner when you're having dinner at a restaurant or surrounded by friends.\n\nA. Setting up a time that your partner is available and isn't distracted can help you take a break from pornography and get to know one another better. [substeps] If your partner is busy with work or has a busy day, this may be uncomfortable for you both.\nB. Look for a time when you are both alone, in a private space, to bring up the topic. [substeps] A good time may be when you are both getting ready for bed, or after you have dinner together at home.\nC. This can come across as snooping in your partner's personal space, or not recognizing when something is wrong. [substeps] Try to find a quiet, distraction-free space where no one is around to tell you what's on your mind.\nD. [substeps] If you know you might have a problem with religious or sexual ideas, then focus on the conversation in private. Ask your partner what they think is missing in the conversations you have before facing them.","prompt":"[header] How to deal with sex problems in marriage [title] Choose a time when you are both alone and not distracted. [step] Having a conversation about sexual issues can be difficult and awkward, especially if you are both feeling the lack of sexual chemistry in the bedroom. Don't spring the conversation on your partner when you're having dinner at a restaurant or surrounded by friends.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5491,"error":null,"output":"B"}
{"question_id":"hellaswag:00002207","item_index":248,"attempt":0,"prompt_hash":"8a85fc81995609e959fdf4ea4c9fd3f1bdb2f7f9b15e6de22fbbbffd358f797e","question":"A news anchor talks in a newsroom. A coffin and is carried by a wagon along with military personell. a military man\n\nA. is carried on a jet ski through the city, past a group of workshops, on 19th street.\nB. presents a flag to a guest.\nC. carries supplies and puts them on top of a wagon to be interred.\nD. poses for picture and is sold a sticker from a bicycle.","prompt":"A news anchor talks in a newsroom. A coffin and is carried by a wagon along with military personell. a military man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5492,"error":null,"output":"B"}
{"question_id":"hellaswag:00000324","item_index":249,"attempt":0,"prompt_hash":"cb42c5437f62c8edaed4eed2ded5b927995126ff43e612f4ca0047ed899c6235","question":". a woman with her hair up in a towel\n\nA. introduces her video showing how she washes her face.\nB. sits on a behind wood chair with her legs curled up underneath her.\nC. is wearing a bikini with one leg tied behind her head and looking down at herself.\nD. walks into the bathroom with her hair down.","prompt":". a woman with her hair up in a towel","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5491,"error":null,"output":"D"}
{"question_id":"hellaswag:00009807","item_index":250,"attempt":0,"prompt_hash":"86fccea7053c14b86c66b66b542076d0c382f5a1f0753fbbe30559ec20028fdd","question":"[header] How to apply lipstick without liner [title] Select a brush to apply your lip color with, if you have one. [step] It will be easier to apply your lip color if you have a brush. Typically, long brushes with a pointed tip work well for lip color application.\n\nA. [substeps] With a brush, you have a lot of control over the line you draw. You can draw cleaner, straighter lines with a brush as opposed to a lip liner.\nB. You may also look for good ones that say nutrition in the edge or that makes your chin look fuller. Hold your brush in your dominant hand at a high angle.\nC. If you want to add just a little more texture to your lips, a brush can help you apply your gloss after it dries. In other words, it will allow your lip paint to run onto the brush without some clumping.\nD. Do not use a brush that has a pointed tip, which cannot blend well with your lipstick. [substeps] Look for a brush with a soft tip.","prompt":"[header] How to apply lipstick without liner [title] Select a brush to apply your lip color with, if you have one. [step] It will be easier to apply your lip color if you have a brush. Typically, long brushes with a pointed tip work well for lip color application.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5492,"error":null,"output":"A"}
{"question_id":"hellaswag:00000851","item_index":251,"attempt":0,"prompt_hash":"5c7ca7915881e039f7c2c96ac3b90cb1a5d6b73ce8e00b53b13c1e2244e8242e","question":"First she fills her shaker up halfway with ice and then she adds vodka and chambord and she puts one ounce of both of them and then she adds a little bit of pineapple juice and cranberry juice. next she\n\nA. puts a lot of acetone from a bottle into her shaker.\nB. shakes everything up and strains the drink into the cocktail cup.\nC. adds a quarter ounce of jack daniels and she shakes it and then she pours a garnish into the shaker, then she adds a drop of jack from the shaker to it and glides it right on to the opposite rim.\nD. adds a liquid solution and a cherry in the solution and she gargles about three times and she puts the loaded shaker in the kitchen sink.","prompt":"First she fills her shaker up halfway with ice and then she adds vodka and chambord and she puts one ounce of both of them and then she adds a little bit of pineapple juice and cranberry juice. next she","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5551,"error":null,"output":"B"}
{"question_id":"hellaswag:00003923","item_index":252,"attempt":0,"prompt_hash":"3a643f2ced73340c3ab207b2c5e6800a5896b43696cc000600189d217c9a19fd","question":"[header] How to get your wedding party properly prepped to talk [title] Meet all party participants. [step] If you're the bride, groom, or maid of honor, provide some icebreakers, perhaps even a quick game to get everyone acquainted and more at ease. All the wedding events will be uncomfortable if you don't meet the other attendants.\n\nA. Some photos may feel appropriate, but also you don't want to be out and about in the middle of a crowded city. [title] Get out your waterproof leather purses.\nB. Make sure that everyone you invite knows the etiquette of the occasion. Doing so should help and not affect the conversation.\nC. If you aren't formally introduced, make it a point to introduce yourself and explain how you know the bride or groom. Try, \" hi, i'm rick.\nD. Still, getting to know the staff while jumping around can help out. [substeps] Serving a meal and getting to know various faces of the crowd can take a little time.","prompt":"[header] How to get your wedding party properly prepped to talk [title] Meet all party participants. [step] If you're the bride, groom, or maid of honor, provide some icebreakers, perhaps even a quick game to get everyone acquainted and more at ease. All the wedding events will be uncomfortable if you don't meet the other attendants.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5550,"error":null,"output":"C"}
{"question_id":"hellaswag:00007558","item_index":253,"attempt":0,"prompt_hash":"0a08e5be2eb1c57f01bd2c2c7b8846db12248a28289295e1e5c312db98a46385","question":"[header] How to buy a carbon offset [title] Research how carbon offsets work. [step] When you buy a carbon offset, your money goes to support a project happening somewhere else in the world that reduce carbon dioxide emissions. Tree planting and soil management projects actively remove carbon dioxide from the atmosphere, while energy efficiency projects, such as wind and solar farms, reduce overall emissions.\n\nA. Learning more at these levels will help you learn how these outputs work. [substeps] For local companies, observe them in the same way you observe a computer.\nB. [substeps] Other environmental environments may be contributing to how the buildings are affected by carbon emissions. Learning about fertilizer use gives you a much better idea of how energy emissions are produced and how they effect trees, livestock, and people.\nC. [substeps] Projects are undertaken all over the planet, so you have your choice of projects to support with your carbon offset dollars. Choose a project based on your own personal interests.\nD. [title] Plant carbon offsets if you want to get started. [step] In their most basic forms, carbon emissions are planted so that they aren't seeded during the season.","prompt":"[header] How to buy a carbon offset [title] Research how carbon offsets work. [step] When you buy a carbon offset, your money goes to support a project happening somewhere else in the world that reduce carbon dioxide emissions. Tree planting and soil management projects actively remove carbon dioxide from the atmosphere, while energy efficiency projects, such as wind and solar farms, reduce overall emissions.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5549,"error":null,"output":"C"}
{"question_id":"hellaswag:00010003","item_index":254,"attempt":0,"prompt_hash":"86b03b28cff51b06003ef35aec344215be5f64a5764c38c5bc0d0a48e674f2ac","question":"[header] How to enforce change in your personal life and work [title] Overcome the fear of failure. [step] Resistance to change may be deeply embedded in fear. We tend to cling to the past which is more secure and predictable.\n\nA. If something has worked well for us in the past, we may fear not accomplishing as much in the future. [title] Change your bad habits of the unwillingness to learn.\nB. Subjectively we see or experience uncertainty--sometimes because we are so afraid of something we not know what will happen someday. This is partly what protects us from the potential for panic.\nC. Now, our fears come alive and we need to embrace the past because it is the foundation of our current situation. Failure to be resilient leads to attempts to manipulate and suppress our emotions and transform our ideas into actions.\nD. Often we can stifle and discourage possible things to happen by dwelling on the possibilities which are too much to imagine. You may be in a battle against a fear that you like to focus on.","prompt":"[header] How to enforce change in your personal life and work [title] Overcome the fear of failure. [step] Resistance to change may be deeply embedded in fear. We tend to cling to the past which is more secure and predictable.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5549,"error":null,"output":"A"}
{"question_id":"hellaswag:00006431","item_index":255,"attempt":0,"prompt_hash":"1fe8765f0fafb3435f3489d6777346949a8676849568fbf702214e580ad4aa4c","question":"[header] How to grow a handlebar mustache [title] Gather your supplies. [step] First, choose a wax for your hair. There are many kinds of mustache wax available.\n\nA. If your mustache hair is thick and wiry you should buy a stiff wax. Inversely, if your mustache hair is thin, you can buy a softer wax.\nB. You need to pick a hair type that you like. [substeps] Consider growing mustache hair on a slightly smaller facial section.\nC. Although these wax aren't as sparkly as your everyday specialty wax, they're still attractive. You may be able to use other wax when you're creating your facial beard.\nD. You can go for the shaved version or the more traditional style. Most handlebar mustache wax is made from natural ingredients.","prompt":"[header] How to grow a handlebar mustache [title] Gather your supplies. [step] First, choose a wax for your hair. There are many kinds of mustache wax available.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5549,"error":null,"output":"A"}
{"question_id":"hellaswag:00002861","item_index":256,"attempt":0,"prompt_hash":"1af77bcc240add3f046585d098c209932db7f84b16345586a7dd617e184ef518","question":"A man in a black speedo is standing on a diving board with his hand down on his sides. He then begins running and does two hops to get to the end of the board. finally\n\nA. he jumps down and is shown again at the bottom.\nB. he goes back down as he gives a thumbs up.\nC. , he is at the end of the diving board and does a flip into the water as the crowd on the side watches.\nD. he takes off and does three belly kicks to wind up before making it to the end, and there shorts has been thrown from the beginning of the board.","prompt":"A man in a black speedo is standing on a diving board with his hand down on his sides. He then begins running and does two hops to get to the end of the board. finally","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5536,"error":null,"output":"C"}
{"question_id":"hellaswag:00009477","item_index":257,"attempt":0,"prompt_hash":"36e6c6061bb8548e988ce8ea8781f474f2cea88be9665e78c3fc94ee10535e9b","question":"[header] How to stop being bullied [title] Don't react with more bullying. [step] Although it might seem tempting to fight back with bullying, it can cause further problems. Bullies enjoy feeling as though they have power over you or that they are able to get a reaction from you.\n\nA. Know your strengths and weaknesses and take control. Remember that bullies are considered strong people and deserve to be treated with respect.\nB. Bullying back or getting angry will let the bully know they are hurting you. Do not respond to the bully with more aggression in order to get them to leave or stop.\nC. Not only will this lower your chances of getting bullied by a bully, but it also prevents you from getting the reaction you want. [substeps] You can try to break the bullying cycle by not being mean and allowing it to just get worse.\nD. [substeps] When you are bullied in the workplace, it can be one of the most intense times of the day and you might feel weak or wobbly in your seat. You might find yourself climbing all over your co-workers to get your attention.","prompt":"[header] How to stop being bullied [title] Don't react with more bullying. [step] Although it might seem tempting to fight back with bullying, it can cause further problems. Bullies enjoy feeling as though they have power over you or that they are able to get a reaction from you.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5529,"error":null,"output":"B"}
{"question_id":"hellaswag:00007080","item_index":258,"attempt":0,"prompt_hash":"827e6af706e6f517231d53b7b6c982edf31bc92f556b2151abb7acdba64dd7c7","question":"[header] How to use connectify hotspot [title] Check the version of your operating system. [step] Connectify hotspot can be used with windows server 2008 r2, windows 2012, windows 7 , 8 , 8.1, and 10. If you have an older version of windows, such as xp or vista, you'll need to upgrade before installing connectify hotspot.\n\nA. [substeps] All operating systems designed for windows server 2008 are designed with the server citency feature. [title] Understand the connectify hotspot menu.\nB. This option will be listed in the summary bar at the top of your browser. [title] Find the tab labeled \" connect \" at the bottom of your desktop.\nC. Find out which version of windows you have by pressing, then typing the word. Click \" about your pc \" or \" about this computer \" and look next to \" edition.\nD. [substeps] You can use this to optimize or remove dots and dashes on your desktops. [title] Click add view & settings] network.","prompt":"[header] How to use connectify hotspot [title] Check the version of your operating system. [step] Connectify hotspot can be used with windows server 2008 r2, windows 2012, windows 7 , 8 , 8.1, and 10. If you have an older version of windows, such as xp or vista, you'll need to upgrade before installing connectify hotspot.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5529,"error":null,"output":"C"}
{"question_id":"hellaswag:00004167","item_index":259,"attempt":0,"prompt_hash":"739eb709a5e54882792da799a680d7e940991e807ae406130ddadece04fcbd4c","question":"[header] How to make posters for a school election [title] Figure out the number of posters you are allowed to hang. [step] Try asking the person who is in charge of the school elections or the principal. Choose a time when they are not busy and, if you easily forget things, bring a notebook and pen to take notes.\n\nA. [substeps] If you have a locker, try to choose a day when the people in charge of the school elections are not busy, such as recess and the winter holidays. During the school election, ensure that posters should not be lost.\nB. [title] Figure out how many posters you must have. [step] The poster must be at least 8 inches by 8 inches (20.\nC. A quick internet search is always good to get a good idea of the number of votes you could raise. [title] Find these posters and use these to put signs on schools' doors.\nD. Ask for other specific details, such as where/not you are allowed to hang the posters. Also, ask what they would like you use to hang the posters.","prompt":"[header] How to make posters for a school election [title] Figure out the number of posters you are allowed to hang. [step] Try asking the person who is in charge of the school elections or the principal. Choose a time when they are not busy and, if you easily forget things, bring a notebook and pen to take notes.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5533,"error":null,"output":"B"}
{"question_id":"hellaswag:00009624","item_index":260,"attempt":0,"prompt_hash":"2e1255cc0811408384c7f6dc2f67986296e0c716ce4c9217c63b9862a20348b3","question":"[header] How to give a sensual massage [title] Clean and groom yourself. [step] To make the experience as sensual as possible, both of you should feel comfortable, clean, and fresh. Bathe and groom yourself, paying particular attention to your hands and fingernails.\n\nA. Put on lotion, wash your hands, and polish your nails. [substeps] If you feel the urge to lick your lips while you are shaving, you should avoid it.\nB. Your fingers should be clean and clipped before you give a sensual massage. [substeps] If you're giving your partner a massage, consider bathing together to initiate the intimacy.\nC. Lather and scrub your body with a loofah to remove any lingering oils, soap, or dirt. Use a fine-tooth comb to groom any hair that falls between your palms.\nD. After showering, apply moisturizer, and get your skin nice and smooth. [substeps] If you use aftershave at the time of use, do so a week before your massage.","prompt":"[header] How to give a sensual massage [title] Clean and groom yourself. [step] To make the experience as sensual as possible, both of you should feel comfortable, clean, and fresh. Bathe and groom yourself, paying particular attention to your hands and fingernails.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5528,"error":null,"output":"B"}
{"question_id":"hellaswag:00001807","item_index":261,"attempt":0,"prompt_hash":"dc7ae1df91a7f1150ee1ddd94efd5aec78783a17a7da9051a0f8c5140b5d5a4e","question":"First the man wakes up with his eyes buck wide and he washes his face and gets dressed. Then he heads outside and plays a game of solo shuffle and he scores good points. he\n\nA. then goes back to the state of his face but airs it so it looks like he just inhaled some turn it could be scotch it looks like he just spit it.\nB. laughs his entire time and the other team keeps playing to keep score.\nC. even pushes the ball using a strange angle and the ball still makes it to number 10.\nD. completes the game, stands up and claps and the crowd cheers.","prompt":"First the man wakes up with his eyes buck wide and he washes his face and gets dressed. Then he heads outside and plays a game of solo shuffle and he scores good points. he","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5532,"error":null,"output":"D"}
{"question_id":"hellaswag:00000411","item_index":262,"attempt":0,"prompt_hash":"516b082e3e39ec7c9eb475454a49e2e0539e81c3469e062c8f8ea01192e67dc8","question":"A man wearing a beanie is inside a room. He is talking in front of a disassembled bike. he\n\nA. shows a small tool that he has installed into the bike.\nB. cleans the tire and scissored it, replacing areas where he thinks it's damaged.\nC. shows the different tools he needs to replace the chain.\nD. uses a scraper to remove peeling paint from the tire.","prompt":"A man wearing a beanie is inside a room. He is talking in front of a disassembled bike. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5532,"error":null,"output":"C"}
{"question_id":"hellaswag:00009954","item_index":263,"attempt":0,"prompt_hash":"eff5c3fbae2463691e8e9ea6103371a76adf1106de5a9a93a26a68efc74bce1b","question":"[header] How to make an amazon account [title] Get started. [step] Go to amazon.com. In the upper right-hand corner, you'll see a menu marked \" your account.\n\nA. \" click on \" manage account \" to switch this to \" new. \" from this link, you'll be redirected to the amazon settings page.\nB. \" under the heading, you'll see the password, passcode, and e-mail, which are all desired extensions of your account. Click on the e-mail link, which will take you to amazon.com.\nC. \" scroll down and click the \" account \" button, then choose the account you want to reserve your account. [title] Select the text field for the account you want to reserve.\nD. \" when you hover over it, a drop-down menu appears. Click on the \" start here \" link directly beneath the yellow \" sign in \" button.","prompt":"[header] How to make an amazon account [title] Get started. [step] Go to amazon.com. In the upper right-hand corner, you'll see a menu marked \" your account.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5533,"error":null,"output":"D"}
{"question_id":"hellaswag:00005682","item_index":264,"attempt":0,"prompt_hash":"7cec7f562c6afcc2f14f02d75e9ec6dfbb26938b35e432f564e8223cf317a400","question":"[header] How to breed dwarf hamsters [title] Pick out male and female dwarf hamsters. [step] If you don't already have a sexually mature male and female hamster, you should find one at the pet store. Check the genitalia to ensure that you are getting a male and a female.\n\nA. If the genitalia isn't visibly red, they are not dwarf hamsters. [substeps] Female dwarf hamsters are docile, quiet animals and shy from humans.\nB. [substeps] Males are generally considered more sexual than females and dwarf hamsters only coexist with female hamsters and dwarf hamsters. Female hamsters, however, cohabitationers and males all share the same racial background in scientific disciplines.\nC. It is pretty tricky to tell the difference. For males, the testicles and anus are far apart.\nD. You should also find out if you have a lemur, pelvis, or other fat of any sort. [substeps] Dwarf hamsters have meeled conditions, which can cause the meats to overreact.","prompt":"[header] How to breed dwarf hamsters [title] Pick out male and female dwarf hamsters. [step] If you don't already have a sexually mature male and female hamster, you should find one at the pet store. Check the genitalia to ensure that you are getting a male and a female.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5528,"error":null,"output":"C"}
{"question_id":"hellaswag:00008537","item_index":265,"attempt":0,"prompt_hash":"576add43795d5ec3a5f5d2715440d1ec1f624a43f23f79e8714993dbb42a8796","question":"[header] How to post something on linkedin on pc or mac [title] Go to https: // www.linkedin.com in a web browser. [step] If you're not already signed in to your linkedin account, enter your username and password into the blanks at the top of the screen, then click. [title] Click the share an article, photo, video, or idea box.\n\nA. [step] It's near the top-center part of the page. This expands the box to display additional post options.\nB. [step] Find this in the drop-down menu, then click share it. [title] Describe what the sharing box is about.\nC. [step] Your share item will be posted using the share dialog box. [title] Click add project, create new project, link, or save.\nD. [step] Tap \" join profile \" under the share link. [substeps] You can add or delete your posts, videos, and ideas into the sharing box beneath your profile image.","prompt":"[header] How to post something on linkedin on pc or mac [title] Go to https: // www.linkedin.com in a web browser. [step] If you're not already signed in to your linkedin account, enter your username and password into the blanks at the top of the screen, then click. [title] Click the share an article, photo, video, or idea box.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5527,"error":null,"output":"A"}
{"question_id":"hellaswag:00007591","item_index":266,"attempt":0,"prompt_hash":"cdb50cca1cbf8533263113ceaf1b56f65dd7504277ff76a83366f679c1c4aedf","question":"[header] How to use flipboard for windows 8 [title] Download and install flipboard from the windows app market. [step] Just select the app from the application list to begin downloading and installing it on your pc. [title] Launch flipboard.\n\nA. [step] Click the \" start \" button to begin an internet search and launch the app. [title] Navigate to the website that you want to install flipboard on your windows 8 device.\nB. [step] Locate the app on your desktop and tap on it. This will launch flipboard and begin installing it on your computer.\nC. [step] Right-click on its icon on your desktop. [title] Visit the email address associated with flipboard.\nD. [step] Tap its icon from your desktop to launch the application. The first thing you'll see is the login screen.","prompt":"[header] How to use flipboard for windows 8 [title] Download and install flipboard from the windows app market. [step] Just select the app from the application list to begin downloading and installing it on your pc. [title] Launch flipboard.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5532,"error":null,"output":"B"}
{"question_id":"hellaswag:00005604","item_index":267,"attempt":0,"prompt_hash":"3af26e57966cbf5acafe3d88b422989c2e182129d01eb122b08141e32de5b9f7","question":"[header] How to get bad smells out of a microwave [title] Place 4 teaspoons (20 ml) of white vinegar in a microwave-safe bowl with water and microwave on high for 6 minutes. [step] Leave the solution in the microwave to steam for an hour and then wipe the insides. Clean the microwave with soap and water.\n\nA. [substeps] An alternative is to place a bowl of white vinegar in the microwave--without turning on the microwave--and allow the vinegar to sit there until the odor is gone. Then clean the microwave with soap and water.\nB. Give the vinegar time to settle, but do not scrub. This method will not work if all you smell is the food.\nC. [substeps] If your microwave has a \" wipes \" function, you may not be able to clean it. [title] Add food flavorings (salt, basil, cinnamon, lemon).\nD. [substeps] Vinegar has a powerful residual smell if not removed quickly. With about 5 minutes of steam to a room or restaurant, vinegar is easily removed.","prompt":"[header] How to get bad smells out of a microwave [title] Place 4 teaspoons (20 ml) of white vinegar in a microwave-safe bowl with water and microwave on high for 6 minutes. [step] Leave the solution in the microwave to steam for an hour and then wipe the insides. Clean the microwave with soap and water.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5526,"error":null,"output":"A"}
{"question_id":"hellaswag:00006982","item_index":268,"attempt":0,"prompt_hash":"f42eac873a43f2a35b215e6234efa6325ad81ab12d396bcad89d2944291a50af","question":"[header] How to make olive oil hair masks [title] Prepare the mask. [step] Add 1 tablespoon of colorless henna with the same amount of olive oil. Mix together thoroughly.\n\nA. This will help to distribute the solution evenly through your hair, as well as draw out the sebum more easily. [title] Use your hair dipped in the mixture.\nB. Pour the mixture onto your hair, one tablespoon at a time. Run a comb through your hair to remove any tangles.\nC. This will give your mask an oil-rich finish. [title] Pour the mask into a large bowl.\nD. Add egg yolk, a teaspoon of honey and a teaspoon of brandy. [title] Apply the mask to the roots of your hair.","prompt":"[header] How to make olive oil hair masks [title] Prepare the mask. [step] Add 1 tablespoon of colorless henna with the same amount of olive oil. Mix together thoroughly.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5530,"error":null,"output":"D"}
{"question_id":"hellaswag:00003896","item_index":269,"attempt":0,"prompt_hash":"b9105fe04361f09753494edc84d534f0064bd2d7194cd2bb9665b426676ad74f","question":"[header] How to light kwanzaa candles [title] Get out your kwanzaa items on the morning of december 26th. [step] Kwanzaa starts the evening of december 26th, so you'll need to get all your items together that morning and afternoon. If you celebrate christmas or any other holiday, put those decorative items away before you set up for kwanzaa.\n\nA. You'll need about three warm-toned coloured candles which you can buy from stores such as salvation army, dollar store, costume house, and walmart. [title] Take any writing items with you to the store.\nB. [step] If you're not sure what to put aside, look up websites like ebay and amazon to find something livable to put aside at the end of the night. Save your empty kwanzaa candle holders.\nC. [substeps] Self-determination and the specific culture of africa is an important part of kwanzaa. You can still celebrate other holidays, but be sure the only holiday decorations up on december 26th are for kwanzaa.\nD. Don't forget to put your candles back where you found them, if you replaced them. [title] Set up your table for kwanzaa candles.","prompt":"[header] How to light kwanzaa candles [title] Get out your kwanzaa items on the morning of december 26th. [step] Kwanzaa starts the evening of december 26th, so you'll need to get all your items together that morning and afternoon. If you celebrate christmas or any other holiday, put those decorative items away before you set up for kwanzaa.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5527,"error":null,"output":"C"}
{"question_id":"hellaswag:00001549","item_index":270,"attempt":0,"prompt_hash":"3cd90119b081c6ac09aa29c40945da0d3318defde4f15941b5db75b32442bc8e","question":"A boy in young boy drives a small tractor lawn mower around the yard of a house maneuvering around obstacles. the boy\n\nA. mows over some small flowers in the grass.\nB. walks in a circle the approached the camera, and then went back the same way he came from.\nC. stops then passes the tractor.\nD. pushes the edge of the mower up and down along the lawn path as leaves begin to fall on the lawn.","prompt":"A boy in young boy drives a small tractor lawn mower around the yard of a house maneuvering around obstacles. the boy","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5529,"error":null,"output":"A"}
{"question_id":"hellaswag:00002572","item_index":271,"attempt":0,"prompt_hash":"3b6117fb84002a327422e1a221720864d69ae5ff89ea27bc5c8f6ef99a480229","question":"The woman speaks and then rinses her face off and then dries her face with a towel. when the woman's face\n\nA. is clean she starts laughing and being silly.\nB. is dry she walks to open the door and when she's knocked out, a man comes in and holds an item up to the camera.\nC. is fully dried, the woman pours soap and then drips it on her face.\nD. is all dried she continues to talk.","prompt":"The woman speaks and then rinses her face off and then dries her face with a towel. when the woman's face","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5529,"error":null,"output":"D"}
{"question_id":"hellaswag:00004472","item_index":272,"attempt":0,"prompt_hash":"a308580380b69c318e0b43fa36fd4ddc52d83552f54dcc4dfed63700789fa588","question":"[header] How to straighten permed hair [title] Choose the right flat iron. [step] If you just want to temporarily remove your perm, a flat iron is the best route. It will not permanently remove the perm, but will allow your hair to temporarily straighten.\n\nA. Make sure you select the right flat iron for your hair type. [substeps] If you have short, thick, or fine hair look for an iron with narrow plates.\nB. Straightening permed hair can be left to rest for two or three days before completing the daily use. [substeps] Once your hair is relaxed, it should be straightened into a un-curly bun or poof to highlight your curls.\nC. Select a flat iron with a satin or matte finish. You can also purchase a fixed or semi-soft iron with a different flat size.\nD. Flat irons differ in the type of look you want to wear, though they can vary by body type, length, thickness, and order. [title] Grab a small container with a zipper.","prompt":"[header] How to straighten permed hair [title] Choose the right flat iron. [step] If you just want to temporarily remove your perm, a flat iron is the best route. It will not permanently remove the perm, but will allow your hair to temporarily straighten.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5527,"error":null,"output":"A"}
{"question_id":"hellaswag:00005505","item_index":273,"attempt":0,"prompt_hash":"3bbfe38592871df14467a04bf057718913d70b162bc3ad2c2bd1da7cbba99a61","question":"[header] How to take care of naturally curly hair [title] Wash your hair often, but not every day. [step] A mistake most people make is shampooing everyday. It may seem like a good idea, but it strips your scalp of the natural oils it needs to be healthy.\n\nA. [title] Moisturize your hair at least once a week. [step] Even cold showers, push yourself further into the care zone by applying a scalp moisturizer after every shower.\nB. Naturally curly hair requires all of the moisture it can get to stay healthy due to the shape/form, and excessive shampooing can leave your hair dull and dry. Remember that you can still shower without shampooing.\nC. Using a gentle shampoo and conditioner may help dilute your hair just enough so that it is less saturated with oils. [substeps] Go for a conditioner only if your hair really needs it.\nD. A more intense shampoo and conditioner will only gently exfoliate your scalp, scalp, and hair follicles, rather than stripping them of natural oils and damaging them. [substeps] Start using a good conditioner after a shower and then reduce the amount of conditioner that you use.","prompt":"[header] How to take care of naturally curly hair [title] Wash your hair often, but not every day. [step] A mistake most people make is shampooing everyday. It may seem like a good idea, but it strips your scalp of the natural oils it needs to be healthy.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5528,"error":null,"output":"B"}
{"question_id":"hellaswag:00009804","item_index":274,"attempt":0,"prompt_hash":"f5d9733a98dd872c52ce687769f2819dd1129e5964ca38a692e83e8dfffb988f","question":"[header] How to sleep in a hammock [title] Get a hammock made of nylon. [step] Look for hammocks made of parachute nylon, as they will be the most comfortable to lay in for a long period of time. Avoid hammocks made of rope or slats, as they can cause discomfort and rope burns.\n\nA. [title] Make sure the hammock is 4 feet (1.1 m) wide. [step] Hammocks made of other materials like nylon for durability are better for sleeping outdoors than those made of synthetic materials.\nB. [substeps] Nylon is heavier, meaning that it will wear out faster, or shrink faster, after being wet for long periods of time. [title] Buy a hammock made of a light material.\nC. [substeps] You can buy parachute nylon hammocks at your local outdoors store or online. [title] Choose a hammock that fits your height and weight.\nD. [substeps] You can use an existing hammock that you already own or purchase one for a lower price. Hammocks made of other materials that have fallen into use are preferred over smaller ones.","prompt":"[header] How to sleep in a hammock [title] Get a hammock made of nylon. [step] Look for hammocks made of parachute nylon, as they will be the most comfortable to lay in for a long period of time. Avoid hammocks made of rope or slats, as they can cause discomfort and rope burns.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5528,"error":null,"output":"C"}
{"question_id":"hellaswag:00001230","item_index":275,"attempt":0,"prompt_hash":"6d2c0295999786231e0a28b47128ed849f812b9d641abf38c9cd8c7b8e9ea5b1","question":"A man in a field strikes a match. He bends down and tries to start a fire, but it doesn't light. he\n\nA. chases after the other people he strikes at, but they all run away.\nB. walks away as another man comes to check the fire.\nC. tries again with another match.\nD. shades his eyes and walks, dragging his dog with him.","prompt":"A man in a field strikes a match. He bends down and tries to start a fire, but it doesn't light. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5527,"error":null,"output":"C"}
{"question_id":"hellaswag:00003617","item_index":276,"attempt":0,"prompt_hash":"958944ba1e0bbc6960a5f2dce4d77f792065840fb31165118055c976d83fcd65","question":"[header] How to keep hornets away [title] Cover all food sources, such as compost piles and trashcans. [step] Unlike bees, wasps and hornets like to eat meat and other protein foods, as well as sweet nectars. Hornets will remember the food source and continue to come around after it has been sealed off or taken away, so it's best to prevent all open-air garbage.\n\nA. [substeps] The first thing wasps most likely to notice and become interested in is beans and fruits, both natural and edible. Female hornets recognize insects and are attracted to them on a daily basis.\nB. [title] Reduce sweet smells during summer or fall. [step] This is the point when wasps prefer blossoms, juices and fruit.\nC. [substeps] Wasps and hornets also like to eat poultry, fruits, vegetables, and natural foods like carrots, lettuce, and oranges. Keep them away from anything that has bees on it.\nD. [substeps] Spread cover around plants and garden beds that are organic. Only put out as much as you can-these plants are bait for wasps.","prompt":"[header] How to keep hornets away [title] Cover all food sources, such as compost piles and trashcans. [step] Unlike bees, wasps and hornets like to eat meat and other protein foods, as well as sweet nectars. Hornets will remember the food source and continue to come around after it has been sealed off or taken away, so it's best to prevent all open-air garbage.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5528,"error":null,"output":"C"}
{"question_id":"hellaswag:00002169","item_index":277,"attempt":0,"prompt_hash":"a9657cd08ab7931c2521dddc9d23dfa6309786a79746503660a93d00c0e240f6","question":"The lady continues playing the harmonica as the elevator moves further and stops at other levels and more people walk in. people\n\nA. wrench the ramps and levels as the camera follows their movements that lead to the lady hitting harmonies and some hokey syllables.\nB. are moving in a line as the lady plays.\nC. start to take pictures of the harmonica as they walk through the lobby and outside to the bike repair shop.\nD. continue to get on and off the elevator but she continues to play her harmonica.","prompt":"The lady continues playing the harmonica as the elevator moves further and stops at other levels and more people walk in. people","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5526,"error":null,"output":"D"}
{"question_id":"hellaswag:00009576","item_index":278,"attempt":0,"prompt_hash":"c52d736e072dde0a4e2086e9f5a6c4f68359c307af7d89cf36cfd22312fa0ec7","question":"[header] How to cook tapioca [title] Put the water and boba in a saucepan and bring it to a boil over high heat. [step] Keep stirring! You don't want the boba to stick to the bottom of the pan. And be sure to keep a 8:1 ratio of water to boba; in other words, if you have 1/4 cup of boba, use 2 cups of water.\n\nA. You can limit the amount of boba you add to boiling water. [title] Remove the boba from the water pot and put it back in the saucepan.\nB. Only need 1/8 cup? Then 1 cup of water will suffice. [substeps] Some recipes call for pre-soaking your boba.\nC. If you want a different result, just add enough water to the boba. [title] Bring the water to a boil and turn down the heat once the water comes to a boil.\nD. Boba, unlike other fish, is heavy. So be careful, if it starts to stick to the bottom, lower the heat.","prompt":"[header] How to cook tapioca [title] Put the water and boba in a saucepan and bring it to a boil over high heat. [step] Keep stirring! You don't want the boba to stick to the bottom of the pan. And be sure to keep a 8:1 ratio of water to boba; in other words, if you have 1/4 cup of boba, use 2 cups of water.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5126,"error":null,"output":"C"}
{"question_id":"hellaswag:00000117","item_index":279,"attempt":0,"prompt_hash":"3b77b0f4c7d24c4e923165fdef61e5e0faa7dd6e2da0bf564b6c3a8cb966e5c4","question":"The surfer who is in front is happy to riding the wave for so long that he raises his two arms up in the air and then lowers it back down. the surfer in the back\n\nA. kept one arm raised up and the surfer in front of him raise his left arm.\nB. wave has a big wave almost bigger than he is paddling, and this makes it pretty fun because it causes the surfer riding the wave to put up a lot of resistance so he just does it and crash onto the waves making the wave even bigger then it was.\nC. does not have to go as far as he needed to to go as he is ready to go.\nD. begins to lose balance and falls off of his board while the man in the front is still standing up.","prompt":"The surfer who is in front is happy to riding the wave for so long that he raises his two arms up in the air and then lowers it back down. the surfer in the back","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5126,"error":null,"output":"D"}
{"question_id":"hellaswag:00001586","item_index":280,"attempt":0,"prompt_hash":"b8c1eb04d2a5d2cfe9aaa7fd40e4769621c182efed04c0c4231821e31cfc0f15","question":"A tight rope is strung across the length of the pool and a woman is attempting to walk across, she falls several times. the last time she falls\n\nA. she gets up and stands on edge of the pool, she tries to pole vault into the pool.\nB. , the camera rewinds and shows her falling backwards.\nC. down, she tries to look across but the rope is too tight.\nD. , the man shoots her several times before he stops her.","prompt":"A tight rope is strung across the length of the pool and a woman is attempting to walk across, she falls several times. the last time she falls","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5127,"error":null,"output":"B"}
{"question_id":"hellaswag:00000207","item_index":281,"attempt":0,"prompt_hash":"de6c319c4d80897e8d327c7cd20a21c66933af046cbd121fa3fc3bf722fab1e3","question":"One of the swimmers is underwater, trying to reach the ball. He throws the ball across to another swimmer who catches the ball. two swimmers\n\nA. advance on the beach, trying to catch the balls.\nB. are outside, working out as they of the group of swimmers.\nC. are under water trying to reach the ball.\nD. are together in a diving pool, putting on mask.","prompt":"One of the swimmers is underwater, trying to reach the ball. He throws the ball across to another swimmer who catches the ball. two swimmers","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5126,"error":null,"output":"C"}
{"question_id":"hellaswag:00000680","item_index":282,"attempt":0,"prompt_hash":"0f2ede3aeaaeecb55b3ab7344419339c3da9b4c20f79834e2d0c35911c8be5f7","question":"The man jumps again but this time at 5'10 high, he clears it, and once again it replays. the man\n\nA. is now running and jumping over the pole that is now 6 feet high, he clears it like he did all the other heights and once again it replays.\nB. performs one last trick on high, this time with an upper hand but he misses again and flies.\nC. jumps again at 610 and doesn't stick it this time at 611 or 890, he gets a second round and attempts to get off successfully.\nD. jumps continuously until it replays to show the height of the spike of the charge and the exact amount of jump result.","prompt":"The man jumps again but this time at 5'10 high, he clears it, and once again it replays. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5126,"error":null,"output":"A"}
{"question_id":"hellaswag:00002521","item_index":283,"attempt":0,"prompt_hash":"dfcba643af4953cb018eb67539c4599a04431bda5076636c4882575a4874ad38","question":"A lady puts her hair in a bun on the side. the lady\n\nA. shows us products to describe a blow dryer.\nB. uses white brush to make her hair stand up.\nC. puts pins in her hair on the side.\nD. puts her hair in a ponytail.","prompt":"A lady puts her hair in a bun on the side. the lady","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4882,"error":null,"output":"D"}
{"question_id":"hellaswag:00003967","item_index":284,"attempt":0,"prompt_hash":"c3b454cee25f01f484e747000c0d05e29968751b97759f9c4cad854e5cf8231f","question":"[header] How to stop nails from breaking [title] Keep your nails trimmed. [step] Keeping your nails trimmed down may be a good solution if your nails keep breaking. You don't have to stop wearing your nails long, but wearing your nails a little shorter might help.\n\nA. Try trimming your nails a centimeter or two shorter than you normally would. [substeps] Clip your nails when they are soft, such as after taking a shower or bath.\nB. [substeps] Short nails can wear out as they age, and you may have to switch to top ofths. The length of your nails change as you age.\nC. It might also make them become more and more broken, so you may want to wear clip-in nails as a reminder of when they begin to break, as well. If you want to start over in a new level, it's not a bad idea to cut your nails.\nD. [substeps] A cuticle pusher is a good way to keep your nails from getting too long. You can buy one with a handle on one end, or you can clip or file your nails yourself at home.","prompt":"[header] How to stop nails from breaking [title] Keep your nails trimmed. [step] Keeping your nails trimmed down may be a good solution if your nails keep breaking. You don't have to stop wearing your nails long, but wearing your nails a little shorter might help.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4882,"error":null,"output":"A"}
{"question_id":"hellaswag:00000367","item_index":285,"attempt":0,"prompt_hash":"a5246c4c1e41bee08002f2642378af526a19c1a77558c2b3684be69a244bbdb4","question":"A lawn mower is tied to a white rope. it\n\nA. is being finally measured for size.\nB. lifts the side tire of the mower, and then rolls it onto the grass.\nC. is pushing leaves with the handle.\nD. is turned on and starts going in circles around the rope mowing the lawn.","prompt":"A lawn mower is tied to a white rope. it","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4882,"error":null,"output":"D"}
{"question_id":"hellaswag:00003721","item_index":286,"attempt":0,"prompt_hash":"d778e8888a76d81123b7ff27f63c2341acac8279d2e487aaff3fbe9c669c5c1d","question":"[header] How to make acquisitions in a down economy [title] Investigate the current market value of the asset under consideration. [step] This may or may not be the same as the asking price for the asset. Before making a commitment, you want to know how much you could sell the asset for in today's less than desirable economy and compare that amount to the seller's price.\n\nA. Before committing to making a purchase, you should know how the sale will affect the current market price of the asset. Realize that any upfront costs are nothing but bonuses, no matter how much you purchase.\nB. If the 2 figures are somewhat close, further consideration of the acquisition may be worthwhile. [title] Project the future movement of that market value.\nC. Remember, a lower down market value means better performance. [substeps] Start by deciding if there are any restrictions before making a purchase of the asset.\nD. [substeps] What is left at the selling price should be kept for the future. When making a bid on the asset, think about how much the value of the asset would cost the seller.","prompt":"[header] How to make acquisitions in a down economy [title] Investigate the current market value of the asset under consideration. [step] This may or may not be the same as the asking price for the asset. Before making a commitment, you want to know how much you could sell the asset for in today's less than desirable economy and compare that amount to the seller's price.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4882,"error":null,"output":"B"}
{"question_id":"hellaswag:00004764","item_index":287,"attempt":0,"prompt_hash":"df16942f11deb4500e6f66e4879ddd6afe58a9541a5879be011ac27b96db2a1b","question":"[header] How to clean your shoelaces [title] Remove the laces from the shoes. [step] It is much easier and more effective to clean your shoe laces once they have been removed from the shoes. [title] Brush any loose dirt or grime from the laces with your hands or a scrub brush.\n\nA. [step] Dried and caked on dirt or mud should come off fairly easily with a little bit of scrubbing. [title] Make a solution of 3 tablespoons (44.4 ml) of bleach to 1 gallon (3.8 l) of water in a sink or basin.\nB. [step] A toothbrush is necessary as toothpaste will have a particularly strong odor and could cause smudges. [substeps] If you have any dirt in your shoes, it is a good idea to rinse the dirt from these items and throw them in the garbage.\nC. [title] Locate the laces at the top, bottom or side of the shoe. [step] It may take a little bit of effort to find the laces.\nD. [step] Use your fingers to clean the shoes in small sections about three quarters of the way up each side. [title] Fill a tub or basin with cool water.","prompt":"[header] How to clean your shoelaces [title] Remove the laces from the shoes. [step] It is much easier and more effective to clean your shoe laces once they have been removed from the shoes. [title] Brush any loose dirt or grime from the laces with your hands or a scrub brush.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4883,"error":null,"output":"A"}
{"question_id":"hellaswag:00000200","item_index":288,"attempt":0,"prompt_hash":"b78b0c246b203c63d5dbdd8494fb18cf5b1a928775f5db28b842063d1101a915","question":". a colorful intro with a lot of flashing words\n\nA. , leads into a dirt track.\nB. flash across the screen along with a logo and a banner with foreign language written on it.\nC. appears giving a description of the video.\nD. it says as follows.","prompt":". a colorful intro with a lot of flashing words","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4911,"error":null,"output":"B"}
{"question_id":"hellaswag:00001814","item_index":289,"attempt":0,"prompt_hash":"272e3bc9d93dbbf379ba75a586f56a919d696d01a3135f34d8a78611ed4acbe2","question":"A man tries to stop a man from shooting a basket ball. The ball is passed across the room and men chase it across the court. a boy\n\nA. catches and tosses the ball behind him which is caught by another boy on his team.\nB. stands in the middle of the court feeling bad.\nC. breaks a basket and and messes with the man's hair.\nD. tries to stop a man from playing a nine ball game.","prompt":"A man tries to stop a man from shooting a basket ball. The ball is passed across the room and men chase it across the court. a boy","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4908,"error":null,"output":"A"}
{"question_id":"hellaswag:00008618","item_index":290,"attempt":0,"prompt_hash":"c7934fb618788f223c7bde73a5c1423bee7835ed2254dda3a58f0bc621d34524","question":"[header] How to supplement vitamin d in babies [title] Determine how much vitamin d your baby needs. [step] The american academy of pediatrics recommends that breast-fed infants get 400 international units (iu) of vitamin d every day. If your baby takes a cow's milk-based or other formula, ask whether you should supplement.\n\nA. Most formulas are fortified with the recommended 400 iu of vitamin d (although these babies might need supplements later in life). [substeps] Breastfeeding mothers should get 600 iu of vitamin d a day.\nB. [substeps] In order to supplement your baby's breast milk, you'll need to feed it two to three times a day. This equals 600 iu of vitamin d every day.\nC. [substeps] Studies have shown that women who are over 40 are able to get at least 300 iu every day. If the child is about 36 or older, they are deficient or at higher risk for developing breast cancer.\nD. [substeps] Supplemental milk can provide the equivalent of 250 , 000 units to the baby at eight million milligrams per day. However, is not recommended if your baby needs 2 , 500 units of iu.","prompt":"[header] How to supplement vitamin d in babies [title] Determine how much vitamin d your baby needs. [step] The american academy of pediatrics recommends that breast-fed infants get 400 international units (iu) of vitamin d every day. If your baby takes a cow's milk-based or other formula, ask whether you should supplement.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4904,"error":null,"output":"A"}
{"question_id":"hellaswag:00001537","item_index":291,"attempt":0,"prompt_hash":"b8fceaf817a2775331e634d42b7a81a7c2632fe42debb37bf02f88eec6d3895f","question":"A chef in the kitchen wearing a chef hat is giving instructions on how to sharpen a knife. He brings the knife to a sharpener and begins slowly to demonstrate. he\n\nA. picks up the sharpening tool and talks about what he will do next and begins to sharpen again.\nB. explains the proper way to cut a clam shape.\nC. wets the knife in the sharpener and pours water into it.\nD. keeps the knife from sharpener down until it's an inch from sharpener.","prompt":"A chef in the kitchen wearing a chef hat is giving instructions on how to sharpen a knife. He brings the knife to a sharpener and begins slowly to demonstrate. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4907,"error":null,"output":"A"}
{"question_id":"hellaswag:00000632","item_index":292,"attempt":0,"prompt_hash":"6a985974c1948cf7a6aafbdec7c6c74176738d5cd630a6ab9c540c78d2d9d849","question":"The chemical is captured displayed on the sit of the wooden chair. The person sprays chemical on a cotton pads and whips the wooden chair. the chair\n\nA. is left to dry.\nB. reclines as the person touches the wood with the cotton pads.\nC. is moved on the side of the room assisting in the testing process.\nD. is then photographed in a decoder display.","prompt":"The chemical is captured displayed on the sit of the wooden chair. The person sprays chemical on a cotton pads and whips the wooden chair. the chair","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4909,"error":null,"output":"A"}
{"question_id":"hellaswag:00009896","item_index":293,"attempt":0,"prompt_hash":"648c360fff19e7733785d9b55a07e19bce5f950e3e65a3c9696b1e613fff8710","question":"[header] How to increase fertility with hypnosis [title] Daily practice of self-hypnosis is invaluable in increasing your chances of conception. [step] Spend five minutes, once or twice each day, in this relaxed state with this new belief and it will begin to plant the seeds necessary to create success. [title] Find a place where you will not be disturbed for a while.\n\nA. [step] This could be a bench, the floor of a street, the sofa or even your bed! Find somewhere completely private with no television and other distractions and lie down for a short period. Rediscover the aspects you already enjoy.\nB. [step] Do not sit in the privacy of your home, bathroom or living room for hours. Find somewhere quiet and solitary where no one will disturb you.\nC. [step] Settle yourself comfortably, either sitting or lying down. Close your eyes, get quiet and still, become aware of your breathing, relax....\nD. [step] Because hypnosis can be performed in a variety of places, including small isolated areas, you will want to keep your activity private. Keep your noise to a minimum to safeguard yourself against loud music and' strange' people entering your home.","prompt":"[header] How to increase fertility with hypnosis [title] Daily practice of self-hypnosis is invaluable in increasing your chances of conception. [step] Spend five minutes, once or twice each day, in this relaxed state with this new belief and it will begin to plant the seeds necessary to create success. [title] Find a place where you will not be disturbed for a while.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4904,"error":null,"output":"C"}
{"question_id":"hellaswag:00007487","item_index":294,"attempt":0,"prompt_hash":"122828e3585a1215f1abf053cbf30c1e16a148db7f996147a9e2c4db8bec21dd","question":"[header] How to stay safe when shoveling snow [title] Don't underestimate the task or overestimate your fitness. [step] Don't sell the exertion required to shovel snow short. If you would have trouble completing a good jog or weight-training session, you may have difficulty clearing your driveway and sidewalk.\n\nA. It's important to make it clear to your neighbors that it's your work for them in case you drop on them again. [substeps] Don't say you'll be doing your job right after ice water starts to appear.\nB. Upper body workouts like clearing snow actually put more strain on the heart than activities like running or biking. [substeps] If you have any reason to doubt the adequacy of your fitness level, consult a physician before undertaking snow shoveling.\nC. [substeps] If you've always told your friends that you don't need long walks in your neighborhood, this will save you money and time. If you really want to keep a snow shovel on you, build one until you have two vehicles available.\nD. [title] Avoid using everyday tools, like electric tools or rakes. [step] Be aware that this will prevent you from being able to work it under the supervision of professionals.","prompt":"[header] How to stay safe when shoveling snow [title] Don't underestimate the task or overestimate your fitness. [step] Don't sell the exertion required to shovel snow short. If you would have trouble completing a good jog or weight-training session, you may have difficulty clearing your driveway and sidewalk.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4902,"error":null,"output":"B"}
{"question_id":"hellaswag:00004423","item_index":295,"attempt":0,"prompt_hash":"79eaaccaf89731e9c2476f4749b5f5a5682f9cc670e3b0b8aa2e244a0708fd29","question":"[header] How to care for your micro bikini [title] Instructions. [step] Read and carefully follow any fabric care instructions that come with or are listed on your bikini wear. You don't want to ruin it and use something that's going to make the color fade or stain even more.\n\nA. In order to be even more sure, determine how to care for it. The first thing to do is to take the time to familiarize yourself with the basic components of the bikini and decide if it's the right size.\nB. [title] Carefully measure the waist and locate the cutout of the fabric on your bikini. [step] The waist is the point at which the cut is visible through the fabric.\nC. The best tips for ensuring a color blooming, photogenic look to your bikini are that you can always paint and/or steam tan later. [title] Plan your trim after you have washed.\nD. Determine if you are going to hand wash or machine wash your bikini wear. Take in consideration the fabric, texture and ornaments, if any, that may be affected.","prompt":"[header] How to care for your micro bikini [title] Instructions. [step] Read and carefully follow any fabric care instructions that come with or are listed on your bikini wear. You don't want to ruin it and use something that's going to make the color fade or stain even more.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4903,"error":null,"output":"D"}
{"question_id":"hellaswag:00005930","item_index":296,"attempt":0,"prompt_hash":"17be17b99c6a4ba060a88d8b42fdcdf3f903a0a0ebd69514ed6767dae374a620","question":"[header] How to apply different styles of eyeshadow [title] Apply a sheer colour all over your lid, up to your eyebrow. [title] Softly line your eye with a pencil liner. [step] It should be a dark or medium shade, like black or grey.\n\nA. Smudge the liner upwards to create a small line, rather than in a dramatic swoop. [substeps] If you use the pencil, smudge it slightly with your finger before closing your eyes.\nB. [title] Apply a shade of eyeliner with the darkest eyeshadow you can think of. [step] Use tiny dots to close your eyelid, so the shade doesn't come out behind your eye.\nC. [title] Focus on the edges of your eyes, blending with the darkest edge. [step] This will make your eyeshadow blend more quickly and take away from the normal depth of intensity.\nD. The line should go from the inner corner to the outer corner. Use the same pencil to dot below your your lower lashes.","prompt":"[header] How to apply different styles of eyeshadow [title] Apply a sheer colour all over your lid, up to your eyebrow. [title] Softly line your eye with a pencil liner. [step] It should be a dark or medium shade, like black or grey.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4902,"error":null,"output":"D"}
{"question_id":"hellaswag:00009778","item_index":297,"attempt":0,"prompt_hash":"4ffb2ae357268b3628a82520cf5c324d56a0823aea764f0316544bd49ad0589c","question":"[header] How to get kool aid out of hair [title] Boil a pot of water and pour it into a container or small tub. [step] Fill the pot with enough water to dunk your hair into. Heat it on the stove until it's boiling, then remove it from the heat.\n\nA. Once the water boils, lower your hair into the water and let it sit for 20 minutes to make the water soften. [substeps] Look up the label on your kool-aid bottles to figure out how long it'll take to soften your hair.\nB. Store in an airtight container in the refrigerator. [substeps] Whole milk to add to dried foods is a popular way to make kool-aid.\nC. [title] Pour the kool-aid into a glass bottle. [step] Try a liter of kool-aid or pre-packaged kool-aid.\nD. Pour the hot water into a glass bowl, plastic tub, or container large enough to hold it. [substeps] Dyed tips will probably only need around 4 cups of water, while larger sections of hair may require more.","prompt":"[header] How to get kool aid out of hair [title] Boil a pot of water and pour it into a container or small tub. [step] Fill the pot with enough water to dunk your hair into. Heat it on the stove until it's boiling, then remove it from the heat.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4903,"error":null,"output":"A"}
{"question_id":"hellaswag:00002019","item_index":298,"attempt":0,"prompt_hash":"540ba2a3ad82c5318afb24ce216ef3e1f0f1ec30c2e8842d01289d8f54af5ccd","question":"A person is pressing buttons on an oven. a girl\n\nA. pops air into an inflated pan.\nB. removes the oven door to grab a paper plate and a scoop of vegetables.\nC. pours ingredients into a bowl together.\nD. lifts the oven lid to see how much heat it gets.","prompt":"A person is pressing buttons on an oven. a girl","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4905,"error":null,"output":"C"}
{"question_id":"hellaswag:00008275","item_index":299,"attempt":0,"prompt_hash":"ea6a033f60652d215360e6ffd8d65882c9dd91c56ffecbc946d69003c8602ed2","question":"[header] How to design a fashion show room [title] Create a vision and capture the overview on paper or on tape. [title] Rent or find a place to actually have the fashion show. [step] It could be as simple as your living room without furniture, or as complex as a rented office building floor.\n\nA. Use your creativity to plan for the show. [title] Find a location free from distractions.\nB. You also want to take into consideration how many people are coming to the fashion show. If only twenty people are coming but, there's a huge empty space with no one in it, it is going to seem empty.\nC. [title] Add wallpaper, backgrounds and props to make the chamber feel like a fashion show. [step] Have quality logos of designers, if possible.\nD. [title] Sketch a name of the event and ask anyone who will be able to attend it to be present. [step] It might be a business event, two people posing as models for a wedding, etc.","prompt":"[header] How to design a fashion show room [title] Create a vision and capture the overview on paper or on tape. [title] Rent or find a place to actually have the fashion show. [step] It could be as simple as your living room without furniture, or as complex as a rented office building floor.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4903,"error":null,"output":"B"}
{"question_id":"hellaswag:00007490","item_index":300,"attempt":0,"prompt_hash":"b3c60681d0bba550c9cf6ee4b5dfebf967a3457ccf4909800047b7fc796e4323","question":"[header] How to be a grandpa [title] Give your grandchildren lots of love. [step] As the grandfather, one of the most important things you can do is simply to shower your grandchildren with love. Give them hugs and kisses and let them know how much they mean to you.\n\nA. Tell them how beautiful, smart, or funny they are and make them see that not a day goes by that you don't think about them. Being as loving and caring as possible is the most important thing you can do.\nB. [substeps] Take them out for a walk where they can be around you. Give them books and toys that are fun to read.\nC. You could say, \" wow, how did you feel when your grandma planted these flowers in your yard last week? It sounds amazing. How long have you been a grandfather? \" also, consider taking time to spend with your grandchildren-every day.\nD. For instance, you could even spend the day naming your grandchildren onto each other! [title] Plan visits to your grandparents for your grandkids. [step] Grandparents live far enough away to spend time regularly together, and tend to connect quite easily by sharing common interests.","prompt":"[header] How to be a grandpa [title] Give your grandchildren lots of love. [step] As the grandfather, one of the most important things you can do is simply to shower your grandchildren with love. Give them hugs and kisses and let them know how much they mean to you.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4902,"error":null,"output":"A"}
{"question_id":"hellaswag:00005744","item_index":301,"attempt":0,"prompt_hash":"2df3561ebf931b00de3ca6211e3f8a5ebea3c0e41c207c975c4d433e950552da","question":"[header] How to take care of an older horse [title] Remember that insurance companies classify old horses as over 15. [step] This means they have higher premiums, as they are more at risk of diseases. [title] Even though a horse may be old, horses age at different rates.\n\nA. [step] Older horses tend to age faster if they have a higher income. Visit the horse shop and ask to be recommended by a veterinarian.\nB. [step] Horses which are in their 20s may be able to cope with work, because every horse if different. [title] Remember that smaller, native breeds often live longer, such as welsh ponies.\nC. [step] This means when equestrian or big time horses are 35 and older, that are who the horse looks like first. For example, if you have a arabian he will look like the golden retriever.\nD. [step] Just because you're buying a horse from a reputable breeder doesn't mean that they don't have higher premiums. [title] Consider the time and environment of the horse.","prompt":"[header] How to take care of an older horse [title] Remember that insurance companies classify old horses as over 15. [step] This means they have higher premiums, as they are more at risk of diseases. [title] Even though a horse may be old, horses age at different rates.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4903,"error":null,"output":"D"}
{"question_id":"hellaswag:00000617","item_index":302,"attempt":0,"prompt_hash":"90f6e007b373ab0adcd91adc4d6bbf9718ff4154c842d6ef465b66525919b21c","question":"A little boy pours mouthwash into his mouth from a cup. he\n\nA. spits it into the sink, then starts crying from the bad taste.\nB. then spits it out on to the floor in front of him.\nC. spits out alcohol and spits it back in.\nD. takes a drink of the cup.","prompt":"A little boy pours mouthwash into his mouth from a cup. he","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4905,"error":null,"output":"B"}
{"question_id":"hellaswag:00009974","item_index":303,"attempt":0,"prompt_hash":"1bcd55ac4340ddb3807dde0f3b446978812c70a1eec71c3489437ef2fa77882c","question":"[header] How to put a streak of color in your hair [title] Choose a type of colored hair extension. [step] There are two main types of hair extensions: synthetic and real hair. Synthetic extensions are cheaper, and readily found in beauty supply stores and \" punk \" stores, such as hot topic or clairs.\n\nA. [substeps] Synthetic extensions are considered very slick and straight with only a slight tug on the layers. Products containing synthetic hair (such as beauty supply, gel) work as well.\nB. Dyed hair extensions: synthetic extensions use diluted oils in the hair to create the appearance of waves. They are available in many sizes and lengths, including perming, pompadour, and pilling.\nC. Real hair extensions are more expensive, and can be found in salons and beauty-supply stores. Unlike synthetic extensions, they can be straightened and curled.\nD. Real hair extensions come in this color, as well as awesome lengths and changes from light to dark. [substeps] Synthetic extensions come in a wide range of widths.","prompt":"[header] How to put a streak of color in your hair [title] Choose a type of colored hair extension. [step] There are two main types of hair extensions: synthetic and real hair. Synthetic extensions are cheaper, and readily found in beauty supply stores and \" punk \" stores, such as hot topic or clairs.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4903,"error":null,"output":"C"}
{"question_id":"hellaswag:00000656","item_index":304,"attempt":0,"prompt_hash":"06841312b977523e950162129c8518105bff9c8364f6c108926b220997aadfd8","question":"After the man lands the jump he explodes in celebration by clenching his fists, after which a slow replay of the jump plays and a scoreboard displays his score. a series\n\nA. of scenes spread over the video because once again the man is shown jumping high and clearing the jump line with the result that he scored twice and successfully for a goal too.\nB. of still shots of the man smiling with another man looking on, the man receiving a medal for the long jump, another replay of the jump and a final scoreboards are displayed.\nC. of upcoming events occur as a player on red pants then flips over and then gets into a flip too.\nD. of results are then shown and they have a man laughing and speaking about his success with the jump for the physical and emotional aspects.","prompt":"After the man lands the jump he explodes in celebration by clenching his fists, after which a slow replay of the jump plays and a scoreboard displays his score. a series","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4905,"error":null,"output":"B"}
{"question_id":"hellaswag:00000350","item_index":305,"attempt":0,"prompt_hash":"313eae8e38cde851f2f79fd4732bad09a56f4a1f0dfb897b4cb42aea5e0c3e69","question":"Men on the beach playing volleyball in the sand. one of the guys\n\nA. does a layup on his stomach.\nB. dives into the water.\nC. starts to get off and the people on the beach talk to each other.\nD. hits the ball over to the other side and they hit it back.","prompt":"Men on the beach playing volleyball in the sand. one of the guys","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4903,"error":null,"output":"D"}
{"question_id":"hellaswag:00005300","item_index":306,"attempt":0,"prompt_hash":"185ee03f7fbe94cf2b76383d9f1dd7787af4ea209e02afbeabecdedfa78b0d91","question":"[header] How to have an open relationship [title] Ask yourself deeply why you want an open relationship. [step] Are you no longer satisfied with your partner? Do you simply want to try it? Do you think it will enhance your relationship with your current partner? These are all important considerations to make before trying to discuss the topic with your partner. [substeps] Be sure your motives are very strong before trying to discuss the topic with your partner.\n\nA. For some people, the topic is incredibly taboo and negative. Unless your partner has mentioned it in the past, do not assume that your partner will accept it.\nB. Telling your partner you would like to open the relationship is not the right way to start your relationship. Seek to open the relationship for the right reasons.\nC. Your partner's reactions will appear to drive you away. [title] Examine whether you need to be honest with your partner.\nD. They are key to a successful, honest, and supportive relationship. Ideally, you want a partner who is good at showing initiative.","prompt":"[header] How to have an open relationship [title] Ask yourself deeply why you want an open relationship. [step] Are you no longer satisfied with your partner? Do you simply want to try it? Do you think it will enhance your relationship with your current partner? These are all important considerations to make before trying to discuss the topic with your partner. [substeps] Be sure your motives are very strong before trying to discuss the topic with your partner.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4902,"error":null,"output":"B"}
{"question_id":"hellaswag:00007876","item_index":307,"attempt":0,"prompt_hash":"074f41b9e912486b42a7ac69a2658c02f7f2d805dc802b21e4f06efb6ea74c6d","question":"[header] How to set a bookmark shortcut in your home screen on android [title] Navigate to your android web browser. [step] Locate the icon that looks like a globe and tap on it to open. [title] Go to your website of choice.\n\nA. [step] Your bing-like website will open with a list of all your pages saved on your android device. [title] Click the link for your html or jpg (version 6) view drop-down menu.\nB. [step] The first section of your website will tell you about all of the site components and the business section. If you are on a pc, you will have to click here first.\nC. [step] Enter the name of the website in the text bar and press \" enter \" or \" go. \" [title] Tap on the create bookmark icon.\nD. [step] Once you open the app, you'll be directed to your home screen. [substeps] You can also find links on facebook and twitter on your home screen, all of which should contain a link to your bookmark.","prompt":"[header] How to set a bookmark shortcut in your home screen on android [title] Navigate to your android web browser. [step] Locate the icon that looks like a globe and tap on it to open. [title] Go to your website of choice.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4903,"error":null,"output":"C"}
{"question_id":"hellaswag:00003066","item_index":308,"attempt":0,"prompt_hash":"7e5811c3466f030351dba3e4741288f2e7a71849ddcadbc3348f6f77c003c200","question":"A person plays the harmonica holding it with both hands inside a room. then\n\nA. , the accordion player play the tune while the woman plays from the open neck.\nB. , the woman blow the horn in the lost whistle.\nC. , the person stops playing and talks.\nD. , a person blow the fire in the harmonica using the hands.","prompt":"A person plays the harmonica holding it with both hands inside a room. then","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4903,"error":null,"output":"C"}
{"question_id":"hellaswag:00000499","item_index":309,"attempt":0,"prompt_hash":"fe57a28a0dfeae84a4fa39ad3aff0879fcbb17fb670fb5fe4118c9f039cc4a29","question":"A man is working outside in a field. He is using a large tool called a thresher. the people\n\nA. walk around the grass, cutting it with the tool.\nB. are lifting him up and down.\nC. around him continue to pump water from their hose.\nD. in the crowd are watching back on now.","prompt":"A man is working outside in a field. He is using a large tool called a thresher. the people","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4903,"error":null,"output":"A"}
{"question_id":"hellaswag:00004431","item_index":310,"attempt":0,"prompt_hash":"5895263148401683abd6c2fd8144f4842aea8e791bf8d197121a3c2ced5fa6a2","question":"[header] How to figure out her ring size [title] Ask her friends. [step] Her friends may have no idea what size she wears, but it is certainly worth a shot. You'll probably have to tell them why you're asking, so be prepared with a story if you don't feel comfortable telling them the truth.\n\nA. [substeps] You may be able to study the ring size she wears several times before having her finally wear it. If you do not feel comfortable telling her outright, or you just don't think it looks right on her, do not ask her to wear the ring one time.\nB. [substeps] You could also ask one of her friends to take her ring shopping. It is less likely to arouse suspicion if a close girlfriend casually suggests wanting to go ring shopping.\nC. Some good stories to tell are : [substeps] Reference their mother for you. Keep it short: \" but how does our mother look wearing one? \" be as specific and focused as possible.\nD. [substeps] You can do this in person with some friends, but at the very least, they'll probably have had plenty of opportunity to talk to her and to figure out if the ring size is appropriate. If you absolutely cannot get anyone to tell her, hire a friend.","prompt":"[header] How to figure out her ring size [title] Ask her friends. [step] Her friends may have no idea what size she wears, but it is certainly worth a shot. You'll probably have to tell them why you're asking, so be prepared with a story if you don't feel comfortable telling them the truth.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5041,"error":null,"output":"B"}
{"question_id":"hellaswag:00007291","item_index":311,"attempt":0,"prompt_hash":"0326f93bc59cab05e8a216bf9cf7853c977d20f44d7cc6199ff70e742de4e2a7","question":"[header] How to sex tortoises [title] Pick up the tortoise. [step] Hold the tortoise in both hands, being careful not to drop it. Hold the tortoise up high enough to be able to see underneath it.\n\nA. Hold the tortoise up close to your body so that it is not forced against you. Close your arms around its neck and press it into your chest.\nB. Lower yourself until its head is touch your lower ribs. [substeps] Be careful! Tortoises hide their genitals.\nC. Do not flip the tortoise on its back. This is very stressful for the animal.\nD. Explain to the tortoise what you are doing, and gently start stroking the tortoise's belly. [substeps] Speak softly in such a low voice that the tortoise won't notice you are petting it.","prompt":"[header] How to sex tortoises [title] Pick up the tortoise. [step] Hold the tortoise in both hands, being careful not to drop it. Hold the tortoise up high enough to be able to see underneath it.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5041,"error":null,"output":"C"}
{"question_id":"hellaswag:00004836","item_index":312,"attempt":0,"prompt_hash":"ac58e1b68a415c46f6b15c3cb8ebce4a805654ab69885bf2cceea669dd8cb56c","question":"[header] How to make a cheerleading ponytail [title] Make sure your hair is clean, but not wet. [step] Creating hairstyles on wet hair can cause breakage, especially on thick hair. It's a good idea to include conditioner as well as shampoo when washing your hair.\n\nA. This is important for layering, so your hair will stay tight and would look silky under the hair stream. [substeps] You could use a clarifying shampoo to lift any tangles or frizz.\nB. [title] Place a shower cap over your hair after washing it. [step] Being in the shower may strip your hair of some moisture, and not only will this make your hair frizzier, but it can give you a chance to straighten.\nC. When your hair is wet, it's easier to keep a sleek, bouncy ponytail while you work to prevent breakage. [substeps] If your hair is damp from the shower, your hair will be wet, which increases frizz.\nD. [substeps] It is a good idea to create your ponytail on hair that is a day or two old of a wash, if manageable. [title] Put a hair elastic or two on your wrist.","prompt":"[header] How to make a cheerleading ponytail [title] Make sure your hair is clean, but not wet. [step] Creating hairstyles on wet hair can cause breakage, especially on thick hair. It's a good idea to include conditioner as well as shampoo when washing your hair.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5041,"error":null,"output":"B"}
{"question_id":"hellaswag:00001717","item_index":313,"attempt":0,"prompt_hash":"b9791270b5a916562609de81f50d4f1efa974f684e723d9a1334abc752aa4df5","question":"A man is riding a surf board in the ocean water. he\n\nA. does some stunts to stay afloat.\nB. almost capsizes on the waves knocking him over.\nC. is being pulled by a line down by a large wave.\nD. stays balanced as he goes sideways over a wave.","prompt":"A man is riding a surf board in the ocean water. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5040,"error":null,"output":"D"}
{"question_id":"hellaswag:00002620","item_index":314,"attempt":0,"prompt_hash":"002edaded9e891d2bf563384d15c3d286bd06b9ff921652651140b06b3e831e3","question":"A close up of shoes are shown that lead into a person putting socks on and waxing off a surf board. the man\n\nA. uses a clipper to undo the end and puts the boots back on his feet.\nB. is then shown surfing along the water with others as well as landing in the water.\nC. is then seen speaking to the camera on a beach while watching various times on the sand.\nD. continues waxing the shoes while pausing to give a tutorial on how they should be done.","prompt":"A close up of shoes are shown that lead into a person putting socks on and waxing off a surf board. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5040,"error":null,"output":"B"}
{"question_id":"hellaswag:00005037","item_index":315,"attempt":0,"prompt_hash":"5f879c2151fe927f21314e5f94340341c6a92cd97e3f99c98b5cde0cfab35178","question":"[header] How to defend against intrustion of solitude claims [title] Identify the elements the plaintiff must prove. [step] Intrusion of solitude is a state tort. Although each state's law will vary somewhat, the plaintiff must typically prove the following : [substeps] Without authorization, you intentionally invaded the plaintiff's private affairs.\n\nA. Because the plaintiff waited for permission to do so in advance, you invaded the plaintiff's private or protected privacy. Because you used obscene or hazardous materials in violation of a state law.\nB. ( note that this intrusion does not need to be physical; spying and eavesdropping also qualify. ) the invasion would be offensive to a reasonable person.\nC. For instance, some laws restrict how the plaintiff can enter the state property protected under the law. One disadvantage of intrusion of solitude is that the plaintiff cannot tour the state property, which cannot be used to defend you in court.\nD. Trespassers violate the law because they are not citizens of the state. At a minimum, trespassers cannot sue for invasion of privacy.","prompt":"[header] How to defend against intrustion of solitude claims [title] Identify the elements the plaintiff must prove. [step] Intrusion of solitude is a state tort. Although each state's law will vary somewhat, the plaintiff must typically prove the following : [substeps] Without authorization, you intentionally invaded the plaintiff's private affairs.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5262,"error":null,"output":"B"}
{"question_id":"hellaswag:00003209","item_index":316,"attempt":0,"prompt_hash":"a2c66b740a4ca9be25e06c44352e18beda505ab474e67358028791db08fab4b2","question":"The light shines bright on the right and scares the baby who begins to cry hard. The light fades and the baby calms down and looks all around the car. the baby\n\nA. throws his right arm in the air.\nB. laughs and it calms down and starts moving and feetfirst talking.\nC. stands on the car nose to nose with the door handle and screams because her nose is broken.\nD. stops crying then gets down and starts to play with the car.","prompt":"The light shines bright on the right and scares the baby who begins to cry hard. The light fades and the baby calms down and looks all around the car. the baby","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5265,"error":null,"output":"D"}
{"question_id":"hellaswag:00007369","item_index":317,"attempt":0,"prompt_hash":"d3a838555d04ea5cd88b6e972f3a651a507ffb51cf28f3cee9e7a1a3098c013b","question":"[header] How to deal with a controlling spouse [title] Stay calm. [step] For many people, arguing is a natural response to a spouse's controlling behavior. Unfortunately, a controlling person is unlikely to submit and let you win the argument, so this tactic will likely only escalate the situation.\n\nA. Instead, stay calm and find ways to resolve your conflict. [substeps] Take a step back and collect your thoughts.\nB. You and your spouse may disagree on many issues, especially important to a child or toddler : [substeps] One parent cannot take over the entire household all by themselves. The child or toddler has to ask for their parent's help and parents let the other parent handle the children, rather than take charge.\nC. Remind yourself that you can't argue with this person. If you communicate assertively, your spouse will understand.\nD. Instead of arguing, stay as calm and collected as possible. You can disagree with your spouse without yelling or being disrespectful.","prompt":"[header] How to deal with a controlling spouse [title] Stay calm. [step] For many people, arguing is a natural response to a spouse's controlling behavior. Unfortunately, a controlling person is unlikely to submit and let you win the argument, so this tactic will likely only escalate the situation.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5263,"error":null,"output":"D"}
{"question_id":"hellaswag:00007606","item_index":318,"attempt":0,"prompt_hash":"3822fe9d6acdc7a56718c042d155a2461b5a05d89b173e7d300cfe47e83e52cd","question":"[header] How to curl hair by twisting it [title] Get your hair damp. [step] It's important that you do this on damp hair, rather than wet hair. You can do this technique after a shower, once your hair has air dried about halfway.\n\nA. Some popular forms of drying hair include : [substeps] Showering. When your hair is wet, it cools.\nB. If you don't want or need to take a shower, you can simply dampen your hair with a spray bottle. The curls will hold best if done on damp hair.\nC. [substeps] Hold your head underneath the roots, then mist your hair with water to get the dampness out. Follow up by squeezing out all the excess water.\nD. [substeps] If your hair is wet, it may curl unevenly, making it difficult to work with. If you've really thick, thick hair, you might also want to consider using a double-boiler, so that your hair is still very damp and/or mixed in, rather than dripping wet.","prompt":"[header] How to curl hair by twisting it [title] Get your hair damp. [step] It's important that you do this on damp hair, rather than wet hair. You can do this technique after a shower, once your hair has air dried about halfway.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5265,"error":null,"output":"B"}
{"question_id":"hellaswag:00006751","item_index":319,"attempt":0,"prompt_hash":"2c6c310c0b17b5f1c6dee4cf994e9d3773bd8ddb5a6124d5f1ac463a18ad0c3d","question":"[header] How to do the crocodile pose in yoga [title] Assess your health and condition. [step] Even though the crocodile pose is, in most variations, a basic maneuver, it does involve moving and stretching that may exacerbate certain medical conditions. [substeps] While this pose can be quite helpful for back and neck stiffness or soreness, you should consult with a medical professional if you have a back or neck injury.\n\nA. [title] Fully stretch your hamstrings. [step] You can stretch just about any part of your body to some degree with this pose.\nB. The crocodile pose is not recommended if you are pregnant, as it places much of your body weight upon your abdomen while it is pressed to the ground. [title] Find a comfortable spot on the floor.\nC. If you have a heavy load and have recently undergone surgery, you should also discuss your surgery options with your physical care professional. [title] Stand at a comfortable distance from the mat.\nD. You can also incorporate the crocodile pose into more physical fitness exercises if you have a specific medical condition or problems that need improving, such as ptsd and joint pain. [title] Warm up your muscles by stretching them.","prompt":"[header] How to do the crocodile pose in yoga [title] Assess your health and condition. [step] Even though the crocodile pose is, in most variations, a basic maneuver, it does involve moving and stretching that may exacerbate certain medical conditions. [substeps] While this pose can be quite helpful for back and neck stiffness or soreness, you should consult with a medical professional if you have a back or neck injury.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5265,"error":null,"output":"B"}
{"question_id":"hellaswag:00000807","item_index":320,"attempt":0,"prompt_hash":"441ed864c164087a39f6ccc104630e54b82724c7041c66455b37ceae845b67fd","question":"The bald headed man places a menu on the side of the bus. the lady from the ice cream parlour bus\n\nA. guides the bald headed man through a maze of slot machines and refrigerators.\nB. window leaves.\nC. hands an ice cream cone to a client.\nD. has her four funnel cakes and is warming up for her cone.","prompt":"The bald headed man places a menu on the side of the bus. the lady from the ice cream parlour bus","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4395,"error":null,"output":"A"}
{"question_id":"hellaswag:00005552","item_index":321,"attempt":0,"prompt_hash":"f705aa21b4687f070568d322e61a730f7a4bb3d5b94a896d78f1cafd58a2ab20","question":"[header] How to dress up jeans [title] Make sure your jeans fit properly. [step] Choose a pair of jeans that flatters your figure and. Avoid jeans that are too snug in the hips and thighs.\n\nA. For a dressy look, you should also avoid jeans that are loose and baggy. [title] Choose jeans which fit well in the waist.\nB. You want to look like you're slim and graceful when you wear your jeans. Avoid baggy jeans that make you look like you're dead weight.\nC. Look for jeans that are flattering in the seams, areas of the leg with sleek lines, and tops with little to no detailing. [substeps] Avoid jeans made of any fabrics like silk, leather, or too thick and heavy fabrics, like cotton.\nD. You don't want to dress up your jeans the wrong way. Stick to jeans made of shape-neutral or loose-fitting fabrics so that you don't show more skin than you already do.","prompt":"[header] How to dress up jeans [title] Make sure your jeans fit properly. [step] Choose a pair of jeans that flatters your figure and. Avoid jeans that are too snug in the hips and thighs.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4396,"error":null,"output":"B"}
{"question_id":"hellaswag:00001209","item_index":322,"attempt":0,"prompt_hash":"7e2afbfaba6625239ae0063880e17da64be6a14600bc0adae85f971298f024da","question":"A boy is on a concert stage sitting behind drums. the boy\n\nA. begins playing the drums with his hands with sticks behind drums.\nB. begins by moving his hands and then starts to play the drums and cymbals with his hands.\nC. begins playing the drums.\nD. begins to play the drums with drumsticks while drumming his hands on the drums.","prompt":"A boy is on a concert stage sitting behind drums. the boy","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4394,"error":null,"output":"C"}
{"question_id":"hellaswag:00007303","item_index":323,"attempt":0,"prompt_hash":"4703b2852743612178068fd1be3e8400c2038611a6fd88e06c6a3c872efd976b","question":"[header] How to make a carton wallet [title] Print out the required template from here. [step] If you don't have it, download and install the latest adobe reader from. [title] Prepare the milk or juice carton.\n\nA. [step] Sketch a few valleys where the packing instructions say to place the carton. [title] Trace the middle of the carton using a pencil.\nB. [step] Use a marker to mark a placement on the carton. In the uk, use \" stay \", which may not work in all regions.\nC. [step] Print over a piece of paper flat. Peel off the wrapper and the lining material.\nD. [step] Rinse the carton, and cut it open so it lies completely flat. Wipe it down with paper towels, or, if you have time, place it in a dish rack to let dry.","prompt":"[header] How to make a carton wallet [title] Print out the required template from here. [step] If you don't have it, download and install the latest adobe reader from. [title] Prepare the milk or juice carton.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4395,"error":null,"output":"D"}
{"question_id":"hellaswag:00008692","item_index":324,"attempt":0,"prompt_hash":"73b97b517de5ad02730772201edf61036889def7fd6abb9a7123d57bef8b0764","question":"[header] How to have a ninja personality [title] Be a problem solver. [step] The ninja had to be alert at all times and be prepared to solve complicated problems on the spot. They did not believe anything was impossible for them to figure out and they looked at obstacles as opportunities to learn.\n\nA. [substeps] Be a problem solver by sitting at a desk and writing up problems on a notepad. Write your problems down on a sheet of paper and hand write them down one day.\nB. [substeps] Try to approach your daily problems with that same mindset. Along with solving problems, also practice being able to identify exactly when problems occur and what caused them.\nC. You have to be a problem solver. [substeps] You have to be able to solve complex problems and always have a solution in mind.\nD. If you want to be a ninja, you need to be good at it, and you can't just be a problem solver and try something completely out of the ordinary. Try to practice this skill daily, even in the middle of the day before you're going to bed.","prompt":"[header] How to have a ninja personality [title] Be a problem solver. [step] The ninja had to be alert at all times and be prepared to solve complicated problems on the spot. They did not believe anything was impossible for them to figure out and they looked at obstacles as opportunities to learn.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4386,"error":null,"output":"C"}
{"question_id":"hellaswag:00004138","item_index":325,"attempt":0,"prompt_hash":"425ba8cd5632bba362b2774f8e2f0c57ba21aadc9fa23c0d5d3e0526c93c7c7d","question":"[header] How to maximize the benefits of medical marijuana clones [title] Choose a \" mother plant. [step] \" the first step in cloning your plants is to find a good \" mother plant \" for cutting or cloning. \" cutting \" is when you take a portion of the original plant and encourage it to grow new roots and stalks, thus making a new plant or \" clone.\n\nA. \" this is a much more effective way of growing what you want. It will also reduce any waste that is put into your plant.\nB. \" [substeps] \" mother \" plants can grow faster so as to eliminate the need for cutting and cloning when you are trying to prevent disease while growing plant or root systems. [title] Don't make the initial first plant grow any shorter.\nC. \" since the new plants are identical to the original, you need to choose well. [substeps] Marijuana plants are sexed, and male plants do not produce buds.\nD. \" unfortunately, cutting or cloning a plant is tricky. Talk to the department of health and human services for more information.","prompt":"[header] How to maximize the benefits of medical marijuana clones [title] Choose a \" mother plant. [step] \" the first step in cloning your plants is to find a good \" mother plant \" for cutting or cloning. \" cutting \" is when you take a portion of the original plant and encourage it to grow new roots and stalks, thus making a new plant or \" clone.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4386,"error":null,"output":"C"}
{"question_id":"hellaswag:00006493","item_index":326,"attempt":0,"prompt_hash":"8ebf0f7b5b747bb0a9c22eab617f053bf8b36f67cdc8773bfebafdbc41f81ed1","question":"[header] How to cope with arrogant people [title] Enter an encounter with an arrogant person with the strong and confident realization that you are well and strong. [step] When you feel sound within your own self, there is nothing an arrogant person can say or do to undermine you. Your sense of confidence and self-worth will prohibit you from being vulnerable to an arrogant person.\n\nA. [title] Try a variety of approaches to identify the person arrogant and go from there. [step] Once you find someone who you can confidently and correctly identify with, go back to the following strategies.\nB. The arrogant person may be unable to relate to you and might even say noxious or cruel things but you can let it slide when you feel secure in yourself. [title] Use the encounter as an opportunity to improve your own listening skills or tolerance.\nC. Pick the right time and place to approach the person and speak directly. [substeps] You may approach the person after work rather than during a break or when they are not busy working at the office.\nD. You will not be forced to tolerate the pull and manipulation which will drive you to become incredibly insecure. [title] Try meditation or something else that helps you in all aspects of your life.","prompt":"[header] How to cope with arrogant people [title] Enter an encounter with an arrogant person with the strong and confident realization that you are well and strong. [step] When you feel sound within your own self, there is nothing an arrogant person can say or do to undermine you. Your sense of confidence and self-worth will prohibit you from being vulnerable to an arrogant person.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4388,"error":null,"output":"B"}
{"question_id":"hellaswag:00004831","item_index":327,"attempt":0,"prompt_hash":"a6279ced5ec066931ea750b3493017cb79ffd5b3b800d4ba74e5cdbd05f117a8","question":"[header] How to bottle feed calves [title] Purchase or gather the appropriate feeding supplies. [step] You'll need a two quart calf bottle with nipple and raw cow's milk or a bovine milk replacement formula, and possibly colostrum. Choose a milk replacement formula that is at least 25% protein.\n\nA. [substeps] Standard cow's milk nipple (gold) is a popular choice for babies older than 4 months. It is 2.5% protein and is also excellent at feeding bulls the correct amount.\nB. You can get nipple or whey milk backup at your local grocer. [substeps] You'll need 4 pounds (19 kg) of calf's milk for every 1 inch (2.5 cm) of virgin cow's milk that you use.\nC. Make sure the formula is fresh and nonfrozen, or you could mess with the calcium content. Plastic or wool milk replacement bottles help support the weight of your calf after feeding.\nD. Other supplies can include a syringe or turkey baster, which you might find helpful in the beginning when getting the calf accustomed to the bottle. [substeps] A calf bottle that holds two quarts of liquid is all you should need.","prompt":"[header] How to bottle feed calves [title] Purchase or gather the appropriate feeding supplies. [step] You'll need a two quart calf bottle with nipple and raw cow's milk or a bovine milk replacement formula, and possibly colostrum. Choose a milk replacement formula that is at least 25% protein.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4386,"error":null,"output":"D"}
{"question_id":"hellaswag:00003247","item_index":328,"attempt":0,"prompt_hash":"03627e7181d3ea133fa81be4e740bef630d9f4b2da10e5fb81eacb6ab8fe4d69","question":"[header] How to do the \" shoulder stand \" yoga position [title] Set up your yoga station. [step] Find an adequate space without distractions. Unroll your yoga mat evenly onto the floor.\n\nA. In a reclining or a sitting position, roll your body over and place one arm on the yoga mat. The other arm should be resting against a wall or on the floor.\nB. If you need additional padding for high-pressure poses, place it under where your shoulders will be when you lie down. Put on clothing that allows you to move freely.\nC. Find a much smaller and less bulky yoga mat to stand on, so that your shoulder width makes your torso--both sides--more even. [title] Breathe in slowly through your nose and out through your mouth.\nD. Lie down on a yoga mat with your back straight. Your head should be on a diagonal, with your feet shoulder width apart.","prompt":"[header] How to do the \" shoulder stand \" yoga position [title] Set up your yoga station. [step] Find an adequate space without distractions. Unroll your yoga mat evenly onto the floor.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4393,"error":null,"output":"B"}
{"question_id":"hellaswag:00006520","item_index":329,"attempt":0,"prompt_hash":"eaf9e9c1908f45d79d6a57ff23594ec4c154b3498fac271e4ab6002c8af08822","question":"[header] How to make professional looking ripped jeans [title] Research different ripped jeans styles. [step] Look online and in fashion magazines for distressed jeans. Find out what kind of a ripped jeans style you want to create.\n\nA. This can influence the feel of your jeans. [substeps] The fit of distressed jeans fit your body and adds some volume to your look.\nB. Then consider what effect your ripped jeans are on the body and the other components of your wardrobe. [substeps] For example, if you want a more edgy look, look for jeans that are leg skinny or form fitting.\nC. It's not a great idea to try to stop a style that doesn't seem to attract your eye. [title] Find out about the issue of ripped jeans.\nD. [substeps] Pay attention to how many rips jeans have and where those are. Two or three tears are usually sufficient and not overdone, but you should pick what looks most appealing to you.","prompt":"[header] How to make professional looking ripped jeans [title] Research different ripped jeans styles. [step] Look online and in fashion magazines for distressed jeans. Find out what kind of a ripped jeans style you want to create.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4386,"error":null,"output":"D"}
{"question_id":"hellaswag:00000341","item_index":330,"attempt":0,"prompt_hash":"578e5f5ec2359595cd11a8d32e601b5af3c1859176344f906d62b2fc2546f188","question":"The dog is playing on a rug after the bath is over with wet fur on his body. the little puppy\n\nA. is running around the dog with wet fur too.\nB. is doing tricks in the bath tub behind the dog.\nC. is trying his best to keep to himself while getting bathed.\nD. becomes enraged by the bath and runs towards the door.","prompt":"The dog is playing on a rug after the bath is over with wet fur on his body. the little puppy","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4391,"error":null,"output":"A"}
{"question_id":"hellaswag:00003926","item_index":331,"attempt":0,"prompt_hash":"8436b19bb8042e7236cf5f680ee2b3a3363678aeb67400d87f55c8d2350aa739","question":"[header] How to treat melasma [title] See your primary care physician. [step] Talk to your primary care physician about any hormonal medication changes and creams you can try for melasma before going to see a dermatologist. Treatment for melasma may be considered elective and it may not be covered by your insurance.\n\nA. [substeps] Your primary care physician will be able to make specific recommendations as to the treatment that is best for you and for your baby. They may follow these recommendations to determine the best treatment plan for you.\nB. [substeps] Your doctor may suggest radiation therapy or deep thrombosis treatment. Melasma occurs when certain protein is injected into blood vessels through the nose.\nC. Find out the costs of any treatments and procedures before you schedule them. [title] Stop taking medications that could be to blame.\nD. [title] Schedule an appointment with your primary care physician. [step] If you fear that you may have melasma, you should let your primary care physician know as soon as possible.","prompt":"[header] How to treat melasma [title] See your primary care physician. [step] Talk to your primary care physician about any hormonal medication changes and creams you can try for melasma before going to see a dermatologist. Treatment for melasma may be considered elective and it may not be covered by your insurance.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4386,"error":null,"output":"D"}
{"question_id":"hellaswag:00000043","item_index":332,"attempt":0,"prompt_hash":"7c0d46fee08d9a82f005d688ed57bc9645a2879e8571a390c8008372f8a4e907","question":"One of the ping pongs lands in the cup and one of the boys begins to drink the beer. the group\n\nA. of teens is sailing down the river with others sailing in the background.\nB. gets up together and poses with their vests to celebrate.\nC. continue to play beer pong when one final player is able to throw and land a ping pong ball while standing with his back to the cups at which point the boys celebrate, and laugh.\nD. does a trick and the liquid is poured on a second table.","prompt":"One of the ping pongs lands in the cup and one of the boys begins to drink the beer. the group","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4392,"error":null,"output":"C"}
{"question_id":"hellaswag:00001585","item_index":333,"attempt":0,"prompt_hash":"5dd8fb804b588301e8cf92ba48f87c5e9560535f91bf9667ef7848f619b469cb","question":"The video takes place in the pool of a backyard. a tight rope\n\nA. is strung across the length of the pool and a woman is attempting to walk across, she falls several times.\nB. is attached to several waterproof ropes inside the fence.\nC. is tied between two trees.\nD. moves quickly about the pool.","prompt":"The video takes place in the pool of a backyard. a tight rope","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4391,"error":null,"output":"C"}
{"question_id":"hellaswag:00003234","item_index":334,"attempt":0,"prompt_hash":"5d42d7e09942db717fa4ea45aa65bc4de4ffe52a45255a578fe4297935973f4d","question":"We see two pieces of ancient asian art. we\n\nA. see people playing a game outdoors.\nB. see the legos and we see a woman smiling.\nC. then see people lifting weights in a gym.\nD. then see people performing martial arts in a plaza and a field.","prompt":"We see two pieces of ancient asian art. we","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4389,"error":null,"output":"D"}
{"question_id":"hellaswag:00006338","item_index":335,"attempt":0,"prompt_hash":"402d9ac51199471558d0fe3a4f31d5fe4e4fd3948661bcdd61bb91f179238da2","question":"[header] How to help kids cope with the death of their cat [title] Talk with your child privately in the evening. [step] Waiting until the day winds down is a wise idea. This will give your child time to grieve and process before they have to go out and do anything.\n\nA. That said, do not tell them right before bed, as this would not give them time to talk to you or think about the news. [substeps] For example, you might aim for telling them right after dinner.\nB. It gives them time to digest and contemplate their emotions. Or, it may result in them going for a walk or taking a relaxing bath.\nC. However, it's still a good idea to sit your child down and let him or her grieve for 7 days. [title] Avoid letting your child back into the house during this time of grieving.\nD. If your child is going to the vet or board for a checkup or a confirmation meeting, be sure to give them plenty of time to be alone and think. [substeps] You might be tempted to lie to your child about the deceased pet, but don't lie all the time.","prompt":"[header] How to help kids cope with the death of their cat [title] Talk with your child privately in the evening. [step] Waiting until the day winds down is a wise idea. This will give your child time to grieve and process before they have to go out and do anything.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4388,"error":null,"output":"A"}
{"question_id":"hellaswag:00005429","item_index":336,"attempt":0,"prompt_hash":"db70de233554dbdd69f9bc2ab393913ea30dccbef463798f70bfee948fa0dcf0","question":"[header] How to become a marketing researcher [title] Get a bachelor's degree. [step] The education requirements for becoming a market researcher can vary, but you need at least a bachelor's degree. [title] Choose your major depending on your interests.\n\nA. [step] Different majors require different approaches to market and employees, so your bachelor's degree may not be a good choice. For example, if you love to write video games or design, look for a major that focuses on science or technology.\nB. [step] To be a market researcher you can major in many fields. Common majors are marketing, economics, market research, math, statistics, and computer science.\nC. [step] The field to which you want to work may depend on the field that you are applying to, the potential employers who you would like to work in, or your particular research specialty. You'll also need to present to every prospective employer a picture of your prior experience in the field that will make your application more appealing.\nD. [step] In general, there are nine major majors that you need to be well versed in to become a successful marketing researcher: marketing, sales, journalism, and laboratory research. [title] Pursue a business degree that's relevant to your specific field.","prompt":"[header] How to become a marketing researcher [title] Get a bachelor's degree. [step] The education requirements for becoming a market researcher can vary, but you need at least a bachelor's degree. [title] Choose your major depending on your interests.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4387,"error":null,"output":"B"}
{"question_id":"hellaswag:00009703","item_index":337,"attempt":0,"prompt_hash":"dc326611c4d35d927cf001e2db4668383d5ee6b1cb3321190f95e77c3ea8355c","question":"[header] How to make mineral blush [title] Start with a base of arrowroot powder. [step] Add your arrowroot powder to a small clean makeup container, or any other small container that you will be able to mix and store your powder in. You can likely find arrowroot powder at your local grocery store, but it is also available online if you have difficulty.\n\nA. Make sure you have a highlighter brush as well-you can get one online. Your highlighter brush should be at least 3 in (7.6 cm) tall for use with smoothies.\nB. Arrowroot powder works as a great base for your mineral blush because it adheres nicely to the skin and works as a great medium for mixing colors. [substeps] This makes one small batch of mineral blush.\nC. [substeps] Arrowroot powder is arguably one of the most popular powders in makeup, as it comes in an eye-catching color. [title] Wrap your face in a towel and dip a cotton swab in the mixture.\nD. [substeps] Arrowroot powder is too soft to dissolve. [title] Scoop the arrowroot powder out of the container.","prompt":"[header] How to make mineral blush [title] Start with a base of arrowroot powder. [step] Add your arrowroot powder to a small clean makeup container, or any other small container that you will be able to mix and store your powder in. You can likely find arrowroot powder at your local grocery store, but it is also available online if you have difficulty.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4389,"error":null,"output":"B"}
{"question_id":"hellaswag:00005292","item_index":338,"attempt":0,"prompt_hash":"62f39f807d7b3b56db6008500d0af091fa24236cc2de24882ee6832d1d0228d3","question":"[header] How to stretch suede shoes [title] Try a spray solution to stretch your shoes 1/4 to 1/2 a size. [step] Spraying your shoes then wearing them for a few hours is a quick fix. If you only need to stretch them 1/4 to 1/2 a shoe size, a spray could do the trick.\n\nA. [substeps] Spray solutions are also your most affordable option. [title] Purchase an immediate-use solution marked for suede shoes.\nB. [substeps] The spray can be purchased at a shoe store, at a department store and inside your shoes. Be sure to apply the solution before you bring your shoes inside and dry them before proceeding with your stretching.\nC. You can also add plain water and a bit of superglue for extra grip, if you wish. [title] Try the \" flower \" method, this is done by squeezing the laces out with a tension wrench.\nD. Skip the spraying procedure if you have shoes that require application. [substeps] Be aware that the spray solution should be more than enough when getting your shoes to fit you perfectly so that the spray can be applied more evenly.","prompt":"[header] How to stretch suede shoes [title] Try a spray solution to stretch your shoes 1/4 to 1/2 a size. [step] Spraying your shoes then wearing them for a few hours is a quick fix. If you only need to stretch them 1/4 to 1/2 a shoe size, a spray could do the trick.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4388,"error":null,"output":"B"}
{"question_id":"hellaswag:00004085","item_index":339,"attempt":0,"prompt_hash":"63c5a7b2390c22c82b7408fd06cdfc61f10c372d9ec4af9c2db5450b66b791ff","question":"[header] How to take triphala [title] Use a traditional form of triphala. [step] You can take triphala fruits in their dried form, or prepared as a tea. Both of these traditional forms can be purchased online or from natural health stores.\n\nA. The upper part of the fruit is divided into 12 fruits to serve as pulp, or tempeh. [title] Slice the fruit into sections and fill each section with the fruit.\nB. To prepare the tea, stir a half teaspoon (2.5 g) of the powder into a cup of hot water. Alternatively, you can combine the same amount of powder with honey or ghee and take it before a meal.\nC. [substeps] Both styles of triphala fruits are divided into a single fruit and a tea form. [title] Use dried fruits in place of whole fruit.\nD. The fruit is ripe and seed arranged roughly in a tea bag or capsule or wrapped tightly in some paper, provided you have packed the fruit in about half of the packet. [substeps] There are many different varieties of dried fruit, all of which can be quite a bit healthier.","prompt":"[header] How to take triphala [title] Use a traditional form of triphala. [step] You can take triphala fruits in their dried form, or prepared as a tea. Both of these traditional forms can be purchased online or from natural health stores.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4390,"error":null,"output":"B"}
{"question_id":"hellaswag:00004806","item_index":340,"attempt":0,"prompt_hash":"9606de81b63601d093a071e387d1dfebcb68162c29f33405e2ce163f11f51089","question":"[header] How to serve healthy and appealing toddler snacks [title] Choose kid-sized foods. [step] When you're looking for nutritious foods to feed your toddler, you need to make sure you pick appropriate foods. Choosing items that are small in size or that are \" toddler-sized \" will be helpful.\n\nA. [substeps] Toddlers need to have smaller sized foods. Their hands, fingers, mouth and teeth are smaller than adults'.\nB. [title] Look for baked goods at the grocery store. [step] Look for foods that are a little higher in calories or have more protein, as these will add up better to your toddler's nutritional needs.\nC. Add large items, like yogurt, fruits, vegetables and grains to your toddler's snack tray. [title] Add toddler-sized meals to the snack tray.\nD. Child-sized foods are small and usually only have a small amount of added starch. That way, if your toddler doesn't want to eat something, you can ask if it's snack-size for him/her.","prompt":"[header] How to serve healthy and appealing toddler snacks [title] Choose kid-sized foods. [step] When you're looking for nutritious foods to feed your toddler, you need to make sure you pick appropriate foods. Choosing items that are small in size or that are \" toddler-sized \" will be helpful.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4389,"error":null,"output":"B"}
{"question_id":"hellaswag:00008350","item_index":341,"attempt":0,"prompt_hash":"a16c0bbeaad1f80bca6a11ad0d7fc67896acbf233e63bf9ebb46371e95157c03","question":"[header] How to get a divorce in nevada [title] Schedule a time to talk with your spouse. [step] For an uncontested divorce, meet with your spouse to discuss the terms and come to an agreement. Set aside several hours for the two of you to talk alone.\n\nA. Send the children to visit friends so that you won't be interrupted. You probably will need several meetings before you can reach agreement on important issues.\nB. Plan to meet away from the courthouse or to a private room with good lighting. [substeps] Remember, your finances are fairly low.\nC. [substeps] Agree on some type of agreement so that you can be in touch with each other when you need to come to an agreement. Keep in mind that many courts will require that a contested divorce be considered uncontested in an agreement.\nD. You may want to place a request to speak with your spouse over phone, skype, or even text messaging so that the possibility of speaking in person is less of a burden. [substeps] If you have children together, schedule time for them to get to know your spouse.","prompt":"[header] How to get a divorce in nevada [title] Schedule a time to talk with your spouse. [step] For an uncontested divorce, meet with your spouse to discuss the terms and come to an agreement. Set aside several hours for the two of you to talk alone.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4389,"error":null,"output":"A"}
{"question_id":"hellaswag:00004811","item_index":342,"attempt":0,"prompt_hash":"e2c88bf78e6a6a7e922f885e6990203a8cc1528b02597a866ba50d3ae80d032f","question":"[header] How to make glow in the dark nail polish [title] Gather your supplies. [step] Because you will be using glow sticks, which only glow for a certain amount of time, it is best to have everything on hand and ready. Plan the make this polish just before you need to use it, but be sure to include enough drying time.\n\nA. Here is a list of what you will need : [substeps] 1 glow stick 1 bottle of nail polish (partly full) sharp scissors strainer (recommended ) [title] Choose your nail polish and glow stick colors. [step] You can use a clear polish or a colored one.\nB. You will need to put on some pink eye makeup, and get a color-filled eyeshadow that will make you smile. [substeps] You can also make your own pale pink lipgloss by opening a pink bottle with a candy thermometer and dipping the amount you use in water.\nC. [substeps] You will need to have a base coat that can withstand 5 minutes to 2 hours. Try this in the morning, and remember to take some time to clean it before applying it to your nail polish.\nD. The wheel light process should take about four to six hours. Check with your toothpick about how long to wait in case you get burned.","prompt":"[header] How to make glow in the dark nail polish [title] Gather your supplies. [step] Because you will be using glow sticks, which only glow for a certain amount of time, it is best to have everything on hand and ready. Plan the make this polish just before you need to use it, but be sure to include enough drying time.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4371,"error":null,"output":"A"}
{"question_id":"hellaswag:00002749","item_index":343,"attempt":0,"prompt_hash":"d0e0b15e8faeb5065a5d92c0a27212a32dd24e51e1385350d825a341415f1008","question":"The man mounts the pommel horse and spins his legs around it. The man does a handstand and dismounts. the man\n\nA. holds out his arms while spinning his legs.\nB. takes a bow and starts walking away.\nC. does a handstand and dismounts again.\nD. use his foot to adjust his position and stance to prepare for his dismount.","prompt":"The man mounts the pommel horse and spins his legs around it. The man does a handstand and dismounts. the man","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4370,"error":null,"output":"C"}
{"question_id":"hellaswag:00005421","item_index":344,"attempt":0,"prompt_hash":"ae9c739f7bcb726dd1e294c733951a2ecce8a4e4f702edeba4eedf6a2c27bebf","question":"[header] How to choose a watch [title] Determine why you want a watch. [step] There are hundreds of different types of watches available and they often have very different functions. A luxury watch that you will wear with suits or formal wear will be very different than an athletic watch you will use to time your runs.\n\nA. If your experience is the sole object of running, opt for a watch that has both a physical component and a practical component. Well-respected watches include the firebrick watch, the rhd watch, and the mx watch.\nB. When you are able to determine if this is a special occasion watch, or a watch for everyday use, you can begin to narrow your search. [substeps] Some common types of watches will be for athletic training, formal wear, everyday use, as a tech piece, or an antique piece.\nC. Think about all of the different features and include any specific factors such as who the watch's owners are, including their names and trademark information. If you are looking for a necessity watch, then it is likely that one of these features would be readily available for you.\nD. A traditional watch may look very similar, but it is more expensive and worn on a necklace. [title] Choose a watch that suits your lifestyle.","prompt":"[header] How to choose a watch [title] Determine why you want a watch. [step] There are hundreds of different types of watches available and they often have very different functions. A luxury watch that you will wear with suits or formal wear will be very different than an athletic watch you will use to time your runs.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4371,"error":null,"output":"B"}
{"question_id":"hellaswag:00000639","item_index":345,"attempt":0,"prompt_hash":"f18a60cea3322966a8b8ed12720760c3f478d301c2dd91a340193267343555dd","question":"The music begins and the girls do their routine of dancing, jumping, making different line formations and baton twirling as the crowd cheers them on. the group\n\nA. ends their routine as they dramatically leave the gym one by one.\nB. then moves around the ground doing swaying motions.\nC. continues individually performing and ends without showing how the intro intro came to be.\nD. finishes their routine and the instructor throws their sticks down in the sand.","prompt":"The music begins and the girls do their routine of dancing, jumping, making different line formations and baton twirling as the crowd cheers them on. the group","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4370,"error":null,"output":"D"}
{"question_id":"hellaswag:00005413","item_index":346,"attempt":0,"prompt_hash":"40819885cd7b75d49447297516ddc319272502c15efbac3aa98c1d1e01a59c07","question":"[header] How to write any high school essay [title] Identify your topic. [step] Choose an essay topic that you know well and feel comfortable writing about in your five paragraph essay. [title] Create your thesis.\n\nA. [step] A thesis is a series of in-text questions that you present to your readers in a clear way so that they can be read over. The thesis should be regarding specific points or descriptions of the topic in a given paragraph.\nB. [step] After you've identified why you want to write, create a thesis statement that explicitly addresses the topic. [substeps] Choose one essay topic that clearly tells the reader why you've chosen it and why you are writing it.\nC. [step] Begin, think, present, then write. This will take quite a bit of time.\nD. [step] Your thesis is the statement that your entire essay will make. Your thesis should outline the main idea of the paper.","prompt":"[header] How to write any high school essay [title] Identify your topic. [step] Choose an essay topic that you know well and feel comfortable writing about in your five paragraph essay. [title] Create your thesis.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4370,"error":null,"output":"D"}
{"question_id":"hellaswag:00008187","item_index":347,"attempt":0,"prompt_hash":"841c0ce6f576803f7e05c1a299c1320d047594bf20b6c27f886f275103c16b58","question":"[header] How to make guitar hero notecharts on powerpoint [title] Open microsoft powerpoint on your computer. [title] Make a whole bunch of pages and close out the windows for typing in words. [title] Put in everything you would see in the guitar hero game on the first page like the notes and the lines and the things that the notes go into.\n\nA. [title] Group the notes that are coming at you and copy the notes onto each page. [title] Do the same with the lines that order the notes (the line that the notes are on or the lines that separate the notes).\nB. [title] Get the whole song and put it in quotes. [step] Make a translated quote and put this on each page.\nC. [title] Put it all in the far left window so you can see the meta do all the work. [title] Switch from a few screenshots of the song or album to have it flashes before the screen.\nD. [title] Then click on cell a1 so you can choose chord-space options. [title] Write in the on box that says song title, instrument names, anything you like.","prompt":"[header] How to make guitar hero notecharts on powerpoint [title] Open microsoft powerpoint on your computer. [title] Make a whole bunch of pages and close out the windows for typing in words. [title] Put in everything you would see in the guitar hero game on the first page like the notes and the lines and the things that the notes go into.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4301,"error":null,"output":"A"}
{"question_id":"hellaswag:00008525","item_index":348,"attempt":0,"prompt_hash":"0b74321edf94615fe1fbde7a25b079d63e1d46a28ceb6decadb958e7d59d9ba2","question":"[header] How to open an encrypted email on pc or mac [title] Exchange digital ids with the sender. [step] You must do this before they send you the encrypted message. To do this, send each other digitally-signed emails.\n\nA. When the sender opens the email you will see your name and a notification letting them know that the email is encrypted. [substeps] After sending the email, you will be given 20 minutes to set up gmail (or other email transfer protocol), confirm you want the sender to clear the folder on your pc.\nB. This will link up the email and your country code together to create a encryption. [substeps] You can only exchange international email information with the sender.\nC. [title] Add the sender's digital id to your contacts. [step] This way, any encrypted emails from that person can be opened by you.\nD. To do this, you should take a copy of the message once you send it. Do not share your identity with anyone other than yourself as it would be illegal to transmit illegal messages.","prompt":"[header] How to open an encrypted email on pc or mac [title] Exchange digital ids with the sender. [step] You must do this before they send you the encrypted message. To do this, send each other digitally-signed emails.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4300,"error":null,"output":"C"}
{"question_id":"hellaswag:00000188","item_index":349,"attempt":0,"prompt_hash":"ebdbf6d61c580a558cd7b02f77fedb951a5a9ffdf91f8c7e5daffba16035ac8c","question":"A girl is seen laying on a table with a man piercing her belly button below her. the girl\n\nA. laughs at the camera while holding a can of herself and the man.\nB. holds up a bottle of fluid and puts it on an iv.\nC. under the belly's cycle begins to milk the tube from the bottle that is in her stomach and pans over to the man piercing her belly button.\nD. screams in pain and the man finishes her piercing while she looks to the camera.","prompt":"A girl is seen laying on a table with a man piercing her belly button below her. the girl","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4355,"error":null,"output":"D"}
{"question_id":"hellaswag:00009682","item_index":350,"attempt":0,"prompt_hash":"ec734bf6f0643b374dad666dcfc054048ce6702eeed6f0527341de4d3de1d79e","question":"[header] How to get rid of a fake tan [title] Change into clothing normally reserved for cleaning, moving, or working with harsh chemicals. [step] This prevents baby oil from permanently staining clothing you care about. [title] Apply a coat of baby oil to the section of skin from which you want the fake tan removed.\n\nA. [step] Apply another coat of baby oil to the entire part of your skin from which you desire a fake tan. Again, the resulting oil is necessary for the fake tan to stay on your skin.\nB. [step] Do this daily, or as soon as you tan. [title] Pick a painting or a photograph you love to look at or print out.\nC. [step] Baby oil helps loosen the top layer of dead skin cells that have been tanned. [title] Thoroughly rub and massage the baby oil into your skin.\nD. [step] Baby oil is an essential oil, and moisturizing it on this section means that you won't need to clean your top once the baby oil is removed. [title] Unexpose the skin around your fake tan.","prompt":"[header] How to get rid of a fake tan [title] Change into clothing normally reserved for cleaning, moving, or working with harsh chemicals. [step] This prevents baby oil from permanently staining clothing you care about. [title] Apply a coat of baby oil to the section of skin from which you want the fake tan removed.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4297,"error":null,"output":"C"}
{"question_id":"hellaswag:00007734","item_index":351,"attempt":0,"prompt_hash":"69e015a772fcf5c1c632441cdee2815bf8cbe05403c4af6025bb1862a61fcd4f","question":"[header] How to find a job as a preteen [title] Do extra chores. [step] This does depend on how cooperative your parents are, but if you are polite in asking and explain why you want the spending money, they're likely to listen. It's an easy way to make 10 or 20 bucks (or more, if your parents are generous).\n\nA. You could give the whole bathroom a good scrub down, or vacuum and mop the entire house. [substeps] You could ask your mom which chores she hates doing the most, and pick those up for her.\nB. Anything under 5 dollars (or less) earns your allowance. Here's how : [substeps] Watch a movie together at the end of the night.\nC. [substeps] Keep some activities in mind when doing chores (e.g. going to the bathroom, doing homework at home, walking to and from school, etc.\nD. [title] Invest in little toys for your little children. [step] As young as possible, buy baskets of tiny toy animals with fun stuff inside them or even a cat or dog with a loose ball inside so that you can look like a kid.","prompt":"[header] How to find a job as a preteen [title] Do extra chores. [step] This does depend on how cooperative your parents are, but if you are polite in asking and explain why you want the spending money, they're likely to listen. It's an easy way to make 10 or 20 bucks (or more, if your parents are generous).","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4297,"error":null,"output":"A"}
{"question_id":"hellaswag:00004567","item_index":352,"attempt":0,"prompt_hash":"36f187d100080190e8d761cb90e0aea9917d3712d4b084185603d9ebd79375ed","question":"[header] How to remove facial hair permanently [title] Search online to find electrologists in your area. [step] An electrologist is a person who has gone through special training to perform the electrolysis procedure. Research different electrologists in your area and make a list of the ones you think are most qualified.\n\nA. You should then look through their information on a case-by-case basis to determine which one would be the best fit for you. [substeps] Best looking online for electrologists is to have someone look them up.\nB. [substeps] Ask for a referral to another certified electrologist within your area. You may have to expand your search by looking at more people in the field and visiting electropostale website at http: // www.electrodocs /.\nC. Try to find at least 3-4 electrologists to start with. [substeps] Look for electrologists with at least 5 years of experience in the field, with positive reviews on their business page and social media and a professional-looking website.\nD. [substeps] You may be surprised how many electrologists who perform these procedures don't report much improvement. [title] Ask your friends or family for recommendations.","prompt":"[header] How to remove facial hair permanently [title] Search online to find electrologists in your area. [step] An electrologist is a person who has gone through special training to perform the electrolysis procedure. Research different electrologists in your area and make a list of the ones you think are most qualified.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3692,"error":null,"output":"C"}
{"question_id":"hellaswag:00009967","item_index":353,"attempt":0,"prompt_hash":"a959351895f05fcec764a6e63b0fec537f75fb730c4fd3c0410ec9b14ab40d69","question":"[header] How to welcome a foster child [title] Introduce yourself by the name you would like the child to use. [step] Your welcome begins as soon as the caseworker walks into the home with the foster child. Use a soft, friendly voice to greet the caseworker and the child.\n\nA. Follow up your introduction with a question such as, \" how are you today? \" followed by a warm greeting like, \" hi, i'm molly. \" once the child's name is said, you can say, \" hello, i'm jeffrey and i brought the foster child here today to welcome them.\nB. Introduce yourself by the name you would like the child to use when referring to you. Do not insist the child call you \" mom \" or \" dad, \" as this will be confusing to the child.\nC. [substeps] Say, \" hello! You are the child's grangiver, \" or, \" this is jake. \" look at the name the caseworker gave the child and check for a sign indicating that it is wrong.\nD. [substeps] You can say \" dear, it's nice to meet you mr. sims! \" or \" hello joe, you're sure, \" where appropriate.","prompt":"[header] How to welcome a foster child [title] Introduce yourself by the name you would like the child to use. [step] Your welcome begins as soon as the caseworker walks into the home with the foster child. Use a soft, friendly voice to greet the caseworker and the child.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3693,"error":null,"output":"B"}
{"question_id":"hellaswag:00002758","item_index":354,"attempt":0,"prompt_hash":"fa817d6a613d3089b0c0ea44ef04194d2daa06d149de2d1c635d93a23e571cbe","question":"A man is playing the bagpipes at the bottom of the stairs in the living room of a house, wearing a button up and dress pants, along with socks. he\n\nA. switches to the dollar earphones with his face turned toward the camera.\nB. goes inside of him car, pulls out a camera, and takes a picture.\nC. begins to tap his foot along with the beat of his tune, he starts to tap his left foot in a back and forth rhythm with his right foot.\nD. takes a few puffs from a large pipe and puffs the rest out in the dish, before putting it down, he looks to the camera and laughs.","prompt":"A man is playing the bagpipes at the bottom of the stairs in the living room of a house, wearing a button up and dress pants, along with socks. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3696,"error":null,"output":"C"}
{"question_id":"hellaswag:00001573","item_index":355,"attempt":0,"prompt_hash":"432bd314b47c03ffdb8e67c1d7b438d745aa57ead051ba3eef1e074004362e0c","question":"We wee the fourth rider in the scene. The lady is riding her bike slower. The lady is riding her bike slowly. we\n\nA. see the closed sign on the bottom of the screen.\nB. see the lady climbing on the bunk bed in the dark room.\nC. see the closing screen on a blue background.\nD. see the lady in the water then the people in the swimming pool at sunset.","prompt":"We wee the fourth rider in the scene. The lady is riding her bike slower. The lady is riding her bike slowly. we","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3695,"error":null,"output":"A"}
{"question_id":"hellaswag:00008104","item_index":356,"attempt":0,"prompt_hash":"145b672e9290fe9612ded47c20640d7a0d4967fb454bee53c243fec3bdad3cae","question":"[header] How to choose a flower girl dress [title] Choose a color for the dress. [step] Choose a dress color that will complement other colors in the wedding. Match the flower girl dress to your bridesmaids' dresses.\n\nA. Alternatively, choose a hue that will match another element in your wedding, like the flowers in your bouquet. Consider finding a flower girl dress that will contrast with and complement the colors in your wedding decor.\nB. [substeps] Whenever possible, try pairing the flower girl dress with a dress contrasting your bridesmaids' dresses. The color scheme works well in formal weddings because bridesmaids can pick and match flowers.\nC. Think about what you like and what you want when choosing a color for your bouquet. [substeps] If you are not big on color, choose a dress that is a darker color than your wedding dress.\nD. This will give you a reasonable opportunity to have fun. [substeps] For example, if you have black dresses made of black around the edges, opt for a dress in an earthy shade like bronze.","prompt":"[header] How to choose a flower girl dress [title] Choose a color for the dress. [step] Choose a dress color that will complement other colors in the wedding. Match the flower girl dress to your bridesmaids' dresses.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3694,"error":null,"output":"A"}
{"question_id":"hellaswag:00004136","item_index":357,"attempt":0,"prompt_hash":"cfaf23180c4f2a75e96a82c3aae05ebd0d5f7ae7fc3fb30e2d033962edd70a12","question":"[header] How to use steam rollers for hair [title] Start with dry hair. [step] You can curl your hair just after you washed it, but it needs to be perfectly dry. Steam rollers will work the best on straight hair, but you can try using them to define curly or wavy hair.\n\nA. Since hair can be dry, leave it down once you're out of the shower. [title] Use a hair-straightening comb to detangle your hair.\nB. Begin by pouring a small amount of water onto a foam brush. When your hair is moist and damp, focus on putting an extension under the roller.\nC. Just curl your hair before washing your hair, or wet your hair. [title] Gather your hair into three high ponytails.\nD. If you have natural or kinky hair, however, you will need to start with a blow-out. Otherwise, the curls will not have as much definition.","prompt":"[header] How to use steam rollers for hair [title] Start with dry hair. [step] You can curl your hair just after you washed it, but it needs to be perfectly dry. Steam rollers will work the best on straight hair, but you can try using them to define curly or wavy hair.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3693,"error":null,"output":"D"}
{"question_id":"hellaswag:00009495","item_index":358,"attempt":0,"prompt_hash":"4047e87065de8e58841d02b05d8cd742e2e2662245e291d52a0cbc620c0d7cbb","question":"[header] How to buzz cut hair [title] Use high-quality clippers with secured guards. [step] If you don't already have a pair of high-quality clippers, you can purchase a pair costing between $20 and $120 online. Before you start buzzing, make sure the guard is securely fastened to the clippers.\n\nA. Otherwise, you could end up with a bald spot if it falls off while in use. [substeps] Many clippers will come as a kit, including not only the clippers but the charging station, guards, combs, and barber scissors.\nB. [substeps] Go ahead and set the guard so it's to your right when you start to buzz. This will ensure your hair doesn't get in the way if you accidentally buzz it.\nC. [title] Shave down the sides of your head and across your temples. [step] In order to buzz your hair, you'll want to shave down to the tips of your ears.\nD. [substeps] Wear a pair of safety gloves to protect your hands from getting scratched by the prongs of the clippers. [title] Decide how many pieces of hair you want to buzz.","prompt":"[header] How to buzz cut hair [title] Use high-quality clippers with secured guards. [step] If you don't already have a pair of high-quality clippers, you can purchase a pair costing between $20 and $120 online. Before you start buzzing, make sure the guard is securely fastened to the clippers.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3689,"error":null,"output":"D"}
{"question_id":"hellaswag:00007593","item_index":359,"attempt":0,"prompt_hash":"70a51481252db2080369086c8d9dc3dd8d210430ecfc3fb7de46ae8fcc50bdc4","question":"[header] How to show off your curves [title] Get properly fitted for a bra and shape wear. [step] Having the right sized bra and undergarments, such as spandex, is important to accentuate or create curves. Ill-fitting undergarments will not enhance your curves, but they may also be uncomfortable.\n\nA. [substeps] Stores that specialize in quality lingerie can offer you qualified salespeople who will find for you the proper size bra. Note that sizing varies from one company to the next, so you may have a different bra size for each brand you wear.\nB. [substeps] If your clothes feel restrictive, this is because your body is not able to expel the excess moisture that lessens the shape. The amount of pressure worn on your body is important, as clothes are not because they are comfortable.\nC. Dress in a swimsuit that is appropriate to wear to work or school. [substeps] Make sure that your fit is not too tight and that you are not wearing anything too revealing, with plunging necklines that are too big or too small.\nD. Consider an altered bra that fits your shape better. If you are choosing a bra made from a solid material, ask for advice on how to make it a good fit.","prompt":"[header] How to show off your curves [title] Get properly fitted for a bra and shape wear. [step] Having the right sized bra and undergarments, such as spandex, is important to accentuate or create curves. Ill-fitting undergarments will not enhance your curves, but they may also be uncomfortable.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3690,"error":null,"output":"A"}
{"question_id":"hellaswag:00003260","item_index":360,"attempt":0,"prompt_hash":"1f9ec9a22ba69a20754a9b65786954106dc7272e6a8c484fcc49de18e3209e3f","question":"[header] How to become a sports announcer [title] Research the field. [step] Before you pursue any formal education, do enough research to know what you can expect from this career field. Check into your expected duties, work environment, expected pay, and job outlook.\n\nA. Keep a running log of the projected jobs and positions you plan to apply for. Check at your recruitment agency for more information.\nB. [substeps] Most sports announcers are either play-by-play announcers, talk show hosts, or sports anchors. Your duties may require you to announce plays, provide commentary, and interview guests.\nC. Avoid going into a job bureau job applying for a position you have no past experience with. Most schools will offer career-related job regimes for those who haven't gone to college; unfortunately, online courses can be incredibly high the last few years.\nD. This will be especially useful in all types of sports sports. [title] Prepare to show the world you're capable of performing one day.","prompt":"[header] How to become a sports announcer [title] Research the field. [step] Before you pursue any formal education, do enough research to know what you can expect from this career field. Check into your expected duties, work environment, expected pay, and job outlook.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3689,"error":null,"output":"B"}
{"question_id":"hellaswag:00005264","item_index":361,"attempt":0,"prompt_hash":"cf8b859bdfafa6bae3271021fefda2855af78aafd91d4bc3aed197fec83dd07f","question":"[header] How to assess the safe firmness of an infant mattress to prevent asphyxiation [title] Gather your supplies. [step] Take your pen or marker, and your ruler. Place your ruler at the bottom edge of the carton.\n\nA. Trace a line across the bottom of the carton starting from the bottom edge. Once you've got the line all the way across, take your paper or pencil.\nB. You'll also want your tape measure to be approximately 2 inches (5.08 centimeters) longer than the mattress. Write down the order in which you measured your mattress.\nC. Measure up an inch and a half (40 mm). Make a mark here and then draw a line running parallel to the bottom edge.\nD. A ruler is a long, skinny, flat measuring tape measuring 24 inches (55 cm) tall. [substeps] Note that this rule applies to mattresses since the mattress is so large that it requires you to write your measurements in centimeters.","prompt":"[header] How to assess the safe firmness of an infant mattress to prevent asphyxiation [title] Gather your supplies. [step] Take your pen or marker, and your ruler. Place your ruler at the bottom edge of the carton.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3689,"error":null,"output":"A"}
{"question_id":"hellaswag:00007240","item_index":362,"attempt":0,"prompt_hash":"303bf70c1c04e04bd8d9237659761de6f81f877f5d69af26d824fe766337a11a","question":"[header] How to go on a fitness retreat [title] Determine your budget for the retreat. [step] Start by deciding how much you can afford to spend on the fitness retreat. There are fitness retreats that are more high end and will be fairly expensive, around $3 , 000-$5 , 000 or more per week, not including travel costs.\n\nA. [substeps] If you plan to consult with your friends and family, ask them to keep track of how much money they typically spend on a fitness retreat. You will need this information to decide who to invite to your retreat.\nB. There are also fitness retreats that are more affordable, less than $2 , 000, not including travel costs. Do your research and compare several fitness retreats to find one that will fit your budget.\nC. [substeps] You will need to factor in a lump sum of various fee amounts to ensure that you are not bringing too much. With the cost set at $30 , 000 up front, you would pay for additional things such as parking and equipment.\nD. [substeps] Some fitness retreats will offer limited access to a gym, some may offer free runs, others may offer light jogs, while still others may charge a flat rate. You can qualify for an outdoor retreat and have quick returns on your investment before you check into the retreat.","prompt":"[header] How to go on a fitness retreat [title] Determine your budget for the retreat. [step] Start by deciding how much you can afford to spend on the fitness retreat. There are fitness retreats that are more high end and will be fairly expensive, around $3 , 000-$5 , 000 or more per week, not including travel costs.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3688,"error":null,"output":"B"}
{"question_id":"hellaswag:00005981","item_index":363,"attempt":0,"prompt_hash":"9bd271393f1efb6f8ecb7e9d72e97e6c37e347fd8e33c7b383c96ed18ddb8c1d","question":"[header] How to control afro style hair [title] Purchase conditioner, moisturizer, food-grade oil (olive oil, coconut oil, etc. [step] ) for your hair. The longer you let the conditioner sit on your hair, the better.\n\nA. You won't need as much while its cleansing, which is more natural. [title] If you have loose or beach hair, wet your damp hair and then, while the conditioner absorbs, start putting in more product (e.g., conditioner) around the roots.\nB. [title] Be cautious of shampoos, most of them strip the moisture from your hair. [step] Moisture is your number 1 ally in achieving healthy manageable hair.\nC. It should be enough for a few hours or even a full day. The less conditioner you use, the tighter it will become.\nD. [title] Style your hair in a cleansing style with sulfate-free hairspray or leave-in products and let it sit for 10 minutes. [step] This helps restore normal hairs and rejuvenates the hair.","prompt":"[header] How to control afro style hair [title] Purchase conditioner, moisturizer, food-grade oil (olive oil, coconut oil, etc. [step] ) for your hair. The longer you let the conditioner sit on your hair, the better.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3689,"error":null,"output":"B"}
{"question_id":"hellaswag:00004502","item_index":364,"attempt":0,"prompt_hash":"d383c9aef66e5ec147309b8a0ed7f8354f6ccc2e1421cd6e78180fd79cf9b83b","question":"[header] How to start a fashion show [title] Get the location. [step] Find out where you want to run your fashion show, whether it's in the mall, your schools hallways, the cafeteria, or even your own backyard. Wherever it is.\n\nA. Find a location that can accommodate the large crowd of people already entering. [title] Get a group of people to get you started.\nB. Places such as target or the mall will typically have a lot more different ideas, never mind the venue, as models at the events will be looking at clothing they haven't worn before. By studying places such as the mall, you will be able to capture emotion from other models.\nC. Make sure to get the stylist to come clean and deal with marketing problems. If you plan to have the stylist come up with a unique idea for the show, consider planning a one-on-one discussion with your stylist that might lead to collaboration.\nD. Always book ahead and ask permission to use a facility, as this will steer you clear of potential problems. [substeps] Even if it's your own backyard, make sure your family has no plans to use it that day.","prompt":"[header] How to start a fashion show [title] Get the location. [step] Find out where you want to run your fashion show, whether it's in the mall, your schools hallways, the cafeteria, or even your own backyard. Wherever it is.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3688,"error":null,"output":"D"}
{"question_id":"hellaswag:00001161","item_index":365,"attempt":0,"prompt_hash":"b7eb9491fd2a2b79e791bdf7368138088680a4d82f248e1a8c32e52c881f844d","question":"A view is seen of snow covered ground and people on the ice. A man sets up a fishing line in a hole in a lake. he\n\nA. then pulls a fish through the hole.\nB. reels in the fish and catches a bigger fish.\nC. reels in the hook and begins reeling in the fish.\nD. stands and catches a fish and throws it in the hole with the other fish.","prompt":"A view is seen of snow covered ground and people on the ice. A man sets up a fishing line in a hole in a lake. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3690,"error":null,"output":"A"}
{"question_id":"hellaswag:00009326","item_index":366,"attempt":0,"prompt_hash":"fb392cb8a9464b67fd68b95e4b58d40da9ae6407e35241170e7d6a5f3e7ffb90","question":"[header] How to diagnose spondylosis deformans in boxers [title] Pay attention to stiffness in your dog. [step] Is your dog less agile than is used to be? If your dog has a less mobility and is reluctant to move, it may be due to this disease. This is because the spine has become fused with bone, making the dog more stiff and less flexible than it was in earlier life.\n\nA. This is also because muscle strain is brought on by arthritis and develops at a faster rate, which means that parts of the spine get loose and reduce mobility. [substeps] Look for impaired coordination in your dog.\nB. [title] Pay attention to ways your dog moves. [step] Sometimes, dogs twist in the opposite direction than they are used to.\nC. [title] Notice a difference in your dog's posture. [step] Your dog may have stiffness that causes it to slump forward, making its jaw and back muscles become stiffer.\nD. [substeps] Happily, it rarely produces symptoms other than stiffness and is most often detected when the dog is x-rayed for another reason. [title] Look for the symptoms in older dogs.","prompt":"[header] How to diagnose spondylosis deformans in boxers [title] Pay attention to stiffness in your dog. [step] Is your dog less agile than is used to be? If your dog has a less mobility and is reluctant to move, it may be due to this disease. This is because the spine has become fused with bone, making the dog more stiff and less flexible than it was in earlier life.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3687,"error":null,"output":"B"}
{"question_id":"hellaswag:00007924","item_index":367,"attempt":0,"prompt_hash":"c19321e6197ce7a9b6024dd85e6d18f000c3bb38ebe94595bf3c568b610d384c","question":"[header] How to earn fuel rewards [title] Research fuel rewards programs. [step] Many different gas stations, grocery stores, and credit card companies have fuel rewards programs. Do an internet search for programs in your area.\n\nA. Look for programs that will be the most convenient for you, such as one through the grocery store or gas station you visit the most, or a program through a credit card you already have. [substeps] Read the fine print carefully so you know how rewards are earned, what the maximum discount is, when the rewards expire, and so forth.\nB. Go to your local paper shopping for a corporate fuel service program. Search the program if it is convenient to you.\nC. [substeps] Check local ranger stations, coffee shops, and grocery stores for fuel rewards programs. See what the prices are for fuel rewards programs.\nD. Don't rely on the exactly how much you spent on fuel. For example, walmart may train computers or credit cards at the same time.","prompt":"[header] How to earn fuel rewards [title] Research fuel rewards programs. [step] Many different gas stations, grocery stores, and credit card companies have fuel rewards programs. Do an internet search for programs in your area.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3690,"error":null,"output":"A"}
{"question_id":"hellaswag:00002398","item_index":368,"attempt":0,"prompt_hash":"02334e2b94cc1b472c944a9f0a33050ef35402b48e949eeb37398d8c966aebed","question":"Man is standing in the middle of a ring jumping the rope. referee\n\nA. is standing in the middle of the man doing the distance.\nB. is standing playing with the man in th ring.\nC. is standing on a corner of a ring.\nD. is acing man to do exercises to try to keep the man going.","prompt":"Man is standing in the middle of a ring jumping the rope. referee","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3689,"error":null,"output":"A"}
{"question_id":"hellaswag:00005282","item_index":369,"attempt":0,"prompt_hash":"f4527c035807c242e98abf79adec5712196e381dfc38d70a3639a5129e0e933d","question":"[header] How to get great looking skin [title] Wash your hands properly. [step] If you don't, the bacteria and oil from your fingers can get in your pores and create infections and breakouts. It may seem like a simple task, but it may also be easily forgotten.\n\nA. Always wash your hands before applying any moisturizers or lotion. [substeps] If you don't wash your hands often, you can purchase special soaps and lotions designed for washing skin.\nB. Think about all the things you come in to contact with during a day, and the times you subconsciously touch your face. Always keep clean hands when doing any facial cleansing so you aren't being counter-productive.\nC. [substeps] The first thing you do to wash your hands is to grab a paper towel and dab it onto the side of your face using circular motions to wipe off dirt and grime. This will remove cleansers and other dirt from your face and eyes, which could damage your skin.\nD. Use a soft wash cloth to take a few gentle scrubs of your hands. Blot your hands dry and wipe excess water from your face with a clean towel.","prompt":"[header] How to get great looking skin [title] Wash your hands properly. [step] If you don't, the bacteria and oil from your fingers can get in your pores and create infections and breakouts. It may seem like a simple task, but it may also be easily forgotten.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3690,"error":null,"output":"A"}
{"question_id":"hellaswag:00001123","item_index":370,"attempt":0,"prompt_hash":"8d0049c2fde8d18bb5297f89c6915e398ccb263dbc7c1d9bec8d9e653a7e7cc7","question":"A man comes onto the screen thanking his viewers for the comments they have made about his videos. Teams to play the harmonica and perform a song for the video. when he\n\nA. 's done he takes a drink as reporters interfix themselves in between his sailing down the river.\nB. is done playing, he stands up and puts his hands in the air and shakes his fist.\nC. 's done, he comes back on the screen.\nD. is done he turns off the camera.","prompt":"A man comes onto the screen thanking his viewers for the comments they have made about his videos. Teams to play the harmonica and perform a song for the video. when he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3689,"error":null,"output":"D"}