This is the optimum way to compile a multiple-choice test, according to psychology research

GettyImages-880136168.jpgBy guest blogger Bradley Busch

Let’s start with a quick multiple-choice test about multiple-choice tests: when designing them, should you a) avoid using complex questions, b) have lots of potential answers for each question, c) all of the above or d) none of the above? The correct answer is (a), though as we’ll see, this was not a very well-crafted multiple-choice question. 

The issue of how best to design multiple-choice questions is important since they have been popular in both education and business settings for many years now. This is due to them being quick to administer and easy to mark and grade. Furthermore, many students often report preferring them over other test formats.

As well as being a useful assessment tool, if they are well-designed they can also aid learning. This is because of the Testing Effect – the way that retrieving knowledge helps consolidate it in memory. 

Thankfully in a recent paper in Journal of Applied Research in Memory and Cognition Andrew Butler of Washington University in St. Louis has reviewed the parallel literatures on how best to design multiple-choice tests for learning and assessment, and from this he’s recommended six evidence-based tips:  

  • Avoid using complex questions or answer formats. When things are too complex, it leads to misunderstandings and students guessing the answer. By keeping the format simple, it makes it a more reliable assessment of what the students do or don’t know. 
  • Create questions that focus on specific parts of knowledge or thought processes that you want to assess. For example, questions could focus on retrieval of a specific fact, contrasting two concepts, or applying a theory to a new situation.
  • Avoid using “None of the Above” and/or “All of the above” as potential answers. The main issue with “None of the Above” is that if it is the correct answer, then the test-taker has been exposed to numerous false answers. This therefore represents a missed opportunity to reinforce what the correct answer was. “All of the Above” may be helpful when it is the correct answer because, as Butler explains, “the test taker is only exposed to the correct information”, but may do more harm than good if it is the incorrect answer because of exposure to false answers.
  • Be economical with the number of answer options. The topic will have a bearing on the optimal number to offer, but three potential answers usually provides enough difficulty whilst also being time efficient. Excessive potential answers may be problematic if the main aim is to enhance learning, as it will “expose test takers to a lot of incorrect information, [which] is worrisome because they could potentially learn it”.
  • Ensure the answer choice is “moderately difficult”. If it is too easy or too hard then little will be learnt. Butler states that “the ideal difficulty level is a bit higher than the midpoint between chance and perfect performance”.
  • Ensure you provide feedback on the correct answers. When giving feedback to students, Butler notes that “one important caveat is that the effectiveness of feedback depends upon students being motivated to process it and their motivation tends to decrease over time”. Therefore, if feedback is delayed, it would be wise to take steps to ensure that students are “required or incentivized to process it”.

This paper has useful implications for anyone designing multiple-choice tests. Butler notes that more research is still needed, however. For example, one area that shows promise is having students assign a confidence rating to their answer, as this helps measure partial knowledge. However, the exact format for doing this is still being studied.

Multiple-Choice Testing in Education: Are the Best Practices for Assessment Also Good for Learning?

Post written by Bradley Busch (@Inner_Drive) for the BPS Research Digest. Bradley is a registered psychologist and director of InnerDrive. He has worked with Premiership and International footballers and is the author of Release Your InnerDrive

4 thoughts on “This is the optimum way to compile a multiple-choice test, according to psychology research”

  1. I’d disagree with ‘Ensure the answer choice is “moderately difficult”’ – a well-designed test has to be able to discriminate across the board. Not only should weaker testees (or less prepared ones, anyway) get low grades, and better testees get good grades, but in order to discriminate between, say, A- and B-grade students, you need some “very difficult” questions too, and to discriminate between D- and F-students, you need some “pretty easy” ones as well.

    I use MCQ test on my first year research methods module and have done for years, and have learned also to manage expectations – let students know that there are going to be very difficult questions that hardly anyone will get right, otherwise it can lead to despair!

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s