The Art Of Creating Test Questions Online

How to Create Exam Questions for Online Assessments

If you’re unsure of how to effectively create exam questions for online assessment programs, using a test delivery software platform that offers advanced features is a great place to start. Familiarize yourself with the wide variety of question types available and be sure you understand when and how each type should be used. Design tests to include question randomization, scenarios and sections, and use multimedia elements such as video, photos, and infographics.

If you’re tasked with creating test questions for your organization’s online skills assessment program, it’s important to understand these key things. In this article, we explain the different types of basic and advanced question types, features, and program settings you can use as well as the best practices for when and why to use specific questions, and tips for how to write them more effectively.

 

Multiple Choice

Consisting of a question with at least three or more answers to choose from, multiple choice questions are one of the most commonly used assessment methods. Typically one answer is correct and the other answers are distractors. Distractor answers tend to resonate with candidates who hold misconceptions or thinking and reasoning errors, and help testing managers pinpoint candidates with less ability or understanding.

Multiple choice questions can be used to test most subjects and situations, and can be used to:

• Make associations and draw comparisons
• Identify terms and definitions
• Verify understanding of vocabulary
• Solve problems
• Show recall of concepts and information

It’s best to not use “all of the above” or “none of the above” for your answers, and be sure your answers are independent of one another without overlapping content between questions. Phrase questions and answers without subjective qualifiers such as “best” or “worst”. Also be sure to only include one subject in your question and to phrase and structure questions in a way that does not give away the answers.

 

True or False

These questions are seemingly simple, but when worded effectively, true-or-false questions can be an excellent tool for testing a candidate’s understanding of the subject by assessing their ability to:

• Analyze statements about the learning material
• Understand related terminology and definitions
• Recall detailed information and concepts
• Complete surveys and give feedback

In order for these questions to be effective, the statements must be written so they are 100% true or 100% false. There must not be any room for interpretation or your results will be skewed.

A few things to avoid when writing true-or-false questions:

• Double negatives – saying “she was not unmotivated” is confusing. Instead say “she was motivated”.
• Absolutes – words such as always, never, all, and must state an absolute fact that might not be true. Be sure it’s unarguable factual before using these words.

 

Fill-in-the-Blank

These questions let you objectively assess the candidate’s understanding of the answer, instead of a multiple choice question, which allows them to recognize and choose an answer when they see it. Use fill-in-the-blank questions to test:

• Subject matter terms and definitions
• Problem-solving abilities
• Capacity to recall information, concepts, and principles
• Vocabulary understanding

To write effective fill-in-the-blank questions, be sure your question and answer align grammatically so they don’t unintentionally prompt a candidate to give an incorrect answer. Use ‘a(n) ______’ instead of ‘a ______” or ‘an _______” to eliminate the possibility of guessing. Also, limit the number of blank spaces to a maximum of two per question.

Matching

Use matching test questions to assess knowledge of a large amount of material using a variety of scenarios. These questions are best used to test a candidate’s understanding of:

• Subject matter terms and definitions
• Scenarios and responses
• Causes and effects
• Parts and functions
• Vocabulary

To write effective matching sets questions, be sure to include more answer possibilities than prompts to avoid being able to answer by the process of elimination. Review prompts to ensure they aren’t structured to give away the answer and that there is no overlap between prompts and answers, which can result in multiple correct responses.

Rank-and-Order

These questions ask candidates to place items on a list in ranking order of dates, steps, or importance. They’re used to assess knowledge of:

• Chronological order of when events occurred in time
• Order in which to complete the steps of a task
• Importance of an item

To write effective rank-and-order tests, be sure to chop them into manageable chunks that are focused on a central idea or theme. Always provide context for the items being ranked and provide a key for the number range and what they mean (1 is most important to 10 is least important, 1 is first to 10 is last).

Advanced Features & Settings

Gauge offers more advanced features and settings that allow you to easily design and deliver fair, accessible, secure, and effective tests that automatically turns results into analytic data reports.

Scenarios

These kinds of questions pose a hypothetical situation to test takers and they apply their theoretical knowledge to respond to the situation. Scenarios help you evaluate lesson material mastery by seeing how candidates use their understanding and knowledge in real-world situations.

Scenarios are used to:

• Demonstrate problem-solving abilities
• Assess how learning experiences stick with a candidate
• Provide a safe space to practice and develop skills
• Learn from mistakes through reinforcing feedback
• Offer an alternate question style for learners who struggle with other formats

To write effective scenarios, ensure all of the included content is relevant to the conclusion and that it’s a challenging problem. Use branching scenarios to illustrate consequences of decisions and continue the story. Make sure your language is accurate and relatable to test takers and that settings are in realistic time and place. Use appropriate point of view and details to add to the logic, not to distract, and make sure there’s a clear beginning, middle, and end.

Question Banks

You can maximize security and accessibility by utilizing question banks, which are a pool of test questions that you can add to the test dynamically so no two test takers receive the same exact version. Question banks in Gauge can be configured independently or at the test level and because they exist independent of any one test, you can reuse them to build any new tests you create in the future.

Sections

Tests that include sections utilize reusable question banks but the tests are broken up into individual sections, often by subject matter, and taken in sequence. This is an excellent method of delivering the kinds of high-stakes tests that keep employees certified to perform their job duties and Gauge makes it easy to configure tests into sections.

Multimedia

Advanced platforms like Gauge let you enhance the user experience by including photos, videos, infographics, and even HTML5/Scorm content to both questions and answers.

Randomization

Use Gauge question bank settings to randomize both the questions and the possible answers that will be delivered to candidates who are testing for the same learning content.

Anchor Questions

Configure specific questions to be anchor questions so that they are always shown when you create randomized tests.

Difficulty Ratings

Configure a question’s difficulty setting so it can be included in test results reports on randomized question tests. The question randomization uses the difficulty setting to balance the test questions chosen at random.

Explanations

Set up answers to display an explanation of why the given answer is correct or incorrect.

Proctoring

You can easily subscribe to remote/digital proctoring services from within Gauge to enhance your skills assessment program integrity and security.

Some other things to consider when designing and delivering effective skills assessment tests:

• Design learning modules that use media and progress through material clearly and logically.
• Set expectations with clear instructions inside the test and with external communications to candidates, which you can do directly from within Gauge with messaging and email.
• Provide practice tests along with learning materials or modules.
• Employ diverse subject matter experts to help write and review questions to ensure they are equitable and unbiased.
• Ensure accessibility for differently abled individuals.

As you can see, there are a lot of important details to take into consideration when creating test questions and building tests. Hopefully, this information helps you make the most effective assessments possible so you can collect the data you need to make important decisions for your organization. We’ve built flexible, adaptable, and easy-to-use features into Gauge for nearly thirty years and would love to show you how you can transform your testing program with our test delivery software platform.

---

If your certification program is running on tools that were never built for it — and you’re starting to feel the friction in your daily operations — Gauge was designed for exactly this situation. It handles the full exam lifecycle for professional credentialing programs, from item banking through results and credential issuance, without requiring you to stitch together separate systems.