Quiz and Awareness Questions

The best practices I've seen have involved relating the awareness and quiz questions to content based on the instructional objectives for the material.

The types of questions should be driven by the performance goal of the learner.

A learner that is only required to perform at an awareness level should have less rigid assessment requirements than someone needing advanced level performance.

With that in mind, it should be clear that True/False type of questions are mostly going to be testing awareness of information, where a drag and drop or matching interaction may be designed to test higher levels of understanding.

Someone needing advanced levels of performance would likely need a form of a performance observation tool completed by an independent observer after completing the lower level knowledge checks.

As for how many questions, that will depend on your content and subject matter expert's opinions.

For a recent state level evaluation on Emergency Management topics, our team reviewed the hundreds of slides being taught and grouped the related questions into pools based on the subject matter. Once grouped, the pools were analyzed to determine the number of questions available for each topic.

Our requirements included delivering multiple versions of the exam, so we then determined the size of the pools available should be divided into 4 different exams.

The topic areas that only had a few questions were randomized and a small subset was included: 2 of 3, 4 of 5 or 4 of 6 questions.(The types of questions were also considered so that one exam wouldn't be perceived as 'easier' than another.)

Those with larger pools, we could limit the pool to a smaller number and ensure there was no overlap of questions from exam to exam.

These exams were delivered as paper copies in a live classroom, but the strategies can be applied to Captivate as well.