Approaching the CAT is a little like jumping a high bar. The bar is set at a certain height for the EMT exam, and you need a certain amount of speed and skill to make it over the first time. If you do, the next time you jump, the bar may be set higher.
On the other hand, if you don’t make the first jump, the bar may be adjusted a little lower before you try again.
A CAT exam begins like the first high bar setting. You see a couple of multiple-choice questions, in a content area like EMS operations, that have been judged to be of medium or medium-easy difficulty. Say that you’re Mr. or Ms. Smarty Pants, and you answer these questions correctly.
The computer records the information and sets a more difficult question in front of you. You, being as smart as you are, answer that question correctly too. The computer records that result and adds it to the previous results.
It then continues to give you increasingly difficult questions to answer until statistically you’ve proven you know your stuff in this area. The computer then switches to another content area, say medical and obstetrics/gynecology, and begins the process all over again.
If you correctly and consistently answer a majority of questions that are above a set minimum standard, the test abruptly stops, and you’re done. If you’re really prepared and knowledgeable about all things EMT, you may only have to answer as few as 70 questions for the computer to decide that you’re ready to go.
If this happens to you, you may think, “Wow, I must have passed since the test shut off after 70 questions.” But that’s not necessarily the case.
Imagine the same scenario, but this time, say you miss the first couple questions of medium difficulty. The computer records this, just as before. But now it shows you a question that’s easier. You end up missing that one too. The computer records your result and adds it to the others.
If you continue to miss similarly easy questions, eventually you demonstrate that you don’t know the material. The computer switches to the next content area and repeats the same process, until you’ve demonstrated that you’re not ready to go. This may also take as few as 70 questions.
The result? If your test shuts off after 70 or so questions, you either did really great or really poorly. And to add insult to injury, you may not know how you did. Because the test adapts the difficulty of each question to your level of ability, every question will appear to be difficult. Many EMT students report how uncertain they feel about how they did after an NREMT exam.
The two preceding scenarios represent the extremes. The fact is, most people need to answer somewhere between 70 and 120 questions before the computer decides statistically that they’re either at or above the standard or below it. For example, say you do just fabulous on the EMS operations section, but when it comes to medical and obstetrics/gynecology, you flub a question that’s considered medium in difficulty.
The computer then asks you a similar question at a lower level of difficulty. You answer that question correctly. The computer then asks a more difficult question related to the same topic and continues to do so until you show that you are at or above the standard. Then it moves on to the next section.
As a result of a few ups and downs, you have to answer more questions to show that you have enough understanding compared to someone who has a better level of comprehension. The number of questions you have to answer doesn’t matter though; as long as you meet or exceed the standard in all sections, you pass the exam. That’s what matters, which brings to mind a joke:
What do you call someone who graduates from an EMT program at the bottom of the class?
An EMT.
In other words, passing the NREMT exam means that your understanding of being an EMT is enough for you to be a safe, beginning practitioner.
Educators utilize a concept known as classical test theory to create a paper-based exam. Classical test theory basically says that by asking a few well-constructed and well-chosen questions about a particular topic area, your performance on those questions can predict how well you’d do if you were asked every possible question about the same topic. Writing good test questions and choosing the right ones to ask is challenging.
A computer adaptive test, on the other hand, uses item response theory as a basis for choosing its questions. Item response theory says that how well you do on a question is based upon how good your knowledge base is and how difficult the question is. CAT, combined with item response theory, delivers a much more targeted exam process that’s highly accurate in determining your readiness level.