How are exams scored?
There is a science behind how the ABD scores exams. Psychometrics helps ensure that the results are fair and accurate. It’s far more complicated than tallying up the number of correct answers.
Once a candidate completes an exam, a psychometrician analyzes and validates the results. Subject matter experts (certified dermatologists who volunteer to assess results) determine a fair passing standard. Quality control methods are employed to assure accuracy of scoring, especially for candidates who don’t pass.
Did we mention that scoring certifying exams is a complicated process? However, it is an important process that the ABD wants all candidates to understand. Below is a list of definitions and explanations to help clarify how exams are scored.
If you still have questions, please contact us.
PSYCHOMETRICIAN
A psychometrician is a professional who practices the science of measurement, or psychometrics. After an ABD exam is administered, the scoring data (including all candidate exam responses and the corresponding answer keys) are sent to a firm that specializes in measurement and analysis and employs a staff of credentialed psychometricians.
ANSWER KEY VALIDATION
After an exam is administered, the psychometrician reviews and analyzes the way questions were answered. The psychometrician looks for questions with a high number of incorrect responses or questions with answer patterns that suggest additional correct options beyond the designated answer key(s). The ABD then evaluates these items to determine whether the designated correct answer really is correct and that none of the other options could also be considered correct. At this point, questions deemed unfair are eliminated from scoring and questions with additional correct options are updated to give credit for these options.
STANDARD SETTING
To establish a fair passing standard for an exam, the ABD conducts a standard setting exercise by what’s known as the Modified Angoff Method. This is a customary process for certification exams that involves an item-by-item review by a panel of subject matter experts. The passing score is not set using a curve, but rather by what a specialist needs to know based on the consensus of the panel of experts. As a result, it is completely possible for everyone who takes the exam to pass it, because there is not a predetermined percentage of candidates who must fail.
QUALITY CONTROL
To ensure the accuracy of exam results, the psychometrics firm follows strict quality control measures. For internal quality assurance, after a psychometrician calculates the results, another psychometrician at the firm performs an independent calculation. The two results are compared to ensure that the pass/fail status, and individual scores, match for every candidate. To further validate the accuracy, the psychometrician then performs a manual review of all exams with failing scores. This entails:
1. Manually comparing each response to the answer key
2. Calculating the number of correct responses
3. Comparing the number of correct responses to the system-generated number of correct responses
4. Converting the number of correct responses into an exam score
5. Confirming the exam score fell below the established passing score
As a final step towards accuracy assurance, the ABD then engages an additional, outside psychometrician (not employed by the primary psychometrics firm) to review exam data and confirm the pass/fail status of each candidate. This rigorous review of exam data assures the ABD, and our candidates, that the results are accurate.*
*This additional step is only conducted for the APPLIED, MDS, Pediatric Dermatology and the February Administration of the CORE Exam for DRY3 candidates.
