Took my last practice test yesterday and I scored 6 points higher than my previous test, and 18 points since my first test in early May. I’ve been slowly and steadily improving. They say that these practice tests (“USMLEWorld Simulated Assessments”) do a reasonable job of approximating the quality and difficulty of the actual test, and I hope so. I chose to take this test instead of the NBME because it is more difficult – meaning that it does a better job of discriminating in the +1 to + 2 SD range. Here’s what I mean by that …
One of the flaws of the NBME practice exams (and hopefully, NOT of Step 1) is that they are geared at determining whether you will pass Step 1 and not whether you will get a +1 vs +2 SD score (a.k.a “245” vs. “265”). I say this because the % correct on the NBME practice tests that is converted to a +1SD test result is about 90%, leaving only 10% of the questions to determine the difference between +1SD, +2SD, +3SD etc. The “mean” score is about 80% correct. The test therefore “spends” 80% of its questions below the mean, trying to discriminate between -3SD (fail) vs. -2SD (marginal pass) vs. -1SD. Which is, in the end, what the NBME wants in a licensing exam. The NBME want to be very sure that the folks who pass the test are really above the threshold … and if the folks at the top end have more variability in their scores, then that’s fine by the NBME. Not so fine from the perspective of folks who lose sleep over getting into competitive residencies … but c’est la vie.
Unlike the NBME, the UWSA sets the difficulty of test so that the “mean” score (+0 SD, “225”) equates to 70% correct. This gives them 30% to discriminate on the upper end, instead of 20%. I think this makes the UWSA a slightly better assessment, assuming that their conversion from %raw to Step 1 equivalent score is accurate. It very well may not be. There are myriad forums in the cesspools of the medical student internet (you know who you are …) that discuss this topic using self-selected, self-reported n=1 anecdotes as though they were data.
Of course, one of the major confounds of the UWSA is that their questions are VERY similar to the UW question bank questions, which I have completed twice (well, technically 1.8 times by now, but I’ll finish it off this week). Accordingly, I’m aware that the UWSA may be over-estimating my score and I take it with a grain of salt. All I know is that I am slowly and steadily improving, and I have the potential to score in the top 10% (+1.3 SD). Nothing is guaranteed.
As an aside, for those of you who don’t know how USMLE Step 1 scores are reported, the “three digit” score ranges from 140 – 310 (see why, below). The national mean for the three digit score is ~225 with an SD of ~22, normally distributed. Passing is usually ~185 (roughly -2SD). The average step 1 score for an applicant matching in the most competitive specialities (say, plastic surgery) is ~245 (roughly +1SD). Since there are ~20,000 test takers per year, there should be few +4SD scores and -4SD scores (~ 1 per year). Thus, the range is roughly from +4SD to -4SD, which equates to a range of 140 to 310.
There used to be a “two digit score” but it has been eliminated since it was meaningless.