Bryan Carmody

Beyond that quote, when I read the article, what struck me was this: The CEOs of the NBME and FSMB expressed numerous concerns about adverse effects of making USMLE Step 1 pass/fail... ...but loss of revenue to the @NBMEnow was not mentioned.

I’m going to share some data on the NBME’s finances, and let you draw your own conclusions about whether the evolution and current financial structure of the NBME might lead to #COI. But let’s begin somewhere else. Why does the NBME exist in the first place?

The NBME was founded in 1916 in large part to address an annoying problem for physicians. Since medical licensure is regulated at the state level, a physician had to take a separate board examination in each and every state in which he intended to practice.

So that’s pretty cool, right? Thanks to the NBME, all MDs can take the same test and then seek licensure wherever they want. (Compare that to attorneys, where taking a new state bar exam is a significant barrier to movement and may lead to workforce inefficiencies.)

Unlike many nonprofits, the NBME neither solicits nor accepts charitable donations. They derive a small amount of income (~3%) from investments. But the overwhelming majority of their $ comes from their programs (i.e., the USMLE, which accounts for ~75% of their revenue).

For those of you wondering, this increase is NOT explained by inflation. Using the Bureau of Labor Statistics consumer price index, $47.5 million dollars in 2001 would be equivalent to $65.6 million in 2017.

To put this in some perspective, consider this: From the start of 2001 to the close of 2017, the Dow Jones Industrial Average increased from 10,646 to 24,719 - a factor of 2.3. NBME program revenues increased by a factor of 3.24.

So what led to such a steep increase in revenue? Two things: increased cost for the USMLE, and new revenue streams from new USMLE programs.

As far as the cost of USMLE over time: here’s where I need some help from #medtwitter. The data I can find are spotty, and I don’t have the cost for every year. If someone can help me, I will put together a chart.

As far as new revenue streams, the NBME has created several. The increase in program revenue around 2004-2005 was from the introduction of a new test: the USMLE Step 2 Clinical Skills (CS) exam.

Step 2 CS is the most expensive product in the USMLE line - $1285 for U.S. students in 2018. The cost to students is large - but because 96% of U.S. first-time test takers pass, the discriminatory value to licensing agencies is minimal. https://www.ncbi.nlm.nih.gov/pubmed/23465102 

Much, much more about the (lack of) value of Step 2 CS has been discussed previously. If you missed it, I’ll leave it to the rest of the #meded community to fill you in.

In the background throughout the time period shown was the steadily-building crescendo of Step 1 mania that I discussed yesterday. The NBME is not primarily responsible for that - but they have undoubtedly benefitted financially by selling new products.

For example, in 2007, they began offering Customized Assessment Services (CAS), a program that allows medical schools to create examinations using “cold” NBME test items. By 2017, 83 US medical schools were using CAS, and the number of exams administered had doubled since 2013.

They also sold more “shelf exams” - the subject tests used at the end of clinical clerkships. By the way, all of the medical students at my school pay for the shelf exams and CAS - the costs are just less transparent since the dollar amounts are rolled into the tuition bill.

The NBME has also increased revenue from its Self Assessment Services. Here, students can purchase NBME question blocks at $60 apiece to help them prepare for the mandatory NBME tests later on.

And what does the NBME do with all of this revenue? It’s time to discuss salaries.

Dr. Melnick’s compensation rose along the rising financial tides at the NBME, increasing from $399,160 in 2001 to over $1.2 million in 2015, almost perfectly in parallel with the tripling of program revenue over the same time frame.

The NBME president’s salary is on par with what other CEOs make. But it is substantially higher than what almost any practicing physician makes in a year - especially early career MDs, from whom the bulk of those dollars were derived.

So why am I pointing all this out? Accountability, for one thing. All nonprofit, tax-exempt organizations deserve some scrutiny to be sure that their dollars align with the organization’s mission and stakeholder values. That’s the whole reason the tax forms are public.

Second, the NBME operates in a strange space where normal economic forces don’t have as much gravity. Paying “customers” (i.e., students) have little choice about whether or not to buy their “product” (the USMLE), at whatever price the NBME chooses to sell it.