Clinical Informatics Exam Post-Mortem

I rarely break from literature review in my blog posts (although, I used to make the occasional post about Scotch).  However, there are probably enough folks out there in academia planning on taking this examination, or considering an Emergency Medicine Clinical Informatics fellowship – like the ones at Mt. Sinai, BIDMC, and Arizona – to make this diversion of passing interest to a few.

Today is the final day of the 2015 testing window, so everyone taking the test this year has already sat for it or is suffering through it at this moment.  Of course, I’m not going to reveal any specific questions, or talk about a special topic to review (hint, hint), but more my general impressions of the test – as someone who has taken a lot of tests.

The day started out well, as the Pearson Vue registration clerk made a nice comment that I’d gone bald since my picture at my last encounter with them, presumably for USMLE Step 3.  After divesting myself of Twitter-enabled devices, the standard computer-based multiple-choice testing commenced.

First of all, for those who aren’t aware, this is only the second time the American Board of Preventive Medicine has administered the Clinical Informatics board examination.  Furthermore, there are few – probably zero – clinicians currently taking this examination who have completed an ACGME Clinical Informatics fellowship.  They simply don’t exist.  Thus, there is a bit of a perfect storm in which none of us have undergone a specific training curriculum preparing us for this test, plus minimal hearsay/experience from folks who have taken the test, plus a test which is essentially still experimental.

Also, the majority (>90%) of folks taking the test use one of AMIA’s review courses – either the in-person session or the online course and assessment.  These courses step through the core content areas describe for the subspecialty of Clinical Informatics, and, in theory, review the necessary material to obtain a passing score.  After all, presumably, the folks at AMIA designed the subspecialty and wrote most of the questions – they ought to know how to prep for it, right?

Except, as you progress through the computer-based examination, you find the board review course has given you an apparently uselessly superficial overview of many topics.  Most of us taking the examination today, I assume, are current or former clinicians, with some sort of computer science background, and are part-time researchers in a subset of clinical informatics.  This sort of experience gets you about half the questions on the exam in the bag.  Then, about a quarter of the course – if you know every detail of what’s presented in the review course regarding certification organizations, standards terminologies, process change, and leadership – that’s another 50 out of 200 questions you can safely answer.  But, you will need to really have pointlessly memorized a pile of acronyms and their various relationships to get there.  Indeed, the use of acronyms is pervasive enough it’s almost as though their intention is more to weed out those who don’t know some secret handshake of Clinical Informatics, rather than truly assess your domain expertise.

The last quarter of the exam?  The ABPM study guide for the examination states 40% of the exam covers “Health Information Systems” and 20% covers “Leading and Managing Change”.  And, nearly every question I was trying to make useful guesses towards came from those two areas – and covered details either absent from or addressed in some passing vagueness in the AMIA study course.  And, probably as some consequence of this being one of the first administrations of this test, I wasn’t particularly impressed the questions – which were heavy on specific recall, and not hardly on application of knowledge or problem solving.  I’m not sure exactly what resources I’d use to study prior to retaking if I failed, but most of the difference would come down to just rote memorization.

However, because the pass rate was 92% last year, and nearly everyone taking the test used the AMIA course, an average examinee with the average preparation ought yet to be in good shape.  So, presumably, despite my distasteful experience overall – one likely shared by many – we’ll all receive passing scores.

Check back mid-December for the exciting conclusion to this tale.

Update (as noted in comments below):  Passed!

Hopefully future editions of prep courses will gradually attune themselves to the board content, once a few iterations have progressed.  Individuals taking this exam, in the meantime, will need to rely heavily on their medical or prior technical experience, particularly as the curricula for fellowships are fleshed out.  Additionally, the CI exam content is so broad, fellowship trainees will need to specifically target their coursework to areas they lack – for, example, “Leading and Managing Change” as a major content area of the examination will definitely force many informaticians into a knowledge gap.

Interesting times!

12 thoughts on “Clinical Informatics Exam Post-Mortem”

  1. Congratulations Ryan! I was part of the inaugural class, and it's an honor to have you join the ranks of emergency physician/informaticists – it's a great group!

  2. I absolutely agree w/your summary of the experience. Thanks for sharing. I am still a bit perplexed as to why many of those questions were so esoteric…

  3. Thanks for posting this.

    I guess as I move through the Kubler-Ross stages after taking the exam (hey, that was in the curriculum) I just have to think – what a wasted opportunity. These folks are defining a new specialty that has an interesting history and theoretical underpinnings. There are big lessons to be learned.

    Instead of focusing the curriculum & exam on that, they put too much focus on alphabet soup of agencies and standards, many of which will be obsolete in a few years.

  4. Am curious if folks with formal training in the discipline found it easier than those of us with OTJ training (and extensive studying/online course). Based on the way the test was…I'd think even those with a degree in the field found it to be…frustrating.

  5. I had both formal training – an M.S. in Health Informatics – as well as a cognitive science-type undergraduate degree, applied informatics research experience, and serve on QI and safety committees as my institution. And I did the online course + the 150 question assessment. And I still felt like about 25% of the questions were just plain trivia.

    Sounds like everyone felt the same way after the first examination – and 92% passed ….

  6. Results are available on the website, and score report e-mails should have begun going out.

    Congratulations to everyone else who passed! It appears the pass rate was a similar ~91% this year.

    I would have liked to have seen my raw score, rather than just a scaled score, to truly understand the context of whether the questions were as discordant with our preparation as it seemed while taking the exam. It seems, from the distribution of scaled scores, anyway, most folks must have had generally similar shared experience in preparation and ultimate outcomes.

    Now we have to do a second specialty worth of MOC. : |

  7. I don't believe so – I don't think any of the board examinations are such.

    I have taken the GRE, which is computer-adaptive. You can definitely notice the questions ramp up in complexity during that exam.

Comments are closed.