Joe and I met when we were both speaking at a Dyslexia Conference. Joe was demonstrating this tool that had been developed to measure student performance and behavior in a very rich and intensive fashion. I’m sure he thought I was a lunatic because I rushed the podium right after his talk and explained that I was convinced this was something we could use as a way of measuring Residents’ progress to their Milestones, which is something that I was very involved in the development of. I was on the advisory committee of the child neurology and neurodevelopment disabilities milestones for the ACGME. So we had this great thing, 37 fields of data on 19 Residents measured in some way, and I think everyone was bewildered on how we were going collect this data in an effective fashion. When I saw what [Joe] had, I knew we could adapt his technology with Milestones. And, he turned out with this terrific tool that in the end has been incredibly easy to use and incredibly rich in the data we collect.
So, I was pretty convinced that using this we could capture information much more in the moment, we could use it to give feedback to Residents to say, “You’re doing well here and here, but not as well here.”
How is the database working with MedInsight, your residents, and with your positions in general?
First of all, now that it’s live and running, it is fantastic. In any old system, what tended to happen at the end of the month-long rotation was people would hand me a piece of paper or direct me to websites, usually 3 weeks afterwards. Upon reviewing their information I’d think, “How’d that guy do? I can’t really tell.” We tended to get really pastel data. Everybody was always like Lake Wobegon; just wonderful and above average.
It also gives me a real sense of how many data points I’m accumulating. So, I can drill down to a single observation by a single attendee and that is not only captured and available, but it is incredibly and intuitively obvious when you’re just starting to look at the graphic displays.
The other thing that is great about it is the fact that even the most technologically adverse members of our faculty have absolutely embraced this thing and think its just great. [Attendees] can open [MedInsight] on their iPad and walk around. At the end of the day, they pick out what they’ve seen. They “tap” out where they feel someone is on the set of metrics we have, and then they’re required to actually have some text input about what [this observation] was about. Plus, since they’re on an iPad, they can use the voice-to-text feature to save even more time.
The amount of data that I’m getting now compared to our old system is just orders of magnitudes larger. I’m going to be able to sit down in my semi-annual meetings, or quarterly meetings – depending upon how the Resident is doing, and what I have to say is much more robust. Not based on hearsay or the last thing anybody saw or the last thing anybody knows, but based on real data. It allows me to say “Look, we don’t have to worry about this anymore; on the basis of this I can quickly tell that you’ve actually hit the Milestone we want. You’ve hit that metric and now we can stop worrying about that. We’ll come back and check on it every now and again, but really, what we need to work on is that you’ve actually not shown that proficiency in neuromuscular disease and demodulating disorders – or other parts of the curriculum.”
Therefore, it allows me to design out-patient rotations more intelligently. And say, “Look, you obviously need more experience here. We’re going to make you heavy on – fill in the blank.”
[MedInsight] allows me to individualize a Residents’ education based on actual information that we have as to how they are performing and what our goals for them are. That, in turn, allows for a much more individualized program that responds to their needs. On top of which, I feel is much more responsible to the public to whom I’m ultimately accountable.
I think that they’re equally excited about the data because they get feedback thats more than the white bread stuff they were getting before which was basically just a slap on the back. This allows us to be much more granular in terms and that’s what I find in a program like ours is what residents actually really want from us. They really want to know how they’re doing in a real way, and not just being a “great job.”
When you’re finishing here, you aren’t done because I said you’re done, but actually because of information that in some sense I can be transparent about and say that yes, people are actually that are graduating from this program. I can say with a high degree of numeric-level confidence that they have in fact achieved the goals we have set out for them, and that makes me feel a lot better.