EMR and EHR Ratings Confusion

I recently got a comment from Tammie on my post about EMR and EHR rating websites. Here last line really hits home:

If we cannot trust the reliability of web published EMR/EHR ratings, then are we not to trust the surveys and polls conducted by the professional organizations or the selections by the Regional Exchange Centers either? I can’t recall ever researching something so thoroughly and have so much information available to me and yet be still so utterly confused.

It’s a definite challenge. I’d suggest going through a methodical process that’s described in my free EMR Selection e-Book. Still a challenge, but this approach helps.

About the author

John Lynn

John Lynn

John Lynn is the Founder of the HealthcareScene.com, a network of leading Healthcare IT resources. The flagship blog, Healthcare IT Today, contains over 13,000 articles with over half of the articles written by John. These EMR and Healthcare IT related articles have been viewed over 20 million times.

John manages Healthcare IT Central, the leading career Health IT job board. He also organizes the first of its kind conference and community focused on healthcare marketing, Healthcare and IT Marketing Conference, and a healthcare IT conference, EXPO.health, focused on practical healthcare IT innovation. John is an advisor to multiple healthcare IT companies. John is highly involved in social media, and in addition to his blogs can be found on Twitter: @techguy.


  • One sure way to get physicians comfortable that they are making the right purchase decisions is to provide them with competitive benchmarking, vendor report cards and the protection of an EMR Lemon Law. I outlined this plan last year in a blog posting: http://blog.srssoft.com/2009/08/emr-reform-a-plan-to-spur-adoption/.

    The newly released KLAS Performance Report by Specialty is a step in the right direction, but absent EMR Reform, physicians are on their own to perform comprehensive due diligence to ferret out complete and unbiased information.

  • Evan,
    I like many of the suggestions in your blog post. Although, I think it’s funny that you mention KLAS. I think they’re ratings contribute nothing to resolve the confusion. There are so many issues of their ratings.

    Of course, it makes sense that you’d like the KLAS ratings since SRSsoft was ranked highly in those ratings. Although, I do have to ask if you paid KLAS to be rated by them.

    EMR vendors paying the rating companies is enough that really needs to be said about KLAS ratings.

  • KLAS is one key source of data in an industry sorely lacking objective product evaluations. It should be noted that KLAS keeps working on improving their reports as evidenced by their most recent specialty ratings.

    As for vendor support, vendors pay KLAS based on their size and you will notice that some of the larger vendors have some of the lower ratings, so I don’t think you can conclude bias based on funding.

  • Thus all the confusion and desire for information since KLAS is the best of the worst that’s available (and there’s only worst).

    Bias based on funding doesn’t really need to be shown. It’s just inherent in the concept.

    The real problem with KLAS ratings is the lack of data points and how easily those data points can be biased. In many cases they’re doing ratings with only a few data points PROBLEM. In other cases, EMR vendors have sent emails to their users asking them to rate them highly. I’m sure there are other cases where EMR vendors have done much more.

    No doubt it’s a hard nut to crack. If it was easy or if there was a good solution I’d have done it already.

    I will admit that lists are fun to look at and I’m ok with people using them as one data point as long as they’re well informed of the validity or potential lack thereof.

  • Only things more meaningless than some debateble ‘Consumer Reports’ rating would be its ATCB MU certification and selection to your REC’s short list of preferred providers.

    Third party ‘stampers’ aren’t responsible for delivering health care or delivering a healthy bottom line. Those who look for a KLAS-like scoring entity are the same folks who read product reviews for the car they just bought and are only looking to make themselves feel good.

  • One way to compare EHR vendor is to see if they have a legitimate value prop over another. Many vendors only talk about themselves and never really explain why they are the right choice. Practice Fusion, a free, web-based
    EHR, has a section on their site that compares other vendor programs with their own, clearly drawing the line on why they are superior. I think when looking for a vendor one needs to look at price and quality. Legacy vendors not only charge $20-40,000 per server but they then charge for support, maintenance and upgrades. With web-based solutions there is no need for those types of fees because once the system upgrades itself, all users will have the upgraded version. Quality is also very crucial when selecting a EHR. Practice Fusion has 100% free 5-days a week support. They do not outsource (unlike some competitors) so they are even able to send consultants to practices for in-person help.

  • I’ll modify my stance on validity of 3rd party assessments. I’m actually more comfortable with subjective customer feedback than an objective scoring process that does not roll up to accurately capture the best overall products.

    What got me thinking differently was this blog post in this morning’s in HealthcareCIO:


    You’ve Shared Your Input, Now What?
    Posted by Adam Gale on November 5th, 2010

    Do you ever wonder what happens to the candid feedback you share once you finish your KLAS evaluation? What in the world do the vendors actually do with your anonymous critique?

    The obvious answer is that those vendors who score well love to tell the world about it, which makes sense. Who wouldn’t want to shout out the fact that their customers rank them #1. But what about the majority of solutions that don’t rank “Best in KLAS”? What happens when you vote them #5, or worse yet, the unofficial worst in KLAS? Many of the vendors take your aggregated input seriously and try to figure out how to do better the next time. Like you, they are trying to be good at what they do. However, when someone is being evaluated the input isn’t always easy to take. As one vendor executive put it, “Sometimes KLAS research is like a punch in the face. But…that wakeup call is just what we need to keep improving.”

    The other alternative, if a vendor doesn’t score well, is to fight the process. The reality is that what we do is not that complex. We talk with you as a healthcare provider to get your input on your vendors, aggregate the scores, and then publish the results. The same rules apply to all vendors. Unlike Consumer Reports, we don’t actually take the product out for a test drive or give our own opinions. We learn from those of you that have had experience driving these products for years. In that sense we are more like J.D. Powers and Associates, in collecting, aggregating and publishing the satisfaction data provided by you, the customer. See more about The KLAS Difference.

    What do you typically hear from those that don’t like the results? Probably that we talk to the wrong people (I’ve even heard the thought that we talk with the parking attendants) or that vendors pay for a KLAS subscription to see a higher score. Though it may be fun to talk with parking attendants, who typically have some good stories, they’re not a good source for input on healthcare technology solutions. The reality is that we talk with the executives, directors and managers using these solutions, including you. As for the “pay to play” theory, even small firms can outperform larger enterprise vendors. For example, Unibased Systems Architecture (a small yet high performing vendor in Chesterfield, MO) was “Best in KLAS” in 2009 for Enterprise Scheduling.

    Thank you for your continued participation with KLAS, and just know that your input is likely being read by your vendor. This is, after all, your voice they are listening to.

  • I actually think that KLAS is one of the more sincere companies when it comes to trying to rate and rank EMR vendors. I really think they’re trying to do ratings while still having a business model. My biggest problem with KLAS is that they just don’t have the volume of doctor ratings that would make the results valuable. You can certainly find bad and good reviews for every EMR vendor out there. Especially ones that have a pretty large install base. So, without any volume you’re not giving users a real benefit. As a research company you’d think they’d understand principles of sample size and random sampling and non-response better than they seem to portray in their results.

    Plus, as I’ve said other places, the number rating isn’t nearly as valuable as a qualitative assessment of the EMR.

Click here to post a comment