Beware of Errors on Test or Demo EMR System

I’m sure that many of my readers have experienced the awkwardness of an error happening during the demo or training of an EMR system. I’ve been on both sides of the fence (watching or doing a demo) and let’s just say it’s really uncomfortable for both sides. Those that have experienced it know that the most common explanation for the error is “This is the demo system and so we haven’t finished setting everything up.” Or in the case of the training system, “This is the training system and so with all of the people training on this system it has some errors from those training on it.”

In some cases, this is completely true. When I’m training my staff for an update to our EMR software, there has been a number of occasions where I was just too lazy to set something up on our test database and it doesn’t work quite right. So, it does happen.

The difference between myself (most of the time) and those demoing and training you on an EMR system is that I’ll make note of the problem and make sure that indeed it was something I could easily fix. If I can’t, then I escalate it to our EMR vendor for resolution before we proceed with the upgrade. Those showing you the demo or training you might do the same. However, if you’re training on the system, there’s little chance the fixes they request will be implemented before you implement the system.

Even more to the point is that far too often it’s not something to do with the test or demo system, but is often an error in the program itself. It’s a good idea to evaluate the error you saw. This can be a real challenge since the trainer is often going to blow by the error as quickly as possible. However, don’t be afraid to call them out on the error. This is going to be the heart of your practice. Make sure you really know if those errors were temporary or chronic. Nothing’s more of a pain than regular errors from your EMR software.

About the author

John Lynn

John Lynn

John Lynn is the Founder of the, a network of leading Healthcare IT resources. The flagship blog, Healthcare IT Today, contains over 13,000 articles with over half of the articles written by John. These EMR and Healthcare IT related articles have been viewed over 20 million times.

John manages Healthcare IT Central, the leading career Health IT job board. He also organizes the first of its kind conference and community focused on healthcare marketing, Healthcare and IT Marketing Conference, and a healthcare IT conference,, focused on practical healthcare IT innovation. John is an advisor to multiple healthcare IT companies. John is highly involved in social media, and in addition to his blogs can be found on Twitter: @techguy.


  • Having experience with many vendors on both sides of the fence, I would say that this is not quite the issue that may be implied here. Reason being, most vendors are going to being showing you their “latest and greatest” alpha code (that may not be live anywhere) to give you a feel for where they are heading. Frankly, functionality is what sells in a demo.

    A site visit and/or customer references are the best place to assess reliability and stability of production releases. Having been involved in over 100 implementations of various products, I have never experienced a demo issue that was indicative of a bigger problem that wasn’t flushed out (better) in another part of the process.

    If you are genuinely nervous about this, specifically ask for a demo of a system that is in production, on a production quality server…eg, no laptop demos.

  • I’m surprised you don’t think this is a potential problem. I have seen this happen a number of times. Which is why I wrote about it.

    Like you said, the best path is to see a demo of the system in an actual patient setting. Hopefully someone you know and not just a referral from the EHR company. Post a request on here or some other online forum for EMRs and you’re bound to find another user.

  • It is certainly something to be aware of, but I haven’t seen this as a problem per se, again because a demo isn’t the right place to assess stability for a lot of reasons. First off, sales guys (gals) in most cases demo from laptops. Laptops don’t generally run the server operating system or an enterprise version of the database. They are also forced to be both client and server, which is not how the system is used in production. Now, if every other screen “crashes”, that’s a bit of a different story.

    Here’s what buyers need to do. Watch it work at a site visit. Talk to users. Ask for a complete client list, not just a list of references. The demo is the place to find out where the vendor is heading with the first production release anticipated at your go live. In this world of hundreds of vendors prepping for CCHIT, potential buyers should expect to see a lot of “raw” code.

    So, I guess you’re right I don’t really see this as a problem. Buyers need to understand how to get the most out of various stages of the decision making process…from RFP to demo to site visit to final selection. I wouldn’t let a demo error by itself be a significant issue, provided I did my other homework.

  • It’s not just the demo systems that we’re talking about, but also the training systems where these errors occur. Those training systems should be their production equipment and the most stable release of their software.

    I agree that a demo error itself shouldn’t nix the selection of a product, but the error is worthy of further consideration. Too many red flags like this and you should proceed to another EMR. There’s plenty to choose from!

  • Not to split hairs, but if we’re talking about a training system, then the purchase decision likely has already been made…..too late to proceed to another EMR unless you want to get lawyers involved. That said, I get your general gist. I just think there’s a reality at play here that needs to be acknowledged. “New code” will be presented in the early stages of the game, and I think that’s what we want. If the vendor can’t get alpha code to stable code, that will come out in other parts of the process.

  • It’s a good point. Although, you can also consider delaying your implementation for the next release if you find out those errors are fixed in the next version. Not an easy choice, but good to have the choice.

    Also, you’re kind of assuming the old model (and still very popular) of paying for an EMR. Basically, that they’ve already sunk tens or hundreds of thousands of dollars on an EMR. If all you’ve paid is your first month for an EMR or some small payment like that, then it’s much easier to stop if you see too many red flags like errors during the demos. Still a difficult choice since you’ll have already sunk training time in, but those are sunk costs that can’t be recovered. Best to make sure you know the choice you’re making if you’re choosing that EMR.

  • I find that the physicians’/practices’ choice of payment models has depended on two things:

    1. the cycle of popularity of ASP/datacenter models (sometimes they’re up, sometimes they’re down)
    2. time of year/tax implications – lots more capital purchases seem to happen at the end of the year

    These two things aside, provided the effective interest rate is right, I’d personally let the vendor carry the note.

    When it comes to terminating a software agreement, the legal fight is the same in either model, as arbiters and state laws usually consider them as financed debt. (Of course, this depends on precisely how the contract is written.) In the end, you have more leverage if you haven’t paid much in to date.

    When it comes to software agreement termination processes (at least those cases that can’t be resolved without lawyers), I find two trends:

    1. They’re usually not related to software performance. Instead that have to do with pure lack of payment, in many cases because of an acquisition where new owners are bringing in their technology and they simply don’t want to pay for two systems.
    2. The vendors usually win, at least on paper. (Actually the lawyers usually win the most, so it is best to not have to take it that far.)

    Bottom line, when you sign on that bottom line, you generally are already in a marriage whether or not money has exchanged hands… this statement has never been more true:

    “Best to make sure you know the choice you’re making if you’re choosing that EMR.”

  • “Best to make sure you know the choice you’re making if you’re choosing that EMR.”

    That line can’t be repeated enough.

    However, you’re still missing some models which are just pay as you go with no contractual obligation to keep paying. Getting out of this is easy. You just stop paying and there’s no legal recourse since there’s no contractual obligation. You’re only out what you actually used.

  • I’m sure those models exist, but they aren’t as common with the more established vendors, especially those CCHIT certified. With the playing field more narrow (with CCHIT), I wonder how long those models might need to exist.

    Of course, when it comes to getting out with any of them, “easy” is a relative term, especially when you begin to get into issues of getting your data back out, which part of the data/structure the customer is privy to (eg, can they actually get the db back up to do their own extractions or is the data structure itself intellectual property), cooperation with subsequent vendors, etc. If it sounds cynical, well that’s what the school of hard knocks will do to you!

  • Maybe those models will be the reason they can still exist in spite of CCHIT.

    No doubt getting data out of any EHR is a lot of work. At least until we get some standards of interoperability. I won’t be holding my breathe for that one.

  • I see a lot of recylced data in test systems. The same fake patient records are used repeatedly with only small modifications which eventually causes rejections. This can be misunderstood as a system problem when it is only a data problem.

    Don’t be afraid to make some new test patients (it’s free!)

  • Thanks for the comment squarie. I do think some people are scared to break something on a test system. The point of testing it is in order to break the system. Better to break the test one than to break your production one and then ask the questions.

  • The above solutions suggested working with a small set of manually created test data, or using “real patient data” which most definitely is HIPAA / HITECH risk.

    Instead, how about testing with fully synthetic medical test data that is not based upon de-identified or masked data? Large scale data sets, with anomaly insertion to replicate human error, and with a “Truth File” to serve as a guide what your test results should be..

    Melanie Hiranandani
    Client Executive
    ExactData, LLC

Click here to post a comment