Where Medical Devices Fall Short: Can More Testing Help? (Part 2 of 2)

As we saw in the previous article, networks of medical devices suffer from many problems intrinsic to the use of wireless technology. But testimony at the joint workshop held by the FCC and FDA on March 31 revealed that problems with the devices themselves run deep. One speaker reported uncovering departures from the standards for transmitting information, which led to incompatibilities and failures. Another speaker found repeated violations of security standards. As a trivial example, many still use the insecure and long-deprecated WEP authentication instead of WPA.

Most devices incorporate generic radio transmitters from third parties, just as refrigerators use replaceable compressors and lawnmowers depend on engines from just a few manufacturers. When markets become commoditized in this way, one would expect reliability. Whatever problems radio transmitters may have, though, are compounded by the software layered on top. Each device needs unique software that can affect the transmissions.

The WiFi Alliance is a consortium of manufacturers that tests devices for reliability and interoperability. But because it doesn’t contain users or government representatives, some panelists thought it was too lenient toward manufacturers. The test plan itself is a trade secret (although it was described at a high level in the workshop by Mick Conley). Several speakers testified that devices could be certified by the Alliance and still perhaps fail to connect.

To my mind, testing is a weak response to design problems. It happens after the fact, and can punish a poor engineering process but not fix it. You can test-drive a car and note that the steering is a bit sluggish, but can you identify the software or the part that is causing the problem? And can you explain it to the salesman, presumably to be conveyed back to the engineers?

Cars tend to be reliable first because of widespread competition that extends internationally, and partly because lawsuits keep the managers of the automobile companies alert to engineering problems. It would be a shame to need lawsuits to correct technical problems with medical devices, but refusing to buy them might do the trick. Test beds do provide warnings that can aid purchase decisions.

Unfortunately, the forum produced no real progress on the leading question of the day, whether a national test bed would be a good idea. It was recognized generally that test beds have to reflect the particular conditions at different institutions, and that multiple test beds would be needed to cover a useful range of settings. Without further clarification of what a test bed would look like, or who would build it, a couple panelists called for the creation of national test beds. More usefully, in my opinion, one speaker suggested a public repository of tests, which are currently the proprietary sects of vendors or academic researchers.

So none of the questions about test beds received answers at the workshop, and no practical recommendations emerged. One would expect that gathering the leading experts in medical devices for seven and a half hours would allow them to come up with actionable next steps or at least a framework for proceeding, but much of the workshop was given over to rhetoric about the importance of medical devices, the need for them to interoperate, and other standard rallying cries of health care reform. I sometimes felt that I was in hearing a pitch for impressionable financial backers. And of course, there was always time taken up by vendors, providers, and regulators trying to point the finger at someone else for the problems.

Device and networking expert David Höglund has written up how the workshop fell short. I would like someone to add up all the doctors, all the senior engineers, all the leading policy makers in the room, calculate how much they are paid per minute, and add up the money wasted every time a speaker extols patient engagement, interoperability, or some other thesis that is already well known to everybody in the room. (Or perhaps they aren’t well-known–another challenge to the medical field.)

Personally, I would write off most of the day as a drain on the US economy. But I have tried to synthesize the points we need to look at going forward, so that I hope you feel the time you devoted to reading the article was well-spent.