When Computers Kill: Radiation Overdose

I was watching BBC News on EyeTV this morning, and caught the tail end of a horrific story about hundreds of French patients who received crippling, and sometimes fatal, overdoses of radiation.

Earlier this year, a major scandal erupted in France when it was discovered that between 1989 and 2006, two radiotherapy units had accidentally given hundreds of cancer patients too high a dose of radiation. Five patients have since died and many others have been left in crippling pain.

My first thought was how eerily similar this is to Therac-25. But this incident could be worse once all the facts are out. 5 are already dead, and hundreds affected, according to the BBC.

A major investigation is now under way to try to establish how so many mistakes could have been made…. Incredibly, one of the lines of inquiry will be why the instruction booklets that accompanied the equipment were in English when the hospital staff of course were French.

This investigation is very much worth following. A lot can be learned about designing safe and usable systems from this disaster. Cynically, I worry that the massive liability involved will lead to politics and cover ups, instead of through investigation. Be prepared to read between the lines.

… staff then explained to newcomers how to operate the programmes, who later explained to subsequent trainees, and so on. To add to the confusion, the procedures were all in English.

Eventually, an incorrect default setting was made that resulted in a number of patients being given overdoses of between 20% and 40%.

Poor training is an issue, sure. But the real question I have here is, how could the software be designed so that it could possible be rendered lethal by default?

According to the AP “In both the Epinal and Lyon incidents, hospitals blamed the problems on human error.” I agree, but I think the humans at fault were the designers, not the operators. “Human error” is usually a euphemism for “operator error”, or “customer error”, or “blame them”. Disasters are a chain of failures; operators are only one link in that chain. The system as implemented in the hospitals included hardware, software, training, and standard operating procedures. From all accounts, it looks like there were systematic errors, over a period of years — about the strongest indicator you can have that the system was deeply flawed.

What Therac-25 was to engineering, this could be to interaction design. I think there were probably engineering mistakes made, but if the instructions weren’t even in the right language, chances are usability was a bigger factor. Actually, the similarities to Therac-25 still bother me. It’s a bit of history that should not be repeated.

I’ve said it before and I’ll say it again, these incidents are worth following. I just wish more hard facts were public (and in English as well, I can’t read French).

Explore posts in the same categories: Accessibility, Bugs, Design, Programming, Research, Security, Usability