Did IBM overhype Watson Health's AI promise?

IBM's Watson Health division has been under fire for not delivering on its promise to use AI to enable smarter, more personalized medicine. But IBM officials maintain that hospitals are seeing benefits.

1 2 Page 2
Page 2 of 2

Computerworld also reached out to Memorial Sloan Kettering Cancer Center and the Mayo Clinic, two of IBM's top development partners on Watson Health who've been cited as success stories for training Watson and using it for clinical trial matching.

Begun in 2014, Watson's job at the Mayo Clinic was to sift through thousands of medical studies and ensure that more patients are accurately and consistently matched with promising clinical trials. (IBM has announced that enrollment rates for breast cancer clinical trials at the Mayo Clinic had increased dramatically.)

A request for comment from the Mayo Clinic on Watson's effectiveness was not returned. A Mayo Clinic spokesperson said multiple attempts had been made at reaching the physician in charge of the Watson project but were unsuccessful.

A Memorial Sloan Kettering Cancer Center spokesperson referred questions to IBM, stating that IBM receives feedback on Watson for Oncology directly from its customers, and while the hospital trains Watson's AI with its data, "we do not use it here."

Another clinic touted early on by IBM is the Highlands Oncology Group (HOG), which participated in a feasibility study of IBM Watson to increase the efficiency and accuracy of the clinical trial matching. Located in Northeast Arkansas, HOG has 15 physicians and 310 staff members working across three sites; the facility's pilot lasted 16 weeks and used data from 2,620 visits by lung and breast cancer patients.

In an initial pre-screening test, the HOG clinical trial coordinator took 1 hour and 50 minutes to process 90 patients against three breast cancer screenings. Conversely, when the Watson's clinical trial matching platform was used, that job took 24 minutes. "This represents a significant reduction in time of 86 minutes or 78%," HOG said in a statement.

Computerworld reached out to HOG about the Watson trial, and asked specifically if there were any problems during the pilot; HOG's medical director said the clinic had signed a confidentiality agreement with IBM and was not allowed to give out any information.

"So, IBM Watson would be the ones that provide you the concerns and road blocks they've run into," a HOG spokesperson wrote via email.

An IBM buying spree, and what comes next

In 2015, IBM purchased healthcare data analytics-firm Explorys and patient communications company Phytel for an undisclosed amount. In 2016, IBM acquired Truven Health Analytics for $2.6 billion. Upon completing all three acquisitions, IBM boasted its Watson Health Cloud housed "one of the world's largest and most diverse collections of health-related data, representing an aggregate of approximately 300 million patient lives acquired from three companies."

"They all in their own right, before they were acquired, were very successful companies and had good, strong, loyal client bases and were plugging along. I think IBM thought, 'We should buy these guys and throw in some AI and really take the market by storm,'" Burghard said. "As far as I can tell, that hasn't happened."

At least one of those acquisitions, Truven, was recently cited by IBM's Kelly as key to moving insurance provider data onto the IBM Watson Health platform now that it's going to be offered through a hybrid cloud.

In late October, IBM announced plans to seed its new hybrid cloud model for Watson by first moving data from insurance payer systems. For that, Truven will be key.

"They [Truven] are very big in the payer space," Kelly said. "We process payer claims and we have payer records. So, what does it cost for a certain procedure in a state or in a hospital – that is a very rich data set we can apply AI to to dramatically reduce cost."

Once payer data is moved to the hybrid cloud, the electronic medical records (EMRs) acquired through the Explorys acquisition will follow, Kelly said.

Copyright © 2018 IDG Communications, Inc.

1 2 Page 2
Page 2 of 2
Download: EMM vendor comparison chart 2019