J10 years ago, a committee of atmospheric experts published a report that revolutionized the weather industry in a single sentence: from research to operations (R2O). This term was coined to describe the challenge of transitioning satellite data into operational use, or as it has been described, bypassing the “valley of death” that engulfed research before it could see the day.
The report posed an important question: what if industry could bridge research and operations?
What he did. After defining R2O, the meteorological community has seen an immense increase in bridge building initiatives. The National Oceanic and Atmospheric Administration, for example, has developed a new initiative focused on best practices and incentive programs to reduce the research burden on operations and help research become a reality.
Today, the healthcare industry faces an eerily similar chasm.
While completing a master’s degree at Tufts University School of Medicine in Boston, I explored the multitude of challenges posed by the R2O revolution and the lessons that can be learned from it in the healthcare industry. Identifying the challenges was a simpler task: obtaining innovative research on healthcare workflows is well documented.
Take the use of artificial intelligence (AI) and machine learning (ML) in medicine. According to a report, less than 10% of machine learning models go into production across all industries. The percentage is even lower in health care, given the additional barriers of safety, accessibility, specialization and regulation. Since 1997, when the FDA first approved an AI/ML-enabled healthcare device, PubMed lists more than 26,000 publications on machine learning, AI, and healthcare. Yet, as of this writing, there are only 343 FDA-cleared AI/ML-enabled healthcare devices left (including software services).
The first clear step was to create a common term to discuss the problem. As renowned meteorologist William Hooke eloquently stated in a post for the American Meteorology Society’s “The Front Page” blog, “R2O matters. Simply put, it is the key to realizing the societal benefits of research and development.
With a common goal in mind, meteorology looked to interoperability solutions: by developing connections between satellites and data systems, the industry was able to develop better real-time reports. Interoperability will also save lives in healthcare and help close the R2O chasm facing the industry, but the transition is not the place to stop.
The key finding of my work landed here: interoperability is not enough. Just because a clinician has access to new research does not mean he can use it to treat his patients. Focusing on artificial intelligence, I identified three determinants for actualizing the value and promise of AI: explainable, transparent, and actionable. In a nutshell, models must be understood, reliable and, above all, useful.
As John D. Halamka, Suchi Saria, and Nigam Shah recently wrote in First Opinion, “To realize the full potential of artificial intelligence (AI) and machine learning (ML) for patients, researchers must foster greater confidence in the accuracy, fairness, and usefulness of clinical AI algorithms.
Several programs are already at work to address R2O in healthcare. Here are some examples :
Bridge2AI, a program funded by the National Institutes of Health Common Fund to “propel biomedical research forward”
MedPerf, an open benchmarking platform for medical artificial intelligence
Model Cards, a standard developed by my company, Google, to bring consistency to best practices in data transparency and explainability
- HealthIT.gov focus areas, providing opportunities to engage in regulatory change
Moving forward requires prioritizing and rewarding work that focuses on the real implementation of research – and therefore in the lives of patients. R2O means encouraging researchers to see their work beyond publication. It develops interoperable standards and uses them meaningfully. It strives to foster trust in technologies by breaking down biases and building with fairness in mind. R2O works across the full continuum of care to ensure that all users from all walks of life can understand, trust and use the technology.
Operations research is a key approach to unlocking innovation in healthcare.
Vivian Neilley is Head of Interoperability at Google Cloud and Healthcare Standards Advocate for Alphabet.