Creating dependable AI solutions for the healthcare industry
New research proposes a system to determine the relative accuracy of predictive AI in a hypothetical medical setting and when the system should defer to a human clinician. Artificial intelligence (AI) has great potential to enhance how people work across a range of industries. However, to integrate AI tools into the workplace in a safe and responsible way, we need to develop more robust methods for understanding when they can be most useful.
So when is AI more accurate, and when is a human? This question is particularly important in healthcare, where predictive AI is increasingly used in high-stakes tasks to assist clinicians. Today in Nature Medicine, we’ve published our joint paper with Google Research, which proposes CoDoC (Complementarity-driven Deferral-to-Clinical Workflow), an AI system that learns when to rely on predictive AI tools or defer to a clinician for the most accurate interpretation of medical images.
CoDoC explores how we could harness human-AI collaboration in hypothetical medical settings to deliver the best results. In one scenario, CoDoC reduced the number of false positives by 25% for a large, de-identified UK mammography dataset, compared to commonly used clinical workflows—without missing any true positives. This work is a collaboration with several healthcare organizations, including the United Nations Office for Project Services’ Stop TB Partnership. To support further development in improving the transparency and safety of AI models for real-world applications, we’ve also open-sourced CoDoC’s code.
CoDoC: Add-on tool for human-AI collaboration
Building more reliable AI models often requires re-engineering the complex inner workings of predictive AI models. However, for many healthcare providers, it’s not feasible to redesign a predictive AI model. CoDoC can potentially enhance predictive AI tools for users without requiring them to alter the underlying AI tool itself.
When developing CoDoC, we had three criteria:
1. Non-machine learning experts, like healthcare providers, should be able to deploy the system and run it on a single computer.
2. Training would require a relatively small amount of data—typically, just a few hundred examples.
3. The system could be compatible with any proprietary AI models and would not need access to the model’s inner workings or the data it was trained on.
Determining when predictive AI or a clinician is more accurate
With CoDoC, we propose a straightforward and usable AI system to improve reliability by helping predictive AI systems to ‘know when they don’t know’. We examined scenarios where a clinician might have access to an AI tool designed to help interpret an image, such as analyzing a chest x-ray for whether a tuberculosis test is needed.
For any theoretical clinical setting, CoDoC’s system requires only three inputs for each case in the training dataset:
1. The predictive AI outputs a confidence score between 0 (certain no disease is present) and 1 (certain that disease is present).
2. The clinician’s interpretation of the medical image.
3. The ground truth of whether disease was present, as established via biopsy or other clinical follow-up.
Note: CoDoC requires no access to any medical images.
CoDoC learns to establish the relative accuracy of the predictive AI model compared with clinicians’ interpretations, and how that relationship fluctuates with the predictive AI’s confidence scores. Once trained, CoDoC could be integrated into a hypothetical future clinical workflow involving both an AI and a clinician. When a new patient image is evaluated by the predictive AI model, its associated confidence score is fed into the system. Then, CoDoC assesses whether accepting the AI’s decision or deferring to a clinician will ultimately yield the most accurate interpretation.
Increased accuracy and efficiency
Our comprehensive testing of CoDoC with multiple real-world datasets—including only historic and de-identified data—has shown that combining the best of human expertise and predictive AI produces greater accuracy than either alone. In addition to achieving a 25% reduction in false positives for a mammography dataset, hypothetical simulations where an AI was permitted to act autonomously on certain occasions revealed that CoDoC could reduce the number of cases needing a clinician’s review by two-thirds. We also demonstrated how CoDoC could hypothetically enhance the triage of chest X-rays for further testing for tuberculosis.
Responsibly developing AI for healthcare
While this work is theoretical, it illustrates CoDoC’s potential adaptability: it was able to improve performance on interpreting medical images across varied demographic populations, clinical settings, medical imaging equipment, and disease types. CoDoC serves as a promising example of how we can leverage the benefits of AI in conjunction with human strengths and expertise. We are actively working with partners to rigorously evaluate our research and the system’s potential benefits. To ensure technology like CoDoC can be safely implemented in real-world medical settings, healthcare providers and manufacturers must understand how clinicians interact differently with AI and validate systems with specific medical AI tools and settings.
Learn more about CoDoC:
