Doctor and patient information needs for adaptive explainable artificial intelligence: lessons learned from a preliminary study

Abstract

Artificial Intelligence (AI) has the potential to enhance clinical decision-making and communication by supporting diagnostic reasoning, but it also raises legal, ethical, and decision-related challenges due to its 'black box' nature. Explainable AI (XAI) aims to address this challenge by attempting to enhance the transparency of AI decisions, and by providing user-tailored explanations that clarify how these decisions are made. The present work investigated how AI-based diagnostic insights can be presented to satisfy the informational needs of doctors and patients. A total of 58 participants (26 doctors) were asked through open questions to first describe in their own words what kind of information they would like to receive from an AI system. Then, they were presented with scenarios and asked to select which information they would like to receive selecting among four types of explanations identified in literature: contrastive, counterfactual, causal, and mechanistic explanation. The quantitative results showed that doctors were more likely to select contrastive explanations, potentially such explanations can help them distinguish between alternative diagnoses. Contrastingly, patients favored counterfactual explanations that serve them to identify alternative scenarios and possible outcomes, significantly differing from doctors' preferences. Doctors focused on detailed, data-driven symptom information, whereas patients emphasized prognosis, risks, and treatment options. Despite these differences, both groups valued understanding symptom progression, treatment pathways, and the reasoning behind AI-driven diagnosis. The findings of this preliminary study provide actionable guidance for XAI developers, highlighting exactly which aspects of AI-assisted decision-making should be made explainable to different user groups. Tailoring explanations in this way can improve interpretability, trust, and practical utility of AI-assisted diagnostic tools in healthcare.