Counterfactual Explanations for Incorrect Predictions Made by AI Models

2023 | International Conference on Information Systems | Citations: 0

Authors: Asrzad, Amir; Li, Xiaobai

Abstract: Advanced AI models are powerful in making accurate predictions for complex prob ... Expand

Abstract: Advanced AI models are powerful in making accurate predictions for complex problems. However, these models often operate as black boxes. This lack of interpretability poses significant challenges, especially in high-stakes applications such as finance, healthcare, and criminal justice. Explainable AI seeks to address the challenges by developing methods that can provide meaningful explanations for humans to understand. When black box models are used for prediction, they inevitably produce errors. It is important to appropriately explain incorrect predictions. This problem, however, has not been addressed in the literature. In this study, we propose a novel method to provide explanations for misclassified cases made by black box models. The proposed method takes a counterfactual explanation approach. It builds a decision tree to find the best counterfactual examples for explanations. Incorrect predictions are rectified using a trust score measure. We validate the proposed method in an evaluation study using real-world data. Collapse

Topics: artificial intelligence explainable artificial intelligence local interpretable model agnostic explanation decision making enterprise information system

Methods: machine learning decision tree classification literature study random forest classification experiment

Expl(AI)ned: The Impact of Explainable Artificial Intelligence on Users’ Information Processing

2023 | Information Systems Research | Citations: 0

Authors: Bauer, Kevin; von Zahn, Moritz; Hinz, Oliver

Abstract: Because of a growing number of initiatives and regulations, predictions of moder ... Expand

Abstract: Because of a growing number of initiatives and regulations, predictions of modern artificial intelligence (AI) systems increasingly come with explanations about why they behave the way they do. In this paper, we explore the impact of feature-based explanations on users’ information processing. We designed two complementary empirical studies where participants either made incentivized decisions on their own, with the aid of opaque predictions, or with explained predictions. In Study 1, laypeople engaged in the deliberately abstract investment game task. In Study 2, experts from the real estate industry estimated listing prices for real German apartments. Our results indicate that the provision of feature-based explanations paves the way for AI systems to reshape users’ sense making of information and understanding of the world around them. Specifically, explanations change users’ situational weighting of available information and evoke mental model adjustments. Crucially, mental model adjustments are subject to the confirmation bias so that misconceptions can persist and even accumulate, possibly leading to suboptimal or biased decisions. Additionally, mental model adjustments create spillover effects that alter user behavior in related yet disparate domains. Overall, this paper provides important insights into potential downstream consequences of the broad employment of modern explainable AI methods. In particular, side effects of mental model adjustments present a potential risk of manipulating user behavior, promoting discriminatory inclinations, and increasing noise in decision making. Our findings may inform the refinement of current efforts of companies building AI systems and regulators that aim to mitigate problems associated with the black-box nature of many modern AI systems. History: Alessandro Acquisti, senior editor; Jason Chan, associate editor. Funding: This work was supported by the Deutsche Forschungsgemeinschaft (DFG) (Projek 449023539), Volkswagen Foundation (ML2MT), and LeibnizInstitute for Financial Research SAFE. Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2023.1199. Collapse

Topics: explainable artificial intelligence artificial intelligence shapley additive explanation local interpretable model agnostic explanation decision making

Methods: F-test regression analysis method experiment field study robustness check

A Bitter Pill to Swallow? The Consequences of Patient Evaluation in Online Health Question-and-Answer Platforms

2023 | Information Systems Research | Citations: 0

Authors: Chen, Chen; Walker, Dylan

Abstract: Online health question-and-answer (Q&A) platforms (OHQPs), where patients post h ... Expand

Abstract: Online health question-and-answer (Q&A) platforms (OHQPs), where patients post health-related questions, evaluate advice from multiple doctors, and direct a bounty (monetary reward) to their most preferred answer, have become a prominent channel for patients to receive medical advice in China. To explore the quality of medical advice on these platforms, we analyzed data on patients’ evaluation of ∼497,000 answers to ∼114,000 questions on one of the most popular OHQPs, 120ask.com, over a three-month period. We assembled a panel of independent physicians and instructed them to evaluate the quality of ∼13,000 answers. We found that the quality of medical advice offered on the platform was high on average, and that low-quality answers were rare (6%). However, our results also indicate that patients lacked the ability to discriminate advice quality. They were as likely to choose the best answer as the worst. The medical accuracy of patient evaluation was worse in critical categories (cancer, internal medicine) and for vulnerable subpopulations (pediatrics). Given that millions of patients seek medical advice from OHQPs in China annually, the social and economic implications of this finding are troubling. To understand how patients evaluate advice, we trained deep neural networks to think like patients, allowing us to identify patients’ positive and negative responses to different heurist cues. Although our results indicate that OHQPs perform well, we identified several concerns that should be addressed through platform design and policy changes. Because the Q&A process lacks peer review mechanisms, signals of advice quality are not conveyed to patients, forcing them to rely on heuristic cues, which cannot effectively guide them toward the best advice. We also found that the platform reputation metric was not correlated with the quality of the advice giver’s advice, may effectively encourage patients to select lesser quality medical advice, and increased the risk of moral hazard for malicious players to intentionally provide less accurate but more agreeable advice for personal gain. Our analysis revealed bad actors on the platform, including drug promoters and spammers. Finally, we found that OHQPs exacerbated care avoidance. We discuss several potential policy changes to address these shortcomings. History: Param Singh, Senior Editor; Idris Adjerid, Associate Editor. Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2022.1158. Collapse

Topics: decision making local interpretable model agnostic explanation digital platform health information system pandemic

Methods: artificial neural network psychometrics survey deep learning natural language processing

Theories: lemon market theory

Explanation matters: An experimental study on explainable AI

2023 | Electronic Markets | Citations: 0

Authors: Hamm, Pascal; Klesel, Michael; Coberger, Patricia; Wittmann, H. Felix

Abstract: Explainable artificial intelligence (XAI) is an important advance in the field o ... Expand

Abstract: Explainable artificial intelligence (XAI) is an important advance in the field of machine learning to shed light on black box algorithms and thus a promising approach to improving artificial intelligence (AI) adoption. While previous literature has already addressed the technological benefits of XAI, there has been little research on XAI from the user’s perspective. Building upon the theory of trust, we propose a model that hypothesizes that post hoc explainability (using Shapley Additive Explanations) has a significant impact on use-related variables in this context. To test our model, we designed an experiment using a randomized controlled trial design where participants compare signatures and detect forged signatures. Surprisingly, our study shows that XAI only has a small but significant impact on perceived explainability. Nevertheless, we demonstrate that a high level of perceived explainability has a strong impact on important constructs including trust and perceived usefulness. A post hoc analysis shows that hedonic factors are significantly related to perceived explainability and require more attention in future research. We conclude with important directions for academia and for organizations. Collapse

Topics: explainable artificial intelligence artificial intelligence perceived ease of use perceived usefulness digital dashboard

Methods: experimental group experiment parametric test experimental task ANOVA

Theories: information systems theory trust theory

Black-box Models’ Explainability: A Theoretical and Practical Perspective

2023 | Americas Conference on Information Systems | Citations: 0

Authors: He, Yuanqi; Gu, Bin; Li, Chunxiao; Yan, Lingling; Ding, Yanqing

Abstract: The lack of explainability remains a critical challenge to the widespread adopt ... Expand

Abstract: The lack of explainability remains a critical challenge to the widespread adoption of artificial intelligence (AI) in many fields. "Understanding brings in trust", while machine-learning models offer superior prediction accuracy, understanding the underlying logic is equally important to foster trust in these models. In this paper, we present eXplainable AI (XAI) as a solution to this challenge. Our research focuses on three key aspects of XAI: mathematics, humanities and social sciences, and practical applications. We demonstrated the feasibility of XAI through the use of artificially-constructed and model-derived ground truth, and verified performances of different XAIs. We also explored three dimensions of explainable consistency and emphasized the significance of human-machine consistency. Finally, we applied our research to a real-world scenario by cooperating with a national bank in China. Our findings highlight that XAI is both mathematically and practically meaningful, but more efforts need to be dedicated to this human-machine communication field. Collapse

Topics: explainable artificial intelligence shapley additive explanation local interpretable model agnostic explanation decision making accounting

Methods: computational algorithm principal component analysis data transformation XGBoost experiment

ICU Outcome Predictions Using Real-Time Signals with Wavelet-Transform-based Convolutional Neural Network

2023 | HICSS | Citations: 0

Authors: Jiang, Yiqun; Wang, Shaodong; Li, Qing; Zhang, Wenli

Abstract: Intensive care units (ICUs) serve patients with life-threatening conditions. The ... Expand

Abstract: Intensive care units (ICUs) serve patients with life-threatening conditions. The limited ICU resources cause severe economic and healthcare burdens worldwide. It is critical to conduct ICU outcome predictions at an early stage and promote efficient use of ICU resources. However, all the current prediction methods have limitations such as unsatisfactory accuracy and depending on resource-demanding laboratory tests or expert domain knowledge. In this research, we design a wavelet-transformed-based convolutional neural network, WTCNN, which only requires patients’ vital sign series and information at ICU admission for real-time ICU outcome predictions. The model is evaluated using a large real-world ICU database and outperforms state-of-art baselines on both ICU mortality and length-of-stay prediction tasks. We conduct LIME for model interpretation and prescriptive analysis. Our work provides an efficient tool for ICU outcome predictions, allowing healthcare providers to take action promptly on patients at risk and reduce the negative impacts on patient outcomes. Collapse

Topics: local interpretable model agnostic explanation database system missing data societal evaluation criteria healthcare data

Methods: machine learning longitudinal research time series analysis deep learning electrocardiography

ROLEX: A Novel Method for Interpretable Machine Learning Using Robust Local Explanations

2023 | Management Information Systems Quarterly | Citations: 2

Authors: Kim, Buomsoo (Raymond); Srinivasan, Karthik; Kong, Sung Hye; Kim, Jung Hee; Shin, Chan Soo; Ram, Sudha

Abstract: Recent developments in big data technologies are revolutionizing the field of h ... Expand

Abstract: Recent developments in big data technologies are revolutionizing the field of healthcare predictive analytics (HPA), enabling researchers to explore challenging problems using complex prediction models. Nevertheless, healthcare practitioners are reluctant to adopt those models as they are less transparent and accountable due to their black-box structure. We believe that instance-level, or local, explanations enhance patient safety and foster trust by enabling patient-level interpretations and medical knowledge discovery. Therefore, we propose the RObust Local EXplanations (ROLEX) method to develop robust, instance-level explanations for HPA models in this study. ROLEX adapts state-of-the-art methods and ameliorates their shortcomings in explaining individual-level predictions made by black-box machine learning models. Our analysis with a large real-world dataset related to a prevalent medical condition called fragility fracture and two publicly available healthcare datasets reveals that ROLEX outperforms widely accepted benchmark methods in terms of local faithfulness of explanations. In addition, ROLEX is more robust since it does not rely on extensive hyperparameter tuning or heuristic algorithms. Explanations generated by ROLEX, along with the prototype user interface presented in this study, have the potential to promote personalized care and precision medicine by providing patient-level interpretations and novel insights. We discuss the theoretical implications of our study in healthcare, big data, and design science. Collapse

Topics: explainable artificial intelligence usability user interface local interpretable model agnostic explanation problem solving

Methods: machine learning design artifact computational algorithm longitudinal research ablation analysis

Theories: lemon market theory

How Explainable AI Methods Support Data-Driven Decision Making

2023 | International Conference on Business Informatics | Citations: 0

Authors: Stoffels, Dominik; Grabl, Susanne; Fischer, Thomas; Fiedler, Marina

Abstract: Explainable AI (XAI) holds great potential to reveal the patterns in black-box ... Expand

Abstract: Explainable AI (XAI) holds great potential to reveal the patterns in black-box AI models and to support data-driven decision-making. We apply four post-hoc explanatory methods to demonstrate the explanatory capabilities of these methods for data-driven decision-making using the illustrative example of unwanted job turnover and human resource management (HRM) support. We show that XAI can be a useful aid in data-driven decision-making, but also highlight potential drawbacks and limitations of which users in research and practice should be aware. Collapse

Topics: explainable artificial intelligence shapley additive explanation decision making local interpretable model agnostic explanation human resource management

Methods: machine learning support vector machine AdaBoost company material computational algorithm

MDSCAN: AN EXPLAINABLE ARTIFICIAL INTELLIGENCE ARTIFACT FOR MENTAL HEALTH SCREENING

2023 | European Conference On Information Systems | Citations: 0

Authors: Tutun, Salih; Bhattacherjee, Anol; Topuz, Kazim; Tosyali, Ali; Li, Gorden

Abstract: This paper presents a novel artifact called MDscan that can help mental health ... Expand

Abstract: This paper presents a novel artifact called MDscan that can help mental health professionals quickly screen a large number of patients for ten mental disorders. MDscan uses patient responses to the SCL-90-R clinical questionnaire to create a full-color image, similar to radiological images, which identifies which disorder or combination of disorders may afflict a patient, the severity of the disorder, and the underlying logic of this prediction, using an explainable artificial intelligence (XAI) approach. While prior artificial intelligence (AI) tools have seen limited acceptance in clinical practice because of the lack of transparency and interpretability in their "black box" models, the XAI approach used in MDscan is a "white box" model that elaborates which patient feature contributes to the predicted outcome and to what extent. Using patient data from a mental health clinic, we demonstrate that MDscan outperforms current (expert-based) clinical practice by an average of 20%. Collapse

Topics: explainable artificial intelligence shapley additive explanation local interpretable model agnostic explanation IT security artificial intelligence

Methods: computational algorithm machine learning data transformation t-distributed stochastic neighbor embedding survey

An AI-based Decision Support System for Predicting Mental Health Disorders

2023 | Information Systems Frontiers | Citations: 0

Authors: Tutun, Salih; Johnson, Marina E.; Ahmed, Abdulaziz; Albizri, Abdullah; Irgil, Sedat; Yesilkaya, Ilker; Ucar, Esma Nur; Sengun, Tanalp; Harfouche, Antoine

Abstract: Approximately one billion individuals suffer from mental health disorders, such ... Expand

Abstract: Approximately one billion individuals suffer from mental health disorders, such as depression, bipolar disorder, schizophrenia, and anxiety. Mental health professionals use various assessment tools to detect and diagnose these disorders. However, these tools are complex, contain an excessive number of questions, and require a significant amount of time to administer, leading to low participation and completion rates. Additionally, the results obtained from these tools must be analyzed and interpreted manually by mental health professionals, which may yield inaccurate diagnoses. To this extent, this research utilizes advanced analytics and artificial intelligence to develop a decision support system (DSS) that can efficiently detect and diagnose various mental disorders. As part of the DSS development process, the Network Pattern Recognition (NEPAR) algorithm is first utilized to build the assessment tool and identify the questions that participants need to answer. Then, various machine learning models are trained using participants’ answers to these questions and other historical data as inputs to predict the existence and the type of their mental disorder. The results show that the proposed DSS can automatically diagnose mental disorders using only 28 questions without any human input, to an accuracy level of 89%. Furthermore, the proposed mental disorder diagnostic tool has significantly fewer questions than its counterparts; hence, it provides higher participation and completion rates. Therefore, mental health professionals can use this proposed DSS and its accompanying assessment tool for improved clinical decision-making and diagnostic accuracy. Collapse

Topics: artificial intelligence decision support system explainable artificial intelligence information portal decision making

Methods: computational algorithm support vector machine design process lasso regression ridge regression