Counterfactual Explanations for Incorrect Predictions Made by AI Models

2023 | International Conference on Information Systems | Citations: 0

Authors: Asrzad, Amir; Li, Xiaobai

Abstract: Advanced AI models are powerful in making accurate predictions for complex prob ... Expand

Abstract: Advanced AI models are powerful in making accurate predictions for complex problems. However, these models often operate as black boxes. This lack of interpretability poses significant challenges, especially in high-stakes applications such as finance, healthcare, and criminal justice. Explainable AI seeks to address the challenges by developing methods that can provide meaningful explanations for humans to understand. When black box models are used for prediction, they inevitably produce errors. It is important to appropriately explain incorrect predictions. This problem, however, has not been addressed in the literature. In this study, we propose a novel method to provide explanations for misclassified cases made by black box models. The proposed method takes a counterfactual explanation approach. It builds a decision tree to find the best counterfactual examples for explanations. Incorrect predictions are rectified using a trust score measure. We validate the proposed method in an evaluation study using real-world data. Collapse

Topics: artificial intelligence explainable artificial intelligence local interpretable model agnostic explanation decision making enterprise information system

Methods: machine learning decision tree classification literature study random forest classification experiment

Expl(AI)ned: The Impact of Explainable Artificial Intelligence on Users’ Information Processing

2023 | Information Systems Research | Citations: 0

Authors: Bauer, Kevin; von Zahn, Moritz; Hinz, Oliver

Abstract: Because of a growing number of initiatives and regulations, predictions of moder ... Expand

Abstract: Because of a growing number of initiatives and regulations, predictions of modern artificial intelligence (AI) systems increasingly come with explanations about why they behave the way they do. In this paper, we explore the impact of feature-based explanations on users’ information processing. We designed two complementary empirical studies where participants either made incentivized decisions on their own, with the aid of opaque predictions, or with explained predictions. In Study 1, laypeople engaged in the deliberately abstract investment game task. In Study 2, experts from the real estate industry estimated listing prices for real German apartments. Our results indicate that the provision of feature-based explanations paves the way for AI systems to reshape users’ sense making of information and understanding of the world around them. Specifically, explanations change users’ situational weighting of available information and evoke mental model adjustments. Crucially, mental model adjustments are subject to the confirmation bias so that misconceptions can persist and even accumulate, possibly leading to suboptimal or biased decisions. Additionally, mental model adjustments create spillover effects that alter user behavior in related yet disparate domains. Overall, this paper provides important insights into potential downstream consequences of the broad employment of modern explainable AI methods. In particular, side effects of mental model adjustments present a potential risk of manipulating user behavior, promoting discriminatory inclinations, and increasing noise in decision making. Our findings may inform the refinement of current efforts of companies building AI systems and regulators that aim to mitigate problems associated with the black-box nature of many modern AI systems. History: Alessandro Acquisti, senior editor; Jason Chan, associate editor. Funding: This work was supported by the Deutsche Forschungsgemeinschaft (DFG) (Projek 449023539), Volkswagen Foundation (ML2MT), and LeibnizInstitute for Financial Research SAFE. Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2023.1199. Collapse

Topics: explainable artificial intelligence artificial intelligence shapley additive explanation local interpretable model agnostic explanation decision making

Methods: F-test regression analysis method experiment field study robustness check

Addressing Learners' Heterogeneity in Higher Education: An Explainable AI-based Feedback Artifact for Digital Learning Environments

2023 | International Conference on Business Informatics | Citations: 0

Authors: Haag, Felix; Günther, Sebastian A.; Hopf, Konstantin; Handschuh, Philipp; Klose, Maria; Staake, Thorsten

Abstract: ... Expand

Abstract: Collapse

Topics: explainable artificial intelligence online learning system analytical information system project management IT project management

Methods: design science survey design principle design artifact two sample t-test with equal variance

Overcoming Anchoring Bias: The Potential of AI and XAI-based Decision Support

2023 | International Conference on Information Systems | Citations: 0

Authors: Haag, Felix; Stingl, Carlo; Zerfass, Katrin; Hopf, Konstantin; Staake, Thorsten

Abstract: Information systems (IS) are frequently designed to leverage the negative effec ... Expand

Abstract: Information systems (IS) are frequently designed to leverage the negative effect of anchoring bias to influence individuals ' decision-making (e.g., by manipulating purchase decisions). Recent advances in Artificial Intelligence (AI) and the explanations of its decisions through explainable AI (XAI) have opened new opportunities for mitigating biased decisions. So far, the potential of these technological advances to overcome anchoring bias remains widely unclear. To this end, we conducted two online experiments with a total of N=390 participants in the context of purchase decisions to examine the impact of AI and XAI-based decision support on anchoring bias. Our results show that AI alone and its combination with XAI help to mitigate the negative effect of anchoring bias. Ultimately, our findings have implications for the design of AI and XAIbased decision support and IS to overcome cognitive biases. Collapse

Topics: decision support explainable artificial intelligence artificial intelligence task productivity decision making

Methods: experiment experimental group computational algorithm ordinary least square Student's t-test

Empowering Recommender Systems in ITSM: A Pipeline Reference Model for AI-Based Textual Data Quality Enrichment

2023 | DESRIST | Citations: 0

Authors: Reinhard, Philipp; Li, Mahei Manhai; Dickhaut, Ernestine; Peters, Christoph; Leimeister, Jan Marco

Abstract: AI-based recommendation systems to augment working conditions in the field of IT ... Expand

Abstract: AI-based recommendation systems to augment working conditions in the field of IT service management (ITSM) have attracted new attention. However, many IT support organizations possess high volumes of tickets but are confronted with low quality, to which they train the underlying models of their AI systems. In particular, support tickets are documented insufficiently due to time pressure and lack of motivation. Following design science research, we design and evaluate an analytics pipeline to address the data quality issue. The pipeline can be applied to assess and extract high-quality support tickets for subsequent model training and operation. Based on a data set of 60.000 real-life support tickets from a manufacturing company, we develop the artifact, instantiate a recommender system and achieve a higher prediction performance in comparison to naïve enrichment methods. In terms of data management literature, we contribute to the understanding of assessing textual ticket data quality. By deriving a pipeline reference model, we move towards a general approach to designing machine learning-driven data quality analytics pipelines for attached recommender systems. Collapse

Topics: data quality recommender system system support shapley additive explanation database system

Methods: machine learning reference modelling design artifact cluster analysis topic model

When machines trade on corporate disclosures: Using text analytics for investment strategies

2023 | Decision Support Systems | Citations: 0

Authors: Schmitz, Hans Christian; Lutz, Bernhard; Wolff, Dominik; Neumann, Dirk

Abstract: Can you make profits by trading on corporate disclosures using machine learning? ... Expand

Abstract: Can you make profits by trading on corporate disclosures using machine learning? In this study, we aim to obtain a conservative estimate of profitability, while accounting for the combination of several important realworld aspects. Specifically, we consider the holistic research problem that combines model predictions based on the textual content of corporate disclosures and trading strategies while accounting for transaction costs, order clearance periods, post-publication returns, and liquidity filtering. Furthermore, we aim to understand how the resulting profits are influenced by different model and trading strategy parameters. Based on 354,992 form 8-K filings and 10,204 ad hoc announcements, we find that the proposed trading strategies yield up to 7.81 % and 9.34 % out-of-sample annualized return. In addition, our results suggest that machine learning models should be provided with additional features about prior disclosures, while being trained on the ternary prediction problem that allows for predictions of neutral market reactions. We complement our results with several sensitivity analyses that show how profitability is influenced by transaction costs, different ensemble sizes, return neutrality thresholds, and liquidity filtering. Ultimately, we provide useful insights for practitioners by describing how the machine learning models arrive at decisions in terms of Shapley Additive Explanations. Collapse

Topics: shapley additive explanation accounting telephone electronic mail

Methods: machine learning sensitivity analysis term frequency–inverse document frequency GloVe random forest classification

Theories: transaction cost economics game theory

How Explainable AI Methods Support Data-Driven Decision Making

2023 | International Conference on Business Informatics | Citations: 0

Authors: Stoffels, Dominik; Grabl, Susanne; Fischer, Thomas; Fiedler, Marina

Abstract: Explainable AI (XAI) holds great potential to reveal the patterns in black-box ... Expand

Abstract: Explainable AI (XAI) holds great potential to reveal the patterns in black-box AI models and to support data-driven decision-making. We apply four post-hoc explanatory methods to demonstrate the explanatory capabilities of these methods for data-driven decision-making using the illustrative example of unwanted job turnover and human resource management (HRM) support. We show that XAI can be a useful aid in data-driven decision-making, but also highlight potential drawbacks and limitations of which users in research and practice should be aware. Collapse

Topics: explainable artificial intelligence shapley additive explanation decision making local interpretable model agnostic explanation human resource management

Methods: machine learning support vector machine AdaBoost company material computational algorithm

Bayesian Networks for Interpretable Cyberattack Detection

2023 | HICSS | Citations: 0

Authors: Yang, B.; Hoffman, M.; Brown, N.

Abstract: The challenge of cyberattack detection can be illustrated by the complexity of ... Expand

Abstract: The challenge of cyberattack detection can be illustrated by the complexity of the MITRE ATT&CK TM matrix, which catalogues >200 attack techniques (most with multiple sub-techniques). To reliably detect cyberattacks, we propose an evidence-based approach which fuses multiple cyber events over varying time periods to help differentiate normal from malicious behavior. We use Bayesian Networks (BNs)probabilistic graphical models consisting of a set of variables and their conditional dependencies -for fusion/classification due to their interpretable nature, ability to tolerate sparse or imbalanced data, and resistance to overfitting. Our technique utilizes a small collection of expert-informed cyber intrusion indicators to create a hybrid detection system that combines data-driven training with expert knowledge to form a host-based intrusion detection system (HIDS). We demonstrate a software pipeline for efficiently generating and evaluating various BN classifier architectures for specific datasets and discuss explainability benefits thereof. Collapse

Topics: shapley additive explanation application programming interface Python Microsoft Windows intrusion detection system

Methods: random forest classification naïve bayes machine learning cluster analysis experiment

Best of Both Worlds: Combining Predictive Power with Interpretable and Explainable Results for Patient Pathway Prediction

2023 | European Conference On Information Systems | Citations: 0

Authors: Zilker, Sandra; Weinzierl, Sven; Zschech, Patrick; Kraus, Mathias; Matzner, Martin

Abstract: Proactively analyzing patient pathways can help healthcare providers to anticip ... Expand

Abstract: Proactively analyzing patient pathways can help healthcare providers to anticipate treatment-related risks, detect undesired outcomes, and allocate resources quickly. For this purpose, modern methods from the field of predictive business process monitoring can be applied to create data-driven models that capture patterns from past behavior to provide predictions about running process instances. Recent methods increasingly focus on deep neural networks (DNN) due to their superior prediction performances and their independence from process knowledge. However, DNNs generally have the disadvantage of showing black-box characteristics, which hampers dissemination in critical environments such as healthcare. To this end, we propose the design of HIXPred, a novel artifact combining predictive power with explainable results for patient pathway predictions. We instantiate HIXPred and apply it to a real-life healthcare use case for evaluation and demonstration purposes and conduct interviews with medical experts. Our results confirm high predictive performance while ensuring sufficient interpretability and explainability to provide comprehensible decision support.Keywords: Patient pathway prediction, predictive business process monitoring, explainable artificial intelligence, interpretable artificial intelligence, deep neural networks, business process management. Collapse

Topics: healthcare data shapley additive explanation explainable artificial intelligence business process management decision support

Methods: event log qualitative interview machine learning design artifact artificial neural network

Assessing the Impact of Virtual Health Communities and Environmental Characteristics of Chronic Pain Mobile Health Apps on Users’ Privacy Decisions: A Multilevel Perspective

2022 | Americas Conference on Information Systems | Citations: 0

Authors: Bantan, May; Alrefaei, Doaa; Alharbi, Raed

Abstract: Chronic pain has been identified as one of the most widespread health-related p ... Expand

Abstract: Chronic pain has been identified as one of the most widespread health-related problems. Potential chronic pain apps users seek health communities for current and previous reviews to assess the quality of the apps and make a decision regarding disclosing their information to these apps. In this study, we present a multilevel perspective on how virtual health communities and environmental characteristics of chronic pain mobile health apps impact users' privacy decisions. We used Exploratory Data Analysis and Machine Learning (ML) to operationalize the Theory of Multilevel Information Privacy. The results revealed that the most influential factors affecting users' cost-benefit analysis are Chronic Pain MHA's characteristics related to user's information privacy. The ML results indicate that the existence of information privacy policy can be predicted through the ways the apps use to Collect Data, App's Category, Country, and Store Type, which in turn affect users' decisions. Collapse

Topics: privacy mobile application mobile application market digital distribution shapley additive explanation

Methods: machine learning support vector machine exploratory data analysis natural language processing sentiment analysis

Theories: theory of multilevel information privacy privacy calculus theory