Risk and the future of AI: Algorithmic bias, data colonialism, and marginalization

2023 | Information and Organization | Citations: 0

Authors: Arora, A.; Barrett, M.; Lee, E.; Oborn, E.; Prince, K.

Abstract: ... Expand

Abstract: Collapse

Topics: artificial intelligence algorithmic bias responsible artificial intelligence legal context outsourcing

Methods: computational algorithm design methodology generative adversarial network

The Impact of Stigmatizing Language in EHR Notes on AI Performance and Fairness

2023 | International Conference on Information Systems | Citations: 0

Authors: Liu, Yizhi; Wang, Weiguang; Gao, Gordon; Agarwal, Ritu

Abstract: Today, there is significant interest in using electronic health record data to ... Expand

Abstract: Today, there is significant interest in using electronic health record data to generate new clinical insights for diagnosis and treatment decisions. However, there are concerns that such data may be biased and result in accentuating racial disparities. We study how clinician biases reflected in EHR notes affect the performance and fairness of artificial intelligence models in the context of mortality prediction for intensive care unit patients. We apply a Transformer-based deep learning model and explainable AI techniques to quantify negative impacts on performance and fairness. Our findings demonstrate that stigmatizing language written by clinicians adversely affects AI performance, particularly so for black patients, highlighting SL as a source of racial disparity in AI model development. As an effective mitigation approach, removing SL from EHR notes can significantly improve AI performance and fairness. This study provides actionable insights for responsible AI development and contributes to understanding clinician EHR note writing. Collapse

Topics: artificial intelligence explainable artificial intelligence open source moral hazard Microsoft Windows

Methods: transformer model machine learning BERT deep learning experiment

Reinforcing data bias in crisis information management: The case of the Yemen humanitarian response

2023 | International Journal of Information Management | Citations: 0

Authors: Paulus, David; De Vries, Gerdien; Janssen, Marijn; Van De Walle, Bartel

Abstract: The complex and uncertain environment of the humanitarian response to crises can ... Expand

Abstract: The complex and uncertain environment of the humanitarian response to crises can lead to data bias, which can affect decision-making. Evidence of data bias in crisis information management (CIM) remains scattered despite its potentially significant impact on crisis response. To understand what biases emerge in complex crises and how they affect CIM, we conducted a combined interview and document analysis study. Focusing on the largest humanitarian crisis in the world, i.e., the conflict in Yemen, we conducted 25 interviews with managers and analysts of response organizations, and assessed 47 reports and datasets created by response organizations in Yemen. We find evidence of a cycle of bias reinforcement through which bias cascades between field, head­ quarters and donor levels of crisis response. Researchers, as well as practitioners, need to consider these un­ derlying biases and reinforcement loops because they influence what data can be collected when, by whom, from whom, and how the data is shared and used. To the CIM literature, we contribute an in-depth understanding of how four types of data bias emerge in crises: political, accessibility, topical, and sampling bias. Collapse

Topics: information management data quality electronic mail emergency management decision making

Methods: qualitative interview qualitative content analysis qualitative coding case study personal interview

THE POLITICS OF BIOMETRIC TECHNOLOGIES: Borders control and the making of data citizens in Africa

2023 | European Conference On Information Systems | Citations: 0

Authors: Polito, Carolina; Alaimo, Cristina

Abstract: Biometric technologies are complex hardware and software infrastructures that l ... Expand

Abstract: Biometric technologies are complex hardware and software infrastructures that link biometric data such as fingerprints, iris scans, face scans, or DNA data with personal data. A handful of foreign private actors have implemented biometric solutions in more than half of African countries. This paper investigates the politics of biometric artifacts, it looks at how biometric data furnish the basis for the emergence and institutionalization of certain political discourses and power configurations. To this aim, we link the study of biometric data artifacts to the role of private contractors and the full-scale involvement of public institutions in the establishment of border control markets. The empirical context of the research is the work practices of the actors involved in the export of biometric technologies for border security solutions in Namibia. Preliminary findings suggest that the technological and political rationalities of biometric solutions introduce a set of novel problems in the making and management of data profiles. Border control as a political issue seems to be increasingly intermeshed with a logic of economic profit and technological efficiency raising questions of data justice and political accountability. Collapse

Topics: biometric database system datafication algorithmic bias IT work

Methods: biometric measurement case study literature study qualitative interview

Theories: theory of affordance

Perceived Fairness of Human Managers Compared with Artificial Intelligence in Employee Performance Evaluation

2023 | Journal of Management Information Systems | Citations: 0

Authors: Qin, Shaojun; Jia, Nan; Luo, Xueming; Liao, Chengcheng; Huang, Ziyao

Abstract: Human managers are increasingly challenged by artificial intelligence (AI) techn ... Expand

Abstract: Human managers are increasingly challenged by artificial intelligence (AI) technologies in performing managerial functions. We undertook a field experiment that used AI vis-à-vis human managers to perform structured, data-intensive evaluations of employee performance. We generate two sets of insights. First, employees considered AI to be both fairer and more accurate in evaluating their performance than the average human manager. Second, to catch up with AI, human managers' fairness perceived by employees played a first-order role by (a) helping human managers, to a greater extent than those managers' evaluation accuracy, to close the performance gap of the employees evaluated by them compared with that of those evaluated by AI, and (b) constraining the effect of human managers' perceived accuracy of evaluations on employees' performance. Thus, facing the competition from AI, it is all the more important for human managers to treat employees fairly and build positive interpersonal relationships with employees. Collapse

Topics: artificial intelligence individual productivity implicit knowledge electronic finance organization learning

Methods: experiment survey field experiment computational algorithm survey design

Theories: organizational theory behavioral theory

Fairness of E-Commerce Platform Product Recommendations: Understanding Customers’ Perceived Fairness and Equity Sensitivity

2023 | Americas Conference on Information Systems | Citations: 0

Authors: Sharma, Sachin; Singh, Vivek; Joshi, Kailash

Abstract: Recommendation systems (RSs) are artificial intelligence (AI) algorithms that u ... Expand

Abstract: Recommendation systems (RSs) are artificial intelligence (AI) algorithms that utilize customers' product interests or preferences in their response to product search requests. However, AI-powered RSs have been investigated for perpetuating biases. This research extends the theoretical understanding of how customers perceive the fairness of RSs and the factors that impact this perception. Although equity theory has been studied in the management literature to explain the variations in individuals' reactions towards (un)fairness of managerial and organizational decisions, its effect on the perceived fairness of AI systems such as RSs has yet to be studied. Our study extends the extant research by 1) examining how customers' sensitivity to equity impact their perceived fairness of RSs and 2) how the perceived fairness of such systems influences customers' satisfaction, attitudinal, and behavioral loyalties toward e-commerce platforms recommendation systems. Further, our study will help e-commerce managers better understand customers' psychological profiles and predict their responses to RSs (un)fairness. KeywordsRecommendation systems, equity sensitivity, perceived fairness, customer satisfaction, attitudinal and behavioral loyalty. Collapse

Topics: electronic commerce recommender system customer satisfaction algorithmic bias IS impact

Methods: computational algorithm theory development theoretical contribution literature study

Theories: equity theory

Mitigating Bias in Organizational Development and Use of Artificial Intelligence

2023 | International Conference on Information Systems | Citations: 0

Authors: Tanriverdi, Hüseyin; Akinyemi, John-Patrick; Neumann, Terrence

Abstract: We theorize why some artificial intelligence (AI) algorithms unexpectedly treat ... Expand

Abstract: We theorize why some artificial intelligence (AI) algorithms unexpectedly treat protected classes unfairly. We hypothesize that mechanisms by which AI assumes agencies, rights, and responsibilities of its stakeholders can affect AI bias by increasing complexity and irreducible uncertainties: e.g., AI's learning method, anthropomorphism level, stakeholder utility optimization approach, and acquisition mode (make, buy, collaborate). In a sample of 726 agentic AI, we find that unsupervised and hybrid learning methods increase the likelihood of AI bias, whereas "strict" supervised learning reduces it. Highly anthropomorphic AI increases the likelihood of AI bias. Using AI to optimize one stakeholder's utility increases AI bias risk, whereas jointly optimizing the utilities of multiple stakeholders reduces it. User organizations that co-create AI with developer organizations instead of developing it in-house or acquiring it off-the-shelf reduce AI bias risk. The proposed theory and the findings advance our understanding of responsible development and use of agentic AI. Collapse

Topics: artificial intelligence algorithmic bias Google+ database system algorithmic transparency

Methods: computational algorithm supervised learning survey design optimization model theory development

How Does AI Fail Us? A Typological Theorization of AI Failures

2023 | International Conference on Information Systems | Citations: 0

Authors: Zhan, Xinhui; Sun, Heshan; Miranda, Shaila M.

Abstract: AI incidents, often resulting from the complex interplay of algorithms, human a ... Expand

Abstract: AI incidents, often resulting from the complex interplay of algorithms, human agents, and situations, violate norms and can cause minor or catastrophic errors. This study systematically examines these incidents by developing a typology of AI failure and linking these modes to AI task types. Using a computationally intensive grounded theory approach, we analyzed 466 unique reported real-world AI incidents from 2013 to 2023. Our findings reveal an AI failure typology with six modes, including artifact malfunction, artifact misuse, algorithmic bias, agency oversight, situational unresponsiveness, and value misalignment. Furthermore, we explore the relationship between these failure modes and the tasks performed by AI, uncovering four propositions that provide a framework for future research. Our study contributes to the literature by offering a more holistic perspective on the challenges faced by AI-powered systems, beyond the critical challenges of fairness, transparency, and responsibility noted by the literature. Collapse

Topics: artificial intelligence algorithmic bias database system Python missing data

Methods: qualitative coding computational algorithm grounded theory machine learning spatial analysis

Integrating Behavioral, Economic, and Technical Insights to Understand and Address Algorithmic Bias: A Human-Centric Perspective

2022 | ACM Transactions on Management Information Systems | Citations: 0

Authors: Adomavicius, Gediminas; Yang, Mochen

Abstract: Many important decisions are increasingly being made with the help of informatio ... Expand

Abstract: Many important decisions are increasingly being made with the help of information systems that use artificial intelligence and machine learning models. These computational models are designed to discover useful patterns from large amounts of data, which augment human capabilities to make decisions in various application domains. However, there are growing concerns regarding the ethics challenges faced by these automated decision-making (ADM) models, most notably on the issue of algorithmic bias, in which the models systematically produce less favorable (i.e., unfair) decisions for certain groups of people. In this commentary, we argue that algorithmic bias is not just a technical (e.g., computational or statistical) problem, and its successful resolution requires deep insights into individual and organizational behavior, economic incentives, as well as complex dynamics of the sociotechnical systems in which the ADM models are embedded. We discuss a human-centric, fairness-aware ADM framework that highlights the holistic involvement of human decision makers in each step of ADM. We review the emerging literature on fairness-aware machine learning and then discuss various strategic decisions that humans need to make, such as formulating proper fairness objectives, recognizing fairness-induced trade-offs and implications, utilizing machine learning model outputs, and managing/governing the decisions of ADM models. We further illustrate how these strategic decisions are jointly informed by behavioral, economic, and design sciences. Our discussions reveal a number of future research opportunities uniquely suitable for Management Information Systems (MIS) researchers to pursue. Collapse

Topics: algorithmic bias artificial intelligence decision making innovation management reference discipline

Methods: machine learning cross sectional research supervised learning conceptual modelling binary classification

Theories: organizational behavior theory socio technical theory sociotechnical systems theory

Research Perspectives: Toward Theoretical Rigor in Ethical Analysis: The Case of Algorithmic Decision-Making Systems

2022 | Journal of the Association for Information Systems | Citations: 0

Authors: Gal, Uri; Hansen, Sean; Lee, Allen

Abstract: Algorithmic decision-making systems (ADMS) are increasingly being used by publi ... Expand

Abstract: Algorithmic decision-making systems (ADMS) are increasingly being used by public and private organizations to enact decisions traditionally made by human beings across a broad range of domains, including business, law enforcement, education, and healthcare. Their growing prevalence engenders profound ethical challenges, which, we maintain, should be examined in a structured and theoretically informed fashion. However, much of the ethical exploration of ADMS within the IS field draws upon an atheoretical application of ethics. In this paper, we argue that the "big three" ethical theories of consequentialism, deontology, and virtue ethics can inform a structured comparative analysis of the ethical significance of ADMS. We demonstrate the value of such an approach through an illustrative case study of an ADMS in use by an Australian bank. Building upon this analysis, we address four characteristics of ADMS from the three theoretical perspectives, provide guidance on the contexts within which the application of each theory might be particularly fruitful, and highlight the advantages of theoretically grounded ethical analyses of ADMS. Collapse

Topics: database system IT workforce algorithmic bias big data business model

Methods: computational algorithm literature sample literature study experiment theory development

Theories: ethical theory