Benchmarking the Robustness of Phishing Email Detection Systems

2023 | Americas Conference on Information Systems | Citations: 0

Authors: Ampel, Benjamin; Gao, Yang; Hu, James; Samtani, Sagar; Chen, Hsinchun

Abstract: Social engineering attacks are currently the most cited cybersecurity threat to ... Expand

Abstract: Social engineering attacks are currently the most cited cybersecurity threat to organizations. Phishing emails are the most salient form of social engineering attacks. Organizations are increasingly implementing AI-enabled systems to detect phishing emails. However, AI-enabled systems are often susceptible to textual perturbations, where an adversary makes a small change to cause a misclassification. In this study, we sought to identify the performance of prevailing phishing email detection systems (PEDS) against character, word, sentence, and multi-level adversarial text perturbations. Through a principled benchmarking framework, we quantitatively demonstrated the lack of robustness prevailing PEDS have to specific types of text-based adversarial perturbations (e.g., character, word, sentence, multi-level). The results of this study provide new insights into the robustness of AI-based PEDS and highlight the need for organizations to adopt a multi-layered approach to phishing protection. Additionally, organizations can implement our benchmark framework to test their PEDS against adversarial perturbations. Collapse

Topics: artificial intelligence electronic mail email phishing phishing IT security

Methods: word2vec random forest classification naïve bayes literature study pre-trained language model

Give me 3W1H: A Bibliometric View on Accountable AI

2023 | International Conference on Information Systems | Citations: 0

Authors: Bartsch, Sebastian; Schmidt, Jan-Hendrik

Abstract: Accountability is crucial to make stakeholders of Artificial Intelligence (AI)- ... Expand

Abstract: Accountability is crucial to make stakeholders of Artificial Intelligence (AI)-based systems justify their actions, thereby explaining the harm such systems cause to AI users. Due to the importance of accountability in the context of AI, accountability was introduced into IS research through literature reviews. Therefore, while IS research's understanding of accountability covers the necessary depth, it comes at the expense of its essential breadth. Using a bibliometric analysis with 19,978 English-language papers, we shed light on the essential breadth posing three W-and one H-questions (When, What, Whereof, and How). Therefore, we contribute to IS research by highlighting the urgent need to revise existing definitions of accountability in the context of AI and establish them in IS research. We argue that a missing revision leads to non-transferrable findings within IS research. Accordingly, this study serves as a starting point for adapting definitions and creating a shared understanding in IS research. Collapse

Topics: artificial intelligence explainable artificial intelligence legal context database system IT security

Methods: word embedding word2vec bibliometric analysis machine learning supervised learning

AI-Enhanced Hybrid Decision Management

2023 | Business & Information Systems Engineering | Citations: 0

Authors: Bork, Dominik; Ali, Syed Juned; Dinev, Georgi Milenov

Abstract: The Decision Model and Notation (DMN) modeling language allows the precise speci ... Expand

Abstract: The Decision Model and Notation (DMN) modeling language allows the precise specification of business decisions and business rules. DMN is readily understandable by business users involved in decision management. However, as the models get complex, the cognitive abilities of humans threaten manual maintainability and comprehensibility. Proper design of the decision logic thus requires comprehensive automated analysis of e.g., all possible cases the decision shall cover; correlations between inputs and outputs; and the importance of inputs for deriving the output. In the paper, the authors explore the mutual benefits of combining human-driven DMN decision modeling with the computational power of Artificial Intelligence for DMN model analysis and improved comprehension. The authors propose a model-driven approach that uses DMN models to generate Machine Learning (ML) training data and show, how the trained ML models can inform human decision modelers by means of superimposing the feature importance within the original DMN models. An evaluation with multiple real DMN models from an insurance company evaluates the feasibility and the utility of the approach. Collapse

Topics: artificial intelligence explainable artificial intelligence perceived usefulness price management extensible markup language

Methods: computational algorithm decision tree classification experiment conceptual modelling simulation

Editor's Comments: Producing Significant Research

2023 | Management Information Systems Quarterly | Citations: 0

Authors: Burton-Jones, Andrew; Gray, Peter; Majchrzak, Ann

Abstract: ... Expand

Abstract: Collapse

Topics: data quality artificial intelligence filter bubble social media citizen science

Methods: field study machine learning computational algorithm field experiment applicability check

A Bitter Pill to Swallow? The Consequences of Patient Evaluation in Online Health Question-and-Answer Platforms

2023 | Information Systems Research | Citations: 0

Authors: Chen, Chen; Walker, Dylan

Abstract: Online health question-and-answer (Q&A) platforms (OHQPs), where patients post h ... Expand

Abstract: Online health question-and-answer (Q&A) platforms (OHQPs), where patients post health-related questions, evaluate advice from multiple doctors, and direct a bounty (monetary reward) to their most preferred answer, have become a prominent channel for patients to receive medical advice in China. To explore the quality of medical advice on these platforms, we analyzed data on patients’ evaluation of ∼497,000 answers to ∼114,000 questions on one of the most popular OHQPs, 120ask.com, over a three-month period. We assembled a panel of independent physicians and instructed them to evaluate the quality of ∼13,000 answers. We found that the quality of medical advice offered on the platform was high on average, and that low-quality answers were rare (6%). However, our results also indicate that patients lacked the ability to discriminate advice quality. They were as likely to choose the best answer as the worst. The medical accuracy of patient evaluation was worse in critical categories (cancer, internal medicine) and for vulnerable subpopulations (pediatrics). Given that millions of patients seek medical advice from OHQPs in China annually, the social and economic implications of this finding are troubling. To understand how patients evaluate advice, we trained deep neural networks to think like patients, allowing us to identify patients’ positive and negative responses to different heurist cues. Although our results indicate that OHQPs perform well, we identified several concerns that should be addressed through platform design and policy changes. Because the Q&A process lacks peer review mechanisms, signals of advice quality are not conveyed to patients, forcing them to rely on heuristic cues, which cannot effectively guide them toward the best advice. We also found that the platform reputation metric was not correlated with the quality of the advice giver’s advice, may effectively encourage patients to select lesser quality medical advice, and increased the risk of moral hazard for malicious players to intentionally provide less accurate but more agreeable advice for personal gain. Our analysis revealed bad actors on the platform, including drug promoters and spammers. Finally, we found that OHQPs exacerbated care avoidance. We discuss several potential policy changes to address these shortcomings. History: Param Singh, Senior Editor; Idris Adjerid, Associate Editor. Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2022.1158. Collapse

Topics: decision making local interpretable model agnostic explanation digital platform health information system pandemic

Methods: artificial neural network psychometrics survey deep learning natural language processing

Theories: lemon market theory

Near-Infrared Spectroscopy for Bladder Monitoring: A Machine Learning Approach

2023 | ACM Transactions on Management Information Systems | Citations: 0

Authors: Fechner, Pascal; König, Fabian; Kratsch, Wolfgang; Lockl, Jannik; Röglinger, Maximilian

Abstract: Patients living with neurogenic bladder dysfunction can lose the sensation of th ... Expand

Abstract: Patients living with neurogenic bladder dysfunction can lose the sensation of their bladder filling. To avoid over-distension of the urinary bladder and prevent long-term damage to the urinary tract, the gold standard treatment is clean intermittent catheterization at predefined time intervals. However, the emptying schedule does not consider actual bladder volume, meaning that catheterization is performed more often than necessary, which can lead to complications such as urinary tract infections. Time-consuming catheterization also interferes with patients' daily routines and, in the case of an empty bladder, uses human and material resources unnecessarily. To enable individually tailored and volume-responsive bladder management, we design a model for the continuous monitoring of bladder volume. During our design science research process, we evaluate the model's applicability and usefulness through interviews with affected patients, prototyping, and application to a real-world in vivo dataset. The developed prototype predicts bladder volume based on relevant sensor data (i.e., near-infrared spectroscopy and acceleration) and the time elapsed since the previous micturition. Our comparison of several supervised state-of-the-art machine and deep learning models reveals that a long short-term memory network architecture achieves a mean absolute error of 116.7 ml that can improve bladder management for patients. Collapse

Topics: computer hardware open source mobile application user interface programming language

Methods: design artifact qualitative interview design process support vector machine computational algorithm

Taking the Person Seriously: Ethically Aware IS Research in the Era of Reinforcement Learning-Based Personalization

2023 | Journal of the Association for Information Systems | Citations: 0

Authors: Greene, Travis; Shmueli, Galit; Ray, Soumya

Abstract: Advances in reinforcement learning and implicit data collection on large-scale ... Expand

Abstract: Advances in reinforcement learning and implicit data collection on large-scale commercial platforms mark the beginning of a new era of personalization aimed at the adaptive control of human user environments. We present five emergent features of this new paradigm of personalization that endanger persons and societies at scale and analyze their potential to reduce personal autonomy, destabilize social and political systems, and facilitate mass surveillance and social control, among other concerns. We argue that current data protection laws, most notably the European Union's General Data Protection Regulation, are limited in their ability to adequately address many of these issues. Nevertheless, we believe that IS researchers are well-situated to engage with and investigate this new era of personalization. We propose three distinct directions for ethically aware reinforcement learning-based personalization research uniquely suited to the strengths of IS researchers across the sociotechnical spectrum. Collapse

Topics: usability personalization legal context General Data Protection Regulation information privacy law

Methods: reinforcement learning computational algorithm simulation language model experiment

Theories: social sciences theory

Detection of Stock Manipulation Influencer Content using Supervised Learning

2023 | International Conference on Information Systems | Citations: 0

Authors: Haase, Frederic; Rath, Oliver; Lauten, Julia; Schoder, Detlef

Abstract: In recent years, social media influencers have emerged as key players in stock ... Expand

Abstract: In recent years, social media influencers have emerged as key players in stock manipulation schemes. Despite their growing impact, methods to detect such activities remain scarcely explored. In this study, we examine the social media content of stock manipulation influencers (SMIs) implicated in a $100 million fraud case by the U.S. Securities and Exchange Commission (SEC) in 2022. Leveraging natural language processing (NLP) techniques, we first investigate the linguistic characteristics present in the social media content published by SMIs. Next, we develop and evaluate supervised learning models to detect manipulative content. Our results have significant implications for investors, regulators, and the broader financial community. They reveal the unique linguistic characteristics of SMI content and demonstrate the potential of machinelearning and deep-learning-based techniques in advancing fraud detection systems. Collapse

Topics: Twitter social media fraud detection database system social media analytics

Methods: machine learning RoBERTa DistilBERT computational algorithm natural language processing

Co-evolution of neural architectures and features for stock market forecasting: A multi-objective decision perspective

2023 | Decision Support Systems | Citations: 0

Authors: Hafiz, Faizal; Broekaert, Jan; La Torre, Davide; Swain, Akshya

Abstract: In a multi-objective setting, a portfolio manager’s highly consequential decisio ... Expand

Abstract: In a multi-objective setting, a portfolio manager’s highly consequential decisions can benefit from assessing alternative forecasting models of stock index movement. The present investigation proposes a new approach to identify a set of non-dominated neural network models for further selection by the decision-maker. A new coevolution approach is proposed to simultaneously select the features and topology of neural networks (collectively referred to as neural architecture), where the features are viewed from a topological perspective as input neurons. Further, the co-evolution is posed as a multi-criteria problem to evolve sparse and efficacious neural architectures. The well-known dominance and decomposition based multi-objective evolutionary algorithms are augmented with a non-geometric crossover operator to diversify and balance the search for neural architectures across conflicting criteria. Moreover, the co-evolution is augmented to accommodate the data-based implications of distinct market behaviors prior to and during the ongoing COVID-19 pandemic. A detailed comparative evaluation is carried out with the conventional sequential approach of feature selection followed by neural topology design, as well as a scalarized co-evolution approach. The results on three market indices (NASDAQ, NYSE, and S&P500) in pre- and peri-COVID time windows convincingly demonstrate that the proposed co-evolution approach can evolve a set of non-dominated neural forecasting models with better generalization capabilities. Collapse

Topics: covid 19 pandemic decision support system decision making

Methods: computational algorithm artificial neural network data transformation machine learning principal component analysis

Theories: co-evolution theory

Fraud Detection in Mobile Payment Systems using an XGBoost-based Framework

2023 | Information Systems Frontiers | Citations: 0

Authors: Hajek, Petr; Abedin, Mohammad Zoynul; Sivarajah, Uthayasankar

Abstract: Mobile payment systems are becoming more popular due to the increase in the numb ... Expand

Abstract: Mobile payment systems are becoming more popular due to the increase in the number of smartphones, which, in turn, attracts the interest of fraudsters. Extant research has therefore developed various fraud detection methods using supervised machine learning. However, sufficient labeled data are rarely available and their detection performance is negatively affected by the extreme class imbalance in financial fraud data. The purpose of this study is to propose an XGBoost-based fraud detection framework while considering the financial consequences of fraud detection systems. The framework was empirically validated on a large dataset of more than 6 million mobile transactions. To demonstrate the effectiveness of the proposed framework, we conducted a comparative evaluation of existing machine learning methods designed for modeling imbalanced data and outlier detection. The results suggest that in terms of standard classification measures, the proposed semi-supervised ensemble model integrating multiple unsupervised outlier detection algorithms and an XGBoost classifier achieves the best results, while the highest cost savings can be achieved by combining random under-sampling and XGBoost methods. This study has therefore financial implications for organizations to make appropriate decisions regarding the implementation of effective fraud detection systems. Collapse

Topics: fraud detection mobile system mobile payment electronic payment Python

Methods: XGBoost machine learning anomaly detection supervised learning experiment