Augmenting Social Bot Detection with Crowd-Generated Labels

2023 | Information Systems Research | Citations: 0

Authors: Benjamin, Victor; Raghu, T. S.

Abstract: Social media platforms are facing increasing numbers of cyber-adversaries seekin ... Expand

Abstract: Social media platforms are facing increasing numbers of cyber-adversaries seeking to manipulate online discourse by using social bots (i.e., social media software robots) to help automate and scale their attacks. Likewise, some social media users can identify social bot activity at varying degrees of confidence. In this research, human reactions to social bot messages are used to augment existing social bot detection capabilities. Speech act theory is used to inspire a framework for assessing the credibility of instances where users identify potential bot activity, as not all user responses are of equal credibility for assisting with the bot detection task. The framework is then operationalized through deep learning methodologies to develop a computational system for identifying social bots. Real-world performance and practicality of the developed framework is demonstrated on a live, crowd-sourced data set collected from a real-world social media platform. Results show that consideration of crowd reactions to suspected bots can significantly improve bot detection performance. Furthermore, consideration of speech acts to evaluate crowd reactions can even further augment the system’s performance, although speech acts themselves are not necessary to observe performance boost through crowd intelligence. This study serves as a grounding point for future work that can explore an augmented model for detecting other forms of algorithmically generated content within social media platforms.History: Olivia Sheng, Senior Editor; Jesse Bockstedt, Associate Editor.Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2022.1136. Collapse

Topics: Reddit social media crowdsourcing IT productivity social media analytics

Methods: experiment BERT machine learning support vector machine random forest classification

Theories: speech act theory

Interpretable cost-sensitive regression through one-step boosting

2023 | Decision Support Systems | Citations: 0

Authors: Decorte, Thomas; Raymaekers, Jakob; Verdonck, Tim

Abstract: In most practical prediction problems, such as regression and classification, th ... Expand

Abstract: In most practical prediction problems, such as regression and classification, the different types of prediction errors are not equally costly in the decision-making process. Although there exist numerous real-world costsensitive regression problems, ranging from loan charge-off forecasting to house price predictions, the literature on cost-sensitive learning mainly focuses on classification and only a few solutions are proposed for regression problems. These regressions are typically characterized by an asymmetric cost structure, where over- and underpredictions of a similar magnitude face vastly different costs. In this paper, we present a one-step boosting method (OSB) for cost-sensitive regression. The proposed methodology leverages a secondary learner to incorporate cost-sensitivity into an already trained cost-insensitive regression model. The secondary learner is defined as a linear function of certain variables deemed interesting for cost-sensitivity. These variables do not necessarily need to be the same as in the already trained model. An efficient optimization algorithm is achieved through iteratively reweighted least squares using the asymmetric cost function. The obtained results become interpretable through bootstrapping, enabling decision makers to distinguish important variables for cost-sensitivity as well as facilitating statistical inference. Applying different cost functions and various initial cost-insensitive learning methods on several public datasets consistently yields a significant reduction in the average misprediction cost, illustrating the excellent performance of our approach. Collapse

Topics: Python decision making ease of use operating system Microsoft Windows

Methods: computational algorithm regression analysis method LightGBM boosting method linear regression analysis

Near-Infrared Spectroscopy for Bladder Monitoring: A Machine Learning Approach

2023 | ACM Transactions on Management Information Systems | Citations: 0

Authors: Fechner, Pascal; König, Fabian; Kratsch, Wolfgang; Lockl, Jannik; Röglinger, Maximilian

Abstract: Patients living with neurogenic bladder dysfunction can lose the sensation of th ... Expand

Abstract: Patients living with neurogenic bladder dysfunction can lose the sensation of their bladder filling. To avoid over-distension of the urinary bladder and prevent long-term damage to the urinary tract, the gold standard treatment is clean intermittent catheterization at predefined time intervals. However, the emptying schedule does not consider actual bladder volume, meaning that catheterization is performed more often than necessary, which can lead to complications such as urinary tract infections. Time-consuming catheterization also interferes with patients' daily routines and, in the case of an empty bladder, uses human and material resources unnecessarily. To enable individually tailored and volume-responsive bladder management, we design a model for the continuous monitoring of bladder volume. During our design science research process, we evaluate the model's applicability and usefulness through interviews with affected patients, prototyping, and application to a real-world in vivo dataset. The developed prototype predicts bladder volume based on relevant sensor data (i.e., near-infrared spectroscopy and acceleration) and the time elapsed since the previous micturition. Our comparison of several supervised state-of-the-art machine and deep learning models reveals that a long short-term memory network architecture achieves a mean absolute error of 116.7 ml that can improve bladder management for patients. Collapse

Topics: computer hardware open source mobile application user interface programming language

Methods: design artifact qualitative interview design process support vector machine computational algorithm

Chatbot Catalysts: Improving Team Decision-Making Through Cognitive Diversity and Information Elaboration

2023 | International Conference on Information Systems | Citations: 0

Authors: Gurkan, Necdet; Yan, Bei

Abstract: As the integration of artificial intelligence (AI) into team decision-making co ... Expand

Abstract: As the integration of artificial intelligence (AI) into team decision-making continues to expand, it is both theoretically and practically pressing for researchers to understand the impact of the technology on team dynamics and performance. To investigate this relationship, we conducted an online experiment in which teams made decisions supported by chatbots and employed computational methods to analyze team interaction processes. Our results indicated that compared to those assisted by chatbots in later phases, teams receiving chatbot assistance during the initial phase of their decisionmaking process exhibited increased cognitive diversity (i.e., diversity in shared information) and information elaboration (i.e., exchange and integration of information). Ultimately, teams assisted by chatbots early on performed better. These results imply that introducing AI at the beginning of the process can enhance team decision-making by promoting effective information sharing among team members. Collapse

Topics: artificial intelligence decision making information exchange team productivity human AI collaboration

Methods: chatbot parametric test experiment ANOVA self reported survey

Early Chatbot Assistance Can Enhance Team Decision-Making by Promoting Cognitive Diversity and Information Elaboration

2023 | Americas Conference on Information Systems | Citations: 0

Authors: Gurkan, Necdet; Yan, Bei

Abstract: As AI increasingly assists teams in decision-making, the study examines how tec ... Expand

Abstract: As AI increasingly assists teams in decision-making, the study examines how technology shapes team processes and performance. We conducted an online experiment of team decision-making assisted by chatbots and analyzed team interaction processes with computational methods. We found that teams assisted by a chatbot offering information in the first half of their decision-making process performed better than those assisted by the chatbot in the second half. The effect was explained by the variation in teams' information-sharing process between the two chatbot conditions. When assisted by the chatbot in the first half of the decision-making task, teams showed higher levels of cognitive diversity (i.e., the difference in the information they shared) and information elaboration (i.e., exchange and integration of information). The findings demonstrate that if introduced early, AI can support team decision-making by acting as a catalyst to promote team information sharing. Collapse

Topics: decision making artificial intelligence team productivity information exchange video conference

Methods: chatbot parametric test ANOVA experiment two-way ANOVA

Explanation matters: An experimental study on explainable AI

2023 | Electronic Markets | Citations: 0

Authors: Hamm, Pascal; Klesel, Michael; Coberger, Patricia; Wittmann, H. Felix

Abstract: Explainable artificial intelligence (XAI) is an important advance in the field o ... Expand

Abstract: Explainable artificial intelligence (XAI) is an important advance in the field of machine learning to shed light on black box algorithms and thus a promising approach to improving artificial intelligence (AI) adoption. While previous literature has already addressed the technological benefits of XAI, there has been little research on XAI from the user’s perspective. Building upon the theory of trust, we propose a model that hypothesizes that post hoc explainability (using Shapley Additive Explanations) has a significant impact on use-related variables in this context. To test our model, we designed an experiment using a randomized controlled trial design where participants compare signatures and detect forged signatures. Surprisingly, our study shows that XAI only has a small but significant impact on perceived explainability. Nevertheless, we demonstrate that a high level of perceived explainability has a strong impact on important constructs including trust and perceived usefulness. A post hoc analysis shows that hedonic factors are significantly related to perceived explainability and require more attention in future research. We conclude with important directions for academia and for organizations. Collapse

Topics: explainable artificial intelligence artificial intelligence perceived ease of use perceived usefulness digital dashboard

Methods: experimental group experiment parametric test experimental task ANOVA

Theories: information systems theory trust theory

What Do They Meme? Exploring the Role of Memes as Cultural Symbols of Online Communities

2023 | International Conference on Information Systems | Citations: 0

Authors: Henn, Theresa; Posegga, Oliver

Abstract: Analyzing symbols shared within online communities (OCs) is essential to better ... Expand

Abstract: Analyzing symbols shared within online communities (OCs) is essential to better understand communities' expressed cultures. To evaluate how OCs differ in their expressed culture and analyze the effects of community rules (CR) and moderation policies (MP), we examined meme sharing of subreddit and interaction communities on Reddit. To detect memes shared within subreddits automatically, we trained a convolutional neural network and applied a feature-matching algorithm to create meme networks with components consisting of visually similar memes. Based on each community's component composition, we created community-specific meme languages that we compared across subreddit and interaction communities. Our results show that memes can be aggregated to characteristic meme languages linked to individual OCs; yet MP and CR do not impact the homogeneity of shared memes. Based on these findings, we plan to analyze dynamically the relationship between memes and OCs, examining memes' textual content and diving deeper into users' individual meme languages. Collapse

Topics: Reddit internet technology social media

Methods: computational algorithm nonparametric test Mann Whitney U test convolutional neural network Keras

Improved learning performance for small datasets in high dimensions by new dual-net model for non-linear interpolation virtual sample generation

2023 | Decision Support Systems | Citations: 0

Authors: Lin, Liang-Sian; Lin, Yao-San; Li, Der-Chiang; Liu, Yun-Hsuan

Abstract: The number of reliable samples obtained in early decision-making activity is usu ... Expand

Abstract: The number of reliable samples obtained in early decision-making activity is usually relatively small. Due to variable distribution and incomplete structure of tiny datasets, it is challenging to create reliable and robust predictive modeling using classic statistical and machine learning models in small sample settings. The virtual sample generation (VSG) technique improves model learning accuracies for minimal datasets across diverse applications. Virtual samples on independent variables were generated using established VSG methods predi­ cated on the assumption of a probability distribution or a membership function to fill data gaps. However, in the actual world, non-linear function interactions between variables are common. To address this issue, this paper developed a novel VSG method called Dual-VSG, which generates non-linear interpolation virtual samples using a self-supervised learning (SSL) framework to improve learning performance on small datasets. We generated non-linear interpolation virtual samples without labels by estimating non-linear functions and transforming them into a high-dimensional space using the proposed dual-net model. The weights of the dual-net model are transferred to a downstream task to generate virtual sample labels. To demonstrate the effectiveness of the suggested strategy, this research employed five datasets. On the Backpropagation Neural Networks (BPNN) predictive model, we compared the suggested method’s prediction performance to two state-of-the-art VSG approaches. To assess prediction performance on a regression dataset, the Mean Absolute Percentage Error (MAPE) and the Root Mean Square Error (RMSE) are used. Furthermore, the classification accuracy (ACC) and the Fl measure are used to assess classification capability on classification datasets. In addition, the paired t-test was utilized to see if the suggested Dual-VSG approach differed significantly from the other VSG methods in terms of RMSE, MAPE, accuracy (ACC), or F1 score. For short datasets, the suggested Dual-VSG method out­ performs those VSG methods, according to our experimental results. Collapse

Topics: Python programming language

Methods: data transformation t-distributed stochastic neighbor embedding experiment machine learning parametric test

The Impact of Stigmatizing Language in EHR Notes on AI Performance and Fairness

2023 | International Conference on Information Systems | Citations: 0

Authors: Liu, Yizhi; Wang, Weiguang; Gao, Gordon; Agarwal, Ritu

Abstract: Today, there is significant interest in using electronic health record data to ... Expand

Abstract: Today, there is significant interest in using electronic health record data to generate new clinical insights for diagnosis and treatment decisions. However, there are concerns that such data may be biased and result in accentuating racial disparities. We study how clinician biases reflected in EHR notes affect the performance and fairness of artificial intelligence models in the context of mortality prediction for intensive care unit patients. We apply a Transformer-based deep learning model and explainable AI techniques to quantify negative impacts on performance and fairness. Our findings demonstrate that stigmatizing language written by clinicians adversely affects AI performance, particularly so for black patients, highlighting SL as a source of racial disparity in AI model development. As an effective mitigation approach, removing SL from EHR notes can significantly improve AI performance and fairness. This study provides actionable insights for responsible AI development and contributes to understanding clinician EHR note writing. Collapse

Topics: artificial intelligence explainable artificial intelligence open source moral hazard Microsoft Windows

Methods: transformer model machine learning BERT deep learning experiment

A decision support framework to incorporate textual data for early student dropout prediction in higher education

2023 | Decision Support Systems | Citations: 0

Authors: Phan, Minh; De Caigny, Arno; Coussement, Kristof

Abstract: Managing student dropout in higher education is critical, considering its substa ... Expand

Abstract: Managing student dropout in higher education is critical, considering its substantial impacts on students’ lives, academic institutions, and society as a whole. Using predictive modeling can be instrumental for this task, as a means to identify dropouts proactively on the basis of student characteristics and their academic performance. To enhance these predictions, textual student feedback also might be relevant; this article proposes a hybrid decision support framework that combines predictive modeling with student segmentation efforts. A real-life data set from a French higher education institution, containing information of 14,391 students and 62,545 feedback documents, confirms the superior performance of the proposed framework, in terms of the area under the curve and top decile lift, compared with various benchmarks. In contributing to decision support system research, this study (1) proposes a new framework for automatic, data-driven segmentation of students based on textual data; (2) compares multiple text representation methods and confirms that incorporating student textual feedback data improves the predictive performance of student dropout models; and (3) establishes useful insights to help decision-makers anticipate and manage student dropout behaviors. Collapse

Topics: decision support decision support system electronic mail

Methods: machine learning doc2vec data transformation BERT computational algorithm