Results

Found 24 articles

Clear filters

A Signal Detection Theory Approach for Predicting Immunity to Pandemic Vaccine Fake News

2023 | HICSS | Citations: 0
Authors: Barajas, J.; John, R.

Abstract: Managing a contagious disease pandemic, such as , requires that the public unde ... Expand

Abstract: Managing a contagious disease pandemic, such as , requires that the public understand and cooperate with behavioral guidelines to reduce viral transmission. This research uses Signal Detection Theory (SDT) to explore how U.S. adults distinguish between true and false information related to COVID-19 vaccines. A total of 372 U.S. adults categorized 17 true and 17 false COVID-19 vaccine headlines. Item Response Theory analyses suggest that the ability to identify true information about the pandemic is a construct that can be reliably measured with our novel methodology. Signal Detection Theory analyses indicate high accuracy (AUC = 0.861), with no bias favoring either true or false responses. Overall, U.S. adults correctly classified 7 of 10 true and 8 of 10 false headlines. Multiple regression analyses on individual performance metrics reveal substantially lower accuracy among conservatives and those with lower scores on a measure of Actively Open-minded Thinking. Implications and limitations of these findings within the pandemic news context are discussed. Collapse

Semantic filters: signal detection theory

Topics: misinformation fake news pandemic social media covid 19

Methods: machine learning survey multiple linear regression survey design regression analysis method

Theories: signal detection theory
Learning not to take the bait: a longitudinal examination of digital training methods and overlearning on phishing susceptibility
PDF
2023 | European Journal of Information Systems | Citations: 0
Authors: Nguyen, Christopher; Jensen, Matthew; Day, Eric

Abstract: As phishing becomes increasingly sophisticated and costly, interventions that im ... Expand

Abstract: As phishing becomes increasingly sophisticated and costly, interventions that improve and prolong resistance to attacks are needed. Previous research supported digital training as a method to reduce phishing susceptibility. However, the effects of training degrade with time. Therefore, we investigate overlearning as an approach that may increase skill retention through repetition and developing automaticity. We performed a longitudinal experiment crossing overlearning with anti-phishing digital training (rule-based, mindfulness, and control). Participants were tested using email identification tests (immediately following and 10 weeks after training) and mock phishing messages delivered to their inboxes (1 week and 8 weeks following training). Results showed that compared to rule-based training, mindfulness training resulted in significantly greater retention in terms of better email discrimination and less susceptibility to phishing attacks but similar levels of caution towards phishing after 2 months. Overlearning resulted in significantly less susceptibility to phishing attacks and more caution towards phishing compared to no overlearning but did not impact the digital training approaches. Even so, mindfulness was more beneficial compared to overlearning. Altogether, the results demonstrate the stability of the benefits of mindfulness training over time in terms of mitigating phishing susceptibility without influencing the chances of missing legitimate emails. Collapse

Semantic filters: signal detection theory

Topics: phishing electronic mail mindfulness email phishing self efficacy

Methods: survey longitudinal research experiment survey design longitudinal experiment

Theories: signal detection theory big five model
Phish Finders: Improving Cybersecurity Training Tools Using Citizen Science

2022 | International Conference on Information Systems | Citations: 0
Authors: Ahuja, Vinod; Rosser, Holly; Grover, Andrea; Hale, Matthew

Abstract: Malicious web content includes phishing emails, social media posts, and website ... Expand

Abstract: Malicious web content includes phishing emails, social media posts, and websites that imitate legitimate sites. Phishing attacks are rising, and human-centered phishing risk mitigation is often an afterthought eclipsed by technical system-centric efforts like firewalls. Training tools can be deployed for combating phishing but often lack sufficient labeled training content. Using signal detection theory, this paper assesses the feasibility of using citizen science and crowdsourcing volunteers to label images for use in cybersecurity training tools. Crowd volunteer performance was compared to gold standard content and prior studies of Fortune 500 company employees. Findings show no significant statistical differences between crowd volunteers and corporate employees' performance on gold standard content in identifying phishing. Based on these findings, citizen scientists can be valuable for generating annotated images for cybersecurity training tools. Collapse

Semantic filters: signal detection theory

Topics: phishing citizen science IT security crowdsourcing IT workforce

Methods: nonparametric test machine learning Mann Whitney U test experiment survey design

Theories: signal detection theory
MEET YOUR NEW COLLE(AI)GUE – EXPLORING THE IMPACT OF HUMAN-AI INTERACTION DESIGNS ON USER PERFORMANCE

2022 | European Conference On Information Systems | Citations: 0
Authors: Braun, Marvin; Greve, Maike; Riquel, Johannes; Brendel, Alfred Benedikt; Kolbe, Lutz

Abstract: Artificial Intelligence (AI) has an increasing impact on industries, establishi ... Expand

Abstract: Artificial Intelligence (AI) has an increasing impact on industries, establishing a new way of solving tasks and automating work routines. While AI-based systems have become new colleagues for some processes, the tasks of some humans have shifted towards supervising AI. Essentially, humans need to adapt to a new form of interaction with AI-based systems because AI functioning is more similar to cognitive processes of humans than traditional information systems, e.g., in terms of their intransparent decision making. Previous research indicates that AI adds new challenges to human-computer interaction, and new frameworks for human-AI interaction are developed. However, current research lacks empirical research on the design of such interactions. We conducted a 2x2x2 experiment of AIsupported information extraction and measured the ability of participants to validate the extracted information by the AI. Our results indicate that the design of human-AI interaction significantly impacts users' supervising performance.interaction and the current state of research. Second, the SDT is explained and applied to derive hypotheses regarding human-AI interaction. Collapse

Semantic filters: signal detection theory

Topics: artificial intelligence human AI collaboration decision support system human computer interaction human resource management

Methods: experiment ANOVA parametric test information extraction theoretical contribution

Theories: signal detection theory
Tell me the Truth: Separating Fact from Fiction in Social Media Following Extreme Events
PDF
2021 | HICSS | Citations: 1
Authors: Byrd, Katie; John, Richard

Abstract: With increased reliance on social media to spread important information during e ... Expand

Abstract: With increased reliance on social media to spread important information during extreme events, users’ reported inability to distinguish fact from fiction is a growing concern. This experiment (N=398) tests whether feedback training improves performance in identifying true and false social media content during extreme events. Respondents completed two sets of 16 binary classification judgments (true or false) of actual social media posts following either natural disasters or soft-target terror attacks. Respondents randomly assigned to the feedback training condition received feedback after each of the 16 training judgments, while those assigned to the control condition did not receive any feedback following the training judgments. Feedback training did not increase social media content classification performance for either natural disasters or soft-target terror events. Individuals’ performance for correctly identifying false content was negatively related to selfidentified political conservatism and was positively related to a measure of cognitive reflection. Collapse

Semantic filters: signal detection theory

Topics: social media natural hazard misinformation fake news deception detection

Methods: machine learning regression analysis method experimental group psychometrics self reported survey

Theories: signal detection theory big five model
Too Busy to Be Manipulated: How Multitasking with Technology Improves Deception Detection in Collaborative Teamwork
PDF
2020 | Journal of Management Information Systems | Citations: 2
Authors: Twyman, Nathan W.; Proudfoot, Jeffrey G.; Cameron, Ann-Frances; Case, Eric; Burgoon, Judee K.; Twitchell, Douglas P.

Abstract: Deception is an unfortunate staple in group work. Guarding against team members’ ... Expand

Abstract: Deception is an unfortunate staple in group work. Guarding against team members’ deceptive tactics and alternative agendas is difficult and may seem even more difficult in technology-driven business environments that have made multitasking during teamwork increasingly commonplace. This research develops a foundation for a nuanced theoretical understanding of deception detection under these conditions. The intersection of information technology multitasking and deception detection theories is shown to produce various and sometimes competing ideas about how this type of multitasking might affect truthfulness assessments in real-time teamwork. A laboratory study involving a collaborative game helped evaluate the different ideas using manipulated deception and multitasking behaviors in a real-time, virtual group environment. The results provide evidence that information multitasking can actually improve deception detection, likely because multitaskers engage less in the team conversation, making themselves less manipulable. As understanding of multitasking benefits increases, managers and designers can incorporate effective multitasking into collaborative processes. Collapse

Semantic filters: signal detection theory

Topics: deception detection personality task productivity decision making

Methods: regression analysis method mediation analysis survey design survey structural equation modeling

Theories: signal detection theory
Advancing the assessment of automated deception detection systems: Incorporating base rate and cost into system evaluation
PDF
2019 | Information Systems Journal | Citations: 2
Authors: Twitchell, Douglas P.; Fuller, Christie M.

Abstract: In the last two decades, there has been an increased interest in automated decep ... Expand

Abstract: In the last two decades, there has been an increased interest in automated deception detection systems (ADDs) for use in screening, although little attention has been paid to the usefulness of these systems. ADDs use various means, both invasive and non‐invasive, to ascertain individual intent to deceive or engage in malicious behaviour. Many papers introducing ADDs use signal detection theory to compare a technique's ability to detect malicious intent with other techniques, but in doing so, they do not include contextual information such as base rate and cost. In this paper, we aim to improve future research by showing how the inclusion of contextual information provides a more realistic picture of the research. Through both theoretical arguments and a real‐data example, we show that especially for those contexts where malicious intent is infrequent (ie, with low base rates of deception) that not factoring in the base rate overestimates the accuracy and therefore usefulness. We conclude with recommendations for how future research should provide a fuller picture of the accuracy and usefulness of ADDs. Collapse

Semantic filters: signal detection theory

Topics: deception detection crowdfunding crowdsourcing criminality insider threat

Methods: machine learning support vector machine Bayesian analysis literature study within subject experiment

Theories: expected utility theory signal detection theory
Fake News on Social Media: The (In)Effectiveness of Warning Messages

2018 | International Conference on Information Systems | Citations: 16
Authors: Ross, Björn; Heisel, Jennifer; Jung, Anna-Katharina; Stieglitz, Stefan

Abstract: Warning messages are being discussed as a possible mechanism to contain the circ ... Expand

Abstract: Warning messages are being discussed as a possible mechanism to contain the circulation of false information on social media. Their effectiveness for this purpose, however, is unclear. This article describes a survey experiment carried out to test two designs of warning messages: a simple one identical to the one used by Facebook, and a more complex one informed by recent research. We find no evidence that either design is clearly superior to not showing a warning message. This result has serious implications for brands and politicians, who might find false information about them spreading uncontrollably, as well as for managers of social media platforms, who are struggling to find effective means of controlling the diffusion of misinformation. Collapse

Semantic filters: signal detection theory

Topics: fake news Facebook social media social network misinformation

Methods: survey ANOVA parametric test descriptive statistic statistical power analysis

Theories: signal detection theory elaboration likelihood model
Designing Warning Messages for Detecting Biased Online Product Recommendations: An Empirical Investigation
PDF
2015 | Information Systems Research | Citations: 54
Authors: Xiao, Bo; Benbasat, Izak

Abstract: The increasing adoption of product recommendation agents (PRAs) by e-commerce me ... Expand

Abstract: The increasing adoption of product recommendation agents (PRAs) by e-commerce merchants makes it an important area of study for information systems researchers. PRAs are a type of Web personalization technology that provides individual consumers with product recommendations based on their product-related needs and preferences expressed explicitly or implicitly. Whereas extant research mainly assumes that such recommendation technologies are designed to benefit consumers and focuses on the positive impact of PRAs on consumers’ decision quality and decision effort, this study represents an early effort to examine PRAs that are designed to produce their recommendations on the basis of benefiting e-commerce merchants (rather than benefiting consumers) and to investigate how the availability and the design of warning messages (a potential detection support mechanism) can enhance consumers’ performance in detecting such biased PRAs. Drawing on signal detection theory, the literature on warning messages, and the literature on message framing, we identified two content design characteristics of warning messages—the inclusion of risk-handling advice and the framing of risk-handling advice—and investigated how they influence consumers’ detection performance. The results of an online experiment reveal that a simple warning message without accompanying advice on how to detect bias is a double-edged sword, because it increases correct detection of biased PRAs (hits) at the cost of increased incorrect detection (false alarms). By contrast, including in warning messages risk-handling advice about how to check for bias (particularly when the advice is framed to emphasize the loss from not following the advice) increases correct detection and, more importantly, also decreases incorrect detection. The patterns of findings are in line with the predictions of signal detection theory. With an enriched understanding of how the availability and the content design of warning messages can assist consumers in the context of PRA-assisted online shopping, the results of this study serve as a basis for future theoretical development and yield valuable insights that can guide practice and the design of effective warning messages. Collapse

Semantic filters: signal detection theory

Topics: electronic commerce website marketing management electronic business decision making

Methods: experiment parametric test ANOVA experimental task analysis of covariance

Theories: signal detection theory
An Efficient Frontier in Organization Design: Organizational Structure as a Determinant of Exploration and Exploitation
PDF
2013 | Organization Science | Citations: 33
Authors: Csaszar, Felipe A.

Abstract: This paper develops a parsimonious process-level theory that connects organizati ... Expand

Abstract: This paper develops a parsimonious process-level theory that connects organizational structure to exploration and exploitation. Toward this end, it develops a mathematical model of organizational decision making that combines an information processing approach in the spirit of Sah and Stiglitz [Sah RK, Stiglitz JE (1986) The architecture of economic systems: Hierarchies and polyarchies. Amer. Econom. Rev. 76(4):716–727] with elements from signal detection theory. The model is first used to explore a “design space” of organizations and identify trade-offs and dominance relationships among alternative organization designs. The paper then studies open questions in the organization design literature, such as the extent to which exploration and exploitation can be produced by one organization and what is the effect of organization size on exploration. More broadly, this research speaks to calls for the introduction of more process-level explanations in the organizations literature. The paper concludes with testable hypotheses and managerially relevant insights. Collapse

Semantic filters: signal detection theory

Topics: organization structure innovation management decision making Google+ anonymity

Methods: computational algorithm Markov chain Monte Carlo robustness check mathematical model descriptive statistic

Theories: signal detection theory decision theory

Page 1 of 3 Next

AIS11

Journals

Conferences

Semantic Filter

Found 24 articles

A Signal Detection Theory Approach for Predicting Immunity to Pandemic Vaccine Fake News

Learning not to take the bait: a longitudinal examination of digital training methods and overlearning on phishing susceptibility

Phish Finders: Improving Cybersecurity Training Tools Using Citizen Science

MEET YOUR NEW COLLE(AI)GUE – EXPLORING THE IMPACT OF HUMAN-AI INTERACTION DESIGNS ON USER PERFORMANCE

Tell me the Truth: Separating Fact from Fiction in Social Media Following Extreme Events

Too Busy to Be Manipulated: How Multitasking with Technology Improves Deception Detection in Collaborative Teamwork

Advancing the assessment of automated deception detection systems: Incorporating base rate and cost into system evaluation

Fake News on Social Media: The (In)Effectiveness of Warning Messages

Designing Warning Messages for Detecting Biased Online Product Recommendations: An Empirical Investigation

An Efficient Frontier in Organization Design: Organizational Structure as a Determinant of Exploration and Exploitation