Understanding the Role of Expert Intuition in Medical Image Annotation: A Cognitive Task Analysis Approach

2023 | HICSS | Citations: 0

Authors: Leiser, F.; Warsinsky, S.; Daum, M.; Schmidt-Kraepelin, M.; Thiebes, S.; Wagner, M.; Sunyaev, A.

Abstract: To improve contemporary machine learning (ML) models, research is increasingly ... Expand

Abstract: To improve contemporary machine learning (ML) models, research is increasingly looking at tapping in and incorporating the knowledge of domain experts. However, expert knowledge often relies on intuition, which is difficult to formalize for incorporation into ML models. Against this backdrop, we investigate the role of intuition in the context of expert medical image annotation. We apply a cognitive task analysis approach, where we observe and interview six expert medical image annotators to gain insights into pertinent decision cues and the role of intuition during annotation. Our results show that intuition plays an important role in various steps of the medical image annotation process, particularly in the appraisals of very easy or very difficult images, and in case purely cognitive appraisals remain inconclusive. Overall, we contribute to a better understanding of expert intuition in medical image annotation and provide possible interfaces to incorporate said intuition into ML models. Collapse

Topics: decision making knowledge representation

Methods: qualitative interview survey self reported survey triangulation cognitive task analysis

Identifying Expertise Gaps in Cyber Incident Response: Cyber Defender Needs vs. Technological Development

2021 | HICSS | Citations: 0

Authors: Nyre-Yu, Megan

Abstract: Incident response is an area within cyber defense that is responsible for detect ... Expand

Abstract: Incident response is an area within cyber defense that is responsible for detecting, mitigating, and preventing threats within a given network. Like other areas of cyber security, incident response is experiencing a shortage of qualified workers which has led to technological development aimed at alleviating labor-related pressures on organizations. A cognitive task analysis was conducted with incident response experts to capture expertise requirements and used an existing construct to help prioritize development of new technology. Findings indicated that current software development incorporates factors such as analyst efficiency and consistency. Gaps were identified regarding communication and team navigation that are inherent to dynamic team environments. This research identified which expertise areas are needed at lower-tier levels of incident response and which of those areas current automation platforms are addressing. These gaps help focus future studies by bridging expertise research to development efforts. Collapse

Topics: spreadsheet information exchange systems development

Methods: qualitative interview descriptive statistic qualitative coding ethnography design science

Developing a Mechanism to Study Code Trustworthiness

2017 | HICSS | Citations: 3

Authors: Walter, Charles; Gamble, Rose; Alarcon, Gene; Jessup, Sarah; Calhoun, Chris

Abstract: When software code is acquired from a third party or version control repository, ... Expand

Abstract: When software code is acquired from a third party or version control repository, programmers assign a level of trust to the code. This trust prompts them to use the code as-is, make minor changes, or rewrite it, which can increase costs and delay deployment. This paper discusses types of degradations to code based on readability and organization expectations and how to present that code as part of a study on programmer trust. Degradations were applied to sixteen of eighteen Java classes that were labeled as acquired from reputable or unknown sources. In a pilot study, participants were asked to determine a level of trustworthiness and whether they would use the code without changes. The results of the pilot study are presented to provide a baseline for the continuance of the study to a larger set of participants and to make adjustments to the presentation environment to improve user experience. Collapse

Topics: Java software developer logistics management web application programming language

Methods: regression analysis method experiment ANOVA parametric test post-hoc analysis

The SEC-system reuse: support for scheduling system development

1999 | Decision Support Systems | Citations: 0

Authors: van Wezel, Wout; Jorna, René J

Abstract: The development process of computer systems for task support of planning and sch ... Expand

Abstract: The development process of computer systems for task support of planning and scheduling is not a unique activity that must be performed from scratch each time. Reuse of previously created conceptual and technical products is possible and recommendable. We claim that adequate computer support starts with a (cognitive) task analysis which reveals general as well as unique aspects. The general aspects can be reused. However, a conceptual framework is necessary that separates general from unique aspects in the task domain and task performance. We propose the Scheduling Expertise Concept (SEC) as such a framework for planning and scheduling. The framework is domain independent and takes into account organisational and task performance characteristics. This generic framework is implemented in a computer system, the SEC-system. The SEC-models and SEC-system enable faster development processes in the planning and scheduling domain. Collapse

Topics: decision support decision support system knowledge representation task productivity problem solving

Methods: survey cognitive task analysis computational algorithm conceptual modelling ontological modelling

Theories: semiotic theory

Display Design for Navy Landing Signal Officers: Supporting Decision Making under Extreme Time Pressure

1998 | Americas Conference on Information Systems | Citations: 1

Authors: Thordsen, Marvin

Abstract: A cognitive task analysis (CTA) was used to determine the critical decisions and ... Expand

Abstract: A cognitive task analysis (CTA) was used to determine the critical decisions and judgments of landing signal officers (LSOs) who are responsible for guiding in the aircraft who land on the U.S. Navy’s aircraft carriers. Based on this CTA, a decision requirements table was constructed which outlined the critical decisions, the critical cues, any potential errors, why the decision is difficult, degrees of uncertainty, tools currently used, and conflicting goals and objectives. With the information, designs were generated for the LSO displays on the LSO platform of the aircraft carriers. Our approach is referred to as decision-centered design. Decisioncentered design adopts the concept of using the critical decisions and judgments of the individuals as the basis for any system or design development with the goal of basing the designs on the decisions, rather than available data, sub-system configurations, or tasks. The recommendations were accepted by LSO instructors and a Phase II effort is underway to further revise the recommendations to encompass additional aircraft types and environmental conditions. This effort will be completed in early 2000. Collapse

Topics: systems design

Methods: design science cognitive task analysis

Expert-Novice Differences in Searching the Web

1998 | Americas Conference on Information Systems | Citations: 1

Authors: Tabatabai, Diana; Luconi, Francesca

Abstract: Three web-user experts and three novices were observed in a problem-solving cont ... Expand

Abstract: Three web-user experts and three novices were observed in a problem-solving context. Their task was to search and evaluate information on the World Wide Web in order to write a research paper about an unfamiliar topic. A cognitive task analysis and verbal protocols were used. Results indicate differences in the ways experts approach and solve problem. These differences were in terms of (a) knowledge base; (b) problem space; (c) strategies; and (d) affect. Novice-expert differences have instructional implications for teachers, Information Technologists and Web site designers. Collapse

Topics: problem solving knowledge base knowledge acquisition online learning system

Methods: qualitative coding design science cognitive task analysis qualitative interview survey