A Method for Evaluating End-User Development Technologies

2017 | Americas Conference on Information Systems | Citations: 0

Authors: Melo, Claudia de O

Abstract: End-user development (EUD) is a strategy that can reduce a considerable amount o ... Expand

Abstract: End-user development (EUD) is a strategy that can reduce a considerable amount of business demand on IT departments. Empowering the end-user in the context of software development is only possible through technologies that allow them to manipulate data and information without the need for deep programming knowledge. The successful selection of appropriate tools and technologies is highly dependent on the context in which the end-user is embedded. End-users should be a central piece in any software package evaluation, being key in the evaluation process in the end-user development context. However, little research has empirically examined software package evaluation criteria and techniques in general, and in the end-user development context in particular. This paper aims to provide a method for technology evaluation in the context of end-user development and to present the evaluation of two platforms. We conclude our study proposing a set of suggestions for future research. Collapse

Topics: end user development usability system quality systems evaluation Oracle database

Methods: literature study computational algorithm analytic hierarchy process trade-off analysis method structured literature research

Theories: technology acceptance model

Exploring an alternative method of evaluating the effects of ERP: a multiple case study

2009 | Journal of Information Technology | Citations: 6

Authors: Uwizeyemungu, Sylvestre; Raymond, Louis

Abstract: Previous research has already established that compared to other types of invest ... Expand

Abstract: Previous research has already established that compared to other types of investments, information technology (IT) investments are insufficiently or not at all evaluated. This can be partly explained by the lack of adequate IT evaluation methods and tools. In the case of enterprise resource planning (ERP) systems whose effects on organizational processes and performance are intrinsically profound and wide-ranging compared to those of traditional IT limited to some spheres of organization, evaluation activities may be an issue of great concern. This study thus aims to propose and test an alternative evaluation method adaptable to the organizational context, making it possible to measure the contribution of an ERP system to organizational performance in all its aspects. Combining a process-based model and a scorecard model, the proposed method was first designed from a review of information systems evaluation literature. It has then been validated and refined through a multi-case study of manufacturing firms: an in-depth pilot case study was conducted, and thereafter the study was replicated on two other cases. Results show that the method proposed here enables organizations to determine the extent to which the firm's operational and overall performance has been impacted by the adoption and use of ERP systems, through the automational, informational, and transformational effects of ERP on their business processes. From a practical point of view, three contributions must be mentioned: the proposed method allows for a strong contextualization of its application, it is action-oriented, and it allows comparison across organizations even though organizational contexts may totally differ. Collapse

Topics: enterprise resource planning organizational productivity organizational context business process management systems evaluation

Methods: qualitative interview multiple case study business process modeling literature study cross sectional research

Development of a framework to assess and guide IT investments: An analysis based on a discretionary–mandatory classification

2008 | International Journal of Information Management | Citations: 0

Authors: Joshi, Kailash; Pant, Somendra

Abstract: Assessing the economic feasibility of information technology (IT) projects remai ... Expand

Abstract: Assessing the economic feasibility of information technology (IT) projects remains a challenge for most organizations. On one hand there are concerns for the rising IT costs, on the other hand organizations may lose significantly if they do not make an appropriate investment in IT. Considering the range of IT projects, a single, standardized capital budgeting approach may not serve the best interests of organizations. This paper presents a framework to help evaluate different IT projects through a mix of suitable methods. The framework is based on the classification of IT projects along a discretionary–mandatory dimension into four types: purely discretionary, mainly discretionary, mainly mandatory, and purely mandatory. A set of factors are identified that may influence the final classification of an IT project into one of the four types. This classification is used to arrive at suitable IT evaluation techniques and methods. Use of strict financial evaluation criteria is suggested for discretionary projects. The need for analyzing financial risks associated with IT projects is also identified and it is suggested that higher standards of project acceptance be used for riskier projects that are not deemed mandatory in the earlier analysis. The framework should be useful for information systems researchers and practitioners in streamlining the assessment of IT projects. Collapse

Topics: IT project project management IS evaluation method IT investment systems evaluation

Methods: qualitative interview

Toward IT Value Mapping - An Approach to Value-Based IT Management

2008 | Americas Conference on Information Systems | Citations: 0

Authors: Lanzinner, Steffen; Leimeister, Jan Marco; Krcmar, Helmut

Abstract: Recent research has shown that companies face considerable difficulties in imple ... Expand

Abstract: Recent research has shown that companies face considerable difficulties in implementing a value-based management of their IT portfolios. This paper therefore presents an approach for measuring and managing the business value of IT investments. To ensure practicability, our approach is derived from a set of practice-based requirements extending previous empirical work. In our conceptual model we distinguish between capital budgeting and ongoing value management. For capital budgeting we propose to assess IT investments according to their expected business value by using net present value analysis and a scoring model. For ongoing value management we recommend to keep track of a set of project specific key performance indicators to ensure that the desired benefits are realized. In order to link these two fundamental processes, value drivers and measurable performance indicators have to be identified. Thus we combine established evaluation for an integrated IT Portfolio assessment considering financial, non-financial and risk effects as well. Collapse

Topics: organizational value IS management risk management IT investment spreadsheet

Methods: design artifact design science expert evaluation post analysis conceptual modelling

IT Investment Evaluation as a Socio-Political Process: Determinants to Managerial Adoption and Use

2007 | European Conference On Information Systems | Citations: 0

Authors: Hu, Qing; Frisk, Elisabeth; Hallikainen, P; Eikebrokk, T; Paivarinta, Tero

Abstract: Researchers have long recognized that IT investment evaluation is a complex soci ... Expand

Abstract: Researchers have long recognized that IT investment evaluation is a complex socio-technical process. Some scholars even consider it as a socio-political process based on the argument that evaluations are conducted from the vantage points of the stakeholders. While anecdotal evidences of these arguments are plenty, empirical support based on large samples and rigorous methodologies are rare in the current IT evaluation literature. In this study we present the results of a structural equation analysis of a sociopolitical model of using survey data collected from 312 Swedish companies. Our results show that organizational power and political factors play a significant role in forming the managers’ intention to conduct formal IT investment evaluation, which in turn predicts the actual use of the formal evaluation methods and frameworks. While perceived usefulness still significantly influences intention, as previously reported, the power and political factors are shown to influence the action directly with or without the mediation of intention, supporting the strong influence of organizational politics in the IT investment evaluation process. Collapse

Topics: IT investment perceived usefulness usage intention perceived ease of use social influence

Methods: survey partial least squares regression structural equation modeling conceptual modelling cross sectional research

Theories: technology acceptance model social sciences theory organizational behavior theory theory of planned behavior

The Use of Formal IT Investment Evaluation Methods in Organizations: A Survey of European Countries

2006 | Americas Conference on Information Systems | Citations: 0

Authors: Hallikainen, Petri; Hu, Qing; Frisk, Elisabeth; Pã, Tero

Abstract: IT investment evaluation has been the subject of many academic studies and often ... Expand

Abstract: IT investment evaluation has been the subject of many academic studies and often rated as one of the top concerns of IT managers. While many studies have focused on the issue of how to evaluate IT projects, we are more interested in understanding why formal IT evaluation methods and techniques are used or not used in various organizations. In this paper, we present the preliminary findings of a large scale, multi-country survey on the usage of formal IT evaluation methods in organizations. Our results show that, despite the fact that most IT managers consider IT investment evaluation as a critical issue in IT management, only about one third of IT investment projects are subjected to some formal evaluation processes. What is even more interesting is that only about 10% of the organizations surveyed claim to use formal quantitative evaluation methods such as ROI, NPV, IRR, or C/B, while over half of the organizations report that their IT investment decisions are made based on experience and manager discretion. We find that the managers’perception about the usefulness of and attitudes toward formal evaluation methods and techniques correspond well to the percentage of usage in each of the three countries we surveyed. Our findings provide an update to the state of the art of IT investment evaluation usage in organizations and raise some interesting questions for future research. Collapse

Topics: IT investment electronic mail IT manager IS evaluation method IS management

Methods: survey survey design Student's t-test

Theories: theory of planned behavior

IT investment evaluation: why hasn ’t it become an organization routine?

2006 | European Conference On Information Systems | Citations: 0

Authors: Hu, Qing; Frisk, Elisabeth; Eikebrokk, T; Hallikainen, P; Paivarinta, Tero

Abstract: In this study we attempt to understand why formal evaluations of IT investment p ... Expand

Abstract: In this study we attempt to understand why formal evaluations of IT investment projects have not yet become an organizational routine. Using survey data gathered from business and IT managers in Sweden, we tested the research hypotheses about the factors influencing the attitudes and behaviour of managers towards using formal evaluation methods based on the theory of planned behaviour. We found that the intent to use formal evaluation methods in an organization is determined by the attitudes of the managers towards the formal methods, the common beliefs of the organization about the formal methods, and the perceived ability to perform formal evaluations. Interestingly, we found that the attitudes toward formal methods are determined mostly by the perceived usefulness of the methods and not by the perceived ease of use of these methods, suggesting that the decision to use formal methods is most likely based on rational analyses rather than individual preferences. We also found that awareness and selfefficacy contribute to the use of formal methods via influences on organizational beliefs and perceived ability to perform evaluation tasks. These findings provide some interesting managerial implications for advocating the use of formal methods in organizations. Collapse

Topics: IT investment perceived ease of use perceived usefulness IT manager self efficacy

Methods: survey SEM tool structural equation modeling data transformation factor analysis

Theories: theory of planned behavior technology acceptance model social sciences theory theory of reasoned action

The Effect of Stakeholder Consideration in IT Investment Evaluation on Business Value: Evidence from Sweden

2005 | Americas Conference on Information Systems | Citations: 0

Authors: Frisk, Elisabeth; Roztocki, Narcyz

Abstract: Investments in Information Technology affect key stakeholders, such as employees ... Expand

Abstract: Investments in Information Technology affect key stakeholders, such as employees, owners, customers, and suppliers. According to prior studies, many of the Information Technology investments fail to deliver the expected results because stakeholders’interests and motivations were not adequately addressed. Based on data collected from 70 companies in Sweden, we examine how consideration of key stakeholders in the pre-investment phase may influence the business value obtained from investments in Information Technology. Our findings indicate that a stakeholder view influences the choice of methods and also, at least indirectly, the chances of achieving business value. Nevertheless, a strong key stakeholder consideration does not automatically guarantee successful IT investments, if not backed by proper evaluation methods. Therefore future research may explore the issue of how a portfolio of methods could be assembled considering a company’s unique stakeholders’characteristics and interests. Collapse

Topics: organizational value systems evaluation IS evaluation method IS evaluation IT investment

Methods: survey survey design literature study cross sectional research

Outcome Based Systems Evaluation to Assess Information Technology Impact Using Arima Methods

2003 | Communications of the Association for Information Systems | Citations: 0

Authors: Evans, Gerald E.; Costa, Bruce A.

Abstract: A new method of system evaluation that focuses on the impact the system has on a ... Expand

Abstract: A new method of system evaluation that focuses on the impact the system has on a data series that served as the rationale for systems implementation was designed and modeled by the authors. Called outcome-based evaluation, this method is founded on the concept of intervention analysis and employs interrupted time series designs to determine the impact of an information system on specific organizational goals. Based on a review of the literature on the evolution of systems evaluation methods from focusing on user goals to user satisfaction and system usage, we conclude that user satisfaction and system usage are necessary but not sufficient criteria to establish system effectiveness or success. Thus, we establish a need for the proposed new method of system evaluation. Collapse

Topics: inventory management system job satisfaction point of sale system information system use user satisfaction

Methods: autoregressive integrated moving average longitudinal research time series analysis case study survey

IS Evaluation in the UK Public Sector: Emerging Research Themes and Issues

2003 | Americas Conference on Information Systems | Citations: 1

Authors: Jones, Steve; Irani, Zahir

Abstract: The concern that investment in Information Systems (IS) is not delivering antici ... Expand

Abstract: The concern that investment in Information Systems (IS) is not delivering anticipated value and benefits is compounded by the difficulty in demonstrating IS value using traditional mechanistic evaluation methods. This is currently an important issue in the UK pubic sector, where the audit commission is employing traditional IS evaluation approaches in external inspections. Using the current literature on IS evaluation, together with both previous and new case material to gain insight into IS evaluation practice, the authors argue that IS evaluation in the public sector is very different from the private sector, and that traditional IS evaluation approaches based upon economic factors are largely inappropriate in this domain and indeed can be ritualistic. The paper calls for UK public sector organisations and external audit bodies to review their IS evaluation practice and seek alternative approaches. Finally, the paper argues for a research agenda, which explores alternative, complementary interpretive IS evaluation approaches to help public sector organisations and external audit bodies understand and gauge the impact of IS. Collapse

Topics: IS evaluation IS evaluation method

Methods: case study grounded theory personal interview