Cost-based Optimization of Multistore Query Plans

2023 | Information Systems Frontiers | Citations: 0

Authors: Forresi, Chiara; Francia, Matteo; Gallinucci, Enrico; Golfarelli, Matteo

Abstract: Multistores are data management systems that enable query processing across diff ... Expand

Abstract: Multistores are data management systems that enable query processing across different and heterogeneous databases; besides the distribution of data, complexity factors like schema heterogeneity and data replication must be resolved through integration and data fusion activities. Our multistore solution relies on a dataspace to provide the user with an integrated view of the available data and enables the formulation and execution of GPSJ queries. In this paper, we propose a technique to optimize the execution of GPSJ queries by formulating and evaluating different execution plans on the multistore. In particular, we outline different strategies to carry out joins and data fusion by relying on different schema representations; then, a self-learning black-box cost model is used to estimate execution times and select the most efficient plan. The experiments assess the effectiveness of the cost model in choosing the best execution plan for the given queries and exploit multiple multistore benchmarks to investigate the factors that influence the performance of different plans. Collapse

Topics: database system MongoDB PostgreSQL document database relational database

Methods: computational algorithm experiment case study regression analysis method longitudinal research

Training Machine Learning Models with Simulated Data for Improved Line Fault Events Classification From 3-Phase PMU Field Recordings

2023 | HICSS | Citations: 0

Authors: Otudi, H.; Mohamed, T.; Kezunovic, M.; Hu, Y.; Obradovic, Z.

Abstract: Utilizing data analytics and machine learning (ML) on phasor measurement units ... Expand

Abstract: Utilizing data analytics and machine learning (ML) on phasor measurement units (PMUs) data to analyze faults automatically is the focus of this paper. Insufficient labels and natural uneven distribution of different types of line fault events found in fieldrecorded PMU data make supervised ML model development challenging. To address this issue, we train off-the-shelf Support Vector Machine (SVM) ML models for line fault classification using simulated PMU data obtained from a combination of 12 physical and virtual PMUs placed on a synthetic IEEE 14-bus system as well as using this simulated data unified with field recordings. A conducted sensitivity study is focused on three factors, 1) the number of PMUs used to train the ML model, 2) the voltage level at which the model is trained, and 3) the vicinity of PMUs to transmission line faults. The ML models trained with simulated and field data are evaluated on one-year field-recorded data collected from 38 PMUs sparsely located in the US Western interconnection. We demonstrate that when training ML models with only simulated data, the performance varies significantly with different number of PMUs, voltage level, and PMU placement in separate areas of the synthetic grid (F1 score of 0.78 to 0.92). We obtained an F1 score of 0.94 using the simulated dataset integrated with field recordings. The performance of a ML model developed using simulated data is also evaluated on the three-phase voltage signals extracted from 188 PMUs in the Eastern interconnection accompanied by imprecise labels, where the majority of the labels do not identify the fault type. On this extremely challenging task, we achieved 77% accuracy solely using synthetic data for ML training. Collapse

Topics: Apache Spark analytical information system computer hardware

Methods: simulation machine learning support vector machine experiment event log

Data Quality in Data Ecosystems: Towards a Design Theory

2022 | Americas Conference on Information Systems | Citations: 5

Authors: Altendeitering, Marcel; Dübler, Stephan; Guggenberger, Tobias

Abstract: Data quality is an important aspect for the success of data ecosystems. Sharing ... Expand

Abstract: Data quality is an important aspect for the success of data ecosystems. Sharing low-quality data causes large data preparation efforts, can disrupt the chain for value co-creation, and can damage the mutual trust among partners in the ecosystem. While there are many data quality tools available in literature and practice, there is limited knowledge on the peculiarities of assessing and managing data quality in data ecosystems. In this study, we present the results of a design science research project that was concerned with the development of design principles for ensuring data quality in data ecosystems. The proposed concept extends an existing ecosystem with an artifact that technically enforces data quality checks on shared data in the manufacturing domain. Our work aims to provide the prescriptive design knowledge needed for such systems. For practitioners, we offer generalized design principles that can inform custom implementations of data quality tools in data ecosystems. Collapse

Topics: digital ecosystem mobile application data quality Python Apache Spark

Methods: design principle design artifact design science design requirement qualitative interview

Insights from Niche Markets: Explainable and Predictive Values of Consumption Tendency on Credit Risks

2022 | International Conference on Information Systems | Citations: 0

Authors: Li, Wenwen; Wu, Yue; Zhang, Yingjie; Lu, Tian; Sun, Yiheng; Zheng, Xuehao

Abstract: The rapid development of FinTech drives the growing popularity of digital payme ... Expand

Abstract: The rapid development of FinTech drives the growing popularity of digital payment transactions. This phenomenon, especially given the increasing number of offline and online transactions being recorded in a real-time manner, offers great opportunities for financial service platforms to track consumers' consumption tendencies and dynamically monitor and evaluate their creditworthiness. In our recent research, we first theorized the value of category-level consumption tendency based on the self-regulatory theory and employed econometric methods to empirically test the relationship between category-level consumption tendency and credit behavior. Then, we proposed a Deep Hierarchical Partial Attention-based Model (DHPAM) to predict credit default risk with full employment of product category features. We provided strong experimental evidence to show that the proposed DHPAM outperforms the state-of-the-art machine learning models. This paper, based on theories, empirical analyses, and a prediction model, offers comprehensive and practical guidance on the optimal utilization of consumption information in credit risk management. Collapse

Topics: risk management electronic finance Apache Spark decision making consumer behavior

Methods: deep learning machine learning ablation analysis support vector machine XGBoost

Theories: behavioral theory

Line Faults Classification Using Machine Learning on Three Phase Voltages Extracted from Large Dataset of PMU Measurements

2022 | HICSS | Citations: 15

Authors: Otudi, Hussain; Dokic, Tatjana; Mohamed, Taif; Kezunovic, Mladen; Hu, Yi; Obradovic, Zoran

Abstract: An end-to-end supervised learning method was developed to classify transmission ... Expand

Abstract: An end-to-end supervised learning method was developed to classify transmission line faults in a twoyear field-recorded dataset that includes synchronized measurements of three-phase voltages recorded by 38 phasor measurement units (PMUs) sparsely located in the US Western Grid interconnection. Statistical analysis was performed to extract features from this large dataset to train the support vector machine (SVM), random forest (RF), and extreme gradient boosting (XGBoost) classifiers. The training further leverages a simulated dataset from a synthetic grid with 12 PMUs to increase the number of types of faults infrequently seen in the field-recorded dataset. Training the classification models with the combined dataset resulted in a classification accuracy of 98.58%. This is a significant improvement over 86.87% to 87.17% accuracy obtained by relying on the field-recorded dataset alone. Collapse

Topics: Python data integration Apache Spark

Methods: simulation support vector machine XGBoost experiment computational algorithm

Dealing with the Social Media Polycontextuality of Work

2022 | Information Systems Research | Citations: 17

Authors: Vaast, Emmanuelle; Pinsonneault, Alain

Abstract: Existing scholarship on social media provides us with important but incomplete i ... Expand

Abstract: Existing scholarship on social media provides us with important but incomplete insights regarding how people rely upon social media as they work. Scholarship so far focuses on social media as technologies that enable and constrain people to do certain things. This study proposes a perspective on social media for work that views them not only as technologies that enable people to do certain things, but also as contexts with emerging norms and roles in which people participated. As they do so, people are confronted with opportunities and challenges that are inherent to social media polycontextuality. This study investigates the case of data scientists and how they dealt with the social media polycontextuality of their work. The study reveals that data scientists relied on strategies of enhancing, differentiating, or occasionally reducing their engagement with social media contexts. The study brings novel insights for scholarship on social media and work by unpacking the social media polycontextuality of work and the interconnections in the engagement with multiple social media contexts. It explains how these interconnections manifest in people’s deliberate participation in or temporary withdrawal from diverse and changing social media contexts. This study also contributes to scholarship by highlighting how people deal with an ensemble of social media applications rather than with a single, isolated information technology. History: Manju Ahuja, Senior Editor; Walter Fernandez, Associate Editor. Funding: This work was supported by the Social Sciences and Humanities Research Council of Canada [Grant 435-2015-0241]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2022.1103. Collapse

Topics: social media IT career Slack affordance LinkedIn

Methods: qualitative interview theory development constant comparative analysis grounded theory qualitative coding

Theories: sociomaterialism theory theory of affordance

Fast Extraction and Characterization of Fundamental Frequency Events from a Large PMU Dataset using Big Data Analytics

2021 | HICSS | Citations: 0

Authors: Baembitov, Rashid; Dokic, Tatjana; Kezunovic, Mladen; Hu, Yi; Obradovic, Zoran

Abstract: A novel method for fast extraction of fundamental frequency events (FFE) based ... Expand

Abstract: A novel method for fast extraction of fundamental frequency events (FFE) based on measurements of frequency and rate of change of frequency by Phasor Measurement Units (PMU) is introduced. The method is designed to work with exceptionally large historical PMU datasets. Statistical analysis was used to extract the features and train Random Forest and Catboost classifiers. The method is capable of fast extraction of FFE from a historical dataset containing measurements from hundreds of PMUs captured over multiple years. The reported accuracy of the best algorithm for classification expressed as Area Under the receiver operating Characteristic curve reaches 0.98, which was obtained in out-of-sample evaluations on 109 system-wide events over 2 years observed at 43 PMUs. Then Minimum Volume Enclosing Ellipsoid Algorithm was used to further analyze the events. 93.72% events were correctly characterized, where average duration of the event as seen by the PMU was 9.93 sec. 1 Collapse

Topics: database system Python Apache Spark high-performance computing data quality

Methods: computational algorithm machine learning event log random forest classification gradient boosting

Cache-Based Multi-Query Optimization for Data-Intensive Scalable Computing Frameworks

2021 | Information Systems Frontiers | Citations: 0

Authors: Michiardi, Pietro; Carra, Damiano; Migliorini, Sara

Abstract: In modern large-scale distributed systems, analytics jobs submitted by various u ... Expand

Abstract: In modern large-scale distributed systems, analytics jobs submitted by various users often share similar work, for example scanning and processing the same subset of data. Instead of optimizing jobs independently, which may result in redundant and wasteful processing, multi-query optimization techniques can be employed to save a considerable amount of cluster resources. In this work, we introduce a novel method combining in-memory cache primitives and multi-query optimization, to improve the efficiency of data-intensive, scalable computing frameworks. By careful selection and exploitation of common (sub)expressions, while satisfying memory constraints, our method transforms a batch of queries into a new, more efficient one which avoids unnecessary recomputations. To find feasible and efficient execution plans, our method uses a cost-based optimization formulation akin to the multiple-choice knapsack problem. Extensive experiments on a prototype implementation of our system show significant benefits of worksharing for both TPC-DS workloads and detailed micro-benchmarks. Collapse

Topics: Apache Spark distributed computing application programming interface database system operating system

Methods: experiment computational algorithm design artifact optimization model knapsack problem

TextBenDS: a Generic Textual Data Benchmark for Distributed Systems

2021 | Information Systems Frontiers | Citations: 0

Authors: Truică, Ciprian-Octavian; Apostol, Elena-Simona; Darmont, Jérôme; Assent, Ira

Abstract: Extracting top-k keywords and documents using weighting schemes are popular tech ... Expand

Abstract: Extracting top-k keywords and documents using weighting schemes are popular techniques employed in text mining and machine learning for different analysis and retrieval tasks. The weights are usually computed in the data preprocessing step, as they are costly to update and keep track of all the modifications performed on the dataset. Furthermore, calculation errors are introduced when analyzing only subsets of the dataset, i.e., wrong weighting are computed as weighting schemes use the number of documents for scoring keywords and documents. Therefore, in a Big Data context, it is crucial to lower the runtime of computing weighting schemes, without hindering the analysis process and the accuracy of the machine learning algorithms. To address this requirement for the task of computing top-k keywords and documents (which largely relies on weighting schemes), it is customary to design benchmarks that compare weighting schemes within various configurations of distributedframeworks and database management systems. Thus, we propose TextBenDS - a generic document-oriented benchmark for storing textual data and constructing weighting schemes. Our benchmark offers a generic data model designed with a multidimensional approach for storing text documents. We also propose using aggregation queries with various complexities and selectivities for constructing term weighting schemes, that are utilized in extracting top-k keywords and documents. We evaluate the computing performance of the queries on several distributed environments set within the Apache Hadoop ecosystem. Our experimental results provide interesting insights. As an example, MongoDB shows the best overall performance, while Spark’s execution time remains almost constant regardless of weighting schemes. Collapse

Topics: information retrieval MongoDB database system distributed system MapReduce

Methods: term frequency–inverse document frequency Okapi BM25 experiment design science qualitative content analysis

DMN4DQ: When data quality meets DMN

2021 | Decision Support Systems | Citations: 0

Authors: Valencia-Parra, Álvaro; Parody, Luisa; Varela-Vaca, Ángel Jesús; Caballero, Ismael; Gómez-López, María Teresa

Abstract: To succeed in their business processes, organizations need data that not only at ... Expand

Abstract: To succeed in their business processes, organizations need data that not only attains suitable levels of quality for the task at hand, but that can also be considered as usable for the business. However, many researchers ground the potential usability of the data on its quality. Organizations would benefit from receiving recommendations on the usability of the data before its use. We propose that the recommendation on the usability of the data be supported by a decision process, which includes a context-dependent data-quality assessment based on business rules. Ideally, this recommendation would be generated automatically. Decision Model and Notation (DMN) enables the assessment of data quality based on the evaluation of business rules, and also, provides stakeholders (e.g., data stewards) with sound support for the automation of the whole process of generation of a recom­ mendation regarding usability based on data quality. Collapse

Topics: data quality usability quality management business process management missing data

Methods: case study survey design literature study