Cost-based Optimization of Multistore Query Plans

2023 | Information Systems Frontiers | Citations: 0

Authors: Forresi, Chiara; Francia, Matteo; Gallinucci, Enrico; Golfarelli, Matteo

Abstract: Multistores are data management systems that enable query processing across diff ... Expand

Abstract: Multistores are data management systems that enable query processing across different and heterogeneous databases; besides the distribution of data, complexity factors like schema heterogeneity and data replication must be resolved through integration and data fusion activities. Our multistore solution relies on a dataspace to provide the user with an integrated view of the available data and enables the formulation and execution of GPSJ queries. In this paper, we propose a technique to optimize the execution of GPSJ queries by formulating and evaluating different execution plans on the multistore. In particular, we outline different strategies to carry out joins and data fusion by relying on different schema representations; then, a self-learning black-box cost model is used to estimate execution times and select the most efficient plan. The experiments assess the effectiveness of the cost model in choosing the best execution plan for the given queries and exploit multiple multistore benchmarks to investigate the factors that influence the performance of different plans. Collapse

Topics: database system MongoDB PostgreSQL document database relational database

Methods: computational algorithm experiment case study regression analysis method longitudinal research

Facing Big Data System Architecture Deployments: Towards an Automated Approach Using Container Technologies for Rapid Prototyping

2022 | HICSS | Citations: 0

Authors: Volk, Matthias; Staegemann, Daniel; Islam, Ashraful; Turowski, Klaus

Abstract: Within the last decade, big data became a promising trend for many application a ... Expand

Abstract: Within the last decade, big data became a promising trend for many application areas, offering immense potential and a competitive edge for various organizations. As the technical foundation for most of today´s data-intensive projects, not only corresponding infrastructures and facilities but also the appropriate knowledge is required. Currently, several projects and services exist that not only allow enterprises to utilize but also to deploy related technologies and systems. However, at the same time, the use of these is accompanied by various challenges that may result in huge monetary expenditures, a lack of modifiability, or the risk of vendor lock-ins. To overcome these shortcomings, in the contribution at hand, modern container and task automation technologies are used to wrap complex big data technologies into re-usable and portable resources. Those are subsequently incorporated in a framework to automate the deployment of big data architectures in private and limited resources. Collapse

Topics: Cassandra Docker big data hard disk drive usability

Methods: data modeling literature study design artifact testing design science survey

Teaching NoSQL Data Models: A Tutorial

2021 | Americas Conference on Information Systems | Citations: 0

Authors: Greiner, Martina

Abstract: This tutorial discusses the four most popular categories of NoSQL databases: ke ... Expand

Abstract: This tutorial discusses the four most popular categories of NoSQL databases: key-value databases, document databases, graph databases, and wide column databases. For each NoSQL database type, we explain differences to traditional relational database management systems, how data is organized using Airbnb as an example, and benefits and drawbacks. Collapse

Topics: Airbnb nosql database database system relational database Cassandra

Methods: data modeling conceptual modelling design science entity relationship modeling

Efficient querying of multidimensional RDF data with aggregates: Comparing NoSQL, RDF and relational data stores

2020 | International Journal of Information Management | Citations: 0

Authors: Ravat, Franck; Song, Jiefu; Teste, Olivier; Trojahn, Cassia

Abstract: This paper proposes an approach to tackle the problem of querying large volume o ... Expand

Abstract: This paper proposes an approach to tackle the problem of querying large volume of statistical RDF data. Our approach relies on pre-aggregation strategies to better manage the analysis of this kind of data. Specifically, we define a conceptual model to represent original RDF data with aggregates in a multidimensional structure. A set of translations rules for converting a well-known multidimensional RDF modelling vocabulary into the proposed conceptual model is then proposed. We implement the conceptual model in six different data stores: two RDF triple stores (Jena TDB and Virtuoso), one graph-oriented NoSQL database (Neo4j), one column-oriented data store (Cassandra), and two relational databases (MySQL and PostGreSQL). We compare the querying performance, with and without aggregates, in these data stores. Experimental results, on real-world datasets containing 81.92 million triplets, show that pre-aggregation allows for reducing query runtime in all data stores. Neo4j NoSQL and relational databases with aggregates outperform triple stores speeding up to 99% query runtime. Collapse

Topics: Neo4j Cassandra relational database nosql database MySQL

Methods: experiment conceptual modelling computational algorithm multidimensional modeling optimization model

Information Systems for a Smart Electricity Grid: Emerging Challenges and Opportunities

2018 | ACM Transactions on Management Information Systems | Citations: 1

Authors: Ketter, Wolfgang; Collins, John; Saar-Tsechansky, Maytal; Marom, Ori

Abstract: The drive for sustainability as evidenced by the Paris Accords is forcing a radi ... Expand

Abstract: The drive for sustainability as evidenced by the Paris Accords is forcing a radical re-examination of the way electricity is produced, managed, and consumed. Research on sustainable smart electricity markets is facilitating the emergence of sustainable energy systems and a revolution in the efficiency and reliability of electricity consumption, production, and distribution. Traditional electricity grids and markets are being disrupted by a range of forces, including the rise of weather-dependent and distributed renewable sources, growing consumer involvement in managing their power consumption and production, and the electrification of transport. These changes will likely bring about complex and dynamic smart electricity markets that rely on analysis of information to inform stakeholders, and on effective integration of stakeholders’ actions. We outline a research agenda on how advances in information-intensive processes are fundamental for facilitating these transformations, describe the roles that such processes will play, and discuss Information Systems research challenges necessary to achieve these goals. These challenges span public policy, privacy, and security; market mechanisms; and data-driven decision support. The diverse challenges we outline also underscore that the diverse IS research perspective is instrumental for addressing the complexity and interdisciplinary nature of this research. Collapse

Topics: smart home decision support energy information system privacy Cassandra

A Primer on NoSQL Databases for Enterprise Architects: The CAP Theorem and Transparent Data Access with MongoDB and Cassandra

2018 | HICSS | Citations: 0

Authors: Marungo, Fumbeya

Abstract: MongoDB and Apache Cassandra are the dominant “Not Only SQL” (NoSQL) database ma ... Expand

Abstract: MongoDB and Apache Cassandra are the dominant “Not Only SQL” (NoSQL) database management systems for persisting structured records. Moreover, the pair are respectively in the top-five and top-ten of database management systems generally. Therefore this work seeks to present the two leading systems, along with the underlying principle of the CAP Theorem, in the context of creating transparent data access tiers capable of supporting flexible enterprise architectures. Collapse

Topics: Cassandra MongoDB database system enterprise architecture nosql database

XLab: Early Indications & Warnings from Open Source Data with Application to Biological Threat

2018 | HICSS | Citations: 0

Authors: Simek, Olga; Davis, Curtis; Heier, Andrew; Mohindra, Sanjeev; O'Brien, Kyle; Passarelli, John; Waugh, Frederick

Abstract: XLab is an early warning system that addresses a broad range of national securit ... Expand

Abstract: XLab is an early warning system that addresses a broad range of national security threats using a flexible, rapidly reconfigurable architecture. XLab enables intelligence analysts to visualize, explore, and query a knowledge base constructed from multiple data sources, guided by subject matter expertise codified in threat model graphs. This paper describes a novel system prototype that addresses threats arising from biological weapons of mass destruction. The prototype applies knowledge extraction analytics-”including link estimation, entity disambiguation, and event detection-”to build a knowledge base of 40 million entities and 140 million relationships from open sources. Exact and inexact subgraph matching analytics enable analysts to search the knowledge base for instances of modeled threats. The paper introduces new methods for inexact matching that accommodate threat models with temporal and geospatial patterns. System performance is demonstrated using several simplified threat models and an embedded scenario. Collapse

Topics: knowledge graph knowledge base database system open source graph database

Methods: computational algorithm design artifact longitudinal research literature study named entity recognition

Batch to Real-Time: Incremental Data Collection & Analytics Platform

2017 | HICSS | Citations: 0

Authors: Aydin, Ahmet; Anderson, Ken

Abstract: Real-time data collection and analytics is a desirable but challenging feature t ... Expand

Abstract: Real-time data collection and analytics is a desirable but challenging feature to provide in dataintensive software systems. To provide highly concurrent and efficient real-time analytics on streaming data at interactive speeds requires a welldesigned software architecture that makes use of a carefully selected set of software frameworks. In this paper, we report on the design and implementation of the Incremental Data Collection & Analytics Platform (IDCAP). The IDCAP provides incremental data collection and indexing in real-time of social media data; support for real-time analytics at interactive speeds; highly concurrent batch data processing supported by a novel data model; and a front-end web client that allows an analyst to manage IDCAP resources, to monitor incoming data in real-time, and to provide an interface that allows incremental queries to be performed on top of large Twitter datasets. Collapse

Topics: Twitter Cassandra mobile application Redis Apache Solr

Methods: design artifact literature study data modeling

A Big Data system supporting Bosch Braga Industry 4.0 strategy

2017 | International Journal of Information Management | Citations: 0

Authors: Santos, Maribel Yasmina; Oliveira e Sá, Jorge; Andrade, Carina; Vale Lima, Francisca; Costa, Eduarda; Costa, Carlos; Martinho, Bruno; Galvão, João

Abstract: People, devices, infrastructures and sensors can constantly communicate exchangi ... Expand

Abstract: People, devices, infrastructures and sensors can constantly communicate exchanging data and generating new data that trace many of these exchanges. This leads to vast volumes of data collected at ever increasing velocities and of different variety, a phenomenon currently known as Big Data. In particular, recent developments in Information and Communications Technologies are pushing the fourth industrial revolution, Industry 4.0, being data generated by several sources like machine controllers, sensors, manufacturing systems, among others. Joining volume, variety and velocity of data, with Industry 4.0, makes the opportunity to enhance sustainable innovation in the Factories of the Future. In this, the collection, integration, storage, processing and analysis of data is a key challenge, being Big Data systems needed to link all the entities and data needs of the factory. Thereby, this paper addresses this key challenge, proposing and implementing a Big Data Analytics architecture, using a multinational organisation (Bosch Car Multimedia – Braga) as a case study. In this work, all the data lifecycle, from collection to analysis, is handled, taking into consideration the different data processing speeds that can exist in the real environment of a factory (batch or stream). Collapse

Topics: big data production management industry 4.0 data visualization Cassandra

Methods: data modeling case study design artifact multidimensional modeling design science

A snail shell process model for knowledge discovery via data analytics

2016 | Decision Support Systems | Citations: 80

Authors: Li, Yan; Thomas, Manoj A.; Osei-Bryson, Kweku-Muata

Abstract: The rapid growth of big data environment imposes new challenges that traditional ... Expand

Abstract: The rapid growth of big data environment imposes new challenges that traditional knowledge discovery and data mining process (KDDM) models are not adequately suited to address. We propose a snail shell process model for knowledge discovery via data analytics (KDDA) to address these challenges. We evaluate the utility of the KDDA process model using real-world analytic case studies at a global multi-media company. By comparing against traditional KDDM models, we demonstrate the need and relevance of the snail shell model, particularly in addressing faster turnaround and frequent model updates that characterize knowledge discovery in the big data environment. Collapse

Topics: big data Hadoop Distributed File System data quality data integration Tableau software

Methods: business process modeling data transformation design science exploratory data analysis case study