Big Data Storage

Abstract

This chapter provides an overview of big data storage technologies. It is the result of a survey of the current state of the art in data storage technologies in order to create a cross-sectorial technology roadmap. This chapter provides a concise overview of big data storage systems that are capable of dealing with high velocity, high volumes, and high varieties of data. It describes distributed file systems, NoSQL databases, graph databases, and NewSQL databases. The chapter investigates the challenge of storing data in a secure and privacy-preserving way. The social and economic impact of big data storage technologies is described, open research challenges highlighted, and three selected case studies are provided from the health, finance, and energy sector. Some of the key insights on big data storage are (1) in-memory databases and columnar databases typically outperform traditional relational database systems, (2) the major technical barrier to widespread up-take of big data storage solutions are missing standards, and (3) there is a need to address open research challenges related to the scalability and performance of graph databases.

Publication
New Horizons for a Data-Driven Economy - A Roadmap for Usage and Exploitation of Big Data in Europe