A scalable smart meter data generator using spark

N Iftikhar, X Liu, S Danalachi, FE Nordbjerg… - On the Move to …, 2017 - Springer
N Iftikhar, X Liu, S Danalachi, FE Nordbjerg, JH Vollesen
On the Move to Meaningful Internet Systems. OTM 2017 Conferences: Confederated …, 2017Springer
Today, smart meters are being used worldwide. As a matter of fact smart meters produce
large volumes of data. Thus, it is important for smart meter data management and analytics
systems to process petabytes of data. Benchmarking and testing of these systems require
scalable data, however, it can be challenging to get large data sets due to privacy and/or
data protection regulations. This paper presents a scalable smart meter data generator
using Spark that can generate realistic data sets. The proposed data generator is based on …
Abstract
Today, smart meters are being used worldwide. As a matter of fact smart meters produce large volumes of data. Thus, it is important for smart meter data management and analytics systems to process petabytes of data. Benchmarking and testing of these systems require scalable data, however, it can be challenging to get large data sets due to privacy and/or data protection regulations. This paper presents a scalable smart meter data generator using Spark that can generate realistic data sets. The proposed data generator is based on a supervised machine learning method that can generate data of any size by using small data sets as seed. Moreover, the generator can preserve the characteristics of data with respect to consumption patterns and user groups. This paper evaluates the proposed data generator in a cluster based environment in order to validate its effectiveness and scalability.
Springer
以上显示的是最相近的搜索结果。 查看全部搜索结果