SQLS: A Storm-Based Query Language System for Real-Time Stream Data Analysis

JI Yimu; ZHANG Dianchao; SUN Yanfei; ZHAO Chuanxin; HE Jing; WANG Ruchuan; CHEN Guoliang

doi:10.1049/cje.2016.10.003

Volume 25 Issue 6

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2016 > 25(6): 1025-1033

JI Yimu, ZHANG Dianchao, SUN Yanfei, et al., “SQLS: A Storm-Based Query Language System for Real-Time Stream Data Analysis,” Chinese Journal of Electronics, vol. 25, no. 6, pp. 1025-1033, 2016, doi: 10.1049/cje.2016.10.003

Citation:

JI Yimu, ZHANG Dianchao, SUN Yanfei, et al., “SQLS: A Storm-Based Query Language System for Real-Time Stream Data Analysis,” Chinese Journal of Electronics, vol. 25, no. 6, pp. 1025-1033, 2016, doi: 10.1049/cje.2016.10.003

Citation:

PDF( 1575 KB)

SQLS: A Storm-Based Query Language System for Real-Time Stream Data Analysis

doi: 10.1049/cje.2016.10.003

1.
Computer of Science, Nanjing University of Posts and Telecommunications, Nanjing 210023, China;
2.
Jiangsu High Technology Research Key Laboratory for Wireless Sensor Networks, Nanjing 210003, China;
3.
Key Laboratory of Computer Network and Information Integration, Southeast University, Nanjing 211102, China;
4.
Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, Nanjing University of Science and Technology, Nanjing 210094, China;
5.
Computer of Telecommunications, Nanjing University of Posts and Telecommunications, Nanjing 210023, China;
6.
Department of Computer, Anhui Normal University, Wuhu 241002, China;
7.
Engineering and Science, Victoria University, Melbourne, Victoria 8001, Australia

Funds: This work is supported by ZTE Foundation (No.2014W58), the National Natural Foundation (No.61170065, No.61373017), and Jiangsu Natural Science of Young Foundation (No.BK20130876).

Received Date: 2016-01-18
Rev Recd Date: 2016-05-19
Publish Date: 2016-11-10

Abstract

Abstract

In this work, a Storm-based query language System (SQLS) is proposed for real-time data stream analysis. The system is compatible with Continuous query language (CQL) specification. It supports both continuous queries and one-time queries over streaming data, and meets the requirements of user experience (traditional SQL queries) and QoS (such as real-time and throughput). In order to better meet the requirement of throughput and enhance the processing efficiency, the load shedding algorithm and cache optimization are employed during the implementation of SQL-like operators. Finally, performance testing of the proposed SQLS has been conducted on stand-alone Storm platform and Storm clusters. Experimental results show that our system can not only meet the needs of users, but also extend the function of real-time streaming queries processing.
- Continuous query,
- Ad hoc query,
- Storm,
- CQL,
- Real-time query processing

FullText(HTML)

References(18)

References

A. Arasu, B. Babcock, S. Babu, et al, "Stream: The stanford data stream management system", Data Stream Management, Springer Berlin Heidelberg, pp. 271-275, 2016.

Daniel J. Abadi, Don Carney, Ugur Cetintemel, et al, "Aurora: A new model and architecture for data stream management", The VLDB Journal-The International Journal on Very Large Data Bases, Vol.12, No.2, pp.120-139, 2003.

D.J. Abadi, Y. Ahmad, M. Balazinska, et al, "The Design of the Borealis Stream Processing Engine", Second Biennial Conference on Innovative Data Systoms Researoh (CIDR), Asilomar, CA, 2005.

Storm, http://storm-project.net,2014-6-20.

Spark, "lightning-fast cluster computing", http://sparkproject.org,2013.

L. Neumeyer, B. Robbins, A. Nair, A. Kesari, "S4: Distributed stream computing platform", Proc. of the 10th IEEE Intl Conf on Data Mining Workshops (ICDMW 2010), Sydney Australia, pp.170-177, 2010.

D.W. Sun, G.Y. Zhang and W.M. Zheng, "Big data stream computing: technologies and instances", Journal of Software, Vol.4, pp.839-862, 2014.

R. Motwani, J. Widom, A. Arasu, et al, "Query processing, resource management, and approximation in a data stream management system", IEEE VLSI Test Symposium, Vol.35, pp.245-256, 2003.

A. Arasu, B. Babcock, S. Babu, et al, "STREAM: The stanford stream data manager", Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data, pp.665-665, 2003.

D. Abadi, D. Carney, U. Cetintemel, et al, "Aurora: A data stream management system", Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data, San Diego, USA pp.666-672, 2003.

Spark SQL, http://spark.apache.org/sql,2014.

Presto, http://prestodb.io,2014.

F. Farag and M.A. Hammad, "Adaptive execution of stream window joins in a limited memory environment", 11th International Database Engineering and Applications Symposium (IDEAS), IEEE, Banff, Canada, pp.12-20, 2007.

Redis, http://redis.io,2007.

W.H. Xu and Z. Qin, "Constructing decision trees for mining high-speed data streams", Chinese Journal of Electronics, Vol.21, No.2, pp.215-220, 2012.

W.P. Li, J. Yang and J.P. Zhang, "Sampling Streaming Data Along Geodesic", Chinese Journal of Electronics, Vol.24, No.2, pp.251-257, 2015.

HBase, http://hbase.apache.org,2015.

Phoenix, http://phoenix.apache.org,2015.

Relative Articles

Supplements(0)

Cited By

Proportional views