JI Yimu, ZHANG Dianchao, SUN Yanfei, ZHAO Chuanxin, HE Jing, WANG Ruchuan, CHEN Guoliang. SQLS: A Storm-Based Query Language System for Real-Time Stream Data Analysis[J]. Chinese Journal of Electronics, 2016, 25(6): 1025-1033. doi: 10.1049/cje.2016.10.003
Citation: JI Yimu, ZHANG Dianchao, SUN Yanfei, ZHAO Chuanxin, HE Jing, WANG Ruchuan, CHEN Guoliang. SQLS: A Storm-Based Query Language System for Real-Time Stream Data Analysis[J]. Chinese Journal of Electronics, 2016, 25(6): 1025-1033. doi: 10.1049/cje.2016.10.003

SQLS: A Storm-Based Query Language System for Real-Time Stream Data Analysis

doi: 10.1049/cje.2016.10.003
Funds:  This work is supported by ZTE Foundation (No.2014W58), the National Natural Foundation (No.61170065, No.61373017), and Jiangsu Natural Science of Young Foundation (No.BK20130876).
  • Received Date: 2016-01-18
  • Rev Recd Date: 2016-05-19
  • Publish Date: 2016-11-10
  • In this work, a Storm-based query language System (SQLS) is proposed for real-time data stream analysis. The system is compatible with Continuous query language (CQL) specification. It supports both continuous queries and one-time queries over streaming data, and meets the requirements of user experience (traditional SQL queries) and QoS (such as real-time and throughput). In order to better meet the requirement of throughput and enhance the processing efficiency, the load shedding algorithm and cache optimization are employed during the implementation of SQL-like operators. Finally, performance testing of the proposed SQLS has been conducted on stand-alone Storm platform and Storm clusters. Experimental results show that our system can not only meet the needs of users, but also extend the function of real-time streaming queries processing.
  • loading
  • A. Arasu, B. Babcock, S. Babu, et al, "Stream: The stanford data stream management system", Data Stream Management, Springer Berlin Heidelberg, pp. 271-275, 2016.
    Daniel J. Abadi, Don Carney, Ugur Cetintemel, et al, "Aurora: A new model and architecture for data stream management", The VLDB Journal-The International Journal on Very Large Data Bases, Vol.12, No.2, pp.120-139, 2003.
    D.J. Abadi, Y. Ahmad, M. Balazinska, et al, "The Design of the Borealis Stream Processing Engine", Second Biennial Conference on Innovative Data Systoms Researoh (CIDR), Asilomar, CA, 2005.
    Storm, http://storm-project.net,2014-6-20.
    Spark, "lightning-fast cluster computing", http://sparkproject.org,2013.
    L. Neumeyer, B. Robbins, A. Nair, A. Kesari, "S4: Distributed stream computing platform", Proc. of the 10th IEEE Intl Conf on Data Mining Workshops (ICDMW 2010), Sydney Australia, pp.170-177, 2010.
    D.W. Sun, G.Y. Zhang and W.M. Zheng, "Big data stream computing: technologies and instances", Journal of Software, Vol.4, pp.839-862, 2014.
    R. Motwani, J. Widom, A. Arasu, et al, "Query processing, resource management, and approximation in a data stream management system", IEEE VLSI Test Symposium, Vol.35, pp.245-256, 2003.
    A. Arasu, B. Babcock, S. Babu, et al, "STREAM: The stanford stream data manager", Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data, pp.665-665, 2003.
    D. Abadi, D. Carney, U. Cetintemel, et al, "Aurora: A data stream management system", Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data, San Diego, USA pp.666-672, 2003.
    Spark SQL, http://spark.apache.org/sql,2014.
    Presto, http://prestodb.io,2014.
    F. Farag and M.A. Hammad, "Adaptive execution of stream window joins in a limited memory environment", 11th International Database Engineering and Applications Symposium (IDEAS), IEEE, Banff, Canada, pp.12-20, 2007.
    Redis, http://redis.io,2007.
    W.H. Xu and Z. Qin, "Constructing decision trees for mining high-speed data streams", Chinese Journal of Electronics, Vol.21, No.2, pp.215-220, 2012.
    W.P. Li, J. Yang and J.P. Zhang, "Sampling Streaming Data Along Geodesic", Chinese Journal of Electronics, Vol.24, No.2, pp.251-257, 2015.
    HBase, http://hbase.apache.org,2015.
    Phoenix, http://phoenix.apache.org,2015.
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Article Metrics

    Article views (185) PDF downloads(507) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return