基于GPU的关系型流处理系统实现与优化

黄皓; 李志方; 王嘉伦; 翁楚良

doi:10.3969/j.issn.1000-5641.2019.05.015

基于GPU的关系型流处理系统实现与优化

doi: 10.3969/j.issn.1000-5641.2019.05.015

华东师范大学数据科学与工程学院, 上海 200062

基金项目:

国家重点研发计划 2018YFB1003400

详细信息

作者简介:
黄皓, 男, 硕士研究生, 研究方向为内存型数据库.E-mail:haohuang@stu.ecnu.edu.cn

通讯作者:
翁楚良, 男, 教授, 博士生导师, 研究方向为并行与分布式系统.E-mail:clweng@dase.ecnu.edu.cn

中图分类号: TP315
计量
- 文章访问数: 114
- HTML全文浏览量: 93
- PDF下载量: 1
- 被引次数: 0
出版历程
- 收稿日期: 2019-07-29
- 刊出日期: 2019-09-25

Implementation and optimization of GPU-based relational streaming processing systems

School of Data Science and Engineering, East China Normal University, Shanghai 200062, China

摘要

摘要: 现有的基于CPU的流处理系统在功能上已支持在大规模数据集上的复杂分析查询，但由于CPU计算能力与特性的限制，无法在性能上同时满足高吞吐量和低响应时间的要求.本文提出一种基于GPU的流处理系统框架Serval，通过充分利用CPU-GPU异构资源，实现了关系型流查询的高效处理.Serval框架采用流水线模型和流执行缓存技术以优化吞吐量和响应时间，并实现多种调优策略以适应不同场景.实验表明，单节点Serval的吞吐量与响应时间性能均优于现有GPU数据库MapD和三节点分布式服务器上的Spark Streaming.
- 流处理系统 /
- 关系型查询 /
- GPU~数据库
Abstract: State-of-the-art CPU-based streaming processing systems support complex queries on large-scale datasets. However, limited by CPU computational capability, these systems suffer from the performance tradeoff between throughput and response time, and cannot achieve the best of both. In this paper, we propose a GPU-based streaming processing system, named Serval, that co-utilizes CPU and GPU resources and efficiently processes streaming queries by micro-batching. Serval adopts the pipeline model and uses streaming execution cache to optimize throughput and response time on large scale datasets. To meet the demands of various scenarios, Serval implements multiple tuning policies by scaling the micro-batch size dynamically. Experiments show that a single-server Serval outperforms a 3-server distributed Spark Streaming by 3.87x throughput with a 91% response time on average, reflecting the efficiency of the optimization.
- streaming processing system /
- relational query /
- GPU database

HTML全文

图 1 Serval架构总览

Fig. 1 Architecture overview of Serval

下载: 全尺寸图片幻灯片

图 2 流水线模型

Fig. 2 Pipeline model

下载: 全尺寸图片幻灯片

图 3 与现有流处理系统对比:吞吐量

Fig. 3 Comparison of different streaming processing systems: throughput

下载: 全尺寸图片幻灯片

图 4 与现有流处理系统对比:响应时间

Fig. 4 Comparison of different streaming processing systems: latency

下载: 全尺寸图片幻灯片

图 5 Q14 (SF100)执行分析

Fig. 5 Q14 (SF100) execution analysis

下载: 全尺寸图片幻灯片

图 6 执行时间花销分析

Fig. 6 Execution time cost analysis

下载: 全尺寸图片幻灯片

参考文献(18)

[1]	ZAHARIA M, DAS T, LI H Y, et al. Discretized streams: An efficient and fault-tolerant model for stream processing on large clusters[C]//Proceedings of the 4th Workshop on Hot Topics in Cloud Computing. USENIX Association, 2012.
[2]	IQBAL M H, SOOMRO T R. Big data analysis:Apache storm perspective[J]. International Journal of Computer Trends and Technology, 2015, 19(1):9-14. doi: 10.14445/22312803/IJCTT-V19P103
[3]	BRESS S, KÖCHER B, HEIMEL M, et al. Ocelot/HyPE:Optimized data processing on heterogeneous hardware[J]. Proceedings of the VLDB Endowment, 2014, 7(13):1609-1612. doi: 10.14778/2733004.2733042
[4]	CARBONE P, KATSIFODIMOS A, EWEN S, et al. Apache Flink:Stream and batch processing in a single engine[J]. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 2015, 36(4):28-38. http://cn.bing.com/academic/profile?id=af3e8fbd6cabf93159674a3b0713e6b1&encoded=0&v=paper_preview&mkt=zh-cn
[5]	ZHANG S, HE J, HE B, et al. Omnidb:Towards portable and efficient query processing on parallel CPU/GPU architectures[J]. Proceedings of the VLDB Endowment, 2013, 6(12):1374-1377. doi: 10.14778/2536274.2536319
[6]	CHEN C, LI K, OUYANG A, et al. GFlink:An in-memory computing architecture on heterogeneous CPU-GPU clusters for big data[J]. IEEE Transactions on Parallel and Distributed Systems, 2018, 29(6):1275-1288. doi: 10.1109/TPDS.2018.2794343
[7]	Nvidia Cooperation. CUDA C Programming Guide[R/OL].(2018-04-01)[2019-05-02]. https://docs.nvidia.com/cuda/archive/9.1/pdf/CUDACProgrammingGuide.pdf.
[8]	BRESS S, HEIMEL M, SIEGMUND N, et al. GPU-accelerated database systems: Survey and open challenges[M]//Transactions on Large-Scale Data and Knowledge-Centered Systems XV. Berlin: Springer, 2014: 1-35.
[9]	MOSTAK T. An overview of MapD (massively parallel database)[R]. White paper. Massachusetts Institute of Technology, 2013.
[10]	ROOT C, MOSTAK T. MapD: A GPU-powered big data analytics and visualization platform[C]//ACM SIGGRAPH 2016 Talks. ACM, 2016: 73.
[11]	Kinetica DB Inc. Kinetica high performance analytics database[EB/OL].[2019-05-11]. https://www.kinetica.com.
[12]	SQream Technologies. SQream: Big Data SQL database[EB/OL].[2019-05-02]. https://sqream.com/.
[13]	CHEN Z, XU J, TANG J, et al. GPU-accelerated high-throughput online stream data processing[J]. IEEE Transactions on Big Data, 2016, 4(2):191-202. http://cn.bing.com/academic/profile?id=0bbd388e2d6d6e5d91b2beb0d7b08246&encoded=0&v=paper_preview&mkt=zh-cn
[14]	CHEN C, LI K, OUYANG A, et al. GFlink:An in-memory computing architecture on heterogeneous CPU-GPU clusters for big data[J]. IEEE Transactions on Parallel and Distributed Systems, 2018, 29(6):1275-1288. doi: 10.1109/TPDS.2018.2794343
[15]	ZHANG Y, MUELLER F. GStream: A general-purpose data streaming framework on GPU clusters[C]//2011 International Conference on Parallel Processing. IEEE, 2011: 245-254.
[16]	KIM J, SEO S, LEE J, et al. SnuCL: An OpenCL framework for heterogeneous CPU/GPU clusters[C]//Proceedings of the 26th ACM International Conference on Supercomputing. ACM, 2012: 341-352.
[17]	HEWANADUNGODAGE C, XIA Y, LEE J J. GStreamMiner: A GPU-accelerated data stream mining framework[C]//Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. ACM, 2016: 2489-2492.
[18]	HUYNH H P, HAGIESCU A, WONG W F, et al. Scalable framework for mapping streaming applications onto multi-GPU systems[C]//ACM Sigplan Notices. ACM, 2012, 47(8): 1-10.