针对高竞争电商负载的事务处理优化原型系统

张舒燕; 王清帅; 张蓉

doi:10.3969/j.issn.1000-5641.202091005

针对高竞争电商负载的事务处理优化原型系统

doi: 10.3969/j.issn.1000-5641.202091005

华东师范大学数据科学与工程学院, 上海　200062

基金项目: 国家重点研发计划(2018YFB1003404)

详细信息

通讯作者:
张　蓉, 女, 教授, 博士生导师, 研究方向为分布式数据管理. E-mail: rzhang@dase.ecnu.edu.cn

中图分类号: TP392
计量
- 文章访问数: 181
- HTML全文浏览量: 123
- PDF下载量: 14
- 被引次数: 0
出版历程
- 收稿日期: 2020-08-05
- 网络出版日期: 2020-09-24
- 刊出日期: 2020-09-24

High contention transaction processing prototype for e-commerce workloads

School of Data Science and Engineering, East China Normal University, Shanghai　200062, China

摘要

摘要: 现代多核主存数据库在高竞争的负载下仍然不能达到理想的性能. 获得高吞吐量的障碍是试图访问相同数据的并发冲突事务. 这些事务争用相同的资源, 在传统数据库中必须串行执行. 促销活动中的电子商务(电商)负载就是这种高冲突的事务. 本研究从两个方面对电商负载的事务处理方案进行了优化. 首先, 由于产品数量有限, 许多购买请求不会成功. 数据库系统可以通过提前过滤掉无效请求来节省资源、降低锁竞争. 其次, 大量的写操作针对同一商品, 故在写操作之间实现锁共享, 再次降低锁竞争. 基于此想法本文实现了原型系统Filmer. 大量的实验表明, 过滤和合并可以提高处理高竞争电商负载的效率.
- 事务处理 /
- 并发控制 /
- 高竞争
Abstract: Modern multi-core main-memory databases still cannot achieve ideal performance under high contention. Throughput is considered to be choked by concurrent transactions trying to modify the same data. These transactions contend for the same resources and must be executed serially in a traditional database. Unfortunately, e-commerce workloads often meet with high contentions due to promotions. In this paper, we optimize the transaction processing scheme for high contention e-commerce workloads from two aspects. First, prefiltering is designed to filter invalid modifications to the databases, which can mitigate the contention for locks. Second, if a large number of writes are similar, we propose to do lock sharing among similar writes. We implement a prototype of our proposed system, Filmer, to demonstrate the idea. Extensive experiments have shown that filtering and merging improve efficiency in handling high contention e-commerce workloads.
- transaction processing /
- concurrency control /
- high contention

HTML全文

图 1 Filmer的整体架构和数据流

Fig. 1 The overall architecture and workflow of Filmer

下载: 全尺寸图片幻灯片

图 2 PO-Map结构

Fig. 2 The structure of the PO-Map

下载: 全尺寸图片幻灯片

图 3 商品数不足时Filmer吞吐随时间的变化

Fig. 3 Change in the Filmer throughput when the number of items is insufficient

下载: 全尺寸图片幻灯片

图 4 探索高竞争下Filmer处理无效操作的可扩展性

Fig. 4 Exploring the scalability of Filmer to handle ineffective operations

下载: 全尺寸图片幻灯片

图 5 高冲突下Filmer合并功能提升系统处理效率

Fig. 5 Efficiency is improved by Filmer’s merging algorithm under high conflict scenarios

下载: 全尺寸图片幻灯片

图 6 高并发下Filmer合并功能提升系统处理效率

Fig. 6 Efficiency is improved by Filmer’s merging algorithm under high concurrency scenarios

下载: 全尺寸图片幻灯片

参考文献(18)

[1]	HUANG G, CHENG X, WANG J, et al. X-Engine: An optimized storage engine for large-scale E-commerce transaction processing [C]// Proceedings of the 2019 International Conference on Management of Data, SIGMOD Conference 2019. Amsterdam, 2019: 651-665.
[2]	PAN W, LI Z, ZHANG Y, et al. The new hardware development trend and the challenges in data management and analysis [J]. Data Science and Engineering, 2018, 3(3): 263-276.
[3]	NARULA N, CUTLER C, KOHLER E, et al. Phase reconciliation for contended in-memory transactions [C]// 11th USENIX Symposium on Operating Systems Design and Implementation, OSDI’14. Broomfield, CO, 2014: 511-524.
[4]	FALEIRO J M, ABADI D J. Rethinking serializable multiversion concurrency control [J]. Proceedings of the VLDB Endowment, 2015, 8(11): 1190-1201.
[5]	PANDIS I, JOHNSON R, HARDAVELLAS N, et al. Data-oriented transaction execution [J]. Proceedings of the VLDB Endowment, 2010, 3(1): 928-939.
[6]	HASTORUN D, JAMPANI M, KAKULAPATI G, et al. Dynamo: Amazon’s highly available key-value store [C]// Proceedings of the 21st ACM Symposium on Operating Systems Principles 2007, SOSP 2007. Stevenson, Washington, 2007: 205-220.
[7]	REN K, FALEIRO J M, ABADI D J. Design principles for scaling multi-core oltp under high contention [C]// Proceedings of the 2016 International Conference on Management of Data, SIGMOD Conference 2016. San Francisco, 2016: 1583-1598.
[8]	TIAN B, HUANG J, MOZAFARI B, et al. Contention-aware lock scheduling for transactional databases [J]. Proceedings of the VLDB Endowment, 2018, 11(5): 648-662.
[9]	WANG T, KIMURA H. Mostly-optimistic concurrency control for highly contended dynamic workloads on a thousand cores [J]. Proceedings of the VLDB Endowment, 2016, 10(2): 49-60.
[10]	SELLIS T K. Multiple-query optimization [J]. ACM Transactions on Database Systems, 1988, 13(1): 23-52.
[11]	MAKRESHANSKI D, GIANNIKIS G, ALONSO G, et al. MQJoin: Efficient shared execution of main-memory joins [J]. Proceedings of the Endowment, 2016, 9(6): 480-491.
[12]	CANDEA G, POLYZOTIS N, VINGRALEK R. Predictable performance and high query concurrency for data analytics [J]. The VLDB Journal, 2011, 20(2): 227-248.
[13]	GIANNIKIS G, ALONSO G, KOSSMANN D. SharedDB: Killing one thousand queries with one stone [J]. Proceedings of the VLDB Endowment, 2012, 5(6): 526-537.
[14]	MAKRESHANSKI D, GICEVA J, BARTHELS C, et al. BatchDB: Efficient isolated execution of hybrid OLTP+ OLAP workloads for interactive applications [C]// Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD Conference 2017. Chicago: ACM, 2017: 37-50.
[15]	REHRMANN R, BINNIG C, BÖHM A, et al. Oltpshare: The case for sharing in OLTP workloads [J]. Proceedings of the VLDB Endowment, 2018, 11(12): 1769-1780.
[16]	ZHANG C, LI Y, ZHANG R, et al. Benchmarking on intensive transaction processing [J]. Frontiers of Computer Science, 2020, 14(5): 1-18.
[17]	BERNSTEIN P A, HADZILACOS V, GOODMAN N. Concurrency Control and Recovery in Database Systems [M]. Massachusetts: Addison-Wesley, 1987.
[18]	RODEH O. B-trees, shadowing, and clones [J]. ACM Transactions on Storage, 2008, 3(4): 1-27.