Woodpecker: Fine-grained contention simulation database testing framework

LI Jie-ying; LI Yu-ming; ZHANG Xiao-lei; ZHANG Rong

doi:10.3969/j.issn.1000-5641.2018.02.008

Issue 2

Mar. 2018

Turn off MathJax

Article Contents

Article Navigation > Journal of East China Normal University (Natural Sciences) > 2018 > (2): 77-88

LI Jie-ying, LI Yu-ming, ZHANG Xiao-lei, ZHANG Rong. Woodpecker: Fine-grained contention simulation database testing framework[J]. Journal of East China Normal University (Natural Sciences), 2018, (2): 77-88. doi: 10.3969/j.issn.1000-5641.2018.02.008

Citation:

LI Jie-ying, LI Yu-ming, ZHANG Xiao-lei, ZHANG Rong. Woodpecker: Fine-grained contention simulation database testing framework[J]. Journal of East China Normal University (Natural Sciences), 2018, (2): 77-88. doi: 10.3969/j.issn.1000-5641.2018.02.008

Citation:

PDF( 695 KB)

Woodpecker: Fine-grained contention simulation database testing framework

doi: 10.3969/j.issn.1000-5641.2018.02.008

School of Data Science and Engineering, East China Normal University, Shanghai 200062, China

Received Date: 2017-06-19
Publish Date: 2018-03-25

Abstract

Abstract

With the rich and the growing peformance requirement of applications, various database developments appear, such as row-based, column-based, main memory database and so on. Hence database testing becomes more and more important. Existing automated testing frameworks of database have some problems in flexibility of test, integrality of function and the granularity of test. Firstly they can't support functional test, performance test and DBMS functional test at the same time. Secondly popular benchmark TPC-C simulates workload contention by warehouse which can't quantify contention and maybe execution is different with expectation. Finally the use of test case is static, they can't organize and run test case dynamically. As a consequence this paper presents a flexible database testing framework which is called Woodpecker, its test task descriptive language has abundant primitives, easy-to-use and could write test case efficiently. Woodpecker is the first test framework that can mix three above-mentioned tests in one test case, and simulates workload contention in fine granularity which include read-write contention and write-write contention. We use a simple case in auto increment testing to show the descriptive language's advantage, last we verify the capacity of mixing three tests in Woodpecker, and besides, we show the results of workload contention simulation in performance test.
- database,
- testing framework,
- contention simulation

FullText(HTML)

References(19)

References

[1]	GEORGE L. HBase-The Definitive Guide: Random Access to Your Planet-Size Data[M]. [S. l. ]: O'Reilly, 2011.
[2]	STONEBRAKER M, WEISBERG A. The VoltDB Main Memory DBMS[J]. IEEE Data Eng Bull, 2013, 36(2):21-27. http://www.researchgate.net/publication/297362705_The_voltdb_main_memory_dbms
[3]	FARBER F, CHA S K, PRIMSCH J, et al. SAP HANA database:Data manä gement for modern business applications[J]. ACM Sigmod Record, 2012, 40(4):45-51. doi: 10.1145/2094114
[4]	MARICK B. When should a test be automated?[C/OL]//Proceedings of the International Software/Internet Quality Week. [2017-04-21]. http://www.uml.org.cn/Test/12/automate.pdf.
[5]	ECNU. CEDAR: A distributed, shared-nothing relational database[EB/OL]. (2016-09-26)[2017-04-21]. https://github.com/daseECNU/Cedar.
[6]	ORACLE. The MySQL Test Framework[EB/OL]. (2016-02-11)[2017-07-12]. https://dev.mysql.com/doc/mysqltest/2.0/en/.
[7]	LEUTENEGGER S T, DIAS D. A modeling study of the TPC-C benchmark[J]. ACM Sigmod Record, 1993, 22(2):22-31. doi: 10.1145/170036
[8]	YAN C, CHEUNG A. Leveraging lock contention to improve OLTP application performance[J]. Proceedings of the VLDB Endowment, 2016, 9(5):444-455. doi: 10.14778/2876473
[9]	PARRISH A S, ZWEBEN S H. Clarifying some fundamental concepts in software testing[J]. IEEE Transactions on Software Engineering, 1993, 19(7):742-746. doi: 10.1109/32.238573
[10]	KOREL B. Automated software test data generation[J]. Software Engineering IEEE Transactions on, 1990, 16(8):870-879. doi: 10.1109/32.57624
[11]	DEMILLO R A. Progress toward automated software testing[C]//International Conference on Software Engineering. IEEE Computer Society Press, 1991: 180-183.
[12]	AHMAD S. XML APIs testing using advance data driven techniques (ADDT)[EB/OL]. (2003-11-17)[2017-04-21]. https://www.stickyminds.com/article/xml-apis-testing-using-advance-data-driven-techniques-addt.
[13]	曹娜. 大型关系数据库自动化测试工具的设计与实现[D]. 武汉: 华中科技大学, 2009. http://cdmd.cnki.com.cn/Article/CDMD-10487-2010212391.htm
[14]	FRANKL P G, WEYUKER E J. Testing software to detect and reduce risk[J]. Journal of Systems & Software, 2000, 53(3):275-286. http://www.sciencedirect.com/science/article/pii/S0164121200000182
[15]	COOPER B F, SILBERSTEIN A, TAM E, et al. Benchmarking cloud serving systems with YCSB[C]//Proceedings of the 1st ACM Symposium on Cloud Computing. ACM, 2010: 143-154.
[16]	IBM. TATP: Telecom application transaction processing benchmark[EB/OL]. (2011-07-06)[2017-04-21]. http://tatpbenchmark.sourceforge.net.
[17]	KODAGANALLUR V. Incorporating language processing into Java applications:A JavaCC tutorial[J]. IEEE Software, 2004, 21(4):70-77. doi: 10.1109/MS.2004.16
[18]	SILBERSCHATZ A, KORTH H F, SUDARSHAN S. Database System Concepts[M], 4th ed. Columbus:McGraw-Hill Education, 2001.
[19]	BERNSTEIN P A, GOODMAN N. Multiversion concurrency control-theory and algorithms[J]. ACM Transactions on Database Systems, 1983, 8(4):465-483. doi: 10.1145/319996.319998