{
    "componentChunkName": "component---src-templates-case-study-detail-ts",
    "path": "/case-studies/20",
    "result": {"pageContext":{"next":{"id":27,"attributes":{"feature":false,"title":"Apache ShardingSphere & openGauss: Breaking the Distributed Database Performance Record with 10 Million tpmC","content":"Our open source community has cooperated Huawei’s openGauss to build a distributed solution with [Apache ShardingSphere](https://shardingsphere.apache.org/) and [openGauss](https://opengauss.org/en/).\n\nWe tested performance together with openGauss on 16 servers for more than one hour. The results were great: our joint solution broke the performance bottleneck of a single machine with a benchmark result of 10 million transactions per minute (tpmC) on average.\n\n\n## Breaking the 10 Million tpmC Barrier\nIn this test, the openGauss community ran this TPC-C testing on [BenchmarkSQL](https://www.postgresql.org/message-id/CAGBW59f9q2Y4v-B3D8gje3xUsWz2Z-WaK9wYx4%3DhCY1zN%3D7%2BBQ%40mail.gmail.com) 5.0, which is an open source implementation of the popular TPC/C OLTP database benchmark.\n\nIn terms of stand-alone performance, openGauss with ShardingSphere broke the limit of multi-core CPU: two-way 128-Core Huawei Kunpeng reached 1.5 million tpmC, and the memory-optimized table (MOT) engine reached 3.5 million tpmC.\n\nThese are great results, but we’re not done. We’ll never stop pushing the boundaries for better database performance — especially in today’s Big Data scenarios and their thirst for top notch database performance.\n\n![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tz0oyppd6vh7l1466ug5.png)\n\nIn this case, the openGauss team used 7 machines to run BenchmarkSQL adapted to [ShardingSphere-JDBC](https://shardingsphere.apache.org/document/current/en/overview/#shardingsphere-jdbc), connected 8 openGauss databases, and deployed [ShardingSphere-Proxy](https://shardingsphere.apache.org/document/current/en/overview/#shardingsphere-jdbc) for data initialization, consistency verification, and other maintenance operations.\n\nThanks to its database sharding capability, ShardingSphere enabled a total of 8,000 bins of data (over 800 GB) to be distributed across 8 openGauss nodes. Following over 1 hour of test, not only sharding was perfect but the average results also reached over 10 million tpmC, which is the best industry performance at this scale.\n\n\n## ShardingSphere & openGauss: Building an Ecosystem Cooperation\n\nThe Apache ShardingSphere community has been working closely with the openGauss community since 2021.\n\nFaced with the diversification of business scenarios and data volume expansion, the traditional solution that centrally stores data to a single node has since become unable to meet needs in terms of performance, availability, and affordable operation cost.\n\nDatabase sharding can solve problems of performance, availability, as well as single-point backup and recovery of stand-alone databases — but it also makes distributed architecture more complex.\n\nAs the proponent of the Database Plus concept, Apache ShardingSphere aims to build a criterion and ecosystem above heterogeneous databases and enhance the ecosystem with sharding, elastic scaling, encryption features & more. Placed above databases, ShardingSphere focuses on the collaborative way of databases to make reasonable and full use of database compute and storage capabilities.\n\nCurrently Apache ShardingSphere has a microkernel plus plugin-oriented architecture model, and on this basis, it continues to improve the capabilities of its kernel and functions to provide increasingly flexible solutions.\n\nThanks to the design concept of its pluggable architecture, ShardingSphere can support openGauss without additional changes and only needs to increase implementations of the corresponding openGauss database based on the SPI extension points provided by each ShardingSphere module .\n\nOur two communities have collaborated to create a distributed database solution suitable for highly-concurrent Online Transaction Processing (OLTP) scenarios by combining the powerful standalone performance of openGauss with the distributed capabilities provided by the Apache ShardingSphere ecosystem.\n\n\n## Building an openGauss-based Distributed Database Solution with ShardingSphere\nApache ShardingSphere includes many features such as database sharding, read/write splitting, data encryption, and shadow database. The features can be used independently or in combination.\n\n![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jq8uhm2w25yk8r3n8xvh.png)\n\nCurrently, ShardingSphere provides users with two access methods, namely ShardingSphere-JDBC and ShardingSphere-Proxy.\n\nShardingSphere-JDBC can easily and transparently perform operations such as sharding and read/write splitting on databases while meeting high concurrency and low latency needs.\n\nShardingSphere-Proxy is deployed to add some database capabilities and operations at the proxy level, enabling users to operate ShardingSphere as if it was a native database for a better user experience.\n\nShardingSphere-JDBC and ShardingSphere-Proxy can be deployed together. We recommend using this mixed deployment in order to make the system user-friendly and perform better.\n\n\n![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nla348ta7qgxuwa8ghb6.png)\n\nFrom the perspective of the openGauss system, Apache ShardingSphere can shard the database horizontally to greatly enhance compute and storage capabilities, as well as database performance.\n\nThis means it can effectively solve problems caused by increasing data volume in a single table and can be combined with business data flows to flexibly and smoothly scale out data nodes, intelligently split reads and writes, and implement automatic load balancing of distributed databases.\n\n\n## Conclusion\nApache ShardingSphere and openGauss can seek potential cooperation opportunities.\n\nConsidering the increasingly diversified applicaiton scenarios and increasing data volume, the requirements for database performance are at an all time high and will only continue to increase in the future.\n\nThe success of our two communities cooperation is just the beginning of ourtwo communities building a collaborative database ecosystem.\n\n💡 About openGauss\n\nopenGauss is an open source relational database management system. It has enterprise-grade features such as multi-core high performance, full-link security, and intelligent operation.\n\nIt integrates Huawei’s years of kernel development experience in the database field and makes adaptations and optimizations on architecture, transaction, storage engine, optimizer, and ARM architecture.\n\n💡 About TPC-C\n\nTransaction Processing Performance Council Benchmark C or TPC-C is a benchmark used to compare the performance of online transaction processing (OLTP) systems. It was released by Transaction Processing Performance Council (TPC) in 1992. The latest update is TPC-C v5.11 published in 2010.\n\nTPC-C involves a mix of five concurrent transactions of different types and complexity either executed online or queued for deferred execution. The database is comprised of nine types of tables with a wide range of record and population sizes.\n\nTPC-C is measured in transactions per minute (tpmC). While the benchmark portrays the activity of a wholesale supplier, TPC-C is not limited to the activity of any particular business segment, but, rather represents any industry that must manage, sell, or distribute a product or service.\n\n\nApache ShardingSphere Project Links:\n\n[ShardingSphere Github](https://github.com/apache/shardingsphere/issues?page=1&q=is%3Aopen+is%3Aissue+label%3A%22project%3A+OpenForce+2022%22)\n\n[ShardingSphere Twitter](https://twitter.com/ShardingSphere)\n\n[ShardingSphere Slack](https://join.slack.com/t/apacheshardingsphere/shared_invite/zt-sbdde7ie-SjDqo9~I4rYcR18bq0SYTg)\n\n[Contributor Guide](https://shardingsphere.apache.org/community/cn/contribute/)","date":"2022-04-21","author":"SphereEx","excerpt":"ShardingSphere collaborated with openGauss to create a distributed database solution suitable for highly-concurrent Online Transaction Processing (OLTP) scenarios.","metaDescription":"openGauss is an open source relational database management system that is released with the Mulan PSL v2. with the kernel built on Huawei’s years of experience in the database field and continuously provides competitive features tailored to enterprise-grade scenarios. In addition, openGauss is an open source database platform that encourages community contribution and collaboration.","createdAt":"2022-09-26T08:12:02.238Z","updatedAt":"2022-09-27T07:00:09.560Z","publishedAt":"2022-09-27T07:00:09.551Z","locale":"en","caseStudiesType":{"data":{"id":10,"attributes":{"name":"Software Development","createdAt":"2022-09-26T10:34:53.523Z","updatedAt":"2022-09-27T06:59:43.907Z","publishedAt":"2022-09-27T06:59:43.905Z","locale":"en"}}},"cover":{"data":{"id":348,"attributes":{"name":"openGauss2.png","alternativeText":"openGauss2.png","caption":"openGauss2.png","width":720,"height":192,"formats":{"thumbnail":{"name":"thumbnail_openGauss2.png","hash":"thumbnail_open_Gauss2_da22098be0","ext":".png","mime":"image/png","width":245,"height":65,"size":9.07,"path":null,"url":"https://official-media.sphere-ex.com/thumbnail_open_Gauss2_da22098be0.png"},"small":{"name":"small_openGauss2.png","hash":"small_open_Gauss2_da22098be0","ext":".png","mime":"image/png","width":500,"height":133,"size":26.12,"path":null,"url":"https://official-media.sphere-ex.com/small_open_Gauss2_da22098be0.png"}},"hash":"open_Gauss2_da22098be0","ext":".png","mime":"image/png","size":34.31,"url":"https://official-media.sphere-ex.com/open_Gauss2_da22098be0.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-09-26T08:11:48.341Z","updatedAt":"2022-09-26T08:11:48.341Z"}}},"metaLogo":{"data":{"id":361,"attributes":{"name":"openGauss logo.png","alternativeText":"openGauss logo.png","caption":"openGauss logo.png","width":275,"height":183,"formats":{"thumbnail":{"name":"thumbnail_openGauss logo.png","hash":"thumbnail_open_Gauss_logo_7b11da9186","ext":".png","mime":"image/png","width":234,"height":156,"size":11.4,"path":null,"url":"https://official-media.sphere-ex.com/thumbnail_open_Gauss_logo_7b11da9186.png"}},"hash":"open_Gauss_logo_7b11da9186","ext":".png","mime":"image/png","size":3.07,"url":"https://official-media.sphere-ex.com/open_Gauss_logo_7b11da9186.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-09-26T09:11:52.021Z","updatedAt":"2022-09-26T09:11:52.021Z"}}},"localizations":{"data":[]}}},"prev":{"id":19,"attributes":{"feature":false,"title":"Apache ShardingSphere & Huimin: Huimin’s Transaction Middle Platform Architecture","content":"The FMCG (fast moving consumer-goods) industry can no longer meet current development requirements. The traditional way of only upgrading the end of the supply chain cannot achieve high-quality business development. [Huimin](https://huimin.cn/) focuses on order fulfillment and delivery service for over one million community supermarkets, transforming and upgrading from serving retail terminal stores to promoting the digitization of the whole FMCG industry chain.\n\nIt is committed to building a B2B2C data closed-loop operating system, providing brand owners, distributors, and community supermarkets with transformed and upgraded solutions for the whole chain covering purchasing, marketing, operation, and distribution. All these commitments cannot be fulfilled without its powerful “middle platform” system.\n\n## Introducing the “middle platform”\nHuimin began to transferred [PHP](https://www.php.net/) technology stack to the Java technology stack, and shifted to microservices from 2016 to 2017, and started to build the “middle platform” in 2018.\n\nThe phase-one project of its transaction middle platform went live in early 2021, and phase-two in March 2022.\n\nOwing to business growth and various changes, Huimin launched a strong “middle platform” development strategy to improve efficiency and reduce costs. Measures were taken to reduce the coupling degree between systems, increase scalability, and ensure low latency while extracting business commonalities. At the beginning of 2021, the company finihsed rebuilding the previous order management system (OMS) and launched a packaged business capability project for a transaction middle platform centered on the transaction process of orders, which is divided into the following four core modules: order status flow system, order management system, order fulfillment system and order cost calculation system. At the same time, based on business stratification, the project is also split into an order query system, report system, and order management backend.\n\nTo synergize technical and business requirements during the rebuilding process, the go-live process was divided into two phases: application system splitting and data splitting.\n\n## System splitting\nR&D teams having to frequently adjust order management system and adapt it to multiple business lines may cause inefficiencies. In response to these problems, the following principles were formulated to split and rebuild the system:\n\n1. Avoid the logic coupling of different lines of business. Different business lines should contain and minimize influence on each other to reduce logic complexity.\n2. Responsibilities are divided according to the order lifecycle to ensure that each system is independent with an exclusive responsibility.\n3. Read/write splitting. Ensure the stability of core functions and avoid the impact of frequent iterations on core functions.\n4. [ElasticSearch](https://www.elastic.co/) was introduced to solve the external index problem, and reduce database index pressure.\n![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1223xl1pboe6emvrjian.png)\n \n\n## Data splitting\nThe purpose of data splitting is to reduce the maintenance pressure of a single table. When the amount of data reaches tens of millions, the index and field maintenance of database tables will have a great impact on the online environment. Before splitting a table, you have to consider the following questions:\n\n1. Splitting rules: what tables are data-intensive? What about the number of splits?\n2. Data read/write: how to solve the read/write problem after splitting tables?\n3. Go-live solution: how to go live smoothly?\n\n**Splitting rules**\n\nDuring a 3 years iteration period, the number of splits = ((amount of data added per month * 36) + Amount of historical data) / maximum amount of data in a single table.\n\nWhen we use the [InnoDB](https://en.wikipedia.org/wiki/InnoDB) engine for DDL operation to maintain [Alibaba Cloud](https://www.alibabacloud.com/?spm=a2796.143805.6791778070.2.b8fa46e3JLecKh) RDS MySQL, we come up with the following results:\n\nWhen the amount of data is less than 5 million, the execution time is about tens of seconds. Between 5 million and less than 10 million, the execution time is about 500 seconds; between 10 million and less than 50 million, the execution time is about 1000 seconds; and when more than 50 million, the execution time is more than 2000 seconds.\n\nThe upper limit for splitting a single table depends on the impact of a single table operation on business. Of course, some solutions can tackle the problem of table lock resulting from DDL operation. For example, dual-table synchronization and changes in table names can reduce the impact to within seconds.\n\nThe split number of the library table is preferably 2 to the power of N, which is conducive to later horizontal scaling.\n\n**Technology selection**\n\nAfter the data table is split, data hashing and query issues need to be resolved. As a SME (small & medium-sized enterprise), the cost of developing a set of sharding middleware is too high, and it will cause various risks in the early stage. A reasonable solution is to adopt a set of open source database sharding middleware.\n\n**[Apache ShardingSphere](https://shardingsphere.apache.org/)**\n\nApache ShardingSphere is an open source ecosystem that allows you to transform any database into a distributed database system. The project includes a JDBC, a Proxy and Sidecar (Planning), which can be deployed independently and support mixed deployment. They all provide functions of data scale-out, distributed transactions and distributed governance.\n\nApache ShardingSphere 5.x introduces the concept of [Database Plus](https://faun.pub/whats-the-database-plus-concepand-what-challenges-can-it-solve-715920ba65aa), and the new version focuses on a plugin oriented architecture. Currently, functions including data sharding, read/write splitting, data encryption, shadow library pressure testing, as well as SQL and protocol support such as [MySQL](https://www.mysql.com/), [PostgreSQL](https://www.postgresql.org/), [SQL Server](https://www.microsoft.com/en-us/sql-server/sql-server-downloads), [Oracle](https://www.oracle.com/index.html), etc. are woven into the project through plugins, allowing developers to customize their own unique systems.\n\nWhy Huimin chose Apache ShardingSphere:\n\n1. Apache ShardingSphere has a variety of functions and can solve current challenges with strong scalability.\n\n2. The Apache ShardingSphere community is very active, which means the possibility to find support quickly.\n\n3. The company adopts the SpringCloud technology stack which is convenient for integration and can lower costs.\n\n4. Its performance can fully support Huimin’s existing business.\n\nShardingSphere supports the following 3 modes:\n![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/19343yqtstyzsmvtu69j.png)\n \n\nCurrently, Huimin’s server-side technology stack only involves Java language, so there is no need to consider heterogeneous scenarios. Taking flexibility, code intrusion and deployment cost into consideration, the implementation of [ShardingSphere-JDBC](https://shardingsphere.apache.org/document/current/en/overview/#shardingsphere-jdbc) wasfinally selected.\n\n**Technology implementation**\n\nThe most complicated part of the process is actually the go-live part, not the plugin integration part as would someone expect.\n\nTo achieve non-perception and rollback, we divided the go-live process into the following steps, as shown in the figure:\n![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dyrbcvaql5nhnw4m0xns.png)\n \n\n**Step 1:** Processing all the data needs to ensure the data consistency between the old and new libraries. Since there currently is no synchronization tool supporting data sharding policy, we developed a data synchronization tool to support the configuration of the sharding policy.\n\n**Step 2:** Process incremental data to keep data consistency between the old and new libraries. Our implementation scheme here was to use the open source component, to listen to the binlog of the database and synchronize the database changes to the message queue, and then the data synchronization tool listens to the messages and writes them into the new library.\n\nThe architecture scheme is not complicated, but great importance was attached to the data consistency in this process. Incremental data cannot be lost, and single-line data should be written in sequential order to prevent ABA problems.\n\n**Step 3:** Process the gray-release of read-traffic. At this point, we were writing an old library and reading a new library in the database. In this process, we should paid attention to the sensitivity of business to data consistency, because the delay from an old library to a new library is within seconds. Reading an old library is needed when we come across some sensitive scenarios requiring high consistency.\n\n**Step 4:** Switch all read-traffic of the application to the new library, maintaining the status of writing the old library and reading the new library.\n\n**Step 5:** Processing write-traffic didn’t take gray-release scenarios into consideration in order to reduce the complexity of the program. Of course, the grayscale scheme can also achieve the same effect. We supported the rollback strategy by writing data from the new library back to the old library, and then uniformly transferring all traffic to the new library within one release.\n\nSo far, the whole process of release and going live has been completed. Due to the low cost of completing the transformation with the help of ShardingSphere-JDBC middleware to rewrite SQL and merge results, there was almost no code intrusion and going live went smoothly.\n\nThere are also some issues that you may need to be aware of when accessing the ShardingSphere-JDBC scheme:\n\n1. ShardingSphere-JDBC supports a default processing for data insertion logic. If there is no shard column in SQL, the same data will be inserted into each shard table.\n\n2. ShardingSphere-JDBC parses only the logil table of the first SQL for batches of SQL semantic analysis, which results in an error. This problem has been reported and should be fixed in the near future. Additionally, a detailed SQL sample is provided, which lists in detail the scope of support.\n\n3. ShardingSphere connection mode selection. The number of database connections visited by businesses should be limited for the sake of resource control. It can effectively prevent a service operation from occupying too many resources, which depletes the database connection resources and affects the normal access of other services. Especially in the case of many sub-tables in a database instance, a logical SQL without shard keys will produce a large number of actual SQL that falls on different tables in the same library. If each actual SQL occupies a separate connection, a query will undoubtedly occupy too many resources.\n\nFrom the perspective of execution efficiency, maintaining a separate database connection for each shard query can make more effective use of multi-threading to improve execution efficiency. Building a separate thread for each database connection allows I/O consumption to be processed concurrently, maintains a separate database connection for each shard, and avoids premature loading of query result data into memory. A separate database connection can hold a reference to the cursor location of the query result set and move the cursor when it needs to fetch the corresponding data.\n\nThe result merging method by moving the result set cursor down is called streaming merge. It does not have to load all the result data into memory, which can effectively save memory resources and reduce the frequency of garbage collection. If each shard query cannot be guaranteed to have a separate database connection, the current query result set needs to be loaded into memory before reusing the database connection to obtain the query result set of the next shard table. Therefore, even if streaming merge could be used, it would degenerate into a memory merge in this scenario.\n\nIt is a problem for the ShardingSphere execution engine to balance the relationship between the control and protection of database connection resources and saving middleware memory resources by adopting a better merging mode. Specifically, if a SQL needs to operate on 200 tables under a database instance after its sharding in ShardingSphere, whether to create 200 connections and execute them concurrently or create one connection and execute it in sequence? How to choose between efficiency and resource control?\n\nIn response to the above scenario, ShardingSphere provides a solution. It introduces the concept of Connection Mode and divides it into `MEMORY_STRICTLY` mode and `CONNECTION_STRICTLY` mode.\n\n`MEMORY_STRICTLY`\n\nThe premise of using this pattern is that ShardingSphere does not limit the number of database connections required by an operation. If the actual SQL executed needs to operate on 200 tables in a database instance, a new database connection is created for each table and processed concurrently through multi-threading to maximize the execution efficiency. When SQL meets the conditions, streaming merge is preferred to prevent memory overflow or frequent garbage collection.\n\n`CONNECTION_STRICTLY`\n\nThe premise for using this pattern is that ShardingSphere strictly limits the number of database connections required for an operation. If the actual SQL executed needs to operate on 200 tables in a database instance, only one single database connection is created and its 200 tables are processed sequentially. If shards in an operation are scattered across different databases, multi-threading is still used to handle operations on different libraries, but only one database connection is created for each operation of each library. The practice prevents the problem of consuming too many database connections for one request. This pattern adopts memory merge all the time.\n\nThe `MEMORY_STRICTLY` mode is applicable for OLAP operations and can improve system throughput by relaxing limits on database connections. The `CONNECTION_STRICTLY` mode applies to OLTP operations, which usually have shard keys and are routed to a single shard. Therefore, it is a wise choice to strictly control database connections to ensure that database resources in an online system can be used by more applications.\n\nWe found that in the `MEMORY_STRICTLY` mode, the operation would become I/O-intensive due to the cache buffer loading strategy of MySQL InnoDB engine, resulting in SQL timeouts. Its solution is to add another layer of processing without changing database resources. If there is no shard key, verify the shard key in the external index, and then use the shard key to perform database retrieval.\n\n## ShardingSphere Benefits\n**1. Performance improvement**\nThrough architecture rebuilding, it can effectively control the amount of single table data, and greatly reduce the slow SQL, down nearly 50%.\n\n**2. Save R&D resources and lower cost**\n\nThe introduction of Apache ShardingSphere does not require the re-development of sharding components, which reduces the R&D cost and lowers the risks.\n\n**3. Strong scalability**\n\nApache ShardingSphere has good scalability in terms of data encryption, distributed transactions, shadow library and other aspects.\n\n**Conclusion**\nIn his book [Building Microservices, Sam Newman](https://samnewman.io/books/building_microservices/) writes, “ Our requirements shift more rapidly than they do for people who design and build buildings — as do the tools and techniques at our disposal. The things we create are not fixed points in time. Once launched into production, our software will continue to evolve as the way it is used changes. For most things we create, we have to accept that once the software gets into the hands of our customers we will have to react and adapt, rather than it being a never-changing artifact. Thus, our architects need to shift their thinking away from creating the perfect end product, and instead focus on helping create a framework in which the right systems can emerge, and continue to grow as we learn more (Sam Newman, 2015: 15).” Apache ShardingSphere is doing exactly what the book says, and is definitely on a promising path.\n\n## Author\n**Zhai Yang**\n\nSoftware architect at Zhongshang Huimin E-Commerce platform team.\n\nParticipated in the 0–1 process of middle platform construction, mainly responsible for the R&D and building the transaction and commodity middle platform and the search platform.","date":"2022-05-17","author":"SphereEx","excerpt":"HuiMin does order fulfillment for over 1 million supermarkets. With ShardingSphere HuiMin shifted to microservices & built a new \"middle platform\" architecture with packaged business capability for order transaction processing.","metaDescription":"Huimin is a B2B e-commerce platform that focuses on small-scale supermarkets that provides community supermarket websites for communities.","createdAt":"2022-09-26T07:01:29.374Z","updatedAt":"2022-09-27T06:59:59.417Z","publishedAt":"2022-09-27T06:59:59.413Z","locale":"en","caseStudiesType":{"data":{"id":2,"attributes":{"name":"E-commerce","createdAt":"2021-12-11T23:02:08.395Z","updatedAt":"2021-12-11T23:02:11.020Z","publishedAt":"2021-12-11T23:02:10.947Z","locale":"en"}}},"cover":{"data":{"id":338,"attributes":{"name":"Zhongshanghuimin.png","alternativeText":"Zhongshanghuimin.png","caption":"Zhongshanghuimin.png","width":720,"height":339,"formats":{"thumbnail":{"name":"thumbnail_Zhongshanghuimin.png","hash":"thumbnail_Zhongshanghuimin_6ceb4a8b12","ext":".png","mime":"image/png","width":245,"height":115,"size":19.04,"path":null,"url":"https://official-media.sphere-ex.com/thumbnail_Zhongshanghuimin_6ceb4a8b12.png"},"small":{"name":"small_Zhongshanghuimin.png","hash":"small_Zhongshanghuimin_6ceb4a8b12","ext":".png","mime":"image/png","width":500,"height":235,"size":60.18,"path":null,"url":"https://official-media.sphere-ex.com/small_Zhongshanghuimin_6ceb4a8b12.png"}},"hash":"Zhongshanghuimin_6ceb4a8b12","ext":".png","mime":"image/png","size":73.86,"url":"https://official-media.sphere-ex.com/Zhongshanghuimin_6ceb4a8b12.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-09-26T07:01:49.173Z","updatedAt":"2022-09-26T07:01:49.173Z"}}},"metaLogo":{"data":{"id":353,"attributes":{"name":"zhongshanghuimin logo.png","alternativeText":"zhongshanghuimin logo.png","caption":"zhongshanghuimin logo.png","width":608,"height":430,"formats":{"thumbnail":{"name":"thumbnail_zhongshanghuimin logo.png","hash":"thumbnail_zhongshanghuimin_logo_978897c9d6","ext":".png","mime":"image/png","width":221,"height":156,"size":9.77,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/thumbnail_zhongshanghuimin_logo_978897c9d6.png"},"small":{"name":"small_zhongshanghuimin logo.png","hash":"small_zhongshanghuimin_logo_978897c9d6","ext":".png","mime":"image/png","width":500,"height":354,"size":31.4,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/small_zhongshanghuimin_logo_978897c9d6.png"}},"hash":"zhongshanghuimin_logo_978897c9d6","ext":".png","mime":"image/png","size":12.04,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/zhongshanghuimin_logo_978897c9d6.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-09-26T08:42:11.374Z","updatedAt":"2022-10-12T05:56:46.710Z"}}},"localizations":{"data":[]}}},"article":{"id":20,"attributes":{"feature":false,"title":"Apache ShardingSphere & Zhuanzhuan: Zhuanzhuan’s Transaction System with 100s of Millions of Records","content":"## Background and Challenges\n[Zhuanzhuan](https://www.zhuanzhuan.com/index.html) is an internet platform that allows it users to sell their second-hand stuff — sort of an eBay of the East. Its business had been booming, and with it the ordering system started to face increasing performances challenges. The order database is the cornerstone of the system, and its performance should not be underestimated.\n\n**Challenges:**\n\n- During promotions and special discount periods the burden on databases is heaby with tens of thousands of single database queries per second (qps) taking up huge database resources, and causing a significant reduction in write performance.\n- Increased data pressure, with a single database containing several large tables with hundreds of millions of data, which challenges the capacity limit of the server.\n- Overwhelming data volume, and data backup and recovery take a long time, posing high risks of data loss in extreme cases.\n\n## Why ShardingSphere?\nIn the beginning, ZhuanZhuan’s team took adjustment measures to ease the database pressure. Exmaples include:\n\n**- Optimized major transactions, reduced transactions, and even eliminated transactions**\n![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jgvh0xy3l3xvjkbn7c4j.png)\n \n\nWe adjusted the original transaction order by putting table generation, the core step at the end, and keeping the transaction only in the order primary database. When the operation of the main table was abnormal, dirty reads were allowed on other order-related tables.\n\n**- Order data cache**\n![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8ueib5czc3uni28w0yk5.png)\n \n\nData consistency was the trickiest part of the cache. As order data involved account settlements and commission, non-real-time and inconsistent data would cause serious accidents.\n\nStrictly keeping cache data consistency would complex coding and reduce system concurrency. Therefore, we made some compromises on cache plans:\n\n1. Allowing direct query when cache failed.\n2. Adding version serial number, and querying the latest version’s data to ensure real-time data.\n3. Complex queries were conducted by [Elasticsearch (ES)](https://www.elastic.co/) and primary and secondary separation, and for some large tables, we adopted hot and cold data separation.\n![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k92mmvjo5y2ldbkskqc6.png)\n \n\nThrough these optimizations, database pressure was eased. However, it still seemed overwhelming under high concurrency scenarios, such as discount season.\n\nTo fundamentally solve the performance problem of order database, ZhuanZhuan decided to adopt data sharding (database and table splitting) on the `order` database so that we wouldn’t have to worry about order capacity in the future 3–5 years.\n\nZhuangzhuang chose [ShardingSphere](https://shardingsphere.apache.org/) after comparing the efficiency, stability, learning cost and etc. of different data sharding components.\n\nAdvantages of ShardingSphere:\n\n- It provides standardized data sharding, distributed transactions and database governance, and it’s applicable in a variety of situations such as Java isomorphism, heterogeneous language and cloud native.\n- It has flexible sharding strategies, supporting multiple sharding methods.\n- It’s easy to integrate with other components and has a low level of transaction intrusions.\n- It has extensive documentation and an active community.\n\nShardingSphere initiated the Database Plus concept and adopts a plugin oriented architecture where all modules are independent of each other, allowing each to be used individually or flexibly combined.\n\nIt consists of three products, namely [JDBC](https://shardingsphere.apache.org/), [Proxy](https://shardingsphere.apache.org/document/current/en/overview/#shardingsphere-proxy) and [Sidecar ](https://shardingsphere.apache.org/document/current/en/overview/#shardingsphere-sidecartodo)(Planning), which supports both independent and hybrid deployment.\n\nBelow is a feature comparison of the three products:\n![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7517wpdgk6gmgb0s8202.png)\n \n\nBy comparison, and considering the high order concurrency, we chose ShardingSphere-JDBC as our data sharding middleware.\n\nShardingSphere-JDBC is a lightweight Java framework, proving extra service at the JDBC layer. It directly connects to the database by the client-side, provides services by Jar package, and requires no extra deployment and reliance. It can be seen as an enhanced JDBC driver, fully compatible with JDBC and other Object-Relational Mapping(ORM) frameworks.\n![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/luou74i1f1gpgowequk9.png)\n \n\n## Key Points in Project Implementation\n\n**- Sharding Key**\nThe current order ID is generated by `timestamp+user identification code+machine code+incremental sequence`. The user identification code is taken from bits 9 to 16 of the buyer ID, a true random number when the user ID is generated, and is thus suitable as a sharding key.\n\nChoosing user identification code as the sharding key has some advantages:\n\n- The data can be distributed as evenly as possible to each database and table.\n- Specific sharding locations can be quickly located either by order ID or user ID.\n- Data of the same buyer can be distributed to the same databases and tables, facilitating the integrated query of the buyer information.\n![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jkilhtbnnwk9gcdn7l5x.png)\n \nThe sharding strategy: we adopt 16 databases and 16 tables. User identification codes are used to split databases, and higher 4 bits are used to split tables.\n\n**- Data Migration between Old and New Databases**\nThe migration must be online, and downtime migration cannot be accepted, as there will be new data writes during the migration process.\n\nThe data should be intact, and the migration process should be insensible to the client-side. After the migration, data in the new database should be consistent with the ones in the old databases.\n\nThe migration should allow rollback, so that when a problem occurs during the migration process, it should be able to roll back to the source database without impacting system availability.\n\nData migration steps are as follows: dual writes-> migrate historical data-> verify-> old database offline.\n\n## Effects and Benefits\n\n- It solves the problem of single database capacity limit.\n- The data volume of a single database and table is greatly reduced after sharding. The data volume of a single table is reduced from nearly a hundred million level to several millions level, which greatly improves the overall performance.\n- It reduces the risk of data losses due to oversized single databases and tables in extreme cases and eases the pressure of operation and maintenance.\nThe following is a comparison of the number of interface calls of the order placement service and the time consumed by the interface during two promotion and discount periods:\n\n**Promotion before adopting ShardingSphere**\n![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l8d6f5tchgq7uipwbip0.png)\n \n\n**Promotion after adopting ShardingSphere**\n![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3s9mtqhr0kpkecyz94kl.png)\n \n\n## Summary\nShardingSphere simplifies the development of data sharding with its well-designed architecture, highly flexible, pluggable and scalable capabilities, allowing R&D teams to focus only on the business itself, thus enabling flexible scaling of the data architecture.\n\n**Apache ShardingSphere Project Links:**\n\n[ShardingSphere Github](https://github.com/apache/shardingsphere/issues?page=1&q=is%3Aopen+is%3Aissue+label%3A%22project%3A+OpenForce+2022%22)\n\n[ShardingSphere Twitter](https://twitter.com/ShardingSphere)\n\n[ShardingSphere Slack](https://join.slack.com/t/apacheshardingsphere/shared_invite/zt-sbdde7ie-SjDqo9~I4rYcR18bq0SYTg)\n\n[Contributor Guide](https://shardingsphere.apache.org/community/cn/contribute/)","date":"2022-04-29","author":"SphereEx","excerpt":"Sales promotions drive traffic, and put great pressure on databases. Here's how \n ShardingSphere solves an e-commerce platform's data challenges created by sales promotion days.","metaDescription":"Zhuanzhuan is an online marketplace on which users can buy and sell secondhand goods.","createdAt":"2022-09-26T07:06:47.526Z","updatedAt":"2022-09-27T07:00:05.205Z","publishedAt":"2022-09-27T07:00:05.202Z","locale":"en","caseStudiesType":{"data":{"id":9,"attributes":{"name":"Internet","createdAt":"2022-09-26T10:34:03.086Z","updatedAt":"2022-09-27T06:59:34.838Z","publishedAt":"2022-09-27T06:59:34.836Z","locale":"en"}}},"cover":{"data":{"id":339,"attributes":{"name":"zhuanzhuan.png","alternativeText":"zhuanzhuan.png","caption":"zhuanzhuan.png","width":720,"height":534,"formats":{"thumbnail":{"name":"thumbnail_zhuanzhuan.png","hash":"thumbnail_zhuanzhuan_a2afd90ff3","ext":".png","mime":"image/png","width":210,"height":156,"size":11,"path":null,"url":"https://official-media.sphere-ex.com/thumbnail_zhuanzhuan_a2afd90ff3.png"},"small":{"name":"small_zhuanzhuan.png","hash":"small_zhuanzhuan_a2afd90ff3","ext":".png","mime":"image/png","width":500,"height":371,"size":36.3,"path":null,"url":"https://official-media.sphere-ex.com/small_zhuanzhuan_a2afd90ff3.png"}},"hash":"zhuanzhuan_a2afd90ff3","ext":".png","mime":"image/png","size":47.13,"url":"https://official-media.sphere-ex.com/zhuanzhuan_a2afd90ff3.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-09-26T07:07:25.422Z","updatedAt":"2022-09-26T07:07:25.422Z"}}},"metaLogo":{"data":{"id":354,"attributes":{"name":"zhuanzhuan logo.png","alternativeText":"zhuanzhuan logo.png","caption":"zhuanzhuan logo.png","width":130,"height":130,"formats":null,"hash":"zhuanzhuan_logo_fd59d35610","ext":".png","mime":"image/png","size":1.85,"url":"https://official-media.sphere-ex.com/zhuanzhuan_logo_fd59d35610.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-09-26T08:45:59.332Z","updatedAt":"2022-09-26T08:45:59.332Z"}}},"localizations":{"data":[]}}}}},
    "staticQueryHashes": []}