{
    "componentChunkName": "component---src-templates-case-study-detail-ts",
    "path": "/case-studies/7",
    "result": {"pageContext":{"next":{"id":26,"attributes":{"feature":false,"title":"openGauss & ShardingSphere: A Top Distribution Solutions","content":"##About openGauss\n\nopenGauss is an open source relational database management system. Since its launch in June 2020, it has made waves the database management vertical. With an increasing number of partners and developers, it’s already reaping the benefits of the significant efforts & resources poured into building a successful database ecosystem.\n\n##Project Background\n\nFacing problems like data volume explosion, and super-high levels of concurrent data processing, openGauss made a choice: to adopt a distributed solution. It started to work on big data storage, throughput improvement for super-high concurrency, improving performance bottlenecks caused by large data amounts in a table, etc.\n\nApache ShardingSphere facilitates the distributed feature development of openGauss. Such collaboration on the full stack and open source solution helps openGauss make significant breakthroughs.\n\n##The Distributed Solution\n\nopenGauss integrates many open source components to optimize its full stack & open source distributed solution for horizontal scaling, distributed transactions, and distributed governance. Its framework is shown in Fig.1\n\n![Figure 1: The Structure of the ShardingSphere X openGauss Distribution Solution](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s9knu2pwkmpsmvx6iwm2.png)\n\nShardingSphere-Proxy is the open source distributed database solution. \nShardingSphere’s well-known features include but are not limited to data sharding, distributed transaction, elastic scaling, and read-write splitting.\n\nHAProxy uses Patroni REST API to identify database master nodes and guarantee high availability, as well as load balancer.\n\nEvery high availability node of Patroni supports one master with multiple slaves and uses the Praxos protocols to keep data consistency, so that nodes are allowed to be deployed in the same site or in different sites to ensure multiple sites and centers data security.\n\nIn terms of the solution, ShardingSphere-Proxy contributes with its mighty distribution features, Kubernetes helps manage clusters, and prometheus monitors cluster status. This all-star team’s integration make the project a really powerful full stack, open source distribution solution.\n\n##What’s Special About the Project?\n\n + _Great extensibility and elastic scaling_\n\nHorizontal scaling helps realize the linear extension of computing and storage capabilities. The maximum shard count increases to 6400. Database performance improves accordingly and the problem of data volume explosion in single database is effectively managed.\n\nBased on users’ traffic condition, a user can flexibly scale data nodes, and adopt intelligent read-write splitting to realize automatic load balancing of a distributed database.\n\n+ _Enhanced enterprise-grade features_\n\nThe project provides users with various enterprise-grade features such as distributed data store, Trigger, distributed governance, comprehensive end-to-end encryption, and Workload Diagnosis Report (WDR)\n\n+ _Simple deployment_\n\nIt may take only a few seconds to deploy the project. Its deployment is simple while also being efficient as it adopts standardized image deployment to keep delivery consistent in different environments. It also uses container deployment as well as resource pooling so that it is less platform dependent.\n\n+ _Super-high availability and remote disaster recovery_\n\nThe project can effectively manage clusters and facilitate operations & maintenance. It follows the Paxos protocols to ensure data security and strong consistency and provides various features for disaster recovery, whose recovery point object (RPO) is zero. It supports resilient recovery methods for one region, cross-region, and multi-region & multi-center.\n\n+ _Open source ecosystem_\n\nThe open source project openGauss warmly welcomes more contributors, and encourages joint efforts to grow the open source database ecosystem.\n\n##Application Scenarios\n\n+ Financial system\n\nThe product is extremely reliable even for demanding financial systems. Paxos is a family of protocols for solving consensus in a network of unreliable and fallible processors. Besides being Paxos-based, the product includes powerful features such as distributed transaction with strong consistency, and multi-region multi-center disaster recovery whose RPO is 0.\n\n+ Government and enterprise operations\n\nIt’s really safe and convenient for government and enterprise operations due to its comprehensive end-to-end security protection, its hardware and software ecosystem support, and its high availability capabilities.\n\n+ Intelligent grid\n\nThe product’s flexible disaster recovery features can improve smart grid database scalability.\n\n##ShardingSphere Community:\n\n[ShardingSphere Github](https://github.com/apache/shardingsphere)\n\n[ShardingSphere Twitter] \n(https://twitter.com/ShardingSphere)\n\n[ShardingSphere Slack Channel](apacheshardingsphere.slack.com)\n\n\n\n\n\n","date":"2021-09-28","author":"SphereEx","excerpt":null,"metaDescription":"openGauss is an open source relational database management system that is released with the Mulan PSL v2. with the kernel built on Huawei’s years of experience in the database field and continuously provides competitive features tailored to enterprise-grade scenarios. In addition, openGauss is an open source database platform that encourages community contribution and collaboration.","createdAt":"2022-09-26T08:09:50.554Z","updatedAt":"2022-09-27T07:00:49.291Z","publishedAt":"2022-09-27T07:00:36.447Z","locale":"en","caseStudiesType":{"data":{"id":10,"attributes":{"name":"Software Development","createdAt":"2022-09-26T10:34:53.523Z","updatedAt":"2022-09-27T06:59:43.907Z","publishedAt":"2022-09-27T06:59:43.905Z","locale":"en"}}},"cover":{"data":{"id":346,"attributes":{"name":"openGauss1.png","alternativeText":"openGauss1.png","caption":"openGauss1.png","width":720,"height":411,"formats":{"thumbnail":{"name":"thumbnail_openGauss1.png","hash":"thumbnail_open_Gauss1_0781ec1284","ext":".png","mime":"image/png","width":245,"height":140,"size":31.7,"path":null,"url":"https://official-media.sphere-ex.com/thumbnail_open_Gauss1_0781ec1284.png"},"small":{"name":"small_openGauss1.png","hash":"small_open_Gauss1_0781ec1284","ext":".png","mime":"image/png","width":500,"height":285,"size":97.34,"path":null,"url":"https://official-media.sphere-ex.com/small_open_Gauss1_0781ec1284.png"}},"hash":"open_Gauss1_0781ec1284","ext":".png","mime":"image/png","size":124.92,"url":"https://official-media.sphere-ex.com/open_Gauss1_0781ec1284.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-09-26T08:07:47.594Z","updatedAt":"2022-09-26T08:07:47.594Z"}}},"metaLogo":{"data":{"id":361,"attributes":{"name":"openGauss logo.png","alternativeText":"openGauss logo.png","caption":"openGauss logo.png","width":275,"height":183,"formats":{"thumbnail":{"name":"thumbnail_openGauss logo.png","hash":"thumbnail_open_Gauss_logo_7b11da9186","ext":".png","mime":"image/png","width":234,"height":156,"size":11.4,"path":null,"url":"https://official-media.sphere-ex.com/thumbnail_open_Gauss_logo_7b11da9186.png"}},"hash":"open_Gauss_logo_7b11da9186","ext":".png","mime":"image/png","size":3.07,"url":"https://official-media.sphere-ex.com/open_Gauss_logo_7b11da9186.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-09-26T09:11:52.021Z","updatedAt":"2022-09-26T09:11:52.021Z"}}},"localizations":{"data":[]}}},"prev":{"id":6,"attributes":{"feature":true,"title":"Apache ShardingSphere & JD Baitiao: Story of an Implementation Journey","content":">Apache ShardingSphere has become the best solution for JD Baitiao’s very large storage scenarios and scale-up. JD’s shopping festival weighed very much on our big data storage. I mean, like hundreds of millions of data sets.\n>\n>JD Baitiao, an internet credit products provider launched in 2014, has always been challenged by data explosion. Every shopping festival is like a technical test for us, but every time the technical staff professionals take proactive strategic initiatives in data architecture development to ensure everything goes smoothly.\n>\n>— Dongfang Zhang，JD Batiao R&D Director\n\n## The Baitiao Data Architecture Evolution\n\nLaunched in Feb. 2014, JD Baitiao’s data architecture has already evolved for several times because of rapid increase in data volume and rapid business growth.\n\n### From 2014 to 2015\n\nBaitiao first adopted the Solr + HBase solution to connect core business systems and other systems with key databases. Solr indexing is able to achieve fast search responses and HBase is used to store a massive amount of data.\n\n- A Solr cluster can help the core database reduce reads and writes stress.\n- However, Solr is not perfect: it has unsatisfying extensibility and intrusions.\n\n### From 2015 to 2016\n\nThe company chose NoSQL to store monthly data tables in Mongo DB clusters, temporarily meeting the demand for importing/exporting massive data in payment settlement scenarios.\n\n- The change made it quite efficient to query hot data. In addition, thanks to such unstructured data storage, staff can easily modify table structures.\n- Nevertheless, the plan was still problematic: bad extensibility, intrusions, and memory hog.\n\n### From 2016 to 2017\n\nGrowing business and growing data. Tens of billions of data volume imposed more performance and volume pressure on MongoDB. JD Baitiao’s Big Data Platform used DBRep and MySQL Slave to capture data changes and store the information data in its message center, to later be written to ES and HBase.\n\n- This plan was actually much better. It focused on real-time data and improved extensibility.\n- However, Baitiao faced high costs to maintain its code because of the data sharding problem in its architecture.\n\nThe architecture evolution of JD Baitiao is just an illustration of rapidly growing Internet consumer finance. All the methods Baitiao adopted in the past were not comprehensive, indicating that every solution would be outdated soon.\n\n## Time to Adopt a Decoupled Architecture\n\nTo ensure good system performance even with rapidly increasing data volumes, the technical team first chose the sharding architecture pattern that not only guaranteed excellent performance, but also kept code under control.\n\n### The data striping plan was actually based on application architecture:\n\nThe product continuously developed but the early solution had now become one of the biggest problems. The old data sharding plan makes code even more complex and increases maintenance expenditure. Developers have to spend much time adjusting sharding every time when the application upgrades, so they cannot concentrate on their own developments. In fact, tight coupling is to blame for the problem.\n\nTherefore, the team decided to use a mature sharding component that simplifies system upgrades and architecture changes. To compare Baitiao’s shard and ShardingSphere shard, their differences are shown in the following table.\n\n![jd_en_1.png](https://official-media.sphere-ex.com/jd_en_1_eb365b4a93.png)\n\nFor Baitiao, decoupling was the next task.\n\nApparently, JD Baitiao‘s data architecture will experience a brand-new journey to decoupling. The following three directions definitely fueled its transformation:\n\n**Concentration:** instead of database sharding built in its architecture, it applied a sharding component to place more energy on it own product development.\n\n**Easier Upgrade:** it used a decoupled architecture to simplify the R&D process of system upgrades.\n\n**Future-Oriented Plan:** it aims to improve system extensibility so that Baitiao will be more capable of confidently holding large online shopping festivals like the 618 Shopping Festival and Double 11 Shopping Festival (also known as Singles Day on November 11th).\n\nJD Baitiao is a huge business, so its business scenario is truly related to finance, high concurrency, and massive data volumes. Baitiao’s sharding component must have the following features:\n\n1. Be a mature product\n2. Have excellent performance\n3. Be able to handle big data\n4. Have an extensible architecture\n\n## The Apache ShardingSphere Solution\n\nThe lightweight Java framework ShardingSphere-JDBC is Apache ShardingSpheres’s first product providing services such as a Java Database Connectivity (JDBC) API. ShardingSphere JDBC uses jar archive file installation package and allows the client-side to directly connect to databases. So, it requires no extra deployment dependencies. It is like an enhanced JDBC driver fully compatible with JDBC and ORM frameworks.\n\nShardingSphere — JDBC provides the following features, making it the best solution in the Baitiao scenario.\n\nMature Product: ShardingSphere — JDBC is a mature product developed for years, and its open source community is very active.\n\nExcellent Performance: its micro-kernel and lightweight design hardly hinder performance.\n\nMinimum Changes: it supports native MySQL protocol and minimizes R&D workload.\n\nExtensibility: users can combine with the migration sync component to easily extend data.\n\n![jd_en_2.png](https://official-media.sphere-ex.com/jd_en_2_c92fb9e5ef.png)\n\nAfter being systematically tested for several times, Apache ShardingSphere became JD Baitiao’s first choice of data sharding middleware. The cooperation started at the end of 2018.\n\n## Product Adaptation\n\nTo provide Baitiao with better services and support the business, Apache ShardingSphere has made many improvements to its product features and performance during the implementation process. At the same time, the user case helps optimize the product in return.\n\n### Upgrade the SQL Engine\n\nBaitiao’s business logic is extremely complicated; its diversified scenario demands require the best SQL compatibility levels. Thus, Apache ShardingSphere restructures the SQL parser module to support more SQLs.\n\n- Single data node routing: 100% SQL compatibility\n- Multi data node routing：comprehensively support DML, DDL, DCL, TCL and partly DAL. Support features such as pagination, deduplication, ranking, grouping, aggregation and correlated query.\n\n### Distributed Key\n\nApache ShardingSphere provides built-in distributed key generators, such as UUID, and SNOWFLAKE. It also provides its distributed key generation API so that users can develop custom key generation algorithms to meet their special needs.\n\n### Shard Key Value Injection\n\nIf a SQL has no shard condition, Apache ShardingSphere uses ThreadLoad to manage shard key value, and users can program and add a shard condition to HintManager, making the condition only effective in the current thread. That’s so-called SQL zero intrusion.\n\nAdditionally, Apache ShardingSphere continues to optimize its other features in order to meet Baitiao’s need for high performance such as：\n\n- SQL parsing result cache\n- JDBC metadata cache\n- Bind table & Broadcast table\n- Automated execution engine & Stream merge\n\nJD Baitiao & ShardingSphere teams joint efforts amde all indicators of the product live up to their expectation. The resulting final performance is almost the same as that of a native JDBC.\n\n![jd_en_3.png](https://official-media.sphere-ex.com/jd_en_3_582e2afe89.png)\n\n## Cutover\n\nApache ShardingSphere uses its custom HASH strategy to shard data, effectively avoiding the hot data problem. The total of data nodes almost reaches ten thousand. The cutover process lasts for about 4 weeks.\n\n1. After DBRep reads data, Apache ShardingSphere synchronizes the data to the target database cluster.\n2. Two clusters run together. After a data migration, Baitiao uses its own tool to verify business and data.\n\nDBRep is fundamental to the product design of ShardingSphere-Scaling. The automation feature of Scaling definitely facilitates migration and scale-up.\n\n## Apache ShardingSphere Brings Many Benefits\n\n### Simplified Upgrade Path\n\nThe decoupled architecture effectively simplifies a technology stack required in a system upgrade, so the developer team no longer need to worry about table sharding. Instead, they can focus on the business itself. In brief, ShardingSphere greatly helps Baitiao optimize the upgrade path.\n\n### Save R&D Cost\n\nBaitiao directly uses the mature product Apache ShardingSphere, so they do not need to reinvent the wheel and save a lot of time and energy.\n\n### Improve Architecture Extensibility\n\nBaitiao also uses the sync migration component Scaling that makes system scale-up even more flexible. It’s the company’s secret for online shopping festival successes.\n\n## Summary\n\nThe growth of JD Baitiao’s business greatly stimulates its data architecture upgrades. This time, it chose Apache ShardingSphere to decouple its architecture and make future upgrades less complicated. Thank to this from now on, developers only have to concentrate on their own product optimization because its data architecture is extensible enough. The case is really a good example of ShardingSphere application in the consumer finance scenario.\n\nNow, there are more and more internet credit payment models. In the future, Apache ShardingSphere in collaboration with JD Technology will explore more business scenarios, and contribute finance-related technological innovations, further improving Internet finance.\n\n### About Apache ShardingSphere\n\nShardingSphere is one of the Apache Software Foundation Top-Level Open Source projects, used by over 170 enterprises worldwide, across various verticals such as finance, e-commerce, cloud services, tourism, logistics, education, and entertainment. Its GitHub community has so far cumulated over 14,000 stars.\n\nWe welcome more technical professionals to write articles and share their experiences and thoughts. If you are interested, feel free to contact us:\n\nShardingSphere Github: https://github.com/apache/shardingsphere\n\nShardingSphere Twitter: https://twitter.com/ShardingSphere\n\nShardingSphere Slack Channel: apacheshardingsphere.slack.com","date":"2021-12-01","author":"SphereEx","excerpt":"Apache ShardingSphere has become the best solution for JD Baitiao’s very large storage scenarios and scale-up. JD’s shopping festival weighed very much on our big data storage. I mean, like hundreds of millions of data sets.","metaDescription":"JD Technology Group is a business subgroup of JD Group that focuses on technology as an industry service, and is committed to providing technical products and solutions for enterprises, financial institutions, governments and other types of customers across the value chain.","createdAt":"2021-12-12T12:30:44.957Z","updatedAt":"2022-09-26T09:16:15.245Z","publishedAt":"2021-12-12T12:35:08.683Z","locale":"en","caseStudiesType":{"data":{"id":2,"attributes":{"name":"E-commerce","createdAt":"2021-12-11T23:02:08.395Z","updatedAt":"2021-12-11T23:02:11.020Z","publishedAt":"2021-12-11T23:02:10.947Z","locale":"en"}}},"cover":{"data":{"id":345,"attributes":{"name":"JDT.png","alternativeText":"JDT.png","caption":"JDT.png","width":720,"height":540,"formats":{"thumbnail":{"name":"thumbnail_JDT.png","hash":"thumbnail_JDT_2704eac855","ext":".png","mime":"image/png","width":208,"height":156,"size":8.85,"path":null,"url":"https://official-media.sphere-ex.com/thumbnail_JDT_2704eac855.png"},"small":{"name":"small_JDT.png","hash":"small_JDT_2704eac855","ext":".png","mime":"image/png","width":500,"height":375,"size":28.29,"path":null,"url":"https://official-media.sphere-ex.com/small_JDT_2704eac855.png"}},"hash":"JDT_2704eac855","ext":".png","mime":"image/png","size":14.08,"url":"https://official-media.sphere-ex.com/JDT_2704eac855.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-09-26T08:02:50.084Z","updatedAt":"2022-09-26T08:02:50.084Z"}}},"metaLogo":{"data":{"id":10,"attributes":{"name":"jd_logo.png","alternativeText":"jd_logo.png","caption":"jd_logo.png","width":300,"height":44,"formats":{"thumbnail":{"name":"thumbnail_jd_logo.png","hash":"thumbnail_jd_logo_591b2e5e14","ext":".png","mime":"image/png","width":245,"height":36,"size":6.43,"path":null,"url":"https://official-media.sphere-ex.com/thumbnail_jd_logo_591b2e5e14.png"}},"hash":"jd_logo_591b2e5e14","ext":".png","mime":"image/png","size":2.36,"url":"https://official-media.sphere-ex.com/jd_logo_591b2e5e14.png","previewUrl":null,"provider":"dine-strapi-provider-upload-aws-s3","provider_metadata":null,"createdAt":"2021-12-12T12:04:06.233Z","updatedAt":"2021-12-14T07:19:09.532Z"}}},"localizations":{"data":[{"id":1,"attributes":{"feature":true,"title":"Apache ShardingSphere 在京东白条场景的落地之旅","content":"\n>京东白条使用 Apache ShardingSphere 解决了千亿数据存储和扩容的问题，为大促活动奠定了基础。\n>\n>2014 年初，“京东白条”作为业内互联网信用支付产品，数据量爆发式的增长，每一次大促备战都是对技术人员的考验，每一次的战略转型驱动着数据架构的成长。\n>\n>— 张栋芳，京东白条研发负责人\n\n### 项目背景\n\n自 2014 年 2 月京东白条业务上线起，为满足快速发展的业务和激增的海量数据，白条的数据架构经历了数次演进。\n\n白条数据架构的演进间接地反应了互联网消费金融的飞速发展，也说明了每一种解决方案在不同背景下都有不同的保质期。\n\n### 业务挑战\n\n随着产品升级迭代，早期的解决方案演变成为了眼前的问题，通过业务框架实现的数据分片方案导致业务代码复杂度增加、维护成本不断攀升，紧耦合的弊端原形毕露，应用每次升级都需要投入较多的精力对分片做相应调整，研发人员难以专注于业务本身。\n\n技术团队经权衡后开始考虑使用成熟的分库分表组件来承担这部分工作，让业务系统升级和架构调整不再复杂。\n\n### 解决方案\n\nShardingSphere-JDBC 是 Apache ShardingSphere 的第一款产品，它定位为轻量级 Java 框架，在 Java 的 JDBC 层提供的额外服务。它使用客户端直连数据库，以 jar 包形式提供服务，无需额外部署和依赖，可理解为增强版的 JDBC 驱动，完全兼容 JDBC 和各种 ORM 框架。\n\nShardingSphere-JDBC 的以下特点能够很好地满足白条业务场景：\n\n- 产品成熟：经数年打磨产品成熟度高，且社区活跃；\n- 性能良好：微内核、轻量化的设计，性能损耗极小；\n- 改造量小：支持原生的 MySQL 协议，研发工作量小；\n- 扩展灵活：搭配使用迁移同步组件轻松实现数据扩展。\n\n经内部大量系统性验证之后，Apache ShardingSphere 成为了京东白条数据分片中间件的首选方案，2018 年底正式开始对接。\n\n![jd_zh_1.png](https://official-media.sphere-ex.com/jd_zh_1_fac28cd6ec.png)\n\n### 用户收益\n\n- 简化升级路径\n\n通过架构解耦，业务系统升级所涉及技术栈得到有效缩短，研发团队不再需要关注分表设计，精力全部聚焦于业务本身，升级路径得到大幅度优化；\n\n- 节省研发力量\n\n引入成熟的 Apache ShardingSphere 无需重新开发分表组件，在简化业务升级路径的基础上节省了大量研发力量；\n\n- 架构灵活扩展\n\n搭配使用 Scaling 同步迁移组件从容面对“618”和“11.11”等大型活动，系统灵活扩容。\n\n**SphereEx，让数据与应用连接更简单。**\n\n","date":"2021-12-01","author":"SphereEx","excerpt":"京东白条使用 Apache ShardingSphere 解决了千亿数据存储和扩容的问题，为大促活动奠定了基础。","metaDescription":"京东科技集团是京东集团旗下专注于以技术为产业服务的业务子集团，致力于为企业、金融机构、政府等各类客户提供全价值链的技术性产品与解决方案。","createdAt":"2021-12-11T23:10:03.172Z","updatedAt":"2021-12-14T07:22:15.304Z","publishedAt":"2021-12-11T23:10:07.307Z","locale":"zh"}}]}}},"article":{"id":7,"attributes":{"feature":false,"title":"eHualu Story’s Data Processing Shortcut","content":"> ShardingSphere has greatly simplified sharding development and maintenance, which has played a very important role in our rapid product release. We estimate that ShardingSphere has saved us at least 4 months of R&D costs.\n> \n> –– Shi Moxuan, Technical Director，E-Hualu\n\nAt the beginning of this year, E-hualu officially launched its cloud service product “Hulu App” that targets non-business users. Since then, its architecture’s data stress has increased on a daily basis.\n\nThe R&D team of Hulu App chose to use ShardingSphere’s sharding features and scale their data out. The flexible and agile features of ShardingSphere met the requirements of Hulu App. It requires data extensibility, and needs to prevent the team from “reinventing the wheel”.\n\nShardingSphere can simplify sharding development and maintenance to the greatest extent. As the service is growing, the work will be increasingly complicated.\n\n### Hulu App’s growth pressure: feature extension and new services\n\nThe high requirement for data storage and computing is in the genes of the Hulu App technical team.\n\nBecause Hulu App is rapidly growing, its service and feature changes are relatively frequent. Therefore its technical team needs to adapt quickly when a front-end service changes. As its users and service data volume are growing rapidly, its databases are faced with greater pressure.\n\nThe official Hulu App was released on May 17, 2020, and then its user data and service volume also showed a rapid growth trend. Therefore it was inevitable to scale out its databases for many times. At the same time, with business requirements changing very quickly, new challenges have emerged:\n\n- How to ensure feature extensibility\n\nBecause of increasing users and developing product versions, its user data has exploded. The storage and computing capability of its old architecture has been severely challenged. Therefore, Hulu App requires that its back-end data processing platform shall have both extensible features, and flexibility.\n\n- How to improve efficiency\n\nFaced with its changeable services, the R&D team of Hulu App needs to make rapid adjustments when necessary, so as to make its architecture adapt to its services. A highly-flexible, extensible data architecture can greatly improve development efficiency of the team. On the other hand, because of such large amounts of data, database retrieval efficiency will inevitably have problems such as delays, slow read and write. These problems will ultimately affect user experience.\n\n- How to facilitate feature release\n\nAll new products will face problems like frequent new feature release, bringing the scheduled release date forward, when they are just released. These problems pose a great challenge to a development team. At first, Hulu’s development team planned to base on their service demands and create their own sharding plan all by themselves. However, they don’t have enough time to do that, so it’s urgent for them to choose a new sharding plan.\n\n- How to guarantee system stability\n\nA newly-added technology will greatly challenge system stability. It’s especially true for bottom-level technologies, most of which can influence platform services and probably challenge system stability. So what Hulu App needs is a data application product that does not greatly stress databases, but quickly adapts to them and that is also highly stable.\n\n### Leverage ShardingSphere’s features to Create a Flexible, Highly-Available Data Architecture Solution\n\nHulu App’s team studied their needs and spent two weeks evaluating ShardingSphere and other similar solutions. They had evaluative indicators like product feature, maturity, stability and performance. At last, they chose ShardingSphere because of its powerful features, its support and its high maturity can fully meet their needs.\n\nThe Hulu team deployed ShardingSphere above Alibaba Cloud’s RDS. Instead of adjusting ontology databases, the Hulu team prefers to conduct data governance above its databases. ShardingSphere can effectively improve Hulu’s data architecture, because it can achieve three things: high scalability brought by its extensible architecture, closely adaptive to Hulu’s service. ShardingSphere brings Hulu with obvious improvement:\n\n#### “High scalability” of the pluggable architecture\n\nDue to its service characteristics, Hulu App’s limited storage space is consumed very quickly, and the problem gradually begins to affect response efficiency at the front end. Therefore, the team can quickly develop enhanced features by using ShardingSphere, which can provide optimized solutions for its subsequent architectural adjustments. The example strengthens the ShardingSphere’s advantage in sharding.\n\n#### Closely adaptive to service\n\nThe smaller the architecture change is, the more controllable it is for developers. ShardingSphere, an ecosystem located above databases, is closer to service and it deployment is even more lightweight. So it is undoubtedly the best solution to solve the contradiction between the front-end service changes and back-end architecture adjustment of Hulu App. In addition, in terms of flexibility, ShardingSphere supports instant achievement of relevant configurations, which greatly helps the team save a lot of sharding-related work. So it plays a very important role in Hulu App’s quick releases.\n\n#### “Zero attack” to the service architecture\n\nHulu App chose ShardingSphere’s Proxy deployment mode. It can manage the real database cluster through Proxy without. replacing underlying databases. It basically completed the separation of service and data at the architectural level without any changes. So it can avoid the risks caused by database changes, such as unavailable service and long stable cycle. In addition, the non-status mode of ShardingSphere hardly has any perceptible impact on users at the front end and the service layer does not need to pay attention to the data storage method .\n\nSo, Hulu App is a product with tight go-live schedule and fast feature iteration. ShardingSphere-Proxy can reuse original databases and help the Hulu development team to develop incremental features above databases such as sharding and data encryption. In addition, ShardingSphere allows developers not to worry about database configuration and shields user perception, so it quickly builds service-oriented database direct connection, which is well separated from the system architecture level, and ensures that daily maintenance work at the database agent, such as bug repair and version update, will not affect the services.\n\nThis time, ShardingSphere collaborated with the R&D team of E-hualu’s Hulu App, and helped them overcome data volume explosion for several times. Actually, this is just a tiny example of ShardingSphere’s various application scenarios around the world.\n\n### About E-hualu & Hulu App\n\nFounded in 2001, Beijing E-hualu Information Technology Co., Ltd. is a listed company under the state-owned enterprise China Hualu Group Co., LTD. directly administrated by the State-owned Assets Supervision and Administration Commission of the State Council (SASAC). Its mission is to reduce energy consumption and the cost of long-term data management.\n\nHulu App is the first to-customer product of E-hualu.\n\n### ShardingSphere Community:\n\nShardingSphere Github: https://github.com/apache/shardingsphere\n\nShardingSphere Twitter: https://twitter.com/ShardingSphere\n\nShardingSphere Slack Channel: apacheshardingsphere.slack.com","date":"2021-11-29","author":"SphereEx","excerpt":"ShardingSphere has greatly simplified sharding development and maintenance, which has played a very important role in our rapid product release. We estimate that ShardingSphere has saved us at least 4 months of R&D costs.","metaDescription":"Founded in 2001, Beijing E-hualu Information Technology Co., Ltd. is a listed company under the state-owned enterprise China Hualu Group Co., LTD. directly administrated by the State-owned Assets Supervision and Administration Commission of the State Council (SASAC). Its mission is to reduce energy consumption and the cost of long-term data management.","createdAt":"2021-12-12T12:33:14.398Z","updatedAt":"2022-09-26T10:27:50.013Z","publishedAt":"2021-12-12T12:35:01.820Z","locale":"en","caseStudiesType":{"data":{"id":1,"attributes":{"name":"Finance","createdAt":"2021-12-11T23:01:56.177Z","updatedAt":"2021-12-11T23:01:58.635Z","publishedAt":"2021-12-11T23:01:58.567Z","locale":"en"}}},"cover":{"data":{"id":7,"attributes":{"name":"ehualu.png","alternativeText":"ehualu.png","caption":"ehualu.png","width":1030,"height":720,"formats":{"thumbnail":{"name":"thumbnail_ehualu.png","hash":"thumbnail_ehualu_6d29db12b5","ext":".png","mime":"image/png","width":223,"height":156,"size":6.16,"path":null,"url":"https://official-media.sphere-ex.com/thumbnail_ehualu_6d29db12b5.png"},"large":{"name":"large_ehualu.png","hash":"large_ehualu_6d29db12b5","ext":".png","mime":"image/png","width":1000,"height":699,"size":53.63,"path":null,"url":"https://official-media.sphere-ex.com/large_ehualu_6d29db12b5.png"},"medium":{"name":"medium_ehualu.png","hash":"medium_ehualu_6d29db12b5","ext":".png","mime":"image/png","width":750,"height":524,"size":33.7,"path":null,"url":"https://official-media.sphere-ex.com/medium_ehualu_6d29db12b5.png"},"small":{"name":"small_ehualu.png","hash":"small_ehualu_6d29db12b5","ext":".png","mime":"image/png","width":500,"height":350,"size":18.22,"path":null,"url":"https://official-media.sphere-ex.com/small_ehualu_6d29db12b5.png"}},"hash":"ehualu_6d29db12b5","ext":".png","mime":"image/png","size":24.54,"url":"https://official-media.sphere-ex.com/ehualu_6d29db12b5.png","previewUrl":null,"provider":"dine-strapi-provider-upload-aws-s3","provider_metadata":null,"createdAt":"2021-12-12T12:04:05.768Z","updatedAt":"2021-12-14T07:19:57.408Z"}}},"metaLogo":{"data":{"id":3,"attributes":{"name":"ehualu_logo.png","alternativeText":"ehualu_logo.png","caption":"ehualu_logo.png","width":303,"height":153,"formats":{"thumbnail":{"name":"thumbnail_ehualu_logo.png","hash":"thumbnail_ehualu_logo_26122a7add","ext":".png","mime":"image/png","width":245,"height":124,"size":22.6,"path":null,"url":"https://official-media.sphere-ex.com/thumbnail_ehualu_logo_26122a7add.png"}},"hash":"ehualu_logo_26122a7add","ext":".png","mime":"image/png","size":22.6,"url":"https://official-media.sphere-ex.com/ehualu_logo_26122a7add.png","previewUrl":null,"provider":"dine-strapi-provider-upload-aws-s3","provider_metadata":null,"createdAt":"2021-12-12T12:04:05.036Z","updatedAt":"2021-12-14T07:19:30.734Z"}}},"localizations":{"data":[{"id":2,"attributes":{"feature":false,"title":"葫芦 App 后台数据处理的逻辑捷径","content":"> ShardingSphere 大大简化了分库分表的开发和维护工作，对于业务的快速上线起到了非常大的支撑作用，保守估计 ShardingSphere 至少为我们节省了 4 个月的研发成本。\n> \n> –– 史墨轩，易华录·技术总监\n\n### 项目背景\n\n伴随着易华录旗下面向个人用户的云服务产品【葫芦 App】正式上线，后台架构所承受的业务压力也与日俱增。\n\n为此，葫芦 App 研发团队选择采用 ShardingSphere 分库分表的功能对数据进行了横向的拆分，围绕 ShardingSphere 灵活敏捷的特性，满足了葫芦 App 业务对数据层扩展性的要求，避免团队重复“造轮子”，最大程度简化了随着业务增长而带来的愈发复杂化的分库分表的开发与维护工作。\n\n### 业务挑战\n\n- 扩展能力\n- 效率提升\n- 上线提前\n- 成熟技术\n\n### 解决方案\n\n- 可插拔架构的『高扩展性』\n\n由于业务特性，葫芦 App 原本有限的存储空间被消耗得非常快，并逐渐开始影响用户在前端的响应效率。通过采用 ShardingSphere 的分片策略，葫芦研发团队在应对海量计算+存储所带来的业务问题同时，能够确保分片扩展策略的灵活性。基于此，葫芦团队可以在ShardingSphere 上快速做出相应的功能扩展，为后续架构调整提供优化方案，进一步强化突出了 ShardingSphere 分库分表的优势。\n\n- 距离业务更紧密的『贴合性』\n\n后台架构的变化越小，对于业务而言就越可控。ShardingSphere 这种位于数据库之上的生态，距离业务更近，部署起来也更加轻量，无疑是解决葫芦 App 前台业务变更与后台架构调整之间矛盾的最优解。此外在灵活性层面，ShardingSphere 通过相关配置即可实现的方式，极大简化了葫芦研发团队在分库分表层面的研发与维护工作，对于业务的快速上线起到了非常大的支撑作用。\n\n- 对于业务架构的『零侵入性』\n\n葫芦 App 选择了 ShardingSphere-Proxy 部署模式，在不更换底层数据库的前提下通过 Proxy 来管理真实的数据库集群，基本无需对业务进行改造就已经完成了业务与数据在架构层面的分离，并避免了因更换数据库导致的业务不可用、漫长稳定周期等风险。另外，ShardingSphere 的无状态模式，几乎不会对前端用户产生任何可感知的影响，业务层也无需关注数据的存储方式。\n\n![ehualu_zh_1.png](https://official-media.sphere-ex.com/ehualu_zh_1_3fad8e2ccd.png)\n\n### 用户收益\n\n- 节省 4 人月成本\n- 明显改善业务体验\n- 系统灵活扩容\n\n**SphereEx，让数据与应用连接更简单。**","date":"2021-11-29","author":"SphereEx","excerpt":"ShardingSphere 大大简化了分库分表的开发和维护工作，对于业务的快速上线起到了非常大的支撑作用，保守估计 ShardingSphere 至少为我们节省了 4 个月的研发成本。","metaDescription":"北京易华录信息技术股份有限公司，成立于 2001 年，是国务院国资委直接监管的中央企业中国华录集团旗下控股的上市公司，致力于建设城市数字经济基础设施。","createdAt":"2021-12-11T23:11:13.132Z","updatedAt":"2022-11-30T08:25:32.894Z","publishedAt":"2021-12-11T23:11:16.643Z","locale":"zh"}}]}}}}},
    "staticQueryHashes": []}