{
    "componentChunkName": "component---src-templates-case-study-detail-ts",
    "path": "/case-studies/12",
    "result": {"pageContext":{"next":{"id":1,"attributes":{"feature":true,"title":"Apache ShardingSphere 在京东白条场景的落地之旅","content":"\n>京东白条使用 Apache ShardingSphere 解决了千亿数据存储和扩容的问题，为大促活动奠定了基础。\n>\n>2014 年初，“京东白条”作为业内互联网信用支付产品，数据量爆发式的增长，每一次大促备战都是对技术人员的考验，每一次的战略转型驱动着数据架构的成长。\n>\n>— 张栋芳，京东白条研发负责人\n\n### 项目背景\n\n自 2014 年 2 月京东白条业务上线起，为满足快速发展的业务和激增的海量数据，白条的数据架构经历了数次演进。\n\n白条数据架构的演进间接地反应了互联网消费金融的飞速发展，也说明了每一种解决方案在不同背景下都有不同的保质期。\n\n### 业务挑战\n\n随着产品升级迭代，早期的解决方案演变成为了眼前的问题，通过业务框架实现的数据分片方案导致业务代码复杂度增加、维护成本不断攀升，紧耦合的弊端原形毕露，应用每次升级都需要投入较多的精力对分片做相应调整，研发人员难以专注于业务本身。\n\n技术团队经权衡后开始考虑使用成熟的分库分表组件来承担这部分工作，让业务系统升级和架构调整不再复杂。\n\n### 解决方案\n\nShardingSphere-JDBC 是 Apache ShardingSphere 的第一款产品，它定位为轻量级 Java 框架，在 Java 的 JDBC 层提供的额外服务。它使用客户端直连数据库，以 jar 包形式提供服务，无需额外部署和依赖，可理解为增强版的 JDBC 驱动，完全兼容 JDBC 和各种 ORM 框架。\n\nShardingSphere-JDBC 的以下特点能够很好地满足白条业务场景：\n\n- 产品成熟：经数年打磨产品成熟度高，且社区活跃；\n- 性能良好：微内核、轻量化的设计，性能损耗极小；\n- 改造量小：支持原生的 MySQL 协议，研发工作量小；\n- 扩展灵活：搭配使用迁移同步组件轻松实现数据扩展。\n\n经内部大量系统性验证之后，Apache ShardingSphere 成为了京东白条数据分片中间件的首选方案，2018 年底正式开始对接。\n\n![jd_zh_1.png](https://official-media.sphere-ex.com/jd_zh_1_fac28cd6ec.png)\n\n### 用户收益\n\n- 简化升级路径\n\n通过架构解耦，业务系统升级所涉及技术栈得到有效缩短，研发团队不再需要关注分表设计，精力全部聚焦于业务本身，升级路径得到大幅度优化；\n\n- 节省研发力量\n\n引入成熟的 Apache ShardingSphere 无需重新开发分表组件，在简化业务升级路径的基础上节省了大量研发力量；\n\n- 架构灵活扩展\n\n搭配使用 Scaling 同步迁移组件从容面对“618”和“11.11”等大型活动，系统灵活扩容。\n\n**SphereEx，让数据与应用连接更简单。**\n\n","date":"2021-12-01","author":"SphereEx","excerpt":"京东白条使用 Apache ShardingSphere 解决了千亿数据存储和扩容的问题，为大促活动奠定了基础。","metaDescription":"京东科技集团是京东集团旗下专注于以技术为产业服务的业务子集团，致力于为企业、金融机构、政府等各类客户提供全价值链的技术性产品与解决方案。","createdAt":"2021-12-11T23:10:03.172Z","updatedAt":"2021-12-14T07:22:15.304Z","publishedAt":"2021-12-11T23:10:07.307Z","locale":"zh","caseStudiesType":{"data":{"id":4,"attributes":{"name":"电商","createdAt":"2021-12-11T23:02:39.013Z","updatedAt":"2021-12-11T23:02:41.225Z","publishedAt":"2021-12-11T23:02:41.156Z","locale":"zh"}}},"cover":{"data":{"id":6,"attributes":{"name":"jd_cover.png","alternativeText":"jd_cover.png","caption":"jd_cover.png","width":1030,"height":720,"formats":{"thumbnail":{"name":"thumbnail_jd_cover.png","hash":"thumbnail_jd_cover_38f39465dd","ext":".png","mime":"image/png","width":223,"height":156,"size":9.38,"path":null,"url":"https://official-media.sphere-ex.com/thumbnail_jd_cover_38f39465dd.png"},"large":{"name":"large_jd_cover.png","hash":"large_jd_cover_38f39465dd","ext":".png","mime":"image/png","width":1000,"height":699,"size":74.1,"path":null,"url":"https://official-media.sphere-ex.com/large_jd_cover_38f39465dd.png"},"medium":{"name":"medium_jd_cover.png","hash":"medium_jd_cover_38f39465dd","ext":".png","mime":"image/png","width":750,"height":524,"size":47.51,"path":null,"url":"https://official-media.sphere-ex.com/medium_jd_cover_38f39465dd.png"},"small":{"name":"small_jd_cover.png","hash":"small_jd_cover_38f39465dd","ext":".png","mime":"image/png","width":500,"height":350,"size":25.77,"path":null,"url":"https://official-media.sphere-ex.com/small_jd_cover_38f39465dd.png"}},"hash":"jd_cover_38f39465dd","ext":".png","mime":"image/png","size":46.19,"url":"https://official-media.sphere-ex.com/jd_cover_38f39465dd.png","previewUrl":null,"provider":"dine-strapi-provider-upload-aws-s3","provider_metadata":null,"createdAt":"2021-12-12T12:04:05.759Z","updatedAt":"2021-12-14T07:19:51.932Z"}}},"metaLogo":{"data":{"id":10,"attributes":{"name":"jd_logo.png","alternativeText":"jd_logo.png","caption":"jd_logo.png","width":300,"height":44,"formats":{"thumbnail":{"name":"thumbnail_jd_logo.png","hash":"thumbnail_jd_logo_591b2e5e14","ext":".png","mime":"image/png","width":245,"height":36,"size":6.43,"path":null,"url":"https://official-media.sphere-ex.com/thumbnail_jd_logo_591b2e5e14.png"}},"hash":"jd_logo_591b2e5e14","ext":".png","mime":"image/png","size":2.36,"url":"https://official-media.sphere-ex.com/jd_logo_591b2e5e14.png","previewUrl":null,"provider":"dine-strapi-provider-upload-aws-s3","provider_metadata":null,"createdAt":"2021-12-12T12:04:06.233Z","updatedAt":"2021-12-14T07:19:09.532Z"}}},"localizations":{"data":[{"id":6,"attributes":{"feature":true,"title":"Apache ShardingSphere & JD Baitiao: Story of an Implementation Journey","content":">Apache ShardingSphere has become the best solution for JD Baitiao’s very large storage scenarios and scale-up. JD’s shopping festival weighed very much on our big data storage. I mean, like hundreds of millions of data sets.\n>\n>JD Baitiao, an internet credit products provider launched in 2014, has always been challenged by data explosion. Every shopping festival is like a technical test for us, but every time the technical staff professionals take proactive strategic initiatives in data architecture development to ensure everything goes smoothly.\n>\n>— Dongfang Zhang，JD Batiao R&D Director\n\n## The Baitiao Data Architecture Evolution\n\nLaunched in Feb. 2014, JD Baitiao’s data architecture has already evolved for several times because of rapid increase in data volume and rapid business growth.\n\n### From 2014 to 2015\n\nBaitiao first adopted the Solr + HBase solution to connect core business systems and other systems with key databases. Solr indexing is able to achieve fast search responses and HBase is used to store a massive amount of data.\n\n- A Solr cluster can help the core database reduce reads and writes stress.\n- However, Solr is not perfect: it has unsatisfying extensibility and intrusions.\n\n### From 2015 to 2016\n\nThe company chose NoSQL to store monthly data tables in Mongo DB clusters, temporarily meeting the demand for importing/exporting massive data in payment settlement scenarios.\n\n- The change made it quite efficient to query hot data. In addition, thanks to such unstructured data storage, staff can easily modify table structures.\n- Nevertheless, the plan was still problematic: bad extensibility, intrusions, and memory hog.\n\n### From 2016 to 2017\n\nGrowing business and growing data. Tens of billions of data volume imposed more performance and volume pressure on MongoDB. JD Baitiao’s Big Data Platform used DBRep and MySQL Slave to capture data changes and store the information data in its message center, to later be written to ES and HBase.\n\n- This plan was actually much better. It focused on real-time data and improved extensibility.\n- However, Baitiao faced high costs to maintain its code because of the data sharding problem in its architecture.\n\nThe architecture evolution of JD Baitiao is just an illustration of rapidly growing Internet consumer finance. All the methods Baitiao adopted in the past were not comprehensive, indicating that every solution would be outdated soon.\n\n## Time to Adopt a Decoupled Architecture\n\nTo ensure good system performance even with rapidly increasing data volumes, the technical team first chose the sharding architecture pattern that not only guaranteed excellent performance, but also kept code under control.\n\n### The data striping plan was actually based on application architecture:\n\nThe product continuously developed but the early solution had now become one of the biggest problems. The old data sharding plan makes code even more complex and increases maintenance expenditure. Developers have to spend much time adjusting sharding every time when the application upgrades, so they cannot concentrate on their own developments. In fact, tight coupling is to blame for the problem.\n\nTherefore, the team decided to use a mature sharding component that simplifies system upgrades and architecture changes. To compare Baitiao’s shard and ShardingSphere shard, their differences are shown in the following table.\n\n![jd_en_1.png](https://official-media.sphere-ex.com/jd_en_1_eb365b4a93.png)\n\nFor Baitiao, decoupling was the next task.\n\nApparently, JD Baitiao‘s data architecture will experience a brand-new journey to decoupling. The following three directions definitely fueled its transformation:\n\n**Concentration:** instead of database sharding built in its architecture, it applied a sharding component to place more energy on it own product development.\n\n**Easier Upgrade:** it used a decoupled architecture to simplify the R&D process of system upgrades.\n\n**Future-Oriented Plan:** it aims to improve system extensibility so that Baitiao will be more capable of confidently holding large online shopping festivals like the 618 Shopping Festival and Double 11 Shopping Festival (also known as Singles Day on November 11th).\n\nJD Baitiao is a huge business, so its business scenario is truly related to finance, high concurrency, and massive data volumes. Baitiao’s sharding component must have the following features:\n\n1. Be a mature product\n2. Have excellent performance\n3. Be able to handle big data\n4. Have an extensible architecture\n\n## The Apache ShardingSphere Solution\n\nThe lightweight Java framework ShardingSphere-JDBC is Apache ShardingSpheres’s first product providing services such as a Java Database Connectivity (JDBC) API. ShardingSphere JDBC uses jar archive file installation package and allows the client-side to directly connect to databases. So, it requires no extra deployment dependencies. It is like an enhanced JDBC driver fully compatible with JDBC and ORM frameworks.\n\nShardingSphere — JDBC provides the following features, making it the best solution in the Baitiao scenario.\n\nMature Product: ShardingSphere — JDBC is a mature product developed for years, and its open source community is very active.\n\nExcellent Performance: its micro-kernel and lightweight design hardly hinder performance.\n\nMinimum Changes: it supports native MySQL protocol and minimizes R&D workload.\n\nExtensibility: users can combine with the migration sync component to easily extend data.\n\n![jd_en_2.png](https://official-media.sphere-ex.com/jd_en_2_c92fb9e5ef.png)\n\nAfter being systematically tested for several times, Apache ShardingSphere became JD Baitiao’s first choice of data sharding middleware. The cooperation started at the end of 2018.\n\n## Product Adaptation\n\nTo provide Baitiao with better services and support the business, Apache ShardingSphere has made many improvements to its product features and performance during the implementation process. At the same time, the user case helps optimize the product in return.\n\n### Upgrade the SQL Engine\n\nBaitiao’s business logic is extremely complicated; its diversified scenario demands require the best SQL compatibility levels. Thus, Apache ShardingSphere restructures the SQL parser module to support more SQLs.\n\n- Single data node routing: 100% SQL compatibility\n- Multi data node routing：comprehensively support DML, DDL, DCL, TCL and partly DAL. Support features such as pagination, deduplication, ranking, grouping, aggregation and correlated query.\n\n### Distributed Key\n\nApache ShardingSphere provides built-in distributed key generators, such as UUID, and SNOWFLAKE. It also provides its distributed key generation API so that users can develop custom key generation algorithms to meet their special needs.\n\n### Shard Key Value Injection\n\nIf a SQL has no shard condition, Apache ShardingSphere uses ThreadLoad to manage shard key value, and users can program and add a shard condition to HintManager, making the condition only effective in the current thread. That’s so-called SQL zero intrusion.\n\nAdditionally, Apache ShardingSphere continues to optimize its other features in order to meet Baitiao’s need for high performance such as：\n\n- SQL parsing result cache\n- JDBC metadata cache\n- Bind table & Broadcast table\n- Automated execution engine & Stream merge\n\nJD Baitiao & ShardingSphere teams joint efforts amde all indicators of the product live up to their expectation. The resulting final performance is almost the same as that of a native JDBC.\n\n![jd_en_3.png](https://official-media.sphere-ex.com/jd_en_3_582e2afe89.png)\n\n## Cutover\n\nApache ShardingSphere uses its custom HASH strategy to shard data, effectively avoiding the hot data problem. The total of data nodes almost reaches ten thousand. The cutover process lasts for about 4 weeks.\n\n1. After DBRep reads data, Apache ShardingSphere synchronizes the data to the target database cluster.\n2. Two clusters run together. After a data migration, Baitiao uses its own tool to verify business and data.\n\nDBRep is fundamental to the product design of ShardingSphere-Scaling. The automation feature of Scaling definitely facilitates migration and scale-up.\n\n## Apache ShardingSphere Brings Many Benefits\n\n### Simplified Upgrade Path\n\nThe decoupled architecture effectively simplifies a technology stack required in a system upgrade, so the developer team no longer need to worry about table sharding. Instead, they can focus on the business itself. In brief, ShardingSphere greatly helps Baitiao optimize the upgrade path.\n\n### Save R&D Cost\n\nBaitiao directly uses the mature product Apache ShardingSphere, so they do not need to reinvent the wheel and save a lot of time and energy.\n\n### Improve Architecture Extensibility\n\nBaitiao also uses the sync migration component Scaling that makes system scale-up even more flexible. It’s the company’s secret for online shopping festival successes.\n\n## Summary\n\nThe growth of JD Baitiao’s business greatly stimulates its data architecture upgrades. This time, it chose Apache ShardingSphere to decouple its architecture and make future upgrades less complicated. Thank to this from now on, developers only have to concentrate on their own product optimization because its data architecture is extensible enough. The case is really a good example of ShardingSphere application in the consumer finance scenario.\n\nNow, there are more and more internet credit payment models. In the future, Apache ShardingSphere in collaboration with JD Technology will explore more business scenarios, and contribute finance-related technological innovations, further improving Internet finance.\n\n### About Apache ShardingSphere\n\nShardingSphere is one of the Apache Software Foundation Top-Level Open Source projects, used by over 170 enterprises worldwide, across various verticals such as finance, e-commerce, cloud services, tourism, logistics, education, and entertainment. Its GitHub community has so far cumulated over 14,000 stars.\n\nWe welcome more technical professionals to write articles and share their experiences and thoughts. If you are interested, feel free to contact us:\n\nShardingSphere Github: https://github.com/apache/shardingsphere\n\nShardingSphere Twitter: https://twitter.com/ShardingSphere\n\nShardingSphere Slack Channel: apacheshardingsphere.slack.com","date":"2021-12-01","author":"SphereEx","excerpt":"Apache ShardingSphere has become the best solution for JD Baitiao’s very large storage scenarios and scale-up. JD’s shopping festival weighed very much on our big data storage. I mean, like hundreds of millions of data sets.","metaDescription":"JD Technology Group is a business subgroup of JD Group that focuses on technology as an industry service, and is committed to providing technical products and solutions for enterprises, financial institutions, governments and other types of customers across the value chain.","createdAt":"2021-12-12T12:30:44.957Z","updatedAt":"2022-09-26T09:16:15.245Z","publishedAt":"2021-12-12T12:35:08.683Z","locale":"en"}}]}}},"prev":{"id":43,"attributes":{"feature":true,"title":"OPPO云数据库访问服务技术揭秘","content":"## **1、背景**\n\nMySQL是OPPO使用最广泛的关系数据库，不同编程语言的微服务都是通过MySQL官方的SDK直连真实的数据库实例。这种最传统的使用方式，会给业务开发和数据库运维带来一系列影响效率和稳定性的问题。\n\n- 不合理的数据库访问，比如不恰当的连接池设置、高危SQL无拦截、无限流和熔断等治理能力\n- 无弹性伸缩能力，单机数据库性能或者容量不足时，扩容非常繁琐低效\n- 缺乏常用的功能特性，不支持读写分离、影子库表、单元化架构、数据库加密\n- 不支持跨语言，并且与应用服务强耦合，编码和升级都非常困难\n\n以上问题，我们讲通过CDAS来解决单MySQL带来的一系列问题。\n\n## **2、CDAS产品简介**\n\nMySQL在并发能力、稳定性、高可用方面久经考验，是绝大多数网联网产品首选的OLTP场景存储组件，但单机MySQL在超高并发、海量存储、OLAP能力上先天不足，因此当数据到达一定量级后一般采用分库分表的方式来水平扩展MySQL，提升系统的整体处理能力。CDAS基于分片理念设计，目的是解决MySQL单机性能瓶颈，对外提供超过并发、海量存储、支持HTAP的MySQL服务。在一组MySQL集群前搭建一套高可用的代理 + 计算集群，提供分片能力、自动化的弹性伸缩能力、读写分离、影子库、数据加密存储能力，为用户提供一体化的产品。\n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=ZjhkYTExOTEyN2EwNjBlYzE3YjIzZjEzZDRhMGYyMTZfM3dFcXZ0d1RCN2FjVGVxeXpYbWJ2ZTJnNGFuNGFRd1VfVG9rZW46Ym94Y25tWlhpYU85Wjcxb2RVM2NEUlF0SXNiXzE2ODAxNTg5NDE6MTY4MDE2MjU0MV9WNA)\n\nCDAS Proxy使用Java语言开发，基于开源产品Apache ShardingSphere Proxy改造而来，并添加了许多高级特性来支持内部业务。我们的开发理念是依托开源的成熟产品来搭建服务，从社区汲取营养的同时也把发现的问题、特性回馈给社区，积极参与、和社区共同发展。 \n\n**2.1 CDAS产品特点**\n\n**（1）稳定性**\n\nCDAS进行了大量的基准测试、调优，实现了特殊的动态队列 + 线程池模型来保证Proxy的总体并发程度不会由于线程数过多导致上下文切换频繁从而性能下降，同时Proxy在长时间高负载的场景下不会产生Full GC导致业务中断，目前正式环境已有多个QPS:5000+的业务场景使用了CDAS，表现稳定。\n\n**（2）高度可扩展**\n\n我们建议业务在申请分片表时预估未来3~5年的数据总量来设定分片总数，前期可申请1~2个数据库，数据量变大后扩容。理论上可扩展性和分片总数相关，最大可扩展分片数相等的数据库实例，最大可扩展到250TB，同时Proxy集群也支持在线扩容。\n\n**（3）平台化运维**\n\n由云平台统一运维、部署，MySQL和Proxy都支持自动化流程申请和变更，自动接入元数据管理系统。\n\n**（4）兼容MySQL语法**\n\n支持绝大多数MySQL语法。\n\n- 路由至单数据分片100%兼容\n- 支持分页、去重、排序、分组、聚合、关联等常见查询场景\n\n**（5）功能丰富**\n\n- 多种分片算法\n- 自增分布式ID\n- 读写分离，可设置主从同步容忍阈值\n- 影子库\n- HTAP\n- 完善的监控信息：审计日志、慢日志、鉴权 \n\n## **3.核心设计** \n\nCDAS的核心目标是解决用户海量数据存储、访问问题，聚焦到分片场景中，主要体现在解析、路由、重写、聚合等逻辑 。\n\n**3.1 内核架构**\n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=Y2E0NjkzNmEyZDdmMDI1ODhjZmIxMzc1NWEyMzMwZTFfbm00REp3aU9kbFhXRHNCVVdtUTlmcm1ETHlkVVhQcmNfVG9rZW46Ym94Y25HZmtiRWdWSW9MR0RjVUZqRjhDaWVnXzE2ODAxNTg5NDE6MTY4MDE2MjU0MV9WNA)\n\n站在高处看Proxy内核，主要分为连接接入模块、I/O多路复用模块、解析路由执行引擎3个大的部分，左侧可以看到线程模型也分为3块，实际涉及线程资源的还有其他子逻辑，如并行执行线程池等，接下来将围绕多个模型和执行流程了解内核架构的细节\n\n**3.2 线程模型**内核入口层基于Netty实现，从整体上看是一个典型的Netty使用场景，内核图从可以看到主要分为3个部分。 \n\n**（1）Boss Thread**负责Accept connection，也就是接受、建立连接，客户端一般都会使用连接池技术，因此建立连接的请求不会太多，单线程即可处理 \n\n**（2）I/O Threads**即Netty EventLoopGroup ，负责编解码、发起auth认证，基于Epoll事件驱动 I/O多路复用，一个线程可处理多个连接，共CPU核心数 * 2个线程。 \n\n**（3）Worker Threads**负责执行、回写数据等核心流程。Worker线程整体是同步的，其中Parser\\Router\\Rewriter\\RateLimter是纯CPU计算，Executor Engine、ResultManager底层基于JDBC规范，JDBC目前对外暴露的是一个同步的调用模型，在数据库未响应前线程状态为blocked，因此Worker线程池的配置决定了Proxy整体的并发能力。 线程资源在操作系统中是相对昂贵的资源，在CPU核心数固定的情况下，线程数量过大会耗费大量内存和导致上下文频繁切换，降低性能。我们在压测过程中额外关注了线程数对服务的影响，找到了一个合适的计算公式。\n\n```Plaintext\nMath.min(cpuNum * X, maxThreadsThreshold)\n```\n\n- X：默认75，可配置\n- maxThreadsThreshold：默认800，可配置\n\n同时我们自研了DynamicBlockingQueue，在队列积压任务到一定阈值后提前创建新线程执行任务。\n\n**3.3 连接模型**\n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=NmExYWRlYzEzOTBiYmQ0N2MyZTE1YzAyYWYzOWQ3MWJfZ2oyeHM0R3FVM2lqaUEwMk9wc3E4R0p4RW53UDB4bTBfVG9rZW46Ym94Y253YVNQeWpFZUxOM2NiZGRxTEplMzRiXzE2ODAxNTg5NDE6MTY4MDE2MjU0MV9WNA)\n\n在接受客户端连接后，Proxy内部会为每个客户端连接维护1个Backend Connection的逻辑连接。Backend Connection的connections列表保存执行过程中用到的真实连接，请求执行完毕或事务结束后清空connections列表，释放物理连接。同时，Proxy内部使用连接池技术来保存物理连接，减少连接建立时间并控制总的连接数量\n\n**3.4 事务模型**接下来聊一下在事务场景中的执行流程。\n\n**3.4.1 单库事务**假设有以下事务场景：\n\n```Plaintext\nbegin;\nselect * from t_order where order_id = 1;\nupdate t_order set gmt_modify = now() where order_id = 1;\nupdate t_order_item set cnt = 2 where order_id = 1;\ncommit/rollback;\n```\n\n事务过程中一共会和数据库顺序交互5次。 语句1：begin时Proxy并不知道后面要执行什么语句，无法路由到RDS，只是在连接上记录了一个begin状态语句2：执行带分片键的select，还是以16分片为例，最终语句会路由到t_order_1这张表中，获取connection_db1，先执行begin，再执行select，并放入逻辑连接的连接列表中\n\n> connection_db1为t_order_1表所在数据库上的连接\n\n语句3、语句4：路由到t_order_1,t_order_item_1,和语句2路由到相同DB，复用connection_db1执行语句5：单纯的commit或rollback语句，取出当前逻辑连接的所有RDS连接，挨个执行commit/rollback在事务执行过程中，是否会产生分布式事务完全由用户SQL语句来控制，如果事务执行过程中不会切换数据库，则会退化成单纯的RDS transaction，保持完整的ACID特性，如果事务执行过程中出现路由到多个DB的情况，则会产生分布式事务。\n\n**3.4.2 分布式事务**目前分布式事务的解决方案主要有以下3种\n\n**（1）最终一致**\n\n业务发起的最终一致性方案，如Seata\\TCC等，业务有感，多用于跨服务调用场景，如订单和库存系统，某一环提交失败需特定逻辑来整体回滚，不适合Proxy场景\n\n**（2）强一致XA**\n\n多用于跨服务调用场景，目前存在性能不佳，协调器单点\\Recovery锁占用等问题，不适合高并发场景使用\n\n**（3）厂商提供分布式事务**\n\n分布式数据库厂商如OceanBase本身会对数据进行分片，事务执行过程中操作了多个分布式节点从而引入分布式事务。针对单个分布式库内的事务保证ACID，Proxy后端基于MySQL协议，无法在多个RDS间实现这类事务。\n\n**（4）现状**\n\nCDAS Proxy并没有提供XA来保证跨RDS实例的强一致性，也没有在内部支持最终一致性。如果触发多库事务则分开提交，有部分提交成功、部分失败的风险，因此建议业务方在设计事务时尽量不再RDS集群内部产生跨库事务。\n\n**3.5 分片模型**\n\n以4库16分片场景为例阐述CDAS分片方案和传统方案的区别\n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=M2QzMmRkNGU3YWQ4ZjQxOTM0MTM5NDRmZmZiNTZkMjlfc3ZiM2Y0akdiVnBKNjZHTUgyZ1F1bG9rOTJubkR5cmxfVG9rZW46Ym94Y25TWU1xTnhaTFhvaFlhSTZFWXo1Q2xjXzE2ODAxNTg5NDE6MTY4MDE2MjU0MV9WNA)\n\n传统方案每个DB中的分片名都是从后缀0开始的，每个库中有相同的分片数量。有以下劣势：\n\n- 热点分片处理困难 \n- 无法针对分片进行迁移 \n- 分片名无意义，无法自解释 \n\n针对以上问题，我们采用了类似范围分片的思路来优化了分片名。\n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=ZGJkYTA5NDM3YjRmOTA3YzUzMjRjZTY2MWMyNGVjMjRfWk1uaGFVdG5mdWd3QWFEc2wxazVSeVhWSFVZak52ZXFfVG9rZW46Ym94Y253NEJRTHMxejFNVDFTNXQzM2o1SkZjXzE2ODAxNTg5NDE6MTY4MDE2MjU0MV9WNA)\n\n分片名是唯一的，位于不同DB中的分片名互不相同，这种设计方式与Redis/MongoDB中不同slot位于不同实例上很相似，有利于扩展。\n\n| 问题         | 解决方式                                                     |\n| ------------ | ------------------------------------------------------------ |\n| 热点分片     | 热点分片可以通过扩展1个新RDS来解决，如添加DB5，将某个流量特别大的热点迁移到DB5 |\n| 分片迁移     | 单个分片迁移不影响其他分片，也不会被分片算法影响             |\n| 分片名无意义 | 通过分片名唯一让每个分片能表述它在整体集合中的位置           |\n\n t_order_1为热点分片或DB1压力过大时迁移t_order_1到新的DB5中：\n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=Mjg3NDVkZTE5ZDk1ZDViM2ZlZjQzN2RiOGZmYmViMGJfWk1NNEdxTXVBNmhsZGpwdEtZQ0hDZ095dTFoR3JJemhfVG9rZW46Ym94Y25rbDJLQ1dNTXB2RVdHS1Z0cTF1aWpmXzE2ODAxNTg5NDE6MTY4MDE2MjU0MV9WNA)\n\n当然这种迁移方式并不适用于所有场景，如表t_order、t_order_item的分片算法完全相同，则他们一定落到同一个DB中，如果迁移过程中只迁移其中一张表势必会导致不必要的分布式事务，因此迁移时会将绑定关系的表一并迁移。**3.6 执行流程** 以最简单的场景举例，逻辑表：t_order分16片，分片键：order_id\n\n```Plaintext\nselect * from t_order where order_id=10\n```\n\n**（1）解析**\n\nWorker线程拿到这条语句后首先要做的是解析SQL语句，基于开源产品Antlr实现语法树解析。\n\n```Plaintext\n{type：select，table：t_order，condition: order_id=10}\n```\n\n**（2）路由**\n\n在获取到分片表名和条件后，根据表名获取分片规则并计算分片\n\n```Plaintext\norder_id mod 16 => 10 mod 16 => 10\n```\n\n得到真实的table：**t_order_10**（**3）重写**重写流程会将逻辑表名替换为真实表名，并计算真实位于哪个DB中。\n\n```Plaintext\nselect * from t_order_10 where order_id=10\n```\n\n**（4）限流**\n\n限流功能可以控制某类SQL的瞬时并发度，目前仅对部分场景进行了限流，如OLAP请求。\n\n**（5）执行**\n\n语句将会发往真实的数据库中执行，目前OLTP请求使用MySQL官方Connector/J发送，OLAP请求使用ClickHouse官方SDK发送，并且AP流量不会路由+重写，因为在整体设计上ClickHouse集群中的分布式表名和逻辑表名是相同的\n\n**（6）聚合结果**\n\n聚合的主要场景是查询请求被路由到了超过一个真实表的场景，如\n\n```Plaintext\nselect * from t_order where order_id in (10,11)\n```\n\n这种情况一条SQL最终到DB层面会是两次子请求\n\n```Plaintext\nselect * from t_order_10 where order_id in (10,11)select * from t_order_11 where order_id in (10,11)\n```\n\n聚合时，通过JDBC API遍历两个子请求的ResultSet，在内存中新建一个逻辑ResultSet，当逻辑ResultSet被遍历时会触发子ResultSet被遍历，最终将逻辑ResultSet编码在MySQL协议包发往客户端。如果SQL语句中包含order by，聚合时会在内存里面进行归并。内存里面排序需要将所有行数据加载到内存中排序，内存压力会很大，因此虽然有排序能力，我们还是不建议在请求分裂多个子请求的场景中使用order by语法。\n\n**3.7 HTAP**HTAP的是指混合OLTP（Online Transactional Processing）和 OLAP（Online Analytical Processing），在数据库中同时支持在线事务请求和后台分析请求。 MySQL是一个典型的OLTP型数据库，不太适合OLAP型业务，因为分析语句一般需要执行复杂的聚合和查询大量数据，这可能会影响其在OLTP上的稳定性。同时OLAP数据库一般在存储方式上采用列存来最大化压缩效率、空间占用，而MySQL采用行存，一般情况下须先将整行记录查询出来再过滤出某些列，在执行分析请求时会有明显的IO读放大，效率不高。CDAS通过DTS（数据传输）服务将MySQL各分片数据传输并聚合到ClickHouse（分析型列存数据库），形成统一的数据视图。AP流量到达Proxy后会被转发到ClickHouse，数据响应后再以MySQL数据包的形式返回给客户端，通过这种方式，CDAS实现了基于MySQL协议的HTAP能力。\n\n## **4.部署架构**\n\n部署拓扑图如下：\n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=MWNiOGQ1YzAxODA2OTMxYzdhZGYwOWY1OTQzNTBhZmVfMGhLaXF6c1FsTWUyQ2hFbFhGZTYwc3Vmcnd3R0JSUHJfVG9rZW46Ym94Y25acFNIek5pMlJ6NG5BS3c1bFNSd2I4XzE2ODAxNTg5NDE6MTY4MDE2MjU0MV9WNA)\n\n现代架构中业务方对高可用的要求越来越高，目前大多数重要的服务都会采用多机房主备或互为主备的方案来部署，一般采用双机房或双中心化方案，保证其中一个区域挂掉后仍然能对外服务。在设计之初CDAS考虑到作为L7层的数据流量入口拥有极高的高可用要求，因此允许用户申请双机房Proxy集群，保证其中1个机房挂掉后另1个机房依然可以承接流量对外服务，实现机房级的高可用。业务系统通过L4负载均衡来和Proxy建立连接，L4和Proxy紧密协调保证优雅下线以及新实例发现，并感知Proxy节点的健康状态，及时下线不可用节点。同时由L4来做同机房优先路由、流量转发，Proxy后的所有MySQL集群都采用semi-sync的高可用架构模式。这种部署架构使CDAS具备了机房级灾备的能力。\n\n## **5.性能测试**\n\n在测试过程中我们尤其关注了性能方面的损耗，并对Proxy代理、直连RDS进行了对比测试，这样就能直观的得到性能损耗程度。\n\n**5.1 压测工具**\n内部压测平台(内部基于JMeter)\n\n**5.2 压测方式**\nJava SDK + 连接池方式压测 主要分为两组\n\n- 压测Proxy，Proxy连接2个RDS集群  \n- 压测单个RDS集群，作为Proxy性能对比的基准 \n\n压测语句：参考**SysBench**的数据库压测语句，由于分片场景一般不会直接使用主键ID分片，且主键ID在InnoDB中查询效率高于二级索引，因此添加sharding_k列进行分片测试，更符合实际的应用场景。\n\n```Plaintext\nCREATE TABLE `sbctest_x` (  \n`id` bigint(11) NOT NULL,\n`sharding_k` bigint(11) NOT NULL,\n`k` int(11) DEFAULT NOT NULL,  \n`name` varchar(100) NOT NULL,  \n`ts` timestamp NULL DEFAULT NULL,  \n`dt` datetime DEFAULT NULL,  \n`c` char(100) DEFAULT NULL,  \n`f` float DEFAULT NULL,  \n`t` tinyint(4) DEFAULT NULL,  \n`s` smallint(6) DEFAULT NULL,  \nPRIMARY KEY (`id`),  \nKEY `sharding_k_idx` (`sharding_k`),\nKEY `k_idx` (`k`),\n) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;\n\n压测SQL\n#QPS 非事务的简单查询\nSELECT COUNT(1) FROM $table WHERE sharding_k=?\n\n#TPS 包含 1条查询，4条变更语句\nBEGIN;\nSELECT c FROM $table WHERE sharding_k=?\nUPDATE $table SET k=k+1 WHERE sharding_k=?\nUPDATE $table SET c=?,ts=?,dt=? WHERE sharding_k=?\nDELETE FROM $table WHERE sharding_k=?\nINSERT INTO $table (id,sharding_k,k,name,ts,dt,c,pad,f,t,s) VALUES(?,?,?,?,?,?,?,?,?,?,?)\nCOMMIT;\n```\n\n**5.3 压测报告** 以8C16G规格为例：SSD\\100分片\\单片250W行\\数据量大于InnoDB buffer pool\n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=MjczNGU0ZWI0OTczM2I0NzRmNDQxMGJiY2UxMTY2OTlfMFZEQWlLd3BwT1V3aDNaUEt0UjBheGZYTFJjMXVidlZfVG9rZW46Ym94Y254UzQxVW8zM0NsSlZwbzZFOFo0VmFkXzE2ODAxNTg5NDE6MTY4MDE2MjU0MV9WNA)\n\n**（1） QPS 场景**\nProxy QPS只有RDS QPS的1/2，压测过程中观察到RT约为2倍 QPS场景主要是大量非事务查询的场景，Proxy的性能损耗约1/2，QPS压测过程中我们观察到Proxy后的RDS实例CPU使用率较低，于是将Proxy的CPU规格提升1倍后，Proxy QPS立即提升1倍，接近直连RDS的性能。\n**（2） TPS 场景**\n由于TPS涉及事务提交、log等操作，I/O频繁，瓶颈也主要在I/O，即使是SSD磁盘性能也都不高。Proxy后挂载了2个RDS实例，TPS在各并发用户数中都明显高于直连单个RDS，且在并发用户数增加后可达到其2倍，性能损耗几乎可忽略。\n**（3） 结论**\n如果业务场景中非事务查询的请求占绝大多数，且RDS的CPU利用率较高，建议选择Proxy的规格时CPU要高于RDS的规格如果业务场景中事务执行请求较多，Proxy不会成为性能瓶颈，规格可和RDS保持一致\n\n## **6.案例**\n\n目前支持了多个在线业务，拥有完善的指标展示、报警机制，我们例举个业务的控制台界面及监控图。 Proxy列表：\n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=MWU2YjRmMWFmMDNiOGI2OTkwYjI3NWY0MmZmMzQzMTVfUnd2aGpZNVdjT3Jvck5pOEYzQXJXcDE5TFczM1pwd1BfVG9rZW46Ym94Y25uMVpueEg5emlkVUx5eWQ4d1lxeWtkXzE2ODAxNTg5NDE6MTY4MDE2MjU0MV9WNA)\n\n单个节点TPS:\n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=NzYxODMxN2ZlNDRmMDNiMTI2MzliMTMyMjFlYzllMmFfb0djVHU2dThteDZueGxTQW53dlZaRG1RQ0sxOXluSjNfVG9rZW46Ym94Y25DS2JjNjFBY2cxMVRCaE5hM1BmS1ZoXzE2ODAxNTg5NDE6MTY4MDE2MjU0MV9WNA)\n\n单个节点QPS:\n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=ODg3ZDg2NGZkMmVkNjkzMWExYmUwMWQ1YmZkMTkzZTZfcnlwc3NKTFgyQW1MYlM5MGRZakc1dGxFdXZVZnVYVGhfVG9rZW46Ym94Y24ycEhPSzA0RGc4RThtN1ZJR1hrVU9nXzE2ODAxNTg5NDE6MTY4MDE2MjU0MV9WNA)\n\n在多个产品的使用过程中CDAS表现稳定，我们将会继续往更大流量的场景推广，服务更多的业务方。 \n\n## **7.总结与展望**\n\nCDAS为数据分片而生，构建基于MySQL分片场景的内部标准，通过MySQL协议实现跨语言可访问，统一使用方式，简化使用成本。  未来我们将持续专注于CDAS的性能提升和功能补充，主要体现在下面2个方面： \n\n1. ShardingSphere社区已计划将Proxy的数据库驱动从MySQL Connnector/J 替换为事件驱动组件vertx-mysql-client（一款Reactive MySQL Client），基于Netty实现和MySQL交互I/O多路复用、事件驱动替换后worker线程池执行语句时将从blocking等待变为事件驱动，少量的线程即可支持大量的并发，我们将持续关注社区的开发进度、积极参与。 \n\n2. CDAS和MySQL交互过程中会自动解码Rows数据包，产生大量的对象并且都是朝生夕灭的，导致GC压力增大，同时编解码也会耗费大量CPU时间。 Rows数据包在非解密等特殊场景外都可以基于Netty ByteBuf实现零拷贝发送，这是最佳的方案。我们计划在开源版本替换为vertx-mysql-client后对这部分逻辑进行优化，使绝大多数的请求的响应时间能达到接近4层负载均衡的性能。","date":"2022-03-14","author":"SphereEx","excerpt":null,"metaDescription":null,"createdAt":"2023-03-30T07:01:59.275Z","updatedAt":"2023-03-30T07:02:00.368Z","publishedAt":"2023-03-30T07:02:00.365Z","locale":"zh","caseStudiesType":{"data":null},"cover":{"data":null},"metaLogo":{"data":null},"localizations":{"data":[]}}},"article":{"id":12,"attributes":{"feature":true,"title":"增强企业数据服务及管控能力｜SphereEx-Console 可视化数据管控平台正式发布","content":">**1 月 27 日，SphereEx 正式推出可视化数据服务管控平台：SphereEx-Console。**\n>\n>作为一款以管理 ShardingSphere 为核心的 SphereEx 商业数据服务集群平台，SphereEx-Console 以平台的形式简化用户配置，极大简化用户操作和部署流程，进而降低新手入门门槛和企业管控难度，进一步提升企业运维效率。\n\n随着数据体量增长以及前端业务敏捷性的要求，金融、政府、传统制造等领域均面临着提升底层数据架构适配性、面向分布式生态进行架构升级。但传统集中式数据库在海量数据承载能力方面存在先天性不足，于是近些年来分布式数据库不断涌现，很好地满足这一诉求。但我们也看到，分布式架构天然的复杂性，对使用者提出了更高的要求，如何在新架构下简化用户体验，便捷普惠地享受到新架构带来的优势，成为各家重点考虑的因素。\n\n作为近些年火热的开源项目，Apache ShardingSphere 的应用范围越来越广，越来越多的企业选择采用 “数据库 + ShardingSphere” 的方式来实现传统关系型数据库到分布式数据库的转型过渡。以 Apache ShardingSphere 为基础，在原有数据库之上构建一套数据中间平台，将分布式数据服务增强能力部署在中间层，而不需对底层数据架构做出调整，进而帮助企业以更加平滑的方式获得分布式以及云上迁移能力。\n\n因此，为进一步降低用户应用门槛，拓宽 Apache ShardingSphere 的使用范围，**SphereEx 在开源内核 Apache ShardingSphere 的基础上推出 SphereEx-Console 可视化数据管控平台。** 作为一款以管理 ShardingSphere 为核心的 SphereEx 商业数据服务集群平台，SphereEx-Console 以平台的形式简化用户配置，极大简化用户操作和部署流程，进而降低新手入门门槛和企业管控难度，进一步提升企业运维效率。\n\n## 什么是 SphereEx-Console\n\n**SphereEx 控制台主要用于对 SphereEx 数据平台的运维管控，在互联网、金融、政务行业均能够支持运维和开发人员常见的操作场景。**\n\n作为一款以管理 ShardingSphere 为核心的 SphereEx 商业数据服务集群平台，SphereEx-Console 可视化数据管控平台以简便易用的操作界面代替相对复杂代码输入方式，结合内部针对资源、实例、插件等多种功能封装，为用户带来一站式的使用体验。\n\n![1.png](https://official-media.sphere-ex.com/1_5bd2d7ea7f.png)\n\n### 产品优势\n\n**功能易用**\n\n提高了用户体验，避免了配置错误，用户不需要通过配置和命令操作 SphereEx 企业数据服务平台，易用性得到极大提升，实现了平台使用的“零”瓶颈。\n\n**功能全面**\n\n完善开源版本的管理能力，提供了对 SphereEx 企业数据服务平台从基础资源到插件能力的管控，形成了对周边插件实现统一管控的解决方案。\n\n**功能可视**\n\n原生提供可视化的监控数据面板，可以实时在线查看 SphereEx 企业数据服务平台的集群、实例、主机的监控数据。\n\n\n### 行业价值\n\n**本次 SphereEx-Console 的发布，是 SphereEx 在数据管理生态下一次关键尝试，弥补了 Apache ShardingSphere 过去在管理易用性、可视化界面等维度的不足。** SphereEx-Console 以可视的操作界面，极大降低了企业运维人员的学习成本；统一的数据管控平台，也极大提升了企业的运维管理效率。\n\n作为一家以开源项目 Apache ShardingSphere 为基本的商业化公司，SphereEx 将持续增强自身在数据治理、管控等方面的能力，配以周边完善的生态基础，为用户提供更加全面和完善的 Apache ShardingSphere 应用生态以及 SphereEx 产品矩阵能力。\n\n**[点击了解详情](https://sphere-ex.com/zh/products/console/)**","date":"2022-01-26","author":"SphereEx","excerpt":"提供企业级的平台管控能力","metaDescription":null,"createdAt":"2022-03-11T09:39:49.579Z","updatedAt":"2022-03-11T09:55:15.098Z","publishedAt":"2022-03-11T09:40:24.284Z","locale":"zh","caseStudiesType":{"data":null},"cover":{"data":{"id":177,"attributes":{"name":"SphereEx-Console_画板 1.png","alternativeText":"SphereEx-Console_画板 1.png","caption":"SphereEx-Console_画板 1.png","width":3750,"height":1596,"formats":{"thumbnail":{"name":"thumbnail_SphereEx-Console_画板 1.png","hash":"thumbnail_Sphere_Ex_Console_1_3552d392b3","ext":".png","mime":"image/png","width":245,"height":104,"size":26.11,"path":null,"url":"https://official-media.sphere-ex.com/thumbnail_Sphere_Ex_Console_1_3552d392b3.png"},"large":{"name":"large_SphereEx-Console_画板 1.png","hash":"large_Sphere_Ex_Console_1_3552d392b3","ext":".png","mime":"image/png","width":1000,"height":426,"size":185.42,"path":null,"url":"https://official-media.sphere-ex.com/large_Sphere_Ex_Console_1_3552d392b3.png"},"medium":{"name":"medium_SphereEx-Console_画板 1.png","hash":"medium_Sphere_Ex_Console_1_3552d392b3","ext":".png","mime":"image/png","width":750,"height":319,"size":128.74,"path":null,"url":"https://official-media.sphere-ex.com/medium_Sphere_Ex_Console_1_3552d392b3.png"},"small":{"name":"small_SphereEx-Console_画板 1.png","hash":"small_Sphere_Ex_Console_1_3552d392b3","ext":".png","mime":"image/png","width":500,"height":213,"size":73.43,"path":null,"url":"https://official-media.sphere-ex.com/small_Sphere_Ex_Console_1_3552d392b3.png"}},"hash":"Sphere_Ex_Console_1_3552d392b3","ext":".png","mime":"image/png","size":345.65,"url":"https://official-media.sphere-ex.com/Sphere_Ex_Console_1_3552d392b3.png","previewUrl":null,"provider":"dine-strapi-provider-upload-aws-s3","provider_metadata":null,"createdAt":"2022-03-11T09:27:36.623Z","updatedAt":"2022-03-11T09:27:36.623Z"}}},"metaLogo":{"data":null},"localizations":{"data":[{"id":11,"attributes":{"feature":false,"title":"The SphereEx-Console Enterprise Grade Data Governance Platform is Officially Released","content":">On January 27th, SphereEx officially launched SphereEx-Console, a data monitoring and management platform. It provides a one-stop solution incorporating Apache ShardingSphere and SphereEx as the core, a powerful user interface to manage data clusters, and a platform to simplify user configuration.\n>\n>By simplifying operation & deployment processes, reducing entry barriers and data management challenges, SphereEx-Console improves enterprise operations and maintenance efficiency. \n\n\nExpanding data volumes and increasingly demanding agility requirements from front-end businesses have created significant challenges for enterprises in the finance, government and traditional manufacturing sectors. \n\nImproving underlying data architectures' adaptability, and upgrading it to a distributed ecosystem represent some of the new requirements - however, traditional centralized databases are inherently inadequate in terms of massive data carrying capacity. \n\nAs a result, distributed databases have emerged in recent years to meet this demand. Nevertheless, the complexity of distributed architectures poses great challenges to users. Simplifying users' experience has become a key database industry driver.\n\nAs a popular open source project, Apache ShardingSphere has successfully been implemented in multiple production scenarios, with an increasing number of enterprises adopting the Database Plus approach to transition from traditional relational databases to distributed databases. \n\nBased on Apache ShardingSphere a data platform on top of the existing databases is built, enabling distributed data capability to be deployed in the middle layer without requiring adjustments to the underlying data architecture. This approach allows enterprises to smoothly transition to distributed and cloud based platforms.\n\nThe SphereEx Enterprise Data Service Platform was created by leveraging ShardingSphere's open source kernel, and enhancing it with enterprise grade features. It provides improved data service capabilities, such as data sharding, data security etc.\n\n**SphereEx-Console is a powerful dashboard applied to the management and control of the SphereEx Enterprise Data Service Platform. It provides a one-stop solution with a user-friendly experience packaging resources, instances and plugins.**\n\nIt provides a visual approach to manage data clusters with Apache ShardingSphere and SphereEx as the core, while offering a platform to simplify user configuration, and ultimately improve enterprise operations & maintenance efficiency.\n\n## What is SphereEx-Console?\n\nThe SphereEx Enterprise Data Service Platform was created by leveraging ShardnginSphere's open source kernel, and enhancing it with enterprise grade features. It provides improved data service capabilities, such as data sharding, data security etc.\n\nSphereEx-Console is a powerful dashboard applied to the management and control of the SphereEx Enterprise Data Service Platform. It provides a one-stop solution with a user-friendly experience packaging multiple functions such as resource layer, instance layer and application layer.\n\nThe SphereEx-Console is used to manage the operation and maintenance of SphereEx Enterprise Data Service Platform. It supports operation, maintenance and development scenarios in Internet, finance and government industries.\n\nAs a SphereEx business data services cluster platform with ShardingSphere at its core, the SphereEx-Console visual data control platform provides a one-stop solution with a user friendly interface instead of complex code input, packaged with various features for the resource, instance and application layers. \n\n![Diagram Console EN.png](https://official-media.sphere-ex.com/Diagram_Console_EN_5f98720b3f.png)\n\n\n### SphereEx-Console Advantages\n\n**Easy to Use**\n\nIt improves the user experience and eliminates configuration errors. Users are not required to operate the SphereEx Enterprise Data Service Platform through configuration commands, greatly improving user experience. \n\n**Comprehensive**\n\nIt provides a unified solution for the management and control of the SphereEx Enterprise Data Service Platform - from basic resources to plugin capabilities.\n\n**Visual Monitoring**\n\nWith its data monitoring & management dashboard, SphereEx Enterprise Data Service Platform's clusters, instances and hosts monitoring data are visible online and in real time.\n\n### Industry Leading\n\nThe release of SphereEx-Console is an essential milestone for SphereEx's data management ecosystem. It improves and evolves Apache ShardingSphere in areas such as management difficulty and user interface. \n\nThe user interface greatly reduces enterprise operations & maintenance teams' learning costs, while the unified data management platform greatly improves operations and maintenance efficiency.\n\nAs a business founded the open source project Apache ShardingSphere's core team, SphereEx continues to enhance data governance as well as management & control capabilities. We are committed to providing users with a comprehensive Apache ShardingSphere application ecosystem and SphereEx enterprise grade capabilities.\n\n[Click here for more details!](https://sphere-ex.com/en/products/console/)","date":"2022-01-26","author":"SphereEx","excerpt":"Enhancing Enterprise Data Governance Capabilities","metaDescription":null,"createdAt":"2022-03-11T09:38:54.812Z","updatedAt":"2022-09-27T08:42:51.504Z","publishedAt":"2022-03-11T09:59:17.395Z","locale":"en"}}]}}}}},
    "staticQueryHashes": []}