{ "componentChunkName": "component---src-templates-case-study-detail-ts", "path": "/case-studies/13", "result": {"pageContext":{"next":{"id":40,"attributes":{"feature":true,"title":"基于Sharding-JDBC的订单分库⽅案","content":"# **前言** \n\n随着业务的发展，数据量迅速增⻓，单实例 Mysql 数据库的磁盘存储容量很快就会达到上限（阿⾥云 RDB 最⼤上限6T）。因此为了解决⽣产环境的存储瓶颈，需要对数据库进⾏分库设计，加⼤存储空间。**本⽅案基于Apache开源项⽬ ShardingSphere 采 Sharding-JDBC 进⾏订单⽔平分库设计（数据分⽚）缓解存储压⼒。**\n\n \n\n# **方案对比**\n\n⽬前主流的分库分表⽅案就是 Sharding-JDBC 和 Mycat，下⾯主要对⽐这两种。\n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=MGZlMTlmM2IyYTNiZDYzMTk5YjI0NWVlYjJlMzE2NmZfRXVzMURvRUdabmYxR1J4MTJxVWFkRGNuVE1GbjJia21fVG9rZW46Ym94Y25EMDBYVHBVUFhuRm51ZVpvS2Rja3ZjXzE2ODAwNzQ5MzU6MTY4MDA3ODUzNV9WNA)\n\n经上述对⽐，可以发现 Sharding-JDBC 是直连数据库、没有中间层的⽅式，性能上更优，同时由于 Sharding-JDBC 集成在应⽤代码内，并不会增加额外的运维成本，使开发者可以专注于⾃⾝代码逻辑。另外，Sharding-JDBC ⽬前仅⽀持 Java，考虑到⽬前团队技术栈主要以 Java 为主，所以采⽤ Sharding-JDBC 的分库⽅案成为了⼀个不错的选择。 下图反映了 Sharding-JDBC 在应⽤中所处的位置：\n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=NTY1M2UyZjBmNWE1NThiMzg1YmI0YTczYmNmNDZhMjZfdHJtWU1pSUdNVnRLV3JabHo0cktNUndXbGxEWkdXYXhfVG9rZW46Ym94Y25XVGdhUGdOVFVid21ucHlTWnpFSFNkXzE2ODAwNzQ5MzU6MTY4MDA3ODUzNV9WNA)\n\n \n\n# **原理**\n\nSharding-JDBC 提供了精确分⽚扩展接⼝ org.apache.shardingsphere.api.sharding.standard.PreciseShardingAlgorithm. PreciseShardingAlgorithm，**只需要实现其doSharding ⽅法，基于分⽚键做⼀定的算法转换，返回指定的数据源，便可以达到分库的效果。** ⽬前我们使⽤的是⼀致性 hash 算法，⽤于定位分⽚键所在数据源，同时引⼊虚拟节点，可以避免数据倾斜问题（即有的数据源数据多，有的数据源数据少，导致数据源设备压⼒不均衡）算法原理如图： \n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=NWZkMDc1YmUxZTFmMzk5ZTg2ZThlYzhhM2NkZGU5MTRfa2ZzNDdsR3FTbnhJVTJaTENmRGFHMkZBMUN2S3hhN0pfVG9rZW46Ym94Y25qUWtZdWRYTmR0VG9SUHdtcTF5SFVnXzE2ODAwNzQ5MzU6MTY4MDA3ODUzNV9WNA)\n\n **然后基于该分库的分⽚键&分⽚算法，我们还需在 application.yml 中增加 sharding 数据源配置；同时需要增加分表相关的配置信息。**最后项⽬启动时，sharding 会⾃动加载这些配置初始化数据源。在请求时从请求参数中提取到分⽚键，应⽤分⽚算法，找到⽬标数据源，操作该数据源。 \n\n# **实战**\n\n在原有订单数据源 datasource_name 的基础上，新加3台订单数据库实例 datasource_name_1&datasource_name_2&datasource_name_3，扩充后共4台订单数据库实例，磁盘存储最⼤可以达到原有容量的4倍。\n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=NjUzZGZlZDA4MmIwNWE4MTRhN2Y1MWIyZWFkNDc3NTVfcUZRMWFIMFNQWnNxR0UxTUVmc21uSmttRElTMjNETnBfVG9rZW46Ym94Y25GZmVJWWphbTJBY3lTOTdCSkZKNURnXzE2ODAwNzQ5MzU6MTY4MDA3ODUzNV9WNA)\n\n**4.1引⼊maven依赖**\n\n```xml\n\n\torg.apache.shardingsphere\n\tsharding-jdbc-spring-boot-starter\n\t4.1.1.001\n\n```\n\n这⾥的4.1.1.001版本是在官⽹ sharding 4.1.1版本上⾃⾏扩充了⼀部分内容：本⾝根据业务分⽚键对业务数据进⾏分⽚后，同⼀类型的数据只会在⼀个数据源中，sharding 只能操作这⼀个数据源的数据，但对于⼀部分分⽚键落⼊新数据源后，意味着数据源发⽣了变动，⽽在旧库数据未迁⼊到新库时，会存在⼀定的兼容时期，此时，改造源码实现了如下双操作（除插⼊外）：a、双查询：查询新旧数据源的数据，然后业务项⽬⾥使⽤切⾯进⾏查询汇总去重，根据 id+updateDatetime 进⾏去重，保留最新数据。b、双更新：更新数据时，会同时更新新旧数据源中满⾜条件的数据。c、双删除：删除数据时，会同时删除新旧数据源中满⾜条件的数据。双删的⽬的是，为了避免只 删除了⼀个库的数据，⽽由于双查导致另⼀个库的数据被查询出来误⽤。d、单插⼊：插⼊数据时，只会插⼊新分⽚算法所落⼊的数据源中。这样做的好处是，在不影响业 务的情况下，也减少了⼀部分数据迁移的⼯作量，即分库上线后新数据直接插⼊新库，不⽤做任何处理。\n\n改造源码如下：\n\n```java\npackage org.apache.shardingsphere.core.rule;\n\n/**\n * Databases and tables sharding rule.\n */\n@Getter\npublic class ShardingRule implements BaseRule {\n\n // 定义：旧的分库分片策略\n private final ShardingStrategy oldDatabaseShardingStrategy;\n \n public ShardingRule(final ShardingRuleConfiguration shardingRuleConfig, final Collection dataSourceNames) {\n Preconditions.checkArgument(null != shardingRuleConfig, \"ShardingRuleConfig cannot be null.\");\n Preconditions.checkArgument(null != dataSourceNames && !dataSourceNames.isEmpty(), \"Data sources cannot be empty.\");\n this.ruleConfiguration = shardingRuleConfig;\n shardingDataSourceNames = new ShardingDataSourceNames(shardingRuleConfig, dataSourceNames);\n tableRules = createTableRules(shardingRuleConfig);\n broadcastTables = shardingRuleConfig.getBroadcastTables();\n bindingTableRules = createBindingTableRules(shardingRuleConfig.getBindingTableGroups());\n // 加载旧的分库分片策略\n ShardingStrategyConfiguration oldDatabaseShardingStrategyConfig = shardingRuleConfig.getOldDatabaseShardingStrategyConfig();\n if (oldDatabaseShardingStrategyConfig != null) {\n oldDatabaseShardingStrategy = createDefaultShardingStrategy(oldDatabaseShardingStrategyConfig);\n } else {\n oldDatabaseShardingStrategy = null;\n }\n defaultDatabaseShardingStrategy = createDefaultShardingStrategy(shardingRuleConfig.getDefaultDatabaseShardingStrategyConfig());\n defaultTableShardingStrategy = createDefaultShardingStrategy(shardingRuleConfig.getDefaultTableShardingStrategyConfig());\n defaultShardingKeyGenerator = createDefaultKeyGenerator(shardingRuleConfig.getDefaultKeyGeneratorConfig());\n masterSlaveRules = createMasterSlaveRules(shardingRuleConfig.getMasterSlaveRuleConfigs());\n encryptRule = createEncryptRule(shardingRuleConfig.getEncryptRuleConfig());\n }\n}\n```\n\n\n\n```java\npackage org.apache.shardingsphere.sharding.route.engine;\n\n/**\n * Sharding route decorator.\n */\npublic final class ShardingRouteDecorator implements RouteDecorator {\n \n @SuppressWarnings(\"unchecked\")\n @Override\n public RouteContext decorate(final RouteContext routeContext, final ShardingSphereMetaData metaData, final ShardingRule shardingRule, final ConfigurationProperties properties) {\n SQLStatementContext sqlStatementContext = routeContext.getSqlStatementContext();\n // SQL参数集合\n List parameters = routeContext.getParameters();\n ShardingStatementValidatorFactory.newInstance(\n sqlStatementContext.getSqlStatement()).ifPresent(validator -> validator.validate(shardingRule, sqlStatementContext.getSqlStatement(), parameters));\n // 获取分片条件\n ShardingConditions shardingConditions = getShardingConditions(parameters, sqlStatementContext, metaData.getSchema(), shardingRule);\n boolean needMergeShardingValues = isNeedMergeShardingValues(sqlStatementContext, shardingRule);\n if (sqlStatementContext.getSqlStatement() instanceof DMLStatement && needMergeShardingValues) {\n checkSubqueryShardingValues(sqlStatementContext, shardingRule, shardingConditions);\n mergeShardingConditions(shardingConditions);\n }\n ShardingRouteEngine shardingRouteEngine = ShardingRouteEngineFactory.newInstance(shardingRule, metaData, sqlStatementContext, shardingConditions, properties);\n RouteResult routeResult = null;\n // ShardingStandardRoutingEngine特殊处理\n if(shardingRouteEngine instanceof ShardingStandardRoutingEngine) {\n SQLStatement sqlStatement = sqlStatementContext.getSqlStatement();\n // 查询/更新/删除操作，开启双操作模式\n boolean isCompatibleMode = (sqlStatement instanceof SelectStatement) || (sqlStatement instanceof UpdateStatement) || (sqlStatement instanceof DeleteStatement);\n routeResult = shardingRouteEngine.route(shardingRule, isCompatibleMode);\n } else {\n routeResult = shardingRouteEngine.route(shardingRule);\n }\n if (needMergeShardingValues) {\n Preconditions.checkState(1 == routeResult.getRouteUnits().size(), \"Must have one sharding with subquery.\");\n }\n return new RouteContext(sqlStatementContext, parameters, routeResult);\n }\n}\n```\n\n\n\n```java\npackage org.apache.shardingsphere.sharding.route.engine.type.standard;\n\n/**\n * Sharding standard routing engine.\n */\npublic final class ShardingStandardRoutingEngine extends BaseShardingRoutingEngine {\n\n /**\n * 寻找sharding路由结点信息\n * @param shardingRule\n * @param tableRule\n * @param databaseShardingValues\n * @param tableShardingValues\n * @param isCompatibleMode 兼容模式（即同时操作新旧分片算法定位到的数据库，查询/更新/删除自动开启兼容模式）\n * @return\n */\n private Collection route0(final ShardingRule shardingRule, final TableRule tableRule, final List databaseShardingValues, final List tableShardingValues,\n final boolean isCompatibleMode) {\n Collection result = new LinkedList<>();\n // 双查 & 双更 & 双删核心逻辑\n // 1. 加载旧数据节点路由信息\n ShardingStrategy oldDatabaseShardingStrategy = shardingRule.getOldDatabaseShardingStrategy();\n Collection oldDataSources = null;\n if (oldDatabaseShardingStrategy != null && isCompatibleMode) {\n oldDataSources = routeDataSources(shardingRule, tableRule, databaseShardingValues, true);\n for (String each : oldDataSources) {\n result.addAll(routeTables(shardingRule, tableRule, each, tableShardingValues, true));\n }\n }\n // 2. 加载新数据节点路由信息\n Collection routedDataSources = routeDataSources(shardingRule, tableRule, databaseShardingValues, false);\n for (String each : routedDataSources) {\n // 新数据节点未被加载时，加载新数据节点\n if(oldDataSources == null || !oldDataSources.contains(each)) {\n result.addAll(routeTables(shardingRule, tableRule, each, tableShardingValues, false));\n }\n }\n return result;\n }\n \n private Collection routeDataSources(final ShardingRule shardingRule, final TableRule tableRule, final List databaseShardingValues, final boolean isOld) {\n if (databaseShardingValues.isEmpty()) {\n return tableRule.getActualDatasourceNames();\n }\n // 根据isOld参数，区分加载新/旧分片策略\n Collection result = new LinkedHashSet<>();\n if (isOld) {\n result.addAll(shardingRule.getOldDatabaseShardingStrategy().doSharding(tableRule.getActualDatasourceNames(), databaseShardingValues, this.properties));\n } else {\n result.addAll(shardingRule.getDatabaseShardingStrategy(tableRule).doSharding(tableRule.getActualDatasourceNames(), databaseShardingValues, this.properties));\n }\n Preconditions.checkState(!result.isEmpty(), \"no database route info\");\n Preconditions.checkState(tableRule.getActualDatasourceNames().containsAll(result), \n \"Some routed data sources do not belong to configured data sources. routed data sources: `%s`, configured data sources: `%s`\", result, tableRule.getActualDatasourceNames());\n return result;\n }\n\n}\n```\n\n **4.2数据分片**基于业务进⾏分库的分⽚键选择，最好能均匀的将数据拆分到不同数据库实例。本⽅案假设选择⽤⼾ID作为分库的分⽚键，然后⾃定义新的分库分⽚算法，**算法命名为 DatasourceShardingAlgorithmV2**，算法内容如下：\n\n```java\nimport org.apache.shardingsphere.api.sharding.standard.PreciseShardingAlgorithm;\nimport org.apache.shardingsphere.api.sharding.standard.PreciseShardingValue;\nimport java.nio.ByteBuffer;\nimport java.nio.ByteOrder;\nimport java.util.*;\n\n/**\n * 新的分库数据分片算法\n *\n * @author jingmin.yang\n * @date 2022/4/18 19:23\n */\npublic class DatasourceShardingAlgorithmV2 implements PreciseShardingAlgorithm {\n // 排序存储结构：SortedMap<虚拟节点，物理节点>\n private static final SortedMap virtualToRealMap = new TreeMap();\n\n /**\n * 基于所有数据源初始化订单分库虚拟节点\n * @return\n */\n static {\n List datasources = new ArrayList();\n datasources.add(\"datasource-name\");\n datasources.add(\"datasource-name-1\");\n datasources.add(\"datasource-name-2\");\n datasources.add(\"datasource-name-3\");\n // 初始化分库虚拟节点数据\n init(virtualToRealMap, datasources);\n }\n\n @Override\n public String doSharding(Collection availableTargetNames, PreciseShardingValue preciseShardingValue) {\n // shardingKey即为分片键值\n String shardingKey = preciseShardingValue.getValue();\n return getShardingDatasource(virtualToRealMap, shardingKey);\n }\n\n /**\n * 获取分库物理节点\n * @param virtualToRealMap 分库虚拟节点\n * @param shardingKey 分库的分片键标识\n * @author: jingmin.yang\n * @create: 2021/9/10 10:41\n * @return\n */\n static String getShardingDatasource(SortedMap virtualToRealMap, String shardingKey) {\n int hashValue = hash(shardingKey);\n // 顺时针找最近的虚拟节点\n SortedMap subVirtualToRealMap = virtualToRealMap.tailMap(hashValue);\n if (subVirtualToRealMap.isEmpty()) {\n return virtualToRealMap.get(virtualToRealMap.firstKey());\n }\n return subVirtualToRealMap.get(subVirtualToRealMap.firstKey());\n }\n\n /**\n * 初始化hash环\n *\n * @param: virtualToRealMap\n * @param: realNodes\n * @author: jingmin.yang\n * @create: 2021/9/14 16:29\n */\n static void init(SortedMap virtualToRealMap, List realNodes) {\n for(String node: realNodes) {\n virtualToRealMap.put(hash(node), node);\n\n // 虚拟出指定数量的虚拟节点\n int count = 0;\n int i = 0;\n int virtualNum = realNodes.size();\n while (count < virtualNum) {\n i++;\n String virtualNode = node + \"#\" + i;\n // 计算hash值\n int hashValue = hash(virtualNode);\n if (!virtualToRealMap.containsKey(hashValue)) {\n // 虚拟节点放到环上去 数据结构：红黑树\n virtualToRealMap.put(hashValue, node);\n count++;\n }\n }\n }\n }\n\n /**\n *\n * MurmurHash是一种非加密型哈希函数，适用于一般的哈希检索操作,与其它流行的哈希函数相比，对于规律\n * 性较强的key，MurmurHash的随机分布（离散性）特征表现更好，而且具有低碰撞率。Redis对节点进行\n * shard时采用的是这种算法。\n *\n * @param: key 分片键\n * @author: jingmin.yang\n * @create: 2021/9/14 19:17\n */\n static int hash(String key) {\n ByteBuffer buf = ByteBuffer.wrap(key.getBytes());\n int seed = 305441741;\n ByteOrder byteOrder = buf.order();\n buf.order(ByteOrder.LITTLE_ENDIAN);\n int m = 1540483477;\n byte r = 24;\n\n int h;\n int k;\n for (h = seed ^ buf.remaining(); buf.remaining() >= 4; h ^= k) {\n k = buf.getInt();\n k *= m;\n k ^= k >>> r;\n k *= m;\n h *= m;\n }\n\n if (buf.remaining() > 0) {\n ByteBuffer finish = ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN);\n finish.put(buf).rewind();\n h ^= finish.getInt();\n h *= m;\n }\n\n h ^= h >>> 13;\n h *= m;\n h ^= h >>> 15;\n buf.order(byteOrder);\n if (h < 0) {\n h = Math.abs(h);\n }\n return h;\n }\n}\n```\n\n测试： \n\n```java\npublic static void main(String[] args) { \n System.out.println(DatasourceShardingAlgorithmV2.getShardingDatasource(virtualToRealMap, \"U0000000001\"));\n}\n```\n\n结果：\n\n```Plaintext\ndatasource-name-1\n```\n\n **4.3规则配置**Sharding-JDBC 可以通过 Java, YAML, Spring 命名空间 和 Spring Boot Starter 这 4种⽅式进⾏配置，开发者可根据场景选择适合的配置⽅式。当前⽅案使⽤ YAML ⽅式进⾏规则配置，application.ym 内容如下（**红⾊部分为⾃⾏扩展内容**）： \n\n```yaml\nspring: \n shardingsphere:\n # sharding数据源配置\n datasource:\n # 对应下方自定义的数据源名称\n names: datasource-name,datasource-name-1,datasource-name-2,datasource-name-3\n # 数据源基础配置\n datasource-name:\n # 数据源驱动类型\n type: com.alibaba.druid.pool.DruidDataSource\n # 数据库驱动类\n driver-class-name: com.mysql.jdbc.Driver\n # 数据库连接地址\n url: jdbc:mysql://127.0.0.1:3306/datasource_name_0\n # 数据库用户名\n username: username\n # 数据库用户密码\n password: password\n # 自定义第二个数据源（新数据源1）\n datasource-name-1:\n type: com.alibaba.druid.pool.DruidDataSource\n driver-class-name: com.mysql.jdbc.Driver\n url: jdbc:mysql://127.0.0.1:3306/datasource_name_1\n username: username\n password: password\n # 自定义第三个数据源（新数据源2）\n datasource-name-2:\n type: com.alibaba.druid.pool.DruidDataSource\n driver-class-name: com.mysql.jdbc.Driver\n url: jdbc:mysql://127.0.0.1:3306/datasource_name_2\n username: username\n password: password\n # 自定义第四个数据源（新数据源3）\n datasource-name-3:\n type: com.alibaba.druid.pool.DruidDataSource\n driver-class-name: com.mysql.jdbc.Driver\n url: jdbc:mysql://127.0.0.1:3306/datasource_name_3\n username: username\n password: password\n # 采用sharding-jdbc进行分库分表设计\n sharding:\n # 默认数据源\n default-data-source-name: datasource-name\n # oldDatabaseStrategy是自行扩展配置（可选），用于双操作（双查询&双更新&双删除）中获取原有数据源，pom需要对应引入4.1.1.001自行扩展版本\n oldDatabaseStrategy:\n standard:\n # 分库分片键\n shardingColumn: sharding_key\n # 原有的数据源分片算法，本算法返回的是固定的原有数据源datasource-name\n preciseAlgorithmClassName: com.yang.sharding.DatasourceShardingAlgorithm\n # 订单分库新的分片算法，根据sharding_key进行分库\n default-database-strategy:\n standard:\n # 分库分片键\n shardingColumn: sharding_key\n # 新的分库分片算法\n preciseAlgorithmClassName: com.yang.sharding.DatasourceShardingAlgorithmV2\n # 分库分表配置，只有分库&分表的表需要在此配置，不分库不分表不需要配置，数据会存入默认数据源\n tables:\n # table_0使用分库+分表\n table_0:\n actual-data-nodes: datasource-name.table_0_$->{0..63},datasource-name-$->{1..3}.table_0_$->{0..63}\n logic-table: table_0\n # 数据库表策略配置\n table-strategy:\n standard:\n # 表分片键\n shardingColumn: sharding_key\n # 表分片算法\n preciseAlgorithmClassName: com.yang.TableShardingAlgorithm\n # table_1使用分库，不使用分表\n table_1:\n actual-data-nodes: datasource-name.table_1, datasource-name-$->{1..3}.table_1\n logic-table: table_1\n```\n\n**4.4代码改造准备**由于 sharding 是对分⽚键使⽤分⽚算法，获取到对应的数据源进⾏操作，⽽分⽚键是从请求 sql 中提取⽽来，所以需要在代码层⾯改造⽀持分库，所有 CRUD 操作必须要带上分⽚键参数。\n\n```sql\n\n\nINSERT INTO order_info('id', 'user_id', 'update_time') VALUES('O0000000001', 'U0000000001', NOW());\n\nSELECT * FROM order_info WHERE user_id='U0000000001';\n\nUPDATE order_info SET update_time=NOW() WHERE user_id='U0000000001';\n\nDELETE FROM order_info WHERE user_id='U0000000001';\n```\n\n \n\n# **总结**\n\n**⾄此，基于 Sharding-JDBC 完整的订单分库改造完成，包含选择分库分⽚键、定义分库数据分⽚算法，添加数据源&表配置，代码改造等。**本⽅案已有实际应⽤案例，如有需要的⼩伙伴可作为参考，有问题也可直接联系我，⼀起讨论学习。","date":"2022-04-24","author":"杨敬敏 Ginee后端工程师","excerpt":"本⽅案基于Apache开源项⽬ ShardingSphere 采 Sharding-JDBC 进⾏订单⽔平分库设计（数据分⽚）缓解存储压⼒。","metaDescription":null,"createdAt":"2023-03-29T07:36:03.794Z","updatedAt":"2023-03-29T07:36:04.607Z","publishedAt":"2023-03-29T07:36:04.603Z","locale":"zh","caseStudiesType":{"data":null},"cover":{"data":null},"metaLogo":{"data":null},"localizations":{"data":[]}}},"prev":{"id":41,"attributes":{"feature":true,"title":"喜马拉雅基于Apache ShardingSphere实践","content":"# **背景** \n\n \n\n喜马拉雅成立之初，各个业务管理各自的数据库、缓存，个业务都要了解中间件的各种部署情况，导致业务间的合作，需要运维、开发等方面的人工介入，效率较低，扩展困难，安全风险也很高，资源利用率也不高。喜马拉雅在发展中，逐渐意识到需要在公司层面，提供统一的定制化的数据访问平台的重要性。为此，我们推出了自己的PaaS化平台，PaaS化就是对资源的使用做了统一的入口，业务只需要申请一个资源ID，就能使用数据库，达到对资源使用的全部系统化，其中对数据库的访问我们基于 Apache ShardingSphere 来实现，并基于 Apache ShardingSphere 强大功能做些优化和增强。\n\n \n\n# **整体架构**\n\n \n\n我们PaaS平台建设中，负责和数据层通信的dal层中间件我们叫Arena,其中对数据库的访问我们叫Arean-Jdbc\n\nArena-Jdbc 层的能力基本是基于 Apache ShardingSphere 的能力建设，我们只是基于喜马拉雅需要的特性做了增强和优化,整体架构如下：\n\n \n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=NmU5ZDNhMjRlNDczZjdiYzg3NTE0Yzg4OTc5N2Q0M2ZfUzg3c3FCbkozTG0zUzZDR1dFb1dHT29Od3Nod0VWQmRfVG9rZW46Ym94Y25UY1VJQlcxeG1qZ2RVU01aUlIwQjdmXzE2ODAwNzcwNDk6MTY4MDA4MDY0OV9WNA)\n\n \n\n# **Pull Frame**\n\n \n\nConsul Pull Frame 是我们对Consul 的配置自动拉起封装为统一的Pull框架，我们除了数据库，还有缓存，每种还有不同的使用方式，我们对不同的使用方式只需要实现对应的实现类和初始化，更新，切好这些接口就行，框架会统一把解析好的数据给到，具体一种场景不需要关心和Consul的交互，为后面的资源PaaS化提供了简单的接入能力。\n\n \n\n# **故障容灾**\n\n \n\n***· 自动重连***\n\n我们对故障容灾在设计时就考虑了平时通用的一些故障场景，比如数据库server 挂了，我们最做自动重链，不需要业务做重启操作。\n\n \n\n***· 本地快照***\n\n本地快照是为了防止Consul不可用时，业务不能启动，所以我们在拉到远程配置后，会本地存储一份，在拉配置时，如果远程失败，就用本地的配置，保证Consul挂了，不影响业务，每次拉到新的配置时，会更新本地的快照。\n\n \n\n***· 灰度更新***\n\n灰度更新是为了支持配置变更时找灰度的逻辑，对于数据库层面的变更，是非常危险的，如果一下就全量变更，有可能会触发线上事故，所以通过灰度变更的机制，业务可以先选择一个容器实例来变更，没有问题后，再全量变更，把风险降到最低。\n\n \n\n***· 密码安全***\n\n没有PaaS化之前，我们的数据库密码都是dba统一管理的，但PaaS化后，访问数据库的密码就存在配置文件中，如果明文，就太不安全，所以我们对密码统一做了加密处理，在 Arean-Jdbc 层统一做解密，确保密码不回泄露出去。\n\n \n\n# **统一数据源**\n\n \n\n为了让业务做最低成本的改造，Arena-jdbc 需要提供一个统一的数据源，不论上层用什么框架，不影响,业务只需要替换数据源接入即可，对于数据库连接池我们默认使用 HikariCP DataSource 也支持个性化的业务，业务可以通过配置指定连接池。\n\n我们基于Apache ShardingSphere 的连接池封装了一个我们自己的DataSource,我们叫ArenaDataSource，通过ArenaDataSource封装了各种不同场景聚合，的使用一个 ArenaDataSource 支持三种使用方式：\n\n1. 支持原生直接连接\n\n2. 支持Proxy模式，也是Apache ShardingSphere的proxy\n\n3. 支持直接连接分库分表\n\n业务只需要一个 DataSource，即支持分库分表，也支持简单的直接连接的模式，这样的好处是业务以后要分库分表，就不再需要升级中间件了，为了彻底解决业务升级的成本，我们做了配置自动升级，就是你之前是简单直接链接使用，为了PaaS化，后来业务发展了，需要分库分表了，以及从分库分表需要多活部署了，这些都不要再升级依赖了，只需要配置动即可。\n\n\n\n# **资源动态变更**\n\n \n\n资源动态变更是PaaS平台基本的能力，接入PaaS后，业务修改数据库的任何属性，都不再需要业务方代码变更，重新发布 \n\nApache ShardingSphere 也支持数据库属性的动态变更，我们基于自己的内部系统的特征，实现了基于Consul的资源变根通知。我们的资源存在Consul。\n\nArena-Jdbc 支持对使用的资源做无损的变更，Arena-Jdbc 收到资源变更时，会先对新下发的资源做预热处理，预热后，再切换使用的数据源，切换成功后，再销毁老的数据源，业务无感知。\n\n如果新的资源预热失败，则不会做变更处理，保证下发的资源时可用的，规避错误下发的问题。\n\n扩容和缩容也是同理，一期数据需要运维手动迁移，迁移好了后，直接在PaaS平台下发新的配置即可，二期支持自动迁移数据和配置变更结合。\n\n同时支持Proxy的无损上下线机制，通过PaaS平台对Proxy的变更，把需要下线的Proxy节点去掉，通知 Arena-Jdbc，Arena-Jdbc 会把缩容的Proxy节点去掉，做到无损下线。\n\n \n\n# **读写分离**\n\n \n\n读写分离我们完全基于Apache ShardingSphere的来实现，我们根据喜马拉雅业务的特性，对强制路由做了增强，不需要规则配置为 Hint 模式，只要线程上下文带有强制路由的标志，就可以路由到指定的库和表，不受分表规则的影响，我们重写了ShardingStandardRoutingEngine的Sharding时路由库和表的逻辑：\n\n```java\nprivate Collection routeDataSources(final TableRule tableRule, final ShardingStrategy databaseShardingStrategy, final List databaseShardingValues) {\n //先判断是否存在Hint上下文路由标，如果有,则优先根据用户指定的规则路由库\n Collection> databaseShardings = HintManager.getDatabaseShardingValues(tableRule.getLogicTable());\n if (databaseShardings != null && databaseShardings.size() > 0) {\n List list = new ArrayList<>(4);\n for (Comparable databaseSharding : databaseShardings) {\n list.add((String) databaseSharding);\n }\n if (log.isDebugEnabled()) {\n log.debug(\"route dataSources, find HintManager, so hint to: {}\", list);\n }\n return list;\n }\n //没有Hint路由规则，则按Sharding 规则路由\n if (databaseShardingValues.isEmpty()) {\n return tableRule.getActualDatasourceNames();\n }\n Collection result = new LinkedHashSet<>(databaseShardingStrategy.doSharding(tableRule.getActualDatasourceNames(), databaseShardingValues, properties));\n Preconditions.checkState(!result.isEmpty(), \"no database route info\");\n Preconditions.checkState(tableRule.getActualDatasourceNames().containsAll(result), \n \"Some routed data sources do not belong to configured data sources. routed data sources: `%s`, configured data sources: `%s`\", result, tableRule.getActualDatasourceNames());\n return result;\n }\n```\n\n \n\n路由表也是同样的逻辑，我们重写了ShardingStandardRoutingEngine的routeTables方法，和上面一样。先从Hint的上下文获取。这样通过上下文的方式能很好的满足业务个性化的路由规则，能和Sharding规则共存。\n\n \n\n# **Database Plus**\n\n \n\nApache ShardingSphere 除了提供基本的分库分表，读写分离的能力外，在上层还提供了很多的插件和扩展的机制，这让我们在基于数据库提供更偏向业务的起的能力非常容易，成本非常低,这叫 Database Plus\n\nDatabase Plus 简单的说就是你用 Apache ShardingSphere 的数据库中间件，不仅仅是提供了分库分表这一基本能力，通过对底层数据的封装为统一的交互标准插件模式，可以在上面实现很多业务的通用的场景的需求，比如喜马拉雅除了用到 Apache ShardingSphere 基础的能力外，我们也享受了 Database Plus 的威力，我们在它的基础上轻松实现了支持压测的影子库和影子表，数据加解密，机房级别容灾的同城双读，分布式唯一ID。\n\n \n\n# **影子库和影子表**\n\n \n\n影子库影子表我们对Apache ShardingSphere做了改动，Apache ShardingSphere需要修改sql，我们认为对业务有改造成本，同时结合我们自己的压测平台，我们和业界一样，我们也实现了影子标记，通过全链路压测标的传递来判断是否路由到影子库/影子表，业务无需任何改造，即可使用影子库影子表来做压测，同时不需要在运行时对sql改写，提升了性能，我们重写了ShadowSQLRouter,\n\n```java\npublic class ArenaShadowSQLRouter extends ShadowSQLRouter {\n @Override\n public boolean isShadow(final SQLStatementContext sqlStatementContext, final List parameters, final ShadowRule rule) {\n if (sqlStatementContext instanceof InsertStatementContext || sqlStatementContext instanceof WhereAvailable\n || sqlStatementContext instanceof UpdateStatementContext) {\n //这里就是判断是否有压测标，如果有，ShardingSphere则会找影子的逻辑。\n return ArenaUtilities.checkPeakRequest();\n }\n return false;\n }\n}\n```\n\n \n\n通过spi的方式把我们自定义的ArenaShadowSQLRouter给Apache ShardingSphere加载使用，不得不说Apache ShardingSphere的插件设计很赞，很方便自定义和扩展。\n\n配置还是和Apache ShardingSphere的一样：\n\n```yaml\n# 配置影子库规则\n- !SHADOW\n # true-影子表，false-影子库(默认)\n enableShadowTable: true \n\n # 源库名称(对应DataSources数据源配置中的名称)，影子库才需要配，影子表不需要配置\n sourceDataSourceNames: \n - ds0 # 源库，与影子库shadow_ds0对应\n - ds1 # 源库，与影子库shadow_ds1对应\n\n # 影子库名称(对应dataSources数据源配置中的名称)，影子库才需要配，影子表不需要配置\n shadowDataSourceNames: \n - shadow_ds0 # 影子库，与源库ds0对应\n - shadow_ds1 # 影子库，与源库ds1对应\n```\n\n \n\nenableShadowTable 我们新增了该属性，来确定是使用影子库还是影子表。\n\n \n\n# **影子库**\n\n \n\n影子库，一定要填sourceDataSourceNames和shadowDataSourceNames，enableShadowTable不用设置，或者设置为false。\n\n \n\nsourceDataSourceNames\n\n按顺序映射，一一对应：\n\nds --> shadow_ds\n\nds1--> shadow_ds1\n\n影子库/影子表是最后一个路由规则，如果发现有影子库/影子表，则根据实际的库找到对应的影子库/影子表，执行sql\n\n \n\n# **同城多活**\n\n \n\n基于喜马拉雅的业务特性，读多写少，我们只实现了对读业务的双机房部署，写业务还是路由到主机房。\n\n为了增强容灾能力，喜马拉雅搭建了双机房，同时承载业务流量，当一个机房故障时，可以把流量快速切换到另一个机房，我们在dal层设计上支持双写，这里充分利用了 Apache ShardingSphere 的读写分离功能，读和写可以配置独立的数据源，我们只需要在上面做了一层封装，在切换时候动态变更对应的数据源即可，为了切换时不影响业务的流量，我们是先预热新的数据源，再销毁老的数据源。\n\n架构图如下：\n\n \n\n![img](https://u01f1kqxrl.feishu.cn/space/api/box/stream/download/asynccode/?code=ODgzZmMyZDliZDcwMTViYTIzYmE5MGE1MWE4YWJjNjBfQVU2NXhzWkIxTzdwZTJqT3MxNVNuRmdENUNqUExhRHNfVG9rZW46Ym94Y250TkVLaXJ1dzJZSlNnTDdlMkJOWk9lXzE2ODAwNzcwNDk6MTY4MDA4MDY0OV9WNA)\n\n \n\n另外我们也对双写做了研究和探索，关键在于数据库的双向同步，基于阿里开源的 otter 做了改造，支持基于 gtid 模式同步，不依赖打标，打标回有性能开销，在一些业务做了试用。\n\n \n\n# **分布式唯一ID**\n\n \n\n分库分表后，唯一id是必须要满足的需求，Apache ShardingSphere默认提供了snowfake和uuid算法，但不是很时候db的场景，db需要保证顺序和格式，所以我们基于Apache ShardingSphere提供的接口，也实现自己的唯一id生成策略：数据分片后，不同Mysql实例生成全局唯一主键是非常棘手的问题。Arena-Jdbc 实现了Apache ShardingSphere的分布式主键生成器接口，通过集成喜马拉雅内部的全局唯一id生成服务，提供了适用于喜马拉雅内部的自增主键生成算法-BoushId主键生成策略。\n\n \n\n# **监控和报警**\n\n \n\n做一个数据库中间件，监控是必不可少的部分，就像我们的眼睛，没有监控就是瞎抓，以及对异常情况的报警也是非常重要的部分，只有完善的监控和报警才能算是一个完整的产品，得益于 Apache ShardingSphere 在设计时就提供了钩子，我们能非常小的成本就能实现对sql层面的监控和报警。\n\n \n\nArena-Jdbc 客户端通过钩子回调，从多维度数据来分析使用数据库的运行情况，以30s为一次统计周期，每个周期统计的数据包括：Mysql总请求量，新增、删除、修改和查询的请求量，失败的请求量和慢请求量，影子库的流量，以及统计响应时间的TP百分比，还有连接池的等待时间、建连时间、连接数等信息。这些指标会发送给专门的收集指标服务，并持久化到时序数据库，PaaS平台可以从时序数据库中查询数据，展示给各个业务，对于异常sql和慢sql，做报警等后续处理。\n\n \n\n# **其他**\n\n \n\n我们除了基于Apache ShardingSphere实现上述关键特性外，我们还对Apache ShardingSphere做了一些优化和改进，以更适合喜马拉雅的业务。\n\n \n\n· 优化分片规则，启动时，如果分片的真实表不存在的情况则报错，将配置错误前置\n\n \n\n· 有的业务方有几百，甚至几千的分表，这种情况下，由于Apache ShardingSphere中的联邦查询需要依次扫表，启动速度很慢，达到了分钟级别。针对这种情况，我们新增了props配置项，不再初始化联邦查询，打打加快了启动速度，并且再使用中，也没有用联邦查询\n\n \n\n· 优化了Apache ShardingSphere，执行sql异常不报错误的情况\n\n \n\n· 由于有的业务方，对重要的表采用了大写的表名和列名，我们去掉Apache ShardingSphere中，对配置中的大写的表名列名强制小写的情况，允许大写的表名和列名\n\n \n\n· 新增了props配置项，可以调节Apache ShardingSphere的编译缓存的大小\n\n \n\n· 优化Apache ShardingSphere复合分片算法，精确匹配分片字段\n\n \n\n在ComplexShardingStrategyConfiguration中，添加shardingColumnList字段：修复Apache ShardingSphere，批量insert，不返回主键的问题，这个问题在mybatis-plus中比较常见\n\n \n\n· 不分片的表，支持使用默认的主键id生成策略\n\n \n\n# **总结**\n\n \n\n基于Apache ShardingSphere实现的数据库中间件Arena-Jdbc，经过半年的时间，已经覆盖了喜马拉雅的70%的核心业务，目前没有发现任何问题，表现的非常稳定，通过和我们的PaaS平台结合，业务也非常愿意接入，另外我们使用Apache ShardingSphere时，社区还没有发布stable的版本，所以我们在使用过程中也遇到了些问题，基本上我们都解决了，有的社区也有对应的解决方案，得益于社区非常活跃，我们以后也希望把我们做的一些feature能回馈到社区，为Apache ShardingSphere的发展做出一点点小贡献。\n\n非常感谢基础架构团队胡建华、彭荣新提出的宝贵意见和建议，感谢喜马拉雅基础小伙伴们在项目推广过程中的大力支持，让Apache ShardingSphere在喜马拉雅生根发芽。\n\n非常感谢亮哥亲自来喜马拉雅对Apache ShardingSphere的技术内幕和规则做了一次全面的分享，非常关心我们在使用Apache ShardingSphere过程中遇到的问题，在现场对小伙伴提的问题都一一作了 深入的解答，非常感谢亮哥，祝Apache ShardingSphere越来越好。","date":"2022-05-11","author":"沈辉 基础架构","excerpt":"基于Apache ShardingSphere实现的数据库中间件Arena-Jdbc，经过半年的时间，已经覆盖了喜马拉雅的70%的核心业务，目前没有发现任何问题，表现的非常稳定，通过和我们的PaaS平台结合，业务也非常愿意接入","metaDescription":null,"createdAt":"2023-03-29T08:09:25.219Z","updatedAt":"2023-03-29T08:09:26.226Z","publishedAt":"2023-03-29T08:09:26.216Z","locale":"zh","caseStudiesType":{"data":null},"cover":{"data":null},"metaLogo":{"data":null},"localizations":{"data":[]}}},"article":{"id":13,"attributes":{"feature":true,"title":"ShardingSphere 助力当当 WMS：订单效率提升 30%、节约成本上千万","content":">Apache ShardingSphere 简单易用且成熟稳定，当当自研 WMS 如虎添翼，在供应链系统改革中起到了至关重要的作用，是我们的杀手锏。\n——李勇，当当 WMS 技术负责人\n\n>Apache ShardingSphere 5.0.0 GA 已于 2021 年 11 月发布，5.1.0 版本也于上周发布。历经两年的迭代与打磨，可插拔生态趋近成熟，标志着 Apache ShardingSphere 从分库分表中间件蜕变为以 Database Plus 为理念的分布式数据库生态系统。\n\n近年来，随着“建设社会主义文化强国”政策的持续落实，全民阅读在移动互联快速发展中得到大力倡导，阅读给当代人提供了丰富的精神食粮。当当成立于 1999 年底，作为新技术与传统行业的结合体，当当融合了互联网新技术和传统图书行业，二十余年深耕图书领域助力全民阅读。\n\n当当在刚成立的几年，也是中国互联网腾飞的阶段，电商逐渐多元化发展的趋势迎来了行业野蛮生长期，各电商平台的发展如逆水行舟，不进则退。当当对战略和管理等方面进行调整的同时，也对技术架构层进行了硬核升级。\n\n当当成立之初的仓储物流系统并非自研，随着业务体量、技术实力等多方面的变化，为了满足业务快速发展需要，当当决定重构 WMS（Warehouse Management System）和 TMS（Transportation Management System）。硬件层使用 x86 取代小机，数据层由传统集中式转型为更灵活的分布式架构。如何解决海量仓储货品信息问题，是重构待解决的核心问题之一。互联网技术栈比较常用的分库分表被率先尝试使用，但由于当时开源数据库中间件的成熟度及功能方面不满足预期，当当决定自研分库分表产品，Sharding-JDBC 就这样应运而生，为“连接”带来更多可能。\n\n**自研 WMS 的上线完成了仓储层智能改革，五年、十余仓，Apache ShardingSphere 同 WMS 完美支撑了数个“读书节”、“618”和“11.11”等大型促销活动。**\n\n\n## 业务挑战\n\n当当在自研 WMS 之前使用的是第三方仓储管理系统，其数据库是基于 IBM 小机的 Oracle，结合当当业务体量以及电商仓储物流在促销期间密集请求的业务特点，传统集中式数据库架构给 WMS 带来了极大的挑战。\n\n在业务快速发展的冲击下，数据库计算和存储能力的上限越发明显，Scale-up 方案难以满足超级促销期间的业务需求，技术团队需要对业务层多次做拆分调整来缓解“存算”瓶颈，产能存在受限的风险。\n\n- 计算、存储能力双受限\n\n传统集中式架构扩展能力相对较差，数据库的计算和存储能力成为明显瓶颈；\n\n- 业务开发维护成本高昂\n\n在扩展能力相对差的基础上，业务侧需要不断的去让步做拆分，开发和维护成本持续攀升；\n\n- 系统架构开放度不足\n\n系统架构开放度不足将会使系统面临灵活度欠佳、功能缺失以及转型难度高等问题，基于现有架构难以快速响应新业务的开展及相关政策的调整，如云原生、SQL 审计、数据脱敏以及分布式治理等多方面需求。\n\n\n## 解决方案\n\n当当技术部结合现状及未来发展等多维度的调研，最终确定了自研 WMS 的方案。在硬件方面全面采用通用的 x86 设备替换 IBM 小型机，数据库层面引入开源数据库 MySQL 来替换 Oracle。当时已有多个开源数据库中间件，但成熟度和功能方面与预期不符，因此选择了自研数据库中间件，Sharding-JDBC 闪亮登场。\n\nShardingSphere-JDBC 定位为轻量级 Java 框架，在 Java 的 JDBC 层提供额外服务。具备轻量、高效、易用以及高兼容等特点。它使用客户端直连数据库，以 jar 包形式提供服务，无需额外部署和依赖，可理解为增强版的 JDBC 驱动，完全兼容 JDBC 和各种 ORM 框架。\n\n- 适用于任何基于 JDBC 的 ORM 框架，如：JPA、Hibernate、Mybatis、Spring JDBC Template 或直接使用 JDBC； \n\n- 支持任何第三方的数据库连接池，如：DBCP、C3P0、BoneCP、HikariCP 等；\n\n- 支持任意实现 JDBC 规范的数据库，目前支持 MySQL、PostgreSQL、Oracle、SQL Server 以及任何可使用 JDBC 访问的数据库。\n\n![当当1.png](https://official-media.sphere-ex.com/1_16c3bbe14b.png)\n\nShardingSphere-JDBC 方案\n\n目前 Apache ShardingSphere 由 JDBC、Proxy 和 Sidecar（规划中）这 3 款既能够独立部署，又支持混合部署配合使用的产品组成，在当当自研 WMS 中使用的是 ShardingSphere-JDBC。\n\n那么 ShardingSphere-JDBC 在 WMS 中具体是如何应用的呢？\n\n在 WMS 系统中，城市的“仓”就是仓储系统的管理单位，每个仓有对应的业务系统和数据库，单个仓使用 3 套 MySQL 主从集群承载指定城市仓库的数据。目前，当当在全国各地设有十余个自建的仓库，主要分布在订单量较大的城市，这种自建仓库模式在具备灵活管理等特性的同时，也可以降低企业的长期仓储成本。\n\n在架构规划上，WMS 通过 ShardingSphere-JDBC 按业务类型进行分库， 每套集群负责存储指定的业务数据，单个仓的 3 套 MySQL 集群分工如下：\n\n- 基础库集群：User，Area，Menu；\n\n- 业务库集群：Order，Package；\n\n- 库存库集群：Stock，Working。\n\n上线前，系统会根据仓库的基础数据进行初始化，如货位等信息，这里不做深入讨论。\n\n分布式数据库中间件的架构解决了 WMS 存算受限、开发维护成本高以及系统灵活度不足的问题，以成熟的底座和熟悉的技术栈去承载业务，架构相对透明化，兼容度良好且没有入侵性。\n\n![当当2.png](https://official-media.sphere-ex.com/2_783843d774.png)\n\n当当 WMS 系统拓扑图（单仓）\n\n\n## 用户收益\n\n在当当自研 WMS 的过程中发挥了非常重要的作用，带来了五方面的收益如下：\n\n1、性能极致\n\nShardingSphere-JDBC 轻量级的框架使其性能接近原生 JDBC，提供分库能力的同时性能可发挥极致，WMS 产能不再有受限的风险；\n\n2、系统稳定\n\n自 2016 年 WMS 上线至今，数年来系统一直稳定运行；\n\n3、低风险、零侵入\n\n当当底层系统为 2000 年演进而来，Apache ShardingSphere 凭借自身零侵入性的特点，用很小的改动就即完成了兼容适配，满足了业务需求；\n\n4、专注业务开发\n\n研发团队无需再关心业务拆分的工作，可将更多的精力聚焦于业务需求；\n\n5、降本增效\n\n高兼容的特点可以在无需大量的重构和改造前提下满足业务增长的需求，改造迁移成本低； \n\n通过仓店智能化、入库商品自动匹配最优货位的设计，仓店订单效率提升了 30%，人力成本节约上千万。\n\n\n## Apache ShardingSphere 的前世今生\n\n有人说 ShardingSphere 是当当的产品，其实更准确的说：ShardingSphere 起源于当当，在 2018 年 11 月 10 日捐赠给 Apache，经 17 个月的孵化于 2020 年 4 月 15 毕业，是 2020 年度第一个从 Apache 孵化器毕业的顶级项目。\n\n前段时间，在 ShardingSphere 进入 Apache 基金会三周年的纪念日（2021/11/10）里，社区发布了期待已久的 5.0 正式版，借着这个机会，一起回顾 Apache ShardingSphere 的前世今生。\n\n- 2014 年，当当针对电商平台推出了统一开发框架 dd-frame，目的是通过分离业务和技术代码，实现开发框架统一化、技术组件标准化以及跨组沟通高效化，使研发人员的精力最大化的聚焦在业务开发上。开发框架中名为 dd-rdb 的关系型数据库模块专门用于负责数据访问，可实现简单的分库分表功能，它就是 Sharding-JDBC 的前身，也是 dd-frame 2.x 中重点规划的部分。 \n \n- 2015 年，当当决定重构 WMS 和 TMS，设计上需要引用分库分表方案，天时地利人和，当当架构部在 9 月正式启动了项目，同年 12 月份 Sharding-JDBC 1.0.0 在当当内部发布，正式开启了 ShardingSphere 的征途。\n \n- 2016 年初，Sharding-JDBC 从 dd-rdb 中剥离，向社区开源，产品形态是增强版 JDBC 驱动，以 jar 包形式提供服务。\n \n- 2017 年底，发布 2.0.0 版本，打开了数据库治理功能。\n\n- 2018 年，进入 Apache 软件基金会。3.0.0 版本是一个华丽的转身，独立服务形态的 Sharding-Proxy 面世，实现了异构开发语言的需求，同时项目名称从 Sharding-JDBC 更名为 ShardingSphere，迈出了构建数据库上层标准和生态的第一步。 \n \n- 2019 年，发布 4.0.0 版本，可支持更多的数据库产品。\n \n- 2020 年，毕业成为 Apache 软件基金会的顶级项目。 \n\n- 2021 年 11 月 10 日，发布 5.0.0 GA 版，作为加入 Apache 软件基金会三周年纪念日的礼物，呈现给社区及整个分布式数据库和安全生态领域。 \n\n![当当3.png](https://official-media.sphere-ex.com/3_ae2b522cd5.png)\n Apache ShardingSphere - RoadMap\n\n从 5.0.0 版本开始，Apache ShardingSphere 正式开启了以可插拔架构为核心的全新阶段，实现了由过去数据分片单一应用场景到现在复杂应用环境下综合数据治理的身份转变与能力提升。在新版本中，Apache ShardingSphere 具备了更多新特性，并拥有了更加成熟的大数据解决方案。\n\n首先，基于可插拔架构对内核进行了全面改造，内核中的各个功能可以任意组合并叠加使用。其次，为了提升 SQL 分布式查询能力，5.0.0 GA 版打造了全新的 Federation 执行引擎，来满足用户复杂的业务场景。此外，5.0.0 GA 版在内核功能 API 层面也进行了大量优化，旨在降低用户使用这些功能的成本。\n\n## 写在结尾\n\n全链条数字化驱动着当当高质量发展，让全民阅读触手可及。在必不可少的自研 WMS 一环中，ShardingSphere 在数据层提供了强力支撑。\n\nApache ShardingSphere 历经两年的迭代与打磨，5.0.0 GA 版本已面世。趋近成熟的可插拔生态，标志着 Apache ShardingSphere 从分库分表中间件工具蜕变为以 Database Plus 理念为新航标。重塑分布式可插拔体系，助力整个数据库行业带来新风向和更有价值的解决方案。\n\n更多内容欢迎点击阅读原文链接访问中文社区 OpenSEC（Open SphereEx Community），一起畅聊开源。\n\n","date":"2022-04-29","author":"SphereEx","excerpt":"历经两年的迭代与打磨，可插拔生态趋近成熟，标志着 Apache ShardingSphere 从分库分表中间件蜕变为以 Database Plus 为理念的分布式数据库生态系统。","metaDescription":null,"createdAt":"2022-04-29T08:49:24.970Z","updatedAt":"2022-11-02T05:36:20.351Z","publishedAt":"2022-04-29T08:49:28.304Z","locale":"zh","caseStudiesType":{"data":null},"cover":{"data":null},"metaLogo":{"data":null},"localizations":{"data":[{"id":14,"attributes":{"feature":false,"title":"Asia’s E-Commerce Giant Dangdang Increases Order Processing Speed by 30% — Saves Over Ten Million in Technology Budget with Apache ShardingSphere","content":">Apache ShardingSphere is an easy-to-use and stable product, making Dangdang’s warehouse management system (WMS) even more powerful. Integrated with WMS, ShardingSphere plays a vital role in reforming the supply chain system.\n>-- Li Yong, Head of WMS Technology, Dangdang\n\nFollowing Apache ShardingSphere 5.0.0 GA release in November 2021, the 5.1.0 version was released last month. Having gone through over two years of polishing, ShardingSphere’s plugin-oriented ecosystem is beginning to take shape, and the project embarks on the evolution from a simple data sharding middleware to a mature distributed database ecosystem driven by the concept of Database Plus.\n\nDangdang, established at the end of 1999, has become a leading e-commerce platform selling books of any kind, and by integrating new Internet technologies with the traditional book industry. Dangdang was founded during the surge in China’s Internet industry in the early 2000s.\n\nLater, the e-commerce industry became exremely competitive in the country, and the market saturated. Facing fierce market competition, e-commerce platforms had to adapt to remain competitive.\n\nIn response, Dangdang not only adjusted its business strategies and management approaches, but also upgraded its technology architecture. Dangdang didn’t have its warehouse management and transportation system at that time. However, with growing business volume and technological capabilities, Dangdang needed to rebuild its warehouse management system and transportation management system (TMS) to better satisfy its business needs. For instance, in terms of hardware, it replaced mini-computer with x86, while its old centralized system was transformed to a distributed system with more flexibility.\n\nOne of the biggest challenges was massive warehousing data storage. The engineers wanted to adopt the data sharding technology that was often chosen by other big Internet companies. Disappointingly, they failed to find a mature and versatile open source database middleware in the marketplace, and therefore,started to develop a new data sharding product. That’s the origin of Sharding-JDBC. The product was created to bring more possibilities to data services.\n\nDangdang released its new WMS five years ago, which meant that it completed its intelligent warehousing transformation. Since then, with Apache ShardingSphere, the WMS has enabled Dangdang to hold large online shopping events every year such as Dangdang’s April Reading Festival, Double-Eleven Online Shopping Festival (aka Singles Day), and Mid-Year Shopping Festival, and to manage over a dozen of smart warehouses.\n\n## Business Challenges\n\nWhen Dangdang adopted a third-party WMS, the database in use was Oracle based on IBM minicomputers.\n\nHowever, considering the increasing business volume and warehouse order requests, especially during the online shopping festivals, the traditional centralized database architecture of the old WMS became inadequate because the computing and storage capabilities were limited. Additionally, the scale-up solution couldn’t support the system during online shopping festivals, and therefore the developers must do scale-up and adjust the business layer several times to alleviate the storage and computing limits and avoid production risks.\n\n- **Limited computing and storage capabilities**\n\nA centralized architecture is less scalable, making database computing and storage capabilities become the bottleneck.\n\n- **Expensive development and maintenance cost**\n\nBecause of poor scalability, developers have to make concessions and scale-up, which increases system development and maintenance costs.\n\n- **Exclusiveness**\n\nIf the architecture is not open enough, the system is less flexible, with fewer functions, and difficult to be transformed. The current architecture makes it difficult to quickly adopt new business services such as cloud native ones, SQL audit, data encryption, and distributed governance.\n\n## The Solution\n\nBased on the situation described above, Dangdang’s tech team proposed a warehouse management system solution: in terms of hardware, IBM minicomputer would be replaced with all-purpose x86, and MySQL would replace Oracle.\n\nHowever, at that time, there wasn’t a versatile and mature enough open source database middleware living up to Dangdang’s expectations, so they created one and named it Sharding-JDBC.\n\nShardingSphere-JDBC is positioned as a lightweight Java framework that provides additional services at the Java Database Connectivity (JDBC) layer. It is lightweight, efficient, easy to use, and compatible.\n\nWith ShardingSphere providing services in the form of the .jar files package, users can connect the client directly to the database without additional deployment and dependencies. It can be seen as an enhanced JDBC driver, fully compatible with JDBC and all ORM frameworks.\n\n- Compatible with JDBC and any JDBC-based ORM framework such as JPA, Hibernate, Mybatis, Spring JDBC Template.\n\n- Supports all third-party database connection pools such as DBCP, C3P0, BoneCP, HikariCP, etc.\n\n- Supports all databases implementing JDBC standards. Currently, ShardingSphere-JDBC supports MySQL, PostgreSQL, Oracle, SQL Server, and any database that can be accessed via JDBC.\n\n![1.jpeg](https://official-media.sphere-ex.com/1_3d03cbca69.jpeg)\nThe ShardingSphere-JDBC Topography\n\nCurrently, Apache ShardingSphere consists of three products, i.e. JDBC, Proxy, and Sidecar (TODO). ShardingSphere-JDBC and ShardingSphere-Proxy can be deployed independently or together.\n\nIt is ShardingSphere-JDBC that is used in Dangdang’s warehouse management system.\n\n\n>So how is ShardingSphere-JDBC exactly utilized?\n\nIn the warehouse management system, each warehouse positioned in a physical city is referred to as a unit with its corresponding business system and databases. Each warehouse has three sets of MySQL primary-secondary clusters to load the warehousing data of the designated city. So far, Dangdang has more than ten self-built warehouses all over China, mostly in cities where customers place a large number of orders. This self-built warehouse model is flexible for warehouse management and helps reduce storage costs in the long run.\n\nIn terms of architecture, the WMS uses ShardingSphere-JDBC to do database sharding according to their business types, and each cluster stores specified business data. The three MySQL clusters of a single warehouse are divided into three types as follows:\n\n-** Basic:** stores user, area, and menu data.\n- **Business:** stores order and package data.\n- **Inventory:** stores stock and working data.\n\nBefore the release, the system was initialized based on the basic data of a warehouse, such as storage locations.\n\nNext, by deploying the distributed database middleware, Dangdang successfully solved a series of problems such as limited storage and computing capabilities, high costs, and lack of flexibility.\n\n![1.jpeg](https://official-media.sphere-ex.com/1_b7199ae49f.jpeg)\nTopography of Dangdang’s Warehouse Management System (of a Single Warehouse)\n\n## User Advantages\n\nApache ShardingSphere played a significant role in helping Dangdang develop its WMS. There are five main benefits:\n\n- **Extraordinary performance**\n\nThe ShardingSphere-JDBC lightweight framework makes its performance close to a native JDBC. Apart from its great database sharding capability, it helps database performance be taken to the extreme, which allows the WMS to work at full capacity.\n\n- **Keep the system stable**\n\nWMS has been functioning well since its release in 2016.\n\n- **Low risk & zero invasion**\n\nThe underlying system of Dangdang has been evolving since 2000. Thanks to its zero-intrusion nature, Apache ShardingSphere can be compatible with others with small modifications to meet Dangdang’s business requirements.\n\n- **Allow developers to focus on the business side**\n\nThe developer team does not need to worry about sharding any more and can concentrate on developing the system to meet business needs.\n\n- **Cost effective and efficient**\n\nSince ShardingSphere is known for its high compatibility, to satisfy increasing business needs, developers don’t have to reconstruct or upgrade the system, minimizing the migration cost.\n\nWarehousing order processing speed is increased by 30% and accordingly, tens of millions of manpower costs are reduced due to the smart warehouses and the auto storage location matching technology.\n\n**Apache ShardingSphere’s Roadmap**\n\nSome said that ShardingSphere is a product created by Dangdang. To be precise, ShardingSphere was derived from the company and it also donated ShardingSphere to the Apache Software Foundation (ASF) on November 10, 2018.\n\nAfter 17-months in the ASF incubator, Apache ShardingSphere successfully graduated on April 15, 2020, as a Top-Level Apache Project.\n\nRecently, to celebrate the third anniversary of ShardingSphere entering Apache Software Foundation(ASF), the community released ShardingSphere 5.0.0. Below is a brief review of Apache ShardingSphere.\n\n- In 2014, Dangdang introduced a centralized development framework targeting at its e-commerce platform called dd-frame. It was created to unify the development framework, standardize its technical components, and achieve efficient cross-team communication by separating business code from technical code. In this way, engineers can devote all their efforts to the business side. The relational database module named dd-rdb in the framework was developed to handle data access and implement the data sharding function. It was the precursor of Sharding-JDBC, as well as a major part of dd-frame 2.x.\n\n- In 2015, Dangdang decided to rebuild its WMS and TMS. As it needed a data sharding plan, the team launched the project in September. In December, 2015, Sharding-JDBC 1.0.0 was released and used within Dangdang.\n\n- In early 2016, Sharding-JDBC was separated from dd-rdb and became open source. The product is an enhanced JDBC driver providing service in .jar files.\n\n- At the end of 2017, Version 2.0.0 was released with the new data governance function.\n\n- In 2018, ShardingSphere was enrolled into Apache Incubator. The release of Version 3.0.0 was a notable turnaround: Sharding-Proxy was released as an independent service. It supported heterogeneous languages, and the project was renamed from Sharding-JDBC to ShardingSphere. It’s in 2018 that the community decided to build the criteria and ecosystem above databases.\n\n- In 2019, Version 4.0.0 was released capable of supporting more database products.\n\n- In 2020, ShardingSphere graduated as a Top-Level Project of the ASF.\n\n- On November 10, 2021, Version 5.0.0 GA was released as a third-anniversity celebration with the whole Apache ShardingSphere community, and the distributed database industry.\n\n![1.jpeg](https://official-media.sphere-ex.com/1_83a06f3e37.jpeg)\nApache ShardingSphere—Roadmap\n\nSince Version 5.0.0, Apache ShardingSphere has embarked on its new journey: with the plugin oriented architect at its core, it evloved from a data sharding application to a comprehensive and enhanced data governance tool applicable to various complex application scenarios. Concurrently, Apache ShardingSphere also has more features, and big data solutions.\n\n## Conclusion\n\nDigitization motivated Dangdang to achieve high-quality development and fulfill its mission. ShardingSphere is glad to support Dangdang’s WMS with its cutting-edging data services.\n\nHaving gone through two years‘ development, Apache ShardingSphere 5.0.0 GA has been released. The pluggable ecosystem marks an evolution from a data sharding middleware tool to a pioneer in the industry following the “Database Plus” concept.","date":"2022-03-11","author":"SphereEx","excerpt":"Apache ShardingSphere is easy-to-use and stable, making Dangdang’s warehouse management system (WMS) even more powerful.","metaDescription":"Dangdang is a B2C-based e-commerce company in China offering a large selection of products both online and offline.","createdAt":"2022-04-29T10:52:10.330Z","updatedAt":"2022-09-28T02:42:43.896Z","publishedAt":"2022-04-29T10:52:12.009Z","locale":"en"}}]}}}}}, "staticQueryHashes": []}