[tag subnets] add tag by subnets for near-rpc

dubbo-extensions-dependencies-bom/pom.xml

@@ -135,6 +135,8 @@
         <resteasy_version>3.0.20.Final</resteasy_version>
         <polaris_adapter_version>0.2.1</polaris_adapter_version>
         <maven_flatten_version>1.2.5</maven_flatten_version>
+        <commons_net_version>3.9.0</commons_net_version>
+        <snakeyaml_version>2.0</snakeyaml_version>
     </properties>
 
     <dependencyManagement>
@@ -489,6 +491,16 @@
                 <artifactId>polaris-adapter-dubbo</artifactId>
                 <version>${polaris_adapter_version}</version>
             </dependency>
+            <dependency>
+                <groupId>commons-net</groupId>
+                <artifactId>commons-net</artifactId>
+                <version>${commons_net_version}</version>
+            </dependency>
+            <dependency>
+                <groupId>org.yaml</groupId>
+                <artifactId>snakeyaml</artifactId>
+                <version>${snakeyaml_version}</version>
+            </dependency>
         </dependencies>
     </dependencyManagement>
 

dubbo-tag-extensions/README.md

@@ -0,0 +1,65 @@
+# dubbo tag subnets
+
+[中文](./README.md)
+
+dubbo-tag-subnets will generate a tag by subnets, then consumer will prefer rpc provider in the same subnets. 
+
+- example1: there are 3 zone(cn-shanghai-a/cn-shanghai-b/cn-shanghai-c) in aliyun-VPC, service rpc in the same zone
+```
+cn|shanghai|a:
+- 172.37.66.0/24 #zone=cn-shanghai-a
+cn|shanghai|b:
+- 172.37.68.0/24 #zone=cn-shanghai-b
+cn|shanghai|c:
+- 172.37.69.0/24 #zone=cn-shanghai-c
+```
+- example2: there is a big zone and there are 3 cells in it , service rpc in the same cell
+```
+cn|shanghai|a|cell-1:
+- 172.31.35.0/24 #zone=cn-shanghai-a
+cn|shanghai|a|cell-2:
+- 172.31.36.0/24 #zone=cn-shanghai-a
+cn|shanghai|a|cell-3:
+- 172.31.37.0/24 #zone=cn-shanghai-a
+```
+- example3: there 3 seperate zones(a/b/c) and a common zone(d), consumer prefer rpc in near zones(a/b/c), and then rpc common zone(d).
+```
+cn|shanghai|a:
+- 172.37.66.0/24 #zone=cn-shanghai-a
+cn|hangzhou|b:
+- 172.37.67.0/24 #zone=cn-hangzhou-b
+cn|shenzhen|c:
+- 172.37.68.0/24 #zone=cn-shenzhen-c
+"":
+- 172.37.69.0/24 #zone=cn-shanghai-d, as common service
+```
+
+## How to use?
+
+1. import dependency
+
+```
+<dependency>
+    <groupId>org.apache.dubbo.extensions</groupId>
+    <artifactId>dubbo-tag-subnets</artifactId>
+    <version>${dubbo-tag-subnets-version}</version>
+</dependency>
+
+```
+
+2. set tag-subnets in config-center path=/dubbo/config/tag.subnets, zookeeper example:
+```
+echo 'cn|cn-northwest|cell-1:
+- 172.37.66.0/24 #cn-northwest-1a
+cn|cn-north|cell-2:
+- 172.37.67.0/24 #cn-northwest-1b
+cn|cn-north:
+- 192.168.1.0/24 #cn-north-1a
+' | tee tag.subnets.yaml
+./zkCli.sh create /dubbo/config/tag.subnets "" 
+./zkCli.sh set /dubbo/config/tag.subnets "$(cat tag.subnets.yaml)"
+./zkCli.sh get /dubbo/config/tag.subnets   
+```
+
+
+3. start your provider, you will see tag generate by subnets in registry. 

dubbo-tag-extensions/README_zh.md

@@ -0,0 +1,65 @@
+# dubbo tag subnets
+
+[中文](./README.md)
+
+dubbo-tag-subnets 会根据子网生成tag, 然后同子网内的tag相同, 服务调用会优先发生在同子网中.
+
+- 示例1: 在阿里云VPC, 有三个可用区(cn-shanghai-a/cn-shanghai-b/cn-shanghai-c) , 服务在同一可用区中调用
+```
+cn|shanghai|a:
+- 172.37.66.0/24 #zone=cn-shanghai-a
+cn|shanghai|b:
+- 172.37.68.0/24 #zone=cn-shanghai-b
+cn|shanghai|c:
+- 172.37.69.0/24 #zone=cn-shanghai-c
+```
+- 示例2: 有三个单元在一个大的可用区cn-shanghai-a , 服务在同单元中调用
+```
+cn|shanghai|a|cell-1:
+- 172.31.35.0/24 #zone=cn-shanghai-a
+cn|shanghai|a|cell-2:
+- 172.31.36.0/24 #zone=cn-shanghai-a
+cn|shanghai|a|cell-3:
+- 172.31.37.0/24 #zone=cn-shanghai-a
+```
+- 示例3: 有三个"独立的可用区"(a/b/c) ,有一个"公共的可用区"(d), 服务消费方优先在自己"独立可用区"中调用, 然后调用"公共的可用区".
+```
+cn|shanghai|a:
+- 172.37.66.0/24 #zone=cn-shanghai-a
+cn|hangzhou|b:
+- 172.37.67.0/24 #zone=cn-hangzhou-b
+cn|shenzhen|c:
+- 172.37.68.0/24 #zone=cn-shenzhen-c
+"":
+- 172.37.69.0/24 #zone=cn-shanghai-d, as common service
+```
+
+## 如何使用?
+
+1. 引入依赖
+
+```
+<dependency>
+    <groupId>org.apache.dubbo.extensions</groupId>
+    <artifactId>dubbo-tag-subnets</artifactId>
+    <version>${dubbo-tag-subnets-version}</version>
+</dependency>
+
+```
+
+2. 在配置中心设置 tag-subnets ,路径为/dubbo/config/tag.subnets, zookeeper 配置示例:
+```
+echo 'cn|cn-northwest|cell-1:
+- 172.37.66.0/24 #cn-northwest-1a
+cn|cn-north|cell-2:
+- 172.37.67.0/24 #cn-northwest-1b
+cn|cn-north:
+- 192.168.1.0/24 #cn-north-1a
+' | tee tag.subnets.yaml
+./zkCli.sh create /dubbo/config/tag.subnets "" 
+./zkCli.sh set /dubbo/config/tag.subnets "$(cat tag.subnets.yaml)"
+./zkCli.sh get /dubbo/config/tag.subnets   
+```
+
+
+3. 启动服务提供方, 您将在注册中心看到根据子网生成的服务标签. 

dubbo-tag-extensions/dubbo-tag-subnets/pom.xml

@@ -0,0 +1,62 @@
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+  -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <parent>
+        <artifactId>dubbo-tag-extensions</artifactId>
+        <groupId>org.apache.dubbo.extensions</groupId>
+        <version>${revision}</version>
+        <relativePath>../pom.xml</relativePath>
+    </parent>
+    <modelVersion>4.0.0</modelVersion>
+
+    <version>1.0.2-SNAPSHOT</version>
+    <artifactId>dubbo-tag-subnets</artifactId>
+    <name>${project.artifactId}</name>
+    <description>The SOFARegistry module of Dubbo project</description>
+
+    <dependencies>
+        <dependency>
+            <groupId>org.apache.dubbo</groupId>
+            <artifactId>dubbo-registry-api</artifactId>
+            <optional>true</optional>
+        </dependency>
+        <dependency>
+            <groupId>org.apache.dubbo</groupId>
+            <artifactId>dubbo-common</artifactId>
+            <optional>true</optional>
+        </dependency>
+        <dependency>
+            <groupId>org.apache.dubbo</groupId>
+            <artifactId>dubbo-config-api</artifactId>
+            <optional>true</optional>
+        </dependency>
+        <dependency>
+            <groupId>commons-net</groupId>
+            <artifactId>commons-net</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>org.yaml</groupId>
+            <artifactId>snakeyaml</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>org.testcontainers</groupId>
+            <artifactId>testcontainers</artifactId>
+            <scope>test</scope>
+        </dependency>
+    </dependencies>
+</project>

dubbo-tag-extensions/dubbo-tag-subnets/src/main/java/org/apache/dubbo/tag/subnets/config/SubnetTagConfigPostProcessor.java

@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.dubbo.tag.subnets.config;
+
+import org.apache.dubbo.common.config.ConfigurationUtils;
+import org.apache.dubbo.common.extension.Activate;
+import org.apache.dubbo.common.utils.StringUtils;
+import org.apache.dubbo.config.ConfigPostProcessor;
+import org.apache.dubbo.config.ServiceConfig;
+import org.apache.dubbo.rpc.model.ApplicationModel;
+import org.apache.dubbo.rpc.model.ScopeModelAware;
+import org.apache.dubbo.tag.subnets.utils.SubnetUtil;
+
+import static org.apache.dubbo.common.utils.NetUtils.getLocalHost;
+
+
+@Activate
+public class SubnetTagConfigPostProcessor implements ConfigPostProcessor, ScopeModelAware {
+    private ApplicationModel applicationModel;
+
+    @Override
+    public void setApplicationModel(ApplicationModel applicationModel) {
+        this.applicationModel = applicationModel;
+    }
+
+    @Override
+    public void postProcessServiceConfig(ServiceConfig serviceConfig) {
+        if (StringUtils.isNotBlank(serviceConfig.getTag())) {
+            return;
+        }
+        if (SubnetUtil.isEmpty()) {
+            String content = ConfigurationUtils.getCachedDynamicProperty(applicationModel.getDefaultModule(), SubnetUtil.TAG_SUBNETS_KEY, null);
+            SubnetUtil.init(content);
+        }
+        if (SubnetUtil.isEmpty()) {
+            return;
+        }
+        String providerHost = serviceConfig.getProvider().getHost();
+        if (StringUtils.isBlank(providerHost)) {
+            providerHost = getLocalHost();
+        }
+        String tagLevel = SubnetUtil.getTagLevelByHost(providerHost);
+        serviceConfig.setTag(tagLevel);
+    }
+
+
+}

dubbo-tag-extensions/dubbo-tag-subnets/src/main/java/org/apache/dubbo/tag/subnets/utils/SubnetUtil.java

@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.dubbo.tag.subnets.utils;
+
+import org.apache.dubbo.common.utils.StringUtils;
+
+import org.apache.commons.net.util.SubnetUtils;
+import org.yaml.snakeyaml.LoaderOptions;
+import org.yaml.snakeyaml.Yaml;
+import org.yaml.snakeyaml.constructor.SafeConstructor;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+
+public class SubnetUtil {
+    public static final String TAG_SUBNETS_KEY = "tag.subnets";
+
+    private static final Map<String, List<SubnetUtils.SubnetInfo>> cellSubnets = new ConcurrentHashMap<>();
+    private static final ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
+    private static final ReentrantReadWriteLock.ReadLock readLock = lock.readLock();
+    private static final ReentrantReadWriteLock.WriteLock writeLock = lock.writeLock();
+
+    public static boolean isEmpty() {
+        try {
+            readLock.lock();
+            return cellSubnets.isEmpty();
+        } finally {
+            readLock.unlock();
+        }
+    }
+
+    public static void init(String content) {
+        if (StringUtils.isBlank(content)) {
+            return;
+        }
+        try {
+            writeLock.lock();
+            Yaml yaml = new Yaml(new SafeConstructor(new LoaderOptions()));
+            cellSubnets.clear();
+            Map<String, List<String>> tmpPathSubnet = (Map<String, List<String>>) yaml.load(content);
+            for (Map.Entry<String, List<String>> entry : tmpPathSubnet.entrySet()) {
+                String path = entry.getKey();
+                List<SubnetUtils.SubnetInfo> subnetInfos = cellSubnets.computeIfAbsent(path, f -> new ArrayList<SubnetUtils.SubnetInfo>());
+                entry.getValue().forEach(e -> subnetInfos.add(new SubnetUtils(e.trim()).getInfo()));
+            }
+        } finally {
+            writeLock.unlock();
+        }
+    }
+
+    public static String getTagLevelByHost(String host) {
+        try {
+            readLock.lock();
+            for (Map.Entry<String, List<SubnetUtils.SubnetInfo>> entry : cellSubnets.entrySet()) {
+                for (SubnetUtils.SubnetInfo subnetInfo : entry.getValue()) {
+                    if (subnetInfo.isInRange(host)) {
+                        return entry.getKey();
+                    }
+                }
+            }
+            return null;
+        } finally {
+            readLock.unlock();
+        }
+    }
+}

dubbo-tag-extensions/dubbo-tag-subnets/src/main/resources/META-INF/dubbo/org.apache.dubbo.config.ConfigPostProcessor

@@ -0,0 +1 @@
+tag-subnets=org.apache.dubbo.tag.subnets.config.SubnetTagConfigPostProcessor