mxli441 opened a new issue, #21100: URL: https://github.com/apache/shardingsphere/issues/21100
## Bug Report **For English only**, other languages will not accept. Before report a bug, make sure you have: - Searched open and closed [GitHub issues](https://github.com/apache/shardingsphere/issues). - Read documentation: [ShardingSphere Doc](https://shardingsphere.apache.org/document/current/en/overview). Please pay attention on issues you submitted, because we maybe need more details. If no response anymore and we cannot reproduce it on current information, we will **close it**. Please answer these questions before submitting your issue. Thanks! ### Which version of ShardingSphere did you use? shardingsphere-proxy-5.2.0 etcd Version: 3.4.9 opengauss 3.0.0 ### Which project did you use? ShardingSphere-JDBC or ShardingSphere-Proxy? shardingsphere-proxy ### Expected behavior ShardingSphere-Proxy can recognize new primary db after switchovering ### Actual behavior ShardingSphere-Proxy can't recognize new primary db after switchovering ### Reason analyze (If you can) Maybe it is a bug or my configuration is wroing? ### Steps to reproduce the behavior, such as: SQL to execute, sharding rule configuration, when exception occur etc. ### Example codes for reproduce this issue (such as a github link). =============server.yaml============== mode: type: Cluster repository: type: Etcd props: namespace: /governance_ds server-lists: http://172.32.153.23x:12379,http://172.32.153.24x:12379,http://172.32.153.23x:12379 retryIntervalMilliseconds: 500 timeToLiveSeconds: 60 maxRetries: 3 - !AUTHORITY users: - root@%:root - sharding@:sharding provider: type: ALL_PERMITTED - !TRANSACTION defaultType: XA providerType: Atomikos - !SQL_PARSER sqlCommentParseEnabled: true sqlStatementCache: initialCapacity: 2000 maximumSize: 65535 parseTreeCache: initialCapacity: 128 maximumSize: 1024 props: max-connections-size-per-query: 1 kernel-executor-size: 16 # Infinite by default. proxy-frontend-flush-threshold: 128 # The default value is 128. proxy-hint-enabled: false sql-show: true ===========config-database-discovery.yaml=========== databaseName: readwrite_db # dataSources: ds_0: url: jdbc:opengauss://172.32.153.24x:31000/d1 username: appuser password: appuser@1 connectionTimeoutMilliseconds: 3000 idleTimeoutMilliseconds: 60000 maxLifetimeMilliseconds: 1800000 maxPoolSize: 50 minPoolSize: 1 ds_1: url: jdbc:opengauss://172.32.153.23x:31000/d1 username: appuser password: appuser@1 connectionTimeoutMilliseconds: 3000 idleTimeoutMilliseconds: 60000 maxLifetimeMilliseconds: 1800000 maxPoolSize: 50 minPoolSize: 1 ds_2: url: jdbc:opengauss://172.32.153.23X:31000/d1 username: appuser password: appuser@1 connectionTimeoutMilliseconds: 3000 idleTimeoutMilliseconds: 60000 maxLifetimeMilliseconds: 1800000 maxPoolSize: 50 minPoolSize: 1 rules: - !DB_DISCOVERY dataSources: readwrite_ds: dataSourceNames: - ds_0 - ds_1 - ds_2 discoveryHeartbeatName: opengauss-heartbeat discoveryTypeName: openGauss discoveryHeartbeats: opengauss-heartbeat: props: keep-alive-cron: '0/5 * * * * ?' discoveryTypes: openGauss: type: openGauss.NORMAL_REPLICATION props: group-name: 92504d5b-6dec-11e8-91ea-246e9612aaf1 - !READWRITE_SPLITTING dataSources: readwrite_ds: dynamicStrategy: autoAwareDataSourceName: readwrite_ds writeDataSourceQueryEnabled: true loadBalancerName: random loadBalancers: random: type: ROUND_ROBIN ===============before switchovering =============== Current cluster topology + Cluster: pg_cluster (630762503572893914) --+---------+---------+----+-----------+-----------------+ | Member | Host | Role | State | TL | Lag in MB | Pending restart | +---------------------+----------------------+---------+---------+----+-----------+-----------------+ | node-172.32.153.23x | 172.32.153.23x:31000 | Replica | running | 1 | 0 | * | | node-172.32.153.23x | 172.32.153.23x:31000 | Replica | running | 1 | 0 | * | | node-172.32.153.24x | 172.32.153.24x:31000 | Leader | running | 1 | | * | +---------------------+----------------------+---------+---------+----+-----------+-----------------+ committed command：etcdctl get --endpoints=127.0.0.1:12379 --prefix /governance_ds/nodes /governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_0 replicationDelayMilliseconds: 0 role: primary status: enabled /governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_1 replicationDelayMilliseconds: 0 role: member status: enabled /governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_2 replicationDelayMilliseconds: 0 role: member status: enabled inserting can commit ==============After switchovering=============== + Cluster: pg_cluster (630762503572893914) --+---------+---------+----+-----------+-----------------+ | Member | Host | Role | State | TL | Lag in MB | Pending restart | +---------------------+----------------------+---------+---------+----+-----------+-----------------+ | node-172.32.153.23x | 172.32.153.23x:31000 | Leader | running | 2 | | * | | node-172.32.153.23x | 172.32.153.23x:31000 | Replica | running | 2 | 0 | * | | node-172.32.153.24x | 172.32.153.24x:31000 | Replica | running | 2 | 0 | * | +---------------------+----------------------+---------+---------+----+-----------+-----------------+ committed command：etcdctl get --endpoints=127.0.0.1:12379 --prefix /governance_ds/nodes 主库无变化 /governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_0 replicationDelayMilliseconds: 0 role: primary status: enabled /governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_1 replicationDelayMilliseconds: 0 role: member status: enabled /governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_2 replicationDelayMilliseconds: 0 role: member status: enabled ================Connected to database with port 3307================ gsql -h 172.32.153.23X -p 3307 -U root readwrite_db -r Password for user root: gsql ((openGauss 3.0.0 build b592536d) compiled at 2022-04-08 15:30:18 commit 0 last mr ) Non-SSL connection (SSL connection is recommended when requiring high-security) Type "help" for help. readwrite_db=> \d List of relations Schema | Name | Type | Owner | Storage --------+------+-------+---------+---------------------------------- public | test | table | appuser | {orientation=row,compression=no} public | t2 | table | appuser | {orientation=row,compression=no} public | t1 | table | appuser | {orientation=row,compression=no} (3 rows) readwrite_db=> insert into t1 select 1; ERROR: cannot execute INSERT in a read-only transaction ==I restarted shardingsphere-proxy process then shardingsphere-proxy recognized the new primary db ====== [root@nhgapp05 conf]# ps -ef |grep -i shard root 55939 1 16 17:31 pts/30 00:00:42 /usr/bin/java -Djava.awt.headless=true -server -Xmx2g -Xms2g -Xmn1g -Xss1m -XX:AutoBoxCacheMax=4096 -XX:+UseNUMA -XX:+DisableExplicitGC -XX:LargePageSizeInBytes=128m -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -Dio.netty.leakDetection.level=DISABLED -classpath /opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/conf:/opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/conf:.:/opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/lib/*:/opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/ext-lib/* org.apache.shardingsphere.proxy.Bootstrap -1 /opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/conf root 57052 56001 0 17:35 pts/30 00:00:00 grep --color=auto -i shard [root@nhgapp05 conf]# kill 55939 [root@nhgapp05 conf]# sh ../bin/start.sh /usr/bin/java we find java version: java8, full_version=1.8.0_181, full_path=/usr/bin/java The classpath is /opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/conf:/opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/conf:.:/opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/lib/*:/opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/ext-lib/* main class org.apache.shardingsphere.proxy.Bootstrap -1 /opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/conf Starting the ShardingSphere-Proxy ... PID: 57342 Please check the STDOUT file: /opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/logs/stdout.log ===============committed etcdctl command ====================================== etcdctl get --endpoints=127.0.0.1:12379 --prefix /governance_ds/nodes /governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_0 replicationDelayMilliseconds: 0 role: member status: enabled /governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_1 replicationDelayMilliseconds: 0 role: primary status: enabled /governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_2 replicationDelayMilliseconds: 0 role: member status: enabled ============================================================ gsql -h 172.32.153.23x -p 3307 -U root readwrite_db -r Password for user root: gsql ((openGauss 3.0.0 build b592536d) compiled at 2022-04-08 15:30:18 commit 0 last mr ) Non-SSL connection (SSL connection is recommended when requiring high-security) Type "help" for help. readwrite_db=> insert into t1 select 1; INSERT 0 1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
