unbridled-41 opened a new issue, #11045:
URL: https://github.com/apache/rocketmq/issues/11045

   ### Before Creating the Bug Report
   
   - [x] I found a bug, not just a question.
   - [x] I searched open GitHub Issues and pull requests and found no duplicate.
   - [x] I confirmed that this bug belongs to Apache RocketMQ.
   
   ### Runtime platform environment
   
   All platforms; reproduced with a deterministic unit test on the current 
`develop` branch.
   
   ### RocketMQ version
   
   `develop` at `ff8f6f74c`
   
   ### JDK Version
   
   All
   
   ### Describe the Bug
   
   `DefaultBrokerHeartbeatManager#onBrokerHeartbeat` never rebinds the channel 
of an existing `BrokerLiveInfo` (the `prev != null` branch only refreshes the 
timestamp/timeout/priority/epoch), so the live entry stays pinned to the 
broker's **first-ever** channel. When that old channel eventually fires 
`channelInactive` / `onChannelException` / `onChannelIdle`, 
`onBrokerChannelClose` matches it against the stale stored channel, removes the 
live entry of a broker that is healthy on a new channel, and fires the 
broker-inactive lifecycle event, which makes 
`ControllerManager#onBrokerInactive` trigger a master election for a perfectly 
healthy master.
   
   This is realistic in at least two ordinary situations:
   
   1. The broker-to-controller connection dies half-open on the controller 
side; the broker client detects it first, reconnects, and keeps heartbeating on 
the new channel. Minutes later the controller's TCP stack learns the old 
channel is dead and fires the close event.
   2. The controller-side idle/close event for the old channel is only 
delivered after the broker has already re-registered on a new channel.
   
   ### Steps to Reproduce
   
   1. Broker B (master of its broker-set) sends a heartbeat to the controller 
on channel A → `BrokerLiveInfo` holds channel A.
   2. The broker reconnects and keeps heartbeating on channel B (every 
heartbeat hits the `prev != null` branch, channel stays A).
   3. Channel A fires `onBrokerChannelClose(A)` on the controller.
   4. Observe that B's live entry is removed and `notifyBrokerInActive` fires 
although B is actively heartbeating on channel B; 
`ControllerManager#onBrokerInactive` then calls `triggerElectMaster` because 
the evicted broker is the current master.
   
   ### What Did You Expect to See?
   
   A close event for a channel that the broker no longer uses must not evict 
the live entry: the heartbeat path should rebind `BrokerLiveInfo` to the 
current channel, so `onBrokerChannelClose` only removes entries whose stored 
channel is the one that actually closed.
   
   ### What Did You See Instead?
   
   The live entry of the re-registered master is removed, a broker-inactive 
notification is fired, and the controller bumps the master epoch / runs a 
failover for a healthy broker-set.
   
   ### Additional Context
   
   The sibling NameServer implementation (`RouteInfoManager`) updates the 
broker's channel on every heartbeat, so the controller behavior is inconsistent 
with the nameserver. The fix is to rebind the channel in the `prev != null` 
branch of `onBrokerHeartbeat` when a non-null channel arrives; a regression 
test in `DefaultBrokerHeartbeatManagerTest` reproduces the eviction 
deterministically.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to