gaodayue opened a new issue #3893:
URL: https://github.com/apache/incubator-doris/issues/3893
In a cluster with frequent load activities, FE will ignore most tablet
report from BE because currently it only handle reports whose version >= BE's
latest report version (which is increased each time a transaction is
published). This can be observed from FE's log, with many
logs like `out of date report version 15919277405765 from
backend[177969252]. current report version[15919277405766]` in it.
However many system functionalities rely on TabletReport processing to work
properly. For example
1. bad or version miss replica is detected and repaired during TabletReport
2. storage medium migration decision and action is made based on TabletReport
3. BE's old transaction is cleared/republished during TabletReport
After reading `ReportHandler.tabletReport`, I think the strict report
version check is not required. In fact it's not possible for FE to make
decision based on the latest state of BE, because BE's state and report version
could still change during FE's processing of tabletReport. **In practice, we
have removed the version check on many of our clusters for more than a month,
nothing bad has happened.** However, we do record the version of each BE's last
report and make sure only report of bigger version is handled.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
