dutyu opened a new pull request, #18252:
URL: https://github.com/apache/doris/pull/18252
# Proposed changes
Issue Number: close #18251
## Problem summary
`lastSyncedEventId` of `HMSExternalCatalog` must set to -1 firstly. If
`lastSyncedEventId` is not initialized, `lastSyncedEventId` will be 0, so the
first invocation of `HiveMetaStoreClient.getNextNotification` will be failed
because event id of notification event must be positive and always there is a
ttl for hive metastore's `notifition_log` table, `HiveMetaStoreClient` will
check the returned event id and throws exception if it is illegal.
The relevant implement of `HiveMetaStoreClient` is as follows:
```
@InterfaceAudience.LimitedPrivate({"HCatalog"})
@Override
public NotificationEventResponse getNextNotification(long lastEventId, int
maxEvents,
NotificationFilter
filter) throws TException {
NotificationEventRequest rqst = new
NotificationEventRequest(lastEventId);
rqst.setMaxEvents(maxEvents);
NotificationEventResponse rsp = client.get_next_notification(rqst);
LOG.debug("Got back " + rsp.getEventsSize() + " events");
NotificationEventResponse filtered = new NotificationEventResponse();
if (rsp != null && rsp.getEvents() != null) {
long nextEventId = lastEventId + 1;
for (NotificationEvent e : rsp.getEvents()) {
**// lastEventId=0, `if` statement will return false and an
exception will throws here**
if (e.getEventId() != nextEventId) {
LOG.error("Requested events are found missing in NOTIFICATION_LOG
table. Expected: {}, Actual: {}. "
+ "Probably, cleaner would've cleaned it up. "
+ "Try setting higher value for
hive.metastore.event.db.listener.timetolive. "
+ "Also, bootstrap the system again to get back the
consistent replicated state.",
nextEventId, e.getEventId());
throw new IllegalStateException(REPL_EVENTS_MISSING_IN_METASTORE);
}
if ((filter != null) && filter.accept(e)) {
filtered.addToEvents(e);
}
nextEventId++;
}
}
return (filter != null) ? filtered : rsp;
}
```
## Checklist(Required)
* [ ] Does it affect the original behavior
* [ ] Has unit tests been added
* [ ] Has document been added or modified
* [ ] Does it need to update dependencies
* [ ] Is this PR support rollback (If NO, please explain WHY)
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[[email protected]](mailto:[email protected]) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
