zecookiez commented on code in PR #50123: URL: https://github.com/apache/spark/pull/50123#discussion_r2034518628
########## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStoreCoordinator.scala: ########## @@ -55,6 +56,39 @@ private case class GetLocation(storeId: StateStoreProviderId) private case class DeactivateInstances(runId: UUID) extends StateStoreCoordinatorMessage +/** + * This message is used to report a state store has just finished uploading a snapshot, + * along with the timestamp in milliseconds and the snapshot version. + */ +private case class ReportSnapshotUploaded( + providerId: StateStoreProviderId, + version: Long, + timestamp: Long) Review Comment: This isn't distinguishable at the moment, we send the same RPC message for both types at the moment (treating the timestamp of loading the snapshot version as the upload timestamp). During testing with some benchmark suites, a lot of state stores would be marked as lagging at the start because of the lack of time check (every store has default timestamp of 0, so it was easy to fulfill the time lag check). This is why both of those cases report a proper timestamp now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
