cadonna edited a comment on pull request #10953:
URL: https://github.com/apache/kafka/pull/10953#issuecomment-872811076


   Some details for the other reviewers.
   
   In 2.8 and before, we computed the default grace period with 
`Math.max(maintainDurationMs - sizeMs, 0);` in method `gracePeriodMs()` in 
`TimeWindows`, `SessionWindows`, and `JoinWindows`. That means that the default 
grace period has never been 24 hours but 24 hours - window size. Since 
`gracePeriodMs()` is used to compute the retention time of the changelog topic 
for the corresponding window state store and the segments for the window state 
store it is important to keep the same computation for the deprecated methods. 
Otherwise, Streams app that run with 2.8 and before might not be compatible 
with Streams 3.0 because the retention time of the changelog topics created 
with older Streams apps will be smaller than the assumed retention time for 
Streams apps in 3.0. For example, with a window size of 10 hours, an old 
Streams app would have created a changelog topic with retention time 10 hours 
(window size) + 14 hours (default grace period, 24 hours - 10 hours). A 3.0 
Streams app woul
 d assume a retention time of 10 hours (window size) + 24 hours (deprecated 
default grace period as currently specified on trunk). In the presence of 
failures, where a state store needs to recreated, records might get lost, 
because before the failure the state store of a 3.0 Streams app contained 10 
hours + 24 hours of records whereas the changelog topic that was created with 
the old Streams app would only contain 10 hours + 14 hours of records.
   
   All this happened due to us always stating that the default grace period was 
24 hours although it was not completely correct and a connected and unfortunate 
misunderstanding when we removed deprecated windows APIs (#10378).      
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Reply via email to