Re: [PR] [SPARK-51758][SS] Fix test case related to extra batch causing empty df due to watermark [spark]

[via GitHub]

LuciferYang commented on PR #50626:
URL: https://github.com/apache/spark/pull/50626#issuecomment-2821775469

   I found that it's not just the aforementioned daily test that has been 
affected.
   
   I noticed that after this pull request  was merged into the branch-4.0 
submission pipeline, the log size of the GA task also increased:
   
   
![image](https://github.com/user-attachments/assets/e91694d4-4801-477b-908a-71a5be7e5b8e)
   
   - Revert "[SPARK-51758][SS][FOLLOWUP][TESTS] Fix flaky test around watermark 
due to additional batch causing empty df": 
https://github.com/apache/spark/actions/runs/14513181823
   
   
![image](https://github.com/user-attachments/assets/732a165f-d008-4bad-a9ff-8746319578e4)
   
   
   - [SPARK-51758][SS] Fix test case related to extra batch causing empty df 
due to watermark: https://github.com/apache/spark/actions/runs/14525129747
   
   
![image](https://github.com/user-attachments/assets/d844d130-571c-4bbd-8fee-b467be266b36)
   
   At the same time, after this submission was merged into branch-4.0, similar 
failures occurred as those in the daily test:
   
   - https://github.com/apache/spark/actions/runs/14525129747/job/40754977831
   
   
![image](https://github.com/user-attachments/assets/037fc0b8-ee2b-40df-9c82-646b9588bc66)
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org