LuciferYang commented on PR #50626: URL: https://github.com/apache/spark/pull/50626#issuecomment-2823949253
> > Should we revert first? WDYT? > > I've conducted a general analysis of the log content. The large log file consists of a total of 313,451,920 lines. Among these, the logs starting from line 8,538,126 to the end of the file are all related to the query name `event_time_test_query` (which is the test that this pr claims to have fixed). Personally, I guess this is because the current pr has made this test case more robust, and the Java process has been continuously printing DEBUG logs, leading to disk `no space` in the end. Therefore, I personally think the solution should be to find a way to control the log level of the Java process used by the Pyspark test, rather than revert this pr. I think https://github.com/apache/spark/pull/50675 can resolve this issue. Let me merge it and verify the branch-4.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
