vrozov commented on PR #49276: URL: https://github.com/apache/spark/pull/49276#issuecomment-2591259692
@gengliangwang My understanding is that log level was set to TRACE intentionally and it should not be enabled by default. Please see comment on #45990 > Because every query applies cache, this log could be huge and should be only turned on during some debugging process, and should not enabled by default in production. Note that warnings on line 129 and 145 coexist with changes from #45990 and provide early problem notification. This PR originates from a real issue where I spent large amount of time first isolating memory leak to the `CacheManager` and then debugging it to the `unpersist()` call on a wrong data set. Should the warning be present in the first place, it would help to identify the problem much easier. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
