Kimahriman commented on code in PR #50483:
URL: https://github.com/apache/spark/pull/50483#discussion_r2025291601
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDB.scala:
##########
@@ -1485,11 +1485,12 @@ class RocksDB(
override def log(infoLogLevel: InfoLogLevel, logMsg: String) = {
// Map DB log level to log4j levels
// Warn is mapped to info because RocksDB warn is too verbose
+ // Info is mapped to debug because RocksDB info is too verbose
Review Comment:
This part is the biggest pain for us and I've been trying to find ways to
filter these out. All the statistics are incredibly verbose, and when you are
running with multiple cores per executor and they all just get intermingled
it's basically impossible to find anything useful. +1 to this being a thing to
opt-in to debug logs for debugging purposes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
