Re: [PR] [SPARK-51865][CONNECT][TESTS] Print stacktrace message when `o.a.arrow.memory.BaseAllocator` fails to close [spark]

[via GitHub]

LuciferYang commented on code in PR #50663:
URL: https://github.com/apache/spark/pull/50663#discussion_r2056408481


##########
sql/connect/client/jvm/src/test/resources/log4j2.properties:
##########
@@ -37,3 +37,6 @@ appender.console.layout.pattern = %d{HH:mm:ss.SSS} %p %c: 
%maxLen{%m}{512}%n%ex{
 # Ignore messages below warning level from Jetty, because it's a bit verbose
 logger.jetty.name = org.sparkproject.jetty
 logger.jetty.level = warn
+
+logger.allocator.name = org.apache.arrow.memory.BaseAllocator
+logger.allocator.level = trace

Review Comment:
   1. Firstly, the `close` method of `BaseAllocator` inlines the `toString()` 
method for every exception.
   
   
https://github.com/apache/arrow-java/blob/0d296dffd39e14dbcf288c2da873777e08091cbd/memory/memory-core/src/main/java/org/apache/arrow/memory/BaseAllocator.java#L434-L521
   
   
![image](https://github.com/user-attachments/assets/6c66b8ef-8346-4661-b254-d5cb2eedd290)
   
   
   
   2. In the `toString()` method, if `logger.isTraceEnabled` is true, it will 
print `Verbosity.LOG_WITH_STACKTRACE`, which indicates that both 
`includeHistoricalLog` and `includeStackTraces` are true.
   
   
   
https://github.com/apache/arrow-java/blob/0d296dffd39e14dbcf288c2da873777e08091cbd/memory/memory-core/src/main/java/org/apache/arrow/memory/BaseAllocator.java#L524-L530
   
   
![image](https://github.com/user-attachments/assets/2738ad0d-4182-4bc4-8c1f-551b6aa0dbda)
   
   
   3. When both `includeHistoricalLog` and `includeStackTraces` are true, and 
`DEBUG` is also true (configured with `-Darrow.memory.debug.allocator=true`), 
it will print the resource allocation events recorded in the `HistoricalLog`.
   
   
https://github.com/apache/arrow-java/blob/0d296dffd39e14dbcf288c2da873777e08091cbd/memory/memory-core/src/main/java/org/apache/arrow/memory/BaseAllocator.java#L719-L765
   
   
![image](https://github.com/user-attachments/assets/3d7d19f4-2f80-4b1e-9840-70da634343f3)
   
   
   
https://github.com/apache/arrow-java/blob/a5b86049261530d41fbbdd23324420f0c1d14431/memory/memory-core/src/main/java/org/apache/arrow/memory/util/HistoricalLog.java#L115-L150
   
   
![image](https://github.com/user-attachments/assets/165059ac-64d9-4cfd-b92a-892aef18a2fa)
   
   
   Therefore, by setting the verbosity of the `toString()` method to 
`Verbosity.LOG_WITH_STACKTRACE`, the relevant information will be inlined into 
the error message. 
   
   This is equivalent to having `-Darrow.memory.debug.allocator=true` + 
`logger.isTraceEnabled` being true.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org