parthchandra commented on issue #517: URL: https://github.com/apache/datafusion-comet/issues/517#issuecomment-2153449766
Got it. I think the calls to xxhash64 native code are not being profiled successfully by async_profiler. Looks like the profiler loses track of the native call stack in generated code. I'll experiment and see if I can find some way to get more info. In the run above while most of the executor threads were in hash aggregation (when cpu usage was at its peak), some threads were in the LZ4 decompressor and some in the parquet reader both of which were probably adding to the CPU load. Also, the peak (CPU: 95%) existed for a very short while compared to the overall run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
