gortiz commented on PR #15967: URL: https://github.com/apache/pinot/pull/15967#issuecomment-2933627063
> how long running were the queries that caused this? (what was their configured timeout) They were running for less than 1 second. But the issue is not on their execution time but the fact that we keep a reference to the multi stage operator. Some of these operators keep a reference to large maps (ie joins and aggregates), which is what somehow _leaks_ memory. This simple PR fixes that by cleaning up successful executions, but reduces precision on the stats and doesn't protect us against queries that fail. I'm planning to change the code so that it does not keep the reference to multi-stage operators once the opchain finishes. But I think we should also release these large maps once the operator is no longer going to use them. I've been thinking about other places where we reference operators longer than expected, and the only case I've found is the new physical planner, which can generate larger stages that contain two joins. That means that the right side table of one of the joins will be kept referenced even after the EOS was sent from that operator. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
