Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/21698 )
Change subject: IMPALA-13186: Tag query option scope for tuple cache ...................................................................... Patch Set 16: > (6 comments) > > > Patch Set 11: > > > > (1 comment) > > > > Let's say that we have low cache hit rates. Is there a way we > could tell whether it is related to different query options? Do we > want a way to tune this at startup (i.e. a list of additional query > options to exempt)? > > > > If we wanted an approximation of how many distinct query option > hashes there are, we could maintain an HLL sketch. > > I don't think it makes sense as a startup option, that could lead > to selecting options that aren't actually safe. > > I don't see a good way to track whether low hit rate is related to > query options without grabbing a bunch of queries. Other options > would require tracking the query options cache key as a separate > component and making it part of the cache key, with a metric we > update based on misses. And it would make the cache more > complicated to identify that a miss was specifically due to the > query options cache key. I think reviewing profiles will be > sufficient. If we diagnose low hit rate to be related to differences in query options and those differences are not related to correctness, the startup option would give us a way to workaround it without a new build / release. Someone could use it to blow themselves up, but it also could save someone a lot of hassle waiting for a new build / release. If you don't want to do that in this change, we can file a separate JIRA. -- To view, visit http://gerrit.cloudera.org:8080/21698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1f4802ad9548749cd43df8848b6f46dca3739ae7 Gerrit-Change-Number: 21698 Gerrit-PatchSet: 16 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Yida Wu <[email protected]> Gerrit-Comment-Date: Tue, 08 Oct 2024 00:06:14 +0000 Gerrit-HasComments: No
