Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21698 )
Change subject: IMPALA-13186: Tag query option scope for tuple cache ...................................................................... Patch Set 16: > Patch Set 16: > > > (6 comments) > > > > > Patch Set 11: > > > > > > (1 comment) > > > > > > Let's say that we have low cache hit rates. Is there a way we > > could tell whether it is related to different query options? Do we > > want a way to tune this at startup (i.e. a list of additional query > > options to exempt)? > > > > > > If we wanted an approximation of how many distinct query option > > hashes there are, we could maintain an HLL sketch. > > > > I don't think it makes sense as a startup option, that could lead > > to selecting options that aren't actually safe. > > > > I don't see a good way to track whether low hit rate is related to > > query options without grabbing a bunch of queries. Other options > > would require tracking the query options cache key as a separate > > component and making it part of the cache key, with a metric we > > update based on misses. And it would make the cache more > > complicated to identify that a miss was specifically due to the > > query options cache key. I think reviewing profiles will be > > sufficient. > > If we diagnose low hit rate to be related to differences in query options and > those differences are not related to correctness, the startup option would > give us a way to workaround it without a new build / release. Someone could > use it to blow themselves up, but it also could save someone a lot of hassle > waiting for a new build / release. > > If you don't want to do that in this change, we can file a separate JIRA. I can add a hidden flag. -- To view, visit http://gerrit.cloudera.org:8080/21698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1f4802ad9548749cd43df8848b6f46dca3739ae7 Gerrit-Change-Number: 21698 Gerrit-PatchSet: 16 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Yida Wu <[email protected]> Gerrit-Comment-Date: Tue, 08 Oct 2024 16:50:07 +0000 Gerrit-HasComments: No
