David Rorke has posted comments on this change. ( http://gerrit.cloudera.org:8080/21277 )
Change subject: IMPALA-12988: Calculate an unbounded version of CpuAsk ...................................................................... Patch Set 10: (2 comments) http://gerrit.cloudera.org:8080/#/c/21277/10/fe/src/main/java/org/apache/impala/service/Frontend.java File fe/src/main/java/org/apache/impala/service/Frontend.java: http://gerrit.cloudera.org:8080/#/c/21277/10/fe/src/main/java/org/apache/impala/service/Frontend.java@2322 PS10, Line 2322: scaledCoreReqToCompare = Math.max(scaledCpuAskBounded, scaledCpuAskUnbounded); I'm a little concerned that using the max here will allow the EG size to exceed the sublinear capping that we're trying to do and we might still get excessively large EG assignments in some cases even with scaledCpuAskBounded. Ideally we'd simplify this and just use the sublinear scaled unbounded value for both the EG selection and the actual fragment instance count computation, but I understand this would require a lot of changes elsewhere (not as simple as removing this max comparison). I think the current code should handle the worst cases of runaway EG assignments so I'm OK with this at this stage but we should plan some future work to simplify this. http://gerrit.cloudera.org:8080/#/c/21277/10/fe/src/main/java/org/apache/impala/service/Frontend.java@2442 PS10, Line 2442: sqrtSmallestEGTotalCores Maybe call this nthrootSmallestEGTotalCores -- To view, visit http://gerrit.cloudera.org:8080/21277 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5441e31088f90761062af35862be4ce09d116923 Gerrit-Change-Number: 21277 Gerrit-PatchSet: 10 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: David Rorke <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Comment-Date: Wed, 17 Apr 2024 16:41:55 +0000 Gerrit-HasComments: Yes
