[ https://issues.apache.org/jira/browse/HIVE-26677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17633421#comment-17633421 ]
Chris Nauroth commented on HIVE-26677: -------------------------------------- [~szita] , thank you! > Constrain available processors to Jetty during test runs to prevent thread > exhaustion. > -------------------------------------------------------------------------------------- > > Key: HIVE-26677 > URL: https://issues.apache.org/jira/browse/HIVE-26677 > Project: Hive > Issue Type: Test > Components: Test > Reporter: Chris Nauroth > Assignee: Chris Nauroth > Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > As described during a [release candidate > vote|https://lists.apache.org/thread/8qjf7x9t9v09d79hlzh712ls4zthdwrh]: > HIVE-24484 introduced a change to limit {{hive.server2.webui.max.threads}} to > 4. Jetty enforces thread leasing to warn or abort if there aren't enough > threads available [1]. During startup, it attempts to lease a thread per NIO > selector [2]. By default, the number of NIO selectors to use is determined > based on available CPUs [3]. This is mostly a passthrough to > {{Runtime.availableProcessors()}} [4]. In my case, running on a machine with > 16 CPUs, this ended up creating more than 4 selectors, therefore requiring > more than 4 threads and violating the lease check. I was able to work around > this by passing the {{JETTY_AVAILABLE_PROCESSORS}} system property to > constrain the number of CPUs available to Jetty. > Since we are intentionally constraining the pool to 4 threads during itests, > let's also limit {{JETTY_AVAILABLE_PROCESSORS}} in {{maven.test.jvm.args}} of > the root pom.xml, so that others don't run into this problem later. > [1] > https://github.com/eclipse/jetty.project/blob/jetty-9.4.40.v20210413/jetty-util/src/main/java/org/eclipse/jetty/util/thread/ThreadPoolBudget.java#L165 > [2] > https://github.com/eclipse/jetty.project/blob/jetty-9.4.40.v20210413/jetty-io/src/main/java/org/eclipse/jetty/io/SelectorManager.java#L255 > [3] > https://github.com/eclipse/jetty.project/blob/jetty-9.4.40.v20210413/jetty-io/src/main/java/org/eclipse/jetty/io/SelectorManager.java#L79 > [4] > https://github.com/eclipse/jetty.project/blob/jetty-9.4.40.v20210413/jetty-util/src/main/java/org/eclipse/jetty/util/ProcessorUtils.java#L45 -- This message was sent by Atlassian Jira (v8.20.10#820010)