TessaIO opened a new issue, #18411: URL: https://github.com/apache/druid/issues/18411
### Affected Version 34.0.0 ### Description After upgrading one of our Druid test clusters to 34.0.0, we found that all components (routers, brokers, historicals), failed to communicate with the coordinator, and the whole cluster was down. We see the following error on these components, while there are no issues with the coordinator logs: ``` 2025-08-18T11:50:57,208 WARN [CoordinatorPollingBasicAuthorizerCacheManager-Exec--0] org.apache.druid.java.util.common.RetryUtils - Retrying (9 of 9) in 50,867ms. org.apache.druid.java.util.common.IOE: Retries exhausted, couldn't fulfill request to [http://druid-sb4-coordinators-76dcdd9955-x8z8q:8088/druid-ext/basic-security/authorization/db/BasicMetadataAuthor at org.apache.druid.discovery.DruidLeaderClient.go(DruidLeaderClient.java:219) ~[druid-server-34.0.0.jar:34.0.0] at org.apache.druid.security.basic.authorization.db.cache.CoordinatorPollingBasicAuthorizerCacheManager.tryFetchGroupMappingMapsFromCoordinator(CoordinatorPollingBasicAuthorizerCacheManager.java:4 at org.apache.druid.security.basic.authorization.db.cache.CoordinatorPollingBasicAuthorizerCacheManager.lambda$fetchGroupAndRoleMapFromCoordinator$6(CoordinatorPollingBasicAuthorizerCacheManager.j at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:129) ~[druid-processing-34.0.0.jar:34.0.0] at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:81) ~[druid-processing-34.0.0.jar:34.0.0] at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:163) ~[druid-processing-34.0.0.jar:34.0.0] at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:153) ~[druid-processing-34.0.0.jar:34.0.0] at org.apache.druid.security.basic.authorization.db.cache.CoordinatorPollingBasicAuthorizerCacheManager.fetchGroupAndRoleMapFromCoordinator(CoordinatorPollingBasicAuthorizerCacheManager.java:367) at org.apache.druid.security.basic.authorization.db.cache.CoordinatorPollingBasicAuthorizerCacheManager.lambda$start$1(CoordinatorPollingBasicAuthorizerCacheManager.java:159) ~[?:?] at org.apache.druid.java.util.common.concurrent.ScheduledExecutors$1.call(ScheduledExecutors.java:56) [druid-processing-34.0.0.jar:34.0.0] at org.apache.druid.java.util.common.concurrent.ScheduledExecutors$1.call(ScheduledExecutors.java:52) [druid-processing-34.0.0.jar:34.0.0] at org.apache.druid.java.util.common.concurrent.ScheduledExecutors$2.run(ScheduledExecutors.java:88) [druid-processing-34.0.0.jar:34.0.0] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] at java.base/java.lang.Thread.run(Thread.java:840) [?:?] ``` It looks like it's related to the basic auth configuration. This is the Auth Config: ``` druid.auth.authenticatorChain=["BasicMetadataAuthenticator"] druid.auth.authenticator.BasicMetadataAuthenticator.type=basic # Default password for 'admin' user druid.auth.authenticator.BasicMetadataAuthenticator.initialAdminPassword=<redacted> # Default password for internal 'druid_system' user druid.auth.authenticator.BasicMetadataAuthenticator.initialInternalClientPassword=<redacted> # Uses the metadata store for storing users, you can use authentication API to create new users and grant permissions druid.auth.authenticator.BasicMetadataAuthenticator.credentialsValidator.type=metadata # If true and the request credential doesn't exists in this credentials store, the request will proceed to next Authenticator in the chain. druid.auth.authenticator.BasicMetadataAuthenticator.skipOnFailure=true druid.auth.authenticator.BasicMetadataAuthenticator.authorizerName=BasicMetadataAuthorizer druid.auth.authorizers=["BasicMetadataAuthorizer"] druid.auth.authorizer.BasicMetadataAuthorizer.type=basic ``` Let me know if you need any specific configuration -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
