[jira] [Commented] (SOLR-17589) First use of PUT/POST request with HttpJdkSolrClient generates error log entry on solr server due to initial HEAD request
[
https://issues.apache.org/jira/browse/SOLR-17589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17905810#comment-17905810
]
Paul Blanchaert commented on SOLR-17589:
I'll try to find some time this week.
> First use of PUT/POST request with HttpJdkSolrClient generates error log
> entry on solr server due to initial HEAD request
> -
>
> Key: SOLR-17589
> URL: https://issues.apache.org/jira/browse/SOLR-17589
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: clients - java
>Affects Versions: 9.7
>Reporter: Paul Blanchaert
>Priority: Minor
>
> The first call of the maybeTryHeadRequest of the HttpJdkSolrClient (used in
> the preparePutOrPost) causes an ERROR log entry in the solr server log even
> while the "HEAD" request is considered successful in the client.
> The client only performs this request once.
> The error on server side is due to the "empty" request: "missing content
> stream"
> 2024-12-10 08:31:48 2024-12-10 07:31:48.877 INFO (qtp1380924218-23-solr-123)
> [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1
> t:solr-123] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update
> params={}{} 0 0
> 2024-12-10 08:31:48 2024-12-10 07:31:48.877 ERROR (qtp1380924218-23-solr-123)
> [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1
> t:solr-123] o.a.s.h.RequestHandlerBase Client exception =>
> org.apache.solr.common.SolrException: missing content stream
> 2024-12-10 08:31:48 at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:93)
> 2024-12-10 08:31:48 org.apache.solr.common.SolrException: missing content
> stream
> 2024-12-10 08:31:48 at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:93)
> ~[?:?]
> 2024-12-10 08:31:48 at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:226)
> ~[?:?]
> 2024-12-10 08:31:48 at
> org.apache.solr.core.SolrCore.execute(SolrCore.java:2880) ~[?:?]
> 2024-12-10 08:31:48 at
> org.apache.solr.servlet.HttpSolrCall.executeCoreRequest(HttpSolrCall.java:890)
> ~[?:?]
> 2024-12-10 08:31:48 at
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:576) ~[?:?]
> 2024-12-10 08:31:48 at
> org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:251)
> ~[?:?]
> 2024-12-10 08:31:48 at
> org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilter$0(SolrDispatchFilter.java:208)
> ~[?:?]
> 2024-12-10 08:31:48 at
> org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:243)
> ~[?:?]
> 2024-12-10 08:31:48 at
> org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:213)
> ~[?:?]
> 2024-12-10 08:31:48 at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:202)
> ~[?:?]
> 2024-12-10 08:31:48 at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
> ~[?:?]
> 2024-12-10 08:31:48 at
> org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:210)
> ~[jetty-servlet-10.0.22.jar:10.0.22]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17595: Fix Solr examples for Windows [solr]
malliaridis commented on PR #2909: URL: https://github.com/apache/solr/pull/2909#issuecomment-2544140574 Yeah, I messed up the PR by pushing the first fix first in main. So I have to make sure its included in the backport. I will merge by tomorrow if there is no objection by then. I have only tested the techproducts, so before I merge I will look into the other examples as well to make sure they all work properly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17405) Zookeeper session can be re-established by multiple threads concurrently
[
https://issues.apache.org/jira/browse/SOLR-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17905831#comment-17905831
]
Jan Høydahl commented on SOLR-17405:
I saw the pr for this issue being auto closed, without any discussion or
review. Do you need someone to review your pr?
The fix looks solid.
Guess it is hard to write a test for it?
> Zookeeper session can be re-established by multiple threads concurrently
>
>
> Key: SOLR-17405
> URL: https://issues.apache.org/jira/browse/SOLR-17405
> Project: Solr
> Issue Type: Bug
>Affects Versions: 8.11, 9.6
>Reporter: Pierre Salagnac
>Priority: Major
> Labels: pull-request-available
> Attachments: stack.png
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Because of a bug in SolrCloud, the Zookeeper session can be re-established by
> multiple threads concurrently when an expiration occurs.
> This portion of the code assumes it is mono-threaded. Because of the bug, the
> last thread re-establishing the session can waif for 30 seconds per core,
> waiting for it to be marked {{DOWN}} while it was previously marked
> {{ACTIVE}} by another thread. With a high number of cores, the Solr server
> can hang for hours before taking traffic again.
> Following exception shows two threads were reestablishing the session
> concurrently. {{ZkController.createEphemeralLiveNode()}} should never be
> invoked twice for the same Zookeeper session.
> {code:java}
> thrown: java. lang.RuntimeException:
> org.apache.solr.common.cloud.ZooKeeperException:
> at
> org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:178)
> at org.apache.solr. common.cloud.DefaultConnectionStrategy.
> reconnect(DefaultConnectionStrategy.java:57)
> at
> org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:152)
> at org.apache.
> zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510)
> Caused by: org.apache.solr.common.cloud.ZooKeeperException:
> at
> org.apache.solr.cloud.ZkController$1.command(ZkController.java:462)
> at
> org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:170)
> ... 4 more
> Caused by: org.apache. zookeeper.KeeperException$NodeExistsException.
> KeeperErrorCode = NodeExists
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:
> 126)
> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:1925)
> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:1830)
> at
> org.apache.solr.common.cloud.SolrZkClient.lambda$multi$11(SolrZkClient.java:666)
> at
> org.apache.solr.common.cloud.ZkCmdExecutor.retry0peration(ZkCmdExecutor.java:71)
> at
> org.apache.solr.common.cloud.SolrZkClient.multi(SolrZkClient.java:666)
> at org.apache.sol.cloud.ZkController
> CreateEphemeralLiveNode(ZkController.java:1086)
> at
> org.apache.solr.cloud.ZkController$1.command(ZkController.java:411)
> ... 5 more {code}
> h2. Root cause
> This bug occurs because several threads can re-establish the session
> concurrently.
> It cannot happen at the first expiration of the session, thanks to a thread
> pool with a single thread to execute the zookeeper Watcher.
> Bellow is a code snippet from class {{SolrZkClient.ProcessWatchWithExecutor}}
> {code:java}
> if (watcher instanceof ConnectionManager) {
> zkConnManagerCallbackExecutor.submit(() -> watcher.process(event));
> } else {
>...
> }
> {code}
> Using this dedicated thread pool (with a single thread) is supposed to ensure
> we don’t handle watches for connection related events with multiple threads.
> This works well for the first session expiration.
> Now, when we re-establish the session after the first expiration, we don’t
> use this wrapper to register the watch.
> It is done directly in {{ConnectionManager}} without wrapping the ZK watch.
> In the following snippet, _“this”_ is the ZK watcher instance, but it is not
> wrapper to use a {{{}ProcessWatchWithExecutor{}}}. This means the next events
> will directly be handled by any ZK callback thread.
> {code:java}
> connectionStrategy.reconnect(zkServerAddress,client.getZkClientTimeout(),
> this,
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17541: LBSolrClient implementations should agree on 'getClient()' semantics [solr]
dsmiley commented on code in PR #2899:
URL: https://github.com/apache/solr/pull/2899#discussion_r1885757147
##
solr/core/src/java/org/apache/solr/core/HttpSolrClientProvider.java:
##
Review Comment:
WDYT @iamsanjay ?
##
solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java:
##
@@ -305,16 +306,16 @@ public void init(PluginInfo info) {
sb);
int soTimeout =
getParameter(args, HttpClientUtil.PROP_SO_TIMEOUT,
HttpClientUtil.DEFAULT_SO_TIMEOUT, sb);
-
-this.defaultClient =
+this.httpClientBuilder =
new Http2SolrClient.Builder()
.withConnectionTimeout(connectionTimeout, TimeUnit.MILLISECONDS)
.withIdleTimeout(soTimeout, TimeUnit.MILLISECONDS)
.withExecutor(commExecutor)
.withMaxConnectionsPerHost(maxConnectionsPerHost)
-.build();
-this.defaultClient.addListenerFactory(this.httpListenerFactory);
-this.loadbalancer = new
LBHttp2SolrClient.Builder(defaultClient).build();
+.withListenerFactory(List.of(this.httpListenerFactory));
Review Comment:
I noticed an issue. This overwrites the list of listeners that may already
be present. Shouldn't we be augmenting, not replacing?
##
solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java:
##
@@ -305,16 +306,16 @@ public void init(PluginInfo info) {
sb);
int soTimeout =
getParameter(args, HttpClientUtil.PROP_SO_TIMEOUT,
HttpClientUtil.DEFAULT_SO_TIMEOUT, sb);
-
-this.defaultClient =
+this.httpClientBuilder =
Review Comment:
please name `httpSolrClientBuilder` -- so as to clarify this isn't an
Apache/Jetty HttpClient; it's a SolrClient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17541: LBSolrClient implementations should agree on 'getClient()' semantics [solr]
janhoy commented on PR #2899: URL: https://github.com/apache/solr/pull/2899#issuecomment-2544063669 > `PKIAuthenticationPlugin` as it apparently has a pivotal role, even when there's know "PKI" going on! PKI plugin handles incoming internal requests from other nodes in the cluster. Then there is indeed PKI going on 😉 It is registered whenever any auth plugin is active. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17548: Switch all public Java APIs from File to Path [solr]
dsmiley commented on code in PR #2907:
URL: https://github.com/apache/solr/pull/2907#discussion_r1885858097
##
solr/core/src/java/org/apache/solr/spelling/suggest/fst/AnalyzingInfixLookupFactory.java:
##
@@ -108,7 +109,7 @@ public Lookup create(NamedList params, SolrCore core) {
try {
return new AnalyzingInfixSuggester(
- FSDirectory.open(new File(indexPath).toPath()),
Review Comment:
Wow that line of code was sad.
##
solr/core/src/java/org/apache/solr/util/FileUtils.java:
##
@@ -37,14 +36,14 @@ public class FileUtils {
* If path is absolute, then a File for that path is returned; if it's not
absolute, then a File
* is returned using "path" as a child of "base")
*/
- public static File resolvePath(File base, String path) {
-File r = new File(path);
-return r.isAbsolute() ? r : new File(base, path);
+ public static Path resolvePath(Path base, String path) {
+Path r = Path.of(path);
+return r.isAbsolute() ? r : Path.of(base.toString(), path);
}
- public static void copyFile(File src, File destination) throws IOException {
-try (FileChannel in = new FileInputStream(src).getChannel();
-FileChannel out = new FileOutputStream(destination).getChannel()) {
+ public static void copyFile(Path src, Path destination) throws IOException {
Review Comment:
The only reason this method exists, is because the inputs were File and not
Path. See Files.copy(path, path). So at this point, you can just write that
one-liner then inline this method (IntelliJ will do for you). IntelliJ kicks
but at refactoring; perhaps Eclipse does fine too; I dunno.
##
solr/core/src/java/org/apache/solr/handler/IndexFetcher.java:
##
@@ -1518,12 +1521,14 @@ private void copyTmpConfFiles2Conf(Path tmpconfDir) {
* backup of the old directory is maintained. If the directory move fails,
it will try to revert
* back the original tlog directory.
*/
- private boolean copyTmpTlogFiles2Tlog(File tmpTlogDir) {
+ private boolean copyTmpTlogFiles2Tlog(Path tmpTlogDir) {
Path tlogDir =
FileSystems.getDefault().getPath(solrCore.getUpdateHandler().getUpdateLog().getTlogDir());
Path backupTlogDir =
FileSystems.getDefault()
-.getPath(tlogDir.getParent().toAbsolutePath().toString(),
tmpTlogDir.getName());
+.getPath(
+tlogDir.getParent().toAbsolutePath().toString(),
+tmpTlogDir.getFileName().toString());
Review Comment:
This looks more complicated than it needs to be. It's highly unusual for
anyone to need to refer to FileSystems. Look at the implementation of
`Path.of(...)` which proves we can just use Path.of here.
But hey, copyTmpTlogFiles2Tlog isn't used at all so let's just delete it!
##
solr/modules/analysis-extras/src/test/org/apache/solr/schema/TestICUCollationFieldOptions.java:
##
@@ -26,7 +26,7 @@ public class TestICUCollationFieldOptions extends
SolrTestCaseJ4 {
@BeforeClass
public static void beforeClass() throws Exception {
File testHome = createTempDir().toFile();
-FileUtils.copyDirectory(getFile("analysis-extras/solr"), testHome);
+FileUtils.copyDirectory(getFile("analysis-extras/solr").toFile(),
testHome);
Review Comment:
(same as my last comment)
##
solr/core/src/java/org/apache/solr/rest/ManagedResourceStorage.java:
##
@@ -172,16 +172,23 @@ public void configure(SolrResourceLoader loader,
NamedList initArgs)
throw new IllegalArgumentException(
"Required configuration parameter '" + STORAGE_DIR_INIT_ARG + "'
not provided!");
- File dir = new File(storageDirArg);
- if (!dir.isDirectory()) dir.mkdirs();
+ Path dir = Path.of(storageDirArg);
+ if (!Files.isDirectory(dir)) {
+try {
+ Files.createDirectories(dir);
+} catch (IOException e) {
+ throw new SolrException(
+ SolrException.ErrorCode.SERVER_ERROR, "Unable to create storage
directory " + dir, e);
+}
+ }
- storageDir = dir.getAbsolutePath();
+ storageDir = dir.toAbsolutePath().toString();
log.info("File-based storage initialized to use dir: {}", storageDir);
}
@Override
public boolean exists(String storedResourceId) throws IOException {
- return (new File(storageDir, storedResourceId)).exists();
+ return (Files.exists(Path.of(storageDir, storedResourceId)));
Review Comment:
there's a needless parenthesis here
##
solr/core/src/java/org/apache/solr/core/StandardDirectoryFactory.java:
##
@@ -78,7 +77,7 @@ protected LockFactory createLockFactory(String rawLockType)
throws IOException {
@Override
public String normalize(String path) throws IOException {
-return super.normalize(new File(path).getCanonicalPath());
+return super.normalize(Path.of(path).toAbsolutePath().toString());
Review Comment:
