Re: [PR] SOLR-17582 Stream CLUSTERSTATUS API response [solr]
dsmiley commented on code in PR #2916:
URL: https://github.com/apache/solr/pull/2916#discussion_r1890699859
##
solr/core/src/java/org/apache/solr/handler/admin/ClusterStatus.java:
##
@@ -368,4 +341,77 @@ public static Map
postProcessCollectionJSON(Map
collection.put("health", Health.combine(healthStates).toString());
return collection;
}
+
+ private class SolrCollectionProperiesIterator implements
Iterator> {
+
+final Iterator it;
+Map> collectionVsAliases;
+String routeKey;
+List liveNodes;
+String shard;
+
+public SolrCollectionProperiesIterator(
+Iterator it,
+Map> collectionVsAliases,
+String routeKey,
+List liveNodes,
+String shard) {
+ this.it = it;
+ this.collectionVsAliases = collectionVsAliases;
+ this.routeKey = routeKey;
+ this.liveNodes = liveNodes;
+ this.shard = shard;
+}
+
+@Override
+public boolean hasNext() {
+ return it.hasNext();
+}
+
+@Override
+public NamedList next() {
+ NamedList collectionProps = new SimpleOrderedMap<>();
+ DocCollection clusterStateCollection = it.next();
+ Map collectionStatus;
+ String name = clusterStateCollection.getName();
+
+ Set requestedShards = new HashSet<>();
+ if (routeKey != null) {
+DocRouter router = clusterStateCollection.getRouter();
+Collection slices = router.getSearchSlices(routeKey, null,
clusterStateCollection);
+for (Slice slice : slices) {
+ requestedShards.add(slice.getName());
+}
+ }
+ if (shard != null) {
+String[] paramShards = shard.split(",");
+requestedShards.addAll(Arrays.asList(paramShards));
+ }
Review Comment:
ideally this is done up front; not per iteration
##
solr/core/src/java/org/apache/solr/handler/admin/ClusterStatus.java:
##
@@ -368,4 +341,77 @@ public static Map
postProcessCollectionJSON(Map
collection.put("health", Health.combine(healthStates).toString());
return collection;
}
+
+ private class SolrCollectionProperiesIterator implements
Iterator> {
Review Comment:
I was hoping we wouldn't need a custom Iterator. We do need a method that
takes in DocCollection (and some other context that is the same across
collections) and returns a NamedList. With such a method, we can call it via
streamOfDocCollection.map(collState -> theMethod(collState, routeKey,
liveNodes, etc.).iterator()
##
solr/solrj/src/java/org/apache/solr/client/solrj/impl/BaseHttpClusterStateProvider.java:
##
@@ -140,11 +139,11 @@ private ClusterState fetchClusterState(SolrClient client)
liveNodesTimestamp = System.nanoTime();
}
-var collectionsNl = (NamedList>)
cluster.get("collections");
+var collectionsNl = (Map>)
cluster.get("collections");
Review Comment:
this var name is no longer appropriate. Should be collectionsMap (and we
assume by name).
##
solr/solrj/src/java/org/apache/solr/client/solrj/impl/BaseHttpClusterStateProvider.java:
##
@@ -185,7 +184,9 @@ private DocCollection fetchCollectionState(SolrClient
client, String collection)
SimpleOrderedMap cluster =
submitClusterStateRequest(client, collection,
ClusterStateRequestType.FETCH_COLLECTION);
-var collStateMap = (Map)
cluster.findRecursive("collections", collection);
+var collState = (Map) cluster.findRecursive("collections");
Review Comment:
so findRecursive can only resolve collections and not Map? Seems worth a
small enhancement to do so but I understand if you don't bother or defer.
##
solr/core/src/java/org/apache/solr/handler/admin/ClusterStatus.java:
##
@@ -368,4 +341,77 @@ public static Map
postProcessCollectionJSON(Map
collection.put("health", Health.combine(healthStates).toString());
return collection;
}
+
+ private class SolrCollectionProperiesIterator implements
Iterator> {
+
+final Iterator it;
+Map> collectionVsAliases;
+String routeKey;
+List liveNodes;
+String shard;
+
+public SolrCollectionProperiesIterator(
+Iterator it,
+Map> collectionVsAliases,
+String routeKey,
+List liveNodes,
+String shard) {
+ this.it = it;
+ this.collectionVsAliases = collectionVsAliases;
+ this.routeKey = routeKey;
+ this.liveNodes = liveNodes;
+ this.shard = shard;
+}
+
+@Override
+public boolean hasNext() {
+ return it.hasNext();
+}
+
+@Override
+public NamedList next() {
+ NamedList collectionProps = new SimpleOrderedMap<>();
+ DocCollection clusterStateCollection = it.next();
+ Map collectionStatus;
+ String name = clusterStateCollection.getName();
+
+ Set requestedShards = new HashSet<>();
+ if (routeKey != null) {
+DocRouter router = clusterStateCollection.getRouter();
+Collection slices = router.getSear
[jira] [Updated] (SOLR-17598) Windows Script logic that handles SOLR_LOGS_DIR is faulty
[
https://issues.apache.org/jira/browse/SOLR-17598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christos Malliaridis updated SOLR-17598:
Description:
On Windows, when setting the {{SOLR_LOGS_DIR}} in the environment variables, it
prevents Solr from starting. Additionally, if the variable contains a space,
trying to stop a running Solr service with {{bin\solr.cmd stop --all}} fails.
I have also observed situations where {{SOLR_LOGS_DIR}} in environment
variables is simply ignored, but I am not sure if this is still preproducable.
h3. Steps to reproduce
1. Set the environment variable {{SOLR_LOGS_DIR}} to an absolute path that does
or does not exist
2. ensure that in your current terminal the envrionment variable you set is
present via "echo %SOLR_LOGS_DIR%"
3. Launch Solr with {{bin\solr.cmd start}}
h3. Expected Behavior
Setting the environment variable is supported and should not make a difference
with setting the value in solr.in.cmd
h3. Actual Behavior
The environment variable is not supported.
h3. Workaround
Setting {{SOLR_LOGS_DIR}} via the {{solr.in.cmd}} file (which overrides the
environment variable) works and avoids the issues. Spaces are also supported
this way.
h3. Additional Information
This behavior was tested for the {{{}SOLR_LOGS_DIR{}}}, but it is not
guaranteed that other environment variables work. This may affect all
environment variables on Windows.
was:
h3. Description
On Windows, when setting the {{SOLR_LOGS_DIR}} in the environment variables, it
prevents Solr from starting. Additionally, if the variable contains a space,
trying to stop a running Solr service with {{bin\solr.cmd stop --all}} fails.
I have also observed situations where {{SOLR_LOGS_DIR}} in environment
variables is simply ignored, but I am not sure if this is still preproducable.
h3. Steps to reproduce
1. Set the environment variable {{SOLR_LOGS_DIR}} to an absolute path that does
or does not exist
2. ensure that in your current terminal the envrionment variable you set is
present via "echo %SOLR_LOGS_DIR%"
3. Launch Solr with {{bin\solr.cmd start}}
h3. Expected Behavior
Setting the environment variable is supported and should not make a difference
with setting the value in solr.in.cmd
h3. Actual Behavior
The environment variable is not supported.
h3. Workaround
Setting {{SOLR_LOGS_DIR}} via the {{solr.in.cmd}} file (which overrides the
environment variable) works and avoids the issues. Spaces are also supported
this way.
h3. Workaround
This behavior was tested for the {{SOLR_LOGS_DIR}}, but it is not guaranteed
that other environment variables work. This may affect all environment
variables on Windows.
> Windows Script logic that handles SOLR_LOGS_DIR is faulty
> -
>
> Key: SOLR-17598
> URL: https://issues.apache.org/jira/browse/SOLR-17598
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: cli
>Affects Versions: 9x, 9.8
> Environment: Windows 11
>Reporter: Christos Malliaridis
>Priority: Major
>
> On Windows, when setting the {{SOLR_LOGS_DIR}} in the environment variables,
> it prevents Solr from starting. Additionally, if the variable contains a
> space, trying to stop a running Solr service with {{bin\solr.cmd stop --all}}
> fails.
> I have also observed situations where {{SOLR_LOGS_DIR}} in environment
> variables is simply ignored, but I am not sure if this is still preproducable.
> h3. Steps to reproduce
> 1. Set the environment variable {{SOLR_LOGS_DIR}} to an absolute path that
> does or does not exist
> 2. ensure that in your current terminal the envrionment variable you set is
> present via "echo %SOLR_LOGS_DIR%"
> 3. Launch Solr with {{bin\solr.cmd start}}
> h3. Expected Behavior
> Setting the environment variable is supported and should not make a
> difference with setting the value in solr.in.cmd
> h3. Actual Behavior
> The environment variable is not supported.
> h3. Workaround
> Setting {{SOLR_LOGS_DIR}} via the {{solr.in.cmd}} file (which overrides the
> environment variable) works and avoids the issues. Spaces are also supported
> this way.
> h3. Additional Information
> This behavior was tested for the {{{}SOLR_LOGS_DIR{}}}, but it is not
> guaranteed that other environment variables work. This may affect all
> environment variables on Windows.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17525: Text Embedder Query Parser [solr]
malliaridis commented on PR #2809: URL: https://github.com/apache/solr/pull/2809#issuecomment-2551851228 Please note in case this is backported to 9.X that you will have to use the `/versions.props`for the dependencies and also refactor the gradle file so that it uses the previous notation, which is string-based (like `implementation 'dependency.group:dependency.name'`). Unluckily, changes between v9 and v10 in the dependency parts are quite annoying, which is why I would like to push for the v10 release. If I can be of any help in that part just let me know. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17286) HttpSolrCall.remoteQuery (proxy) should use Jetty HttpClient
[ https://issues.apache.org/jira/browse/SOLR-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906859#comment-17906859 ] David Smiley commented on SOLR-17286: - I also strongly want to rename this. "remoteQuery" is misleading – isn't necessarily a "query" at all. The word "proxy" should be used maybe "remoteProxy". > HttpSolrCall.remoteQuery (proxy) should use Jetty HttpClient > > > Key: SOLR-17286 > URL: https://issues.apache.org/jira/browse/SOLR-17286 > Project: Solr > Issue Type: Sub-task >Reporter: David Smiley >Priority: Major > Labels: newdev > > There is Apache HttpClient usage in HttpSolrCall to do a "remote query". It > should switch to Jetty HttpClient. > Looking at the code details, I don't think > org.apache.solr.servlet.CoreContainerProvider#httpClient (the field and > getter here) needs to exist. It's just a reference but it can be looked up > on-demand from the CoreContainer with less ceremony. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Assigned] (SOLR-17286) HttpSolrCall.remoteQuery (proxy) should use Jetty HttpClient
[ https://issues.apache.org/jira/browse/SOLR-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley reassigned SOLR-17286: --- Assignee: David Smiley > HttpSolrCall.remoteQuery (proxy) should use Jetty HttpClient > > > Key: SOLR-17286 > URL: https://issues.apache.org/jira/browse/SOLR-17286 > Project: Solr > Issue Type: Sub-task >Reporter: David Smiley >Assignee: David Smiley >Priority: Major > Labels: newdev > > There is Apache HttpClient usage in HttpSolrCall to do a "remote query". It > should switch to Jetty HttpClient. > Looking at the code details, I don't think > org.apache.solr.servlet.CoreContainerProvider#httpClient (the field and > getter here) needs to exist. It's just a reference but it can be looked up > on-demand from the CoreContainer with less ceremony. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Resolved] (SOLR-17414) multiThreaded=true can result in RejectedExecutionException
[ https://issues.apache.org/jira/browse/SOLR-17414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley resolved SOLR-17414. - Fix Version/s: 9.8 Resolution: Fixed > multiThreaded=true can result in RejectedExecutionException > --- > > Key: SOLR-17414 > URL: https://issues.apache.org/jira/browse/SOLR-17414 > Project: Solr > Issue Type: Bug >Reporter: David Smiley >Assignee: David Smiley >Priority: Major > Labels: pull-request-available > Fix For: 9.8 > > Attachments: build-out2.txt > > Time Spent: 0.5h > Remaining Estimate: 0h > > Since the new multiThreaded search feature landed, I see a new test > failure involving "RejectedExecutionException" being thrown > [link|https://ge.apache.org/s/5ack462ji4mlu/tests/task/:solr:core:test/details/org.apache.solr.search.TestRealTimeGet/testStressGetRealtime?top-execution=1]. > It is thrown at a low level in Lucene building TermStates > concurrently. I doubt the problem is specific to that test > (TestRealTimeGet) but that test might induce more activity than most > tests, thus crossing some thresholds like the queue size -- apparently > 1000. > *I don't think we should be throwing a RejectedExecutionException > when running a Search query* -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Assigned] (SOLR-17560) DocCollection, don't need getLeaderReplicas(nodeName)
[ https://issues.apache.org/jira/browse/SOLR-17560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley reassigned SOLR-17560: --- Assignee: (was: David Smiley) > DocCollection, don't need getLeaderReplicas(nodeName) > - > > Key: SOLR-17560 > URL: https://issues.apache.org/jira/browse/SOLR-17560 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Reporter: David Smiley >Priority: Minor > Labels: newdev > Fix For: main (10.0) > > > Claim: DocCollection.getLeaderReplicas(nodeName) isn't needed. > Goal: reduce complexity/bookkeeping and weight of DocCollection > In non-test code, it's only needed by > org.apache.solr.servlet.HttpSolrCall#getCoreByCollection for an indexing > request. This could be modified to randomly iterate all replicas and filter > leaders afterwards instead of before. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-15751: Modify v2 `GET /collections/collName` to support full ColStatus response [solr]
dsmiley commented on code in PR #2912:
URL: https://github.com/apache/solr/pull/2912#discussion_r1890892532
##
solr/api/src/java/org/apache/solr/client/api/model/CollectionStatusResponse.java:
##
@@ -82,21 +97,93 @@ public static class LeaderSummary {
// Present with coreInfo=true || sizeInfo=true unless otherwise specified
@JsonProperty public SegmentInfo segInfos;
+
+private Map unknownFields = new HashMap<>();
+
+@JsonAnyGetter
+public Map unknownProperties() {
+ return unknownFields;
+}
+
+@JsonAnySetter
+public void setUnknownProperty(String field, Object value) {
+ unknownFields.put(field, value);
+}
}
- // Present with coreInfo=true || sizeInfo=true unless otherwise specified
+ // Present with segments=true || coreInfo=true || sizeInfo=true ||
fieldInfo=true unless otherwise
+ // specified
public static class SegmentInfo {
-// Present with coreInfo=true || sizeInfo=true unless otherwise specified
@JsonProperty public SegmentSummary info;
+// Present with segments=true || sizeInfo=true || fieldInfo=true
+@JsonProperty public Map segments;
+
// Present with rawSize=true
@JsonProperty public RawSize rawSize;
-// Present with fieldInfo=truethis seems pretty useless in isolation?
Is it maybe a bad
-// param name?
+// Present only with fieldInfo=true
@JsonProperty public List fieldInfoLegend;
}
+ // Present with segment=true || sizeInfo=true
+ public static class SingleSegmentData {
+@JsonProperty public String name;
+@JsonProperty public Integer delCount;
+@JsonProperty public Integer softDelCount;
+@JsonProperty public Boolean hasFieldUpdates;
+@JsonProperty public Long sizeInBytes;
+@JsonProperty public Integer size;
+// A date string of the form "2024-12-17T17:35:18.275Z"
+@JsonProperty public String age;
Review Comment:
Can we use Instant?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-15751: Modify v2 `GET /collections/collName` to support full ColStatus response [solr]
dsmiley commented on code in PR #2912:
URL: https://github.com/apache/solr/pull/2912#discussion_r1890899775
##
solr/api/src/java/org/apache/solr/client/api/model/CollectionStatusResponse.java:
##
@@ -0,0 +1,249 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.client.api.model;
+
+import com.fasterxml.jackson.annotation.JsonAnyGetter;
+import com.fasterxml.jackson.annotation.JsonAnySetter;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+/** Response of the CollectionStatusApi.getCollectionStatus() API */
+public class CollectionStatusResponse extends SolrJerseyResponse {
+
+ @JsonProperty public String name;
+ @JsonProperty public Integer znodeVersion;
+ @JsonProperty public Long creationTimeMillis;
Review Comment:
Use Instant and drop "Millis"?
##
solr/api/src/java/org/apache/solr/client/api/model/CollectionStatusResponse.java:
##
@@ -0,0 +1,249 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.client.api.model;
+
+import com.fasterxml.jackson.annotation.JsonAnyGetter;
+import com.fasterxml.jackson.annotation.JsonAnySetter;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+/** Response of the CollectionStatusApi.getCollectionStatus() API */
+public class CollectionStatusResponse extends SolrJerseyResponse {
+
+ @JsonProperty public String name;
+ @JsonProperty public Integer znodeVersion;
+ @JsonProperty public Long creationTimeMillis;
+ @JsonProperty public CollectionMetadata properties;
+ @JsonProperty public Integer activeShards;
+ @JsonProperty public Integer inactiveShards;
+ @JsonProperty public List schemaNonCompliant;
+
+ @JsonProperty public Map shards;
+
+ // Always present in response
+ public static class CollectionMetadata {
+@JsonProperty public String configName;
+@JsonProperty public Integer nrtReplicas;
+@JsonProperty public Integer pullReplicas;
+@JsonProperty public Integer tlogReplicas;
+@JsonProperty public Map router;
+@JsonProperty public Integer replicationFactor;
+
+private Map unknownFields = new HashMap<>();
+
+@JsonAnyGetter
+public Map unknownProperties() {
+ return unknownFields;
+}
+
+@JsonAnySetter
+public void setUnknownProperty(String field, Object value) {
+ unknownFields.put(field, value);
+}
+ }
+
+ // Always present in response
+ public static class ShardMetadata {
+@JsonProperty public String state; // TODO Make this an enum?
+@JsonProperty public String range;
+@JsonProperty public ReplicaSummary replicas;
+@JsonProperty public LeaderSummary leader;
+ }
+
+ // Always present in response
+ public static class ReplicaSummary {
+@JsonProperty public Integer total;
+@JsonProperty public Integer active;
+@JsonProperty public Integer down;
+@JsonProperty public Integer recovering;
+
+@JsonProperty("recovery_failed")
+public Integer recoveryFailed;
+ }
+
+ // Always present in response unless otherwise specified
+ public static class LeaderSummary {
+@JsonProperty public String coreNode;
+@JsonProperty public String core;
+@JsonProperty public Boolean leader;
+
+@JsonProperty("node_name")
+public String nodeName;
+
+@JsonProperty("base_url")
+public String b
Re: [PR] SOLR-17541: LBSolrClient implementations should agree on 'getClient()' semantics [solr]
dsmiley commented on code in PR #2899:
URL: https://github.com/apache/solr/pull/2899#discussion_r1890903610
##
solr/core/src/java/org/apache/solr/security/PKIAuthenticationPlugin.java:
##
@@ -374,17 +374,13 @@ PublicKey fetchPublicKeyFromRemote(String nodename) {
}
}
- @Override
- public void setup(Http2SolrClient.Builder httpClientBuilder, Http2SolrClient
client) {
-setup(client, httpClientBuilder);
- }
-
@Override
public void setup(Http2SolrClient client) {
-setup(client, null);
+setup(null, client);
}
- private void setup(Http2SolrClient client, Http2SolrClient.Builder builder) {
+ @Override
+ public void setup(Http2SolrClient.Builder httpClientBuilder, Http2SolrClient
client) {
Review Comment:
Suggest naming this param "builder" as it was, or "httpSolrClientBuilder".
Note "client" is simply "client".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-15751) Create a v2 equivalent for 'COLSTATUS'
[
https://issues.apache.org/jira/browse/SOLR-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SOLR-15751:
--
Labels: V2 newdev pull-request-available (was: V2 newdev)
> Create a v2 equivalent for 'COLSTATUS'
> --
>
> Key: SOLR-15751
> URL: https://issues.apache.org/jira/browse/SOLR-15751
> Project: Solr
> Issue Type: Sub-task
> Components: v2 API
>Reporter: Jason Gerlowski
>Assignee: Jason Gerlowski
>Priority: Major
> Labels: V2, newdev, pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Solr's 'COLSTATUS' command under the v1 \{{/solr/admin/collections}} endpoint
> has no full v2 equivalent. The \{{/v2/collections/}} API has
> similar (identical?) output as a vanilla COLSTATUS request, but COLSTATUS can
> also return detailed index information that \{{/v2/collections/}}
> cannot expose. We should add parameters to this v2 API to expose similar data
> to achieve parity with the v1 COLSTATUS API.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17598) Windows Script logic that handles SOLR_LOGS_DIR is faulty
Christos Malliaridis created SOLR-17598:
---
Summary: Windows Script logic that handles SOLR_LOGS_DIR is faulty
Key: SOLR-17598
URL: https://issues.apache.org/jira/browse/SOLR-17598
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: cli
Affects Versions: 9.7
Environment: Windows 11
Reporter: Christos Malliaridis
h3. Description
On Windows, when setting the {{SOLR_LOGS_DIR}} in the environment variables, it
prevents Solr from starting. Additionally, if the variable contains a space,
trying to stop a running Solr service with {{bin\solr.cmd stop --all}} fails.
I have also observed situations where {{SOLR_LOGS_DIR}} in environment
variables is simply ignored, but I am not sure if this is still preproducable.
h3. Steps to reproduce
1. Set the environment variable {{SOLR_LOGS_DIR}} to an absolute path that does
or does not exist
2. ensure that in your current terminal the envrionment variable you set is
present via "echo %SOLR_LOGS_DIR%"
3. Launch Solr with {{bin\solr.cmd start}}
h3. Expected Behavior
Setting the environment variable is supported and should not make a difference
with setting the value in solr.in.cmd
h3. Actual Behavior
The environment variable is not supported.
h3. Workaround
Setting {{SOLR_LOGS_DIR}} via the {{solr.in.cmd}} file (which overrides the
environment variable) works and avoids the issues. Spaces are also supported
this way.
h3. Workaround
This behavior was tested for the {{SOLR_LOGS_DIR}}, but it is not guaranteed
that other environment variables work. This may affect all environment
variables on Windows.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17598) Windows Script logic that handles SOLR_LOGS_DIR is faulty
[
https://issues.apache.org/jira/browse/SOLR-17598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christos Malliaridis updated SOLR-17598:
Affects Version/s: 9.8
> Windows Script logic that handles SOLR_LOGS_DIR is faulty
> -
>
> Key: SOLR-17598
> URL: https://issues.apache.org/jira/browse/SOLR-17598
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: cli
>Affects Versions: 9x, 9.8
> Environment: Windows 11
>Reporter: Christos Malliaridis
>Priority: Major
>
> h3. Description
> On Windows, when setting the {{SOLR_LOGS_DIR}} in the environment variables,
> it prevents Solr from starting. Additionally, if the variable contains a
> space, trying to stop a running Solr service with {{bin\solr.cmd stop --all}}
> fails.
> I have also observed situations where {{SOLR_LOGS_DIR}} in environment
> variables is simply ignored, but I am not sure if this is still preproducable.
> h3. Steps to reproduce
> 1. Set the environment variable {{SOLR_LOGS_DIR}} to an absolute path that
> does or does not exist
> 2. ensure that in your current terminal the envrionment variable you set is
> present via "echo %SOLR_LOGS_DIR%"
> 3. Launch Solr with {{bin\solr.cmd start}}
> h3. Expected Behavior
> Setting the environment variable is supported and should not make a
> difference with setting the value in solr.in.cmd
> h3. Actual Behavior
> The environment variable is not supported.
> h3. Workaround
> Setting {{SOLR_LOGS_DIR}} via the {{solr.in.cmd}} file (which overrides the
> environment variable) works and avoids the issues. Spaces are also supported
> this way.
> h3. Workaround
> This behavior was tested for the {{SOLR_LOGS_DIR}}, but it is not guaranteed
> that other environment variables work. This may affect all environment
> variables on Windows.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17598) Windows Script logic that handles SOLR_LOGS_DIR is faulty
[
https://issues.apache.org/jira/browse/SOLR-17598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christos Malliaridis updated SOLR-17598:
Affects Version/s: 9x
(was: 9.7)
> Windows Script logic that handles SOLR_LOGS_DIR is faulty
> -
>
> Key: SOLR-17598
> URL: https://issues.apache.org/jira/browse/SOLR-17598
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: cli
>Affects Versions: 9x
> Environment: Windows 11
>Reporter: Christos Malliaridis
>Priority: Major
>
> h3. Description
> On Windows, when setting the {{SOLR_LOGS_DIR}} in the environment variables,
> it prevents Solr from starting. Additionally, if the variable contains a
> space, trying to stop a running Solr service with {{bin\solr.cmd stop --all}}
> fails.
> I have also observed situations where {{SOLR_LOGS_DIR}} in environment
> variables is simply ignored, but I am not sure if this is still preproducable.
> h3. Steps to reproduce
> 1. Set the environment variable {{SOLR_LOGS_DIR}} to an absolute path that
> does or does not exist
> 2. ensure that in your current terminal the envrionment variable you set is
> present via "echo %SOLR_LOGS_DIR%"
> 3. Launch Solr with {{bin\solr.cmd start}}
> h3. Expected Behavior
> Setting the environment variable is supported and should not make a
> difference with setting the value in solr.in.cmd
> h3. Actual Behavior
> The environment variable is not supported.
> h3. Workaround
> Setting {{SOLR_LOGS_DIR}} via the {{solr.in.cmd}} file (which overrides the
> environment variable) works and avoids the issues. Spaces are also supported
> this way.
> h3. Workaround
> This behavior was tested for the {{SOLR_LOGS_DIR}}, but it is not guaranteed
> that other environment variables work. This may affect all environment
> variables on Windows.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-15781: Modify v2 `GET /collections/collName` to support full ColStatus response [solr]
gerlowskija commented on code in PR #2912:
URL: https://github.com/apache/solr/pull/2912#discussion_r1890632261
##
solr/core/src/java/org/apache/solr/handler/admin/api/CollectionStatus.java:
##
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.handler.admin.api;
+
+import static org.apache.solr.common.cloud.ZkStateReader.COLLECTION_PROP;
+
+import jakarta.inject.Inject;
+import org.apache.solr.client.api.endpoint.CollectionStatusApi;
+import org.apache.solr.client.api.model.CollectionStatusResponse;
+import org.apache.solr.common.cloud.ZkNodeProps;
+import org.apache.solr.common.params.ModifiableSolrParams;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.handler.admin.ColStatus;
+import org.apache.solr.jersey.PermissionName;
+import org.apache.solr.jersey.SolrJacksonMapper;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.solr.security.PermissionNameProvider;
+
+/** V2 API implementation for {@link CollectionStatusApi}. */
+public class CollectionStatus extends AdminAPIBase implements
CollectionStatusApi {
+
+ @Inject
+ public CollectionStatus(
+ CoreContainer coreContainer,
+ SolrQueryRequest solrQueryRequest,
+ SolrQueryResponse solrQueryResponse) {
+super(coreContainer, solrQueryRequest, solrQueryResponse);
+ }
+
+ @Override
+ @PermissionName(PermissionNameProvider.Name.COLL_READ_PERM)
+ public CollectionStatusResponse getCollectionStatus(
+ String collectionName,
+ Boolean coreInfo,
+ Boolean segments,
+ Boolean fieldInfo,
+ Boolean rawSize,
+ Boolean rawSizeSummary,
+ Boolean rawSizeDetails,
+ Float rawSizeSamplingPercent,
+ Boolean sizeInfo)
+ throws Exception {
+recordCollectionForLogAndTracing(collectionName, solrQueryRequest);
+
+final var params = new ModifiableSolrParams();
+params.add(COLLECTION_PROP, collectionName);
+params.setNonNull(ColStatus.CORE_INFO_PROP, coreInfo);
+params.setNonNull(ColStatus.SEGMENTS_PROP, segments);
+params.setNonNull(ColStatus.FIELD_INFO_PROP, fieldInfo);
+params.setNonNull(ColStatus.RAW_SIZE_PROP, rawSize);
+params.setNonNull(ColStatus.RAW_SIZE_SUMMARY_PROP, rawSizeSummary);
+params.setNonNull(ColStatus.RAW_SIZE_DETAILS_PROP, rawSizeDetails);
+params.setNonNull(ColStatus.RAW_SIZE_SAMPLING_PERCENT_PROP,
rawSizeSamplingPercent);
+params.setNonNull(ColStatus.SIZE_INFO_PROP, sizeInfo);
+
+// TODO Push CollectionStatusResponse down into ColStatus and avoid the
intermediate NL, if all
+// usages can be switched over.
+final var nlResponse = new NamedList<>();
+new ColStatus(
+coreContainer.getSolrClientCache(),
+
coreContainer.getZkController().getZkStateReader().getClusterState(),
+new ZkNodeProps(params))
+.getColStatus(nlResponse);
+// collName is prop/key for a nested NL returned by ColStatus - extract
the inner NL and
+// manually add name to resp
Review Comment:
That model works great anytime that the v1 and v2 responses are exactly the
same (which is almost always the case). I made the v2 response for this API
slightly different from v1 COLSTATUS, so it's hard to have CollectionsHandler
call `V2ApiUtils.squashIntoNamedList(new
CollectionStatus(..).getCollectionStatus(...))` the way we've been doing with
most of our other v1-v2 pairs.
That said, I did a bit of restructuring here so that v1 can *mostly* sit on
top of v2: CollectionsHandler.COLSTATUS now calls a static method
`populateColStatusData` that lives in this class. Hopefully that gets us most
of the benefit. If we eventually tackle the TODO comment at L87, we should be
able to close the gap and have CollectionsHandler.COLSTATUS consume the v2 API
in the normal way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastr
Re: [PR] SOLR-17525: Text Embedder Query Parser [solr]
malliaridis commented on PR #2809: URL: https://github.com/apache/solr/pull/2809#issuecomment-2551951538 It should be manageable in less than an hour to handle the gradle stuff. However, in case you make use of JDK >11 fearures, or the dependencies are not compatible with JDK 11, there may be more than just the gradle stuff that needs to be updated, and even may not be compatible at all. Have you checked these constraints? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] SOLR-17306: fix replication problem on follower restart [solr]
ds-manzinger opened a new pull request, #2918: URL: https://github.com/apache/solr/pull/2918 https://issues.apache.org/jira/browse/SOLR-X # Description Please provide a short description of the changes you're making with this pull request. # Solution Please provide a short description of the approach taken to implement your solution. # Tests Please describe the tests you've developed or run to confirm this patch implements the feature or solves the problem. # Checklist Please review the following and check all that apply: - [ ] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [ ] I have created a Jira issue and added the issue ID to my pull request title. - [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation) - [ ] I have developed this patch against the `main` branch. - [ ] I have run `./gradlew check`. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17306: fix replication problem on follower restart [solr]
ds-manzinger commented on PR #2918: URL: https://github.com/apache/solr/pull/2918#issuecomment-2552157377 @epugh Sorry for that new Pull Request. Sync with apache/solr broke my history -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17541: LBSolrClient implementations should agree on 'getClient()' semantics [solr]
jdyer1 commented on PR #2899: URL: https://github.com/apache/solr/pull/2899#issuecomment-2552174264 > Can you point me at such so I can appreciate the problem better? My apologies, I think I misstated what is going on earlier. `PKIAuthenticationPlugin` on setup injects a `RequestResponseListener` into the passed-in Client. I wanted to ensure the same Listener gets injected both in the Builder and the Client, in case it matters. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17541: LBSolrClient implementations should agree on 'getClient()' semantics [solr]
dsmiley commented on PR #2899: URL: https://github.com/apache/solr/pull/2899#issuecomment-2552178161 Since the direction is SolrClient immutability, I think this auth/PKI setup method should only the builder (mutable). Is this realistic? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17525: Text Embedder Query Parser [solr]
alessandrobenedetti commented on PR #2809: URL: https://github.com/apache/solr/pull/2809#issuecomment-2551958083 Not at all, let's see first if 9.8 is acceptable as a target and we take it from there! Thanks in the meantime for your prompt responses! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17471) upgrade to Lucene 9.12.1
[ https://issues.apache.org/jira/browse/SOLR-17471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Salagnac updated SOLR-17471: --- Summary: upgrade to Lucene 9.12.1 (was: upgrade to Lucene 9.12.x) > upgrade to Lucene 9.12.1 > > > Key: SOLR-17471 > URL: https://issues.apache.org/jira/browse/SOLR-17471 > Project: Solr > Issue Type: Task >Reporter: Christine Poerschke >Priority: Blocker > Fix For: 9.8 > > > Creating and leaving this Jira open for (new) devs to grab. > * https://lucene.apache.org/core/9_12_0/changes/Changes.html or > https://lucene.apache.org/core/9_12_1/changes/Changes.html when released > * https://github.com/apache/solr/blob/main/dev-docs/lucene-upgrade.md -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Assigned] (SOLR-17471) upgrade to Lucene 9.12.1
[ https://issues.apache.org/jira/browse/SOLR-17471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Salagnac reassigned SOLR-17471: -- Assignee: Pierre Salagnac > upgrade to Lucene 9.12.1 > > > Key: SOLR-17471 > URL: https://issues.apache.org/jira/browse/SOLR-17471 > Project: Solr > Issue Type: Task >Reporter: Christine Poerschke >Assignee: Pierre Salagnac >Priority: Blocker > Fix For: 9.8 > > > Creating and leaving this Jira open for (new) devs to grab. > * https://lucene.apache.org/core/9_12_0/changes/Changes.html or > https://lucene.apache.org/core/9_12_1/changes/Changes.html when released > * https://github.com/apache/solr/blob/main/dev-docs/lucene-upgrade.md -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17405) Zookeeper session can be re-established by multiple threads concurrently
[
https://issues.apache.org/jira/browse/SOLR-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906816#comment-17906816
]
David Smiley commented on SOLR-17405:
-
> this bug does not exist anymore since the move to Curator.
Really glad to hear that. The move to Curator for connection management means
less Solr code doing questionable low-level ZK things that Curator is now
responsible for.
> Zookeeper session can be re-established by multiple threads concurrently
>
>
> Key: SOLR-17405
> URL: https://issues.apache.org/jira/browse/SOLR-17405
> Project: Solr
> Issue Type: Bug
>Affects Versions: 8.11, 9.6
>Reporter: Pierre Salagnac
>Priority: Major
> Labels: pull-request-available
> Attachments: stack.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Because of a bug in SolrCloud, the Zookeeper session can be re-established by
> multiple threads concurrently when an expiration occurs.
> This portion of the code assumes it is mono-threaded. Because of the bug, the
> last thread re-establishing the session can waif for 30 seconds per core,
> waiting for it to be marked {{DOWN}} while it was previously marked
> {{ACTIVE}} by another thread. With a high number of cores, the Solr server
> can hang for hours before taking traffic again.
> Following exception shows two threads were reestablishing the session
> concurrently. {{ZkController.createEphemeralLiveNode()}} should never be
> invoked twice for the same Zookeeper session.
> {code:java}
> thrown: java. lang.RuntimeException:
> org.apache.solr.common.cloud.ZooKeeperException:
> at
> org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:178)
> at org.apache.solr. common.cloud.DefaultConnectionStrategy.
> reconnect(DefaultConnectionStrategy.java:57)
> at
> org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:152)
> at org.apache.
> zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510)
> Caused by: org.apache.solr.common.cloud.ZooKeeperException:
> at
> org.apache.solr.cloud.ZkController$1.command(ZkController.java:462)
> at
> org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:170)
> ... 4 more
> Caused by: org.apache. zookeeper.KeeperException$NodeExistsException.
> KeeperErrorCode = NodeExists
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:
> 126)
> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:1925)
> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:1830)
> at
> org.apache.solr.common.cloud.SolrZkClient.lambda$multi$11(SolrZkClient.java:666)
> at
> org.apache.solr.common.cloud.ZkCmdExecutor.retry0peration(ZkCmdExecutor.java:71)
> at
> org.apache.solr.common.cloud.SolrZkClient.multi(SolrZkClient.java:666)
> at org.apache.sol.cloud.ZkController
> CreateEphemeralLiveNode(ZkController.java:1086)
> at
> org.apache.solr.cloud.ZkController$1.command(ZkController.java:411)
> ... 5 more {code}
> h2. Root cause
> This bug occurs because several threads can re-establish the session
> concurrently.
> It cannot happen at the first expiration of the session, thanks to a thread
> pool with a single thread to execute the zookeeper Watcher.
> Bellow is a code snippet from class {{SolrZkClient.ProcessWatchWithExecutor}}
> {code:java}
> if (watcher instanceof ConnectionManager) {
> zkConnManagerCallbackExecutor.submit(() -> watcher.process(event));
> } else {
>...
> }
> {code}
> Using this dedicated thread pool (with a single thread) is supposed to ensure
> we don’t handle watches for connection related events with multiple threads.
> This works well for the first session expiration.
> Now, when we re-establish the session after the first expiration, we don’t
> use this wrapper to register the watch.
> It is done directly in {{ConnectionManager}} without wrapping the ZK watch.
> In the following snippet, _“this”_ is the ZK watcher instance, but it is not
> wrapper to use a {{{}ProcessWatchWithExecutor{}}}. This means the next events
> will directly be handled by any ZK callback thread.
> {code:java}
> connectionStrategy.reconnect(zkServerAddress,client.getZkClientTimeout(),
> this,
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17471) upgrade to Lucene 9.12.1
[
https://issues.apache.org/jira/browse/SOLR-17471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906818#comment-17906818
]
Pierre Salagnac commented on SOLR-17471:
I updated the PR to upgrade to Lucene 9.12.1 and to use the newly introduced
constructor. Everything looks good to me.
Also, some new checks were introduced in Lucene testing framework with [PR
#13578|https://github.com/apache/lucene/pull/13578] to ensure segment files are
always read with {{READONCE}} mode. This was breaking some backup/restore tests.
I updated backup code to use this mode, with definitely makes sense to me.
> upgrade to Lucene 9.12.1
>
>
> Key: SOLR-17471
> URL: https://issues.apache.org/jira/browse/SOLR-17471
> Project: Solr
> Issue Type: Task
>Reporter: Christine Poerschke
>Assignee: Pierre Salagnac
>Priority: Blocker
> Fix For: 9.8
>
>
> Creating and leaving this Jira open for (new) devs to grab.
> * https://lucene.apache.org/core/9_12_0/changes/Changes.html or
> https://lucene.apache.org/core/9_12_1/changes/Changes.html when released
> * https://github.com/apache/solr/blob/main/dev-docs/lucene-upgrade.md
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Assigned] (SOLR-17405) Zookeeper session can be re-established by multiple threads concurrently
[
https://issues.apache.org/jira/browse/SOLR-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Salagnac reassigned SOLR-17405:
--
Assignee: Pierre Salagnac
> Zookeeper session can be re-established by multiple threads concurrently
>
>
> Key: SOLR-17405
> URL: https://issues.apache.org/jira/browse/SOLR-17405
> Project: Solr
> Issue Type: Bug
>Affects Versions: 8.11, 9.6
>Reporter: Pierre Salagnac
>Assignee: Pierre Salagnac
>Priority: Major
> Labels: pull-request-available
> Attachments: stack.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Because of a bug in SolrCloud, the Zookeeper session can be re-established by
> multiple threads concurrently when an expiration occurs.
> This portion of the code assumes it is mono-threaded. Because of the bug, the
> last thread re-establishing the session can waif for 30 seconds per core,
> waiting for it to be marked {{DOWN}} while it was previously marked
> {{ACTIVE}} by another thread. With a high number of cores, the Solr server
> can hang for hours before taking traffic again.
> Following exception shows two threads were reestablishing the session
> concurrently. {{ZkController.createEphemeralLiveNode()}} should never be
> invoked twice for the same Zookeeper session.
> {code:java}
> thrown: java. lang.RuntimeException:
> org.apache.solr.common.cloud.ZooKeeperException:
> at
> org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:178)
> at org.apache.solr. common.cloud.DefaultConnectionStrategy.
> reconnect(DefaultConnectionStrategy.java:57)
> at
> org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:152)
> at org.apache.
> zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510)
> Caused by: org.apache.solr.common.cloud.ZooKeeperException:
> at
> org.apache.solr.cloud.ZkController$1.command(ZkController.java:462)
> at
> org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:170)
> ... 4 more
> Caused by: org.apache. zookeeper.KeeperException$NodeExistsException.
> KeeperErrorCode = NodeExists
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:
> 126)
> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:1925)
> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:1830)
> at
> org.apache.solr.common.cloud.SolrZkClient.lambda$multi$11(SolrZkClient.java:666)
> at
> org.apache.solr.common.cloud.ZkCmdExecutor.retry0peration(ZkCmdExecutor.java:71)
> at
> org.apache.solr.common.cloud.SolrZkClient.multi(SolrZkClient.java:666)
> at org.apache.sol.cloud.ZkController
> CreateEphemeralLiveNode(ZkController.java:1086)
> at
> org.apache.solr.cloud.ZkController$1.command(ZkController.java:411)
> ... 5 more {code}
> h2. Root cause
> This bug occurs because several threads can re-establish the session
> concurrently.
> It cannot happen at the first expiration of the session, thanks to a thread
> pool with a single thread to execute the zookeeper Watcher.
> Bellow is a code snippet from class {{SolrZkClient.ProcessWatchWithExecutor}}
> {code:java}
> if (watcher instanceof ConnectionManager) {
> zkConnManagerCallbackExecutor.submit(() -> watcher.process(event));
> } else {
>...
> }
> {code}
> Using this dedicated thread pool (with a single thread) is supposed to ensure
> we don’t handle watches for connection related events with multiple threads.
> This works well for the first session expiration.
> Now, when we re-establish the session after the first expiration, we don’t
> use this wrapper to register the watch.
> It is done directly in {{ConnectionManager}} without wrapping the ZK watch.
> In the following snippet, _“this”_ is the ZK watcher instance, but it is not
> wrapper to use a {{{}ProcessWatchWithExecutor{}}}. This means the next events
> will directly be handled by any ZK callback thread.
> {code:java}
> connectionStrategy.reconnect(zkServerAddress,client.getZkClientTimeout(),
> this,
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17541: LBSolrClient implementations should agree on 'getClient()' semantics [solr]
jdyer1 commented on PR #2899: URL: https://github.com/apache/solr/pull/2899#issuecomment-2552304777 > Since the direction is SolrClient immutability, I think this auth/PKI setup method should only the builder (mutable). Is this realistic? I fully agree this is the right direction to take. Unfortunately this probably requires a minor redesign to both `UpdateShardHandler` and `HttpSolrClientProvider`, which is out-of-scope here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17587: Prometheus Writer duplicate TYPE information in exposition format [solr]
mlbiscoc commented on PR #2902: URL: https://github.com/apache/solr/pull/2902#issuecomment-2552315416 Going to bump this PR. It's not a blocker for standard Prometheus server metrics collection but it can potentially block users using other exporters/collectors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Coordinator: remove SolrQueryRequest.getCloudDescriptor [solr]
pvcnt commented on code in PR #2913:
URL: https://github.com/apache/solr/pull/2913#discussion_r1890431876
##
solr/core/src/java/org/apache/solr/servlet/CoordinatorHttpSolrCall.java:
##
@@ -160,28 +156,13 @@ protected String getCoreOrColName() {
return collectionName;
}
- public static SolrQueryRequest wrappedReq(
- SolrQueryRequest delegate, String collectionName, HttpSolrCall
httpSolrCall) {
-Properties p = new Properties();
-if (collectionName != null) {
- p.put(CoreDescriptor.CORE_COLLECTION, collectionName);
-}
-p.put(CloudDescriptor.REPLICA_TYPE, Replica.Type.PULL.toString());
-p.put(CoreDescriptor.CORE_SHARD, "_");
+ public static SolrQueryRequest wrappedReq(SolrQueryRequest delegate,
HttpSolrCall httpSolrCall) {
Review Comment:
I believe this method can be removed altogether, can't it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Clarify comment since examples do not use a special directory [solr]
epugh commented on PR #2908: URL: https://github.com/apache/solr/pull/2908#issuecomment-2552105680 Thanks for confirming this...Yeah, no test structure sucks. Hopefully htough, as we work out the set of changes that we introduced in 9.7 and 9.8 that things will calm down -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Clarify comment since examples do not use a special directory [solr]
epugh merged PR #2908: URL: https://github.com/apache/solr/pull/2908 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17599) CrossDC should not use the MirroringUpdateHandler when replaying updates from the transaction log or when doing PeerSync.
Mark Robert Miller created SOLR-17599: - Summary: CrossDC should not use the MirroringUpdateHandler when replaying updates from the transaction log or when doing PeerSync. Key: SOLR-17599 URL: https://issues.apache.org/jira/browse/SOLR-17599 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Components: CrossDC Affects Versions: 9.7 Reporter: Mark Robert Miller Assignee: Mark Robert Miller -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17306: fix replication problem on follower restart [solr]
ds-manzinger closed pull request #2874: SOLR-17306: fix replication problem on follower restart URL: https://github.com/apache/solr/pull/2874 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17597: Fix `CaffeineCache.put()` ramBytes decrement [solr]
magibney commented on PR #2917: URL: https://github.com/apache/solr/pull/2917#issuecomment-2551854131 This is a trivial change with no practical impact on the vast majority of users. I plan to merge this tomorrow, pending any feedback. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17525: Text Embedder Query Parser [solr]
alessandrobenedetti commented on PR #2809: URL: https://github.com/apache/solr/pull/2809#issuecomment-2551920436 thanks @malliaridis to let me know, I totally forgot about the dependency hell! What do you think @anshumg ? In case Anshum is happy to include it, Christos will you have some spare time within the week to jump on a call and do the backporting together? Do you think it's reasonable to do it live in an hour, maybe on a zoom call? I'm definitely not a gradle expert, and if It's me having to do it on my own I'm afraid I won't be able to do it soon, so we'll have to postpone it to 10.0. Let me know! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] SOLR-17405: allow a single thread to reestablish ZK session [solr]
psalagnac opened a new pull request, #2914: URL: https://github.com/apache/solr/pull/2914 https://issues.apache.org/jira/browse/SOLR-17405 # Description `OnReconnect` can be invoked by several threads concurrently (see Jira for the explanation). Same change as #2644, but against `9x` branch. This bug does not exist anymore in `main` since the move to Curator. # Solution Always add the wrapper on the watch that handle the session expiration events. This makes sure the session is re-established in a dedicated thread, and this thread exists only once per server. # Tests No new tests added. Unfortunately, this is super tricky to reproduce. # Checklist Please review the following and check all that apply: - [x] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [x] I have created a Jira issue and added the issue ID to my pull request title. - [x] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [ ] I have developed this patch against the `main` branch. - [x] I have run `./gradlew check`. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17595: Backport "Fix Solr examples for Windows" [solr]
malliaridis merged PR #2911: URL: https://github.com/apache/solr/pull/2911 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17595) CLI examples are broken on Windows
[ https://issues.apache.org/jira/browse/SOLR-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906786#comment-17906786 ] ASF subversion and git services commented on SOLR-17595: Commit 9440ec1fa35109bc4ba79dd86af51164e8a040fc in solr's branch refs/heads/branch_9x from Christos Malliaridis [ https://gitbox.apache.org/repos/asf?p=solr.git;h=9440ec1fa35 ] SOLR-17595: Backport "Fix Solr examples for Windows" (#2911) * Fix argument parsing for -D options with multiple values on Windows (cherry picked from commit 44ea44d1fb07019e8bcd56c7a341301566d64deb) * SOLR-17595: Fix Solr examples for Windows (#2909) * Fix argument parsing for -D options with multiple values on Windows * Fix invalid usage of wildcards in RunExampleTool / PostTool (cherry picked from commit a4229e76771da0125b38204db01ea0934e09c4f4) * Fix unsupported method > CLI examples are broken on Windows > -- > > Key: SOLR-17595 > URL: https://issues.apache.org/jira/browse/SOLR-17595 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: cli, examples, SolrCLI >Affects Versions: 9.7 >Reporter: Christos Malliaridis >Assignee: Christos Malliaridis >Priority: Blocker > Labels: cli, pull-request-available > Fix For: 9.8 > > Attachments: console-output-9.7.txt, console-output.txt > > Time Spent: 50m > Remaining Estimate: 0h > > When trying to run the examples on Windows, it fails with an unspecified > error. I have attached the output for investigation. Bug is present in latest > changes. > A previous state was reporting more details about a potential cause. It was > mentioning the "*/xml" in in RunExampleTool.java, but that may not be anymore > the issue for the current failure (but could be a lead for the error). > The error is slightly different in the branches main, 9x and 9.8. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17595) CLI examples are broken on Windows
[ https://issues.apache.org/jira/browse/SOLR-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906789#comment-17906789 ] ASF subversion and git services commented on SOLR-17595: Commit a86d50db808591eff7b47923641802a42aa30eec in solr's branch refs/heads/branch_9_8 from Christos Malliaridis [ https://gitbox.apache.org/repos/asf?p=solr.git;h=a86d50db808 ] SOLR-17595: Backport "Fix Solr examples for Windows" (#2911) * Fix argument parsing for -D options with multiple values on Windows (cherry picked from commit 44ea44d1fb07019e8bcd56c7a341301566d64deb) * SOLR-17595: Fix Solr examples for Windows (#2909) * Fix argument parsing for -D options with multiple values on Windows * Fix invalid usage of wildcards in RunExampleTool / PostTool (cherry picked from commit a4229e76771da0125b38204db01ea0934e09c4f4) * Fix unsupported method > CLI examples are broken on Windows > -- > > Key: SOLR-17595 > URL: https://issues.apache.org/jira/browse/SOLR-17595 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: cli, examples, SolrCLI >Affects Versions: 9.7 >Reporter: Christos Malliaridis >Assignee: Christos Malliaridis >Priority: Blocker > Labels: cli, pull-request-available > Fix For: 9.8 > > Attachments: console-output-9.7.txt, console-output.txt > > Time Spent: 50m > Remaining Estimate: 0h > > When trying to run the examples on Windows, it fails with an unspecified > error. I have attached the output for investigation. Bug is present in latest > changes. > A previous state was reporting more details about a potential cause. It was > mentioning the "*/xml" in in RunExampleTool.java, but that may not be anymore > the issue for the current failure (but could be a lead for the error). > The error is slightly different in the branches main, 9x and 9.8. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Resolved] (SOLR-17595) CLI examples are broken on Windows
[ https://issues.apache.org/jira/browse/SOLR-17595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christos Malliaridis resolved SOLR-17595. - Resolution: Fixed > CLI examples are broken on Windows > -- > > Key: SOLR-17595 > URL: https://issues.apache.org/jira/browse/SOLR-17595 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: cli, examples, SolrCLI >Affects Versions: 9.7 >Reporter: Christos Malliaridis >Assignee: Christos Malliaridis >Priority: Blocker > Labels: cli, pull-request-available > Fix For: 9.8 > > Attachments: console-output-9.7.txt, console-output.txt > > Time Spent: 50m > Remaining Estimate: 0h > > When trying to run the examples on Windows, it fails with an unspecified > error. I have attached the output for investigation. Bug is present in latest > changes. > A previous state was reporting more details about a potential cause. It was > mentioning the "*/xml" in in RunExampleTool.java, but that may not be anymore > the issue for the current failure (but could be a lead for the error). > The error is slightly different in the branches main, 9x and 9.8. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Resolved] (SOLR-17551) zk upconfig command broken on Windows
[
https://issues.apache.org/jira/browse/SOLR-17551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christos Malliaridis resolved SOLR-17551.
-
Fix Version/s: 9.8
Resolution: Resolved
This issue is resolved via the refactoring done in SOLR-17483. Since there will
not be a 9.7.1 release, the fix version is set to 9.8.
> zk upconfig command broken on Windows
> -
>
> Key: SOLR-17551
> URL: https://issues.apache.org/jira/browse/SOLR-17551
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: 9.7
> Environment: Windows Server 2022, Windows 10
>Reporter: Mingchun Zhao
>Assignee: Christos Malliaridis
>Priority: Critical
> Labels: cli, newdev
> Fix For: 9.8
>
> Attachments: SOLR-17551.patch
>
>
> When running the `solr zk upconfig` command on Windows, the following error
> occurs.
> {code:java}
> C:\Program Files\solr-9.7.0\bin>solr zk upconfig -d sitecore_configs -n
> sitecoreconfigset -z 10.1.0.11:2181,10.1.0.12:2181,10.1.0.13:2181
> 'ELSE' is not recognized as an internal or external command,
> operable program or batch file.
> Failed to parse command-line arguments due to: Unrecognized option: --zk-host
> ZK_HOSTusage: bin/solr upconfig -d -n [-url ] [-z
> ]List of options:
> -d,--conf-dir Local directory with configs.
> -n,--conf-name Configset name in ZooKeeper.
> -url,--solr-url Base Solr URL, which can be used to determine the
> zk-host if that's not known; defaults to:
> http://Solrcloud-test-Solr01:8983.
> -z,--zk-host Zookeeper connection string; unnecessary if ZK_HOST
> is defined in solr.in.sh; otherwise,
> defaults to localhost:9983.Please see the Reference
> Guide for more tools documentation:
> https://solr.apache.org/guide/solr/latest/deployment-guide/solr-control-script-reference.html
> WARN - 2024-11-07 11:29:32.428; org.apache.solr.common.cloud.SolrZkClient;
> Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it
> creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to
> Zookeeper nodes
> WARN - 2024-11-07 11:29:33.666; org.apache.solr.common.cloud.SolrZkClient;
> Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates
> 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes
> Uploading C:\Program
> Files\solr-9.7.0\server\solr\configsets\sitecore_configs\conf for config
> sitecoreconfigset to ZooKeeper at 10.1.0.11:2181 {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Comment Edited] (SOLR-17551) zk upconfig command broken on Windows
[
https://issues.apache.org/jira/browse/SOLR-17551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906795#comment-17906795
]
Christos Malliaridis edited comment on SOLR-17551 at 12/18/24 4:19 PM:
---
This issue is resolved via the refactoring done in SOLR-17483. Since there will
not be a 9.7.1 release, the fix version is set to 9.8, which includes the
refactoring done in SOLR-17483.
was (Author: JIRAUSER305622):
This issue is resolved via the refactoring done in SOLR-17483. Since there will
not be a 9.7.1 release, the fix version is set to 9.8.
> zk upconfig command broken on Windows
> -
>
> Key: SOLR-17551
> URL: https://issues.apache.org/jira/browse/SOLR-17551
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: 9.7
> Environment: Windows Server 2022, Windows 10
>Reporter: Mingchun Zhao
>Assignee: Christos Malliaridis
>Priority: Critical
> Labels: cli, newdev
> Fix For: 9.8
>
> Attachments: SOLR-17551.patch
>
>
> When running the `solr zk upconfig` command on Windows, the following error
> occurs.
> {code:java}
> C:\Program Files\solr-9.7.0\bin>solr zk upconfig -d sitecore_configs -n
> sitecoreconfigset -z 10.1.0.11:2181,10.1.0.12:2181,10.1.0.13:2181
> 'ELSE' is not recognized as an internal or external command,
> operable program or batch file.
> Failed to parse command-line arguments due to: Unrecognized option: --zk-host
> ZK_HOSTusage: bin/solr upconfig -d -n [-url ] [-z
> ]List of options:
> -d,--conf-dir Local directory with configs.
> -n,--conf-name Configset name in ZooKeeper.
> -url,--solr-url Base Solr URL, which can be used to determine the
> zk-host if that's not known; defaults to:
> http://Solrcloud-test-Solr01:8983.
> -z,--zk-host Zookeeper connection string; unnecessary if ZK_HOST
> is defined in solr.in.sh; otherwise,
> defaults to localhost:9983.Please see the Reference
> Guide for more tools documentation:
> https://solr.apache.org/guide/solr/latest/deployment-guide/solr-control-script-reference.html
> WARN - 2024-11-07 11:29:32.428; org.apache.solr.common.cloud.SolrZkClient;
> Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it
> creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to
> Zookeeper nodes
> WARN - 2024-11-07 11:29:33.666; org.apache.solr.common.cloud.SolrZkClient;
> Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates
> 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes
> Uploading C:\Program
> Files\solr-9.7.0\server\solr\configsets\sitecore_configs\conf for config
> sitecoreconfigset to ZooKeeper at 10.1.0.11:2181 {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-15781: Modify v2 `GET /collections/collName` to support full ColStatus response [solr]
gerlowskija commented on code in PR #2912:
URL: https://github.com/apache/solr/pull/2912#discussion_r1890512324
##
solr/api/src/java/org/apache/solr/client/api/model/CollectionStatusResponse.java:
##
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.client.api.model;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+import java.util.List;
+import java.util.Map;
+
+/** Response of the CollectionStatusApi.getCollectionStatus() API */
+public class CollectionStatusResponse extends SolrJerseyResponse {
+
+ @JsonProperty public String name;
+ @JsonProperty public Integer znodeVersion;
+ @JsonProperty public Long creationTimeMillis;
+ @JsonProperty public CollectionMetadata properties;
+ @JsonProperty public Integer activeShards;
+ @JsonProperty public Integer inactiveShards;
+ @JsonProperty public List schemaNonCompliant;
+
+ @JsonProperty public Map shards;
+
+ // Always present in response
+ public static class CollectionMetadata {
+@JsonProperty public String configName;
+@JsonProperty public Integer nrtReplicas;
+@JsonProperty public Integer pullReplicas;
+@JsonProperty public Integer tlogReplicas;
+@JsonProperty public Map router;
+@JsonProperty public Integer replicationFactor;
+ }
+
+ // Always present in response
+ public static class ShardMetadata {
+@JsonProperty public String state; // TODO Make this an enum?
+@JsonProperty public String range;
+@JsonProperty public ReplicaSummary replicas;
+@JsonProperty public LeaderSummary leader;
+ }
+
+ // ALways present in response
+ public static class ReplicaSummary {
+@JsonProperty public Integer total;
+@JsonProperty public Integer active;
+@JsonProperty public Integer down;
+@JsonProperty public Integer recovering;
+
+@JsonProperty("recovery_failed")
+public Integer recoveryFailed;
+ }
+
+ // Always present in response unless otherwise specified
+ public static class LeaderSummary {
+@JsonProperty public String coreNode;
+@JsonProperty public String core;
+@JsonProperty public Boolean leader;
+
+@JsonProperty("node_name")
+public String nodeName;
+
+@JsonProperty("base_url")
+public String baseUrl;
+
+@JsonProperty public String state; // TODO Make this an enum?
+@JsonProperty public String type; // TODO Make this an enum?
+
+@JsonProperty("force_set_state")
+public Boolean forceSetState;
Review Comment:
I can omit the field from the v2 response if you'd like, but this seems like
a much larger concern than this PR. It's only in the v2 response because it's
in the v1 COLSTATUS response. (And it's only there because it's being persisted
to ZK!)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-15781: Modify v2 `GET /collections/collName` to support full ColStatus response [solr]
gerlowskija commented on code in PR #2912:
URL: https://github.com/apache/solr/pull/2912#discussion_r1890516257
##
solr/api/src/java/org/apache/solr/client/api/model/CollectionStatusResponse.java:
##
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.client.api.model;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+import java.util.List;
+import java.util.Map;
+
+/** Response of the CollectionStatusApi.getCollectionStatus() API */
+public class CollectionStatusResponse extends SolrJerseyResponse {
+
+ @JsonProperty public String name;
+ @JsonProperty public Integer znodeVersion;
+ @JsonProperty public Long creationTimeMillis;
+ @JsonProperty public CollectionMetadata properties;
+ @JsonProperty public Integer activeShards;
+ @JsonProperty public Integer inactiveShards;
+ @JsonProperty public List schemaNonCompliant;
+
+ @JsonProperty public Map shards;
+
+ // Always present in response
+ public static class CollectionMetadata {
+@JsonProperty public String configName;
+@JsonProperty public Integer nrtReplicas;
+@JsonProperty public Integer pullReplicas;
+@JsonProperty public Integer tlogReplicas;
+@JsonProperty public Map router;
+@JsonProperty public Integer replicationFactor;
+ }
+
+ // Always present in response
+ public static class ShardMetadata {
+@JsonProperty public String state; // TODO Make this an enum?
+@JsonProperty public String range;
+@JsonProperty public ReplicaSummary replicas;
+@JsonProperty public LeaderSummary leader;
+ }
+
+ // ALways present in response
+ public static class ReplicaSummary {
+@JsonProperty public Integer total;
+@JsonProperty public Integer active;
+@JsonProperty public Integer down;
+@JsonProperty public Integer recovering;
+
+@JsonProperty("recovery_failed")
+public Integer recoveryFailed;
+ }
+
+ // Always present in response unless otherwise specified
+ public static class LeaderSummary {
+@JsonProperty public String coreNode;
+@JsonProperty public String core;
+@JsonProperty public Boolean leader;
+
+@JsonProperty("node_name")
+public String nodeName;
+
+@JsonProperty("base_url")
+public String baseUrl;
+
+@JsonProperty public String state; // TODO Make this an enum?
+@JsonProperty public String type; // TODO Make this an enum?
+
+@JsonProperty("force_set_state")
+public Boolean forceSetState;
+
+// Present with coreInfo=true || sizeInfo=true unless otherwise specified
+@JsonProperty public SegmentInfo segInfos;
+ }
+
+ // Present with coreInfo=true || sizeInfo=true unless otherwise specified
+ public static class SegmentInfo {
+// Present with coreInfo=true || sizeInfo=true unless otherwise specified
+@JsonProperty public SegmentSummary info;
+
+// Present with rawSize=true
+@JsonProperty public RawSize rawSize;
+
+// Present with fieldInfo=truethis seems pretty useless in isolation?
Is it maybe a bad
+// param name?
+@JsonProperty public List fieldInfoLegend;
+ }
+
+ // Present with rawSize=true unless otherwise specified
+ public static class RawSize {
+@JsonProperty public Map fieldsBySize;
+@JsonProperty public Map typesBySize;
+
+// Present with rawSizeDetails=true
+@JsonProperty public Object details;
+
+// Present with rawSizeSummary=true
+@JsonProperty public Map summary;
+ }
+
+ // Present with coreInfo=true || sizeInfo=true unless otherwise specified
+ public static class SegmentSummary {
+@JsonProperty public String minSegmentLuceneVersion;
+@JsonProperty public String commitLuceneVersion;
+@JsonProperty public Integer numSegments;
+@JsonProperty public String segmentsFileName;
+@JsonProperty public Integer totalMaxDoc;
+
+@JsonProperty
+public Map
+userData; // Typically keys are 'commitCommandVer' and 'commitTimeMSec'
Review Comment:
It's ugly for sure. Tidy is well-intentioned, but often a blunt instrument.
It's probably not worth too much of our time trying to clean up after it,
but I'm happy to fix these few bits you highlighted.
--
This is an automated message from the A
Re: [PR] SOLR-15781: Modify v2 `GET /collections/collName` to support full ColStatus response [solr]
gerlowskija commented on code in PR #2912:
URL: https://github.com/apache/solr/pull/2912#discussion_r1890516647
##
solr/core/src/java/org/apache/solr/api/V2HttpCall.java:
##
@@ -188,6 +188,8 @@ public void call(SolrQueryRequest req, SolrQueryResponse
rsp) {
Thread.currentThread().setContextClassLoader(core.getResourceLoader().getClassLoader());
this.path = path = path.substring(prefix.length() +
pathSegments.get(1).length() + 2);
+ parts.put(
+ COLLECTION_PROP, origCorename); // Core-level API, so populate
"collection" template val
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[PR] Optimize dependency resolution via platform module [solr]
malliaridis opened a new pull request, #2915: URL: https://github.com/apache/solr/pull/2915 # Description After the upgrade to version catalogs there were changes made in the way our project resolves the dependencies. The new "flow introduced has proven to be quite confusing and problematic. # Solution This PR aims to solve the issues by introducing a platform module that holds all the constraints with the libraries included in our version catalog. This platform module is added to the root modules `solr:api` and `solr.core` as via the `api` configuration, so that it is inherited across all the other modules. This way, it is no longer necessary to add constraints to the `gradle/validation/dependencies.gradle`, but it is sufficient to include the conflicting dependency in the version catalog. # Tests These changes can be manually tested by comparing the libraries included in the output of `gradlew assemble`. There should not be any differences. # Checklist Please review the following and check all that apply: - [X] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [ ] I have created a Jira issue and added the issue ID to my pull request title. - [X] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation) - [X] I have developed this patch against the `main` branch. - [ ] I have run `./gradlew check`. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] SOLR-17582 Stream CLUSTERSTATUS API response [solr]
mlbiscoc opened a new pull request, #2916: URL: https://github.com/apache/solr/pull/2916 https://issues.apache.org/jira/browse/SOLR-17582 # Description `CLUSTERSTATUS` would aggregate and build the whole response including all collection properties into an `NamedList` before finally passing it to the response writers. With many or thousands of collections, this can use up memory. # Solution Instead of returning a `NamedList`, return a `MapWriter`. The Mapwriter will pull an `Iterator` from the documentStream and create `SolrCollectionProperiesIterator` implementing `iterator` and `override` `next()` to create and write the response in time, instead of writing the whole thing in memory. # Tests Fix appropriate tests that were using `NamedList` to `Map` because of the `MapWriter`. # Checklist Please review the following and check all that apply: - [ ] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [ ] I have created a Jira issue and added the issue ID to my pull request title. - [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation) - [ ] I have developed this patch against the `main` branch. - [ ] I have run `./gradlew check`. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17582) CLUSTERSTATUS API could stream the response to save memory
[ https://issues.apache.org/jira/browse/SOLR-17582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SOLR-17582: -- Labels: pull-request-available (was: ) > CLUSTERSTATUS API could stream the response to save memory > -- > > Key: SOLR-17582 > URL: https://issues.apache.org/jira/browse/SOLR-17582 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Reporter: David Smiley >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The CLUSTERSTATUS API currently creates the entire response before submitting > it. If there are many thousands of collections, this will use up memory. It > could easily be modified to stream the response to compute each collection > information just-in-time. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Feature/solr-17334 Minor bugs in Solr dedicated coordinator mode [solr]
dsmiley commented on PR #2672: URL: https://github.com/apache/solr/pull/2672#issuecomment-2551551068 I'm looking at #2527 and #2672 (this one) which seem the same at a glance; why do we have two PRs? It's really sad this didn't get merged before 9.8. Contributors sometime really need to advocate / push for their changes or they get forgotten if no committer is interested. @patsonluk maybe you could take a look at this PR as you have worked on the Coordinator node and run it production (I assume) Another vote of confidence to this PR (actually #2527 if that matters) is that a colleague took it today in order to try out coordinator node. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17405) Zookeeper session can be re-established by multiple threads concurrently
[
https://issues.apache.org/jira/browse/SOLR-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906779#comment-17906779
]
Pierre Salagnac commented on SOLR-17405:
Thanks for pointing this [~janhoy].
Indeed, I haven't been able to write a test that reproduces the race condition.
But we saw it happening quite frequently in case of overloaded Zookeeper, with
multiple session expirations in a row.
I reviewed code in {{{}main{}}}, and it seems to me this bug does not exist
anymore since the move to Curator. We correctly have a single thread to
reestablish everything to Zookeeper ([code
here|https://github.com/apache/solr/blob/f6c5af9c96adef93588f1a66943706eab1d558ff/solr/solrj-zookeeper/src/java/org/apache/solr/common/cloud/SolrZkClient.java#L209]).
I think it is worth to fix this in 9x. I'll open another PR against 9x with
same change.
We run this for a while in production with no issue, and no more repros of the
initial bug. But for sure a review is always welcomed :)
> Zookeeper session can be re-established by multiple threads concurrently
>
>
> Key: SOLR-17405
> URL: https://issues.apache.org/jira/browse/SOLR-17405
> Project: Solr
> Issue Type: Bug
>Affects Versions: 8.11, 9.6
>Reporter: Pierre Salagnac
>Priority: Major
> Labels: pull-request-available
> Attachments: stack.png
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Because of a bug in SolrCloud, the Zookeeper session can be re-established by
> multiple threads concurrently when an expiration occurs.
> This portion of the code assumes it is mono-threaded. Because of the bug, the
> last thread re-establishing the session can waif for 30 seconds per core,
> waiting for it to be marked {{DOWN}} while it was previously marked
> {{ACTIVE}} by another thread. With a high number of cores, the Solr server
> can hang for hours before taking traffic again.
> Following exception shows two threads were reestablishing the session
> concurrently. {{ZkController.createEphemeralLiveNode()}} should never be
> invoked twice for the same Zookeeper session.
> {code:java}
> thrown: java. lang.RuntimeException:
> org.apache.solr.common.cloud.ZooKeeperException:
> at
> org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:178)
> at org.apache.solr. common.cloud.DefaultConnectionStrategy.
> reconnect(DefaultConnectionStrategy.java:57)
> at
> org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:152)
> at org.apache.
> zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510)
> Caused by: org.apache.solr.common.cloud.ZooKeeperException:
> at
> org.apache.solr.cloud.ZkController$1.command(ZkController.java:462)
> at
> org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:170)
> ... 4 more
> Caused by: org.apache. zookeeper.KeeperException$NodeExistsException.
> KeeperErrorCode = NodeExists
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:
> 126)
> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:1925)
> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:1830)
> at
> org.apache.solr.common.cloud.SolrZkClient.lambda$multi$11(SolrZkClient.java:666)
> at
> org.apache.solr.common.cloud.ZkCmdExecutor.retry0peration(ZkCmdExecutor.java:71)
> at
> org.apache.solr.common.cloud.SolrZkClient.multi(SolrZkClient.java:666)
> at org.apache.sol.cloud.ZkController
> CreateEphemeralLiveNode(ZkController.java:1086)
> at
> org.apache.solr.cloud.ZkController$1.command(ZkController.java:411)
> ... 5 more {code}
> h2. Root cause
> This bug occurs because several threads can re-establish the session
> concurrently.
> It cannot happen at the first expiration of the session, thanks to a thread
> pool with a single thread to execute the zookeeper Watcher.
> Bellow is a code snippet from class {{SolrZkClient.ProcessWatchWithExecutor}}
> {code:java}
> if (watcher instanceof ConnectionManager) {
> zkConnManagerCallbackExecutor.submit(() -> watcher.process(event));
> } else {
>...
> }
> {code}
> Using this dedicated thread pool (with a single thread) is supposed to ensure
> we don’t handle watches for connection related events with multiple threads.
> This works well for the first session expiration.
> Now, when we re-establish the session after the first expiration, we don’t
> use this wrapper to register the watch.
> It is done directly in {{ConnectionManager}} without wrapping the ZK watch.
> In the f
[PR] Coordinator: remove SolrQueryRequest.getCloudDescriptor [solr]
dsmiley opened a new pull request, #2913: URL: https://github.com/apache/solr/pull/2913 I don't love seeing SolrQueryRequest.getCloudDescriptor -- we should be conservative of adding more methods to SolrQueryRequest. It only has one caller. Tests pass. If it serves a purpose, removing it ought to demonstrate something's wrong. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Coordinator: remove SolrQueryRequest.getCloudDescriptor [solr]
dsmiley commented on PR #2913: URL: https://github.com/apache/solr/pull/2913#issuecomment-2551617582 @patsonluk do you know about this? **If** there's a problem, it's HttpShardHandler (the only caller) who passes this cloudDescriptor on to `org.apache.solr.handler.component.HttpShardHandler#canShortCircuit` which we want to return false for any coordinator. Maybe it will do so _anyway_. If we want to guarantee it, we could modify this code to recognize the coordinator somehow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17597) CaffeineCache ramBytes incorrect accounting on `put()`
Michael Gibney created SOLR-17597:
-
Summary: CaffeineCache ramBytes incorrect accounting on `put()`
Key: SOLR-17597
URL: https://issues.apache.org/jira/browse/SOLR-17597
Project: Solr
Issue Type: Bug
Affects Versions: 9.7, main (10.0)
Reporter: Michael Gibney
Assignee: Michael Gibney
In most cases, usage of {{CaffeineCache.put()}} results in incorrect ramBytes
accounting. Replaced values (and keys) decrement {{ramBytes}} via
{{onRemoval()}}, so replaced values currently end up double-decrementing for
the replaced value in
[recordRamBytes()|https://github.com/apache/solr/blob/a4229e76771da0125b38204db01ea0934e09c4f4/solr/core/src/java/org/apache/solr/search/CaffeineCache.java#L297-L301].
After a sufficient number of replacements via {{put()}}, {{ramBytes}} becomes
permanently negative, and is thus useless.
The problem was masked in existing tests due to the behavior of the Caffeine
library in the case of reference equality between the old (replaced) value and
the new (replacement) value for the same ({{.equals()}}) key -- [the case
exercised in existing
tests|https://github.com/apache/solr/blob/a4229e76771da0125b38204db01ea0934e09c4f4/solr/core/src/test/org/apache/solr/search/TestCaffeineCache.java#L342-L345]
-- in which case {{onRemoval()}} is _not_ invoked (so there is no double
decrement).
The problem is also masked in normal usage, since existing/default cache
implementations and invocation all currently use {{computeIfAbsent()}} instead
of {{put()}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17582) CLUSTERSTATUS API could stream the response to save memory
[ https://issues.apache.org/jira/browse/SOLR-17582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906801#comment-17906801 ] Matthew Biscocho commented on SOLR-17582: - Took a look at MapWriter like you said and I think I found a solution that works. Made the PR. > CLUSTERSTATUS API could stream the response to save memory > -- > > Key: SOLR-17582 > URL: https://issues.apache.org/jira/browse/SOLR-17582 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Reporter: David Smiley >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The CLUSTERSTATUS API currently creates the entire response before submitting > it. If there are many thousands of collections, this will use up memory. It > could easily be modified to stream the response to compute each collection > information just-in-time. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17597) CaffeineCache ramBytes incorrect accounting on `put()`
[
https://issues.apache.org/jira/browse/SOLR-17597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SOLR-17597:
--
Labels: pull-request-available (was: )
> CaffeineCache ramBytes incorrect accounting on `put()`
> --
>
> Key: SOLR-17597
> URL: https://issues.apache.org/jira/browse/SOLR-17597
> Project: Solr
> Issue Type: Bug
>Affects Versions: main (10.0), 9.7
>Reporter: Michael Gibney
>Assignee: Michael Gibney
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In most cases, usage of {{CaffeineCache.put()}} results in incorrect ramBytes
> accounting. Replaced values (and keys) decrement {{ramBytes}} via
> {{onRemoval()}}, so replaced values currently end up double-decrementing for
> the replaced value in
> [recordRamBytes()|https://github.com/apache/solr/blob/a4229e76771da0125b38204db01ea0934e09c4f4/solr/core/src/java/org/apache/solr/search/CaffeineCache.java#L297-L301].
> After a sufficient number of replacements via {{put()}}, {{ramBytes}} becomes
> permanently negative, and is thus useless.
> The problem was masked in existing tests due to the behavior of the Caffeine
> library in the case of reference equality between the old (replaced) value
> and the new (replacement) value for the same ({{.equals()}}) key -- [the case
> exercised in existing
> tests|https://github.com/apache/solr/blob/a4229e76771da0125b38204db01ea0934e09c4f4/solr/core/src/test/org/apache/solr/search/TestCaffeineCache.java#L342-L345]
> -- in which case {{onRemoval()}} is _not_ invoked (so there is no double
> decrement).
> The problem is also masked in normal usage, since existing/default cache
> implementations and invocation all currently use {{computeIfAbsent()}}
> instead of {{put()}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17597: Fix `CaffeineCache.put()` ramBytes decrement [solr]
magibney commented on PR #2917: URL: https://github.com/apache/solr/pull/2917#issuecomment-2551828172 This probably does not warrant a CHANGES.txt entry, since it doesn't impact any existing/default implementation or invocation of `CaffeineCache`. NOTE: the second (`V oldValue`) arg to the private method `CaffeineCache.recordRamBytes()` is always `null` as now invoked, and could be removed. I will remove this unless anyone objects. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17306: fix replication problem on follower restart [solr]
epugh commented on PR #2874: URL: https://github.com/apache/solr/pull/2874#issuecomment-2551837349 @ds-manzinger I just went to merge, and of course my old enemy, `CHANGES.txt` has merge conflict. Would you mine fixing that and then I'll click the merge button! Lookign forward to getting this in. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16891) Solr Cloud Graph screen blank
[
https://issues.apache.org/jira/browse/SOLR-16891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906836#comment-17906836
]
David Smiley commented on SOLR-16891:
-
I liked this mostly except for one detail – MapWriter implementing
org.noggit.JSONWriter.Writable. Lots of classes implement MapWriter. Thanks
to MapWriter, there are many methods brought in there. And this new one will
be used by nobody except for Noggit. This is a really minor point; there are
many methods and this is just one and is harmless except for maybe
organizational sensibilities / complexity.
As Solr nowadays wholly owns Noggit, we can have Noggit understand Solr's
MapWriter. But, Noggit has no dependencies so it'd also feel wrong to add one
to Solr, albeit there's nothing to it (it's all in the same source path) so
it's completely consequence-less. Shrug.
> Solr Cloud Graph screen blank
> -
>
> Key: SOLR-16891
> URL: https://issues.apache.org/jira/browse/SOLR-16891
> Project: Solr
> Issue Type: Bug
> Components: Admin UI
>Affects Versions: 9.3
>Reporter: Houston Putman
>Assignee: Houston Putman
>Priority: Blocker
> Fix For: 9.3
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The Solr Admin UI - Cloud - Graph screen is blank. This was not an issue in
> Solr 9.2.1, so an API must have changed to make this no longer work.
> After some investigation, the screen uses the Zookeeper API to look at
> {{{}clusterstate.json{}}}, so there must have been some back-compat logic in
> the Zookeeper API to make that work given that {{clusterstate.json}} no
> longer exists. This may no longer work.
> As an easy fix, we can just move to using the Collections ClusterStatus API
> which returns the same data that used to exist in the
> {{{}clusterstate.json{}}}, so we don't even need to map fields or anything.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-15781: Modify v2 `GET /collections/collName` to support full ColStatus response [solr]
gerlowskija commented on PR #2912: URL: https://github.com/apache/solr/pull/2912#issuecomment-2552199105 Alright - COLSTATUS bug if fixed (see comment above), added some additional tests for both v1 and v2, and tests+check pass for me locally! Will aim to merge this in a few days pending any more feedback -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17600) Deprecate and remove MapSerializable
David Smiley created SOLR-17600: --- Summary: Deprecate and remove MapSerializable Key: SOLR-17600 URL: https://issues.apache.org/jira/browse/SOLR-17600 Project: Solr Issue Type: Task Security Level: Public (Default Security Level. Issues are Public) Reporter: David Smiley MapSerializable has a single method toMap(Map) and that which is intended for serialization. It was basically obsoleted by MapWritable that also has this method. Any class implementing MapSerializable should instead implement MapWritable, thus implementing MapWritable.writeMap, and remove redundant toMap implementations. This amounts to replacing toMap with writeMap. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17600) Deprecate and remove MapSerializable
[ https://issues.apache.org/jira/browse/SOLR-17600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley updated SOLR-17600: Fix Version/s: main (10.0) Labels: newdev (was: ) > Deprecate and remove MapSerializable > > > Key: SOLR-17600 > URL: https://issues.apache.org/jira/browse/SOLR-17600 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: David Smiley >Priority: Minor > Labels: newdev > Fix For: main (10.0) > > > MapSerializable has a single method toMap(Map) and that which is intended for > serialization. It was basically obsoleted by MapWritable that also has this > method. Any class implementing MapSerializable should instead implement > MapWritable, thus implementing MapWritable.writeMap, and remove redundant > toMap implementations. This amounts to replacing toMap with writeMap. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16456) ZkNodeProps to implement MapWriter and ZkDistributedQueue.offer() to support MapWriter
[
https://issues.apache.org/jira/browse/SOLR-16456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906890#comment-17906890
]
David Smiley commented on SOLR-16456:
-
Good stuff here but I don't really like MapWriter.append – undocumented and
called in 2 places. Since MapWriter is implemented by a great many things, we
should be conservative adding methods there. I'll throw up a little PR.
> ZkNodeProps to implement MapWriter and ZkDistributedQueue.offer() to support
> MapWriter
> --
>
> Key: SOLR-16456
> URL: https://issues.apache.org/jira/browse/SOLR-16456
> Project: Solr
> Issue Type: Improvement
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> 100% of the time we are submitting a JSON payload using {{Utils.toJSON()}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[PR] Deprecate MapWriter.append [solr]
dsmiley opened a new pull request, #2919: URL: https://github.com/apache/solr/pull/2919 I don't really like MapWriter.append – undocumented and called in only 2 places. Since MapWriter is implemented by a great many things, we should be conservative adding methods there. append() is kind of clever but I think it's outside the scope of what MapWriter should be. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
