[jira] [Updated] (SOLR-17669) Reduce Memory Consumption when using Dynamic Fields
[ https://issues.apache.org/jira/browse/SOLR-17669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Kroiss updated SOLR-17669: Description: Replace the Pattern.matcher for DynamicFields with standard String Operations. In our Environment that fix reduced the Memory Overhead when mapping the Objects by 80-90% (see Screenshot) Pull Request will be opened created by us. Would be great to fix in Solr 9.8.1 was: Replace the Pattern.matcher for DynamicFields with standard String Operations. In our Environment that fix reduced the Memory Overhead when mapping the Objects by 80-90% (see Screenshot) Pull Request will be opened created by us. Would be great to backport to Solr 9.9 > Reduce Memory Consumption when using Dynamic Fields > --- > > Key: SOLR-17669 > URL: https://issues.apache.org/jira/browse/SOLR-17669 > Project: Solr > Issue Type: Improvement > Components: SolrJ >Affects Versions: 9.8 >Reporter: Peter Kroiss >Priority: Major > Labels: pull-request-available > Fix For: 9.8.1 > > Attachments: Screenshot 2025-02-11 at 11.00.15.png > > Time Spent: 10m > Remaining Estimate: 0h > > Replace the Pattern.matcher for DynamicFields with standard String Operations. > In our Environment that fix reduced the Memory Overhead when mapping the > Objects by 80-90% (see Screenshot) > Pull Request will be opened created by us. Would be great to fix in Solr 9.8.1 > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17669) Reduce Memory Consumption when using Dynamic Fields
[ https://issues.apache.org/jira/browse/SOLR-17669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Kroiss updated SOLR-17669: Fix Version/s: 9.8.1 > Reduce Memory Consumption when using Dynamic Fields > --- > > Key: SOLR-17669 > URL: https://issues.apache.org/jira/browse/SOLR-17669 > Project: Solr > Issue Type: Improvement > Components: SolrJ >Affects Versions: 9.8 >Reporter: Peter Kroiss >Priority: Major > Labels: pull-request-available > Fix For: 9.8.1 > > Attachments: Screenshot 2025-02-11 at 11.00.15.png > > Time Spent: 10m > Remaining Estimate: 0h > > Replace the Pattern.matcher for DynamicFields with standard String Operations. > In our Environment that fix reduced the Memory Overhead when mapping the > Objects by 80-90% (see Screenshot) > Pull Request will be opened created by us. Would be great to backport to Solr > 9.9 > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17669) Reduce Memory Consumption when using Dynamic Fields
Peter Kroiss created SOLR-17669: --- Summary: Reduce Memory Consumption when using Dynamic Fields Key: SOLR-17669 URL: https://issues.apache.org/jira/browse/SOLR-17669 Project: Solr Issue Type: Improvement Components: SolrJ Affects Versions: 9.8 Reporter: Peter Kroiss Attachments: Screenshot 2025-02-11 at 11.00.15.png Replace the Pattern.atcher for DynamicFields with standard String Operations. In our Environment that fix reduced the Memory Overhead when mapping the Objects by 80-90% (see Screenshot) Pull Request will be opened created by us. Would be creat to backport to Solr 9.9 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [I] Basic Auth Bootstrap Not Working In 0.9.0 [solr-operator]
idjemaoune commented on issue #755: URL: https://github.com/apache/solr-operator/issues/755#issuecomment-2653520207 I encountered the same issue with version 9.7.0 when creating my cluster from scratch with this version. However, upgrading from version 9.6.1 (with authentication already in place) to later versions works correctly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17669) Reduce Memory Consumption when using Dynamic Fields
[ https://issues.apache.org/jira/browse/SOLR-17669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17926350#comment-17926350 ] Peter Kroiss commented on SOLR-17669: - Implemented by [~dsmanzinger] > Reduce Memory Consumption when using Dynamic Fields > --- > > Key: SOLR-17669 > URL: https://issues.apache.org/jira/browse/SOLR-17669 > Project: Solr > Issue Type: Improvement > Components: SolrJ >Affects Versions: 9.8 >Reporter: Peter Kroiss >Priority: Major > Labels: pull-request-available > Fix For: 9.8.1 > > Attachments: Screenshot 2025-02-11 at 11.00.15.png > > Time Spent: 10m > Remaining Estimate: 0h > > Replace the Pattern.matcher for DynamicFields with standard String Operations. > In our Environment that fix reduced the Memory Overhead when mapping the > Objects by 80-90% (see Screenshot) > Pull Request will be opened created by us. Would be great to fix in Solr 9.8.1 > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Update enabling-ssl.adoc - fix minor typo in example [solr]
Kordishal commented on code in PR #3175: URL: https://github.com/apache/solr/pull/3175#discussion_r1952792666 ## solr/solr-ref-guide/modules/deployment-guide/pages/enabling-ssl.adoc: ## @@ -245,7 +245,7 @@ Windows:: [source,powershell] -C:\> bin/solr.cmd --property urlSchema --value https --zk-host server1:2181,server2:2181,server3:2181 Review Comment: I did not notice that. As I don't have a windows system I can't really confirm this. The script is running the cluster command with the solrcli class. So probably would be documented there. https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/cli/SolrCLI.java and then uses: https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/cli/ClusterTool.java SolrCli has a cluster command so probably this needs cluster too, but I have found no documentation of SolrCLI at all. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] fix the solr zk invocation [solr-operator]
gerlowskija commented on PR #756: URL: https://github.com/apache/solr-operator/pull/756#issuecomment-2653981478 Hi @elangelo - what Solr version are you running? And are there any particular steps needed to reproduce the issue? I tried playing with this a bit this morning but haven't been able to reproduce on the versions I tried (9.7.0 and the unreleased 10.0): ``` # Try without a ZK_HOST initially $ bin/solr zk cp foo.json zk:/foo.json Neither --zk-host or --solr-url parameters provided so assuming solr url is http://localhost:8983. ERROR: Server refused connection at: http://localhost:8983/solr/admin/info/system ... # With ZK_HOST value $ export ZK_HOST=localhost:9765 $ bin/solr zk cp foo.json zk:/foo.json Copying from 'foo.json' to 'zk:/foo.json'. ZooKeeper at localhost:9765 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] fix the solr zk invocation [solr-operator]
elangelo commented on PR #756: URL: https://github.com/apache/solr-operator/pull/756#issuecomment-2654002210 I was doing this with solr 9.8.0 and solr-operator 0.8 When ZK_HOST is set (which is default in the init container) I always get that error message: `Neither --zk-host or --solr-url parameters provided so assuming solr url is http://localhost:8983` that obviously fails as i'm running this on kubernetes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] fix the solr zk invocation [solr-operator]
epugh commented on PR #756: URL: https://github.com/apache/solr-operator/pull/756#issuecomment-2654030295 just a bit of poking, and i don't think we look up from env var the ZK_HOST: https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/cli/CLIUtils.java#L223 eventually, we do have a look up for a Solr URL, but it doesn't use ZK_HOST, instead it uses these processes: https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/cli/CLIUtils.java#L71 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Update enabling-ssl.adoc - fix minor typo in example [solr]
epugh commented on PR #3175: URL: https://github.com/apache/solr/pull/3175#issuecomment-2654044758 I just triggered the builds, and then this looks ready to merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17669) Reduce Memory Consumption when using Dynamic Fields
[ https://issues.apache.org/jira/browse/SOLR-17669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Kroiss updated SOLR-17669: Description: Replace the Pattern.matcher for DynamicFields with standard String Operations. In our Environment that fix reduced the Memory Overhead when mapping the Objects by 80-90% (see Screenshot) Pull Request will be opened created by us. Would be great to backport to Solr 9.9 was: Replace the Pattern.atcher for DynamicFields with standard String Operations. In our Environment that fix reduced the Memory Overhead when mapping the Objects by 80-90% (see Screenshot) Pull Request will be opened created by us. Would be creat to backport to Solr 9.9 > Reduce Memory Consumption when using Dynamic Fields > --- > > Key: SOLR-17669 > URL: https://issues.apache.org/jira/browse/SOLR-17669 > Project: Solr > Issue Type: Improvement > Components: SolrJ >Affects Versions: 9.8 >Reporter: Peter Kroiss >Priority: Major > Attachments: Screenshot 2025-02-11 at 11.00.15.png > > > Replace the Pattern.matcher for DynamicFields with standard String Operations. > In our Environment that fix reduced the Memory Overhead when mapping the > Objects by 80-90% (see Screenshot) > Pull Request will be opened created by us. Would be great to backport to Solr > 9.9 > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] SOLR-17669: Reduce Memory Consumption by 80-90% when using Dynamic fields (DocumentObjectBinder) [solr]
ds-manzinger opened a new pull request, #3179: URL: https://github.com/apache/solr/pull/3179 https://issues.apache.org/jira/browse/SOLR-17669 # Description Flame Graph in our Production System showed a siginificant high memory consumption of Pattern.matcher # Solution Wildcard can only be at beginning or end of Field Name, we replaced Pattern Regex with startsWith and endsWith # Tests No new tests needed, old tests are working fine with new code # Checklist Please review the following and check all that apply: - [x] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [x] I have created a Jira issue and added the issue ID to my pull request title. - [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation) - [x] I have developed this patch against the `main` branch. - [x] I have run `./gradlew check`. - [x] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17669) Reduce Memory Consumption when using Dynamic Fields
[ https://issues.apache.org/jira/browse/SOLR-17669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SOLR-17669: -- Labels: pull-request-available (was: ) > Reduce Memory Consumption when using Dynamic Fields > --- > > Key: SOLR-17669 > URL: https://issues.apache.org/jira/browse/SOLR-17669 > Project: Solr > Issue Type: Improvement > Components: SolrJ >Affects Versions: 9.8 >Reporter: Peter Kroiss >Priority: Major > Labels: pull-request-available > Attachments: Screenshot 2025-02-11 at 11.00.15.png > > Time Spent: 10m > Remaining Estimate: 0h > > Replace the Pattern.matcher for DynamicFields with standard String Operations. > In our Environment that fix reduced the Memory Overhead when mapping the > Objects by 80-90% (see Screenshot) > Pull Request will be opened created by us. Would be great to backport to Solr > 9.9 > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Update enabling-ssl.adoc - fix minor typo in example [solr]
epugh commented on PR #3175: URL: https://github.com/apache/solr/pull/3175#issuecomment-2654043373 I searched our ref guide for "solrcli" and yeah, the main page, the "Solr Control Script" page doesn't come up!: https://github.com/user-attachments/assets/a4bd3597-a972-4b84-a77d-8b1b888dd0b5"; /> We should add a "aka SolrCLI" on that page so it comes up! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] fix the solr zk invocation [solr-operator]
epugh commented on PR #756: URL: https://github.com/apache/solr-operator/pull/756#issuecomment-2654032493 personally, when it comes to invoking the SolrCLI, I like the verbosity of passing in all the args, so there is less magic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Transfer encryption metadata in the commit when DirectUpdateHandler2.closeWriter is called. [solr-sandbox]
bruno-roustant merged PR #113: URL: https://github.com/apache/solr-sandbox/pull/113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [I] Failure loading SolrCore at startup - java.util.NoSuchElementException: No key id for key ref=0 after server restart [solr-sandbox]
bruno-roustant commented on issue #109: URL: https://github.com/apache/solr-sandbox/issues/109#issuecomment-2654226876 This issue is fixed now. The fix is in Solr 9.8, and the encryption module now depends on 9.8. The PR mentioned above tests the node restart and the successful transfer of encryption commit metadata. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17635: unmarshalling Map to SimpleOrderedMap if key i of type St… [solr]
renatoh commented on code in PR #3163:
URL: https://github.com/apache/solr/pull/3163#discussion_r1953918318
##
solr/solrj/src/java/org/apache/solr/common/util/Utils.java:
##
@@ -290,7 +290,7 @@ public static Object fromJSON(byte[] utf8) {
}
public static Object fromJSON(byte[] utf8, int offset, int length) {
-return fromJSON(utf8, offset, length, STANDARDOBJBUILDER);
+return fromJSON(utf8, offset, length, SIMPLEORDEREDMAPOBJBUILDER);
Review Comment:
Looking at a test case
ClusterStateProviderTest.testClusterStateProviderEmptySolrVersion,
Http2ClusterStateProvider is using JavaBinCode to deserialize, hence we get now
a SOM, ZkClientClusterStateProvider is using JSON and use the Utils.fromJson to
deserialize, hence it still gets LinkedHashMap. In this test it expects the
same data structure from both Http2ClusterStateProvider and
ZkClientClusterStateProvider, hence it's failing now.
If I change the Utils to return SMO instead of a LinkedHashMap, as I did in
this change, we break other tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17635: unmarshalling Map to SimpleOrderedMap if key i of type St… [solr]
renatoh commented on code in PR #3163:
URL: https://github.com/apache/solr/pull/3163#discussion_r1953918318
##
solr/solrj/src/java/org/apache/solr/common/util/Utils.java:
##
@@ -290,7 +290,7 @@ public static Object fromJSON(byte[] utf8) {
}
public static Object fromJSON(byte[] utf8, int offset, int length) {
-return fromJSON(utf8, offset, length, STANDARDOBJBUILDER);
+return fromJSON(utf8, offset, length, SIMPLEORDEREDMAPOBJBUILDER);
Review Comment:
Looking at a test case
ClusterStateProviderTest.testClusterStateProviderEmptySolrVersion,
Http2ClusterStateProvider is using JavaBinCode to deserialize, hence we get now
a SOM, ZkClientClusterStateProvider is using JSON and use the Utils.fromJson to
deserialize, hence it still gets LinkedHashMap. In this test it expects the
same data structure from both Http2ClusterStateProvider and
ZkClientClusterStateProvider, hence it's failing now.
If I change the Utils to return SMO instead of a LinkedHashMap, as I did in
this change, all the tests in ClusterStateProviderTest pass, but it breaks 17
other tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17635: unmarshalling Map to SimpleOrderedMap if key i of type St… [solr]
renatoh commented on code in PR #3163:
URL: https://github.com/apache/solr/pull/3163#discussion_r1953918318
##
solr/solrj/src/java/org/apache/solr/common/util/Utils.java:
##
@@ -290,7 +290,7 @@ public static Object fromJSON(byte[] utf8) {
}
public static Object fromJSON(byte[] utf8, int offset, int length) {
-return fromJSON(utf8, offset, length, STANDARDOBJBUILDER);
+return fromJSON(utf8, offset, length, SIMPLEORDEREDMAPOBJBUILDER);
Review Comment:
Looking at a test case
ClusterStateProviderTest.testClusterStateProviderEmptySolrVersion,
Http2ClusterStateProvider is using JavaBinCode to deserialize, hence we get now
a SOM, ZkClientClusterStateProvider is using JSON and use the Utils.fromJson to
deserialize, and it still gets a LinkedHashMap. In this test it expects the
same data structure from both Http2ClusterStateProvider and
ZkClientClusterStateProvider, hence it's failing now.
If I change the Utils to return SMO instead of a LinkedHashMap, as I did in
this change, all the tests in ClusterStateProviderTest pass, but it breaks 17
other tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17635: unmarshalling Map to SimpleOrderedMap if key i of type St… [solr]
renatoh commented on code in PR #3163:
URL: https://github.com/apache/solr/pull/3163#discussion_r1953918318
##
solr/solrj/src/java/org/apache/solr/common/util/Utils.java:
##
@@ -290,7 +290,7 @@ public static Object fromJSON(byte[] utf8) {
}
public static Object fromJSON(byte[] utf8, int offset, int length) {
-return fromJSON(utf8, offset, length, STANDARDOBJBUILDER);
+return fromJSON(utf8, offset, length, SIMPLEORDEREDMAPOBJBUILDER);
Review Comment:
Looking at a test case
ClusterStateProviderTest.testClusterStateProviderEmptySolrVersion,
Http2ClusterStateProvider is using JavaBinCode to deserialize, hence we get now
a SOM, ZkClientClusterStateProvider is using JSON and usees the Utils.fromJson
to deserialize, and it gets a LinkedHashMap. In this test it expects the same
data structure from both Http2ClusterStateProvider and
ZkClientClusterStateProvider, hence it's failing now.
If I change the Utils to return SMO instead of a LinkedHashMap, as I did in
this change, all the tests in ClusterStateProviderTest pass, but it breaks at
least 17 other tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17663: Protect TimeOut from overflow. [solr]
bruno-roustant merged PR #3173: URL: https://github.com/apache/solr/pull/3173 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17663) TimeOut overflows if the provided interval is large
[ https://issues.apache.org/jira/browse/SOLR-17663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17926698#comment-17926698 ] ASF subversion and git services commented on SOLR-17663: Commit 38f4ec7e1dc2b4145aec985049f8ec76916a5aab in solr's branch refs/heads/main from Bruno Roustant [ https://gitbox.apache.org/repos/asf?p=solr.git;h=38f4ec7e1dc ] SOLR-17663: Protect TimeOut from overflow. (#3173) > TimeOut overflows if the provided interval is large > --- > > Key: SOLR-17663 > URL: https://issues.apache.org/jira/browse/SOLR-17663 > Project: Solr > Issue Type: Bug >Affects Versions: 9.8 >Reporter: Bruno Roustant >Priority: Minor > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > TimeOut constructor is passed a long interval parameter. If this long is > large enough, the timeoutAt field overflows, making all the methods > incorrect, including hasTimedOut() which returns true immediately. > TimeUnit.convert() deals with the same problem by checking the value to > convert ahead of time and by setting fixed Long.MAX_VALUE to prevent > overflow. We should do the same for timeoutAt in TimeOut constructor. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17663) TimeOut overflows if the provided interval is large
[ https://issues.apache.org/jira/browse/SOLR-17663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17926700#comment-17926700 ] ASF subversion and git services commented on SOLR-17663: Commit 933abef4dbe137343e196818b8d16653d70c0667 in solr's branch refs/heads/branch_9x from Bruno Roustant [ https://gitbox.apache.org/repos/asf?p=solr.git;h=933abef4dbe ] SOLR-17663: Protect TimeOut from overflow. (#3173) > TimeOut overflows if the provided interval is large > --- > > Key: SOLR-17663 > URL: https://issues.apache.org/jira/browse/SOLR-17663 > Project: Solr > Issue Type: Bug >Affects Versions: 9.8 >Reporter: Bruno Roustant >Priority: Minor > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > TimeOut constructor is passed a long interval parameter. If this long is > large enough, the timeoutAt field overflows, making all the methods > incorrect, including hasTimedOut() which returns true immediately. > TimeUnit.convert() deals with the same problem by checking the value to > convert ahead of time and by setting fixed Long.MAX_VALUE to prevent > overflow. We should do the same for timeoutAt in TimeOut constructor. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Resolved] (SOLR-17663) TimeOut overflows if the provided interval is large
[ https://issues.apache.org/jira/browse/SOLR-17663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Roustant resolved SOLR-17663. --- Fix Version/s: 9.9 Resolution: Fixed > TimeOut overflows if the provided interval is large > --- > > Key: SOLR-17663 > URL: https://issues.apache.org/jira/browse/SOLR-17663 > Project: Solr > Issue Type: Bug >Affects Versions: 9.8 >Reporter: Bruno Roustant >Priority: Minor > Labels: pull-request-available > Fix For: 9.9 > > Time Spent: 40m > Remaining Estimate: 0h > > TimeOut constructor is passed a long interval parameter. If this long is > large enough, the timeoutAt field overflows, making all the methods > incorrect, including hasTimedOut() which returns true immediately. > TimeUnit.convert() deals with the same problem by checking the value to > convert ahead of time and by setting fixed Long.MAX_VALUE to prevent > overflow. We should do the same for timeoutAt in TimeOut constructor. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17663: Protect TimeOut from overflow. [solr]
bruno-roustant commented on PR #3173: URL: https://github.com/apache/solr/pull/3173#issuecomment-2655756233 Thanks for your review David. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] SOLR-17670: Fix unnecessary memory allocation caused by a large reRankDocs param [solr]
gaojiabao1991 opened a new pull request, #3181:
URL: https://github.com/apache/solr/pull/3181
https://issues.apache.org/jira/browse/SOLR-X
# Description
For a query containing re-ranking, such as:
{
"start": "0",
"rows": 10,
"fl": "ID,score",
"q": "*:*",
"rq": "{!rerank reRankQuery='{!func} 100' reRankDocs=10
reRankWeight=2}"
}
The current execution logic is as follows:
1. Perform normal retrieval using the q parameter.
2. Re-score all documents retrieved in the q phase using the rq parameter.
During the retrieval in phase 1 (using q), a TopScoreDocCollector is
created. Underneath, this creates a PriorityQueue which contains an Object[].
The length of this Object[] continuously increases with reRankDocs without any
limit.
On my local test cluster with limited JVM memory, this can even trigger an
OOM, causing the Solr node to crash. I can also reproduce the OOM situation
using the SolrCloudTestCase unit test.
# Solution
I think limiting the length of the Object[] array using
searcher.getIndexReader().maxDoc() at ReRankCollector would resolve this issue.
This way, when reRankDocs exceeds maxDoc, memory allocation will not continue
to increase indefinitely.
# Tests
TestRerankOnSolrCloud will reproduce the OOM situation without my fix.
# Checklist
Please review the following and check all that apply:
- [ ] I have reviewed the guidelines for [How to
Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my
code conforms to the standards described there to the best of my ability.
- [ ] I have created a Jira issue and added the issue ID to my pull request
title.
- [ ] I have given Solr maintainers
[access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork)
to contribute to my PR branch. (optional but recommended, not available for
branches on forks living under an organisation)
- [ ] I have developed this patch against the `main` branch.
- [ ] I have run `./gradlew check`.
- [ ] I have added tests for my changes.
- [ ] I have added documentation for the [Reference
Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17670) Fix unnecessary memory allocation caused by a large reRankDocs param
[
https://issues.apache.org/jira/browse/SOLR-17670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SOLR-17670:
--
Labels: pull-request-available (was: )
> Fix unnecessary memory allocation caused by a large reRankDocs param
>
>
> Key: SOLR-17670
> URL: https://issues.apache.org/jira/browse/SOLR-17670
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: JiaBaoGao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The reRank function has a reRankDocs parameter that specifies the number of
> documents to re-rank. I've observed that increasing this parameter to test
> its performance impact causes queries to become progressively slower. Even
> when the parameter value exceeds the total number of documents in the index,
> further increases continue to slow down the query, which is counterintuitive.
>
> Therefore, I investigated the code:
>
> For a query containing re-ranking, such as:
> {code:java}
> {
> "start": "0",
> "rows": 10,
> "fl": "ID,score",
> "q": "*:*",
> "rq": "{!rerank reRankQuery='{!func} 100' reRankDocs=10
> reRankWeight=2}"
> } {code}
>
> The current execution logic is as follows:
> 1. Perform normal retrieval using the q parameter.
> 2. Re-score all documents retrieved in the q phase using the rq parameter.
>
> During the retrieval in phase 1 (using q), a TopScoreDocCollector is created.
> Underneath, this creates a PriorityQueue which contains an Object[]. The
> length of this Object[] continuously increases with reRankDocs without any
> limit.
>
> On my local test cluster with limited JVM memory, this can even trigger an
> OOM, causing the Solr node to crash. I can also reproduce the OOM situation
> using the SolrCloudTestCase unit test.
>
> I think limiting the length of the Object[] array using
> searcher.getIndexReader().maxDoc() at ReRankCollector would resolve this
> issue. This way, when reRankDocs exceeds maxDoc, memory allocation will not
> continue to increase indefinitely.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17635: unmarshalling Map to SimpleOrderedMap if key i of type St… [solr]
renatoh commented on code in PR #3163:
URL: https://github.com/apache/solr/pull/3163#discussion_r1953320836
##
solr/solrj/src/java/org/apache/solr/common/util/Utils.java:
##
@@ -290,7 +290,7 @@ public static Object fromJSON(byte[] utf8) {
}
public static Object fromJSON(byte[] utf8, int offset, int length) {
-return fromJSON(utf8, offset, length, STANDARDOBJBUILDER);
+return fromJSON(utf8, offset, length, SIMPLEORDEREDMAPOBJBUILDER);
Review Comment:
this breaks another 17 tests.
@dsmiley Do you think it is OK to run the test ClusterStateProviderTest with
solr.solrj.javabin.mapAsNamedList=false?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[PR] build solr/webapp/build.gradle: remove needless excludes [solr]
dsmiley opened a new pull request, #3180: URL: https://github.com/apache/solr/pull/3180 These exclusions don't match anything. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Resolved] (SOLR-17649) Multivalue facets on enum field type returns empty result when using JsonFacet API
[
https://issues.apache.org/jira/browse/SOLR-17649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Smiley resolved SOLR-17649.
-
Fix Version/s: 9.9
Assignee: David Smiley
Resolution: Fixed
Thanks for contributing Thomas!
> Multivalue facets on enum field type returns empty result when using
> JsonFacet API
> --
>
> Key: SOLR-17649
> URL: https://issues.apache.org/jira/browse/SOLR-17649
> Project: Solr
> Issue Type: Bug
> Components: Facet Module, FacetComponent, faceting
>Affects Versions: 9.4, 9.5, 9.4.1, 9.6, 9.7, 9.6.1, 9.8
>Reporter: Thomas Wöckinger
>Assignee: David Smiley
>Priority: Major
> Labels: pull-request-available
> Fix For: 9.9
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> When using JsonFacet API on a multivalued EnumFieldType with facet method
> 'enum' the {color:#232629}FacetFieldProcessorByArrayDV{color} will be used.
> At line 96 ({color:#232629}FacetFieldProcessorByArrayDV.java){color} there is
> a check about allBuckets or missing buckets, which simply skip collect
> process.
> So at the moment there is no support for Multivalue faceting on facets which
> are using FacetFieldProcessorByArrayDV facet processor for collecting facets.
> Tested this behavior from 9.4 onwards, this feature was working on the 8.x
> releases.
> Multivalue faceting on EnumFieldType should be supported again.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Typo in enabling-ssl.adoc [solr]
janhoy commented on PR #3177: URL: https://github.com/apache/solr/pull/3177#issuecomment-2655131003 Sorry, but I cannot see that this is a bug or why this change would be any better than the default in the docs. The docs contain the same values as commented in our `solr.in.sh/cmd` file, and the paths are resolved relative to solr root dir. I.e. if you place your keystore files in `solr/server/etc/` then this works. Of course, if you place your keystore elsewhere you need to modify those values, but I cannot see why `/etc/` would be a better default? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [I] Unable to install solr-operator 0.8.1 - zookeeper operator post install job failure [solr-operator]
larsskj commented on issue #728: URL: https://github.com/apache/solr-operator/issues/728#issuecomment-2655142543 I'm hit by the same problem - and can elaborate with some more information. I want to run a Solr cloud and thought installing it using the Helm charts would be the easiest. Well - it is, until... But let me start with a bit of context: At home - where I want my Solr cloud - I have a small K8s cluster with four relatively high powered nodes running a lot of websites - all non-profit. The nodes have Core i7 CPUs and 64 GB memory, and on top of that I have installed Rook/Ceph running on four 2 TB SSDs. This works like a charm and runs some medium traffic websites - I'm quite happy about the setup. Next to the cluster, I have a Qnap NAS box offering, among other things, NFS shares. I've spent a couple of days trying to make this work - but I run into exactly the same problems as described here in the opening post. My problems persist, though. I have tried both letting the Solr Helm chart install everyting, and I have tried installing the ZK Helm first. I have tried removing the CRDs from the cluster and reinstalling them in various ways - to no avail. BUT: I can easily install a working solution - as long as I don't use persistent storage. Installing a cluster with ephemeral storage works out of the box. But when I ask the Helm installer to use Ceph RBD or CephFS, the ZK deployment fails exactly as described here. I then made a desperate attempt: Instead of Ceph, I tried using NFS on my NAS box as persistent storage - and voila, Helm build a working setup! That's not a viable solution moving forward as the NAS box becomes a single point of failure - but it proves that the solution works with some persistent storages. I use my Ceph solution a lot and it has never caused problems before - I'm really puzzled as to what goes on here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17649) Multivalue facets on enum field type returns empty result when using JsonFacet API
[
https://issues.apache.org/jira/browse/SOLR-17649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17926610#comment-17926610
]
ASF subversion and git services commented on SOLR-17649:
Commit 17f935fdfbbfcb9f8e47299c4c323b9a558a44a3 in solr's branch
refs/heads/branch_9x from Thomas Wöckinger
[ https://gitbox.apache.org/repos/asf?p=solr.git;h=17f935fdfbb ]
SOLR-17649: Fix JSON faceting on multiValued number types (#3158)
* Add DV type checks to alert to a problem instead of silently returning nothing
* Clarify isNumeric
-
Co-authored-by: Thomas Wöckinger
Co-authored-by: David Smiley
(cherry picked from commit f66aa9ed07fa76b5db45186073a9b2ac20c01fd4)
> Multivalue facets on enum field type returns empty result when using
> JsonFacet API
> --
>
> Key: SOLR-17649
> URL: https://issues.apache.org/jira/browse/SOLR-17649
> Project: Solr
> Issue Type: Bug
> Components: Facet Module, FacetComponent, faceting
>Affects Versions: 9.4, 9.5, 9.4.1, 9.6, 9.7, 9.6.1, 9.8
>Reporter: Thomas Wöckinger
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> When using JsonFacet API on a multivalued EnumFieldType with facet method
> 'enum' the {color:#232629}FacetFieldProcessorByArrayDV{color} will be used.
> At line 96 ({color:#232629}FacetFieldProcessorByArrayDV.java){color} there is
> a check about allBuckets or missing buckets, which simply skip collect
> process.
> So at the moment there is no support for Multivalue faceting on facets which
> are using FacetFieldProcessorByArrayDV facet processor for collecting facets.
> Tested this behavior from 9.4 onwards, this feature was working on the 8.x
> releases.
> Multivalue faceting on EnumFieldType should be supported again.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] build solr/webapp/build.gradle: remove needless excludes [solr]
HoustonPutman commented on PR #3180: URL: https://github.com/apache/solr/pull/3180#issuecomment-2655094380 Yeah, I guess the non-minimized files used to be in source control? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17635: unmarshalling Map to SimpleOrderedMap if key i of type St… [solr]
dsmiley commented on code in PR #3163:
URL: https://github.com/apache/solr/pull/3163#discussion_r1953556107
##
solr/solrj/src/java/org/apache/solr/common/util/TextWriter.java:
##
@@ -84,11 +84,11 @@ default void writeVal(String name, Object val, boolean raw)
throws IOException {
}
} else if (val instanceof IteratorWriter) {
writeIterator(name, (IteratorWriter) val, raw);
-} else if (val instanceof MapWriter) {
+} else if (val instanceof MapWriter && !(val instanceof SimpleOrderedMap))
{
writeMap(name, (MapWriter) val);
} else if (val instanceof ReflectWritable) {
writeVal(name, Utils.getReflectWriter(val));
-} else if (val instanceof MapSerializable) {
+} else if (val instanceof MapSerializable && !(val instanceof
SimpleOrderedMap)) {
Review Comment:
why this line? SOM isn't MapSerializable
##
solr/solrj/src/java/org/apache/solr/common/util/TextWriter.java:
##
@@ -84,11 +84,11 @@ default void writeVal(String name, Object val, boolean raw)
throws IOException {
}
} else if (val instanceof IteratorWriter) {
writeIterator(name, (IteratorWriter) val, raw);
-} else if (val instanceof MapWriter) {
+} else if (val instanceof MapWriter && !(val instanceof SimpleOrderedMap))
{
writeMap(name, (MapWriter) val);
} else if (val instanceof ReflectWritable) {
writeVal(name, Utils.getReflectWriter(val));
-} else if (val instanceof MapSerializable) {
+} else if (val instanceof MapSerializable && !(val instanceof
SimpleOrderedMap)) {
// todo find a better way to reuse the map more efficiently
writeMap(name, ((MapSerializable) val).toMap(new LinkedHashMap<>()),
false, true);
} else if (val instanceof Map) {
Review Comment:
this should simply be ranked higher than MapWriter, where it will then apply
to SimpleOrderedMap.
##
solr/solrj/src/java/org/apache/solr/common/util/Utils.java:
##
@@ -290,7 +290,7 @@ public static Object fromJSON(byte[] utf8) {
}
public static Object fromJSON(byte[] utf8, int offset, int length) {
-return fromJSON(utf8, offset, length, STANDARDOBJBUILDER);
+return fromJSON(utf8, offset, length, SIMPLEORDEREDMAPOBJBUILDER);
Review Comment:
No; we should understand why the test or underlying functionality are
breaking and then make decisions based on that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [I] Failure loading SolrCore at startup - java.util.NoSuchElementException: No key id for key ref=0 after server restart [solr-sandbox]
bruno-roustant closed issue #109: Failure loading SolrCore at startup - java.util.NoSuchElementException: No key id for key ref=0 after server restart URL: https://github.com/apache/solr-sandbox/issues/109 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SolrRequest.getParams never null; and clarify mutability [solr]
dsmiley merged PR #3140: URL: https://github.com/apache/solr/pull/3140 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17649: Fix Json faceting on multivalue number types [solr]
dsmiley merged PR #3158: URL: https://github.com/apache/solr/pull/3158 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17649) Multivalue facets on enum field type returns empty result when using JsonFacet API
[
https://issues.apache.org/jira/browse/SOLR-17649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17926592#comment-17926592
]
ASF subversion and git services commented on SOLR-17649:
Commit f66aa9ed07fa76b5db45186073a9b2ac20c01fd4 in solr's branch
refs/heads/main from Thomas Wöckinger
[ https://gitbox.apache.org/repos/asf?p=solr.git;h=f66aa9ed07f ]
SOLR-17649: Fix JSON faceting on multiValued number types (#3158)
* Add DV type checks to alert to a problem instead of silently returning nothing
* Clarify isNumeric
-
Co-authored-by: Thomas Wöckinger
Co-authored-by: David Smiley
> Multivalue facets on enum field type returns empty result when using
> JsonFacet API
> --
>
> Key: SOLR-17649
> URL: https://issues.apache.org/jira/browse/SOLR-17649
> Project: Solr
> Issue Type: Bug
> Components: Facet Module, FacetComponent, faceting
>Affects Versions: 9.4, 9.5, 9.4.1, 9.6, 9.7, 9.6.1, 9.8
>Reporter: Thomas Wöckinger
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> When using JsonFacet API on a multivalued EnumFieldType with facet method
> 'enum' the {color:#232629}FacetFieldProcessorByArrayDV{color} will be used.
> At line 96 ({color:#232629}FacetFieldProcessorByArrayDV.java){color} there is
> a check about allBuckets or missing buckets, which simply skip collect
> process.
> So at the moment there is no support for Multivalue faceting on facets which
> are using FacetFieldProcessorByArrayDV facet processor for collecting facets.
> Tested this behavior from 9.4 onwards, this feature was working on the 8.x
> releases.
> Multivalue faceting on EnumFieldType should be supported again.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Add NavigableObject.wrap. Deprecate MapWriter.EMPTY and MapWriterMap. [solr]
dsmiley merged PR #3148: URL: https://github.com/apache/solr/pull/3148 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Fix DockMakerTest maxCardinality [solr]
dsmiley commented on PR #3171: URL: https://github.com/apache/solr/pull/3171#issuecomment-2654944730 Will merge tomorrow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SolrParams.equals implementation [solr]
dsmiley merged PR #3141: URL: https://github.com/apache/solr/pull/3141 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17670) Fix unnecessary memory allocation caused by a large reRankDocs param
JiaBaoGao created SOLR-17670:
Summary: Fix unnecessary memory allocation caused by a large
reRankDocs param
Key: SOLR-17670
URL: https://issues.apache.org/jira/browse/SOLR-17670
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Reporter: JiaBaoGao
The reRank function has a reRankDocs parameter that specifies the number of
documents to re-rank. I've observed that increasing this parameter to test its
performance impact causes queries to become progressively slower. Even when the
parameter value exceeds the total number of documents in the index, further
increases continue to slow down the query, which is counterintuitive.
Therefore, I investigated the code:
For a query containing re-ranking, such as:
{code:java}
{
"start": "0",
"rows": 10,
"fl": "ID,score",
"q": "*:*",
"rq": "{!rerank reRankQuery='{!func} 100' reRankDocs=10 reRankWeight=2}"
} {code}
The current execution logic is as follows:
1. Perform normal retrieval using the q parameter.
2. Re-score all documents retrieved in the q phase using the rq parameter.
During the retrieval in phase 1 (using q), a TopScoreDocCollector is created.
Underneath, this creates a PriorityQueue which contains an Object[]. The length
of this Object[] continuously increases with reRankDocs without any limit.
On my local test cluster with limited JVM memory, this can even trigger an OOM,
causing the Solr node to crash. I can also reproduce the OOM situation using
the SolrCloudTestCase unit test.
I think limiting the length of the Object[] array using
searcher.getIndexReader().maxDoc() at ReRankCollector would resolve this issue.
This way, when reRankDocs exceeds maxDoc, memory allocation will not continue
to increase indefinitely.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
