[jira] [Created] (SOLR-17582) CLUSTERSTATUS API could stream the response to save memory
David Smiley created SOLR-17582: --- Summary: CLUSTERSTATUS API could stream the response to save memory Key: SOLR-17582 URL: https://issues.apache.org/jira/browse/SOLR-17582 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: SolrCloud Reporter: David Smiley The CLUSTERSTATUS API currently creates the entire response before submitting it. If there are many thousands of collections, this will use up memory. It could easily be modified to stream the response to compute each collection information just-in-time. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16781: Disable by default on 9.x [solr]
gerlowskija commented on code in PR #2894: URL: https://github.com/apache/solr/pull/2894#discussion_r1867891294 ## solr/core/src/java/org/apache/solr/cli/RunExampleTool.java: ## @@ -620,10 +620,15 @@ protected Map startSolr( if (!isWindows && cwdPath.length() > 1 && solrHome.startsWith(cwdPath)) solrHome = solrHome.substring(cwdPath.length() + 1); +final var syspropArg = Review Comment: It's already gone, afaict. @epugh removed it (unrelated to this change) in [a recent PR here](https://github.com/apache/solr/pull/2861/files#diff-1b6e36f0bb5a2f3801d561bd2fa32cb1e354e8342f6cf80e6e6dbfa9ed54d83c) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SolrClientCache switch to Http2SolrClient [solr]
iamsanjay commented on code in PR #2764: URL: https://github.com/apache/solr/pull/2764#discussion_r1867901890 ## solr/modules/sql/src/test/org/apache/solr/handler/sql/TestSQLHandler.java: ## Review Comment: Yeah this one I commented because somewhere in the code they `InputStreamResponseParser` is being used and but didn't closed properly upon exception. I believe it was to create JsonTupleStream, need to check again. So to avoid the ObjectTracker error I commented it then. In the new commit, I uncomment it and pushed the changes. Now the `TestSQLHandler` should fail! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SolrClientCache switch to Http2SolrClient [solr]
iamsanjay commented on code in PR #2764: URL: https://github.com/apache/solr/pull/2764#discussion_r1867908226 ## solr/modules/sql/src/test/org/apache/solr/handler/sql/TestSQLHandler.java: ## Review Comment: Yup failed! ``` ./gradlew :solr:modules:sql:test --tests "org.apache.solr.handler.sql.TestSQLHandler" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=A8C858120DC9821C -Ptests.timeoutSuite=60! -Ptests.file.encoding=UTF-8 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SolrClientCache switch to Http2SolrClient [solr]
iamsanjay commented on code in PR #2764:
URL: https://github.com/apache/solr/pull/2764#discussion_r1867903389
##
solr/core/src/java/org/apache/solr/handler/designer/SchemaDesignerConfigSetHelper.java:
##
@@ -125,46 +119,22 @@ class SchemaDesignerConfigSetHelper implements
SchemaDesignerConstants {
}
@SuppressWarnings("unchecked")
- Map analyzeField(String configSet, String fieldName, String
fieldText)
+ NamedList analyzeField(String configSet, String fieldName, String
fieldText)
Review Comment:
Good stuff!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16781: Disable by default on 9.x [solr]
gerlowskija commented on code in PR #2894:
URL: https://github.com/apache/solr/pull/2894#discussion_r1867906970
##
solr/core/src/test/org/apache/solr/cli/TestSolrCLIRunExample.java:
##
@@ -70,6 +72,7 @@ public static void beforeClass() {
Paths.get(".").toAbsolutePath().toString().contains(" "));
// to be true
System.setProperty("solr.directoryFactory",
"solr.NRTCachingDirectoryFactory");
+System.setProperty(LIB_ENABLED_SYSPROP, "true");
Review Comment:
Good catch, fixed!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
epugh commented on PR #2835: URL: https://github.com/apache/solr/pull/2835#issuecomment-2511772758 Okay, I've responded (I think!) to @risdenk comments.I think this is ready for merging -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17574: Move host allow list cache to AllowListUrlChecker. [solr]
dsmiley commented on code in PR #2892:
URL: https://github.com/apache/solr/pull/2892#discussion_r1867330538
##
solr/solrj/src/java/org/apache/solr/common/cloud/ClusterState.java:
##
@@ -387,7 +386,7 @@ public boolean equals(Object obj) {
/** Internal API used only by ZkStateReader */
void setLiveNodes(Set liveNodes) {
-this.liveNodes = liveNodes;
+this.liveNodes = Set.of(liveNodes.toArray(new String[0]));
Review Comment:
no, call Set.copyOf
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17576) Remove deprecated master/slave configuration support from Replication
[ https://issues.apache.org/jira/browse/SOLR-17576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17902323#comment-17902323 ] ASF subversion and git services commented on SOLR-17576: Commit f9f4a0764b46aa967e15c0faef9f0b3e873f3f41 in solr's branch refs/heads/main from Eric Pugh [ https://gitbox.apache.org/repos/asf?p=solr.git;h=f9f4a0764b4 ] SOLR-17576: Remove deprecated master/slave support in ReplicationHandler (#2887) > Remove deprecated master/slave configuration support from Replication > - > > Key: SOLR-17576 > URL: https://issues.apache.org/jira/browse/SOLR-17576 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) > Components: replication (java) >Affects Versions: main (10.0) >Reporter: Eric Pugh >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Picking up from the work that Marcus started, we can now remove the > deprecated keywords master/slave. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17574: Move host allow list cache to AllowListUrlChecker. [solr]
dsmiley commented on code in PR #2892: URL: https://github.com/apache/solr/pull/2892#discussion_r1867487293 ## solr/CHANGES.txt: ## @@ -229,6 +229,8 @@ Bug Fixes * SOLR-17575: Fixed broken backwards compatibility with the legacy "langid.whitelist" config in Solr Langid. (Jan Høydahl, Alexander Zagniotov) +* SOLR-17574: Move host allow list cache to AllowListUrlChecker (Bruno Roustant, David Smiley) Review Comment: ```suggestion * SOLR-17574: Fix AllowListUrlChecker when liveNodes changes. Remove ClusterState.getHostAllowList (Bruno Roustant, David Smiley) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17574) ClusterState getHostAllowList isn't always updated when live nodes change
[
https://issues.apache.org/jira/browse/SOLR-17574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17902582#comment-17902582
]
David Smiley commented on SOLR-17574:
-
Even though this is a public method on ClusterState; I think we can simply
remove it without back-compat. It's only there to serve AllowListUrlChecker.
> ClusterState getHostAllowList isn't always updated when live nodes change
> -
>
> Key: SOLR-17574
> URL: https://issues.apache.org/jira/browse/SOLR-17574
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: 8.8.2
>Reporter: David Smiley
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> ClusterState.getHostAllowList was introduced in SOLR-15217 to validate a host
> may be reached for distributed-search. But it's a cached value computed from
> the live nodes, and the live nodes may change. Thus it can happen that a new
> live node arrives, gets replicas, and suddenly, distributed-search fails with
> a 403 ("Forbidden"), with an error string containing this substring: "is
> neither a live node of the cluster nor in the configured".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16781: Remove solrconfig.xml directives [solr]
gerlowskija commented on PR #2875: URL: https://github.com/apache/solr/pull/2875#issuecomment-2514995493 One thing we'll probably want/need to come up with a better story around is model-serving. Several places in our docs described using `` as a way not of loading new JARs, but of accessing model files. Users can still raise the ZK filesize limit and access their models from there, so we're not breaking those usecases. But all the same, it'd be great if there was a way to do this that didn't require modifying the ZK limit. (Does solr.xml's `` tag already get us this maybe?) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] [WIP] Jetty12 + EE8 [solr]
gerlowskija commented on code in PR #2876:
URL: https://github.com/apache/solr/pull/2876#discussion_r1868027727
##
solr/core/src/java/org/apache/solr/api/V2HttpCall.java:
##
@@ -434,7 +434,12 @@ protected void executeCoreRequest(SolrQueryResponse rsp) {
// SolrCore counter
core.close();
core = null;
-response.getHeaderNames().stream().forEach(name ->
response.setHeader(name, null));
+// Skip specific headers
+// workaround for response.setHeader(name, null)
+response.getHeaderNames().stream()
Review Comment:
This looks good for now IMO.
Longer term I'd love to see this code go away in favor of having the v2 API
stuff get handled by its own servlet, but we're pretty far from that being
workable for now. It is a little hacky in the interim though 😦
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17576) Remove deprecated master/slave configuration support from Replication
[
https://issues.apache.org/jira/browse/SOLR-17576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17902695#comment-17902695
]
Eric Pugh commented on SOLR-17576:
--
Looking now.. I noticed that it had the annotation:
{color:#8c8c8c}@LuceneTestCase.Nightly{color}
> Remove deprecated master/slave configuration support from Replication
> -
>
> Key: SOLR-17576
> URL: https://issues.apache.org/jira/browse/SOLR-17576
> Project: Solr
> Issue Type: Task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: replication (java)
>Affects Versions: main (10.0)
>Reporter: Eric Pugh
>Assignee: Eric Pugh
>Priority: Major
> Labels: pull-request-available
> Fix For: main (10.0)
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Picking up from the work that Marcus started, we can now remove the
> deprecated keywords master/slave.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16781: Remove solrconfig.xml directives [solr]
gerlowskija commented on code in PR #2875:
URL: https://github.com/apache/solr/pull/2875#discussion_r1868009405
##
solr/core/src/test-files/solr/configsets/upload/with-lib-directive/solrconfig.xml:
##
@@ -37,8 +37,6 @@
${tests.luceneMatchVersion:LATEST}
-
Review Comment:
Good catch - removed!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16781: Remove solrconfig.xml directives [solr]
HoustonPutman commented on PR #2875: URL: https://github.com/apache/solr/pull/2875#issuecomment-2515161658 > One thing we'll probably want/need to come up with a better story around is model-serving. > > Several places in our docs described using `` as a way not of loading new JARs, but of accessing model files. Users can still raise the ZK filesize limit and access their models from there, so we're not breaking those usecases. But all the same, it'd be great if there was a way to do this that didn't require modifying the ZK limit. (Does solr.xml's `` tag already get us this maybe?) Yeah, it should, but there is also the `$SOLR_ROOT_DIR/lib` option as well -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SolrClientCache switch to Http2SolrClient [solr]
dsmiley commented on PR #2764: URL: https://github.com/apache/solr/pull/2764#issuecomment-2515726655 There is another problem I chased down. So I tried the schema designer locally from this branch just to see if it was working. It's not, yet not captured in a test failure (not good). The problem I encountered was a HTTP 400 for the `.system` collection, which is where the schema designer is storing stuff. This is a special collection that is auto-created in HttpSolrCall on the first POST to it. But the schema designer does a GET. I have some WIP to catch the exception on the GET. I'll get back to this later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17583) Bring back documentation for Adding Custom Expressions
cfeldmann created SOLR-17583:
Summary: Bring back documentation for Adding Custom Expressions
Key: SOLR-17583
URL: https://issues.apache.org/jira/browse/SOLR-17583
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: documentation
Affects Versions: 9.7
Reporter: cfeldmann
A change was pushed to do a major revision of the Streaming Expressions guide
almost 4 years ago in version 8.8: SOLR-13105 A visual guide to Solr Math
Expressions and Streaming Expressions
In this change, the following section was removed:
{quote}Adding Custom Expressions
Creating your own custom expressions can be easily done by implementing the
[Expressible|https://lucene.apache.org/solr/8_5_0/solr-solrj/org/apache/solr/client/solrj/io/stream/expr/Expressible.html]
interface. To add a custom expression to the list of known mappings for the
/stream and /graph handlers, you just need to declare it as a plugin in
solrconfig.xml via:
{quote}
There was no mention of deprecating the feature or removing this section.
Given the massive size of the diff, it looks to be an inadvertent removal.
The feature works. There’s no deprecation of this feature mentioned in any of
the Solr docs since it was removed from the guide without explanation. The
code itself is not deprecated.
The Config API doc still includes
add-expressible/update-expressible/delete-expressible
Since the feature still works, has not been deprecated and seems to have been
accidently removed from the documentation, let's bring this section back.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
