Re: [PR] SOLR-17525: Text Embedder Query Parser [solr]
alessandrobenedetti commented on PR #2809: URL: https://github.com/apache/solr/pull/2809#issuecomment-2484982643 I've finished another round of polishing following review suggestions, thanks @dsmiley , @cpoerschke and anyone else involved! 1) I modified the class loading, bringing the SolrResourceLoader back (it was there in an initial iteration). If no additional feedback comes, I'll consider the SolrEmbeddingModel getInstance, done 2) What I've done in the 'solr/server/etc/security.policy' is recommended/ the correct way to enable what we want here? i.e. the call to external vectorisation services through a third party API? (LangChain4j) If these two points get approval, I'll be happy to align to https://github.com/apache/solr/pull/2706, introduce the right entry in CHANGES and merge! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17567) Improve Stream CLI implementation
[ https://issues.apache.org/jira/browse/SOLR-17567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SOLR-17567: -- Labels: cli pull-request-available streams (was: cli streams) > Improve Stream CLI implementation > - > > Key: SOLR-17567 > URL: https://issues.apache.org/jira/browse/SOLR-17567 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: cli >Reporter: Christos Malliaridis >Priority: Major > Labels: cli, pull-request-available, streams > Time Spent: 10m > Remaining Estimate: 0h > > The current implemention for the CLI's Streams command that was introduced in > SOLR-14673 has multiple issues that should be addressed, including but not > limited to: > - Use of raw strings in {{Option#getOptionValue()}} > - Overlapping options ({{\-f}} and {{\-e}}) > - Multiple type suppressions that can be avoided > - A bug that causes specific bats tests to fail > - StreamTool.LocalCatStream overrides CatStream btu does not properly set > commaDelimitedFilepaths (this may be a bug) > - Obsolete LetStream#getLetParams() was introduced > - Function StreamTool.constructStream seems obsolete and occurrences can be > replaced with implementation > - Implementation of StreamTool.readExpression does not take leading spaces > infront of comment lines into account > - Implementation of StreamTool.readExpression does not support {{*/}} at the > end of line -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] SOLR-17567: Improve Stream CLI implementation [solr]
epugh opened a new pull request, #2872: URL: https://github.com/apache/solr/pull/2872 https://issues.apache.org/jira/browse/SOLR-17567 This is to improve StreamTool based on some assessments by @malliaridis! - [ ] Use of raw strings in Option#getOptionValue() - [ ] Overlapping options (-f and -e) - [ ] Multiple type suppressions that can be avoided - [ ] A bug that causes specific bats tests to fail. **done in seperate PR by @malliaridis** ! - [ ] StreamTool.LocalCatStream overrides CatStream btu does not properly set commaDelimitedFilepaths (this may be a bug) - [ ]Obsolete LetStream#getLetParams() was introduced - [ ] Function StreamTool.constructStream seems obsolete and occurrences can be replaced with implementation - [ ] Implementation of StreamTool.readExpression does not take leading spaces infront of comment lines into account - [ ] Implementation of StreamTool.readExpression does not support */ at the end of line -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17525: Text Embedder Query Parser [solr]
alessandrobenedetti commented on PR #2809: URL: https://github.com/apache/solr/pull/2809#issuecomment-2485082701 hi @malliaridis, first of all nice to meet you! When rebasing on upstream I ended up with the new dependency approach, given it's not released I had to dig a bit in the Pull Request for documentation and ended up with my latest commit (that doesn't work). I am no Gradle expert, I can definitely spend some time on this in the next few days but given you worked on https://issues.apache.org/jira/browse/SOLR-17406, maybe can fix what I didn't do correctly? Thanks in advance! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] SOLR-17306: fix replication problem on follower restart [solr]
ds-manzinger opened a new pull request, #2874: URL: https://github.com/apache/solr/pull/2874 https://issues.apache.org/jira/browse/SOLR-17306 Same as https://github.com/apache/solr/pull/2873 -> but now against main branch # Description If Leader has Replication disabled - do not delete Followers data on restart # Solution Check if Leader Replication is enabled # Tests Implemented Unit Tests, that check different restart scenarios. Enable Directory Storage for Replica, othwerwise tests will not work because memory is cleaned on restart # Checklist Please review the following and check all that apply: - [x] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [x] I have created a Jira issue and added the issue ID to my pull request title. - [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation) - [ ] I have developed this patch against the `main` branch. - [x] I have run `./gradlew check`. - [x] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] SOLR-17567: Revert unnecessary signature changes in CatStream [solr]
malliaridis opened a new pull request, #2878: URL: https://github.com/apache/solr/pull/2878 https://issues.apache.org/jira/browse/SOLR-17567 # Description See https://github.com/apache/solr/pull/2872#issuecomment-2486167699 for context. # Solution Reverts the changes introduced to prevent potential cascading issues of the overridden method / class. # Tests No changes were altered. # Checklist Please review the following and check all that apply: - [X] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [X] I have created a Jira issue and added the issue ID to my pull request title. - [X] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation) - [X] I have developed this patch against the `main` branch. - [X] I have run `./gradlew check`. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17306: fix replication problem on follower restart [solr]
ds-manzinger commented on PR #2873: URL: https://github.com/apache/solr/pull/2873#issuecomment-2486091063 @epugh First, we tried using BATS for testing (as you already mentioned in the JIRA Ticket), but it was easier to create the unit test attached to the Pull Request. We built this because our setup is Leader → Replication Leader → Followers. When we update our catalog, we stop the Leader replication. If the Replication Leader restarts during this time (e.g., due to automated patching), it loses all the data. Followers are an autoscaling group that scales depending on the traffic. We have been running this patch in our production environment for two months without any issues on Solr 9.6. Should we also submit a Pull Request against the *main* branch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17556: Examples should run with the standard & recommended Solr home and solr data dir [solr]
gerlowskija commented on code in PR #2861: URL: https://github.com/apache/solr/pull/2861#discussion_r1848626386 ## solr/CHANGES.txt: ## @@ -237,6 +237,9 @@ led to the suppression of exceptions. (Andrey Bozhko) * SOLR-17545: Upgrade to Gradle 8.10 (Houston Putman) +* SOLR-17556: Examples should run with the standard & recommended Solr home and solr data dir (except for Review Comment: [0] Minor nit - the "should" here might confuse users. Is this how Solr behaves in 9.8? Is it how it should behave? etc. Wording like `"home" and "data" directories used by Solr examples have been updated to align with documented best practices.` might be a little clearer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] [WIP] Jetty12 + EE8 [solr]
risdenk commented on PR #2876: URL: https://github.com/apache/solr/pull/2876#issuecomment-2486366448 This should link to https://issues.apache.org/jira/browse/SOLR-17069 and potentially https://github.com/apache/solr/pull/1509 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] Avoid NPE if snapshotMgr failed during init [solr]
AndreyBozhko opened a new pull request, #2879: URL: https://github.com/apache/solr/pull/2879 https://issues.apache.org/jira/browse/SOLR-X # Description Avoid NPE when SolrCore is closing. If the snapshotMgr failed to initialize (e.g., because it didn't have write permission to create the snapshot metadata folder), SolrCore begins its `close` routine and may encounter a NPE because snapshotMgr is null. ``` 11:07:29.001 [coreLoadExecutor-12-thread-1] ERROR org.apache.solr.core.SolrCore - Error while closing java.lang.NullPointerException: null at org.apache.solr.core.SolrCore.doClose(SolrCore.java:1879) ~[solr-core-9.4.0.jar:9.4.0 71e101bb37497f730078d9afe1991b60d10bfe96 - stillalex - 2023-10-10 19:10:39] at org.apache.solr.core.SolrCore.close(SolrCore.java:1755) ~[solr-core-9.4.0.jar:9.4.0 71e101bb37497f730078d9afe1991b60d10bfe96 - stillalex - 2023-10-10 19:10:39] at org.apache.solr.core.SolrCore.(SolrCore.java:1223) ~[solr-core-9.4.0.jar:9.4.0 71e101bb37497f730078d9afe1991b60d10bfe96 - stillalex - 2023-10-10 19:10:39] at org.apache.solr.core.SolrCore.(SolrCore.java:1056) ~[solr-core-9.4.0.jar:9.4.0 71e101bb37497f730078d9afe1991b60d10bfe96 - stillalex - 2023-10-10 19:10:39] at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1705) ~[solr-core-9.4.0.jar:9.4.0 71e101bb37497f730078d9afe1991b60d10bfe96 - stillalex - 2023-10-10 19:10:39] at org.apache.solr.core.CoreContainer.lambda$loadInternal$12(CoreContainer.java:1043) ~[solr-core-9.4.0.jar:9.4.0 71e101bb37497f730078d9afe1991b60d10bfe96 - stillalex - 2023-10-10 19:10:39] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:234) ~[metrics-core-4.2.20.jar:4.2.20] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:294) ~[solr-solrj-9.4.0.jar:9.4.0 71e101bb37497f730078d9afe1991b60d10bfe96 - stillalex - 2023-10-10 19:10:39] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] ``` # Solution Please provide a short description of the approach taken to implement your solution. # Tests Please describe the tests you've developed or run to confirm this patch implements the feature or solves the problem. # Checklist Please review the following and check all that apply: - [x] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [ ] I have created a Jira issue and added the issue ID to my pull request title. - [x] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation) - [x] I have developed this patch against the `main` branch. - [x] I have run `./gradlew check`. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17570) cursorMark should assume minExactCount=0
David Smiley created SOLR-17570: --- Summary: cursorMark should assume minExactCount=0 Key: SOLR-17570 URL: https://issues.apache.org/jira/browse/SOLR-17570 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Reporter: David Smiley Claim: Someone using cursorMark (deep paging) probably doesn't care about numFound at all. And if they want to know, they either find out on the first page straight away (nextCursorMark is absent) or failing that, it could send a query purely to ascertain what numFound is if it doesn't want to page to the end. I see no test with both functionalities together. I think it should work this way by default. I tried this with a quick hack using DistribCursorPagingTest and it triggered an assertion in {{org.apache.solr.search.SolrIndexSearcher#sortDocSet}} that didn't need to be there -- it can be removed. The functionality worked. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17567: Revert unnecessary signature changes in CatStream [solr]
malliaridis commented on PR #2878: URL: https://github.com/apache/solr/pull/2878#issuecomment-2486720176 If you want you can add the changes to your PR, so that we can close this one and reduce the merges into main. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17567: Revert unnecessary signature changes in CatStream [solr]
epugh commented on PR #2878: URL: https://github.com/apache/solr/pull/2878#issuecomment-2486705064 Do you want me to copy these changes into the Stream tool branch I have, or just keep them parralel? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17567: Fix auth issue in stream tests [solr]
malliaridis commented on PR #2877: URL: https://github.com/apache/solr/pull/2877#issuecomment-2486723318 @epugh this one can be merged into https://github.com/apache/solr/pull/2872 as well. I think it's better to include them all together in one merge. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16503: Replace default USH apache client with Http2SolrClient [solr]
dsmiley commented on code in PR #2741: URL: https://github.com/apache/solr/pull/2741#discussion_r1849091157 ## solr/core/src/java/org/apache/solr/filestore/DistribFileStore.java: ## @@ -180,23 +184,35 @@ private boolean fetchFileFromNodeAndPersist(String fromNode) { ByteBuffer metadata = null; Map m = null; + + InputStream is = null; + var solrClient = coreContainer.getDefaultHttpSolrClient(); + try { +GenericSolrRequest request = new GenericSolrRequest(GET, "/node/files" + getMetaPath()); +request.setResponseParser(new InputStreamResponseParser(null)); +var response = solrClient.requestWithBaseUrl(baseUrl, request::process).getResponse(); +is = (InputStream) response.get("stream"); metadata = -Utils.executeGET( -coreContainer.getUpdateShardHandler().getDefaultHttpClient(), -baseUrl + "/node/files" + getMetaPath(), -Utils.newBytesConsumer((int) MAX_PKG_SIZE)); +Utils.newBytesConsumer((int) MAX_PKG_SIZE).accept((InputStream) response.get("stream")); m = (Map) Utils.fromJSON(metadata.array(), metadata.arrayOffset(), metadata.limit()); Review Comment: I now understand that we must do this because we need the metadata in ByteBuffer form to persist it later, therefore we need the InputStream. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16427: Enable error-prone PatternMatchingInstanceof rule [solr]
malliaridis commented on PR #2867: URL: https://github.com/apache/solr/pull/2867#issuecomment-2486696648 Waiting for https://github.com/apache/solr/pull/2877 to rerun the checks and merge afterwards. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17525: Text Embedder Query Parser [solr]
alessandrobenedetti commented on PR #2809: URL: https://github.com/apache/solr/pull/2809#issuecomment-2486562994 hi @malliaridis, my pleasure to know you! Thanks for the detailed explanation! I just added you to the branch, you should be able to write! If you are willing to help with that, it would be much appreciated! thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17570) cursorMark should assume minExactCount=0
[ https://issues.apache.org/jira/browse/SOLR-17570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley updated SOLR-17570: Description: Claim: Someone using cursorMark (deep paging) probably doesn't care about numFound at all, and thus could set minExactCount=0 param. And if they want to know, they either find out on the first page straight away (nextCursorMark is absent) or failing that, it could send a query purely to ascertain what numFound is if it doesn't want to page to the end. I see no test with both functionalities together. I think it should work this way by default. I tried this with a quick hack using DistribCursorPagingTest and it triggered an assertion in {{org.apache.solr.search.SolrIndexSearcher#sortDocSet}} that didn't need to be there -- it can be removed. The functionality worked. was: Claim: Someone using cursorMark (deep paging) probably doesn't care about numFound at all. And if they want to know, they either find out on the first page straight away (nextCursorMark is absent) or failing that, it could send a query purely to ascertain what numFound is if it doesn't want to page to the end. I see no test with both functionalities together. I think it should work this way by default. I tried this with a quick hack using DistribCursorPagingTest and it triggered an assertion in {{org.apache.solr.search.SolrIndexSearcher#sortDocSet}} that didn't need to be there -- it can be removed. The functionality worked. > cursorMark should assume minExactCount=0 > > > Key: SOLR-17570 > URL: https://issues.apache.org/jira/browse/SOLR-17570 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: David Smiley >Priority: Major > > Claim: Someone using cursorMark (deep paging) probably doesn't care about > numFound at all, and thus could set minExactCount=0 param. And if they want > to know, they either find out on the first page straight away (nextCursorMark > is absent) or failing that, it could send a query purely to ascertain what > numFound is if it doesn't want to page to the end. > I see no test with both functionalities together. I think it should work > this way by default. I tried this with a quick hack using > DistribCursorPagingTest and it triggered an assertion in > {{org.apache.solr.search.SolrIndexSearcher#sortDocSet}} that didn't need to > be there -- it can be removed. The functionality worked. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17525: Text Embedder Query Parser [solr]
malliaridis commented on PR #2809: URL: https://github.com/apache/solr/pull/2809#issuecomment-2486349589 It's nice meeting new people working on this project :D I have no push permission to push the fixes I've created. There are two issues in the referencing, one the libs.langchain4j and the other one was `libs.apache.solr.testframework`. The first one is actually a tricky one and I will have to provide a note in the documentation in case this happens again. When you have in `libs.versions.toml` `langchain4j` and `langchain4j-cohere` (for example), then you have to reference the first dependency as `libs.langchain4j.asProvider()`, otherwise it is just a group of dependencies, not the dependency itself. An alternative solution in this case is to simply add another keyword and extend `langchain4j` in `libs.version.toml`, like I did. Usually for "core" libraries that have simple names like in this case, you can use `core` or the module name itself (`langchain4j-langchain4j`). The second dependency `libs.apache.solr.testframework` could not be resolved, either because you mean `libs.apache.lucene.testframework` or `project(':solr:test-framework')`. I assumed the second one, but if wrong, please update it. If you update the permissions of this PR, I can push these changes together with a cleanup of the newly added license files (there is another issue there) and and update locks / checksums. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16503: Replace default USH apache client with Http2SolrClient [solr]
iamsanjay commented on PR #2741: URL: https://github.com/apache/solr/pull/2741#issuecomment-2486360871 I plan to merge this PR by the end of the week if there are no further comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17567: Improve Stream CLI implementation [solr]
epugh commented on PR #2872: URL: https://github.com/apache/solr/pull/2872#issuecomment-2485509588 @malliaridis I've marked off the items I took care of. The remaining ones I need more detail to more forward on. Regardless, it's nice to get a bit more polish! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17567: Improve Stream CLI implementation [solr]
epugh commented on PR #2872: URL: https://github.com/apache/solr/pull/2872#issuecomment-2486136854 > Wow, did not have SolrOnHdfsSnapshotsTool under my radar. Did you find it throught the forbidden-api rules? I did!!! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17504: CoreContainer calls UpdateHandler.commit. [solr]
bruno-roustant merged PR #2786: URL: https://github.com/apache/solr/pull/2786 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17306: fix replication problem on follower restart [solr]
epugh commented on PR #2873: URL: https://github.com/apache/solr/pull/2873#issuecomment-2486155043 > Should we also submit a Pull Request against the _main_ branch? Please do open aginst _main_... Generally we commit to _main_ and then backport to branch_9x. This should go out in 9.8. Or maybe 9.7.1 if we cut that soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17306: fix replication problem on follower restart [solr]
epugh commented on PR #2873: URL: https://github.com/apache/solr/pull/2873#issuecomment-2486163564 BTW, it would be interesting if you could write a short few paragraphs type story of how you do your lifecycle with leader/folllower/replicas, your ASG etc. In https://github.com/apache/solr/pull/2783/files#diff-b58818a370dac65f7abb0064599f8813a56841b4a40f960ea2b81e398b820f43 we are talking about the architecture you highlight. Would be great if you could review the pros/cons and weigh in. Let me know if you are interested and we can discuss more on that PR... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17567: Improve Stream CLI implementation [solr]
malliaridis commented on PR #2872: URL: https://github.com/apache/solr/pull/2872#issuecomment-2486167699 About the remaining TODOs: _Multiple type suppressions that can be avoided_ Suppressions like `@SuppressWarnings({"rawtypes"})` for classes like `List` can usually be fixed by simply using `List` or using the known generic type if any present. _StreamTool.LocalCatStream overrides CatStream btu does not properly set commaDelimitedFilepaths (this may be a bug)_ In `CatStream` `commaDelimitedFilepaths` is a class property that is also used in `CatStream#toExpression`, which is called by other classes. By passing passing the property as parameter in `validateAndSetFilepathsInSandbox` and overriding the method does not update the class property, resulting to potential inconsistent behavior / values in other classes. Instead, it is possible to simply remove the property from the parameters of `validateAndSetFilepathsInSandbox` (like it was before), and make the property `commaDelimitedFilepaths` visible (protected modifier). That way, overriding `validateAndSetFilepathsInSandbox` will be possible without changing the method signature. If the description is a bit irritating, I can create a separate PR for that. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-14966) Solr should use cache collection state to validate _stateVer_ sent by SolrJ
[ https://issues.apache.org/jira/browse/SOLR-14966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17899572#comment-17899572 ] David Smiley commented on SOLR-14966: - Noted, but this seems more of a minor optimization opportunity than a bug. > Solr should use cache collection state to validate _stateVer_ sent by SolrJ > --- > > Key: SOLR-14966 > URL: https://issues.apache.org/jira/browse/SOLR-14966 > Project: Solr > Issue Type: Bug >Reporter: Varun Thacker >Priority: Major > > Whenever Solr receives a request with the __stateVer__ parameter it's > supposed to validate the version the server has vs what the client has. The > server was meant to cache the state.json for 2 seconds but it's not doing so. > > If you follow this > [code|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/servlet/HttpSolrCall.java#L928] > you'll see _clusterState.getCollectionOrNull(coll)_ resulting it a ZK fetch > ( > [LazyCollectionRef|https://github.com/apache/lucene-solr/blob/master/solr/solrj/src/java/org/apache/solr/common/cloud/ZkStateReader.java#L656] > ) . If the code was doing _clusterState.getCollectionOrNull(coll, true)_ > then we would call ZK only after the 2s TTL has expired -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17525: Text Embedder Query Parser [solr]
malliaridis commented on PR #2809: URL: https://github.com/apache/solr/pull/2809#issuecomment-2486640193 @alessandrobenedetti If you need further assistance with the rest of the issues, just let me know. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17501) CLI: Reduce responsibilities of SolrCLI
[ https://issues.apache.org/jira/browse/SOLR-17501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christos Malliaridis updated SOLR-17501: Fix Version/s: main (10.0) Resolution: Resolved Status: Resolved (was: Patch Available) > CLI: Reduce responsibilities of SolrCLI > --- > > Key: SOLR-17501 > URL: https://issues.apache.org/jira/browse/SOLR-17501 > Project: Solr > Issue Type: Improvement > Components: cli >Reporter: Christos Malliaridis >Assignee: Christos Malliaridis >Priority: Minor > Labels: cli, pull-request-available > Fix For: main (10.0) > > Time Spent: 1h 40m > Remaining Estimate: 0h > > The current state of {{SolrCLI}} contains multiple static fields and methods > that are used across the CLI classes. However, the {{SolrCLI}} class is used > as the main entrypoint for our CLI scripts and should not provide and be > responsible for these utilities. > By following the seperation of concerns, two improvements can be made. These > are > - extract the static fields and helper methods to a utility class > - extract global options to a cli options class > In the context of this ticket only the first improvement should be addressed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16503: Replace default USH apache client with Http2SolrClient [solr]
dsmiley commented on code in PR #2741: URL: https://github.com/apache/solr/pull/2741#discussion_r1849072274 ## solr/core/src/java/org/apache/solr/filestore/DistribFileStore.java: ## @@ -484,14 +504,24 @@ public List list(String path, Predicate predicate) { public void delete(String path) { deleteLocal(path); List nodes = FileStoreUtils.fetchAndShuffleRemoteLiveNodes(coreContainer); -HttpClient client = coreContainer.getUpdateShardHandler().getDefaultHttpClient(); + +final var solrParams = new ModifiableSolrParams(); +solrParams.add("localDelete", "true"); +final var solrRequest = new GenericSolrRequest(DELETE, "/cluster/files" + path, solrParams); + for (String node : nodes) { String baseUrl = coreContainer.getZkController().getZkStateReader().getBaseUrlV2ForNodeName(node); - String url = baseUrl + "/cluster/files" + path + "?localDelete=true"; - HttpDelete del = new HttpDelete(url); - // invoke delete command on all nodes asynchronously - coreContainer.runAsync(() -> Utils.executeHttpMethod(client, url, null, del)); + try { +var solrClient = coreContainer.getDefaultHttpSolrClient(); +// invoke delete command on all nodes asynchronously +solrClient.requestWithBaseUrl(baseUrl, client -> client.requestAsync(solrRequest)); Review Comment: if submitted asynchronously, then we won't actually get the exception (if it happens remotely). requestAsync returns immediately. solrClient.requestWithBaseUrl takes a lambda and declares that it throws (hence forcing you to catch something) but it has no clue that, in this case, nothing will actually ever be thrown. Maybe simply add a code comment to say that we don't expect this will happen due to requestAsync. This is not the only instance of this scenario; @gerlowskija recently encountered it (with my review). ## solr/modules/cross-dc/src/java/org/apache/solr/crossdc/update/processor/MirroringUpdateProcessor.java: ## @@ -233,7 +223,9 @@ public void processDelete(final DeleteUpdateCommand cmd) throws IOException { boolean done = false; while (!done) { q.set(CursorMarkParams.CURSOR_MARK_PARAM, cursorMark); - QueryResponse rsp = client.query(collection, q); + var baseUrl = cmd.getReq().getCore().getCoreContainer().getZkController().getBaseUrl(); + var client = cmd.getReq().getCoreContainer().getDefaultHttpSolrClient(); + QueryResponse rsp = client.requestWithBaseUrl(baseUrl, c -> c.query(collection, q)); Review Comment: I believe getDefaulthttpSolrClient, will already have its URL as that of getZkController.getBaseUrl as it's the same node, the current node. Thus can issue a standard request without `requestWithBaseUrl`. ## solr/core/src/java/org/apache/solr/filestore/DistribFileStore.java: ## @@ -180,23 +184,35 @@ private boolean fetchFileFromNodeAndPersist(String fromNode) { ByteBuffer metadata = null; Map m = null; + + InputStream is = null; + var solrClient = coreContainer.getDefaultHttpSolrClient(); + try { +GenericSolrRequest request = new GenericSolrRequest(GET, "/node/files" + getMetaPath()); +request.setResponseParser(new InputStreamResponseParser(null)); +var response = solrClient.requestWithBaseUrl(baseUrl, request::process).getResponse(); +is = (InputStream) response.get("stream"); metadata = -Utils.executeGET( -coreContainer.getUpdateShardHandler().getDefaultHttpClient(), -baseUrl + "/node/files" + getMetaPath(), -Utils.newBytesConsumer((int) MAX_PKG_SIZE)); +Utils.newBytesConsumer((int) MAX_PKG_SIZE).accept((InputStream) response.get("stream")); m = (Map) Utils.fromJSON(metadata.array(), metadata.arrayOffset(), metadata.limit()); Review Comment: @iamsanjay ping. BTW, updating what I said, if we must have a Map and not a NamedList then NamedList.toMap(10) would be reasonable. ## solr/core/src/java/org/apache/solr/filestore/DistribFileStore.java: ## @@ -221,17 +242,18 @@ boolean fetchFromAnyNode() { try { String baseUrl = coreContainer.getZkController().getZkStateReader().getBaseUrlV2ForNodeName(liveNode); - String reqUrl = baseUrl + "/node/files" + path + "?meta=true&wt=javabin&omitHeader=true"; + final var solrParams = new ModifiableSolrParams(); + solrParams.add("meta", "true"); + solrParams.add("omitHeader", "true"); + + final var request = new GenericSolrRequest(GET, "/node/files" + path, solrParams); boolean nodeHasBlob = false; - Object nl = - Utils.executeGET( - coreContainer.getUpdateShardHandler().getDefaultHttpClient(), - reqUrl, - Utils.JAVAB
Re: [PR] SOLR-17501: Move out CLI utils from SolrCLI [solr]
malliaridis merged PR #2744: URL: https://github.com/apache/solr/pull/2744 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17501) CLI: Reduce responsibilities of SolrCLI
[ https://issues.apache.org/jira/browse/SOLR-17501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17899573#comment-17899573 ] ASF subversion and git services commented on SOLR-17501: Commit e1f56b30c33d9adfcb2c7aac1c01ef5393400c1e in solr's branch refs/heads/main from Christos Malliaridis [ https://gitbox.apache.org/repos/asf?p=solr.git;h=e1f56b30c33 ] SOLR-17501: Move out CLI utils from SolrCLI (#2744) > CLI: Reduce responsibilities of SolrCLI > --- > > Key: SOLR-17501 > URL: https://issues.apache.org/jira/browse/SOLR-17501 > Project: Solr > Issue Type: Improvement > Components: cli >Reporter: Christos Malliaridis >Assignee: Christos Malliaridis >Priority: Minor > Labels: cli, pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > The current state of {{SolrCLI}} contains multiple static fields and methods > that are used across the CLI classes. However, the {{SolrCLI}} class is used > as the main entrypoint for our CLI scripts and should not provide and be > responsible for these utilities. > By following the seperation of concerns, two improvements can be made. These > are > - extract the static fields and helper methods to a utility class > - extract global options to a cli options class > In the context of this ticket only the first improvement should be addressed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17567: Improve Stream CLI implementation [solr]
malliaridis commented on code in PR #2872: URL: https://github.com/apache/solr/pull/2872#discussion_r1848617744 ## gradle/validation/forbidden-apis/commons-cli.commons-cli.all.txt: ## @@ -0,0 +1,5 @@ +@defaultMessage Use a org.apache.commons.cli.Option instead of a String value +org.apache.commons.cli.CommandLine#hasOption(java.lang.String) +org.apache.commons.cli.CommandLine#getOptionValue(java.lang.String) +org.apache.commons.cli.CommandLine#getOptionValue(java.lang.String, java.lang.String) +org.apache.commons.cli.CommandLine#getParsedOptionValue(java.lang.String, java.lang.Object) Review Comment: I would extend the following as well (from `CommandLine`): ``` org.apache.commons.cli.CommandLine#hasOption(char) org.apache.commons.cli.CommandLine#getOptionValue(char) org.apache.commons.cli.CommandLine#getOptionValue(char, java.lang.String) org.apache.commons.cli.CommandLine#getOptionValue(char, Supplier) org.apache.commons.cli.CommandLine#getOptionValues(char) org.apache.commons.cli.CommandLine#getOptionValues(java.lang.String) org.apache.commons.cli.CommandLine#getParsedOptionValue(char) org.apache.commons.cli.CommandLine#getParsedOptionValue(char, Supplier) org.apache.commons.cli.CommandLine#getParsedOptionValue(char, java.lang.Object) org.apache.commons.cli.CommandLine#getParsedOptionValue(java.lang.String) org.apache.commons.cli.CommandLine#getParsedOptionValue(String, Supplier) ``` I haven't written any rules before, so I am not sure if the above have the correct format (especially `Supplier` is likely not correct). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Comment Edited] (SOLR-17219) Exceptions occur while Solr reads some core's configset (java.io.IOException: Error opening /configs/)
[ https://issues.apache.org/jira/browse/SOLR-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17859867#comment-17859867 ] Guillaume Jactat edited comment on SOLR-17219 at 11/19/24 10:04 PM: Also experiencing this issue with Solr 9.6.0. {*}Solr 9.6.1 seems to be fine though{*}. I didn't managed to produce the same error with this version. {*}Solr 9.7.0 seems OK too{*}. was (Author: gjactat): Also experiencing this issue with Solr 9.6.0. {*}Solr 9.6.1 seems to be fine though{*}. I didn't managed to produce the same error with this version. > Exceptions occur while Solr reads some core's configset (java.io.IOException: > Error opening /configs/) > > > Key: SOLR-17219 > URL: https://issues.apache.org/jira/browse/SOLR-17219 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 9.5 > Environment: My first attempts were on Windows via services hosted > through procrun (both zookeeper and solr nodes). I also tried with a Docker > Dekstop ensemble. > It seems that this error occurs less frequently via Docker. But it happens > anyway. >Reporter: Guillaume Jactat >Priority: Major > Attachments: stack.txt > > > Hello, > I'm currently testing SolrCloud to get a better idea of how to recover from > node failures. > I have a simple configuration: one ZooKeeper server and 3 *Solr 9.5* nodes. > I upload a configset in Zookeeper via Solr's Configsets API. I create 200 > collections, all bound to the same configset. > I leave the collections empty for the moment. > When I stop/start one node, the process of recovery happens. And almost > everytime, i get the following error (full stack is attached to this issue): > java.io.IOException: Error opening > /configs/CoreModel–CB38FE6CFE/lang/stopwords_fi.txt > Its not always the same configset's file. Sometimes, everything goes fine. > But when this error occurs, the whole process of recovery seem compromised, > leaving a lot of cores/collections "down". No "retry" happens, maybe because > Solr assumes that the configset is wrong and no retry could fix it ? > I've tried the same setup on Windows Service (procrun) and Docker Desktop > containers. It seems that this error occurs less frequently with docker but > it happens anyway. > I didn't find anything close to this error on the web... I have no clue why > this error happens. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Avoid NPE if snapshotMgr failed during init [solr]
dsmiley commented on PR #2879: URL: https://github.com/apache/solr/pull/2879#issuecomment-2487554064 Thanks. Ideally add a JIRA & CHANGES.txt entry but I'm personally fine with simply merging this trivial change as-is. So if nobody says anything or you don't bother, that's what I'll do by end of week. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Resolved] (SOLR-17535) Remove ClusterState.getCollectionStates() and getCollectionsMap()
[ https://issues.apache.org/jira/browse/SOLR-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley resolved SOLR-17535. - Fix Version/s: 9.8 Resolution: Fixed Deprecated with replacements in 9.8. Removals of this and other stuff can happen later in 10. > Remove ClusterState.getCollectionStates() and getCollectionsMap() > - > > Key: SOLR-17535 > URL: https://issues.apache.org/jira/browse/SOLR-17535 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Reporter: David Smiley >Assignee: David Smiley >Priority: Major > Labels: pull-request-available > Fix For: 9.8 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > To avoid creating intermediary maps for ClusterState consumers, and to limit > the direct usages of CollectionRef, I propose removing both > getCollectionStates and getCollectionsMap. This refactoring helps with any > future evolution of ClusterState. Instead, callers can use forEachCollection > or a new {{Stream}} one proposed here. > {code:java} > /** Streams the resolved DocCollections. Use this sparingly in case there > are many collections. */ > public Stream collectionStream() { > return > collectionStates.values().stream().map(CollectionRef::get).filter(Objects::nonNull); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17516: LBHttpSolrClient: support HttpJdkSolrClient (Generic Version) [solr]
jdyer1 merged PR #2828: URL: https://github.com/apache/solr/pull/2828 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17306: fix replication problem on follower restart [solr]
epugh commented on PR #2873: URL: https://github.com/apache/solr/pull/2873#issuecomment-2485945577 Could https://github.com/apache/solr/pull/1875 be augmented to test your scenario? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17516) LBHttpSolrClient: support HttpJdkSolrClient
[ https://issues.apache.org/jira/browse/SOLR-17516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17899512#comment-17899512 ] ASF subversion and git services commented on SOLR-17516: Commit 72cc2756a7befcdb1e9fff5275261695054bd152 in solr's branch refs/heads/main from James Dyer [ https://gitbox.apache.org/repos/asf?p=solr.git;h=72cc2756a7b ] SOLR-17516: LBHttpSolrClient: support HttpJdkSolrClient (Generic Version) (#2828) > LBHttpSolrClient: support HttpJdkSolrClient > --- > > Key: SOLR-17516 > URL: https://issues.apache.org/jira/browse/SOLR-17516 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Labels: pull-request-available > Fix For: main (10.0) > > Time Spent: 4.5h > Remaining Estimate: 0h > > With SOLR-599 we added a new SolrJ client *HttpJdkSolrClient* which uses > java.net.http.HttpClient internally. We can also support load balancing. > This ticket is to factor out common functionality from the existing > *LBHttp2SolrClient*, creating a new sibling class *LBHttpJdkSolrClient*. > This is a prerequisite for having a version of *CloudSolrClient* that works > with *HttpJdkSolrClient*. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17306: fix replication problem on follower restart [solr]
epugh commented on PR #2873: URL: https://github.com/apache/solr/pull/2873#issuecomment-2485943863 I am hoping someone with more confidence in this area weighs in, but if you don't get a review, please do poke me!!! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17504) CoreContainer calls UpdateHandler.commit
[ https://issues.apache.org/jira/browse/SOLR-17504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17899530#comment-17899530 ] ASF subversion and git services commented on SOLR-17504: Commit 5cc733b9fc60d4887df576aef340aa027ed2fc3e in solr's branch refs/heads/main from Bruno Roustant [ https://gitbox.apache.org/repos/asf?p=solr.git;h=5cc733b9fc6 ] SOLR-17504: CoreContainer calls UpdateHandler.commit. (#2786) > CoreContainer calls UpdateHandler.commit > > > Key: SOLR-17504 > URL: https://issues.apache.org/jira/browse/SOLR-17504 > Project: Solr > Issue Type: Improvement >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > ... and DirectUpdateHandler2.closeWriter supports commit metadata. > This proposal is about unifying calls to UpdateHandler.commit. > CoreContainer.reload may call directly IndexWriter.commit if the core to > reload is readonly (property of the Collection). This is an issue because it > bypasses some logic in UpdateHandler.commit. In addition, the current code > commits without taking the commit lock. > DirectUpdateHandler2.closeWriter may commit if the update log is not empty > (not committed yet). But it does not call DirectUpdateHandler2.commit(), it > copies commit() code instead for some reasons (comments in the code mention > test failures). The proposal is to add the same call to > DirectUpdateHandler2.shouldCommit() as in the commit() method, otherwise > commit metadata are lost. In addition shouldCommit() can be extended > correctly, for example in the solr-sandbox encryption module which needs to > intercept calls to shouldCommit(). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] [WIP] Jetty12 + EE8 [solr]
risdenk commented on code in PR #2876: URL: https://github.com/apache/solr/pull/2876#discussion_r1848801684 ## gradle/libs.versions.toml: ## @@ -295,7 +296,7 @@ bouncycastle-bcprov = { module = "org.bouncycastle:bcprov-jdk18on", version.ref carrot2-core = { module = "org.carrot2:carrot2-core", version.ref = "carrot2-core" } carrotsearch-hppc = { module = "com.carrotsearch:hppc", version.ref = "carrotsearch-hppc" } carrotsearch-randomizedtesting-runner = { module = "com.carrotsearch.randomizedtesting:randomizedtesting-runner", version.ref = "carrotsearch-randomizedtesting" } -# @keep transitive dependency for version alignment Review Comment: not sure why this changed ## gradle/libs.versions.toml: ## @@ -305,7 +306,7 @@ commonsio-commonsio = { module = "commons-io:commons-io", version.ref = "commons cybozulabs-langdetect = { module = "com.cybozu.labs:langdetect", version.ref = "cybozulabs-langdetect" } dropwizard-metrics-core = { module = "io.dropwizard.metrics:metrics-core", version.ref = "dropwizard-metrics" } dropwizard-metrics-graphite = { module = "io.dropwizard.metrics:metrics-graphite", version.ref = "dropwizard-metrics" } -dropwizard-metrics-jetty10 = { module = "io.dropwizard.metrics:metrics-jetty10", version.ref = "dropwizard-metrics" } +dropwizard-metrics-jetty10 = { module = "io.dropwizard.metrics:metrics-jetty12-ee10", version.ref = "dropwizard-metrics" } Review Comment: also the key should change from `dropwizard-metrics-jetty10` ## solr/test-framework/src/java/org/apache/solr/embedded/JettySolrRunner.java: ## @@ -414,16 +419,16 @@ public void contextInitialized(ServletContextEvent event) { // Default servlet as a fall-through root.addServlet(Servlet404.class, "/"); - chain = root; + chain.setHandler(root); } chain = injectJettyHandlers(chain); if (config.enableV2) { RewriteHandler rwh = new RewriteHandler(); rwh.setHandler(chain); - rwh.setRewriteRequestURI(true); - rwh.setRewritePathInfo(false); + // rwh.setRewriteRequestURI(true); Review Comment: commented out? ## solr/core/src/test/org/apache/solr/cloud/OverseerStatusTest.java: ## @@ -24,6 +24,7 @@ import org.junit.BeforeClass; import org.junit.Test; +// @LogLevel("org.eclipse.jetty=DEBUG") Review Comment: left over from debugging? ## gradle/libs.versions.toml: ## @@ -305,7 +306,7 @@ commonsio-commonsio = { module = "commons-io:commons-io", version.ref = "commons cybozulabs-langdetect = { module = "com.cybozu.labs:langdetect", version.ref = "cybozulabs-langdetect" } dropwizard-metrics-core = { module = "io.dropwizard.metrics:metrics-core", version.ref = "dropwizard-metrics" } dropwizard-metrics-graphite = { module = "io.dropwizard.metrics:metrics-graphite", version.ref = "dropwizard-metrics" } -dropwizard-metrics-jetty10 = { module = "io.dropwizard.metrics:metrics-jetty10", version.ref = "dropwizard-metrics" } +dropwizard-metrics-jetty10 = { module = "io.dropwizard.metrics:metrics-jetty12-ee10", version.ref = "dropwizard-metrics" } Review Comment: does it matter that this is ee10 vs ee8 like the description? ## solr/licenses/jetty-ee10-servlet-12.0.10.jar.sha1: ## @@ -0,0 +1 @@ +a8d087bde78f231084901553bb6829663b4a22f1 Review Comment: this is ee10 servlet? ## solr/core/src/java/org/apache/solr/handler/admin/ShowFileRequestHandler.java: ## @@ -100,9 +100,14 @@ public class ShowFileRequestHandler extends RequestHandlerBase implements Permis private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); static { -KNOWN_MIME_TYPES = new HashSet<>(MimeTypes.getKnownMimeTypes()); +KNOWN_MIME_TYPES = new HashSet<>(); +for (MimeTypes.Type type : MimeTypes.Type.values()) { + KNOWN_MIME_TYPES.add(type.toString()); +} KNOWN_MIME_TYPES.add("text/xml"); KNOWN_MIME_TYPES.add("text/javascript"); +KNOWN_MIME_TYPES.add("text/csv"); Review Comment: Why did csv/xhtml get added? ## solr/test-framework/src/java/org/apache/solr/embedded/JettySolrRunner.java: ## @@ -350,7 +351,7 @@ private void init(int port) { connector.setIdleTimeout(THREAD_POOL_MAX_IDLE_TIME_MS); server.setConnectors(new Connector[] {connector}); - server.setSessionIdManager(new DefaultSessionIdManager(server, new Random())); + // server.setSessionIdManager(new DefaultSessionIdManager(server, new Random())); Review Comment: commented out? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] SOLR-17567: Improve Stream CLI implementation [solr]
epugh commented on PR #2872: URL: https://github.com/apache/solr/pull/2872#issuecomment-2485913079 I've also polished up the docs a bit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17306) Solr Repeater or Slave loses data after restart when replication is not enabled on leader
[ https://issues.apache.org/jira/browse/SOLR-17306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SOLR-17306: -- Labels: pull-request-available (was: ) > Solr Repeater or Slave loses data after restart when replication is not > enabled on leader > - > > Key: SOLR-17306 > URL: https://issues.apache.org/jira/browse/SOLR-17306 > Project: Solr > Issue Type: Bug >Affects Versions: 9.2, 9.3, 9.4, 9.5, 9.6 >Reporter: Peter Kroiss >Priority: Major > Labels: pull-request-available > Attachments: solr-replication-test.txt > > Time Spent: 10m > Remaining Estimate: 0h > > We are testing Solr 9.6.2 in a leader - repeater - follower configuration. We > have times where we write the leader heavily, in that time replication is > disabled to save bandwidth. > In the time, when replication is disabled on leader, the repeater restarts > for some reason, the repeater loses all documents and doesn't recover when > the leader is opened for replication. > The documents are deleted but indexVersion and generation properties are set > to the value of the leader, so the repeater or follower doesn't recover when > the leader is opened for replication again. > It recovers only when there are commits on the leader after opening the > replication. > Log: > 2024-05-22 06:18:42.186 INFO (qtp16373883-27-null-23) [c: s: r: x:mycore > t:null-23] o.a.s.c.S.Request webapp=/solr path=/replication > params=\{wt=json&command=details} status=0 QTime=10 > 2024-05-22 06:18:46.195 INFO (indexFetcher-43-thread-1) [c: s: r: x:mycore > t:] o.a.s.h.IndexFetcher Leader's generation: 0 > 2024-05-22 06:18:46.195 INFO (indexFetcher-43-thread-1) [c: s: r: x:mycore > t:] o.a.s.h.IndexFetcher Leader's version: 0 > 2024-05-22 06:18:46.195 INFO (indexFetcher-43-thread-1) [c: s: r: x:mycore > t:] o.a.s.h.IndexFetcher Follower's generation: 2913 > 2024-05-22 06:18:46.195 INFO (indexFetcher-43-thread-1) [c: s: r: x:mycore > t:] o.a.s.h.IndexFetcher Follower's version: 1716300697144 > 2024-05-22 06:18:46.195 INFO (indexFetcher-43-thread-1) [c: s: r: x:mycore > t:] o.a.s.h.IndexFetcher New index in Leader. Deleting mine... > > --> there is no new Index in Leader it is only closed for replication > > > We think the problem is in IndexFetcher > old: if (IndexDeletionPolicyWrapper.getCommitTimestamp(commit) != 0L) { > forceReplication - will probably fix the problem > new : if (forceReplication && > IndexDeletionPolicyWrapper.getCommitTimestamp(commit) != 0L) { > > > > > When investigation the problem we also found some inconsistencies in the > details request. There are two fragments leader. When the leader is closed > for replication the property leader. replicationEnabled is set to true, the > property follower. leaderDetails. Leader. replicationEnabled is correct. > > Example > curl -s > "https://solr9-repeater:8983/solr/mycore/replication?wt=json&command=details"; > | jq '.details | > { indexSize: .indexSize, indexVersion: .indexVersion, generation: > .generation, indexPath: .indexPath, leader: \\{ replicableVersion: > .leader.replicableVersion, replicableGeneration: > .leader.replicableGeneration, replicationEnabled: .leader.replicationEnabled } > , > follower: { leaderDetails: { indexSize: .follower.leaderDetails.indexSize, > generation: .follower.leaderDetails.generation, > indexVersion: .follower.leaderDetails.indexVersion, indexPath: > .follower.leaderDetails.indexPath, > leader: > { replicableVersion: .follower.leaderDetails.leader.replicableVersion , > replicableGeneration: .follower.leaderDetails.leader.replicableGeneration, > replicationEnabled: .follower.leaderDetails.leader.replicationEnabled } > }} > }' > > { > "indexSize": "10.34 GB", > "indexVersion": 1716358708159, > "generation": 2913, > "indexPath": "/var/solr/data/mycore/data/index.20240522061946262", > "leader": > { "replicableVersion": 1716358708159, "replicableGeneration": 2913, > "replicationEnabled": "true" } > , > "follower": { > "leaderDetails": { > "indexSize": "10.34 GB", > "generation": 2913, > "indexVersion": 1716358708159, > "indexPath": "/var/solr/data/mycore/data/restore.20240508131046932", > "leader": > { "replicableVersion": 1716358708159, "replicableGeneration": > 2913, "replicationEnabled": "false" } > } > } > } -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-16781) Remove directives from Solr
[ https://issues.apache.org/jira/browse/SOLR-16781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SOLR-16781: -- Labels: pull-request-available (was: ) > Remove directives from Solr > - > > Key: SOLR-16781 > URL: https://issues.apache.org/jira/browse/SOLR-16781 > Project: Solr > Issue Type: Improvement >Reporter: Ishan Chattopadhyaya >Priority: Blocker > Labels: pull-request-available > Fix For: main (10.0) > > Attachments: SOLR-16781-1.patch, SOLR-16781-2.patch, SOLR-16781.patch > > Time Spent: 10m > Remaining Estimate: 0h > > directives in solrconfig.xml used to be recommended way for including > additional jar files to the classpath for a particular collection or > collections. > For context: This feature required complex handling of "trusted" vs > "non-trusted" configsets in configset upload API to keep Solr secure (i.e. to > stop RCE attacks for non-authentication enabled deployments). This security > feature also broke down recently due to a bug in Schema designer (SOLR-16777). > Supported alternatives exist that are safer: > * user can add the jar files to Solr's classpath > * use packages to use custom jars per collection > In the light of these, there's no need to continue to support the > directive going forward. > I propose to remove the directives handling and functionality through > this issue. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17525: Text Embedder Query Parser [solr]
alessandrobenedetti commented on code in PR #2809: URL: https://github.com/apache/solr/pull/2809#discussion_r1847847749 ## solr/modules/llm/src/java/org/apache/solr/llm/embedding/SolrEmbeddingModel.java: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.llm.embedding; + +import dev.langchain4j.data.embedding.Embedding; +import dev.langchain4j.model.embedding.EmbeddingModel; +import java.lang.reflect.Method; +import java.time.Duration; +import java.util.ArrayList; +import java.util.Map; +import java.util.Objects; +import org.apache.lucene.util.Accountable; +import org.apache.lucene.util.RamUsageEstimator; +import org.apache.solr.llm.store.EmbeddingModelException; + +public class SolrEmbeddingModel implements Accountable { + private static final long BASE_RAM_BYTES = + RamUsageEstimator.shallowSizeOfInstance(SolrEmbeddingModel.class); + private static final String TIMEOUT_PARAM = "timeout"; + private static final String MAX_SEGMENTS_PER_BATCH_PARAM = "maxSegmentsPerBatch"; + private static final String MAX_RETRIES_PARAM = "maxRetries"; + + private final String name; + private final Map params; + private final EmbeddingModel embedder; + private final Integer hashCode; + + public static SolrEmbeddingModel getInstance( + String className, String name, Map params) throws EmbeddingModelException { +try { + EmbeddingModel embedder; + Class modelClass = Class.forName(className); Review Comment: thanks for the explanation, done in the latest commit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Assigned] (SOLR-17516) LBHttpSolrClient: support HttpJdkSolrClient
[ https://issues.apache.org/jira/browse/SOLR-17516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Dyer reassigned SOLR-17516: - Fix Version/s: main (10.0) Assignee: James Dyer > LBHttpSolrClient: support HttpJdkSolrClient > --- > > Key: SOLR-17516 > URL: https://issues.apache.org/jira/browse/SOLR-17516 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Labels: pull-request-available > Fix For: main (10.0) > > Time Spent: 4h 20m > Remaining Estimate: 0h > > With SOLR-599 we added a new SolrJ client *HttpJdkSolrClient* which uses > java.net.http.HttpClient internally. We can also support load balancing. > This ticket is to factor out common functionality from the existing > *LBHttp2SolrClient*, creating a new sibling class *LBHttpJdkSolrClient*. > This is a prerequisite for having a version of *CloudSolrClient* that works > with *HttpJdkSolrClient*. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] SOLR-17567: Fix auth issue in stream tests [solr]
malliaridis opened a new pull request, #2877: URL: https://github.com/apache/solr/pull/2877 https://issues.apache.org/jira/browse/SOLR-17567 # Description In https://github.com/apache/solr/pull/2479 an issue was introduced together with the stream tests that cause the bats tests to fail. # Solution This PR applies a fix that is used in `test_basic_auth.bats` for enabling authentication. Bats tests are not starting with a clean state sometimes, and this is one situation that is affected by that. # Tests A test for missing credentials was added. CLI output could be improved but is not task of this fix / PR. # Checklist Please review the following and check all that apply: - [X] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [X] I have created a Jira issue and added the issue ID to my pull request title. - [X] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation) - [X] I have developed this patch against the `main` branch. - [X] I have run `./gradlew check`. - [X] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org