[jira] [Updated] (SOLR-17663) TimeOut overflows if the provided interval is large
[ https://issues.apache.org/jira/browse/SOLR-17663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Roustant updated SOLR-17663: -- Description: TimeOut constructor is passed a long interval parameter. If this long is large enough, the timeoutAt field overflows, making all the methods incorrect, including hasTimedOut() which returns true immediately. TimeUnit.convert() deals with the same problem by checking the value to convert ahead of time and by setting fixed Long.MAX_VALUE to prevent overflow. We should do the same for timeoutAt in TimeOut constructor. was: TimeOut constructor is passed a long interval parameter. If this long is large enough, the timeoutAt field overflows, making all the methods incorrect, including hasTimedOut() which returns true immediately. TimeUnit.convert() deals with the same problem by checking the value to convert ahead of time and by setting fixed Long.MAX_VALUE to prevent overflow. We should do the same for timeoutAt in TimeOut constructor. Maybe we could also provide a constant NO_TIME_OUT for callers that know there is no timeout? For example when the "timeAllowed" request param is not present. In this case, the caller could use the constant instead of creating a TimeOut with Long.MAX_VALUE as interval. > TimeOut overflows if the provided interval is large > --- > > Key: SOLR-17663 > URL: https://issues.apache.org/jira/browse/SOLR-17663 > Project: Solr > Issue Type: Bug >Affects Versions: 9.8 >Reporter: Bruno Roustant >Priority: Minor > > TimeOut constructor is passed a long interval parameter. If this long is > large enough, the timeoutAt field overflows, making all the methods > incorrect, including hasTimedOut() which returns true immediately. > TimeUnit.convert() deals with the same problem by checking the value to > convert ahead of time and by setting fixed Long.MAX_VALUE to prevent > overflow. We should do the same for timeoutAt in TimeOut constructor. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [I] EncryptionRequestHandler support for collection with multiple shards [solr-sandbox]
bruno-roustant closed issue #110: EncryptionRequestHandler support for collection with multiple shards URL: https://github.com/apache/solr-sandbox/issues/110 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] EncryptionRequestHandler supports encryption requests distribution. [solr-sandbox]
bruno-roustant merged PR #115: URL: https://github.com/apache/solr-sandbox/pull/115 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17663) TimeOut overflows if the provided interval is large
Bruno Roustant created SOLR-17663: - Summary: TimeOut overflows if the provided interval is large Key: SOLR-17663 URL: https://issues.apache.org/jira/browse/SOLR-17663 Project: Solr Issue Type: Bug Affects Versions: 9.8 Reporter: Bruno Roustant TimeOut constructor is passed a long interval parameter. If this long is large enough, the timeoutAt field overflows, making all the methods incorrect, including hasTimedOut() which returns true immediately. TimeUnit.convert() deals with the same problem by checking the value to convert ahead of time and by setting fixed Long.MAX_VALUE to prevent overflow. We should do the same for timeoutAt in TimeOut constructor. Maybe we could also provide a constant NO_TIME_OUT for callers that know there is no timeout? For example when the "timeAllowed" request param is not present. In this case, the caller could use the constant instead of creating a TimeOut with Long.MAX_VALUE as interval. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17663) TimeOut overflows if the provided interval is large
[ https://issues.apache.org/jira/browse/SOLR-17663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SOLR-17663: -- Labels: pull-request-available (was: ) > TimeOut overflows if the provided interval is large > --- > > Key: SOLR-17663 > URL: https://issues.apache.org/jira/browse/SOLR-17663 > Project: Solr > Issue Type: Bug >Affects Versions: 9.8 >Reporter: Bruno Roustant >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > TimeOut constructor is passed a long interval parameter. If this long is > large enough, the timeoutAt field overflows, making all the methods > incorrect, including hasTimedOut() which returns true immediately. > TimeUnit.convert() deals with the same problem by checking the value to > convert ahead of time and by setting fixed Long.MAX_VALUE to prevent > overflow. We should do the same for timeoutAt in TimeOut constructor. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] SOLR-17663: Protect TimeOut from overflow. [solr]
bruno-roustant opened a new pull request, #3173: URL: https://github.com/apache/solr/pull/3173 https://issues.apache.org/jira/browse/SOLR-17663 TimeOut constructor is passed a long interval parameter. If this long is large enough, the timeoutAt field overflows, making all the methods incorrect, including hasTimedOut() which returns true immediately. TimeUnit.convert() deals with the same problem by checking the value to convert ahead of time and by setting fixed Long.MAX_VALUE to prevent overflow. We should do the same for timeoutAt in TimeOut constructor. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Transfer encryption metadata in the commit when DirectUpdateHandler2.closeWriter is called. [solr-sandbox]
bruno-roustant commented on PR #113: URL: https://github.com/apache/solr-sandbox/pull/113#issuecomment-2647900256 I recently updated the Solr core dependency to 9.8, so I can complete this PR to fix the issue. The real fix is inside Solr core 9.8. Here I mainly add a test to verify that when nodes restart without commit, the encryption metadata in the commit metadata will still be transferred at the next commit after the restart. I could also fix EncryptionDirectory to support IndexFetcher copying the Lucene "segments" file (happens when the nodes restart). This file is not encrypted of course. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16503) Switch UpdateShardHandler.getDefaultHttpClient to Jetty HTTP2
[ https://issues.apache.org/jira/browse/SOLR-16503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17925571#comment-17925571 ] ASF subversion and git services commented on SOLR-16503: Commit 82c50e4291e86e773301cc71278f99f33490ad48 in solr's branch refs/heads/branch_9x from Sanjay Dutt [ https://gitbox.apache.org/repos/asf?p=solr.git;h=82c50e4291e ] SOLR-16503: Use Http2SolrClient in SolrClientCache, SchemaDesigner (#2764) - Co-authored-by: David Smiley (cherry picked from commit 1bb9b84b6c43746713561d10612633b61555eef3) > Switch UpdateShardHandler.getDefaultHttpClient to Jetty HTTP2 > - > > Key: SOLR-16503 > URL: https://issues.apache.org/jira/browse/SOLR-16503 > Project: Solr > Issue Type: Sub-task >Reporter: David Smiley >Priority: Major > Labels: pull-request-available > Attachments: Screenshot 2024-03-16 at 9.14.36 PM.png > > Time Spent: 13h 20m > Remaining Estimate: 0h > > Much of Solr's remaining uses of Apache HttpClient (HTTP 1) is due to > {{org.apache.solr.update.UpdateShardHandler#getDefaultHttpClient}} which > underlies most Solr-to-Solr connectivity. This also underlies the > {{{}CoreContainer.getSolrClientCache{}}}. Lets switch to Jetty (HTTP 2). > > In SolrClientCache in particular: > Switch use of CloudLegacySolrClient.Builder to CloudSolrClient.Builder > Switch use of HttpSolrClient.Builder to Http2SolrClient.Builder > Undeprecate all the methods here. They should not have been deprecated in > the first place. > The constructor: switch from Apache HttpClient to a Jetty HttpClient. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17663: Protect TimeOut from overflow. [solr]
dsmiley commented on code in PR #3173: URL: https://github.com/apache/solr/pull/3173#discussion_r1949280001 ## solr/core/src/java/org/apache/solr/util/TimeOut.java: ## @@ -23,15 +23,31 @@ import java.util.function.Supplier; import org.apache.solr.common.util.TimeSource; +/** + * Timeout tool to ease time unit conversion, to check time left or time elapsed, and to easily wait Review Comment: Thanks for adding javadocs! IMO the very first thing and only thing to list is `check time left or time elapsed` as this is the fundamental limited purpose. You wouldn't use this class only for time unit conversion :-). Everything else is minor stuff that hopefully makes this utility easy to use/understand. ## solr/core/src/java/org/apache/solr/util/TimeOut.java: ## @@ -23,15 +23,31 @@ import java.util.function.Supplier; import org.apache.solr.common.util.TimeSource; +/** + * Timeout tool to ease time unit conversion, to check time left or time elapsed, and to easily wait + * on a {@link Supplier}. It is based on {@link TimeSource} to allow choosing the time source, and + * to facilitate time control in tests. + */ public class TimeOut { + // Internally, the time unit is nanosecond. private final long timeoutAt, startTime; private final TimeSource timeSource; + /** + * Internally the timeout is stored in nanoseconds, so it cannot track more than {@link Review Comment: The javadocs for anything should not begin with discussing internals. What you added is great but it's an FYI. And even the FYI should start with that great warning on what can happen (a limitation of sorts), and _then_ elaborate with the _why_ (you started with the _why_) or leave the "why" (which is distracting to the caller; they don't care) as code comment. As I look at this with fresh eyes, I would have loved to see an explanation of what "interval" is which I can eventually guess but the name choice wasn't obvious to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16503: Use Http2SolrClient in SolrClientCache, SchemaDesigner [solr]
iamsanjay merged PR #2764: URL: https://github.com/apache/solr/pull/2764 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16503) Switch UpdateShardHandler.getDefaultHttpClient to Jetty HTTP2
[ https://issues.apache.org/jira/browse/SOLR-16503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17925562#comment-17925562 ] ASF subversion and git services commented on SOLR-16503: Commit 1bb9b84b6c43746713561d10612633b61555eef3 in solr's branch refs/heads/main from Sanjay Dutt [ https://gitbox.apache.org/repos/asf?p=solr.git;h=1bb9b84b6c4 ] SOLR-16503: Use Http2SolrClient in SolrClientCache, SchemaDesigner (#2764) - Co-authored-by: David Smiley > Switch UpdateShardHandler.getDefaultHttpClient to Jetty HTTP2 > - > > Key: SOLR-16503 > URL: https://issues.apache.org/jira/browse/SOLR-16503 > Project: Solr > Issue Type: Sub-task >Reporter: David Smiley >Priority: Major > Labels: pull-request-available > Attachments: Screenshot 2024-03-16 at 9.14.36 PM.png > > Time Spent: 13h 20m > Remaining Estimate: 0h > > Much of Solr's remaining uses of Apache HttpClient (HTTP 1) is due to > {{org.apache.solr.update.UpdateShardHandler#getDefaultHttpClient}} which > underlies most Solr-to-Solr connectivity. This also underlies the > {{{}CoreContainer.getSolrClientCache{}}}. Lets switch to Jetty (HTTP 2). > > In SolrClientCache in particular: > Switch use of CloudLegacySolrClient.Builder to CloudSolrClient.Builder > Switch use of HttpSolrClient.Builder to Http2SolrClient.Builder > Undeprecate all the methods here. They should not have been deprecated in > the first place. > The constructor: switch from Apache HttpClient to a Jetty HttpClient. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17664) SolrCloud cluster creation version in cluster properties (ZK)
David Smiley created SOLR-17664: --- Summary: SolrCloud cluster creation version in cluster properties (ZK) Key: SOLR-17664 URL: https://issues.apache.org/jira/browse/SOLR-17664 Project: Solr Issue Type: Improvement Reporter: David Smiley If a SolrCloud cluster had metadata including a version number (say the SolrVersion of it's creation), then we could toggle new/different ways of doing things in SolrCloud that have to do with what's in ZooKeeper. This way, new code can know whether it needs to do something a new way or old way. Overseer vs distributed-mode is one, PRS is another. There have been other changes in the past, which would have benefited. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17659) Implement basic authentication in Admin UI
[ https://issues.apache.org/jira/browse/SOLR-17659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17925634#comment-17925634 ] Gus Heck commented on SOLR-17659: - {quote}not a service that is run on the server where Solr is running. {quote} This has never been a confusion on my part, however I guess it's good to have this clarified for others who read this later. {quote}I believe it would be way easier to analyze the endpoints from our source code, rather than reverse-engineer the WebAssembly app {quote} This is exactly why version information is important. One would have to analyze the correct version from VCS. Also I am presuming that if it is not now possible, it will soon be possible to feed a WebAssembly app to a Chat based AI like ChatGPT, or possibly a custom trained DeepSeek (Model License terms only stop the good guys) etc... and ask it to list the URL's the application is likely to access. (some trickiness possibly to word the prompt so that it knows what to do with variable URLs like /solr//select of course). WebAssembly (or obfuscated javascript) is hard for humans to read, but a machine doesn't care, and if one can get an LLM to do that, all questions of versioning go away. Also, consider the answer to this [SO post|https://stackoverflow.com/questions/61904135/decoding-wasm-webassembly-files]. {quote}That sounds very much like the concepts from OAuth. And from how I understand your message, what you probably are looking for is "migrating" Solr into a [resource server|https://www.oauth.com/oauth2-servers/the-resource-server/], and allow an external app like Keycloak manage users, access etc. {quote} Resource server is just a description of how you treat the server (aside from expecting a few HTTP responses and codes in failure cases, I suppose). But I DO hope for solr to be friendly to such usage, including not showing an additional login page once you pass the external verification etc. In the current design the UI will need complex "do we already have some other sort of auth?" logic to ensure that the form isn't shown. This will need to be updated for every new Auth mechanism (or new version of auth mechanism) that breaks the heuristic we were using to detect it. I'm pretty sure such checks can be more easily handled and maintained if they are not part of the UI by folks who don't have to understand how to build or update the UI. Also there's less chance that a UI focused contributor not fully appreciating security concerns accidentally breaks it. Basically I'd like to have zero authorization decision making in the UI, and if we have a login form, we definitely have to make those decisions about when to display the form. Also, for the folks who have made a carefully considered decision to rely on "network security" (because that's the only way they can make a profit or they are already inside an air gap facility containing legacy systems far more important than Solr) there's the question of how to disable the form If the login is not in the UI, and authentication is turned off, direct requests to the UI "just work" as expected. > Implement basic authentication in Admin UI > -- > > Key: SOLR-17659 > URL: https://issues.apache.org/jira/browse/SOLR-17659 > Project: Solr > Issue Type: New Feature > Components: Admin UI >Reporter: Christos Malliaridis >Priority: Major > Labels: new-ui, ui > > In the new UI one of the key features that is not implemented yet is user > authentication. In order to secure and securily access Solr, the user should > be able to authenticate against a Solr instance with basic credentials. > h2. Task > Implement basic user authentication (with credentials) according to the [new > designs|https://www.figma.com/design/VdbEfcWQ8mirFNquBzbPk2/Apache-Solr-Admin-UI-v2-Concept?node-id=1190-388&t=vMgOa9QlzQZSdjLf-1]. > h2. Acceptance Criteria > - The user can access a Solr instance that has user authentication enabled > - The user can at least authenticate with credentials (basic auth) > - The credentials form is displayed after the user has established a > connection with a Solr instance, that is, after a Solr instance was found > - The user can return to the start screen where the Solr URL was provided, > if he decides to abort the authentication step > - The user is no longer redirected to the dashboard or any other screen if > user authentication is required > - The credentials are used for any subsequent request > h2. Additional Information > The support for additional authentication options does not have to be > addressed in this issue. If it proves to be straight-forward, feel free to > implement additional auth options as well. Note that additional > authentication options will be added later, and th
[jira] [Comment Edited] (SOLR-17659) Implement basic authentication in Admin UI
[ https://issues.apache.org/jira/browse/SOLR-17659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17925634#comment-17925634 ] Gus Heck edited comment on SOLR-17659 at 2/10/25 4:52 PM: -- {quote}not a service that is run on the server where Solr is running. {quote} This has never been a confusion on my part, however I guess it's good to have this clarified for others who read this later. {quote}I believe it would be way easier to analyze the endpoints from our source code, rather than reverse-engineer the WebAssembly app {quote} This is exactly why version information is important. One would have to analyze the correct version from VCS. Also I am presuming that if it is not now possible, it will soon be possible to feed a WebAssembly app to a Chat based AI like ChatGPT, or possibly a custom trained DeepSeek (Model License terms only stop the good guys) etc... and ask it to list the URL's the application is likely to access. (some trickiness possibly to word the prompt so that it knows what to do with variable URLs like /solr//select of course). WebAssembly (or obfuscated javascript) is hard for humans to read, but a machine doesn't care, and if one can get an LLM to do that, all questions of versioning go away. Also, consider the answer to this [SO post|https://stackoverflow.com/questions/61904135/decoding-wasm-webassembly-files]. {quote}That sounds very much like the concepts from OAuth. And from how I understand your message, what you probably are looking for is "migrating" Solr into a [resource server|https://www.oauth.com/oauth2-servers/the-resource-server/], and allow an external app like Keycloak manage users, access etc. {quote} Resource server is just a description of how you treat the server (aside from expecting a few HTTP responses and codes in failure cases, I suppose). But I DO hope for solr to be friendly to such usage, including not showing an additional login page once you pass the external verification etc. In the current design the UI will need complex "do we already have some other sort of auth?" logic to ensure that the form isn't shown. This will need to be updated for every new Auth mechanism (or new version of auth mechanism) that breaks the heuristic we were using to detect it. I'm pretty sure such checks can be more easily handled and maintained if they are not part of the UI by folks who don't have to understand how to build or update the UI. Also there's less chance that a UI focused contributor not fully appreciating security concerns accidentally breaks it. Basically I'd like to have zero authentication/authorization decision making in the UI, and if we have a login form, we definitely have to make those decisions about when to display the form. Also, for the folks who have made a carefully considered decision to rely on "network security" (because that's the only way they can make a profit or they are already inside an air gap facility containing legacy systems far more important than Solr) there's the question of how to disable the form If the login is not in the UI, and authentication is turned off, direct requests to the UI "just work" as expected. was (Author: gus_heck): {quote}not a service that is run on the server where Solr is running. {quote} This has never been a confusion on my part, however I guess it's good to have this clarified for others who read this later. {quote}I believe it would be way easier to analyze the endpoints from our source code, rather than reverse-engineer the WebAssembly app {quote} This is exactly why version information is important. One would have to analyze the correct version from VCS. Also I am presuming that if it is not now possible, it will soon be possible to feed a WebAssembly app to a Chat based AI like ChatGPT, or possibly a custom trained DeepSeek (Model License terms only stop the good guys) etc... and ask it to list the URL's the application is likely to access. (some trickiness possibly to word the prompt so that it knows what to do with variable URLs like /solr//select of course). WebAssembly (or obfuscated javascript) is hard for humans to read, but a machine doesn't care, and if one can get an LLM to do that, all questions of versioning go away. Also, consider the answer to this [SO post|https://stackoverflow.com/questions/61904135/decoding-wasm-webassembly-files]. {quote}That sounds very much like the concepts from OAuth. And from how I understand your message, what you probably are looking for is "migrating" Solr into a [resource server|https://www.oauth.com/oauth2-servers/the-resource-server/], and allow an external app like Keycloak manage users, access etc. {quote} Resource server is just a description of how you treat the server (aside from expecting a few HTTP responses and codes in failure cases, I suppose). But I DO hope for solr to be friendly to such
Re: [PR] SOLR-17641: Disable the Security Manager for Java 24+ [solr]
dsmiley commented on code in PR #3153: URL: https://github.com/apache/solr/pull/3153#discussion_r1949516798 ## build.gradle: ## @@ -35,6 +35,8 @@ plugins { } // Declare default Java versions for the entire project and for SolrJ separately +rootProject.ext.currentJavaVersion = JavaVersion.current() +rootProject.ext.minJavaVersionSolrJ = JavaVersion.toVersion(libs.versions.java.solrj.get()) Review Comment: duplicates line 41 ## gradle/testing/randomization.gradle: ## @@ -204,6 +204,10 @@ allprojects { if (project.ext.has("useSecurityManager")) { useSecurityManager = project.ext.get("useSecurityManager") } +// Solr does not run with the Security Mangager for Java 24+ Review Comment: ```suggestion // There is no longer a Security Manager in Java 24 ``` ## solr/bin/solr: ## @@ -1155,6 +1155,11 @@ else REMOTE_JMX_OPTS=() fi +# Do not use the java security manager when running Java 24+ +if (( JAVA_VER_NUM >= 24 )) ; then Review Comment: could just combine this condition into line 1164. Ideally this changes the "true" in the default interpretation of that env var to a "false". But it's fine. ## solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-9.adoc: ## @@ -67,6 +67,10 @@ It is always strongly recommended that you fully reindex your documents after a In Solr 8, it was possible to add docValues to a schema without re-indexing via `UninvertDocValuesMergePolicy`, an advanced/expert utility. Due to changes in Lucene 9, that isn't possible any more. +== Solr 9.9 +=== Java Security Manager +Java removed support for the Security Manager starting with Java 24, therefore Solr will disable the feature when run with Java 24 or later. + Review Comment: minor: Okay but arguably you didn't have to bother to say this. There's no action for the user to take. I don't think it's up to Solr to tell people what's coming/going/happening in the Java ecosystem. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] chore(deps): update dependency org.jetbrains.kotlinx:kotlinx-datetime to v0.6.2 [solr]
solrbot opened a new pull request, #3174: URL: https://github.com/apache/solr/pull/3174 This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [org.jetbrains.kotlinx:kotlinx-datetime](https://redirect.github.com/Kotlin/kotlinx-datetime) | dependencies | patch | `0.6.1` -> `0.6.2` | --- ### Release Notes Kotlin/kotlinx-datetime (org.jetbrains.kotlinx:kotlinx-datetime) ### [`v0.6.2`](https://redirect.github.com/Kotlin/kotlinx-datetime/blob/HEAD/CHANGELOG.md#062) - Add the Wasm/WASI target support ([#366](https://redirect.github.com/Kotlin/kotlinx-datetime/pull/366)). Thanks, [@igoriakovlev](https://redirect.github.com/igoriakovlev)! - Add `TimeSource.asClock` ([#164](https://redirect.github.com/Kotlin/kotlinx-datetime/pull/164)). Thanks, [@hfhbd](https://redirect.github.com/hfhbd)! - Implement parsing and formatting day-of-year ([#414](https://redirect.github.com/Kotlin/kotlinx-datetime/issues/414)) - Improve the precision of `NSDate` to `Instant` conversions ([#427](https://redirect.github.com/Kotlin/kotlinx-datetime/issues/427)) - Deprecate `DatePeriod.plus` and `DateTimePeriod.plus` with a warning ([#381](https://redirect.github.com/Kotlin/kotlinx-datetime/issues/381)) - Deprecate `Clock.asTimeSource` with a warning ([#372](https://redirect.github.com/Kotlin/kotlinx-datetime/issues/372)) - Fix a bug in `module-info` preventing using `kotlinx.datetime.format` together with JPMS ([#438](https://redirect.github.com/Kotlin/kotlinx-datetime/pull/438)). Thanks, [@cdelabou](https://redirect.github.com/cdelabou)! - Small tweaks and fixes. --- ### Configuration 📅 **Schedule**: Branch creation - "* * * * *" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://redirect.github.com/solrbot/renovate-github-action) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SolrTestCaseJ4: don't reset HttpClient SSL stuff [solr]
HoustonPutman commented on PR #3037: URL: https://github.com/apache/solr/pull/3037#issuecomment-2648778043 @dsmiley this broke `ConcurrentUpdateSolrClientBuilderTest.testSocketTimeoutOnCommit()` in a fun way. This PR works for all tests that extend `SolrTestCaseJ4`, because all of those tests expect to have the SSLConfig configured by `SolrTestCaseJ4.setupTestCases()`: ```java sslConfig = buildSSLConfig(); // based on randomized SSL config, set SocketFactoryRegistryProvider appropriately HttpClientUtil.setSocketFactoryRegistryProvider( sslConfig.buildClientSocketFactoryRegistryProvider()); ``` Unfortunately, other tests, such as the one that is failing, extend `SolrTestCase`, and expect to have the default SSLConfig configured by default (a good assumption IMO). This test fails because the test ran before it extends `SolrTestCaseJ4` and the randomization selected HTTPS for that run. After the test succeeded, SolrTestCaseJ4 no longer reset the SSLConfig, and then `ConcurrentUpdateSolrClientBuilderTest.testSocketTimeoutOnCommit()` tried to connect to an HTTP endpoint using the HTTPS SSLConfig, which fails. I think we should just revert this and keep the original logic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-14414) New Admin UI
[ https://issues.apache.org/jira/browse/SOLR-14414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17925783#comment-17925783 ] Chris M. Hostetter commented on SOLR-14414: --- {quote}Commit bb27d3e4047dc50849e4c42d7d6bac1119003805 in solr's branch refs/heads/main from Christos Malliaridis {quote} FYI: This commit has made the {{main}} branch un-buildable using older linux kernels due to failures like this... {noformat} > Task :kotlinNpmInstall FAILED /home/hossman/.gradle/nodejs/node-v22.0.0-linux-x64/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /home/hossman/.gradle/nodejs/node-v22.0.0-linux-x64/bin/node) FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':kotlinNpmInstall'. > Process 'Resolving NPM dependencies using yarn' returns 1 {noformat} ...and is affecting at least some of our jenkins builds... [https://ci-builds.apache.org/job/Solr/job/Solr-Docker-Test-main/620/console#gradle-task-76] Unless there is some reason we actually need the latest and greatest version of "nodejs" it may make sense to mitigate he impact of this change by pinning the version of nodejs to an older version? ala: [https://youtrack.jetbrains.com/issue/KT-58502] My current workaround for running tests on my older machine is {{./gradlew check -x kotlinNpmInstall -x :solr:ui:check}} (it's still running, so i can't be certain, but it seems to correctly be skipping anything dependent on node.js) > New Admin UI > > > Key: SOLR-14414 > URL: https://issues.apache.org/jira/browse/SOLR-14414 > Project: Solr > Issue Type: Improvement > Components: Admin UI >Affects Versions: 9.0 >Reporter: Marcus Eagan >Priority: Major > Labels: pull-request-available > Attachments: QueryUX-SolrAdminUIReboot.mov > > Time Spent: 10h > Remaining Estimate: 0h > > We have had a lengthy discussion in the mailing list about the need to build > a modern UI that is both more security and does not depend on deprecated, end > of life code. In this ticket, I intend to familiarize the community with the > efforts of the community to do just that that. While we are nearing feature > parity, but not there yet as many have suggested we could complete this task > in iterations, here is an attempt to get the ball rolling. I have mostly > worked on it in weekend nights on the occasion that I could find the time. > Angular is certainly not my specialty, and this is my first attempt at using > TypeScript besides a few brief learning exercises here and there. However, I > will be engaging experts in both of these areas for consultation as our > community tries to pull our UI into another era. > Many of the components here can improve. One or two them need to be > rewritten, and there are even at least three essential components to the app > missing, along with some tests. A couple other things missing are the V2 API, > which I found difficult to build with in this context because it is not > documented on the web. I understand that it is "self-documenting," but the > most easy-to-use APIs are still documented on the web. Maybe it is entirely > documented on the web, and I had trouble finding it. Forgive me, as that > could be an area of assistance. Another area where I need assistance is > packaging this application as a Solr package. I understand this app is not in > the right place for that today, but it can be. There are still many > improvements to be made in this Jira and certainly in this code. > The project is located in {{lucene-solr/solr/webapp2}}, where there is a > README for information on running the app. > The app can be started from the this directory with {{npm start}} for now. It > can quickly be modified to start as a part of the typical start commands as > it approaches parity. I expect there will be a lot of opinions. I welcome > them, of course. The community input should drive the project's success. > Discussion in mailing list: > https://mail-archives.apache.org/mod_mbox/lucene-dev/202004.mbox/%3CCAF76exK-EB_tyFx0B4fBiA%3DJj8gH%3Divn2Uo6cWvMwhvzRdA3KA%40mail.gmail.com%3E -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17656) Add expert level option to allowe PULL replicas to go ACTIVE w/o RECOVERING
[ https://issues.apache.org/jira/browse/SOLR-17656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris M. Hostetter updated SOLR-17656: -- Attachment: SOLR-17656-1.patch Status: Open (was: Open) Patch updated to include ref-guide additions. > Add expert level option to allowe PULL replicas to go ACTIVE w/o RECOVERING > --- > > Key: SOLR-17656 > URL: https://issues.apache.org/jira/browse/SOLR-17656 > Project: Solr > Issue Type: New Feature >Reporter: Chris M. Hostetter >Assignee: Chris M. Hostetter >Priority: Major > Attachments: SOLR-17656-1.patch, SOLR-17656.patch > > > In situations where a Solr cluster undergoes a rolling restart (or some other > "catastrophic" failure situations requiring/causing solr node restarts) there > can be a snowball effect of poor performance (or even solr node crashing) due > to fewer then normal replicas serving query requests while replicas on > restarting nodes are DOWN or RECOVERING – especially if shard leaders are > also affected, and (restarting) replicas first must wait for a leader > election before they can recover (or wait to finish recovery from an > over-worked leader). > For NRT type usecases, RECOVERING is really a necessary evil to ensure every > replicas is up to date before handling NRT requests – but in the case of PULL > replicas, which are expected to routinely "lag" behind their leader, I've > talked to a lot of Solr users w/usecases where they would be happy to have > PULL replicas back online serving "stale" data ASAP, and let normal > IndexFetching "catchup" with the leader later. > I propose we support a new "advanced" replica property that can be set on > PULL replicas by expert level users, to indicate: on (re)init, these replicas > may skip RECOVERING and go directly to ACTIVE. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Resolved] (SOLR-12224) there is no API to read collection properties
[ https://issues.apache.org/jira/browse/SOLR-12224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gerlowski resolved SOLR-12224. Resolution: Duplicate > there is no API to read collection properties > - > > Key: SOLR-12224 > URL: https://issues.apache.org/jira/browse/SOLR-12224 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 7.3 >Reporter: Hendrik Haddorp >Priority: Major > > Solr 7.3 added the COLLECTIONPROP API call > (https://lucene.apache.org/solr/guide/7_3/collections-api.html#collectionprop) > that allows to set arbitrary properties on a collection. There is however no > API call that returns the data. The only option is to manually read out the > collectionprops.json file in ZK below the collection. > Options could be that the COLLECTIONPROP command has an option to retrieve > properties, have a special command to list the properties and/or to have the > properties listed in the clusterstatus output for a collection. > Would be great if SolrJ would also be supported. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17659) Implement basic authentication in Admin UI
[ https://issues.apache.org/jira/browse/SOLR-17659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17925695#comment-17925695 ] Christos Malliaridis commented on SOLR-17659: - I start to get a better understanding now. What I am wondering now is, would a deep link / a "redirect" to the new UI (either web, desktop client or anything else), with the necessary authentication / authorization token be sufficient? Because the new UI will have to handle such cases, like for OAuth, anyway. Completely hiding the UI from the user would be possible, but there must be then a module that will take care of the auth and that would handle all the redirects and token generation flows. And I think from the perspective of OAuth at least, this could be cumbersome to implement, as OAuth flows are very specific which parties are involved and who is redirecting where. I believe it can work without issues and as expected, as long as there is this "facade" infront of the UI. I'm still not sure though if this could be addressed early enough so that the new UI could make use of that (if it has to do anything differently, like not showing login masks). Perhaps it is for now quicker to go with a more "direct" approach and implement the most important auth options in the new UI, and until we have a working solution for "hiding" the UI, we would use the UIs auth screens? And we should still not forget that we try to address relatively serious issue, the replacement of angularjs that has been discontinued since 2022. With that said, we should definitely not take the security lightly, which is why we discuss these matters in such detail. :) > Implement basic authentication in Admin UI > -- > > Key: SOLR-17659 > URL: https://issues.apache.org/jira/browse/SOLR-17659 > Project: Solr > Issue Type: New Feature > Components: Admin UI >Reporter: Christos Malliaridis >Priority: Major > Labels: new-ui, ui > > In the new UI one of the key features that is not implemented yet is user > authentication. In order to secure and securily access Solr, the user should > be able to authenticate against a Solr instance with basic credentials. > h2. Task > Implement basic user authentication (with credentials) according to the [new > designs|https://www.figma.com/design/VdbEfcWQ8mirFNquBzbPk2/Apache-Solr-Admin-UI-v2-Concept?node-id=1190-388&t=vMgOa9QlzQZSdjLf-1]. > h2. Acceptance Criteria > - The user can access a Solr instance that has user authentication enabled > - The user can at least authenticate with credentials (basic auth) > - The credentials form is displayed after the user has established a > connection with a Solr instance, that is, after a Solr instance was found > - The user can return to the start screen where the Solr URL was provided, > if he decides to abort the authentication step > - The user is no longer redirected to the dashboard or any other screen if > user authentication is required > - The credentials are used for any subsequent request > h2. Additional Information > The support for additional authentication options does not have to be > addressed in this issue. If it proves to be straight-forward, feel free to > implement additional auth options as well. Note that additional > authentication options will be added later, and therefore, the implementation > should be expandable. > The credentials do not have to survive an application restart (desktop). > Storing credentials securely will be addressed in a separate issue. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16390) Cosmetic improvements and migration to JAX-RS (v2 cluster and clusterprop APIs)
[ https://issues.apache.org/jira/browse/SOLR-16390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17925719#comment-17925719 ] Jason Gerlowski commented on SOLR-16390: The cluster property APIs are done, but there were other APIs bundled into this ticket that haven't been tweaked and migrated to JAX-RS yet. So we can't close this quite yet. Also, SOLR-12224 is concerned with COLLECTION properties (i.e. COLLECTION vs CLUSTER), that that ticket is unrelated entirely to the work here. > Cosmetic improvements and migration to JAX-RS (v2 cluster and clusterprop > APIs) > --- > > Key: SOLR-16390 > URL: https://issues.apache.org/jira/browse/SOLR-16390 > Project: Solr > Issue Type: Sub-task > Components: documentation, v2 API >Affects Versions: main (10.0) >Reporter: Jason Gerlowski >Priority: Major > Labels: newdev, pull-request-available > Time Spent: 4.5h > Remaining Estimate: 0h > > As mentioned on SOLR-15781, the v2 API currently has an experimental > designation, and the community has expressed an interest in using this period > to update our v2 endpoints to be more REST-ful and consistent. The current > plan is to follow the specific changes laid out in [this > spreadsheet|https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing], > though of course nothing there is set in stone and there are still warts to > be worked out. > This ticket plans to tackle making the changes required for Solr's "Cluster" > and "cluster-prop" APIs. These APIs are described in detail in the > spreadsheet linked above, but are summarized in the table below for > convenience and easier tracking. > While we're touching the code for these endpoints, we should also convert > them to JAX-RS framework definitions. (This was initially tracked as a > separate effort - see SOLR-16370 - but the edit that were required ended up > overlapping so significantly with the "cosmetic" improvements here that in > practice it almost always makes sense to do the two together.) > *Cosmetic Changes and JAX-RS Conversion* > ||API Name||Original Form||Desired Form||Status||Volunteer|| > |Upsert ClusterProp|POST /api/cluster \{"set-property": \{...\}\}|PUT > /api/cluster/properties/propName \{"value": "propVal"\}|Open|N/A| > |Upsert ClusterProp (Potentially Nested)|POST /api/cluster > \{"set-obj-property": \{...\}\}|PUT /api/cluster/properties \{...\}|Open|N/A| > |Delete ClusterProp|POST /api/cluster \{"set-property": \{"name": "foo", > "value": ""\}\}|DELETE /api/cluster/properties/propName|Open|N/A| > |Delete Single Async Status|DELETE /api/cluster/command-status/123|DELETE > /api/cluster/commands/123|Open|N/A| > |Delete All Async Status|DELETE /api/cluster/command-status|DELETE > /api/cluster/commands|Open|N/A| > |Get Single Async Status|GET /api/cluster/command-status/123|GET > /api/cluster/commands/123|Open|N/A| > |Create rate-limiter|POST /api/cluster/ \{"set-ratelimiter": \{...\}\}|POST > /api/cluster/ratelimiter \{...\}|Open|N/A| -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Reopened] (SOLR-12224) there is no API to read collection properties
[ https://issues.apache.org/jira/browse/SOLR-12224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gerlowski reopened SOLR-12224: I closed this out errantly as a duplicate of SOLR-16390, but that ticket turns out to be unrelated. My mistake! > there is no API to read collection properties > - > > Key: SOLR-12224 > URL: https://issues.apache.org/jira/browse/SOLR-12224 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 7.3 >Reporter: Hendrik Haddorp >Priority: Major > > Solr 7.3 added the COLLECTIONPROP API call > (https://lucene.apache.org/solr/guide/7_3/collections-api.html#collectionprop) > that allows to set arbitrary properties on a collection. There is however no > API call that returns the data. The only option is to manually read out the > collectionprops.json file in ZK below the collection. > Options could be that the COLLECTIONPROP command has an option to retrieve > properties, have a special command to list the properties and/or to have the > properties listed in the clusterstatus output for a collection. > Would be great if SolrJ would also be supported. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-16390) Cosmetic improvements and migration to JAX-RS (v2 cluster and clusterprop APIs)
[ https://issues.apache.org/jira/browse/SOLR-16390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gerlowski updated SOLR-16390: --- Description: As mentioned on SOLR-15781, the v2 API currently has an experimental designation, and the community has expressed an interest in using this period to update our v2 endpoints to be more REST-ful and consistent. The current plan is to follow the specific changes laid out in [this spreadsheet|https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing], though of course nothing there is set in stone and there are still warts to be worked out. This ticket plans to tackle making the changes required for Solr's "Cluster" and "cluster-prop" APIs. These APIs are described in detail in the spreadsheet linked above, but are summarized in the table below for convenience and easier tracking. While we're touching the code for these endpoints, we should also convert them to JAX-RS framework definitions. (This was initially tracked as a separate effort - see SOLR-16370 - but the edit that were required ended up overlapping so significantly with the "cosmetic" improvements here that in practice it almost always makes sense to do the two together.) *Cosmetic Changes and JAX-RS Conversion* ||API Name||Original Form||Desired Form||Status||Volunteer|| |-Upsert ClusterProp-|-POST /api/cluster \{"set-property": \{...\}\}-|-PUT /api/cluster/properties/propName \{"value": "propVal"\}-|-Closed-|-Jason-| |-Upsert ClusterProp (Potentially Nested)-|-POST /api/cluster \{"set-obj-property": \{...\}\}-|-PUT /api/cluster/properties \{...\}-|-Closed-|-Jason-| |-Delete ClusterProp-|-POST /api/cluster \{"set-property": \{"name": "foo", "value": ""\}\}-|-DELETE /api/cluster/properties/propName-|-Closed-|-Jason-| |Delete Single Async Status|DELETE /api/cluster/command-status/123|DELETE /api/cluster/commands/123|Open|N/A| |Delete All Async Status|DELETE /api/cluster/command-status|DELETE /api/cluster/commands|Open|N/A| |Get Single Async Status|GET /api/cluster/command-status/123|GET /api/cluster/commands/123|Open|N/A| |Create rate-limiter|POST /api/cluster/ \{"set-ratelimiter": \{...\}\}|POST /api/cluster/ratelimiter \{...\}|Open|N/A| was: As mentioned on SOLR-15781, the v2 API currently has an experimental designation, and the community has expressed an interest in using this period to update our v2 endpoints to be more REST-ful and consistent. The current plan is to follow the specific changes laid out in [this spreadsheet|https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing], though of course nothing there is set in stone and there are still warts to be worked out. This ticket plans to tackle making the changes required for Solr's "Cluster" and "cluster-prop" APIs. These APIs are described in detail in the spreadsheet linked above, but are summarized in the table below for convenience and easier tracking. While we're touching the code for these endpoints, we should also convert them to JAX-RS framework definitions. (This was initially tracked as a separate effort - see SOLR-16370 - but the edit that were required ended up overlapping so significantly with the "cosmetic" improvements here that in practice it almost always makes sense to do the two together.) *Cosmetic Changes and JAX-RS Conversion* ||API Name||Original Form||Desired Form||Status||Volunteer|| |-Upsert ClusterProp-|-POST /api/cluster \{"set-property": \{...\}\}-|-PUT /api/cluster/properties/propName \{"value": "propVal"\}-|-Closed-|-Jason-| |Upsert ClusterProp (Potentially Nested)|POST /api/cluster \{"set-obj-property": \{...\}\}|PUT /api/cluster/properties \{...\}|Open|N/A| |Delete ClusterProp|POST /api/cluster \{"set-property": \{"name": "foo", "value": ""\}\}|DELETE /api/cluster/properties/propName|Open|N/A| |Delete Single Async Status|DELETE /api/cluster/command-status/123|DELETE /api/cluster/commands/123|Open|N/A| |Delete All Async Status|DELETE /api/cluster/command-status|DELETE /api/cluster/commands|Open|N/A| |Get Single Async Status|GET /api/cluster/command-status/123|GET /api/cluster/commands/123|Open|N/A| |Create rate-limiter|POST /api/cluster/ \{"set-ratelimiter": \{...\}\}|POST /api/cluster/ratelimiter \{...\}|Open|N/A| > Cosmetic improvements and migration to JAX-RS (v2 cluster and clusterprop > APIs) > --- > > Key: SOLR-16390 > URL: https://issues.apache.org/jira/browse/SOLR-16390 > Project: Solr > Issue Type: Sub-task > Components: documentation, v2 API >Affects Versions: main (10.0) >Reporter: Jason Gerlowski >Priority: Major > Labels: newdev, pull-request-available > Time Spent: 4.5h > Remaining Estimate: 0h > > As
[jira] [Updated] (SOLR-16390) Cosmetic improvements and migration to JAX-RS (v2 cluster and clusterprop APIs)
[ https://issues.apache.org/jira/browse/SOLR-16390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gerlowski updated SOLR-16390: --- Description: As mentioned on SOLR-15781, the v2 API currently has an experimental designation, and the community has expressed an interest in using this period to update our v2 endpoints to be more REST-ful and consistent. The current plan is to follow the specific changes laid out in [this spreadsheet|https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing], though of course nothing there is set in stone and there are still warts to be worked out. This ticket plans to tackle making the changes required for Solr's "Cluster" and "cluster-prop" APIs. These APIs are described in detail in the spreadsheet linked above, but are summarized in the table below for convenience and easier tracking. While we're touching the code for these endpoints, we should also convert them to JAX-RS framework definitions. (This was initially tracked as a separate effort - see SOLR-16370 - but the edit that were required ended up overlapping so significantly with the "cosmetic" improvements here that in practice it almost always makes sense to do the two together.) *Cosmetic Changes and JAX-RS Conversion* ||API Name||Original Form||Desired Form||Status||Volunteer|| |-Upsert ClusterProp-|-POST /api/cluster \{"set-property": \{...\}\}-|-PUT /api/cluster/properties/propName \{"value": "propVal"\}-|-Closed-|-Jason-| |Upsert ClusterProp (Potentially Nested)|POST /api/cluster \{"set-obj-property": \{...\}\}|PUT /api/cluster/properties \{...\}|Open|N/A| |Delete ClusterProp|POST /api/cluster \{"set-property": \{"name": "foo", "value": ""\}\}|DELETE /api/cluster/properties/propName|Open|N/A| |Delete Single Async Status|DELETE /api/cluster/command-status/123|DELETE /api/cluster/commands/123|Open|N/A| |Delete All Async Status|DELETE /api/cluster/command-status|DELETE /api/cluster/commands|Open|N/A| |Get Single Async Status|GET /api/cluster/command-status/123|GET /api/cluster/commands/123|Open|N/A| |Create rate-limiter|POST /api/cluster/ \{"set-ratelimiter": \{...\}\}|POST /api/cluster/ratelimiter \{...\}|Open|N/A| was: As mentioned on SOLR-15781, the v2 API currently has an experimental designation, and the community has expressed an interest in using this period to update our v2 endpoints to be more REST-ful and consistent. The current plan is to follow the specific changes laid out in [this spreadsheet|https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing], though of course nothing there is set in stone and there are still warts to be worked out. This ticket plans to tackle making the changes required for Solr's "Cluster" and "cluster-prop" APIs. These APIs are described in detail in the spreadsheet linked above, but are summarized in the table below for convenience and easier tracking. While we're touching the code for these endpoints, we should also convert them to JAX-RS framework definitions. (This was initially tracked as a separate effort - see SOLR-16370 - but the edit that were required ended up overlapping so significantly with the "cosmetic" improvements here that in practice it almost always makes sense to do the two together.) *Cosmetic Changes and JAX-RS Conversion* ||API Name||Original Form||Desired Form||Status||Volunteer|| |Upsert ClusterProp|POST /api/cluster \{"set-property": \{...\}\}|PUT /api/cluster/properties/propName \{"value": "propVal"\}|Open|N/A| |Upsert ClusterProp (Potentially Nested)|POST /api/cluster \{"set-obj-property": \{...\}\}|PUT /api/cluster/properties \{...\}|Open|N/A| |Delete ClusterProp|POST /api/cluster \{"set-property": \{"name": "foo", "value": ""\}\}|DELETE /api/cluster/properties/propName|Open|N/A| |Delete Single Async Status|DELETE /api/cluster/command-status/123|DELETE /api/cluster/commands/123|Open|N/A| |Delete All Async Status|DELETE /api/cluster/command-status|DELETE /api/cluster/commands|Open|N/A| |Get Single Async Status|GET /api/cluster/command-status/123|GET /api/cluster/commands/123|Open|N/A| |Create rate-limiter|POST /api/cluster/ \{"set-ratelimiter": \{...\}\}|POST /api/cluster/ratelimiter \{...\}|Open|N/A| > Cosmetic improvements and migration to JAX-RS (v2 cluster and clusterprop > APIs) > --- > > Key: SOLR-16390 > URL: https://issues.apache.org/jira/browse/SOLR-16390 > Project: Solr > Issue Type: Sub-task > Components: documentation, v2 API >Affects Versions: main (10.0) >Reporter: Jason Gerlowski >Priority: Major > Labels: newdev, pull-request-available > Time Spent: 4.5h > Remaining Estimate: 0h > > As mentioned on SOLR-15781, the v2 API curren
[jira] [Comment Edited] (SOLR-12224) there is no API to read collection properties
[ https://issues.apache.org/jira/browse/SOLR-12224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17925721#comment-17925721 ] Jason Gerlowski edited comment on SOLR-12224 at 2/10/25 8:16 PM: - I closed this out errantly as a duplicate of SOLR-16390, but that ticket turns out to be unrelated. My mistake! I *do* think that adding get and/or list collectionprop APIs would be great for v2 in particular, we just need a volunteer to help with that effort. I'm more than happy to guide and help if someone's willing to drive development. was (Author: gerlowskija): I closed this out errantly as a duplicate of SOLR-16390, but that ticket turns out to be unrelated. My mistake! > there is no API to read collection properties > - > > Key: SOLR-12224 > URL: https://issues.apache.org/jira/browse/SOLR-12224 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 7.3 >Reporter: Hendrik Haddorp >Priority: Major > > Solr 7.3 added the COLLECTIONPROP API call > (https://lucene.apache.org/solr/guide/7_3/collections-api.html#collectionprop) > that allows to set arbitrary properties on a collection. There is however no > API call that returns the data. The only option is to manually read out the > collectionprops.json file in ZK below the collection. > Options could be that the COLLECTIONPROP command has an option to retrieve > properties, have a special command to list the properties and/or to have the > properties listed in the clusterstatus output for a collection. > Would be great if SolrJ would also be supported. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17569) TestLBHttpSolrClient/LBHttp2SolrClientIntegrationTest unreproducible test failures
[ https://issues.apache.org/jira/browse/SOLR-17569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17925763#comment-17925763 ] Houston Putman commented on SOLR-17569: --- [~jdyer] These tests work perfectly on main, but are still very error prone on branch_9x and branch_9_8. Do you have any ideas as to why this would be? I can't reproduce locally, and don't understand why main and branch_9x would differ so much. (The logic looks very similar) > TestLBHttpSolrClient/LBHttp2SolrClientIntegrationTest unreproducible test > failures > --- > > Key: SOLR-17569 > URL: https://issues.apache.org/jira/browse/SOLR-17569 > Project: Solr > Issue Type: Bug >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Labels: pull-request-available > Fix For: main (10.0), 9.8 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > *TestLBHttpSolrClient* (branch_9x) was renamed to > *LBHttp2SolrClientIntegrationTest* and has a high failure rate. These > failures are not reproducible. Consider altering the test technique to > eliminate false-positive failures. > - > {noformat} > Build: > https://ci-builds.apache.org/job/Solr/job/Solr-BadApples-Tests-main/1486/ > 1 tests failed. > Build: > https://ci-builds.apache.org/job/Solr/job/Solr-BadApples-Tests-main/1486/ > FAILED: org.apache.solr.client.solrj.TestLBHttp2SolrClient.testSimple > Error Message: > java.lang.AssertionError: expected:<3> but was:<2> > Stack Trace: > java.lang.AssertionError: expected:<3> but was:<2> > at > __randomizedtesting.SeedInfo.seed([3CF24FAC059E0DDC:4416B52226DD90D]:0) > at org.junit.Assert.fail(Assert.java:89) > at org.junit.Assert.failNotEquals(Assert.java:835) > at org.junit.Assert.assertEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:633) > at > org.apache.solr.client.solrj.TestLBHttp2SolrClient.testSimple(TestLBHttp2SolrClient.java:165) > Reproduce with: ./gradlew :solr:solrj:test --tests > "org.apache.solr.client.solrj.TestLBHttp2SolrClient.testSimple" > -Ptests.jvms=4 -Ptests.haltonfailure=false > "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC > -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" > -Ptests.seed=3CF24FAC059E0DDC -Ptests.multiplier=2 -Ptests.badapples=true > -Ptests.file.encoding=US-ASCII > {noformat} > - > {noformat} > Build: https://ci-builds.apache.org/job/Solr/job/Solr-Test-9.x/6323/ > 1 tests failed. > FAILED: org.apache.solr.client.solrj.TestLBHttp2SolrClient.testTwoServers > Error Message: > org.apache.solr.client.solrj.SolrServerException: Total timeout 2000 ms > elapsed > Stack Trace: > org.apache.solr.client.solrj.SolrServerException: Total timeout 2000 ms > elapsed > at > __randomizedtesting.SeedInfo.seed([C10E68448E178BC0:61E4C6C9555CA5E0]:0) > at > app//org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:562) > at > app//org.apache.solr.client.solrj.impl.LBSolrClient.lambda$doRequest$0(LBSolrClient.java:558) > at > app//org.apache.solr.client.solrj.impl.Http2SolrClient.requestWithBaseUrl(Http2SolrClient.java:604) > at > app//org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:558) > at > app//org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:809) > at > app//org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:753) > at > app//org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:279) > at > app//org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:927) > at > app//org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:940) > at > app//org.apache.solr.client.solrj.TestLBHttp2SolrClient.testTwoServers(TestLBHttp2SolrClient.java:202) > ...etc... > Caused by: java.util.concurrent.TimeoutException: Total timeout 2000 ms > elapsed > at > org.eclipse.jetty.client.HttpDestination$RequestTimeouts.onExpired(HttpDestination.java:608) > at > org.eclipse.jetty.client.HttpDestination$RequestTimeouts.onExpired(HttpDestination.java:591) > at > org.eclipse.jetty.io.CyclicTimeouts.onTimeoutExpired(CyclicTimeouts.java:110) > at > org.eclipse.jetty.io.CyclicTimeouts$Timeouts.onTimeoutExpired(CyclicTimeouts.java:197) > at > org.eclipse.jetty.io.CyclicTimeout$Wakeup.run(CyclicTimeout.java:294) > Reproduce with: ./gradlew :solr:solrj:test --tests > "org.apache.solr.client.solrj.TestLBHttp2SolrClient.testTwoServers" > -Ptests.jvms=4 -Ptests.haltonfailure=false > "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC > -XX:ActiveProcessorCount=1 -XX:Reserv
[jira] [Commented] (SOLR-17656) Add expert level option to allowe PULL replicas to go ACTIVE w/o RECOVERING
[ https://issues.apache.org/jira/browse/SOLR-17656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17925817#comment-17925817 ] David Smiley commented on SOLR-17656: - Sounds useful. > Add expert level option to allowe PULL replicas to go ACTIVE w/o RECOVERING > --- > > Key: SOLR-17656 > URL: https://issues.apache.org/jira/browse/SOLR-17656 > Project: Solr > Issue Type: New Feature >Reporter: Chris M. Hostetter >Assignee: Chris M. Hostetter >Priority: Major > Attachments: SOLR-17656-1.patch, SOLR-17656.patch > > > In situations where a Solr cluster undergoes a rolling restart (or some other > "catastrophic" failure situations requiring/causing solr node restarts) there > can be a snowball effect of poor performance (or even solr node crashing) due > to fewer then normal replicas serving query requests while replicas on > restarting nodes are DOWN or RECOVERING – especially if shard leaders are > also affected, and (restarting) replicas first must wait for a leader > election before they can recover (or wait to finish recovery from an > over-worked leader). > For NRT type usecases, RECOVERING is really a necessary evil to ensure every > replicas is up to date before handling NRT requests – but in the case of PULL > replicas, which are expected to routinely "lag" behind their leader, I've > talked to a lot of Solr users w/usecases where they would be happy to have > PULL replicas back online serving "stale" data ASAP, and let normal > IndexFetching "catchup" with the leader later. > I propose we support a new "advanced" replica property that can be set on > PULL replicas by expert level users, to indicate: on (re)init, these replicas > may skip RECOVERING and go directly to ACTIVE. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17650) Determine executor for UpdateLog reading
[ https://issues.apache.org/jira/browse/SOLR-17650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17925714#comment-17925714 ] Mark Robert Miller commented on SOLR-17650: --- I took a look at this. In regards to multiple ways to read the updateLog: We started with no parallel reply. You need to be able to read in sorted order because updates might be reordered, say going from leader to replica, and you don’t drop based on versions at the tlog level and you have things like dependent updates. So sorted order read will make sure you read in logical order from the tlog with a sorted reader. The reason we wanted to add parallel replay is because if you have buffered and are trying to catch up while updates are continuously coming in, maybe it takes forever to catch up or a long time, because you are buffering as fast or faster than you are replaying. So OrderedExecutor was added, in this case you don’t have to read in sorted order, the OrderedExecutor handles reorders of updates to the same ID and delete by query is handled by waiting for the executor to drain before replaying it. I think Yonik didn’t always use parallel replay because typically you wouldn’t expect to be replaying a lot, and so the coordination overhead is often probably not worth it in the normal case. As to the fails, Houston's analysis looks correct to me, before SOLR-17391, only one thread appears to have been used, and now thread scheduling can changed the order of applied buffered updates. > Determine executor for UpdateLog reading > > > Key: SOLR-17650 > URL: https://issues.apache.org/jira/browse/SOLR-17650 > Project: Solr > Issue Type: Improvement >Reporter: Houston Putman >Priority: Major > > Currently most operations that read from the updateLog use the > OrderedExecutor that is setup for that purpose. However, TLOG replicas, when > taking leadership, read from the updateLog without the executor (using the > inSortedOrder = true parameter). > SOLR-17391 changed the default executor for updateLog reading to use a fixed > coreSize thread pool, which seems like started to enable parallel reading of > the updateLog (which was supposed to be true beforehand, but it looks like > our understanding of the executor was wrong). > This ticket's purpose is to: > # Investigate the correct way of reading the updateLog using an executor > # Determine if there needs to actually be multiple ways > (inSortedOrder=true/false) > # Make sure the tests pass with whatever we decide. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SolrTestCaseJ4: don't reset HttpClient SSL stuff [solr]
dsmiley commented on PR #3037: URL: https://github.com/apache/solr/pull/3037#issuecomment-2649745240 Very impressive debugging Houston! Apparently I overlooked STCJ4.setupTestCases SSL stuff. Maybe I should undo the STCJ4 change (as you suggest), although I like the explicit changes to the 2 tests I also touched here. I ultimately want to be able to switch many tests extending STCJ4 to use SolrTestCase instead. But I don't want STC to become so bloated as STCJ4 is. Perhaps the SSL matters can be encapsulated into a TestRule. Oh I just noticed JettyConfig.Builder is referencing the public static field STCJ4.sslConfig -- _evil_! (separate but related issue; should raise to the dev list) I sense we'd have an easier time with randomization if we change our approach for some scenarios like SSL (also PRS, non-Overseer). Instead of flip-flopping from one test to the next, it'd be easier to track issues and debug if the entire build (all the tests selected) run a consistent SSL choice, and we either have a dedicated build or randomly choose this at the start. If it's the whole build, we'd then use Develocity tags, thus aiding tracking patterns. The choice could be determined in gradle and passed only when enabled. A failure would have a reproducer line showing the additional flags. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17659) Implement basic authentication in Admin UI
[ https://issues.apache.org/jira/browse/SOLR-17659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17925765#comment-17925765 ] Jan Høydahl commented on SOLR-17659: I think this discussion is de-railing the progress for simple BasicAuth support in new UI. Let's keep Solr backend out of this in this phase. This Jira won't change anything about the security stance for Solr. The APIs are as secure as they always were, and I see no reason whatsoever to try to obscure what APIs a solr server has available. We must assume both users and attackers how what Solr is and what APIs it has. The Solr backend won't be less secure by adding another UI frontend, and it won't be more secure by trying to hide APIs or version info. Btw - the Solr APIs *do* implement fairly standard auth headers. If you hit any Solr endpoint without auth, the server will respond with 401 along with the WWW-Authenticate header, which tells the client what auth(s) is enabled on the server. And the old UI parses the WWW-Authenticate header to choose what login form to display. In case of JWT auth enabled, Solr *also* emits [a custom header "X-Solr-AuthData"|https://github.com/apache/solr/blob/main/solr/modules/jwt-auth/src/java/org/apache/solr/security/jwt/JWTAuthPlugin.java#L850-L862] that the UI parses to have enough data for it to handle the OIDC auth flow. Since the Admin UI is 100% static it needs to get this from the server. > Implement basic authentication in Admin UI > -- > > Key: SOLR-17659 > URL: https://issues.apache.org/jira/browse/SOLR-17659 > Project: Solr > Issue Type: New Feature > Components: Admin UI >Reporter: Christos Malliaridis >Priority: Major > Labels: new-ui, ui > > In the new UI one of the key features that is not implemented yet is user > authentication. In order to secure and securily access Solr, the user should > be able to authenticate against a Solr instance with basic credentials. > h2. Task > Implement basic user authentication (with credentials) according to the [new > designs|https://www.figma.com/design/VdbEfcWQ8mirFNquBzbPk2/Apache-Solr-Admin-UI-v2-Concept?node-id=1190-388&t=vMgOa9QlzQZSdjLf-1]. > h2. Acceptance Criteria > - The user can access a Solr instance that has user authentication enabled > - The user can at least authenticate with credentials (basic auth) > - The credentials form is displayed after the user has established a > connection with a Solr instance, that is, after a Solr instance was found > - The user can return to the start screen where the Solr URL was provided, > if he decides to abort the authentication step > - The user is no longer redirected to the dashboard or any other screen if > user authentication is required > - The credentials are used for any subsequent request > h2. Additional Information > The support for additional authentication options does not have to be > addressed in this issue. If it proves to be straight-forward, feel free to > implement additional auth options as well. Note that additional > authentication options will be added later, and therefore, the implementation > should be expandable. > The credentials do not have to survive an application restart (desktop). > Storing credentials securely will be addressed in a separate issue. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Resolved] (SOLR-17654) DistribFileStore._getRealPath() has issues on Windows
[ https://issues.apache.org/jira/browse/SOLR-17654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Houston Putman resolved SOLR-17654. --- Assignee: Houston Putman Resolution: Fixed > DistribFileStore._getRealPath() has issues on Windows > - > > Key: SOLR-17654 > URL: https://issues.apache.org/jira/browse/SOLR-17654 > Project: Solr > Issue Type: Improvement >Reporter: Houston Putman >Assignee: Houston Putman >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > On Windows, many tests that use the DistribFileStore, such as > {{TestPackages}}, {{TestDistribFileStore}} and {{PackageToolTest}} are > failing because of an issue in {{DistribFileStore._getRealPath()}}. > This method tries to remove the beginning slashes from the path, and then > tries to make a new path relative to the file store location. However, in the > tests, it's failing and showing stuff like "Illegal path > \mypkg\v.0.12\jar_a.jar". Clearly in the code, the first "\" should have been > removed, so this code is having an issue with Windows. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org