Re: [PR] SOLR-17609: Remove HDFS module [solr]
epugh commented on PR #2923: URL: https://github.com/apache/solr/pull/2923#issuecomment-2558164589 Question! Should `AbstractRecoveryZkTestBase` be merged back into `RecoveryZkTest` since we no longer have the HDFS contrib tests? That goes for `AbstractChaosMonkeySafeLeaderTestBase`, `AbstractMoveReplicaTestBase`,`AbstractRecoveryZkTestBase`, `AbstractRestartWhileUpdatingTestBase`, `AbstractSyncSliceTestBase`, `AbstractTlogReplayBufferedWhileIndexingTestBase`, `AbstractUnloadDistributedZkTestBase` files as well? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-6999) deprecate the init(NamedList) in favor of init(PluginInfo)
[
https://issues.apache.org/jira/browse/SOLR-6999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17907620#comment-17907620
]
Eric Pugh commented on SOLR-6999:
-
+1! I ran into this confusion writing the UBI plugin...
> deprecate the init(NamedList) in favor of init(PluginInfo)
> --
>
> Key: SOLR-6999
> URL: https://issues.apache.org/jira/browse/SOLR-6999
> Project: Solr
> Issue Type: Task
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
>
> Having different ways to initialize components is confusing and we should
> just make every component {{PluginInfoInitialized}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17609: Remove HDFS module [solr]
epugh commented on PR #2923: URL: https://github.com/apache/solr/pull/2923#issuecomment-2558180476 Opportunity to remove the `isSharedStorage` property of DirectoryFactory? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Adding Powershell snippets to Five Minutes Tutorial [solr]
epugh commented on PR #2482: URL: https://github.com/apache/solr/pull/2482#issuecomment-2558160012 BTW, I think this was a nice idea, and maybe in the future we'll add it if we see more folks using powershell. I'm also thinking that in the future as our CLI maybe gets richer, we may find that it's possible to use the CLI for all the thigns int he tutorial, and not need language specific examples... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17608) v1/v2 serialization should support 'Instant'
Jason Gerlowski created SOLR-17608: -- Summary: v1/v2 serialization should support 'Instant' Key: SOLR-17608 URL: https://issues.apache.org/jira/browse/SOLR-17608 Project: Solr Issue Type: Improvement Components: Response Writers, SolrJ Reporter: Jason Gerlowski David pointed out on a recent PR that it'd be nice if our API responses could use 'java.time.Instant' objects. This is do-able but would require changes to the following: * add stub 'JsonSerializer' and 'JsonDeserializer' implementations to our SolrJacksonMapper * add handling of 'Instant' in 'TextWriter.writeVal', and the corresponding location in JavaBinCodec ** possible read-side changes to match those in TextWriter+JavaBinCodec? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17609) Remove hdfs module
Eric Pugh created SOLR-17609: Summary: Remove hdfs module Key: SOLR-17609 URL: https://issues.apache.org/jira/browse/SOLR-17609 Project: Solr Issue Type: Task Components: hdfs Affects Versions: main (10.0) Reporter: Eric Pugh One of the outcomes of the 2024 Community Survey is that we learned (in our admittedly fairly unscientific) responses that hdfs module is not used. This PR is to understand the impact of removing hdfs in Solr 10. See [https://lists.apache.org/thread/hp6bov79rgrg0gb2ozzbzxxn30k2js0h] for discussion on Dev. I won't merge this PR till we have more consensus. This builds on work started in https://issues.apache.org/jira/browse/SOLR-14660 and https://issues.apache.org/jira/browse/SOLR-14021 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-15751: Modify v2 `GET /collections/collName` to support full ColStatus response [solr]
gerlowskija commented on PR #2912: URL: https://github.com/apache/solr/pull/2912#issuecomment-2558115243 > Still need to look into David's question about use of `Instant` Alright - the main hiccup for using `Instant` is that it's not supported by our serialization layer. Jackson supports it pretty easily on the v2 side, but our v1 "ResponseWriter"-based serialization code doesn't. (The v1 code needs support because of how our v1 APIs are built "on top of" the v2 APIs - "primitive" types used in the v2 POJO end up in the NamedList that the v1 code serializes out.) Anyway, I think it would be nice to support this, but I don't want to squeeze it into this PR so I'm not tempted to skimp on testing, etc. I've filed [a ticket for it](https://issues.apache.org/jira/browse/SOLR-17608), and hope to pursue that shortly. In the meantime I added 'TODO' comments to the places David flagged as being 'Instant'-ready - we can circle back to those either as a part of SOLR-17608, or once it's done. (FYI @dsmiley ) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17540) Remove hadoop-auth module
[ https://issues.apache.org/jira/browse/SOLR-17540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17907638#comment-17907638 ] David Smiley commented on SOLR-17540: - Please remember to deprecate in 9x. You can mention it in CHANGES.txt with this issue and mark one or more of the key plugin classes (that which are loaded by name in configuration) as Deprecated annotation. When Solr loads a plugin that it sees has this annotation, it prints a warning to the user. Perhaps for some users, this will be their wake-up call as to what happened. > Remove hadoop-auth module > - > > Key: SOLR-17540 > URL: https://issues.apache.org/jira/browse/SOLR-17540 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: main (10.0) >Reporter: Eric Pugh >Assignee: Eric Pugh >Priority: Minor > Labels: pull-request-available > Fix For: main (10.0) > > Time Spent: 7h 10m > Remaining Estimate: 0h > > One of the outcomes of the 2024 Community Survey is that we learned (in our > admittedly fairly unscientific) responses that hadoop-auth is not used. > This PR is to understand the impact of removing hadoop-auth in Solr 10. > See [https://lists.apache.org/thread/vnd73j0nq3losfc17lzqp48g10r5tdgg] for > outreach to users mailing list. > See [https://lists.apache.org/thread/lltc0wjdghq18tt37zlrsd8ty35qsytl] for > discussion on Dev. > > I won't merge this PR till we have more consensus. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Deprecate MapWriter.append [solr]
dsmiley commented on code in PR #2919:
URL: https://github.com/apache/solr/pull/2919#discussion_r1894685936
##
solr/solrj/src/java/org/apache/solr/common/MapWriter.java:
##
@@ -70,8 +74,7 @@ public MapWriter.EntryWriter put(CharSequence k, Object v) {
writer.endObject();
}
- void writeMap(EntryWriter ew) throws IOException;
-
+ @Deprecated
Review Comment:
> I did a pass the other week around deprecated code and a lot it wasn't
clear what would trigger the deleting of it.. Or how long it had been
deprecated.
Curious; in that instance, why did you care? In this case, it's just some
small utility method that any caller can do themselves. Thus the deprecation
rationale is not something I think any user would find interesting _for this
method_.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17582 Stream CLUSTERSTATUS API response [solr]
mlbiscoc commented on PR #2916: URL: https://github.com/apache/solr/pull/2916#issuecomment-2558290502 > Uh oh; a back-compat problem is dawning on me. Any SolrJ consumer of this API (especially using the default "javabin" format) is going to break here. The change to BaseHttpClusterStateProvider should have made this glaringly obvious but I overlooked its implication earlier (face-palm). > > I wouldn't mind if we do this only for V2 (which doesn't yet have back-compatibility concerns) but V2 only has one collection status list returning API (/cluster) and I've debated against the very existence of /cluster for V2. There's not yet a proposal for an alternative. Furthermore V2 raises the bigger question of coalescing on a single DocCollection serialization method. > > If we forge ahead anyway (not waiting for V2), we'll have to work around this compatibility. I had a couple ideas but whittled down to one: > > * detect the response format in this handler. If it's Javabin, we write to a NamedList (or better, SimpleOrderedMap), defeating the point of this PR _for that format_. If JSON/XML is used, this PR will accomplish its goal. Need to write a little test for JSON that at least partially sanity checks the result. Such a test should pass without this PR. Well thats unfortunate... I'm not too familiar with SolrJ (I'll maybe play with it more to catch stuff like this) but I am assuming now the API is going to return a `Map` and no longer a `NamedList` to the client thus breaking them now? I'll continue with this and add in your ideas to keep this backwards compatible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17587: Prometheus Writer duplicate TYPE information in exposition format [solr]
mlbiscoc commented on code in PR #2902:
URL: https://github.com/apache/solr/pull/2902#discussion_r1894710055
##
solr/core/src/java/org/apache/solr/metrics/prometheus/core/SolrCoreMetric.java:
##
@@ -30,18 +30,18 @@ public abstract class SolrCoreMetric extends SolrMetric {
public String coreName;
public SolrCoreMetric(Metric dropwizardMetric, String metricName) {
-super(dropwizardMetric, metricName);
+super(dropwizardMetric, metricName.substring(metricName.indexOf(".") + 1));
Review Comment:
Done!
##
solr/core/src/java/org/apache/solr/metrics/prometheus/core/SolrCoreMetric.java:
##
@@ -30,18 +30,18 @@ public abstract class SolrCoreMetric extends SolrMetric {
public String coreName;
public SolrCoreMetric(Metric dropwizardMetric, String metricName) {
-super(dropwizardMetric, metricName);
+super(dropwizardMetric, metricName.substring(metricName.indexOf(".") + 1));
Matcher cloudPattern = CLOUD_CORE_PATTERN.matcher(metricName);
Matcher standalonePattern = STANDALONE_CORE_PATTERN.matcher(metricName);
if (cloudPattern.find()) {
- coreName = cloudPattern.group(1);
- labels.put("core", cloudPattern.group(1));
- labels.put("collection", cloudPattern.group(2));
- labels.put("shard", cloudPattern.group(3));
- labels.put("replica", cloudPattern.group(4));
+ coreName = cloudPattern.group("core");
+ labels.put("core", cloudPattern.group("core"));
+ labels.put("collection", cloudPattern.group("collection"));
+ labels.put("shard", cloudPattern.group("shard"));
+ labels.put("replica", cloudPattern.group("replica"));
} else if (standalonePattern.find()) {
- coreName = standalonePattern.group(1);
- labels.put("core", standalonePattern.group(1));
+ coreName = standalonePattern.group("core");
+ labels.put("core", standalonePattern.group("core"));
Review Comment:
Gotcha. Moved them in loops so we can remove all that hard coded `.put()`
##
solr/core/src/java/org/apache/solr/response/PrometheusResponseWriter.java:
##
@@ -113,8 +114,10 @@ public static void toPrometheus(
Metric dropwizardMetric = dropwizardMetrics.get(metricName);
formatter.exportDropwizardMetric(dropwizardMetric, metricName);
} catch (Exception e) {
-// Do not fail entirely for metrics exporting. Log and try to
export next metric
-log.warn("Error occurred exporting Dropwizard Metric to
Prometheus", e);
+throw new SolrException(
+SolrException.ErrorCode.SERVER_ERROR,
+"Error occurred exporting Dropwizard Metric to Prometheus",
+e);
Review Comment:
Also wanted to mention this. I was thinking to just remove the `log.warn`
and actually throw a SolrException. I think at the time I thought that not
failing metrics entirely and even just partially getting metrics was ok. But
after some thought, adding this would actually fail any metrics from posting
but helps exporting tests actually get caught if there is something wrong or
even a user finding a bug. WDYT?
##
solr/core/src/java/org/apache/solr/metrics/prometheus/core/SolrCoreMetric.java:
##
@@ -30,18 +30,18 @@ public abstract class SolrCoreMetric extends SolrMetric {
public String coreName;
public SolrCoreMetric(Metric dropwizardMetric, String metricName) {
-super(dropwizardMetric, metricName);
+super(dropwizardMetric, metricName.substring(metricName.indexOf(".") + 1));
Matcher cloudPattern = CLOUD_CORE_PATTERN.matcher(metricName);
Matcher standalonePattern = STANDALONE_CORE_PATTERN.matcher(metricName);
Review Comment:
Yeah good point. I feel like we discussed this on the PR when I introduced
this feature... Or maybe not. Regardless, it should probably be in
SolrCoreMetric, so I moved it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[PR] SOLR-17609: Remove HDFS module [solr]
epugh opened a new pull request, #2923: URL: https://github.com/apache/solr/pull/2923 https://issues.apache.org/jira/browse/SOLR-17609 # Description Explore what removing HDFS module from Solr impacts. # Solution Searching for references for `hdfs`, `hadoop`, and other keywords. Running tests. # Tests Only existing tests. # Checklist Please review the following and check all that apply: - [ ] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [ ] I have created a Jira issue and added the issue ID to my pull request title. - [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation) - [ ] I have developed this patch against the `main` branch. - [ ] I have run `./gradlew check`. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17609) Remove hdfs module
[ https://issues.apache.org/jira/browse/SOLR-17609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SOLR-17609: -- Labels: pull-request-available (was: ) > Remove hdfs module > -- > > Key: SOLR-17609 > URL: https://issues.apache.org/jira/browse/SOLR-17609 > Project: Solr > Issue Type: Task > Components: hdfs >Affects Versions: main (10.0) >Reporter: Eric Pugh >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > One of the outcomes of the 2024 Community Survey is that we learned (in our > admittedly fairly unscientific) responses that hdfs module is not used. > This PR is to understand the impact of removing hdfs in Solr 10. > See [https://lists.apache.org/thread/hp6bov79rgrg0gb2ozzbzxxn30k2js0h] for > discussion on Dev. > > I won't merge this PR till we have more consensus. > This builds on work started in > https://issues.apache.org/jira/browse/SOLR-14660 and > https://issues.apache.org/jira/browse/SOLR-14021 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
