Cloudsolrclient.getclusterstateprovider - returns incorrect base_url [http instead of https] - Urgent pls help
Hi Team, We have enabled ssl. Solr version 8.8.2, zk - 3.6.3 When we try to get the base url using cloudsolrclient.getclusterstateprovider, it returns http url instead of HTTPS. Please find the screenshots while I debug, Code used From solr admin server - using /solr/admin/collection?action=CLUSTERSTATUS&wt=json return https as base url Kindly help to fix this. Thanks
Re: Cloudsolrclient.getclusterstateprovider - returns incorrect base_url [http instead of https] - Urgent pls help
your screenshots are unreadable, couldn't you copy and paste the text from the screen? On Wed, Jul 21, 2021 at 9:18 AM Reej M wrote: > Hi Team, > > We have enabled ssl. Solr version 8.8.2, zk - 3.6.3 > When we try to get the base url using > cloudsolrclient.getclusterstateprovider, it returns http url instead of > HTTPS. > Please find the screenshots while I debug, > Code used > > From solr admin server - using > /solr/admin/collection?action=CLUSTERSTATUS&wt=json return https as base url > > Kindly help to fix this. Thanks > -- Vincenzo D'Amore
Re: Cloudsolrclient.getclusterstateprovider - returns incorrect base_url [http instead of https] - Urgent pls help
Hi Sorry,
My offic system doesn't have internet for security reasons.
The issue is we have enabled ssl in solr and we try to get the clusterstate
as below
cloudsolrclient.getclusterstateprovider.getclusterstate()
then using the clusterstate doccollection
DocCollection dc = clusterstate.getCollection(collectionName)
for(Slice slice:dc.getSlices()){
for(Replica replica:slice.getReplicas()){
replica.get("base_url")
}
}
replica.get("base_url") -- returns a http url but what we have configured
is a https url
solr admin page we are able to access using https url and can see the
clusterstatus with base_url having https node url
Not sure how through java it is different
*Thanks,*
*Reej*
On Wed, Jul 21, 2021 at 4:19 PM Vincenzo D'Amore wrote:
> your screenshots are unreadable, couldn't you copy and paste the text from
> the screen?
>
> On Wed, Jul 21, 2021 at 9:18 AM Reej M wrote:
>
>> Hi Team,
>>
>> We have enabled ssl. Solr version 8.8.2, zk - 3.6.3
>> When we try to get the base url using
>> cloudsolrclient.getclusterstateprovider, it returns http url instead of
>> HTTPS.
>> Please find the screenshots while I debug,
>> Code used
>>
>> From solr admin server - using
>> /solr/admin/collection?action=CLUSTERSTATUS&wt=json return https as base url
>>
>> Kindly help to fix this. Thanks
>>
>
>
> --
> Vincenzo D'Amore
>
>
Re: Cloudsolrclient.getclusterstateprovider - returns incorrect base_url [http instead of https] - Urgent pls help
Sorry for being so rude, I was in a hurry and just a while after the mail
was sent I thought you weren't in the position to take a screenshot.
On Wed, Jul 21, 2021 at 10:41 AM Reej Nayagam wrote:
> Hi Sorry,
> My offic system doesn't have internet for security reasons.
> The issue is we have enabled ssl in solr and we try to get the clusterstate
> as below
> cloudsolrclient.getclusterstateprovider.getclusterstate()
> then using the clusterstate doccollection
> DocCollection dc = clusterstate.getCollection(collectionName)
> for(Slice slice:dc.getSlices()){
> for(Replica replica:slice.getReplicas()){
> replica.get("base_url")
> }
> }
>
> replica.get("base_url") -- returns a http url but what we have configured
> is a https url
> solr admin page we are able to access using https url and can see the
> clusterstatus with base_url having https node url
> Not sure how through java it is different
>
>
> *Thanks,*
> *Reej*
>
>
> On Wed, Jul 21, 2021 at 4:19 PM Vincenzo D'Amore
> wrote:
>
> > your screenshots are unreadable, couldn't you copy and paste the text
> from
> > the screen?
> >
> > On Wed, Jul 21, 2021 at 9:18 AM Reej M wrote:
> >
> >> Hi Team,
> >>
> >> We have enabled ssl. Solr version 8.8.2, zk - 3.6.3
> >> When we try to get the base url using
> >> cloudsolrclient.getclusterstateprovider, it returns http url instead of
> >> HTTPS.
> >> Please find the screenshots while I debug,
> >> Code used
> >>
> >> From solr admin server - using
> >> /solr/admin/collection?action=CLUSTERSTATUS&wt=json return https as
> base url
> >>
> >> Kindly help to fix this. Thanks
> >>
> >
> >
> > --
> > Vincenzo D'Amore
> >
> >
>
--
Vincenzo D'Amore
Re: Cloudsolrclient.getclusterstateprovider - returns incorrect base_url [http instead of https] - Urgent pls help
Have you double checked how ssl has been configured?
I think this doc could help
https://solr.apache.org/guide/6_6/enabling-ssl.html#EnablingSSL-SolrCloud
And double check the "clusterprop" urlScheme
On Wed, Jul 21, 2021 at 11:01 AM Vincenzo D'Amore
wrote:
> Sorry for being so rude, I was in a hurry and just a while after the mail
> was sent I thought you weren't in the position to take a screenshot.
>
>
> On Wed, Jul 21, 2021 at 10:41 AM Reej Nayagam wrote:
>
>> Hi Sorry,
>> My offic system doesn't have internet for security reasons.
>> The issue is we have enabled ssl in solr and we try to get the
>> clusterstate
>> as below
>> cloudsolrclient.getclusterstateprovider.getclusterstate()
>> then using the clusterstate doccollection
>> DocCollection dc = clusterstate.getCollection(collectionName)
>> for(Slice slice:dc.getSlices()){
>> for(Replica replica:slice.getReplicas()){
>> replica.get("base_url")
>> }
>> }
>>
>> replica.get("base_url") -- returns a http url but what we have configured
>> is a https url
>> solr admin page we are able to access using https url and can see the
>> clusterstatus with base_url having https node url
>> Not sure how through java it is different
>>
>>
>> *Thanks,*
>> *Reej*
>>
>>
>> On Wed, Jul 21, 2021 at 4:19 PM Vincenzo D'Amore
>> wrote:
>>
>> > your screenshots are unreadable, couldn't you copy and paste the text
>> from
>> > the screen?
>> >
>> > On Wed, Jul 21, 2021 at 9:18 AM Reej M wrote:
>> >
>> >> Hi Team,
>> >>
>> >> We have enabled ssl. Solr version 8.8.2, zk - 3.6.3
>> >> When we try to get the base url using
>> >> cloudsolrclient.getclusterstateprovider, it returns http url instead of
>> >> HTTPS.
>> >> Please find the screenshots while I debug,
>> >> Code used
>> >>
>> >> From solr admin server - using
>> >> /solr/admin/collection?action=CLUSTERSTATUS&wt=json return https as
>> base url
>> >>
>> >> Kindly help to fix this. Thanks
>> >>
>> >
>> >
>> > --
>> > Vincenzo D'Amore
>> >
>> >
>>
>
>
> --
> Vincenzo D'Amore
>
>
--
Vincenzo D'Amore
Re: Cloudsolrclient.getclusterstateprovider - returns incorrect base_url [http instead of https] - Urgent pls help
this is your version,
https://solr.apache.org/guide/8_8/enabling-ssl.html#EnablingSSL-SolrCloud
anyway, pay attention to clusterprop
https://solr.apache.org/guide/8_8/enabling-ssl.html#update-cluster-properties-for-existing-collections
On Wed, Jul 21, 2021 at 11:04 AM Vincenzo D'Amore
wrote:
> Have you double checked how ssl has been configured?
> I think this doc could help
> https://solr.apache.org/guide/6_6/enabling-ssl.html#EnablingSSL-SolrCloud
>
> And double check the "clusterprop" urlScheme
>
> On Wed, Jul 21, 2021 at 11:01 AM Vincenzo D'Amore
> wrote:
>
>> Sorry for being so rude, I was in a hurry and just a while after the mail
>> was sent I thought you weren't in the position to take a screenshot.
>>
>>
>> On Wed, Jul 21, 2021 at 10:41 AM Reej Nayagam wrote:
>>
>>> Hi Sorry,
>>> My offic system doesn't have internet for security reasons.
>>> The issue is we have enabled ssl in solr and we try to get the
>>> clusterstate
>>> as below
>>> cloudsolrclient.getclusterstateprovider.getclusterstate()
>>> then using the clusterstate doccollection
>>> DocCollection dc = clusterstate.getCollection(collectionName)
>>> for(Slice slice:dc.getSlices()){
>>> for(Replica replica:slice.getReplicas()){
>>> replica.get("base_url")
>>> }
>>> }
>>>
>>> replica.get("base_url") -- returns a http url but what we have configured
>>> is a https url
>>> solr admin page we are able to access using https url and can see the
>>> clusterstatus with base_url having https node url
>>> Not sure how through java it is different
>>>
>>>
>>> *Thanks,*
>>> *Reej*
>>>
>>>
>>> On Wed, Jul 21, 2021 at 4:19 PM Vincenzo D'Amore
>>> wrote:
>>>
>>> > your screenshots are unreadable, couldn't you copy and paste the text
>>> from
>>> > the screen?
>>> >
>>> > On Wed, Jul 21, 2021 at 9:18 AM Reej M wrote:
>>> >
>>> >> Hi Team,
>>> >>
>>> >> We have enabled ssl. Solr version 8.8.2, zk - 3.6.3
>>> >> When we try to get the base url using
>>> >> cloudsolrclient.getclusterstateprovider, it returns http url instead
>>> of
>>> >> HTTPS.
>>> >> Please find the screenshots while I debug,
>>> >> Code used
>>> >>
>>> >> From solr admin server - using
>>> >> /solr/admin/collection?action=CLUSTERSTATUS&wt=json return https as
>>> base url
>>> >>
>>> >> Kindly help to fix this. Thanks
>>> >>
>>> >
>>> >
>>> > --
>>> > Vincenzo D'Amore
>>> >
>>> >
>>>
>>
>>
>> --
>> Vincenzo D'Amore
>>
>>
>
> --
> Vincenzo D'Amore
>
>
--
Vincenzo D'Amore
Re: Cloudsolrclient.getclusterstateprovider - returns incorrect base_url [http instead of https] - Urgent pls help
> On 21 Jul 2021, at 5:07 PM, Vincenzo D'Amore wrote:
> Hi,
Is ok sometime all of us just loose our cool.
By the way we have followed the same steps as per the documentation only.
Im trying to clear the zk data, clear everything and recheck again if that
might help. Thanks
> this is your version,
> https://solr.apache.org/guide/8_8/enabling-ssl.html#EnablingSSL-SolrCloud
> anyway, pay attention to clusterprop
> https://solr.apache.org/guide/8_8/enabling-ssl.html#update-cluster-properties-for-existing-collections
>
> On Wed, Jul 21, 2021 at 11:04 AM Vincenzo D'Amore
> wrote:
>
>> Have you double checked how ssl has been configured?
>> I think this doc could help
>> https://solr.apache.org/guide/6_6/enabling-ssl.html#EnablingSSL-SolrCloud
>>
>> And double check the "clusterprop" urlScheme
>>
>> On Wed, Jul 21, 2021 at 11:01 AM Vincenzo D'Amore
>> wrote:
>>
>>> Sorry for being so rude, I was in a hurry and just a while after the mail
>>> was sent I thought you weren't in the position to take a screenshot.
>>>
>>>
>>> On Wed, Jul 21, 2021 at 10:41 AM Reej Nayagam wrote:
>>>
Hi Sorry,
My offic system doesn't have internet for security reasons.
The issue is we have enabled ssl in solr and we try to get the
clusterstate
as below
cloudsolrclient.getclusterstateprovider.getclusterstate()
then using the clusterstate doccollection
DocCollection dc = clusterstate.getCollection(collectionName)
for(Slice slice:dc.getSlices()){
for(Replica replica:slice.getReplicas()){
replica.get("base_url")
}
}
replica.get("base_url") -- returns a http url but what we have configured
is a https url
solr admin page we are able to access using https url and can see the
clusterstatus with base_url having https node url
Not sure how through java it is different
*Thanks,*
*Reej*
On Wed, Jul 21, 2021 at 4:19 PM Vincenzo D'Amore
wrote:
> your screenshots are unreadable, couldn't you copy and paste the text
from
> the screen?
>
> On Wed, Jul 21, 2021 at 9:18 AM Reej M wrote:
>
>> Hi Team,
>>
>> We have enabled ssl. Solr version 8.8.2, zk - 3.6.3
>> When we try to get the base url using
>> cloudsolrclient.getclusterstateprovider, it returns http url instead
of
>> HTTPS.
>> Please find the screenshots while I debug,
>> Code used
>>
>> From solr admin server - using
>> /solr/admin/collection?action=CLUSTERSTATUS&wt=json return https as
base url
>>
>> Kindly help to fix this. Thanks
>>
>
>
> --
> Vincenzo D'Amore
>
>
>>>
>>>
>>> --
>>> Vincenzo D'Amore
>>>
>>>
>>
>> --
>> Vincenzo D'Amore
>>
>>
>
> --
> Vincenzo D'Amore
Solr stop doesn't cope with zombie process - should it?
Hello,
When calling solr stop on linux, this command is used
*CHECK_PID=`ps auxww | awk '{print $2}' | grep -w $SOLR_PID | sort -r | tr
-d ' '`*
https://github.com/apache/solr/blob/122c88a0748769432ef62cc3fb94c2226dd67aa7/solr/bin/solr#L871
If Solr has stopped but remains as a zombie process then its process entry
will remain in the table, so *ps auxww* will continue to show the PID even
after kill -9. So that results in something like this, with 3 minutes
wasted waiting for a dead process to exit.
*[2021-07-21T09:15:12.365Z] Sending stop command to Solr running on port
8983 ... waiting up to 180 seconds to allow Jetty process 12622 to stop
gracefully.[2021-07-21T09:18:13.551Z] [|] Solr process 12622 is still
running; jstacking it now.[2021-07-21T09:18:21.806Z] 12622: Unable to open
socket file /proc/12622/root/tmp/.java_pid12622: target process 12622
doesn't respond within 10500ms or HotSpot VM not
loaded[2021-07-21T09:18:21.806Z] Solr process 12622 is still running;
forcefully killing it now.[2021-07-21T09:18:21.806Z] Killed process
12622[2021-07-21T09:18:31.678Z] ERROR: Failed to kill previous Solr Java
process 12622 ... script fails.*
But the output of ps auxww does identify Zombie processes under STAT:
*USER PID %CPU %MEMVSZ RSS TTY STAT START TIME COMMAND*
*root 12622 1.4 0.0 0 0 pts/1 Z
10:42 0:26 [java]*
So the CHECK_PID could filter out Zombies.
Obviously the bigger issue is why the process has ended up as a Zombie (in
this case it was because of
https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/
and not specifying "--init" when running Solr inside a docker container) so
maybe a message warning that the process is a zombie is worth having, so
that the user has an opportunity to do something about it.
I guess I will raise a JIRA issue with a patch to do that unless there's
some alternative suggestions?
Regards,
Colvin
Solr stop doesn't cope with zombie process - should it?
Hello,
When calling solr stop on linux, this command is used
*CHECK_PID=`ps auxww | awk '{print $2}' | grep -w $SOLR_PID | sort -r | tr
-d ' '`*
https://github.com/apache/solr/blob/122c88a0748769432ef62cc3fb94c2226dd67aa7/solr/bin/solr#L871
If Solr has stopped but remains as a zombie process then its process entry
will remain in the table, so *ps auxww* will continue to show the PID even
after kill -9. So that results in something like this, with 3 minutes
wasted waiting for a dead process to exit.
*[2021-07-21T09:15:12.365Z] Sending stop command to Solr running on port
8983 ... waiting up to 180 seconds to allow Jetty process 12622 to stop
gracefully.[2021-07-21T09:18:13.551Z] [|] Solr process 12622 is still
running; jstacking it now.[2021-07-21T09:18:21.806Z] 12622: Unable to open
socket file /proc/12622/root/tmp/.java_pid12622: target process 12622
doesn't respond within 10500ms or HotSpot VM not
loaded[2021-07-21T09:18:21.806Z] Solr process 12622 is still running;
forcefully killing it now.[2021-07-21T09:18:21.806Z] Killed process
12622[2021-07-21T09:18:31.678Z] ERROR: Failed to kill previous Solr Java
process 12622 ... script fails.*
But the output of ps auxww does identify Zombie processes under STAT:
*USER PID %CPU %MEMVSZ RSS TTY STAT START TIME COMMAND*
*root 12622 1.4 0.0 0 0 pts/1 Z
10:42 0:26 [java]*
So the CHECK_PID could filter out Zombies.
Obviously the bigger issue is why the process has ended up as a Zombie (in
this case it was because of
https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/
and not specifying "--init" when running Solr inside a docker container) so
maybe a message warning that the process is a zombie is worth having, so
that the user has an opportunity to do something about it.
I guess I will raise a JIRA issue with a patch to do that unless there's
some alternative suggestions?
Regards,
Colvin
Error in Fq parsing while using multiple values in solr 8.7
Hi, We have a filter query in our system "fq=negativeattribute:(citychennai%20citydelhi)", in solr 6.5 it was working fine. solr 6.5 parsed query as negativeattribute:citychennai negativeattribute:citydelhi After upgrading the solr to 8.7, this query broke. It is not working as before solr 8.7 parsed query as "parsed_filter_queries": [ "negativeattribute:citychennai citydelhi", } Schema of negattribute field Further this is working fine for the fields which have only string type mapping. Then the fq field is being applied to every value in parenthesis. --
Result set order when searching on "*" (asterisk character)
Hi everyone, When I search on "*" (asterisk character) what's the result sort order based on? Thanks Steven
Re: Result set order when searching on "*" (asterisk character)
if no sort options are configure, just try to add the score field you'll see all the documents (are ordered by score), which usually when there are no clause is 1. On Wed, Jul 21, 2021 at 4:36 PM Steven White wrote: > Hi everyone, > > When I search on "*" (asterisk character) what's the result sort order > based on? > > Thanks > > Steven > -- Vincenzo D'Amore
Re: Solr stop doesn't cope with zombie process - should it?
That seems like a reasonable check to add, the only caution I would advise
is that a lot of developers use macs for local testing so make sure that
whatever flags you invoke are generally cross platform compatible, or
hidden behind appropriate conditions.
On Wed, Jul 21, 2021 at 5:59 AM Colvin Cowie
wrote:
> Hello,
>
> When calling solr stop on linux, this command is used
> *CHECK_PID=`ps auxww | awk '{print $2}' | grep -w $SOLR_PID | sort -r | tr
> -d ' '`*
>
> https://github.com/apache/solr/blob/122c88a0748769432ef62cc3fb94c2226dd67aa7/solr/bin/solr#L871
>
> If Solr has stopped but remains as a zombie process then its process entry
> will remain in the table, so *ps auxww* will continue to show the PID even
> after kill -9. So that results in something like this, with 3 minutes
> wasted waiting for a dead process to exit.
>
>
>
>
>
>
> *[2021-07-21T09:15:12.365Z] Sending stop command to Solr running on port
> 8983 ... waiting up to 180 seconds to allow Jetty process 12622 to stop
> gracefully.[2021-07-21T09:18:13.551Z] [|] Solr process 12622 is still
> running; jstacking it now.[2021-07-21T09:18:21.806Z] 12622: Unable to open
> socket file /proc/12622/root/tmp/.java_pid12622: target process 12622
> doesn't respond within 10500ms or HotSpot VM not
> loaded[2021-07-21T09:18:21.806Z] Solr process 12622 is still running;
> forcefully killing it now.[2021-07-21T09:18:21.806Z] Killed process
> 12622[2021-07-21T09:18:31.678Z] ERROR: Failed to kill previous Solr Java
> process 12622 ... script fails.*
>
> But the output of ps auxww does identify Zombie processes under STAT:
> *USER PID %CPU %MEMVSZ RSS TTY STAT START TIME COMMAND*
> *root 12622 1.4 0.0 0 0 pts/1 Z
> 10:42 0:26 [java]*
>
> So the CHECK_PID could filter out Zombies.
> Obviously the bigger issue is why the process has ended up as a Zombie (in
> this case it was because of
>
> https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/
> and not specifying "--init" when running Solr inside a docker container) so
> maybe a message warning that the process is a zombie is worth having, so
> that the user has an opportunity to do something about it.
>
> I guess I will raise a JIRA issue with a patch to do that unless there's
> some alternative suggestions?
>
> Regards,
> Colvin
>
Re: Error in Fq parsing while using multiple values in solr 8.7
On 7/20/2021 11:37 PM, Satya Nand wrote: We have a filter query in our system "fq=negativeattribute:(citychennai%20citydelhi)", in solr 6.5 it was working fine. solr 6.5 parsed query as name="parsed_filter_queries"> negativeattribute:citychennai negativeattribute:citydelhi After upgrading the solr to 8.7, this query broke. It is not working as before solr 8.7 parsed query as "parsed_filter_queries": [ "negativeattribute:citychennai citydelhi", } Schema of negattribute field The "sow" query parameter (split on whitespace) now defaults to false. This is intentional. Your analysis chain doesn't split the input into tokens, so the value is accepted as-is -- with the space. It is expected that the query analysis definition will do the splitting now, not the query parser. You can add "sow=true" to the query parameters, either on the request or in the handler definition, and regain the behavior you're expecting. But if you actually do intend to have this field be an exact match of all characters including space, that's probably not the best idea. If you change the fq to the following, it would also work: fq=negativeattribute:(citychennai OR citydelhi) Thanks, Shawn
Re: Cloudsolrclient.getclusterstateprovider - returns incorrect base_url [http instead of https] - Urgent pls help
Hi All,
I still face the same issue. Anyone had this issue before?
Im making client connection as below,
CloudSolrClinet client = new CloudSolrClient.Builder("solrURL").build();
clusterstate = client.getClusterstateProvider().getClusterState();
when I check the replicas inside the cluster state the baseurl is http
instead of HTTPS
but when i hit the url in browser
/solr/admin/collections?action=CLUSTERSTATUS, I can see the base_url as
https
Im totally confused on whats wrong. Please help. Thanks
*Thanks,*
*Reej*
On Wed, Jul 21, 2021 at 5:16 PM Reej M wrote:
>
>
> > On 21 Jul 2021, at 5:07 PM, Vincenzo D'Amore wrote:
> > Hi,
> Is ok sometime all of us just loose our cool.
> By the way we have followed the same steps as per the documentation only.
> Im trying to clear the zk data, clear everything and recheck again if that
> might help. Thanks
>
> > this is your version,
> >
> https://solr.apache.org/guide/8_8/enabling-ssl.html#EnablingSSL-SolrCloud
> > anyway, pay attention to clusterprop
> >
> https://solr.apache.org/guide/8_8/enabling-ssl.html#update-cluster-properties-for-existing-collections
> >
> > On Wed, Jul 21, 2021 at 11:04 AM Vincenzo D'Amore
> > wrote:
> >
> >> Have you double checked how ssl has been configured?
> >> I think this doc could help
> >>
> https://solr.apache.org/guide/6_6/enabling-ssl.html#EnablingSSL-SolrCloud
> >>
> >> And double check the "clusterprop" urlScheme
> >>
> >> On Wed, Jul 21, 2021 at 11:01 AM Vincenzo D'Amore
> >> wrote:
> >>
> >>> Sorry for being so rude, I was in a hurry and just a while after the
> mail
> >>> was sent I thought you weren't in the position to take a screenshot.
> >>>
> >>>
> >>> On Wed, Jul 21, 2021 at 10:41 AM Reej Nayagam
> wrote:
> >>>
> Hi Sorry,
> My offic system doesn't have internet for security reasons.
> The issue is we have enabled ssl in solr and we try to get the
> clusterstate
> as below
> cloudsolrclient.getclusterstateprovider.getclusterstate()
> then using the clusterstate doccollection
> DocCollection dc = clusterstate.getCollection(collectionName)
> for(Slice slice:dc.getSlices()){
> for(Replica replica:slice.getReplicas()){
> replica.get("base_url")
> }
> }
>
> replica.get("base_url") -- returns a http url but what we have
> configured
> is a https url
> solr admin page we are able to access using https url and can see the
> clusterstatus with base_url having https node url
> Not sure how through java it is different
>
>
> *Thanks,*
> *Reej*
>
>
> On Wed, Jul 21, 2021 at 4:19 PM Vincenzo D'Amore
> wrote:
>
> > your screenshots are unreadable, couldn't you copy and paste the text
> from
> > the screen?
> >
> > On Wed, Jul 21, 2021 at 9:18 AM Reej M wrote:
> >
> >> Hi Team,
> >>
> >> We have enabled ssl. Solr version 8.8.2, zk - 3.6.3
> >> When we try to get the base url using
> >> cloudsolrclient.getclusterstateprovider, it returns http url instead
> of
> >> HTTPS.
> >> Please find the screenshots while I debug,
> >> Code used
> >>
> >> From solr admin server - using
> >> /solr/admin/collection?action=CLUSTERSTATUS&wt=json return https as
> base url
> >>
> >> Kindly help to fix this. Thanks
> >>
> >
> >
> > --
> > Vincenzo D'Amore
> >
> >
>
> >>>
> >>>
> >>> --
> >>> Vincenzo D'Amore
> >>>
> >>>
> >>
> >> --
> >> Vincenzo D'Amore
> >>
> >>
> >
> > --
> > Vincenzo D'Amore
>
>
RE: Solr nodes crashing
Thanks for the help Shawn and Walter. After increasing the open files setting to 128000 and increasing the JVM-Memory to 16 GB, I was able to load my documents. I now have a collection with 2.3 T rows / ~480 GB running on a 4-node cluster. I have found that complicated queries (searching for two search terms in a field with "AND" for example), often timeout. If I try multiple times the query does eventually complete. I'm assuming this is a caching / warm-up issue. Is there a configuration option I can use to cache the indexes for one of the columns or increase the timeout? Any other advice to get this performing quicker is appreciated. Thanks again, Jon -Original Message- From: Shawn Heisey Sent: Thursday, July 1, 2021 6:48 PM To: users@solr.apache.org Subject: Re: Solr nodes crashing On 7/1/2021 4:23 PM, Jon Morisi wrote: > I've had an indexing job running for 24+ hours. I'm importing 100m+ > documents. After about 8 hours both of the replica nodes crashed but the > primary nodes have continued to run and index. There's a common misconception. Java programs, including Solr, almost never crash. If you've started a recent Solr version on a platform other than Windows, then Solr is started with a Java option that runs a script whenever an OutOfMemoryError exception is thrown by the program. What that script does is simple -- it logs a line to a logfile and then kills Solr with the -9 (kill) signal. Note that there are a number of resource depletion scenarios, other than memory, which can result in an OutOfMemoryError. That's why you were asked about open file and process limits. Most operating systems also have what has been named the "oom killer". When system memory becomes extremely tight, the OS will find programs using a lot of memory and kill one of them. These two things will LOOK like a crash, but they're not really crashes. > JVM-Memory 50.7% > 981.38 MB > 981.38 MB > 497 This indicates that your max heap setting for Solr is in the ballpark of 1GB. This is extremely small, and so you're probably throwing OutOfMemoryError because of heap space. Which, on a non-Windows system, will basically cause Solr to commit suicide. It does this because when OOME is thrown, program operation becomes completely unpredictable, and index corruption is a very real possibility. There are precisely two ways to deal with OOME. One is to increase the size of the resource that is being depleted. The other is to change the program or the program configuration so that it doesn't require as much of that resource. Often, especially with Solr, the second option is simply not possible. Most likely you're going to need to increase Solr's heap far beyond 1GB. There's no way for us to come up with a recommendation for you without asking you a lot of very detailed questions about your setup ... and even with that, it's possible that we would give you an incorrect recommendation. I'll give you a number, and warn you that it could be wrong, either way too small or way too large. Try an 8GB heap. You have lots of memory in this system, 8GB is barely a drop in the bucket. Thanks, Shawn
Re: Solr nodes crashing
You may want to look into enabling docvalues for your fields in your scheme, if not already enabled. That often helps with memory usage during query, but requires a reindex of your data. There are also first searches and new searches queries you can configure in your Solr config, those would be able to warm your caches for you if that is the case. Mike On Wed, Jul 21, 2021 at 11:06 AM Jon Morisi wrote: > Thanks for the help Shawn and Walter. After increasing the open files > setting to 128000 and increasing the JVM-Memory to 16 GB, I was able to > load my documents. > > I now have a collection with 2.3 T rows / ~480 GB running on a 4-node > cluster. I have found that complicated queries (searching for two search > terms in a field with "AND" for example), often timeout. If I try multiple > times the query does eventually complete. I'm assuming this is a caching / > warm-up issue. > > Is there a configuration option I can use to cache the indexes for one of > the columns or increase the timeout? Any other advice to get this > performing quicker is appreciated. > > Thanks again, > Jon > > -Original Message- > From: Shawn Heisey > Sent: Thursday, July 1, 2021 6:48 PM > To: users@solr.apache.org > Subject: Re: Solr nodes crashing > > On 7/1/2021 4:23 PM, Jon Morisi wrote: > > I've had an indexing job running for 24+ hours. I'm importing 100m+ > documents. After about 8 hours both of the replica nodes crashed but the > primary nodes have continued to run and index. > > There's a common misconception. Java programs, including Solr, almost > never crash. > > If you've started a recent Solr version on a platform other than Windows, > then Solr is started with a Java option that runs a script whenever an > OutOfMemoryError exception is thrown by the program. What that script does > is simple -- it logs a line to a logfile and then kills Solr with the -9 > (kill) signal. Note that there are a number of resource depletion > scenarios, other than memory, which can result in an OutOfMemoryError. > That's why you were asked about open file and process limits. > > Most operating systems also have what has been named the "oom killer". > When system memory becomes extremely tight, the OS will find programs > using a lot of memory and kill one of them. > > These two things will LOOK like a crash, but they're not really crashes. > > > JVM-Memory 50.7% > > 981.38 MB > > 981.38 MB > > 497 > > This indicates that your max heap setting for Solr is in the ballpark of > 1GB. This is extremely small, and so you're probably throwing > OutOfMemoryError because of heap space. Which, on a non-Windows system, > will basically cause Solr to commit suicide. It does this because when > OOME is thrown, program operation becomes completely unpredictable, and > index corruption is a very real possibility. > > There are precisely two ways to deal with OOME. One is to increase the > size of the resource that is being depleted. The other is to change the > program or the program configuration so that it doesn't require as much of > that resource. Often, especially with Solr, the second option is simply > not possible. > > Most likely you're going to need to increase Solr's heap far beyond 1GB. > There's no way for us to come up with a recommendation for you without > asking you a lot of very detailed questions about your setup ... and even > with that, it's possible that we would give you an incorrect > recommendation. I'll give you a number, and warn you that it could be > wrong, either way too small or way too large. Try an 8GB heap. You have > lots of memory in this system, 8GB is barely a drop in the bucket. > > Thanks, > Shawn >
Re: Cloudsolrclient.getclusterstateprovider - returns incorrect base_url [http instead of https] - Urgent pls help
Does your `solrURL` value you're passing to `CloudSolrClient.Builder`
start with `https`?
On Wed, Jul 21, 2021 at 10:12 AM Reej Nayagam wrote:
>
> Hi All,
>
> I still face the same issue. Anyone had this issue before?
> Im making client connection as below,
> CloudSolrClinet client = new CloudSolrClient.Builder("solrURL").build();
> clusterstate = client.getClusterstateProvider().getClusterState();
> when I check the replicas inside the cluster state the baseurl is http
> instead of HTTPS
> but when i hit the url in browser
> /solr/admin/collections?action=CLUSTERSTATUS, I can see the base_url as
> https
> Im totally confused on whats wrong. Please help. Thanks
>
> *Thanks,*
> *Reej*
>
>
> On Wed, Jul 21, 2021 at 5:16 PM Reej M wrote:
>
> >
> >
> > > On 21 Jul 2021, at 5:07 PM, Vincenzo D'Amore wrote:
> > > Hi,
> > Is ok sometime all of us just loose our cool.
> > By the way we have followed the same steps as per the documentation only.
> > Im trying to clear the zk data, clear everything and recheck again if that
> > might help. Thanks
> >
> > > this is your version,
> > >
> > https://solr.apache.org/guide/8_8/enabling-ssl.html#EnablingSSL-SolrCloud
> > > anyway, pay attention to clusterprop
> > >
> > https://solr.apache.org/guide/8_8/enabling-ssl.html#update-cluster-properties-for-existing-collections
> > >
> > > On Wed, Jul 21, 2021 at 11:04 AM Vincenzo D'Amore
> > > wrote:
> > >
> > >> Have you double checked how ssl has been configured?
> > >> I think this doc could help
> > >>
> > https://solr.apache.org/guide/6_6/enabling-ssl.html#EnablingSSL-SolrCloud
> > >>
> > >> And double check the "clusterprop" urlScheme
> > >>
> > >> On Wed, Jul 21, 2021 at 11:01 AM Vincenzo D'Amore
> > >> wrote:
> > >>
> > >>> Sorry for being so rude, I was in a hurry and just a while after the
> > mail
> > >>> was sent I thought you weren't in the position to take a screenshot.
> > >>>
> > >>>
> > >>> On Wed, Jul 21, 2021 at 10:41 AM Reej Nayagam
> > wrote:
> > >>>
> > Hi Sorry,
> > My offic system doesn't have internet for security reasons.
> > The issue is we have enabled ssl in solr and we try to get the
> > clusterstate
> > as below
> > cloudsolrclient.getclusterstateprovider.getclusterstate()
> > then using the clusterstate doccollection
> > DocCollection dc = clusterstate.getCollection(collectionName)
> > for(Slice slice:dc.getSlices()){
> > for(Replica replica:slice.getReplicas()){
> > replica.get("base_url")
> > }
> > }
> >
> > replica.get("base_url") -- returns a http url but what we have
> > configured
> > is a https url
> > solr admin page we are able to access using https url and can see the
> > clusterstatus with base_url having https node url
> > Not sure how through java it is different
> >
> >
> > *Thanks,*
> > *Reej*
> >
> >
> > On Wed, Jul 21, 2021 at 4:19 PM Vincenzo D'Amore
> > wrote:
> >
> > > your screenshots are unreadable, couldn't you copy and paste the text
> > from
> > > the screen?
> > >
> > > On Wed, Jul 21, 2021 at 9:18 AM Reej M wrote:
> > >
> > >> Hi Team,
> > >>
> > >> We have enabled ssl. Solr version 8.8.2, zk - 3.6.3
> > >> When we try to get the base url using
> > >> cloudsolrclient.getclusterstateprovider, it returns http url instead
> > of
> > >> HTTPS.
> > >> Please find the screenshots while I debug,
> > >> Code used
> > >>
> > >> From solr admin server - using
> > >> /solr/admin/collection?action=CLUSTERSTATUS&wt=json return https as
> > base url
> > >>
> > >> Kindly help to fix this. Thanks
> > >>
> > >
> > >
> > > --
> > > Vincenzo D'Amore
> > >
> > >
> >
> > >>>
> > >>>
> > >>> --
> > >>> Vincenzo D'Amore
> > >>>
> > >>>
> > >>
> > >> --
> > >> Vincenzo D'Amore
> > >>
> > >>
> > >
> > > --
> > > Vincenzo D'Amore
> >
> >
Re: Cloudsolrclient.getclusterstateprovider - returns incorrect base_url [http instead of https] - Urgent pls help
Hi Reej, I'm used to instantiate a new CloudSolrClient with the zookeeper
ensemble. Well, something like this:
final List zkServers = new ArrayList();
zkServers.add("zookeeper1:2181"); zkServers.add("zookeeper2:2181");
zkServers.add("zookeeper3:2181");
final SolrClient client = new CloudSolrClient.Builder(zkServers,
Optional.empty()).build();
On Wed, Jul 21, 2021 at 6:13 PM Reej Nayagam wrote:
> Hi All,
>
> I still face the same issue. Anyone had this issue before?
> Im making client connection as below,
> CloudSolrClinet client = new CloudSolrClient.Builder("solrURL").build();
> clusterstate = client.getClusterstateProvider().getClusterState();
> when I check the replicas inside the cluster state the baseurl is http
> instead of HTTPS
> but when i hit the url in browser
> /solr/admin/collections?action=CLUSTERSTATUS, I can see the base_url as
> https
> Im totally confused on whats wrong. Please help. Thanks
>
> *Thanks,*
> *Reej*
>
>
> On Wed, Jul 21, 2021 at 5:16 PM Reej M wrote:
>
> >
> >
> > > On 21 Jul 2021, at 5:07 PM, Vincenzo D'Amore
> wrote:
> > > Hi,
> > Is ok sometime all of us just loose our cool.
> > By the way we have followed the same steps as per the documentation only.
> > Im trying to clear the zk data, clear everything and recheck again if
> that
> > might help. Thanks
> >
> > > this is your version,
> > >
> >
> https://solr.apache.org/guide/8_8/enabling-ssl.html#EnablingSSL-SolrCloud
> > > anyway, pay attention to clusterprop
> > >
> >
> https://solr.apache.org/guide/8_8/enabling-ssl.html#update-cluster-properties-for-existing-collections
> > >
> > > On Wed, Jul 21, 2021 at 11:04 AM Vincenzo D'Amore
> > > wrote:
> > >
> > >> Have you double checked how ssl has been configured?
> > >> I think this doc could help
> > >>
> >
> https://solr.apache.org/guide/6_6/enabling-ssl.html#EnablingSSL-SolrCloud
> > >>
> > >> And double check the "clusterprop" urlScheme
> > >>
> > >> On Wed, Jul 21, 2021 at 11:01 AM Vincenzo D'Amore >
> > >> wrote:
> > >>
> > >>> Sorry for being so rude, I was in a hurry and just a while after the
> > mail
> > >>> was sent I thought you weren't in the position to take a screenshot.
> > >>>
> > >>>
> > >>> On Wed, Jul 21, 2021 at 10:41 AM Reej Nayagam
> > wrote:
> > >>>
> > Hi Sorry,
> > My offic system doesn't have internet for security reasons.
> > The issue is we have enabled ssl in solr and we try to get the
> > clusterstate
> > as below
> > cloudsolrclient.getclusterstateprovider.getclusterstate()
> > then using the clusterstate doccollection
> > DocCollection dc = clusterstate.getCollection(collectionName)
> > for(Slice slice:dc.getSlices()){
> > for(Replica replica:slice.getReplicas()){
> > replica.get("base_url")
> > }
> > }
> >
> > replica.get("base_url") -- returns a http url but what we have
> > configured
> > is a https url
> > solr admin page we are able to access using https url and can see
> the
> > clusterstatus with base_url having https node url
> > Not sure how through java it is different
> >
> >
> > *Thanks,*
> > *Reej*
> >
> >
> > On Wed, Jul 21, 2021 at 4:19 PM Vincenzo D'Amore <
> v.dam...@gmail.com>
> > wrote:
> >
> > > your screenshots are unreadable, couldn't you copy and paste the
> text
> > from
> > > the screen?
> > >
> > > On Wed, Jul 21, 2021 at 9:18 AM Reej M wrote:
> > >
> > >> Hi Team,
> > >>
> > >> We have enabled ssl. Solr version 8.8.2, zk - 3.6.3
> > >> When we try to get the base url using
> > >> cloudsolrclient.getclusterstateprovider, it returns http url
> instead
> > of
> > >> HTTPS.
> > >> Please find the screenshots while I debug,
> > >> Code used
> > >>
> > >> From solr admin server - using
> > >> /solr/admin/collection?action=CLUSTERSTATUS&wt=json return https
> as
> > base url
> > >>
> > >> Kindly help to fix this. Thanks
> > >>
> > >
> > >
> > > --
> > > Vincenzo D'Amore
> > >
> > >
> >
> > >>>
> > >>>
> > >>> --
> > >>> Vincenzo D'Amore
> > >>>
> > >>>
> > >>
> > >> --
> > >> Vincenzo D'Amore
> > >>
> > >>
> > >
> > > --
> > > Vincenzo D'Amore
> >
> >
>
--
Vincenzo D'Amore
How to disable transaction logs in solr cloud ?
Hello, I want to disable transaction logs in solr cloud to test an experiment. I am not sure how to do it. I tried to comment out the following section in the solrconfig.xml file but that results in the following error when a hard commit takes place through autoCommit. 2021-07-21 22:10:44.771 ERROR (commitScheduler-33-thread-1) [ ] o.a.s.u.CommitTracker auto commit error...:java.lang.NullPointerException at org.apache.solr.update.CommitTracker.run(CommitTracker.java:261) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source)
Re: Cloudsolrclient.getclusterstateprovider - returns incorrect base_url [http instead of https] - Urgent pls help
Hi Reej, another question, why are you using the solrUrl?
I assume you are using a list because the Builder(List solrUrls):
> Provide a series of Solr URLs to be used when configuring CloudSolrClient
instances.
https://solr.apache.org/docs/8_8_0/solr-solrj/org/apache/solr/client/solrj/impl/CloudSolrClient.Builder.html#Builder-java.util.List-
On Wed, Jul 21, 2021 at 11:50 PM Vincenzo D'Amore
wrote:
> Hi Reej, I'm used to instantiate a new CloudSolrClient with the zookeeper
> ensemble. Well, something like this:
>
>final List zkServers = new ArrayList();
>zkServers.add("zookeeper1:2181"); zkServers.add("zookeeper2:2181");
> zkServers.add("zookeeper3:2181");
>final SolrClient client = new CloudSolrClient.Builder(zkServers,
> Optional.empty()).build();
>
>
> On Wed, Jul 21, 2021 at 6:13 PM Reej Nayagam wrote:
>
>> Hi All,
>>
>> I still face the same issue. Anyone had this issue before?
>> Im making client connection as below,
>> CloudSolrClinet client = new CloudSolrClient.Builder("solrURL").build();
>> clusterstate = client.getClusterstateProvider().getClusterState();
>> when I check the replicas inside the cluster state the baseurl is http
>> instead of HTTPS
>> but when i hit the url in browser
>> /solr/admin/collections?action=CLUSTERSTATUS, I can see the base_url as
>> https
>> Im totally confused on whats wrong. Please help. Thanks
>>
>> *Thanks,*
>> *Reej*
>>
>>
>> On Wed, Jul 21, 2021 at 5:16 PM Reej M wrote:
>>
>> >
>> >
>> > > On 21 Jul 2021, at 5:07 PM, Vincenzo D'Amore
>> wrote:
>> > > Hi,
>> > Is ok sometime all of us just loose our cool.
>> > By the way we have followed the same steps as per the documentation
>> only.
>> > Im trying to clear the zk data, clear everything and recheck again if
>> that
>> > might help. Thanks
>> >
>> > > this is your version,
>> > >
>> >
>> https://solr.apache.org/guide/8_8/enabling-ssl.html#EnablingSSL-SolrCloud
>> > > anyway, pay attention to clusterprop
>> > >
>> >
>> https://solr.apache.org/guide/8_8/enabling-ssl.html#update-cluster-properties-for-existing-collections
>> > >
>> > > On Wed, Jul 21, 2021 at 11:04 AM Vincenzo D'Amore > >
>> > > wrote:
>> > >
>> > >> Have you double checked how ssl has been configured?
>> > >> I think this doc could help
>> > >>
>> >
>> https://solr.apache.org/guide/6_6/enabling-ssl.html#EnablingSSL-SolrCloud
>> > >>
>> > >> And double check the "clusterprop" urlScheme
>> > >>
>> > >> On Wed, Jul 21, 2021 at 11:01 AM Vincenzo D'Amore <
>> v.dam...@gmail.com>
>> > >> wrote:
>> > >>
>> > >>> Sorry for being so rude, I was in a hurry and just a while after the
>> > mail
>> > >>> was sent I thought you weren't in the position to take a screenshot.
>> > >>>
>> > >>>
>> > >>> On Wed, Jul 21, 2021 at 10:41 AM Reej Nayagam
>> > wrote:
>> > >>>
>> > Hi Sorry,
>> > My offic system doesn't have internet for security reasons.
>> > The issue is we have enabled ssl in solr and we try to get the
>> > clusterstate
>> > as below
>> > cloudsolrclient.getclusterstateprovider.getclusterstate()
>> > then using the clusterstate doccollection
>> > DocCollection dc = clusterstate.getCollection(collectionName)
>> > for(Slice slice:dc.getSlices()){
>> > for(Replica replica:slice.getReplicas()){
>> > replica.get("base_url")
>> > }
>> > }
>> >
>> > replica.get("base_url") -- returns a http url but what we have
>> > configured
>> > is a https url
>> > solr admin page we are able to access using https url and can see
>> the
>> > clusterstatus with base_url having https node url
>> > Not sure how through java it is different
>> >
>> >
>> > *Thanks,*
>> > *Reej*
>> >
>> >
>> > On Wed, Jul 21, 2021 at 4:19 PM Vincenzo D'Amore <
>> v.dam...@gmail.com>
>> > wrote:
>> >
>> > > your screenshots are unreadable, couldn't you copy and paste the
>> text
>> > from
>> > > the screen?
>> > >
>> > > On Wed, Jul 21, 2021 at 9:18 AM Reej M wrote:
>> > >
>> > >> Hi Team,
>> > >>
>> > >> We have enabled ssl. Solr version 8.8.2, zk - 3.6.3
>> > >> When we try to get the base url using
>> > >> cloudsolrclient.getclusterstateprovider, it returns http url
>> instead
>> > of
>> > >> HTTPS.
>> > >> Please find the screenshots while I debug,
>> > >> Code used
>> > >>
>> > >> From solr admin server - using
>> > >> /solr/admin/collection?action=CLUSTERSTATUS&wt=json return https
>> as
>> > base url
>> > >>
>> > >> Kindly help to fix this. Thanks
>> > >>
>> > >
>> > >
>> > > --
>> > > Vincenzo D'Amore
>> > >
>> > >
>> >
>> > >>>
>> > >>>
>> > >>> --
>> > >>> Vincenzo D'Amore
>> > >>>
>> > >>>
>> > >>
>> > >> --
>> > >> Vincenzo D'Amore
>> > >>
>> > >>
>> > >
>> > > --
>> > > Vincenzo D'Amore
>> >
>> >
>>
>
>
> --
> Vincenzo D'Amore
>
>
--
Vincenzo D'Amore
Re: Cloudsolrclient.getclusterstateprovider - returns incorrect base_url [http instead of https] - Urgent pls help
Yes the passed url is with https
*Thanks,*
*Reej*
On Thu, Jul 22, 2021 at 3:40 AM Timothy Potter wrote:
> Does your `solrURL` value you're passing to `CloudSolrClient.Builder`
> start with `https`?
>
> On Wed, Jul 21, 2021 at 10:12 AM Reej Nayagam wrote:
> >
> > Hi All,
> >
> > I still face the same issue. Anyone had this issue before?
> > Im making client connection as below,
> > CloudSolrClinet client = new CloudSolrClient.Builder("solrURL").build();
> > clusterstate = client.getClusterstateProvider().getClusterState();
> > when I check the replicas inside the cluster state the baseurl is http
> > instead of HTTPS
> > but when i hit the url in browser
> > /solr/admin/collections?action=CLUSTERSTATUS, I can see the base_url as
> > https
> > Im totally confused on whats wrong. Please help. Thanks
> >
> > *Thanks,*
> > *Reej*
> >
> >
> > On Wed, Jul 21, 2021 at 5:16 PM Reej M wrote:
> >
> > >
> > >
> > > > On 21 Jul 2021, at 5:07 PM, Vincenzo D'Amore
> wrote:
> > > > Hi,
> > > Is ok sometime all of us just loose our cool.
> > > By the way we have followed the same steps as per the documentation
> only.
> > > Im trying to clear the zk data, clear everything and recheck again if
> that
> > > might help. Thanks
> > >
> > > > this is your version,
> > > >
> > >
> https://solr.apache.org/guide/8_8/enabling-ssl.html#EnablingSSL-SolrCloud
> > > > anyway, pay attention to clusterprop
> > > >
> > >
> https://solr.apache.org/guide/8_8/enabling-ssl.html#update-cluster-properties-for-existing-collections
> > > >
> > > > On Wed, Jul 21, 2021 at 11:04 AM Vincenzo D'Amore <
> v.dam...@gmail.com>
> > > > wrote:
> > > >
> > > >> Have you double checked how ssl has been configured?
> > > >> I think this doc could help
> > > >>
> > >
> https://solr.apache.org/guide/6_6/enabling-ssl.html#EnablingSSL-SolrCloud
> > > >>
> > > >> And double check the "clusterprop" urlScheme
> > > >>
> > > >> On Wed, Jul 21, 2021 at 11:01 AM Vincenzo D'Amore <
> v.dam...@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> Sorry for being so rude, I was in a hurry and just a while after
> the
> > > mail
> > > >>> was sent I thought you weren't in the position to take a
> screenshot.
> > > >>>
> > > >>>
> > > >>> On Wed, Jul 21, 2021 at 10:41 AM Reej Nayagam
> > > wrote:
> > > >>>
> > > Hi Sorry,
> > > My offic system doesn't have internet for security reasons.
> > > The issue is we have enabled ssl in solr and we try to get the
> > > clusterstate
> > > as below
> > > cloudsolrclient.getclusterstateprovider.getclusterstate()
> > > then using the clusterstate doccollection
> > > DocCollection dc = clusterstate.getCollection(collectionName)
> > > for(Slice slice:dc.getSlices()){
> > > for(Replica replica:slice.getReplicas()){
> > > replica.get("base_url")
> > > }
> > > }
> > >
> > > replica.get("base_url") -- returns a http url but what we have
> > > configured
> > > is a https url
> > > solr admin page we are able to access using https url and can see
> the
> > > clusterstatus with base_url having https node url
> > > Not sure how through java it is different
> > >
> > >
> > > *Thanks,*
> > > *Reej*
> > >
> > >
> > > On Wed, Jul 21, 2021 at 4:19 PM Vincenzo D'Amore <
> v.dam...@gmail.com>
> > > wrote:
> > >
> > > > your screenshots are unreadable, couldn't you copy and paste the
> text
> > > from
> > > > the screen?
> > > >
> > > > On Wed, Jul 21, 2021 at 9:18 AM Reej M
> wrote:
> > > >
> > > >> Hi Team,
> > > >>
> > > >> We have enabled ssl. Solr version 8.8.2, zk - 3.6.3
> > > >> When we try to get the base url using
> > > >> cloudsolrclient.getclusterstateprovider, it returns http url
> instead
> > > of
> > > >> HTTPS.
> > > >> Please find the screenshots while I debug,
> > > >> Code used
> > > >>
> > > >> From solr admin server - using
> > > >> /solr/admin/collection?action=CLUSTERSTATUS&wt=json return
> https as
> > > base url
> > > >>
> > > >> Kindly help to fix this. Thanks
> > > >>
> > > >
> > > >
> > > > --
> > > > Vincenzo D'Amore
> > > >
> > > >
> > >
> > > >>>
> > > >>>
> > > >>> --
> > > >>> Vincenzo D'Amore
> > > >>>
> > > >>>
> > > >>
> > > >> --
> > > >> Vincenzo D'Amore
> > > >>
> > > >>
> > > >
> > > > --
> > > > Vincenzo D'Amore
> > >
> > >
>
Re: Cloudsolrclient.getclusterstateprovider - returns incorrect base_url [http instead of https] - Urgent pls help
I tried earlier with zk ensemble, but when i try to get the
clusterstateprovider.getclusterstate , it throws me
"NO such method error : org.noggit.JSONParser.getFlags()
so I was using the solrurl,
i've added the jars solr-core-8.8.2, solr-solrj-8.8.2, zookeeper-3.6.3 and
zookeeper-jute-3.6.3
Not sure if I need to add any additional jars, google didn’t help.
*Thanks,*
*Reej*
On Thu, Jul 22, 2021 at 5:51 AM Vincenzo D'Amore wrote:
> Hi Reej, I'm used to instantiate a new CloudSolrClient with the zookeeper
> ensemble. Well, something like this:
>
>final List zkServers = new ArrayList();
>zkServers.add("zookeeper1:2181"); zkServers.add("zookeeper2:2181");
> zkServers.add("zookeeper3:2181");
>final SolrClient client = new CloudSolrClient.Builder(zkServers,
> Optional.empty()).build();
>
>
> On Wed, Jul 21, 2021 at 6:13 PM Reej Nayagam wrote:
>
> > Hi All,
> >
> > I still face the same issue. Anyone had this issue before?
> > Im making client connection as below,
> > CloudSolrClinet client = new CloudSolrClient.Builder("solrURL").build();
> > clusterstate = client.getClusterstateProvider().getClusterState();
> > when I check the replicas inside the cluster state the baseurl is http
> > instead of HTTPS
> > but when i hit the url in browser
> > /solr/admin/collections?action=CLUSTERSTATUS, I can see the base_url as
> > https
> > Im totally confused on whats wrong. Please help. Thanks
> >
> > *Thanks,*
> > *Reej*
> >
> >
> > On Wed, Jul 21, 2021 at 5:16 PM Reej M wrote:
> >
> > >
> > >
> > > > On 21 Jul 2021, at 5:07 PM, Vincenzo D'Amore
> > wrote:
> > > > Hi,
> > > Is ok sometime all of us just loose our cool.
> > > By the way we have followed the same steps as per the documentation
> only.
> > > Im trying to clear the zk data, clear everything and recheck again if
> > that
> > > might help. Thanks
> > >
> > > > this is your version,
> > > >
> > >
> >
> https://solr.apache.org/guide/8_8/enabling-ssl.html#EnablingSSL-SolrCloud
> > > > anyway, pay attention to clusterprop
> > > >
> > >
> >
> https://solr.apache.org/guide/8_8/enabling-ssl.html#update-cluster-properties-for-existing-collections
> > > >
> > > > On Wed, Jul 21, 2021 at 11:04 AM Vincenzo D'Amore <
> v.dam...@gmail.com>
> > > > wrote:
> > > >
> > > >> Have you double checked how ssl has been configured?
> > > >> I think this doc could help
> > > >>
> > >
> >
> https://solr.apache.org/guide/6_6/enabling-ssl.html#EnablingSSL-SolrCloud
> > > >>
> > > >> And double check the "clusterprop" urlScheme
> > > >>
> > > >> On Wed, Jul 21, 2021 at 11:01 AM Vincenzo D'Amore <
> v.dam...@gmail.com
> > >
> > > >> wrote:
> > > >>
> > > >>> Sorry for being so rude, I was in a hurry and just a while after
> the
> > > mail
> > > >>> was sent I thought you weren't in the position to take a
> screenshot.
> > > >>>
> > > >>>
> > > >>> On Wed, Jul 21, 2021 at 10:41 AM Reej Nayagam
> > > wrote:
> > > >>>
> > > Hi Sorry,
> > > My offic system doesn't have internet for security reasons.
> > > The issue is we have enabled ssl in solr and we try to get the
> > > clusterstate
> > > as below
> > > cloudsolrclient.getclusterstateprovider.getclusterstate()
> > > then using the clusterstate doccollection
> > > DocCollection dc = clusterstate.getCollection(collectionName)
> > > for(Slice slice:dc.getSlices()){
> > > for(Replica replica:slice.getReplicas()){
> > > replica.get("base_url")
> > > }
> > > }
> > >
> > > replica.get("base_url") -- returns a http url but what we have
> > > configured
> > > is a https url
> > > solr admin page we are able to access using https url and can see
> > the
> > > clusterstatus with base_url having https node url
> > > Not sure how through java it is different
> > >
> > >
> > > *Thanks,*
> > > *Reej*
> > >
> > >
> > > On Wed, Jul 21, 2021 at 4:19 PM Vincenzo D'Amore <
> > v.dam...@gmail.com>
> > > wrote:
> > >
> > > > your screenshots are unreadable, couldn't you copy and paste the
> > text
> > > from
> > > > the screen?
> > > >
> > > > On Wed, Jul 21, 2021 at 9:18 AM Reej M
> wrote:
> > > >
> > > >> Hi Team,
> > > >>
> > > >> We have enabled ssl. Solr version 8.8.2, zk - 3.6.3
> > > >> When we try to get the base url using
> > > >> cloudsolrclient.getclusterstateprovider, it returns http url
> > instead
> > > of
> > > >> HTTPS.
> > > >> Please find the screenshots while I debug,
> > > >> Code used
> > > >>
> > > >> From solr admin server - using
> > > >> /solr/admin/collection?action=CLUSTERSTATUS&wt=json return https
> > as
> > > base url
> > > >>
> > > >> Kindly help to fix this. Thanks
> > > >>
> > > >
> > > >
> > > > --
> > > > Vincenzo D'Amore
> > > >
> > > >
> > >
> > > >>>
> > > >>>
> > > >>> --
> > > >>> Vincenzo D'Amore
> > > >>>
> > > >>>
> > > >>
> > > >> --
> >
Re: How to disable transaction logs in solr cloud ?
On 7/21/2021 4:24 PM, Pratik Patel wrote: I want to disable transaction logs in solr cloud to test an experiment. I am not sure how to do it. What are you trying to accomplish? The transaction log is a required feature for SolrCloud. I know it's used for recovery operations, but based on what you're seeing it seems to be an integral part of normal SolrCloud function. Don't disable it. Thanks, Shawn
Re: Cloudsolrclient.getclusterstateprovider - returns incorrect base_url [http instead of https] - Urgent pls help
On 7/21/2021 6:37 PM, Reej Nayagam wrote: I tried earlier with zk ensemble, but when i try to get the clusterstateprovider.getclusterstate , it throws me "NO such method error : org.noggit.JSONParser.getFlags() so I was using the solrurl, i've added the jars solr-core-8.8.2, solr-solrj-8.8.2, zookeeper-3.6.3 and zookeeper-jute-3.6.3 Not sure if I need to add any additional jars, google didn’t help. The jar dependencies you need for SolrJ are included in the Solr download. You will find them in dist/solrj-lib. Not all of those jars are required for every usage ... but figuring out which ones you don't need can be challenging, so it's better to include them all. You do not need solr-core unless you're trying to embed a complete Solr server into your app (without http access) ... which we strongly recommend NOT doing for nearly everyone. The noggit library should not be needed for SolrJ, but it IS used by solr-core. You're probably getting that message because you have solr-core. Thanks, Shawn
Re: Error in Fq parsing while using multiple values in solr 8.7
Hi Shawn, Thank you, I also had my suspicion on the sow parameter. but I can't figure out why it is acting differently for analyzed type and non-analyzed. for example, If I give this query to solr 8.7 fq=negativeattribute:(citychennai mcat43120 20mcat43120)&debug=query&fq=mcatid:(43120 26527 43015) It parses both queries as you can see for mcatid field it is working like sow is true. "parsed_filter_queries": [ "negativeattribute:citychennai mcat43120 mcat43120", "mcatid:43120 mcatid:26527 mcatid:43015" ] } Schema of negattribute field < filter class="solr.TrimFilterFactory"/> Schema of mcatid field On Wed, Jul 21, 2021 at 8:42 PM Shawn Heisey wrote: > On 7/20/2021 11:37 PM, Satya Nand wrote: > > We have a filter query in our system > > "fq=negativeattribute:(citychennai%20citydelhi)", in solr 6.5 it was > > working fine. solr 6.5 parsed query as > name="parsed_filter_queries"> negativeattribute:citychennai > > negativeattribute:citydelhi After upgrading the solr to > > 8.7, this query broke. It is not working as before solr 8.7 parsed > > query as "parsed_filter_queries": [ "negativeattribute:citychennai > > citydelhi", } Schema of negattribute field > > The "sow" query parameter (split on whitespace) now defaults to false. > This is intentional. Your analysis chain doesn't split the input into > tokens, so the value is accepted as-is -- with the space. > > It is expected that the query analysis definition will do the splitting > now, not the query parser. > > You can add "sow=true" to the query parameters, either on the request or > in the handler definition, and regain the behavior you're expecting. > But if you actually do intend to have this field be an exact match of > all characters including space, that's probably not the best idea. If > you change the fq to the following, it would also work: > > fq=negativeattribute:(citychennai OR citydelhi) > > Thanks, > Shawn > > --
min_popularity alternative for Solr Relatedness and Semantic Knowledge Graphs
Hi Solr users, I have a question on the relatedness and Semantic Knowledge Graphs feature in Solr. While the results are good with the out of box provision, I need some tweaking on the ability to specify filters or parameters based on only the foreground count. Right now only the min_popularity parameter is available which applies to both the foreground dataset or the background one. The white paper from Trey Grainger and his team mention that the z score is used to calculate the score. As per my understanding, the z score assumes a normal distribution and is applicable when sample size>30 which I assume is the foreground count. So I would like to control this value with a parameter or filter. Right now I am getting the approximate count by doing a reverse calculation on the foreground popularity and the background size to get the foreground count. Kindly correct me if my understanding is different from what it should be.
