Re: Delete by Id in solr cloud

[Satya Nand]

Hi Peter,

I have tried using httpsolrclient. but the receiving shard also forwards
the request to some other shard. Is there any way we can achieve this solr
8.7 ?





On Thu, Jun 30, 2022 at 3:41 AM r ohara  wrote:

> Hi Satya,
> I think it's a bug with using compositeId. We had the same issue, and had
> to use deleteByQuery instead, but like you said, it's much slower. We're
> using solr 8.11
>
> On Tue, Jun 28, 2022 at 4:59 AM Satya Nand  .invalid>
> wrote:
>
> > Thanks, Peter,
> > I am checking that, also UpdateRequest class seems to have methods that
> > take routes as input. I will see if it helps.
> >
> > On Tue, Jun 28, 2022 at 3:19 PM Peter Lancaster <
> > peter.lancas...@findmypast.com> wrote:
> >
> > > Hi Satya,
> > >
> > > I think you would need to use a HttpSolrClient that uses the url of the
> > > shard where the record exists.
> > >
> > > Regards,
> > > Peter.
> > >
> > > -Original Message-
> > > From: Satya Nand 
> > > Sent: 28 June 2022 10:43
> > > To: users@solr.apache.org
> > > Subject: Re: Delete by Id in solr cloud
> > >
> > > EXTERNAL SENDER: Do not click any links or open any attachments unless
> > you
> > > trust the sender and know the content is safe.
> > >
> > >
> > > Hi Radu,
> > >
> > > I am using solrj for executing the query. I couldn't find any function
> > > with accepts additional parameters like routing, shards, solr Params
> etc.
> > >
> > > I also tried delete by query instead of deleteById, But it is very
> slow.
> > >
> > >
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsolr.apache.org%2Fdocs%2F8_1_0%2Fsolr-solrj%2Forg%2Fapache%2Fsolr%2Fclient%2Fsolrj%2Fimpl%2FCloudSolrClient.html&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4d7N5LCpx8TXnEv7GW%2BN2TmoE8YvHa0tgr4c%2FamgOBw%3D&reserved=0
> > > deleteById
> > > <
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsolr.apache.org%2Fdocs%2F7_3_1%2Fsolr-solrj%2Forg%2Fapache%2Fsolr%2Fclient%2Fsolrj%2FSolrClient.html%23deleteById-java.lang.String-java.util.List-int-&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MdBKsoMlbTqUjx5xzUny1Hrop0La2cwkg6cVZgZ76Es%3D&reserved=0
> > > >
> > > (String
> > > <
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.oracle.com%2Fjavase%2F8%2Fdocs%2Fapi%2Fjava%2Flang%2FString.html%3Fis-external%3Dtrue&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3gPAaYNOQvAkYD8coSuGjm28gau5i3lEJabT4Kqu%2BCk%3D&reserved=0
> > > >
> > >  collection, List
> > > <
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.oracle.com%2Fjavase%2F8%2Fdocs%2Fapi%2Fjava%2Futil%2FList.html%3Fis-external%3Dtrue&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=bUQ0Fe0pPkP2kFeRy%2BLg%2FuTIBSEM1HVQdk4EEAdQYCQ%3D&reserved=0
> > > >
> > >  > > <
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.oracle.com%2Fjavase%2F8%2Fdocs%2Fapi%2Fjava%2Flang%2FString.html%3Fis-external%3Dtrue&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3gPAaYNOQvAkYD8coSuGjm28gau5i3lEJabT4Kqu%2BCk%3D&reserved=0
> > > >>
> > > ids,
> > > int commitWithinMs)
> > >
> > >
> > >
> > >
> > > On Tue, Jun 28, 2022 at 12:58 PM Radu Gheorghe <
> > radu.gheor...@sematext.com
> > > >
> > > wrote:
> > >
> > > > Hi Satya,
> > > >
> > > > I didn't try it, but does it work if you add
> "shards=shard1,shard2..."
> > > > to the request?
> > > >
> > > > Worst case scenario, if you have the address of each shard (you can
> > > > get it from Zookeeper), you can run the delete command N times, one
> > > > hitting each shard address.
> > > >
> > > > Best regards,
> > > > Radu
> > > > --
> > > > Elasticsearch/OpenSearch & Solr Consulting, Production Support &
> > > > Training Sematext Cloud - Full Stack Observability
> > > >
> https://gbr01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsemat
> > > > ext.com%2F&data=05%7C01%7Cpeter.lancaster%40findmypast.com
> %7C52e71
> > 

Re: Delete by Id in solr cloud

[Satya Nand]

Hi, r ohara,

Yeah, I found the bug id:https://issues.apache.org/jira/browse/SOLR-8889.
seems like it was fixed in solr 8.10 and 9.

so nothing worked besides using delete by query?

On Thu, Jun 30, 2022 at 3:41 AM r ohara  wrote:

> Hi Satya,
> I think it's a bug with using compositeId. We had the same issue, and had
> to use deleteByQuery instead, but like you said, it's much slower. We're
> using solr 8.11
>
> On Tue, Jun 28, 2022 at 4:59 AM Satya Nand  .invalid>
> wrote:
>
> > Thanks, Peter,
> > I am checking that, also UpdateRequest class seems to have methods that
> > take routes as input. I will see if it helps.
> >
> > On Tue, Jun 28, 2022 at 3:19 PM Peter Lancaster <
> > peter.lancas...@findmypast.com> wrote:
> >
> > > Hi Satya,
> > >
> > > I think you would need to use a HttpSolrClient that uses the url of the
> > > shard where the record exists.
> > >
> > > Regards,
> > > Peter.
> > >
> > > -Original Message-
> > > From: Satya Nand 
> > > Sent: 28 June 2022 10:43
> > > To: users@solr.apache.org
> > > Subject: Re: Delete by Id in solr cloud
> > >
> > > EXTERNAL SENDER: Do not click any links or open any attachments unless
> > you
> > > trust the sender and know the content is safe.
> > >
> > >
> > > Hi Radu,
> > >
> > > I am using solrj for executing the query. I couldn't find any function
> > > with accepts additional parameters like routing, shards, solr Params
> etc.
> > >
> > > I also tried delete by query instead of deleteById, But it is very
> slow.
> > >
> > >
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsolr.apache.org%2Fdocs%2F8_1_0%2Fsolr-solrj%2Forg%2Fapache%2Fsolr%2Fclient%2Fsolrj%2Fimpl%2FCloudSolrClient.html&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4d7N5LCpx8TXnEv7GW%2BN2TmoE8YvHa0tgr4c%2FamgOBw%3D&reserved=0
> > > deleteById
> > > <
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsolr.apache.org%2Fdocs%2F7_3_1%2Fsolr-solrj%2Forg%2Fapache%2Fsolr%2Fclient%2Fsolrj%2FSolrClient.html%23deleteById-java.lang.String-java.util.List-int-&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MdBKsoMlbTqUjx5xzUny1Hrop0La2cwkg6cVZgZ76Es%3D&reserved=0
> > > >
> > > (String
> > > <
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.oracle.com%2Fjavase%2F8%2Fdocs%2Fapi%2Fjava%2Flang%2FString.html%3Fis-external%3Dtrue&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3gPAaYNOQvAkYD8coSuGjm28gau5i3lEJabT4Kqu%2BCk%3D&reserved=0
> > > >
> > >  collection, List
> > > <
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.oracle.com%2Fjavase%2F8%2Fdocs%2Fapi%2Fjava%2Futil%2FList.html%3Fis-external%3Dtrue&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=bUQ0Fe0pPkP2kFeRy%2BLg%2FuTIBSEM1HVQdk4EEAdQYCQ%3D&reserved=0
> > > >
> > >  > > <
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.oracle.com%2Fjavase%2F8%2Fdocs%2Fapi%2Fjava%2Flang%2FString.html%3Fis-external%3Dtrue&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3gPAaYNOQvAkYD8coSuGjm28gau5i3lEJabT4Kqu%2BCk%3D&reserved=0
> > > >>
> > > ids,
> > > int commitWithinMs)
> > >
> > >
> > >
> > >
> > > On Tue, Jun 28, 2022 at 12:58 PM Radu Gheorghe <
> > radu.gheor...@sematext.com
> > > >
> > > wrote:
> > >
> > > > Hi Satya,
> > > >
> > > > I didn't try it, but does it work if you add
> "shards=shard1,shard2..."
> > > > to the request?
> > > >
> > > > Worst case scenario, if you have the address of each shard (you can
> > > > get it from Zookeeper), you can run the delete command N times, one
> > > > hitting each shard address.
> > > >
> > > > Best regards,
> > > > Radu
> > > > --
> > > > Elasticsearch/OpenSearch & Solr Consulting, Production Support &
> > > > Training Sematext Cloud - Full Stack Observability
> > > >
> https://gbr01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsemat
> > > > ext.com%2F&data=05%7C01%7Cpeter.lancaster%40findmypast.com
> 

Re: Delete by Id in solr cloud

[Jeryl Cook]

Unsubscribe

On Thu, Jun 30, 2022, 6:58 AM Satya Nand 
wrote:

> Hi, r ohara,
>
> Yeah, I found the bug id:https://issues.apache.org/jira/browse/SOLR-8889.
> seems like it was fixed in solr 8.10 and 9.
>
> so nothing worked besides using delete by query?
>
> On Thu, Jun 30, 2022 at 3:41 AM r ohara  wrote:
>
> > Hi Satya,
> > I think it's a bug with using compositeId. We had the same issue, and had
> > to use deleteByQuery instead, but like you said, it's much slower. We're
> > using solr 8.11
> >
> > On Tue, Jun 28, 2022 at 4:59 AM Satya Nand  > .invalid>
> > wrote:
> >
> > > Thanks, Peter,
> > > I am checking that, also UpdateRequest class seems to have methods that
> > > take routes as input. I will see if it helps.
> > >
> > > On Tue, Jun 28, 2022 at 3:19 PM Peter Lancaster <
> > > peter.lancas...@findmypast.com> wrote:
> > >
> > > > Hi Satya,
> > > >
> > > > I think you would need to use a HttpSolrClient that uses the url of
> the
> > > > shard where the record exists.
> > > >
> > > > Regards,
> > > > Peter.
> > > >
> > > > -Original Message-
> > > > From: Satya Nand 
> > > > Sent: 28 June 2022 10:43
> > > > To: users@solr.apache.org
> > > > Subject: Re: Delete by Id in solr cloud
> > > >
> > > > EXTERNAL SENDER: Do not click any links or open any attachments
> unless
> > > you
> > > > trust the sender and know the content is safe.
> > > >
> > > >
> > > > Hi Radu,
> > > >
> > > > I am using solrj for executing the query. I couldn't find any
> function
> > > > with accepts additional parameters like routing, shards, solr Params
> > etc.
> > > >
> > > > I also tried delete by query instead of deleteById, But it is very
> > slow.
> > > >
> > > >
> > > >
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsolr.apache.org%2Fdocs%2F8_1_0%2Fsolr-solrj%2Forg%2Fapache%2Fsolr%2Fclient%2Fsolrj%2Fimpl%2FCloudSolrClient.html&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4d7N5LCpx8TXnEv7GW%2BN2TmoE8YvHa0tgr4c%2FamgOBw%3D&reserved=0
> > > > deleteById
> > > > <
> > > >
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsolr.apache.org%2Fdocs%2F7_3_1%2Fsolr-solrj%2Forg%2Fapache%2Fsolr%2Fclient%2Fsolrj%2FSolrClient.html%23deleteById-java.lang.String-java.util.List-int-&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MdBKsoMlbTqUjx5xzUny1Hrop0La2cwkg6cVZgZ76Es%3D&reserved=0
> > > > >
> > > > (String
> > > > <
> > > >
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.oracle.com%2Fjavase%2F8%2Fdocs%2Fapi%2Fjava%2Flang%2FString.html%3Fis-external%3Dtrue&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3gPAaYNOQvAkYD8coSuGjm28gau5i3lEJabT4Kqu%2BCk%3D&reserved=0
> > > > >
> > > >  collection, List
> > > > <
> > > >
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.oracle.com%2Fjavase%2F8%2Fdocs%2Fapi%2Fjava%2Futil%2FList.html%3Fis-external%3Dtrue&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=bUQ0Fe0pPkP2kFeRy%2BLg%2FuTIBSEM1HVQdk4EEAdQYCQ%3D&reserved=0
> > > > >
> > > >  > > > <
> > > >
> > >
> >
> https://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.oracle.com%2Fjavase%2F8%2Fdocs%2Fapi%2Fjava%2Flang%2FString.html%3Fis-external%3Dtrue&data=05%7C01%7Cpeter.lancaster%40findmypast.com%7C52e71d1ca9294234c62808da58eaa4a0%7C75e41e0807c2445db397039b2b54c244%7C0%7C0%7C637920062049080011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3gPAaYNOQvAkYD8coSuGjm28gau5i3lEJabT4Kqu%2BCk%3D&reserved=0
> > > > >>
> > > > ids,
> > > > int commitWithinMs)
> > > >
> > > >
> > > >
> > > >
> > > > On Tue, Jun 28, 2022 at 12:58 PM Radu Gheorghe <
> > > radu.gheor...@sematext.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Satya,
> > > > >
> > > > > I didn't try it, but does it work if you add
> > "shards=shard1,shard2..."
> > > > > to the request?
> > > > >
> > > > > Worst case scenario, if you have the address of each shard (you can
> > > > > get it from Zookeeper), you can run the delete command N times, one
> > > > > hitting each shard address.
> > > > >
> > > > > Best regards,
> > > > > Radu
> > > > > 

Re: Solr 8.1.1 - OpenJDK 11

[Shawn Heisey]

On 6/30/22 00:34, Paul Ryder wrote:

Various docs seem ambiguous on Java version required for Solr 8.1.1

Solr docs says Java 8 or higher and Solr 8.1.1 was continually tested with JDK11

Sitecore docs say use only Java8 for Solr 8.4 or earlier

I'm trying OpenJDK 11 with Solr 8.1.1 - mostly works but in cloud mode I get 
broken pipe error in inter-node communication under SSL (Windows 10)

What's the definitive recommended version of OpenJDK for Solr 8.1.1?


All versions of Solr 8 require Java 8.  I have used 8.11.x with Java 11 
and 17, but my setup is not in cloud mode and I have never enabled SSL.


Solr 9 requires Java 11.  At this time there is only one 9.x version 
available, 9.0.0.  At this moment my Solr install is version 
10.0.0-SNAPSHOT -- built from the main branch in git.  I am running that 
with OpenJDK 11.0.15.


When you say 8.1.1, do you actually mean 8.11.1?  There is an 8.1.1 
version, but it is over three years old now and upgrading is definitely 
recommended.  The most current 8.x version of Solr is 8.11.2.


I strongly recommend that you run the latest OpenJDK 11, and that all of 
your SolrCloud nodes are running the exact same version of Java.  
Interoperability over SSL could be a problem if the nodes are running 
different versions of Java.


Thanks,
Shawn

RE: Solr 8.1.1 - OpenJDK 11

[Paul Ryder]

Hi Shawn

Unfortunately it is 8.1.1 :-(

The version of Sitecore we are on only supports this version... but I will 
press to get that upgraded...

The thing that has me confused is the Solr doc 
(https://solr.apache.org/guide/solr/latest/deployment-guide/system-requirements.html)
 that refers to Solr 8.x and states "Requires Java 8 or higher" and "This 
version did have continuous testing with Java 9, 10, 11, 12 and the pre-release 
version of Java 13."

My setup (3 nodes Cloud + external 3 ZK servers on Windows SSL) works just fine 
with Java 8 but with Java 11 gives broken pipe errors when posting new docs. 
This is such a basic thing to test that I wonder what "This version did have 
continuous testing with Java 9, 10, 11, 12 and the pre-release version of Java 
13." actually refers to...

Ta! Paul

-Original Message-
From: Shawn Heisey  
Sent: 30 June 2022 12:29
To: users@solr.apache.org
Subject: Re: Solr 8.1.1 - OpenJDK 11

On 6/30/22 00:34, Paul Ryder wrote:
> Various docs seem ambiguous on Java version required for Solr 8.1.1
>
> Solr docs says Java 8 or higher and Solr 8.1.1 was continually tested 
> with JDK11
>
> Sitecore docs say use only Java8 for Solr 8.4 or earlier
>
> I'm trying OpenJDK 11 with Solr 8.1.1 - mostly works but in cloud mode 
> I get broken pipe error in inter-node communication under SSL (Windows 
> 10)
>
> What's the definitive recommended version of OpenJDK for Solr 8.1.1?

All versions of Solr 8 require Java 8.  I have used 8.11.x with Java 11 and 17, 
but my setup is not in cloud mode and I have never enabled SSL.

Solr 9 requires Java 11.  At this time there is only one 9.x version available, 
9.0.0.  At this moment my Solr install is version 10.0.0-SNAPSHOT -- built from 
the main branch in git.  I am running that with OpenJDK 11.0.15.

When you say 8.1.1, do you actually mean 8.11.1?  There is an 8.1.1 version, 
but it is over three years old now and upgrading is definitely recommended.  
The most current 8.x version of Solr is 8.11.2.

I strongly recommend that you run the latest OpenJDK 11, and that all of your 
SolrCloud nodes are running the exact same version of Java. Interoperability 
over SSL could be a problem if the nodes are running different versions of Java.

Thanks,
Shawn

Re: Solr 8.1.1 - OpenJDK 11

[Alex The Rocker]

Hi,

I'm running Solr 8.11 since a long time with IBM Semeru 11 (OpenJ9
variant of OpenJDK) on CentOS 7.9 x64, with SolrCloud, but no SSL.
I tried once with Java 17 (still IBM Semeru 17) and found that
everything was working fine, but I was a bit shy to push that in
production, so still running with Java 11...

Alex

Le jeu. 30 juin 2022 à 08:34, Paul Ryder  a écrit :
>
> All,
>
> Various docs seem ambiguous on Java version required for Solr 8.1.1
>
> Solr docs says Java 8 or higher and Solr 8.1.1 was continually tested with 
> JDK11
>
> Sitecore docs say use only Java8 for Solr 8.4 or earlier
>
> I'm trying OpenJDK 11 with Solr 8.1.1 - mostly works but in cloud mode I get 
> broken pipe error in inter-node communication under SSL (Windows 10)
>
> What's the definitive recommended version of OpenJDK for Solr 8.1.1?
>
> Ta! Paul

R: [suspected SPAM] Re: Semantic Knowledge Graph theoric question

[Danilo Tomasoni]

Thank you very much Michael for your answer.
Below the extra information you asked for, and a sample result

QUERY INFORMATION
query=covid
back query = *:*
fore query = mitochondria
sample gene id ="57506" / "54205"

facet code:
"json.facet": "{'titles_gene': {'type': 'terms', 'field': 
'titles_gene_pubtator_annotation_ids', 'limit': 10, 'sort': {'rtitles0': 
'desc'}, 'facet': {'rtitles0': 'relatedness($fore,$back)'}}

CARDINALITY
q=157120
back=157120
fore=321385
fore & back=343

"57506" & back=8
"57506" & fore & back=5

"54205" & back=5
"54205" & fore & back=5

RESULTS
titles_gene
"val": "57506",
"count": 8,
"rtitles0": {
  "relatedness": 0.69182,
  "foreground_popularity": 1e-05,
  "background_popularity": 1e-05
}
abstracts_gene
"val": "54205",
"count": 5,
"rabstracts0": {
  "relatedness": 0.94975,
  "foreground_popularity": 0.00025,
  "background_popularity": 0.00043
}



Here it looks like that the fpopularity and bpopularity are the same for 
titles_gene (but I expected 5/343 and 8/157120  instead..)
but the relatedness of 0.69182 (it should range between -1 and 1) suggests me 
that "57506" is strongly "characteristic"
(meaning that it is occourring more in the fore than in the back, that is a 
superset of fore)
to the fore corpus with respect to the back corpus.


I would like to ask:

  1.  is my interpretation of relatedness correct?
  2.  why foreground_popularity and background_popularity are like this?
  3.  how should I change my json.facet query to require a min_popularity? 
should this solve the strange relatedness values?


thank you
D


Danilo Tomasoni

Fondazione The Microsoft Research - University of Trento Centre for 
Computational and Systems Biology (COSBI)
Piazza Manifattura 1,  38068 Rovereto (TN), Italy
tomas...@cosbi.eu
http://www.cosbi.eu

As for the European General Data Protection Regulation 2016/679 on the 
protection of natural persons with regard to the processing of personal data, 
we inform you that all the data we possess are object of treatment in the 
respect of the normative provided for by the cited GDPR.
It is your right to be informed on which of your data are used and how; you may 
ask for their correction, cancellation or you may oppose to their use by 
written request sent by recorded delivery to The Microsoft Research – 
University of Trento Centre for Computational and Systems Biology Scarl, Piazza 
Manifattura 1, 38068 Rovereto (TN), Italy.
P Please don't print this e-mail unless you really need to

Da: Michael Gibney 
Inviato: martedì 28 giugno 2022 14:50
A: users@solr.apache.org 
Oggetto: Re: [suspected SPAM] Re: Semantic Knowledge Graph theoric question

[CAUTION: EXTERNAL SENDER]
[Please check correspondence between Sender Display Name and Sender Email 
Address before clicking on any link or opening attachments]


It's hard to give a concrete answer without knowing the actual counts
involved, but iiuc significantTerms and relatedness are basically
equivalent (happy to be corrected here if I'm wrong).

> the relatedness function is iffy at best

? -- not sure what is meant by this. It's a function, and afaict based
on information available it's likely to be working as intended (and
most likely equivalent to what you'd get for an analogous config of
significantTerms streaming expression?). If you're getting a hit count
of 1 returned for any of these terms, then you're probably dealing
with numbers low enough that you're going to mostly be getting noise
-- indeed, that's the purpose of the min_popularity setting: to screen
out results that don't have enough instances (e.g., 1) to calculate a
meaningful correlation.

Of course feel free to experiment with the other suggestions above,
but unless you also screen out low-frequency terms (i.e., with
min_popularity) I'd be surprised if shingles would have a helpful
impact on your use case.

Ultimately though, to reiterate: I think it's going to be hard to
provide helpful feedback unless you're able to provide a sense of the
integer counts involved (background set size, and intersection size
(term of interest & foreground_set; term of interest &
background_set). It might also be helpful to have a sense of the
overall cardinality and types of values in the field in question.

On Tue, Jun 28, 2022 at 8:17 AM Danilo Tomasoni  wrote:
>
> Thank you very much Alessandro.
> I will look into the code.
>
>
> Danilo Tomasoni
>
> Fondazione The Microsoft Research - University of Trento Centre for 
> Computational and Systems Biology (COSBI)
> Piazza Manifattura 1,  38068 Rovereto (TN), Italy
> tomas...@cosbi.eu

Semantic Knowledge Graph theoric question

[Danilo Tomasoni]

Thank you very much Michael for your answer.
Below the extra information you asked for, and a sample result

QUERY INFORMATION
query=covid
back query = *:*
fore query = mitochondria
sample gene id ="57506" / "54205"

facet code:
"json.facet": "{'titles_gene': {'type': 'terms', 'field': 
'titles_gene_pubtator_annotation_ids', 'limit': 10, 'sort': {'rtitles0': 
'desc'}, 'facet': {'rtitles0': 'relatedness($fore,$back)'}}

CARDINALITY
q=157120
back=157120
fore=321385
fore & back=343

"57506" & back=8
"57506" & fore & back=5

"54205" & back=5
"54205" & fore & back=5

RESULTS
titles_gene
"val": "57506",
"count": 8,
"rtitles0": {
  "relatedness": 0.69182,
  "foreground_popularity": 1e-05,
  "background_popularity": 1e-05
}
abstracts_gene
"val": "54205",
"count": 5,
"rabstracts0": {
  "relatedness": 0.94975,
  "foreground_popularity": 0.00025,
  "background_popularity": 0.00043
}



Here it looks like that the fpopularity and bpopularity are the same for 
titles_gene (but I expected 5/343 and 8/157120  instead..)
but the relatedness of 0.69182 (it should range between -1 and 1) suggests me 
that "57506" is strongly "characteristic"
(meaning that it is occourring more in the fore than in the back, that is a 
superset of fore)
to the fore corpus with respect to the back corpus.


I would like to ask:

  1.  is my interpretation of relatedness correct?
  2.  why foreground_popularity and background_popularity are like this?
  3.  how should I change my json.facet query to require a min_popularity? 
should this solve the strange relatedness values?
  4.  is it possible to 'test' (statistically) the significativity of a 
z-score, like we do with a p-value?


thank you
D


Danilo Tomasoni

Fondazione The Microsoft Research - University of Trento Centre for 
Computational and Systems Biology (COSBI)
Piazza Manifattura 1,  38068 Rovereto (TN), Italy
tomas...@cosbi.eu
http://www.cosbi.eu

As for the European General Data Protection Regulation 2016/679 on the 
protection of natural persons with regard to the processing of personal data, 
we inform you that all the data we possess are object of treatment in the 
respect of the normative provided for by the cited GDPR.
It is your right to be informed on which of your data are used and how; you may 
ask for their correction, cancellation or you may oppose to their use by 
written request sent by recorded delivery to The Microsoft Research – 
University of Trento Centre for Computational and Systems Biology Scarl, Piazza 
Manifattura 1, 38068 Rovereto (TN), Italy.
P Please don't print this e-mail unless you really need to

group.ngroups=true and multi shard

[Vincenzo D'Amore]

Hi,

I have a collection that is becoming bigger than usual (few millions of
documents) so we are doing a few tests to understand how to divide it in
multiple shards (solrcloud 8.11.1).

Strangely since we divided the index in 3 shards the ngroups value is not
always returned.
To be clear, executing the same query 10 times, can happen that 3/4 times
the ngroups value is not returned. We created the shard using
router.name=compositeId
and we have _route_ in the query.

Any suggestions?

Best regards,
Vincenzo

{"device-product-hash":{
"pullReplicas":"0",
"replicationFactor":"3",
"shards":{
  "shard1":{
"range":"8000-d554",
"state":"active",
"replicas":{
  "core_node7":{
"core":"device-product-hash_shard1_replica_n4",
"node_name":"solr-1.solrcluster.svc.cluster.local:8983_solr",
"base_url":"
http://solr-1.solrcluster.svc.cluster.local:8983/solr";,
"state":"active",
"type":"NRT",
"force_set_state":"false"},
  "core_node20":{
"core":"device-product-hash_shard1_replica_n19",
"node_name":"solr-0.solrcluster.svc.cluster.local:8983_solr",
"base_url":"
http://solr-0.solrcluster.svc.cluster.local:8983/solr";,
"state":"active",
"type":"NRT",
"force_set_state":"false",
"leader":"true"}}},
  "shard2":{
"range":"d555-2aa9",
"state":"active",
"replicas":{
  "core_node13":{
"core":"device-product-hash_shard2_replica_n10",
"node_name":"solr-1.solrcluster.svc.cluster.local:8983_solr",
"base_url":"
http://solr-1.solrcluster.svc.cluster.local:8983/solr";,
"state":"active",
"type":"NRT",
"force_set_state":"false",
"leader":"true"},
  "core_node22":{
"core":"device-product-hash_shard2_replica_n21",
"node_name":"solr-2.solrcluster.svc.cluster.local:8983_solr",
"base_url":"
http://solr-2.solrcluster.svc.cluster.local:8983/solr";,
"state":"active",
"type":"NRT",
"force_set_state":"false"}}},
  "shard3":{
"range":"2aaa-7fff",
"state":"active",
"replicas":{
  "core_node24":{
"core":"device-product-hash_shard3_replica_n23",
"node_name":"solr-0.solrcluster.svc.cluster.local:8983_solr",
"base_url":"
http://solr-0.solrcluster.svc.cluster.local:8983/solr";,
"state":"active",
"type":"NRT",
"force_set_state":"false",
"leader":"true"},
  "core_node28":{
"core":"device-product-hash_shard3_replica_n27",
"node_name":"solr-2.solrcluster.svc.cluster.local:8983_solr",
"base_url":"
http://solr-2.solrcluster.svc.cluster.local:8983/solr";,
"state":"active",
"type":"NRT",
"force_set_state":"false",
"router":{"name":"compositeId"},
"maxShardsPerNode":"-1",
"autoAddReplicas":"false",
"nrtReplicas":"3",
"tlogReplicas":"0"}}




-- 
Vincenzo D'Amore

Problem in enabling basic solr authentication for solr cluster

[Anchal Sharma2]

Hello Team ,

I referred to following link and enabled basic solr authentication  -
Basic Authentication Plugin | Apache Solr Reference Guide 
8.11
Basic Authentication Plugin | Apache Solr Reference Guide 
8.11
Combining Basic Authentication with Other Schemes. When using other 
authentication schemes, such as the JWT Authentication Plugin, you may still 
want to use Basic authentication for a small set of "service account" oriented 
client applications.Solr provides the MultiAuthPlugin to support multiple 
authentication schemes. For example, you may want to integrate Solr with an 
OIDC provider for user ...
solr.apache.org

 It works well for a standalone solrcloud . But when I try these steps in solr 
cluster (3 zookeepers and 2 solrs) . I get intermittent error on running query 
on solr collections . Please let me know if I have missed some configuration. 
Please note the error comes ONLY when running query , and I can login to solr 
just fine using solr credentials.
Solr version 8.11.1 and apache zookeeper version 3.5.8

ERROR




Error 401 require authentication

HTTP ERROR 401 require authentication

URI:/solr/param/select
STATUS:401
MESSAGE:require authentication
SERVLET:default





STEPS DONE
1.Updated security.json on both solr nodes
{"authentication":{
"blockUnknown": true,
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"DefaultEncryptedPassword"},
"realm":"Solr user",
"forwardCredentials": false},
"authorization":{"class":"solr.RuleBasedAuthorizationPlugin",
"user-role":{"solr":"admin"},
"permissions":[{"name":"security-edit","role":"admin"}]
}}

2.Restarted all 3 zookeeper and 2 solr nodes

3.Upload security.json to zookeeper (for both solr nodes)
./solr zk cp //security.json zk:security.json -z 
hostname1:2181,hostname2:2181,hostname3:2181

Thank You
Anchal Sharma

Re: group.ngroups=true and multi shard

[Mikhail Khludnev]

I've thought JSON Facet is the answer.. despite the question.
 https://issues.apache.org/jira/browse/SOLR-7036
https://issues.apache.org/jira/browse/SOLR-4763

On Thu, Jun 30, 2022 at 6:31 PM Vincenzo D'Amore  wrote:

> Hi,
>
> I have a collection that is becoming bigger than usual (few millions of
> documents) so we are doing a few tests to understand how to divide it in
> multiple shards (solrcloud 8.11.1).
>
> Strangely since we divided the index in 3 shards the ngroups value is not
> always returned.
> To be clear, executing the same query 10 times, can happen that 3/4 times
> the ngroups value is not returned. We created the shard using
> router.name=compositeId
> and we have _route_ in the query.
>
> Any suggestions?
>
> Best regards,
> Vincenzo
>
> {"device-product-hash":{
> "pullReplicas":"0",
> "replicationFactor":"3",
> "shards":{
>   "shard1":{
> "range":"8000-d554",
> "state":"active",
> "replicas":{
>   "core_node7":{
> "core":"device-product-hash_shard1_replica_n4",
> "node_name":"solr-1.solrcluster.svc.cluster.local:8983_solr",
> "base_url":"
> http://solr-1.solrcluster.svc.cluster.local:8983/solr";,
> "state":"active",
> "type":"NRT",
> "force_set_state":"false"},
>   "core_node20":{
> "core":"device-product-hash_shard1_replica_n19",
> "node_name":"solr-0.solrcluster.svc.cluster.local:8983_solr",
> "base_url":"
> http://solr-0.solrcluster.svc.cluster.local:8983/solr";,
> "state":"active",
> "type":"NRT",
> "force_set_state":"false",
> "leader":"true"}}},
>   "shard2":{
> "range":"d555-2aa9",
> "state":"active",
> "replicas":{
>   "core_node13":{
> "core":"device-product-hash_shard2_replica_n10",
> "node_name":"solr-1.solrcluster.svc.cluster.local:8983_solr",
> "base_url":"
> http://solr-1.solrcluster.svc.cluster.local:8983/solr";,
> "state":"active",
> "type":"NRT",
> "force_set_state":"false",
> "leader":"true"},
>   "core_node22":{
> "core":"device-product-hash_shard2_replica_n21",
> "node_name":"solr-2.solrcluster.svc.cluster.local:8983_solr",
> "base_url":"
> http://solr-2.solrcluster.svc.cluster.local:8983/solr";,
> "state":"active",
> "type":"NRT",
> "force_set_state":"false"}}},
>   "shard3":{
> "range":"2aaa-7fff",
> "state":"active",
> "replicas":{
>   "core_node24":{
> "core":"device-product-hash_shard3_replica_n23",
> "node_name":"solr-0.solrcluster.svc.cluster.local:8983_solr",
> "base_url":"
> http://solr-0.solrcluster.svc.cluster.local:8983/solr";,
> "state":"active",
> "type":"NRT",
> "force_set_state":"false",
> "leader":"true"},
>   "core_node28":{
> "core":"device-product-hash_shard3_replica_n27",
> "node_name":"solr-2.solrcluster.svc.cluster.local:8983_solr",
> "base_url":"
> http://solr-2.solrcluster.svc.cluster.local:8983/solr";,
> "state":"active",
> "type":"NRT",
> "force_set_state":"false",
> "router":{"name":"compositeId"},
> "maxShardsPerNode":"-1",
> "autoAddReplicas":"false",
> "nrtReplicas":"3",
> "tlogReplicas":"0"}}
>
>
>
>
> --
> Vincenzo D'Amore
>


-- 
Sincerely yours
Mikhail Khludnev

Re: using childFilter to restrict "child" docs by "grandchild" information

[Mikhail Khludnev]

If it's worked partially, only debugger may answer what's wrong with the
particular query.

On Wed, Jun 29, 2022 at 10:51 AM Noah Torp-Smith 
wrote:

> Interestingly, I found that
>
> [child childFilter=$pidfilter limit=-1]&pidfilter=+instance.agency:94
>
> also worked - I get the pid that is present at that library. It's when I
> restrict at both pid and instance level it does not seem to work.
>
>
> --
>
> Noah Torp-Smith (n...@dbc.dk)
>
> 
> Fra: Noah Torp-Smith 
> Sendt: 29. juni 2022 09:26
> Til: users@solr.apache.org 
> Emne: Sv: using childFilter to restrict "child" docs by "grandchild"
> information
>
> [Du f?r ikke ofte mails fra n...@dbc.dk.invalid. F? mere at vide om,
> hvorfor dette er vigtigt, p? https://aka.ms/LearnAboutSenderIdentification
> ]
>
> (apologies if this is not the correct way to respond to a comment to my
> original post)
>
> Hello Mikhail - thanks for responding so quickly.
>
> Your suggestion does not seem to work for me. Using
>
> [child childFilter=$pidfilter limit=-1]&pidfilter=+pid.material_type:bog
>
> works, but
>
> [child childFilter=$pidfilter limit=-1]&pidfilter=+pid.material_type:bog +
> instance.agency:94
>
> or even
>
> [child childFilter=$pidfilter limit=-1]&pidfilter=+pid.material_type:bog +
> instance.agency:*
>
> returns the top-level work document, but no child/pid documents (and I
> have verified that they do exist). Specifying an instance-level restriction
> seems to make all the pid-level documents vanish.
>
> Thanks,
>
> /Noah
>
>
> --
>
> Noah Torp-Smith (n...@dbc.dk)
>
> 
> Fra: Mikhail Khludnev 
> Sendt: 29. juni 2022 08:32
> Til: users@solr.apache.org 
> Emne: Re: using childFilter to restrict "child" docs by "grandchild"
> information
>
> Hello, Noah.
> Could i be something like
> [child childFilter=$pidfilter limit=-1]&pidfilter=+pid.material_type:bog +
> instance.agency:94 +instance.status:onShelf
> ?
>
> On Wed, Jun 29, 2022 at 8:57 AM Noah Torp-Smith 
> wrote:
>
> > To explain my question, first some domain background. We have a search
> > engine where users can search for materials they can borrow at their
> local
> > library.
> >
> > Our top level documents are *works*. An example of a work could be "Harry
> > Potter and the Philosopher's Stone". Examples of information stored at
> this
> > level could be the title, the author of the work, and a genre.
> >
> > At the second level, we have *manifestations" (we call these "pids"). It
> > might be that a work exists as a physical book, an ebook, as an audiobook
> > on CDs, an online audiobook, and there might be several editions of a
> book.
> > Information stored at this level includes material type, year of
> > publication, contributors (can be narrators, artists that have
> illustrated
> > in a particular edition).
> >
> > At the third level, we have *instances*. This includes information about
> > the physical books, and in which libraries they are located, which
> > department, and even down to locations within departments, if they are
> > currently on loan, on the shelf.
> >
> > Each document has a `doc_type` (which is either work, pid, or instance),
> > works have a list of pids, and pids have a list of instances associated
> > with them.
> >
> > Our job is to formulate solr queries on behalf of users that belong to
> > their local library, so that they can search for materials that is
> > available to them. Given a query, we want to return works, along with the
> > manifestations that match the query. A query can specify restrictions at
> > all three levels; you might be interested in the (physical) book from
> last
> > year written by Jussi Adler-Olsen, and it should be available at the
> local
> > branch of the community library.
> >
> > The way we find the appropriate works is pretty much in place. We use the
> > `/query` endpoint of solr, and we formulate a json object where
> >
> > * the `query` field contains the restrictions at the work level,
> something
> > like `work.creator:'Jussi Adler-Olsen'`.
> > * To restrict to works where manifestations/pids apply to the
> restrictions
> > at that level, we use a "parent which" construction in the `filter` part
> of
> > the solr query. Something like `{!parent
> > which='doc_type:work'}(pid.material_type:book AND  pid.year:(2021))`.
> > * To restrict to works where we can find a physical copy at the local
> > library, we add another element to the `filter`. Something like `{!parent
> > which='doc_type:work'}(instance.agency:94 AND
> > instance.status:\"onShelf\")`, where 94 is the id of the local
> library.
> >
> > That seems to work well. We get the works we are interested in. The
> > question I have is, how do I restrict the manifestations we return? We
> use
> > the field list and a `childFilter` to restrict manifestations, something
> > like this: `"fields": "work.workid work.title work.creator, pids, id,
> > pid.year, pid.material_type [child childFilte

Re: Solr 8.1.1 - OpenJDK 11

[Paul Ryder]

Rumour has it that Solr 8.4 is the first of the 8.x releases to support java 11

But only a rumour…! Guess I’ll try that now… but it would be nice if the docs 
gave a definitive answer…

Get Outlook for iOS

From: Alex The Rocker 
Sent: Thursday, June 30, 2022 2:19:30 PM
To: users@solr.apache.org 
Subject: Re: Solr 8.1.1 - OpenJDK 11

Hi,

I'm running Solr 8.11 since a long time with IBM Semeru 11 (OpenJ9
variant of OpenJDK) on CentOS 7.9 x64, with SolrCloud, but no SSL.
I tried once with Java 17 (still IBM Semeru 17) and found that
everything was working fine, but I was a bit shy to push that in
production, so still running with Java 11...

Alex

Le jeu. 30 juin 2022 à 08:34, Paul Ryder  a écrit :
>
> All,
>
> Various docs seem ambiguous on Java version required for Solr 8.1.1
>
> Solr docs says Java 8 or higher and Solr 8.1.1 was continually tested with 
> JDK11
>
> Sitecore docs say use only Java8 for Solr 8.4 or earlier
>
> I'm trying OpenJDK 11 with Solr 8.1.1 - mostly works but in cloud mode I get 
> broken pipe error in inter-node communication under SSL (Windows 10)
>
> What's the definitive recommended version of OpenJDK for Solr 8.1.1?
>
> Ta! Paul

Re: How to add replica cluster-policy

[Koen De Groote]

To answer my own question: it was disk space.

Has to be below 80% used disk space. Or it won't start the replication.

On Sat, Jun 25, 2022 at 12:58 PM Koen De Groote  wrote:

> Using solr7_6
>
> This documentation:
> https://solr.apache.org/guide/7_6/solrcloud-autoscaling-api.html#solrcloud-autoscaling-api
>
> Speaks of "cluster-policy". I recently did a restore of a backup, and it
> does not have this.
>
> The API output for autoscaling only shows:
>
> "config":{
> "cluster-preferences":[{
> "minimize":"cores",
> "precision":1}]}}
>
> There is no "cluster-policy" in the output.
>
> Now I'm faced with a problem where a replica isn't being made on my 2nd
> host and the repair action in the admin interface fails, saying it can't
> find replica locations that don't violate the policies... and then shows me
> an empty list of policies.
>
> I don't understand what is going on.
>
> How can I insert my own list of policies?
>
> Regards,
> Koen De Groote
>
>

Re: group.ngroups=true and multi shard

[Vincenzo D'Amore]

Hi Mikhail,

thanks for the prompt reply, not sure I got your suggestion...
Anyway, reading the jira issues I realized that I can have the ngroups
value simply adding a json.facet on the same field I'm grouping to.
Something like this:

json.facet: "{"categories":{"type": "terms","field": "fieldName","limit":
0,"numBuckets":true}}".

Thanks,
Vincenzo



On Thu, Jun 30, 2022 at 8:44 PM Mikhail Khludnev  wrote:

> I've thought JSON Facet is the answer.. despite the question.
>  https://issues.apache.org/jira/browse/SOLR-7036
> https://issues.apache.org/jira/browse/SOLR-4763
>
> On Thu, Jun 30, 2022 at 6:31 PM Vincenzo D'Amore 
> wrote:
>
> > Hi,
> >
> > I have a collection that is becoming bigger than usual (few millions of
> > documents) so we are doing a few tests to understand how to divide it in
> > multiple shards (solrcloud 8.11.1).
> >
> > Strangely since we divided the index in 3 shards the ngroups value is not
> > always returned.
> > To be clear, executing the same query 10 times, can happen that 3/4 times
> > the ngroups value is not returned. We created the shard using
> > router.name=compositeId
> > and we have _route_ in the query.
> >
> > Any suggestions?
> >
> > Best regards,
> > Vincenzo
> >
> > {"device-product-hash":{
> > "pullReplicas":"0",
> > "replicationFactor":"3",
> > "shards":{
> >   "shard1":{
> > "range":"8000-d554",
> > "state":"active",
> > "replicas":{
> >   "core_node7":{
> > "core":"device-product-hash_shard1_replica_n4",
> > "node_name":"solr-1.solrcluster.svc.cluster.local:8983_solr",
> > "base_url":"
> > http://solr-1.solrcluster.svc.cluster.local:8983/solr";,
> > "state":"active",
> > "type":"NRT",
> > "force_set_state":"false"},
> >   "core_node20":{
> > "core":"device-product-hash_shard1_replica_n19",
> > "node_name":"solr-0.solrcluster.svc.cluster.local:8983_solr",
> > "base_url":"
> > http://solr-0.solrcluster.svc.cluster.local:8983/solr";,
> > "state":"active",
> > "type":"NRT",
> > "force_set_state":"false",
> > "leader":"true"}}},
> >   "shard2":{
> > "range":"d555-2aa9",
> > "state":"active",
> > "replicas":{
> >   "core_node13":{
> > "core":"device-product-hash_shard2_replica_n10",
> > "node_name":"solr-1.solrcluster.svc.cluster.local:8983_solr",
> > "base_url":"
> > http://solr-1.solrcluster.svc.cluster.local:8983/solr";,
> > "state":"active",
> > "type":"NRT",
> > "force_set_state":"false",
> > "leader":"true"},
> >   "core_node22":{
> > "core":"device-product-hash_shard2_replica_n21",
> > "node_name":"solr-2.solrcluster.svc.cluster.local:8983_solr",
> > "base_url":"
> > http://solr-2.solrcluster.svc.cluster.local:8983/solr";,
> > "state":"active",
> > "type":"NRT",
> > "force_set_state":"false"}}},
> >   "shard3":{
> > "range":"2aaa-7fff",
> > "state":"active",
> > "replicas":{
> >   "core_node24":{
> > "core":"device-product-hash_shard3_replica_n23",
> > "node_name":"solr-0.solrcluster.svc.cluster.local:8983_solr",
> > "base_url":"
> > http://solr-0.solrcluster.svc.cluster.local:8983/solr";,
> > "state":"active",
> > "type":"NRT",
> > "force_set_state":"false",
> > "leader":"true"},
> >   "core_node28":{
> > "core":"device-product-hash_shard3_replica_n27",
> > "node_name":"solr-2.solrcluster.svc.cluster.local:8983_solr",
> > "base_url":"
> > http://solr-2.solrcluster.svc.cluster.local:8983/solr";,
> > "state":"active",
> > "type":"NRT",
> > "force_set_state":"false",
> > "router":{"name":"compositeId"},
> > "maxShardsPerNode":"-1",
> > "autoAddReplicas":"false",
> > "nrtReplicas":"3",
> > "tlogReplicas":"0"}}
> >
> >
> >
> >
> > --
> > Vincenzo D'Amore
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>


-- 
Vincenzo D'Amore

Problem in enabling basic solr authentication for solr cluster

[Anchal Sharma2]

Hello Team ,

I referred to following link and enabled basic solr authentication  -
Basic Authentication Plugin | Apache Solr Reference Guide 
8.11
Basic Authentication Plugin | Apache Solr Reference Guide 
8.11
Combining Basic Authentication with Other Schemes. When using other 
authentication schemes, such as the JWT Authentication Plugin, you may still 
want to use Basic authentication for a small set of "service account" oriented 
client applications.Solr provides the MultiAuthPlugin to support multiple 
authentication schemes. For example, you may want to integrate Solr with an 
OIDC provider for user ...
solr.apache.org

 It works well for a standalone solrcloud . But when I try these steps in solr 
cluster (3 zookeepers and 2 solrs) . I get intermittent error on running query 
on solr collections . Please let me know if I have missed some configuration. 
Please note the error comes ONLY when running query , and I can login to solr 
just fine using solr credentials.
Solr version 8.11.1 and apache zookeeper version 3.5.8

ERROR




Error 401 require authentication

HTTP ERROR 401 require authentication

URI:/solr/param/select
STATUS:401
MESSAGE:require authentication
SERVLET:default





STEPS DONE
1.Updated security.json on both solr nodes
{"authentication":{
"blockUnknown": true,
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"DefaultEncryptedPassword"},
"realm":"Solr user",
"forwardCredentials": false},
"authorization":{"class":"solr.RuleBasedAuthorizationPlugin",
"user-role":{"solr":"admin"},
"permissions":[{"name":"security-edit","role":"admin"}]
}}

2.Restarted all 3 zookeeper and 2 solr nodes

3.Upload security.json to zookeeper (for both solr nodes)
./solr zk cp //security.json zk:security.json -z 
hostname1:2181,hostname2:2181,hostname3:2181

Thank You
Anchal Sharma