Re: Suggestions to improve Star queries latencies

2023-11-22 Thread Mikhail Khludnev
> And why is per shard latency so high? (~120 ms) ok. this proves that it's a problem with low level search. Make sure you have swap disabled, and plenty of ram available for mmaping (limited heap). Then, you need to put a search load: some simple script looping keywords with cache and distrib di

Solr Cloud + zookeeper+ kubernetes setup

2023-11-22 Thread Vince McMahon
Hi, I'd like to thank many of you giving your valuable advice. You are liked a big brother I never have. I wish you in the US a Happy Thanksgiving. I have boss that ask for the sky and I need a clarity if this is at all possible. I have very large cores, 200 million docs. The situation calls f

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-22 Thread Ramesh Balasubaramaniam
Yes, Possible. How many shards and replicas you are planning to create for the 200 millions documents all based upon the index size ( how many stored fields ) and all. External zookeeper is best. SOLR in kubernetes setup also works well. Thanks, Ramesh On Wed, Nov 22, 2023 at 3:30 AM Vince McMah

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-22 Thread Jan Høydahl
Hi, I'd have a look at https://solr.apache.org/operator/ for more info about the official support for running Solr in k8s. There is also this 3rd party Helm chart which is well maintained and perhaps easier to get started with: https://artifacthub.io/packages/helm/bitnami/solr Good luck. Jan

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-22 Thread matthew sporleder
It's annoying that zookeeper is a hard dependency when etcd is just sitting there and all of the stuff in zk could easily fit into ConfigMaps. Ditto using a managed service like dynamodb instead of zookeeper :) On Wed, Nov 22, 2023 at 6:24 AM Ramesh Balasubaramaniam wrote: > > Yes, Possible. Ho

Re: pls help with the definitions in schema.xml and how to get a copy of files

2023-11-22 Thread Shawn Heisey
On 11/20/23 20:22, Vince McMahon wrote: I need some help to clarify a schema.xml I am trying to find out where the fieldtype string is defined. I open the defination of the schema.xml from admin UI https://localhost:8984/solr/cookbookCSV/admin/file?wt=json&_=1700536107495&; file=schema.xml&cont

Re: pls help with the definitions in schema.xml and how to get a copy of files

2023-11-22 Thread Vince McMahon
Shawn, The guy left the job. And the pass word is gone. On Wed, Nov 22, 2023, 10:22 AM Shawn Heisey wrote: > On 11/20/23 20:22, Vince McMahon wrote: > > I need some help to clarify a schema.xml I am trying to find out where > the > > fieldtype string is defined. > > > > I open the defination o

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-22 Thread Vince McMahon
Ramesh, Can you elaborate more on the installation of ext. Zookeeper and how to make it work with solr on k8? On Wed, Nov 22, 2023, 6:26 AM Ramesh Balasubaramaniam wrote: > Yes, Possible. How many shards and replicas you are planning to create for > the 200 millions documents all based upon the

Re: Suggestions to improve Star queries latencies

2023-11-22 Thread Shawn Heisey
On 4/16/23 07:24, Rajani Maski wrote: What are your suggestions to improve star queries latencies? By star queries I mean "*:*" or single term queries having boost formulas (such as doc recency and many others) taking 10 or more seconds. It is a large collection with good compute resources, howe

Re: pls help with the definitions in schema.xml and how to get a copy of files

2023-11-22 Thread Shawn Heisey
On 11/22/23 08:57, Vince McMahon wrote: Shawn, The guy left the job. And the pass word is gone. As I said before, if the schema with those ENTITY definitions works, then the files ARE on the system somewhere. If it's Linux or another UNIX-like system, running this as root should locate the

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-22 Thread Vince McMahon
Hi, Jan. Thanks for your links. They are super good. Luv it. On Wed, Nov 22, 2023 at 7:35 AM Jan Høydahl wrote: > Hi, > > I'd have a look at https://solr.apache.org/operator/ for more info about > the official support for running Solr in k8s. > > There is also this 3rd party Helm chart which i

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-22 Thread Jan Høydahl
I think there is a product that speaks ZK api but stores everything in etcd :) Also, you may want to check out https://cwiki.apache.org/confluence/display/SOLR/SIP-18%3A+A+Solr+Kubernetes+Module+for+native+integration , an initiative that will let you use ConfigMaps for configSet files etc. I.e.

Re: pls help with the definitions in schema.xml and how to get a copy of files

2023-11-22 Thread Vince McMahon
Thank you Shawn. On Wed, Nov 22, 2023, 11:10 AM Shawn Heisey wrote: > On 11/22/23 08:57, Vince McMahon wrote: > > Shawn, > > > > The guy left the job. And the pass word is gone. > > As I said before, if the schema with those ENTITY definitions works, > then the files ARE on the system somewhere.