Re: Number of cores per node

2021-11-12 Thread Rahul Goswami
To clarify, I mean the number of Solr cores per JVM instance. How many Solr cores on one node is too many? As mentioned, it would be nice to get the numbers for a Solr instance running in standalone mode (aka non-SolrCloud), but if not, would also be interested to know about people's experience wit

Re: Number of cores per node

2021-11-12 Thread Tulsi Das
Hi Rahul , Are you asking about the cores (number of replica's per node) or the cpu cores? On Sat, Nov 13, 2021, 12:50 AM Walter Underwood wrote: > We’ve run on AWS instances with 72 CPUs. They all get used. Throughput is > linear with the number of CPUs. You need enough free RAM to cache all of

Re: Solr & Kubernetes - how to configure the liveness

2021-11-12 Thread Timothy Potter
oops! sent a little too soon ... also wanted to mention that if you're running Solr 8+, you can use /admin/info/health instead of /admin/info/system for the probe path (see: https://issues.apache.org/jira/browse/SOLR-11126), like this: livenessProbe: httpGet: path: /admin/info/health por

Re: Number of cores per node

2021-11-12 Thread Walter Underwood
We’ve run on AWS instances with 72 CPUs. They all get used. Throughput is linear with the number of CPUs. You need enough free RAM to cache all of the index files in OS file buffers. The entire point of avoiding locking in the Lucene index is so that multiple threads can read it without conten

Function query over score

2021-11-12 Thread Swapnil Vasant Chaudhari
Hi, Is there a way to apply function query over score field. I have a use case where I need to apply a mathematical function over the score of edismax query parser eg: div(score,100) The solr documentation allows using function query to achieve this by recomputing the score using the query fun

Re: Solr & Kubernetes - how to configure the liveness

2021-11-12 Thread Timothy Potter
Some things to consider ... If one out of many Solr cores is down on a pod, I would not want Kubelet to restart my Solr pod (if liveness probe fails) or even remove it from the load-balancer service (if readiness probe fails) because the pod can still serve traffic for the healthy cores. Requiring

Multi polygon spatial search - isochrones

2021-11-12 Thread Dan Rosher
Hi, We're looking at implementing commutability / reachability search to our users with isochrones e.g. https://en.wikipedia.org/wiki/Isochrone_map. There are a number of open and commercial services which we are looking at. Some of these, in particular with public transport, return multiple poly

Re: Solr & Kubernetes - how to configure the liveness

2021-11-12 Thread Mathieu Marie
Beware that using http://node:8983/api/node/health?requireHealthyCores=true for your liveness assumes that ZK is up and running. We are all hoping that ZK is never down, but if it happens, your Solr liveness probe will start to fail too, and K8S will restart all our Solr, adding instability to a cl

Re: Number of cores per node

2021-11-12 Thread Deepak Goel
My guess is (please note it is not a benchmark): you would need a lot of tuning to make Solr use 32 cpu cores per node. After 4 cpu cores, you would have to start tuning Solr, JVM, your app (requirement), IOP'S. Deepak "The greatness of a nation can be judged by the way its animals are treated - M

Number of cores per node

2021-11-12 Thread Rahul Goswami
Hi, Does anyone have benchmarks on performance as the number of cores on a Solr node goes up? I am trying to get an idea about how many cores per node is too much. Assume 31 GB heap size, SSD disk and 32 CPU cores. Preferably non-SolrCloud (aka standalone), but even if you have insights from SolrCl

Re: java.lang.ClassNotFoundException: solr.ICUCollationField for multi language support in Solr 8.9.0

2021-11-12 Thread dinesh naik
`solr.ICUCollationField`, is backed by http://site.icu-project.org[the ICU4J library], provides more flexible configuration, has more locales, is significantly faster, and requires less memory and less index space, since its keys are smaller than those produced by the JDK implementation that backs

Re: java.lang.ClassNotFoundException: solr.ICUCollationField for multi language support in Solr 8.9.0

2021-11-12 Thread dinesh naik
Hi Vishal, What is your solr.install.dir ? See if the pure path mentioned is solrconfig.xml. is correct. Also you might want to try using all jars mentioned in the sample solrconfig.xml from sample_techproducts_configs to start with . On Fri, Nov 12, 2021 at 4:48 PM VIshal Patel wrote: > Hi

Re: Return field with astrick(*) is going to slow in Solr 8.9.0

2021-11-12 Thread Jan Høydahl
Beware of the 'useDocValuesAsStored' setting which I believe is newer than 6.x Read more here https://solr.apache.org/guide/8_10/docvalues.html#retrieving-docvalues-during-search I think that may be what you're seeing. So what it will do is to try to return data from every field in the index eve

Nested Solr spatial query using JSON request API

2021-11-12 Thread Aimeos | Norbert Sendetzky
Hi there How to combine a nested query with a spatial filter in Apache Solr? This is a simplified nested query which should be combined with a spatial filter: curl "http://localhost:8983/solr/corename/query"; -d '{ "query":{ "bool":{ "must":[ "type:product", "product.s

Re: Return field with astrick(*) is going to slow in Solr 8.9.0

2021-11-12 Thread dinesh naik
Hi Vishal, Knowing exactly where the time is spent will be helpful. Is your data set (umber of documents) same in both versions? can your share snippet with debug=true for both versions. refer this for more details on debug param: https://solr.apache.org/guide/6_6/common-query-parameters.html#Com

Re: Solr & Kubernetes - how to configure the liveness

2021-11-12 Thread Vincenzo D'Amore
On Fri, Nov 12, 2021 at 10:54 AM Jan Høydahl wrote: > I agree that this is a risk. It all comes back to your initial sizing of > the cluster. > If you have decided for three nodes, and have HA policy of tolerating loss > of > any one server at a time, then you have to fully stress test your syste

Re: Return field with astrick(*) is going to slow in Solr 8.9.0

2021-11-12 Thread Dominic Humphries
It's not exactly an answer, but we've also experienced slowness when trying to upgrade. Our testing showed we started seeing degraded performance from v8.6 - if 8.5.2 would be "new enough" for you, it may be worth testing performance with that version to see if it's any better. On Fri, 12 Nov 2021

java.lang.ClassNotFoundException: solr.ICUCollationField for multi language support in Solr 8.9.0

2021-11-12 Thread VIshal Patel
Hi I want to support multi language sorting while searching in Solr 8.9.0. For that I have added in my collection schema.xml like Solrconfig.xml When I start the Solr I am getting below ERROR Caused by: java.lang.ClassNotFoundException: solr.ICUCollationField at java.net.URLClas

Return field with astrick(*) is going to slow in Solr 8.9.0

2021-11-12 Thread VIshal Patel
Hello, We have upgraded Solr from 6.1.0 to 8.9.0. After upgrade we are facing slowness when returning field with *. Solr 8.9.0 [300 milliseconds] http://XX.XX.XX.XX:/solr/forms/select?q=id%3AFORM27248682964899&fl=*ORI_FORMTITLE> Solr 6.1.0 [15 milliseconds] http://XX.XX.XX.XX:/solr/fo

Re: Migrating from Solr 7.5 to Solr 8.9

2021-11-12 Thread Robert Pearce
I remember we had an issue with duplicates being created when we moved from 6.6.1 to 8.4.0. One of our cores had a dynamic field definition which was stored but not indexed. We would add an existing document to the core and a duplicate of it would appear; add it again and another duplicate. We

Re: Solr & Kubernetes - how to configure the liveness

2021-11-12 Thread Jan Høydahl
I agree that this is a risk. It all comes back to your initial sizing of the cluster. If you have decided for three nodes, and have HA policy of tolerating loss of any one server at a time, then you have to fully stress test your system with only two of those three nodes. If the two nodes cannot h

Re: Solr & Kubernetes - how to configure the liveness

2021-11-12 Thread Vincenzo D'Amore
Hi Jan, I agree, if liveness is not configured correctly we could end up in an endless loop and the node never be healthy. Please consider another scenario, a common case where there are at least 3 solr instances in production 24/7 high availability with a situation of index light/heavy and query-