Re: LTS, UserManaged Mode

2024-12-14 Thread Shawn Heisey
On 10/9/2024 11:52, mtn search wrote: Does Solr mark any versions for Long Term Support (LTS) like other technology? I have not seen this, but wanted to check. I am not really sure whether this is fully documented anywhere, but I can tell you how we manage support. The current major version

Re: The run-time type of a value

2024-09-14 Thread Shawn Heisey
On 8/29/2024 09:27, org.apache.s...@io7m.com wrote: The client API allows for specifying input documents to be indexed. For example: final var document = new SolrInputDocument(); document.addField("x", "a"); document.addField("y", "b"); document.addField("z", "c"); client.add(collection, docume

Re: Programmatically creating a collection with a schema

2024-09-06 Thread Shawn Heisey
On 8/25/2024 02:59, org.apache.s...@io7m.com wrote: Thanks. So from what I can make out, it looks like the application would be given the name of an existing ConfigSet. It would then download that ConfigSet, replace the schema with its own, and then create a new ConfigSet based on the old one wit

Re: Keeping track of when a Segment Merge happens?

2024-09-06 Thread Shawn Heisey
On 9/6/2024 14:49, Koen De Groote wrote: I'm trying to get more insight into when Segment Merges happen. Right now all I really have to go on is the fact that I sometimes see an increase in used disk space for the partition that the solr data directories are on, and then after a few minutes this

Re: Programmatically creating a collection with a schema

2024-08-24 Thread Shawn Heisey
On 8/24/2024 12:59, org.apache.s...@io7m.com wrote: I have a schema.xml file locally. I want to create a new collection programmatically, and set the schema for that collection to be exactly the contents of schema.xml. No more, no less. There's a ConfigSets API that purports to allow for spe

Re: SOLR-13510 patch installation

2024-08-22 Thread Shawn Heisey
On 8/22/2024 17:55, Shawn Heisey wrote: Then you will need these commands, starting from the root of the checkout: ant ivy-bootstrap cd solr ant clean package I can confirm that Java 17 does NOT work. I installed Java 11 (it was missing on my Windows 11 machine) and added this command

Re: SOLR-13510 patch installation

2024-08-22 Thread Shawn Heisey
On 8/22/2024 17:40, Shawn Heisey wrote: On 8/22/2024 11:38, Raju Vaddeh wrote: Below are the steps we have performed on the local solr environment for the patch install. *1. Clone the solr repo* git clone GitHub - apache/solr: Apache Solr open-source search software <https://github.

Re: SOLR-13510 patch installation

2024-08-22 Thread Shawn Heisey
On 8/22/2024 11:38, Raju Vaddeh wrote: Below are the steps we have performed on the local solr environment for the patch install. *1. Clone the solr repo* git clone GitHub - apache/solr: Apache Solr open-source search software There's your problem right

Re: oom_solr.sh

2024-08-21 Thread Shawn Heisey
On 8/16/2024 15:05, mtn search wrote: oom_solr.sh is provided by Solr to kill the solr process on OOM and log this event. Is it a common practice to then also create supporting functionality to automatically restart solr after the kill? IMHO, in general, it is a bad idea to automatically resta

Re: Index directory is not created when i run the command ...

2024-08-05 Thread Shawn Heisey
On 8/5/2024 00:47, Deepak Goel wrote: Hello From the page: https://lucene.apache.org/core/2_9_4/demo.html#:~:text=First%2C%20you%20should%20download%20the,when%20you%20extracted%20the%20archive. I am trying to run the following command "java org.apache.lucene.demo.IndexFiles {full-path-to-luce

Re: solrcloud 9 -- cluster endpoint

2024-07-16 Thread Shawn Heisey
On 7/8/2024 03:25, Roberto Maggi @ Debian wrote: I'm totally new to apache ecosystem and, after studying solr9 documentation I can't clarify a doubt. Once I created the zookeeper cluster as well as the solr one, the latter cluster elects the "leader" and goes online. At this point I'm wonder

Re: Cannot Run Solr as the Solr User

2024-05-28 Thread Shawn Heisey
On 5/27/24 13:43, Lee Daniel wrote: I'm on Centos 7.5. Can't find it in:  * /usr/lib/systemd/system  * /usr/sbin/ I found it in  * /etc/init.d/ I know for sure when I initially installed I was able to use:  * service solr start/stop/restart  * systemctl start/stop/restart solr.servic

Re: Memory tuning advice for small single-node Solr instance

2024-05-26 Thread Shawn Heisey
On 5/25/24 07:09, Imran Chaudhry wrote: Just a follow-up that since limiting my Solr QuerySet rows to 1000 like this: qs.get_results(rows=1000) ...I have not had a single OOM killer issue. I consider this resolved. Reducing rows can reduce the amount of heap memory that Solr needs because So

Re: Basic configuration of Solr with Docker on Plesk

2024-05-26 Thread Shawn Heisey
On 5/22/24 10:23, Jorge Sánchez Briantes wrote: Due to Plesk requirements, if you need Solr and Zookeeper in the same web server, they must be installed on a Docker container. I have deployed it and I have a fresh Solr 9.6, and Zookeeper 3.9.2., running. I have readthis page

Re: Cannot Run Solr as the Solr User

2024-05-26 Thread Shawn Heisey
On 5/22/24 10:16, Lee Daniel wrote:  * Upon installing Apache Solr, a user */solr/* was automatically created. o Running Solr through from the root is not recommend and will    terminate unless `-force` is used.  * If I switch accounts `su solr`/*, */run `/opt/solr/bin/solr start`,

Re: SolrCloud behavior when Zookeeper has lost a quorum.

2024-05-20 Thread Shawn Heisey
On 5/20/2024 17:46, mtn search wrote: Will a SolrJ client connection (configured with a ZKhost list) to a SolrCloud still work for querying a SolrCloud when the Zookeeper ensemble has lost its quorum, but some ZK nodes remain up? I understand that indexing will not work. Follow-on - will the So

Re: Solr parsing query: Non ended comments inside comment does not work as expected

2024-05-20 Thread Shawn Heisey
On 5/14/24 06:40, Runar Espeland wrote: Solr parsing non-ended comments inside comment, does not work as expected. If there are non-ended comments inside a comment. Then it will treat trailing words as comments Examples samsung /* Motorola iphone */ -- > works. As expected samsung /* motorola

Re: Problems migrating solrj to run with SpringBoot 3

2024-05-20 Thread Shawn Heisey
On 5/16/24 10:40, Todd Stevenson wrote: I'm in the process of upgrading a project that uses SolrJ to run with Spring Boot 3. I've been able to keep the current version of my jars after migrating to SpringBoot 3. SolrJ is working in some cases, but I've run into a problem where it is missing

Re: How to fix unstable solr cluster

2024-05-20 Thread Shawn Heisey
On 5/12/24 10:10, Sarthak Sharma wrote: Hi, We have a production solr cluster setup with 4 shards and 4 replicas on a legacy stack. Same machines have been used to host 5 Zookeeper nodes ensemble. Solr version : 1.1.0.41 ZK version : 3.4.6 I saw in a later message that the Solr version is act

Re: Zookeeper 3.8.3 FAILED TO START.... but why?

2024-05-18 Thread Shawn Heisey
On 5/15/2024 15:37, Peter Johnson wrote: Bottom-line, I want the bare minimum Zookeeper/Solr installation possible on 1 server (and not really interested in learning too much about it), I just need it as a requirement for OT TeamSite. With that in mind, I followed these basic steps (which had w

Re: Out of memory Error - Solr version 8.11.2

2024-04-25 Thread Shawn Heisey
On 4/25/24 00:02, Anuj Bhargava wrote: Out of memory Error - Solr version 8.11.2 *The oom_solr.sh file contains the following* - You did not ask a question and did not provide any kind of error message. What can we help you with? Thanks, Shawn

Re: Solr 6 Replication Question

2024-03-16 Thread Shawn Heisey
On 3/16/2024 11:52, mtn search wrote: While following up on a request to verify replication within our Solr farm, I discovered a state that is puzzling. For a number of Solr 6 cores I see the master and the core replicating to the master having the same doc count, indexVersion number, and LastUp

Re: Solr 9.5.0 and solrj client

2024-03-13 Thread Shawn Heisey
On 3/11/24 08:37, Isabella Trevisan wrote: Hi, I am planning to migrate my solrcloud from 8.11.3 to 9.5.0 . Our applications user SolJ to work with solr. I read on the upgrade Notes that I have to use Http2SolrClient instead of HttpSolrClient . But on Apache SolR 9.5 reference guide I read tha

Re: Migrating from solr8.11.3 to solr 9.5.0

2024-03-13 Thread Shawn Heisey
On 3/13/24 03:06, Isabella Trevisan wrote: Hi , I am plannig to migrate from solr 8.11.3 to 9.5.0 in solrcloud architecture. The idea is to create a new line of solrcloud 9.5 where to create the collections present in solrcloud 8.11.3 and reindex all the content on the new platform. My doubt

Re: HK2 service reification failed starting from Solr 9.5.0

2024-03-08 Thread Shawn Heisey
zOn 3/7/24 06:00, Yvan Zo wrote: I’m trying to upgrade from Solr 9.4.0 to 9.5.0, and I’m now getting the following WARN message in logs when starting the server. I use a custom XML response writer that imports classes from both ` jakarta.xml.bind` and `javax.xml.datatype`. It is built throug

Re: issue starting example core in new 8.11 install

2024-03-08 Thread Shawn Heisey
On 3/7/24 17:49, Scott Derrick wrote: I installed 8.11 and can access it via teh web interface. I created an example core in /var/solr/data/test test/conf     /data I used the solarconfig.xml and managed-schema files from /opt/solr/ example/files/conf When I use the core admin tool to add

Re: Solr 6 Replication Question

2024-03-08 Thread Shawn Heisey
On 3/7/24 16:22, mtn search wrote: Realized my question was not very clear. It pertains to when we bring the primary cores back up, after X number of days, how does Solr determine to do an standard replication of updates or a full replication of the set of index files. There is no "tipping poi

Re: Can a back up just a specific shard be made and rebuild just that shard and how?

2024-03-06 Thread Shawn Heisey
On 3/5/24 07:13, Aleksander Pähn wrote: Thank you for your reply! Do I understand correctly that you are talking about taking a shard from another replica node and moving/restoring to the problematic node? If you have a good replica of the shard in the cluster acting as leader for the shard,

Re: Can a back up just a specific shard be made and rebuild just that shard and how?

2024-03-01 Thread Shawn Heisey
On 3/1/2024 08:52, William Crowell wrote: I am using Apache Solr 7.7.2, and I need to rebuild just one specific shard. Can I take a backup of just one specific shard and rebuild just that shard? How would I do that? Is there any documentation for that? If you have a copy of one of the data d

Re: Upgrade from Solr 8.6 to 8.11

2024-02-22 Thread Shawn Heisey
On 2/21/24 12:58, Storch, Robin wrote: I am trying to upgrade our solr cluster from version 8.6 to 8.11.3. I have looked at https://solr.apache.org/guide/solr/latest/upgrade-notes/solr-upgrade-notes.html but this seems to cover only major version upgrades. Does anyone know how to upgrade w

Re: Need suggestions on performance improvement in Solr based application

2024-02-20 Thread Shawn Heisey
On 2/13/24 19:46, Fiz N wrote: Hi SOLR Experts, I am facing one challenge regarding performance in *Java Spring Boot + Apache Solr Cloud *based application. Details are as follows: *Data present in Solr collection*: 41 million *Shards:* 8 *Replicas:* 2 *Solr Version:* 8.2.0 There is not a

Re: CVE-2021-31684: SOLR- net.minidev_json-smart (Out of Bounds Read)

2024-02-13 Thread Shawn Heisey
On 2/13/2024 10:06, Shahryar Shagoshtasbi wrote: Thank you for your prompt response. Our scans have detected these CVEs in 9.1 and higher (At least the one we have tested). I’d highly appreciate if you could link me to the appropriate changelog for these changes. Solr 8.11.3 was announced onl

Re: Memory usage reported by Solr

2024-02-09 Thread Shawn Heisey
On 2/9/24 06:58, Ishan Chattopadhyaya wrote: Solr reports a memory usage of nearly 100%. This, I believe, is good as that indicates full memory utilization by MMapDirectory. In case Solr is saying that 100% of the 2GB heap memory is used, it is not a good thing. It can be very normal for the

Re: Dataimport - Optimize (Solr Version 8.11.2)

2024-02-01 Thread Shawn Heisey
On 2/1/24 08:09, Shawn Heisey wrote: To verify that what I say is true, manually kick an optimize off when you're NOT doing an import.  If your index is named XXX, this command would do it: curl "http://hostname:port/solr/XXX/update?optimize=true"; That command is not wo

Re: Dataimport - Optimize (Solr Version 8.11.2)

2024-02-01 Thread Shawn Heisey
On 1/31/24 21:59, Anuj Bhargava wrote: Why does *Optimize* in the *dataimport* section of solr Admin UI duplicate the records. A new copy is created of each record. "*id*" and "_*version*_" are different, everything else is the exact copy. Each time I run Optimize, the initial number of records g

Re: Keeping certain stored fields uncompressed

2024-01-26 Thread Shawn Heisey
On 1/26/24 03:38, Srijan wrote: Since upgrading to Solr 9.x, I've observed a drastic decrease in performance – approximately 10 to 20 times slower than before. And this stems from the fact that stored fields in Solr 9.x are now compressed. Decompressing these fields during each search query has i

Re: Optimizing Solr for low memory

2024-01-25 Thread Shawn Heisey
On 1/24/24 01:27, uyil...@vivaldi.net.INVALID wrote: Is there a general guideline to optimize Solr for very little number of documents in the core and low memory? For example, let's say 2000 documents and 100mb of memory. It crashes often due to OOM error with the default configuration. Are t

Re: SOLR data on ECS problem with write lock files

2024-01-22 Thread Shawn Heisey
On 1/18/24 09:08, uyil...@vivaldi.net.INVALID wrote: I had a very similar problem when running Solr on EKS kubernetes cluster. The solution I found was to add a pre_stop shutdown hook to the kubernetes deployment, which runs the command "/opt/solr/bin/solr stop -k solrrocks -p 8983" to graceful

Re: What I'm missing on this query (strange results with OR) ?

2024-01-10 Thread Shawn Heisey
On 1/10/24 15:22, Ing. Andrea Vettori wrote: Is it the expected behaviour that a query like someotherconditions OR -field:value returns different result than someotherconditions OR (-field:value) It seems to me that the first works like the OR is instead and AND. Welcome to the fun world of

Re: Arbitrary Range

2024-01-10 Thread Shawn Heisey
On 1/10/24 05:32, dario.v...@coop.ch wrote: But when you want to use this from java with the solrj client I did not fond a good way to do this. https://solr.apache.org/guide/8_2/json-facet-api.html#range-facet What would be the preferred way to do this? At the URL you linked, it opens with a

Re: SolrJ with Spring Boot 3.2

2024-01-09 Thread Shawn Heisey
On 1/9/24 10:14, dario.v...@coop.ch wrote: Hello Everyone at Solr This Email is possibly a duplicate, but as I did not find my Mail appear in the archives the first time I'm sending it a second time. Today I wanted to bump the Spring Boot version from 3.1.7 to 3.2.1. With the first Version my

Re: SolrCloud availability when a server is down

2024-01-09 Thread Shawn Heisey
On 1/8/24 10:32, Facchin Umberto wrote: If server2 or server3 are down everything continues to work (even the updates), but if server1 is down then everything stops, obviously. Solr has no features to get around this. What you need is something that manages a virtual IP address among multipl

Re: Solr query using full heap and triggers stop the world pause

2024-01-04 Thread Shawn Heisey
On 1/3/24 13:33, rajani m wrote: Solr query with LTR as a re-ranker is using full heap all of sudden and triggering STW pause. Could you please take a look and let me know your thoughts? What is causing this? The STW is putting nodes in an unhealthy state causing nodes to restart and bringi

Re: Correct Procedure for making Basic Authentication Changes LIVE across Multiple Nodes in Solr Cloud 8.10

2023-12-29 Thread Shawn Heisey
On 12/26/2023 00:19, Uday Kumar wrote: Surprisingly, *authentication is enabled on all nodes/shards* immediately after pushing the security.json file to Zookeeper. However, according to the Solr Documentation, a restart of each node is required for authentication changes to take effect across al

Re: Deletion on Solr Failing due to _version_ conflict

2023-12-20 Thread Shawn Heisey
On 12/19/2023 23:38, Saksham Gupta wrote: I have extracted the_version_ field using query: *fq=uniqueid:{some_value}&fl=_version_*, but upon running filter query using this same version I extracted; fq=_version_:{extracted_version} it is unable to find the document. This is the definition of th

Re: Deletion on Solr Failing due to _version_ conflict

2023-12-19 Thread Shawn Heisey
On 12/18/23 21:42, Saksham Gupta wrote: I have written a small script in python to delete a product from solr cloud collection on the basis of unique id, _route_ and _version_. I have extracted the exact values of unique id, _route_ and _version_ from solr index and used them to delete the produ

Re: Java Versions and GC Tuning

2023-12-14 Thread Shawn Heisey
On 12/14/2023 13:41, Walter Underwood wrote: Running this with Corretto 17 on my Apple Silicon MacBook: java -XX:+PrintFlagsFinal I see this line in the 500+ lines of output: bool UseZGC = false {product} {default} I

Re: Java Versions and GC Tuning

2023-12-14 Thread Shawn Heisey
On 12/14/23 13:04, Shawn Heisey wrote: On 12/14/23 12:58, Shawn Heisey wrote: On closer read, that list for Java 17 is applicable to Java itself, not necessarily ZGC. And on a third read, I figured out that the list DOES apply to ZGC. Apologies for the noise! They do have a nice clear

Re: Java Versions and GC Tuning

2023-12-14 Thread Shawn Heisey
On 12/14/23 12:58, Shawn Heisey wrote: On closer read, that list for Java 17 is applicable to Java itself, not necessarily ZGC. And on a third read, I figured out that the list DOES apply to ZGC. Apologies for the noise! Thanks, Shawn

Re: Java Versions and GC Tuning

2023-12-14 Thread Shawn Heisey
On 12/14/23 12:47, Shawn Heisey wrote: On 12/14/23 08:37, Walter Underwood wrote: Thanks for the recommendation. Are you running this on Intel or ARM64? We’ve mostly moved to ARM64. —wunder This is on x64. When ZGC was first released, it only worked on Linux/x64. In Java 17, the supported

Re: Java Upgrade process

2023-12-14 Thread Shawn Heisey
On 12/13/23 01:15, Jim Morgan wrote: So, just as a follow up, I'm going to post my BEFORE and AFTER solr.in.sh configs in case I've removed anything vital. Any comments welcome, as they don't really mean much to me. === BEFORE SOLR_HEAP="8000m" GC_LOG_OPTS="-verbose:gc -XX:+PrintHeapAtGC -X

Re: Java Versions and GC Tuning

2023-12-14 Thread Shawn Heisey
On 12/14/23 08:37, Walter Underwood wrote: Thanks for the recommendation. Are you running this on Intel or ARM64? We’ve mostly moved to ARM64. —wunder This is on x64. When ZGC was first released, it only worked on Linux/x64. In Java 17, the supported architectures are: Linux/x64 Lin

Re: Solr connection refused error

2023-12-14 Thread Shawn Heisey
On 12/13/23 22:25, Deepak Goel wrote: On Thu, 14 Dec 2023, 10:11 Anuj Bhargava, wrote: There are some files in /var/solr/logs. For example - solr_oom_killer-8983-2023-12-14_05_30_42.log and this contains - Running OOM killer script for process 2861377 for Solr on port 8983 Killed process 28

Re: Implementing Authentication for Solr Indexing and Querying in a Spring Boot Project

2023-12-12 Thread Shawn Heisey
On 12/11/23 01:48, Uday Kumar wrote: *1. Authentication for Indexing:* We are currently using the SolrJ client for indexing documents into the Solr index. However, with authentication enabled at Solr, we are uncertain about the best practices for implementing authentication within the SolrJ clie

Re: Solr connection refused error

2023-12-12 Thread Shawn Heisey
On 12/11/23 21:25, Anuj Bhargava wrote: http://xxx.xxx.xx.xxx:8983/solr/#/ I often get the following error - This site can’t be reached xxx.xxx.xx.xxx refused to connect. Try: Checking the connection Checking the proxy and the firewall ERR_CONNECTION_REFUSED 8983 port is open There's not a

Re: Java Upgrade process

2023-12-12 Thread Shawn Heisey
On 12/10/23 20:36, Jim Morgan wrote: Now we're looking at java. Its currently running on openjdk-8. I've tried upgrading to openjdk-11 and openjdk-13. The process, as this is Ubuntu is basically. Don't use Java 13. It is a non-LTS release. Non-LTS releases are only supported for 6 months, so

Re: how to fix full-import indexed document not shown on query search

2023-12-07 Thread Shawn Heisey
On 12/7/23 07:56, Vince McMahon wrote: { "responseHeader": { "status": 0, "QTime": 0 }, "initArgs": [ "defaults", [ "config", "db-data-config.xml" ] ], "command": "status", "status": "idle", "importResponse": "", "statusMessages": {

Re: pls help with the definitions in schema.xml and how to get a copy of files

2023-11-22 Thread Shawn Heisey
On 11/22/23 08:57, Vince McMahon wrote: Shawn, The guy left the job. And the pass word is gone. As I said before, if the schema with those ENTITY definitions works, then the files ARE on the system somewhere. If it's Linux or another UNIX-like system, running this as root should locate the

Re: Suggestions to improve Star queries latencies

2023-11-22 Thread Shawn Heisey
On 4/16/23 07:24, Rajani Maski wrote: What are your suggestions to improve star queries latencies? By star queries I mean "*:*" or single term queries having boost formulas (such as doc recency and many others) taking 10 or more seconds. It is a large collection with good compute resources, howe

Re: pls help with the definitions in schema.xml and how to get a copy of files

2023-11-22 Thread Shawn Heisey
On 11/20/23 20:22, Vince McMahon wrote: I need some help to clarify a schema.xml I am trying to find out where the fieldtype string is defined. I open the defination of the schema.xml from admin UI https://localhost:8984/solr/cookbookCSV/admin/file?wt=json&_=1700536107495&; file=schema.xml&cont

Re: Solr 8.11.2 - remote updates

2023-11-16 Thread Shawn Heisey
On 11/16/23 09:38, Saur, Alexandre (ELS-AMS) wrote: When I tried to delete a document I noticed this message in the logs: 16/11/2023, 16:35:04 ERROR x:..._shard3_replica_n59 ErrorReportingConcurrentUpdateSolrClient Error when calling SolrCmdDistributor$Req: cmd=delete{_version_=-17827353375

Re: G1GC not cleaning old gen

2023-11-16 Thread Shawn Heisey
On 11/16/23 00:14, bilal qureshi wrote: I'm running facet queries with G1GC, and Solr is crashing with OOM, not triggering GC. All caches are disabled in solrconfig.xml. The Java OutOfMemoryError exception can happen for resource depletions that are NOT memory. Until you know exactly what res

Re: Reload greyed out in 9.2 browser GUI

2023-11-15 Thread Shawn Heisey
On 11/15/23 08:10, Oakley, Craig (NIH/NLM/NCBI) [C] wrote: After upgrading from Solr8.11.1 to Solr9.2.1, the browser GUI (at hostname:port/solr/#/~collections/collection_name) has the "Reload" button greyed out and not functional ("Add Collection" and "Delete Collection" have also become greyed

Re: Delete large number of documents using UI

2023-11-15 Thread Shawn Heisey
On 11/14/23 08:59, Vince McMahon wrote: Ishan, within that UI, I have the choice to set "Commit Within". Is there a way to via the UI to pause all indexing operations? And how to resume, afterward? That just tells Solr that within the specified number of milliseconds, a commit will begin. I

Re: How to do fastest loading and indexing

2023-11-12 Thread Shawn Heisey
On 11/11/2023 17:30, Vince McMahon wrote: From the Solr UI, how can I tell the number of threads are set for indexing ? If you are not using threads (or multiple processes) in your program to send data to Solr, then the indexing is single-threaded. Thanks, Shawn

Re: is it in mill-sec or seconds

2023-11-12 Thread Shawn Heisey
On 11/11/2023 19:46, Vince McMahon wrote: Hi, I would like to find out the unit of the QUERY./dataImport.totalTime. Is it in mill-seconds or seconds? image.png The image you attached did not make it. The mailing list eats attachments. All times that I have seen Solr report are in milli

Re: How to do fastest loading and indexing

2023-11-11 Thread Shawn Heisey
On 11/11/2023 12:32, Vince McMahon wrote: What is the fastest way to load and index this large and wide CSV file? It is taking too long, 20+ hours, now. I am assuming here that you are sending the CSV data directly to Solr and letting Solr parse it into documents. If that is incorrect, pleas

Re: G1GC not cleaning old gen

2023-11-10 Thread Shawn Heisey
On 11/10/2023 06:09, bilal qureshi wrote: I'm using solr cloud 9.1.1 on solr cloud cluster start initial heap it takes around 400 MB out of 12 GB. After some queries on solr heap increases to 8 GB. And when I reload the collection it reduces to 7GB. using G1GC in this case. Running the same test

Re: Apach Solr Exercise 1 Index the Techproducts Data step not working

2023-11-04 Thread Shawn Heisey
On 11/4/2023 10:22, Shawn Heisey wrote: java -Dc=techproducts -Dauto -classpath server\solr-webapp\webapp\WEB-INF\lib\solr-core-9.4.0.jar org.apache.solr.cli.SimplePostTool example\exampledocs\* This command also works: bin\solr post -url http://localhost:8983/solr/techproducts/update

Re: Apach Solr Exercise 1 Index the Techproducts Data step not working

2023-11-04 Thread Shawn Heisey
On 11/4/2023 00:44, Adrien Grand wrote: Le sam. 4 nov. 2023, 01:58, Qizhi Zheng a écrit : *java -jar -Dc=techproducts -Dauto example\exampledocs\post.jar example\exampledocs\** But I always got the following error messages: *Error: Could not find or load main class org.apache.solr.util

Re: Suggestions to improve Star queries latencies

2023-11-02 Thread Shawn Heisey
On 11/1/23 20:01, rajani m wrote: The search query "http://host:8983/solr/v9/select?&q=*&rows=10"; consistently took ~500 ms. With "distrib=false" all the 96 shards have QTime 0-25 ms. Does this mean aggregation of results from all the shards is taking ~475 ms? I also tried shards.rows=5 and i

Re: where is the

2023-10-27 Thread Shawn Heisey
On 10/27/23 06:46, Vince McMahon wrote: in our environment, I can't get a shell to make changes . I am finding ways to make the shema change and do a proof of concept. how can I find more examples of solr schema api? i can do POST via curl. But, it seems POST has length limitations. Solr's de

Re: Question on "useColdSearcher"

2023-10-26 Thread Shawn Heisey
On 10/26/23 08:36, Surya R wrote: That was my assumption that some queries were taking longer time during the warmup. That is why I turned off the warmup count and made it to zero, but still the slow behavior. Can you restart Solr and provide the entire solr.log file? Maybe that will have som

Re: Question on "useColdSearcher"

2023-10-25 Thread Shawn Heisey
On 10/25/23 09:05, Surya R wrote: I started playing with this setting " *false* " in the solr config.xml, when i made it "*true*", which implies to use the cold searcher, I get response immediately after a core restart. But if i leave the default, which is "*false*" it takes a long time. It s

Re: SolR 9.4.0 and zookeepr version

2023-10-25 Thread Shawn Heisey
On 10/25/23 05:45, Isabella Trevisan wrote: which version of zookeeper is certified / suggested for solr 9.4.0 ? Solr 9.4.0 comes with the ZK 3.9.0 client. The ZK project guarantees that 3.9.x will work with 3.8.0 through 3.10.x. A wider version range might actually work, that is just th

Re: Performance issue in Wildcard Query in Solr 8.9.0

2023-10-24 Thread Shawn Heisey
On 10/23/23 05:00, Vishal Patel wrote: We want to search data which contains test. So, we are making our query doc_ref:*test*. I think wildcard query is taking high memory and CPU. Sometimes we faced issue that collection goes into recovery mode due to usage of wildcard query. Fo better perfor

Re: Solved/Working but Network Interface Question

2023-10-23 Thread Shawn Heisey
On 10/23/2023 8:18 AM, David Filip wrote: In theory I guess I could bind Solr only to the Ethernet network (it currently binds to 0.0.0.0, a.k.a. all network interfaces), but then I would need to set up a separate proxy from my WiFi network to each of the Solr servers, which I would rather not

Re: Newbie Help: Replicating Between Two SolrCloud Instances (Solr 9.2.1)

2023-10-23 Thread Shawn Heisey
On 10/20/2023 5:23 PM, David Filip wrote: From a matter of perspective, however, I think what I am not clear on is having more than one ZK “server”, and when and why I would need more than one? Perhaps it is just terminology, but if I have three (3x) Solr instances (cores) running on three (3

Re: SOLR 9.3 - JVM Crashes

2023-10-22 Thread Shawn Heisey
On 10/21/2023 2:31 AM, Ing. Andrea Vettori wrote: Hello, we’re using two SOLR servers (same hw, same version of solr and java, same solr config). The SOLR version is 9.3 and JVM is Adoptium JDK 17.0.8.1 on Linux. They both were running fine since a couple years (we upgraded from SOLR 8 to 9 wi

Re: Newbie Help: Replicating Between Two SolrCloud Instances (Solr 9.2.1)

2023-10-20 Thread Shawn Heisey
On 10/20/23 13:34, David Filip wrote: Understand about redundancy and needing an odd number of nodes (I’ve used quorum in other (non-Solr) type of clusters, so I get it). So what I’ve done now is installed ZooKeeper on a separate (physical) node (so no longer using ZooKeeper bundled with Solr,

Re: Newbie Help: Replicating Between Two SolrCloud Instances (Solr 9.2.1)

2023-10-20 Thread Shawn Heisey
On 10/19/23 17:48, David Filip wrote: I think I am getting confused between differences in Solr versions (most links seem to talk about Solr 6, and I’ve installed Solr 9), and SolrCloud vs. Standalone, when searching the ’Net … so I am hoping that someone can point me towards what I need to d

Re: Solr 8.11.2 Admin Panel

2023-10-18 Thread Shawn Heisey
On 10/18/23 06:25, Anuj Bhargava wrote: Hi Jan, Using the following to access - http://xxx.xxx.xx.xx:8983/solr/#/ Getting the message - This site can’t be reached xxx.xxx.xx.xx took too long to respond. Try: Checking the connection Checking the proxy and the firewall ERR_CONNECTION_TIMED_OUT P

Re: A general question about update ordering

2023-10-12 Thread Shawn Heisey
On 10/12/23 13:26, Chris Hostetter wrote: i think you're missunderstanding rules of _version_ and how '0' is handled. I'm about 90 percent sure the update requests are not being sent with the _version_ field populated. I am verifying this right as quickly as I can. Thanks, Shawn

Re: Soft commit takes 5 seconds in Solr 8.9.0

2023-10-10 Thread Shawn Heisey
On 10/10/23 05:05, John Jackson wrote: Now for the 20 questions: What OS do you have this running on? --> centos How much physical memory is in the machine? --> 370 GB Is it running anything other than Solr? --> Yes, we have 3 Wildfly applications (each has 50GB memory) and 2 microapps (each has

Re: Timeout on LTR query causing out of memory

2023-10-09 Thread Shawn Heisey
On 10/8/23 18:24, rajani m wrote: I have filed a Jira SOLR-17018 explaining the issue and think it is just a certain type of queries causing the issue and not any bug, but still looking into it to see if anything else. You need to find out what

Re: Soft commit takes 5 seconds in Solr 8.9.0

2023-10-07 Thread Shawn Heisey
On 10/5/23 23:02, John Jackson wrote: Hello, Kindly guide/help us on this. I already shared the requested files. There's nothing that jumps out at me in the config. You should configure caches ... it won't help with commit time but it is likely to speed up queries unless every query your Sol

Re: SolrCloud sharding. Replicas on different AZ

2023-10-04 Thread Shawn Heisey
On 10/4/23 03:58, Sergio García Maroto wrote: I am trying to migrate from a SolrCloud Cluster which has 1 shard and replication factor of 3. Each server is on a different Availability Zone (AZ) on AWS. In prod today we can run on 1 AZ even if the other 2 can be down. Something to keep in mind:

Re: Some nodes show high query latency

2023-09-29 Thread Shawn Heisey
On 9/29/23 13:28, rajani m wrote: What could cause some nodes in a cluster to have high query latency when compared to the rest? It is possible that some of the nodes have either handled zero queries since the last reboot, or that they have handled fewer queries than the others, and as

Re: Soft commit takes 5 seconds in Solr 8.9.0

2023-09-29 Thread Shawn Heisey
On 9/29/23 02:57, John Jackson wrote: but our requirement NRT so we keep 100 milliseconds. 100 milliseconds is a completely unrealistic target for an automatic commit interval. On one Solr install I ran where each core had over 25 million documents, 10 seconds was a typical commit time, and

Re: Backup from old server and Restore to new server

2023-09-27 Thread Shawn Heisey
On 9/27/23 02:16, Jim Morgan wrote: OK, so I had some more time to work on this. I removed version 9.3 and tried with version 8.11. But still the same story: any of the API restore methods I try are rejected. I have also tried stopping solr, copying the whole /var/solr/data/collectionname direct

Re: A question about some behavior I am seeing

2023-09-26 Thread Shawn Heisey
On 9/26/23 14:06, Rahul Goswami wrote: I typically set the field value to null in an atomic update if I want to remove the field. Eg: If a document is soft deleted via a field "isVisible":false , then I would want to drop its "content" field by setting it to null (since it is large and not requir

A question about some behavior I am seeing

2023-09-26 Thread Shawn Heisey
In SolrJ 4.7.2, if an atomic update request is sent with some fields set to null (the object, not the string), then those fields will be ignored when processing the atomic update -- no change. In SolrJ 9, a field set to null in an atomic update request will be removed from the document. This

Re: A general question about update ordering

2023-09-25 Thread Shawn Heisey
On 9/25/23 10:24, Dmitri Maziuk wrote: (From your description, relying on the 4.x behaviour was a bad idea in the first place: what if "create" request e.g. hit a bad sector on a non-TLER drive and was stuck on i/o for 100x longer?) I agree that the way they are operating now is completely wro

Re: A general question about update ordering

2023-09-25 Thread Shawn Heisey
On 9/25/23 09:44, Walter Underwood wrote: Updates and indexing used to be single-threaded. A while ago (maybe in Solr 6.x?), a bunch of locking was removed so that indexing could use multiple CPUs. That is probably what you are seeing. I don’t think Solr has ever guaranteed ordering for update

Re: A general question about update ordering

2023-09-25 Thread Shawn Heisey
On 9/25/23 08:53, Dmitri Maziuk wrote: On 9/25/23 08:24, Shawn Heisey wrote: ... Solr 4.7 handles this as the customer wants, but Solr 9.1 doesn't. This sounds like a race condition, are you sure it's Solr doing the ordering and not e.g. the network stack? I have seen the reque

A general question about update ordering

2023-09-25 Thread Shawn Heisey
I have a general question about update ordering. Situation: Two indexing requests for the same uniqueKey value sent. One update "creates" the document, the other is an "update" with more fields populated. The "update" is sent by a different system than the "create" ... and it turns out that

Re: What is the process for increasing drive size

2023-09-20 Thread Shawn Heisey
On 9/20/23 06:59, Matias Laino wrote: Hello everyone, we recently upgraded our hard drives on our EC2 instances of our solrcloud, because we were running out of space, after doing that, all of our nodes stopped showing disk information, number of records per collection and solr is in general wo

Re: Restart on a node triggers restart like impact on all the other nodes in cluster

2023-09-14 Thread Shawn Heisey
On 9/14/23 07:16, rajani m wrote: Karl, I like your suggestion and will keep in mind for k8 cluster implementation. For now, interestingly, shards remain in recovery state until the first searcher listen

Re: Restart on a node triggers restart like impact on all the other nodes in cluster

2023-09-13 Thread Shawn Heisey
On 9/12/23 18:28, rajani m wrote: Solr 9.1.1 version, upon restarting solr on any node in the cluster, a unique event is triggered across all the *other* nodes in the cluster that has an impact similar to restarting solr on all the other nodes in the cluster. There is dip in the cpu usage, all

Re: Reg CVE 2021-44832

2023-09-06 Thread Shawn Heisey
On 9/5/23 23:10, ramkrishna vasudevan wrote: Now the tools that we run internally flags CVE-2021-44832 . I did not notice that this was cross-posted to both users and dev. I read dev first, replied, and then saw this message. This list (users

  1   2   3   4   5   6   7   8   >