Re: Restore the collection in solr 9.6.1 version where Backup is done in solr 7.2.1 version

2025-07-16 Thread Rahul Goswami
Solr 9.x will not open any indexes touched by <= Solr 7.x. So you might be out of luck with restoring the backed up index. This version limitation comes from the Lucene layer which powers the index capabilities of Solr. If access to the data is crucial, you could try restoring the index on a Solr

Re: Help with Empty Facets After Upgrading to Solr 9.8.1

2025-06-16 Thread Rahul Goswami
Schema v1.7 has a new behavior ==> docValues defaults to true, univertible defaults to false. From what I could track, this change happened in Solr 9.7. Since TextFields are anyway docValues=false, your behavior must be coming from the uninvertible=false default. You could consider reindexing and

Re: Combine Response Writers and cursorMark

2025-06-04 Thread Rahul Goswami
Can you please explain why the 2 calls? Are you not able to get the result the first time with wt=csv and cursorMark=* ? Rahul On Wed, Jun 4, 2025 at 10:45 AM James Baster < james.bas...@opendataservices.coop> wrote: > I know that when paging through a big set of results, using cursorMark is >

Re: Unexpected behaviour of solr.NumFieldLimitingUpdateRequestProcessorFactory

2025-05-09 Thread Rahul Goswami
Jason, I took a look at NumFieldLimitingUpdateRequestProcessorFactory and see the issue. Looks like a straightforward fix. I am happy to review the PR if it helps. Rahul On Fri, May 9, 2025 at 9:50 AM Andreas Mock wrote: > Hi Jason, > > thank you for your information. As being relatively new to

Re: Automatic upgrade of Solr indexes over multiple versions

2025-04-01 Thread Rahul Goswami
25 19:55:34 UTC-04:00 > > > That's interesting, but brings up the question of what happens if a node > (or the whole cluster) is rebooted in the middle of the process? > > On Mon, Mar 31, 2025 at 10:02 PM Rahul Goswami > wrote: > > > Some good points brought up in

Re: Automatic upgrade of Solr indexes over multiple versions

2025-03-31 Thread Rahul Goswami
. > > > > > > > > > It does come with a limitation that all *source* fields need to be > either > > > stored=true or docValues=true. Any copyField destination fields can be > > > stored=false of course, but as long as the source field (or in general, > > the > > > fields you care about preserving) is either stored or docValues true , > > the > > > tool can reindex in-place and legitimately "upgrade" the index. For > > indexes > > > where this limitation is not a problem (it wasn't for us!), this tool > can > > > remove a lot of operational headaches, especially in environments with > > > hundreds/thousands of very large indexes. > > > > > > > > > I had a conversation about this with some of you during "Apache > Community > > > over Code 2024" in Denver, and I could sense some interest. If this > > feature > > > sounds appealing, I would like to contribute it to Solr on behalf of my > > > employer, Commvault. Please let me know if I should create a JIRA and > get > > > the discussion rolling! > > > > > > > > > Thanks, > > > Rahul Goswami > > > > > > > > > > > > -- > http://www.needhamsoftware.com (work) > https://a.co/d/b2sZLD9 (my fantasy fiction book) > > >

Automatic upgrade of Solr indexes over multiple versions

2025-03-27 Thread Rahul Goswami
ealing, I would like to contribute it to Solr on behalf of my employer, Commvault. Please let me know if I should create a JIRA and get the discussion rolling! Thanks, Rahul Goswami

Dense vector search efforts in Solr

2025-02-19 Thread Rahul Goswami
I came across this JIRA where support for text-to-vector for a query was added: https://issues.apache.org/jira/browse/SOLR-17525 In the ticket, Alessandro mentions: "Other tasks I see coming: - UpdateRequestProcessor to add the indexing time support - performance testing - support for additional

Re: More information about copyField?

2025-02-18 Thread Rahul Goswami
The official documentation should be a good starting point for your questions : https://solr.apache.org/guide/solr/latest/indexing-guide/copy-fields.html Reading the above documentation should help answer some of them and provide a general structure to your understanding of copyFields. Overall,

Re: Using the NOT operator with the AND operator

2025-01-22 Thread Rahul Goswami
This is one of the idiosyncrasies of the NOT boolean operator. I prefer using the "-" operator for boolean NOT . The way to see it is that it behaves like a set operation where you give Solr a r

Re: Unofficial Apache Solr Developer Survey

2025-01-17 Thread Rahul Goswami
e, I'll share the results in full shortly. >> >> On Mon, 13 Jan 2025 at 01:01, Rahul Goswami >> wrote: >> >>> Interesting survey. I would like to see the results myself. Can’t see a >>> link which displays aggregated stats. Will they be shared anywhere

Relevance sorting in federated search

2025-01-17 Thread Rahul Goswami
Hello, Let's say I have a 4 shard Solr collection. When I query the collection, what is the default sorting order for aggregated results assuming the search returns several results from each shard? I understand when querying a single core, documents are sorted based on score. But while assessing t

Re: Unofficial Apache Solr Developer Survey

2025-01-12 Thread Rahul Goswami
Interesting survey. I would like to see the results myself. Can’t see a link which displays aggregated stats. Will they be shared anywhere, maybe later? -Rahul On Sun, Jan 12, 2025 at 10:10 AM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > Hi all, > I want to gather some feedback fro

Re: Keeping track of when a Segment Merge happens?

2024-09-08 Thread Rahul Goswami
Adding to what Shawn already mentioned, you could alternatively also set < infoStream>true in solrconfig.xml. You should find it already present in the section if your solrconfig is built off the techproducts config that ships as an example, or you could manually add it. That would allow Lucene lo

Re: SSL Enabling Certificate Error Message When Starting SOLR

2024-08-12 Thread Rahul Goswami
Looks like the hostname in your certificate doesn't match the hostname you are accessing Solr through. Eg: If you are accessing Solr as abc.myorg.com:8983/solr ,the certificate on abc.myorg.com should have the hostname correctly specified. Conversely if you know the hostname in the certificate you

Re: Error when create a new core

2024-07-31 Thread Rahul Goswami
Can you please provide a full stacktrace of the exception. My hunch is this is a ClassNotFoundException, but need to be sure. The typical class for "integer" type in Solr is "solr.IntPointField" (not "solr.IntField" which you are getting an exception for). Try that and the error should go away. I

Re: Terms query with filters

2024-07-29 Thread Rahul Goswami
I am assuming you are talking about the terms component and calling the /terms handler. There is no fq support for /terms handler at the moment. The Terms component works directly on the Terms index and does not have knowledge of live documents. So if you delete a document, and even if a new sear

Re: Help in testing Solr on Windows needed!

2024-07-26 Thread Rahul Goswami
file > http://home.apache.org/~epugh/solr/solr-cli-zk-args.zip and use that, it > was built off PR 2593. > > Please report any bugs, and ideally, if you can tweak the bin/solr.cmd > script to fix any issues, that would be huge > > > > On 2024/07/26 14:56:20 Rahul Goswa

Re: Help in testing Solr on Windows needed!

2024-07-26 Thread Rahul Goswami
Hey Eric, I am a prolific Windows user of Solr (several different clusters totalling 300+ nodes of Windows servers). I must mention though that we circumvent the use of cmd scripts to be able to directly launch the Jetty Java main class which starts the webserver and thus Solr. I am happy to help

Re: SOLR SSL Enabling

2024-07-02 Thread Rahul Goswami
:40, Rahul Goswami wrote: > > > > > Since you are using a self-signed certificate, the browser won't have a > > corresponding CA certificate for it and hence flags it. The connection is > > still encrypted as with usual SSL. > > wrong: > > >> I c

Re: SOLR SSL Enabling

2024-07-02 Thread Rahul Goswami
This is expected. When you visit any website, the server sends its certificate to the browser. The browser validates this certificate against a pre-loaded CA certificate issued by a trusted authority (Verisign, GoDaddy etc), Since you are using a self-signed certificate, the browser won't have a c

Re: SOLR 8.11.1 SSL Enable Failing

2024-05-18 Thread Rahul Goswami
I remember seeing that exact error. In my case it turned out to be an issue with the certificate. I was creating my own .p12 certificate by bundling the server cert and CA cert in the same .p12 file (which works) using OpenSSL , but can't seem to remember the exact issue with the certificate unfort

Re: Backtick character in field data breaks streaming query

2024-02-28 Thread Rahul Goswami
Submitted https://github.com/apache/solr/pull/2321 I can't assign reviewers (at least it seems so), so would be great if somebody could please take a look. Thanks. -Rahul On Tue, Feb 27, 2024 at 3:10 PM Rahul Goswami wrote: > Thanks. Submitted https://issues.apache.org/jira/browse/SO

Re: Backtick character in field data breaks streaming query

2024-02-27 Thread Rahul Goswami
Thanks. Submitted https://issues.apache.org/jira/browse/SOLR-17186 PR on the way! -Rahul On Tue, Feb 27, 2024 at 1:29 PM Gus Heck wrote: > On Tue, Feb 27, 2024 at 12:13 PM Rahul Goswami > wrote: > > > I can submit a fix for > > this. Should I open a JIRA? > > &

Re: Backtick character in field data breaks streaming query

2024-02-27 Thread Rahul Goswami
Thanks Christine. Unfortunately the change of parser doesn't seem to help either because the replacement of ` with " happens in the StreamExpressionParser itself ( https://github.com/apache/solr/blob/main/solr/solrj-streaming/src/java/org/apache/solr/client/solrj/io/stream/expr/StreamExpressionPars

Re: Backtick character in field data breaks streaming query

2024-02-26 Thread Rahul Goswami
Floating this once again in case anyone has any thoughts. Thanks, Rahul On Sun, Feb 25, 2024 at 11:51 PM Rahul Goswami wrote: > Hello, > I am running Solr 8.11.1 and running into an issue with stream api. Looks > like searches break when the data contains the backtick character(

Backtick character in field data breaks streaming query

2024-02-25 Thread Rahul Goswami
Hello, I am running Solr 8.11.1 and running into an issue with stream api. Looks like searches break when the data contains the backtick character( ` ). Eg: http://host-name:8983/solr/MyCollection/stream?expr=search(MyCollection,q="My_Field:Foto`s",fl="field1",qt="/export")

Re: Partial update slowness with a stored="false" dynamic field and lots of distinct field names

2024-02-20 Thread Rahul Goswami
Did you happen to change the DirectoryFactory in solrconfig to SimpleFSDirectoryFactory or NIOFSDirecotryFactory by any chance? Default is Mmap which is much more performant for atomic updates (and also practical, especially given the small(ish) size of your index). -Rahul On Tue, Feb 20, 2024 at

Re: Performance issue with solr 7.5.0 and docValues

2024-02-13 Thread Rahul Goswami
Can you pass debug=true with your query to find out which phase (query or faceting) takes more time? This is to eliminate chasing the wrong symptom to optimize for. -Rahul On Tue, Feb 13, 2024 at 3:48 AM Mikhail Khludnev wrote: > Hello, Arun. > Why don't you warm a new searcher with a query li

Re: [dev help wanted] /admin/segments handler: expose the term count

2024-01-30 Thread Rahul Goswami
Submitted PR: https://github.com/apache/solr/pull/2233 Open to further discussion on the JIRA/PR if more fields need to be added to the handler output. On Mon, Jan 29, 2024 at 8:03 AM Christine Poerschke (BLOOMBERG/ LONDON) < cpoersc...@bloomberg.net> wrote: > Wonderful. Please feel free to dir

Re: [dev help wanted] /admin/segments handler: expose the term count

2024-01-26 Thread Rahul Goswami
I would love to take this up. On Fri, Jan 26, 2024 at 6:46 AM Christine Poerschke (BLOOMBERG/ LONDON) < cpoersc...@bloomberg.net> wrote: > Hi Everyone, > > Have you used or are you curious about the segments info handler and/or > screen? > https://solr.apache.org/guide/solr/latest/configuration-g

Re: Solr authentication options

2023-12-17 Thread Rahul Goswami
ning it behind a proxy (nginx, varnish, > >> haproxy etc) and adding the authentication at that level. This also > means > >> you can use the proxy as an SSL terminator and so secure traffic. > >> > >>> On Sat, Dec 16, 2023 at 3:03 AM Rahul Goswami > w

Solr authentication options

2023-12-15 Thread Rahul Goswami
Hello, I am using Solr 8.11.1 in standalone mode and need to implement authentication for API calls. I went over the below documentation page to evaluate my options: https://solr.apache.org/guide/8_11/securing-solr.html JWTPlugin seems to be something which could be a good option. But it mentions

Re: A question about some behavior I am seeing

2023-09-26 Thread Rahul Goswami
chain in the solrconfig. This processor would check for such null fields in the solr doc and remove them altogether before it makes it further down the chain. -Rahul On Tue, Sep 26, 2023 at 5:56 PM Shawn Heisey wrote: > On 9/26/23 14:06, Rahul Goswami wrote: > > I typically set the fi

Re: A question about some behavior I am seeing

2023-09-26 Thread Rahul Goswami
Hi Shawn, I typically set the field value to null in an atomic update if I want to remove the field. Eg: If a document is soft deleted via a field "isVisible":false , then I would want to drop its "content" field by setting it to null (since it is large and not required anymore) in an atomic update

Re: [Solr] Reindexing leaving behind 0 live doc segments

2023-09-10 Thread Rahul Goswami
cessing to finish before reclaiming the space, especially when some of the cores are 3-4 TB large. Thanks, Rahul On Sat, Sep 2, 2023 at 4:45 PM Shawn Heisey wrote: > On 9/1/23 16:30, Rahul Goswami wrote: > > Thanks for your response. To your question about locking, I am not doing > >

Re: [Solr] Reindexing leaving behind 0 live doc segments

2023-09-01 Thread Rahul Goswami
rches working fine, except that these 0 live doc 7.x segments get left behind and cause index bloat. Thanks, Rahul On Thu, Aug 31, 2023 at 10:22 PM Shawn Heisey wrote: > On 8/31/23 14:45, Rahul Goswami wrote: > > I am trying to execute a program to read documents segment-by-segment and >

[Solr] Reindexing leaving behind 0 live doc segments

2023-08-31 Thread Rahul Goswami
Hello, I tried floating this question on the Lucene list as well, but thought the answer could also come from Solr's handling of IndexReader. Hence posting here. I am trying to execute a program to read documents segment-by-segment and reindex to the same index. I am reading using Lucene apis and

Re: Re-index after upgrade

2023-08-28 Thread Rahul Goswami
Yep, that check is present in Lucene 9.x as well. It will refuse to open an index created in 7.x. https://github.com/apache/lucene/blob/releases/lucene/9.4.0/lucene/core/src/java/org/apache/lucene/index/SegmentInfos.java#L345 https://github.com/apache/lucene/blob/releases/lucene/9.4.0/lucene/core

Re: Re-index after upgrade

2023-08-24 Thread Rahul Goswami
Even if you delete all docs using deleteByQuery api and start indexing using the code in your email, index will still identify as being created in 7.x. So as Shawn mentioned, the only solution is to bring down the solr service, delete the "index" and "tlogs" folder, bring up the service again and r

Re: Slow softCommits under heavy load?

2023-07-23 Thread Rahul Goswami
mes in the > config XML near various LRU and fastLRU cache definitions. Not seeing > specific queries defined in any XML. > > Regards, > Koen > > On Sun, Jul 23, 2023 at 7:10 PM Rahul Goswami > wrote: > > > “The application in question was creating a document per in

Re: Slow softCommits under heavy load?

2023-07-23 Thread Rahul Goswami
“The application in question was creating a document per interaction and doing a soft commit at the end of the interaction.“ You also mentioned your autoSoftCommit interval is 1 sec. If you really need NRT, I would suggest the client stop sending a softCommit upon each insert since the (extremely)

Re: Atomic updates too slow in Solr 8 vs Solr 7

2023-06-05 Thread Rahul Goswami
> > > Once we have a Solr JIRA, it may make sense to open a mail thread on > > users@lucene list to get some Lucene expertice involved in digging. > > > > Jan > > > > > 1. jun. 2023 kl. 18:15 skrev Rahul Goswami : > > > > > > So I ran the test

Re: Atomic updates too slow in Solr 8 vs Solr 7

2023-06-01 Thread Rahul Goswami
.. > > Jan > > > 31. mai 2023 kl. 23:50 skrev Rahul Goswami : > > > > Sure, I can do that. Let me create an index with a few million docs, call > > RTG with a few million iterations on it and note the times between 7.x > and > > 8.x. I assume this should be s

Re: Atomic updates too slow in Solr 8 vs Solr 7

2023-05-31 Thread Rahul Goswami
ers of magnitude slower in > 8.x than 7.x and is the main culprit. Then we could isolate the testing to > RTG only and not involce Atomic Update? > > Jan > > > 31. mai 2023 kl. 21:33 skrev Rahul Goswami : > > > > I don’t have any nested documents. And the results are

Re: Atomic updates too slow in Solr 8 vs Solr 7

2023-05-31 Thread Rahul Goswami
t; Would be interesting with a comparison with profiling. RTG gets the > > document from transaction log I believe? Could there be some RTG changes > in > > 8.x that caused such slowdown? > > > > Jan Høydahl > > > > > 31. mai 2023 kl. 16:57 skrev Rahul Goswami : &g

Re: Atomic updates too slow in Solr 8 vs Solr 7

2023-05-31 Thread Rahul Goswami
: > On 5/30/23 15:34, Rahul Goswami wrote: > > Environment details: - Java 11 on Windows server - Xms1536m Xmx3072m - > > Indexing client code running 15 parallel threads indexing in batches of > > 1000 - using SimpleFSDirectoryFactory (since Mmap doesn't quite work > &g

Re: Atomic updates too slow in Solr 8 vs Solr 7

2023-05-30 Thread Rahul Goswami
Adding another detail if it matters...all indexing is happening on standalone Solr on a single core. On Tue, May 30, 2023 at 5:34 PM Rahul Goswami wrote: > Hi, > We started experiencing slowness with updates in production after > upgrading from Solr 7.7.2 to 8.11.1. Upon comp

Atomic updates too slow in Solr 8 vs Solr 7

2023-05-30 Thread Rahul Goswami
Hi, We started experiencing slowness with updates in production after upgrading from Solr 7.7.2 to 8.11.1. Upon comparing the performance it turns out that indexing 20 million docs via atomic updates through the same client program (running 15 parallel threads indexing in batches of 1000) takes bel

Re: standard tokenizer seemingly splitting on dot

2023-05-04 Thread Rahul Goswami
Bill, Do you have a WordDelimiterFilterFactory in the analysis chain (with "*preserveOriginal" *attribute likely set to *0*)? That would split the token on the period downstream in the analysis chain even if StandardTokenizer doesn't. -Rahul On Thu, May 4, 2023 at 6:22 AM Mikhail Khludnev wrote:

Re: SOLVED(-ish) Re: SolrCore is loading

2022-10-24 Thread Rahul Goswami
That’s kind of strange that the EOFException doesn’t occur with the change to LocalSystem. But glad it works anyway. For what it’s worth, we run Solr on Windows as a service under the Network Service account which is less privileged than LocalSystem. Rahul On Mon, Oct 24, 2022 at 6:32 PM dmitri m

Re: SolrCore is loading

2022-10-24 Thread Rahul Goswami
Hi Dima, Do you observe anything unusual (blocked thread or something else) in the thread dump when this happens? Or can you share one? Maybe through jstack or the /solr/admin/info/threads endpoint. Rahul On Mon, Oct 24, 2022 at 5:17 PM dmitri maziuk wrote: > Hi all, > > I've a weird problem wi

Solr 7.7.2 going OOM

2022-09-28 Thread Rahul Goswami
Hi, I am running Solr 7.7.2 in standalone mode. 32 GB heap. Proces is throwing an OOM exception. Heap dump analysis shows ~14 GB of FrozeBufferedUpdates and 9 GB held across ~3000 DWPTs . There is a log replay in progress and another indexing thread. My understanding is that Solr flushes when accu

Re: Issue with Atomic update on boolean fields in Solr 8.11

2022-08-29 Thread Rahul Goswami
Created the below JIRA for the issue. https://issues.apache.org/jira/browse/SOLR-16360 -Rahul On Mon, Aug 29, 2022 at 11:29 AM Rahul Goswami wrote: > Dmitri, > I see this issue even when the document is an XML. I would like to note > that clients send data inside CDATA ( [CDATA[my da

Re: Issue with Atomic update on boolean fields in Solr 8.11

2022-08-29 Thread Rahul Goswami
hul On Sun, Aug 28, 2022 at 8:01 PM dmitri maziuk wrote: > On 2022-08-28 5:36 PM, Rahul Goswami wrote: > > Hi Dmitri, > > I am not sure if I understand your second comment. Can you please > elaborate? > > Try doing it in XML instead of JSON. JSON has data types encoded in the

Re: Issue with Atomic update on boolean fields in Solr 8.11

2022-08-28 Thread Rahul Goswami
clients send value 1 for some of the boolean fields and the behavior in 8.x breaks backward compatibility (in a possibly critical way. Saying “no” when you mean “yes” can have disastrous consequences). -Rahul On Sun, Aug 28, 2022 at 12:33 PM dmitri maziuk wrote: > On 2022-08-27 9:47 PM, Ra

Issue with Atomic update on boolean fields in Solr 8.11

2022-08-27 Thread Rahul Goswami
Hi, I am running Solr 8.11. As per the Solr documentation, any value starting with "1","t" or "T" for a boolean field is interpreted as true. https://solr.apache.org/guide/8_11/field-types-included-with-solr.html#recommended-field-types However, I hit a potential Solr bug where if the String valu

Re: Optimize not working, Solr version 9.1.0-SNAPSHOT

2022-08-05 Thread Rahul Goswami
I recently hit this problem on 8.11.1. It was a tiny test index with 2 segments. One of the segments *might* have been from 7.x. I wanted to optimize and rewrite the index into a single segment in 8.x . But optimize didn’t work. In this case though, before optimize, numDocs=maxDocs. So I thought

Re: After Solr 7.7 upgrade to Solr 8.11, there are two pieces of data in the same row, default overwrite is false?

2022-06-18 Thread Rahul Goswami
you'd require the _root_ field to be populated, otherwise not. On Sat, Jun 11, 2022 at 10:19 PM Rahul Goswami wrote: > This doesn't sound like the right behaviour. You should not need _root_ > field if you're not using nested documents. Can someone please confirm? > > On

Re: After Solr 7.7 upgrade to Solr 8.11, there are two pieces of data in the same row, default overwrite is false?

2022-06-11 Thread Rahul Goswami
This doesn't sound like the right behaviour. You should not need _root_ field if you're not using nested documents. Can someone please confirm? On Sat, Jun 11, 2022 at 12:29 PM 戴晓彬 wrote: > There is no _root_ field in the index data before the upgrade. The 8.x > version will judge whether it is

Failing Solr tests during build

2022-06-06 Thread Rahul Goswami
Hi, I am trying to build Solr 8.11.1 on Windows (using GitBash). Most tests execute fine, until I hit the S3 suite of testcases under contrib/s3-repository with the below exception: > Throwable #1: java.security.AccessControlException: *access denied ("java.io.FilePermission" "Z:\.aws\credentials"

Re: JDK version to build source

2022-06-03 Thread Rahul Goswami
Thanks Shawn, Dmitri. I have always used GitBash for building on Windows and it has never failed me. Just wanted to report that 8.11.1 builds fine too on Windows with GitBash. I was just curious to know more in case I am missing something. Thanks, Rahul On Fri, Jun 3, 2022 at 7:06 PM dmitri maziu

Re: JDK version to build source

2022-06-03 Thread Rahul Goswami
y not work on Windows." Thanks, Rahul On Fri, Jun 3, 2022 at 1:05 PM Shawn Heisey wrote: > On 6/3/2022 8:29 AM, Rahul Goswami wrote: > > I am trying to build 8.11.1 from source. Which JDK version should I use? > > I have successfully built it with OpenJDK 8 and OpenJD

JDK version to build source

2022-06-03 Thread Rahul Goswami
Hi, I am trying to build 8.11.1 from source. Which JDK version should I use? Thanks, Rahul

Re: Solr - frequent OOM

2022-05-03 Thread Rahul Goswami
Unfortunately in my experience, Solr doesn’t handle OOMs well and needs to be restarted. For example, if you have an indexing job going on or an expensive group by or collapse query, it will close the IndexWriter or IndexSearcher and the core is just defunct thereafter unless Solr is restarted. I w

Re: Stop a long running query

2022-05-02 Thread Rahul Goswami
, 2022 at 8:25 AM Mikhail Khludnev wrote: > Hello, Rahul. > I probably not fully get what you expect, but curiously I believe that > timeAllowed does what you expect. > Just clarify your context, please. > > On Fri, Apr 29, 2022 at 12:22 AM Rahul Goswami > wrote: > > &

Stop a long running query

2022-04-28 Thread Rahul Goswami
Hello, I am using Solr 7.7.2. Is it possible to stop a long running request ? Using the "timeAllowed" parameter would return partial results, but I want the query to outright terminate and ideally throw an exception so as to not utilize additional resources. Thanks, Rahul

Re: Problem with indexing a String field in SOLR.

2022-04-28 Thread Rahul Goswami
Neha, As Alessandro already mentioned, please share your schema if possible. A wild guess is that sometimes a field is defined as indexed=true stored=false which gives the impression that the document is missing the field. Taking a look at the schema would help clarify that. Thanks, Rahul On Thu,

Re: About IndexUpgraderTool stored=false fields

2022-02-19 Thread Rahul Goswami
No, that is not a requirement. If the index was created in 7.6 you should be fine. But do note that if the 7.6 index is a result of an upgrade from an earlier version and you are now upgrading to 8.11.1, Lucene will not allow you and the IndexUpgraderTool can't help you either. -Rahul On Sat, Feb

Re: Is there an easy way to determine Lucene versions for segments?

2022-01-02 Thread Rahul Goswami
ere could be inbuilt support for such upgrades without having a hard break in between. Thanks, Rahul On Fri, Dec 31, 2021 at 12:51 AM Shawn Heisey wrote: > On 12/30/2021 9:46 PM, Rahul Goswami wrote: > > What is the reason for blocking the upgrade ? If someone has been able to > >

Re: Is there an easy way to determine Lucene versions for segments?

2021-12-30 Thread Rahul Goswami
n Thu, Oct 7, 2021 at 9:36 AM Michael Conrad wrote: > >> >> >> On 10/7/21 8:46 AM, Rahul Goswami wrote: >> > Won’t work. I have tried optimize on 7.7.2 to 8.x where several segments >> > were originally written in 5.x and 6.x. >> > We are scratching o

Re: Log4J saga (CVE-2021-45046)

2021-12-15 Thread Rahul Goswami
We just upgraded to log4j2-2.16. It disables jndi lookups altogether by default. -Rahul On Wed, Dec 15, 2021 at 7:40 AM wrote: > Hi all, > > Looks like we are not done with log4j security problems. Someone has > recommendations about CVE-2021-45046? > > Eric Briere >

Re: Log4j vulnerability- Solr4 - urgent pls

2021-12-11 Thread Rahul Goswami
In case of solr4 which uses log4j-1.2.17.jar, the "log4j2.formatMsgNoLookups=true" system property is neither required nor applicable. In fact, the property was only introduced in log4j-2.10 (refer to the JIRA below). So not just Solr, but any Java application using 2<= log4j <2.10 will not be help

Re: Log4j vulnerability- Solr4 - urgent pls

2021-12-11 Thread Rahul Goswami
As pointed out by the author of log4j 1.x, the library is not susceptible to this attack the way log4j2 is. https://github.com/apache/logging-log4j2/pull/608#issuecomment-991380319 So you should be good. Rahul On Sat, Dec 11, 2021 at 9:51 PM Reej Nayagam wrote: > Hi All, > > In production we a

Re: Solr and CVE-2021-44228

2021-12-10 Thread Rahul Goswami
In addition to the mitigation strategies mentioned on the Solr page, the below blog post indicates that you should be protected if you are using Java 11.0.1 and up https://www.lunasec.io/docs/blog/log4j-zero-day/ On Fri, Dec 10, 2021 at 3:07 PM Mike Drob wrote: > Solr is affected. Please see th

Re: Number of cores per node

2021-11-23 Thread Rahul Goswami
at 7:54 PM Tulsi Das wrote: > > > Probably you can take a look at this jira : > > https://issues.apache.org/jira/browse/SOLR-1028 to get an idea. > > > > On Sat, Nov 13, 2021 at 1:29 AM Rahul Goswami > > wrote: > > > >> To clarify, I mean the number of Solr c

Re: Number of cores per node

2021-11-12 Thread Rahul Goswami
its animals are > > treated > > > - Mahatma Gandhi" > > > > > > +91 73500 12833 > > > deic...@gmail.com > > > > > > Facebook: https://www.facebook.com/deicool > > > LinkedIn: www.linkedin.com/in/deicool > > > > >

Number of cores per node

2021-11-12 Thread Rahul Goswami
Hi, Does anyone have benchmarks on performance as the number of cores on a Solr node goes up? I am trying to get an idea about how many cores per node is too much. Assume 31 GB heap size, SSD disk and 32 CPU cores. Preferably non-SolrCloud (aka standalone), but even if you have insights from SolrCl

Re: Receiving NPE during search.

2021-10-25 Thread Rahul Goswami
Can you please share the complete exception stack trace from the log ? - Rahul On Mon, Oct 25, 2021 at 2:12 PM Michael Conrad wrote: > We have run across an issue where certain searches result in a vague > NPE. Any help into pointing into a correct direction would be appreciated. > > ### Exampl

Any downsides to using RAFDirectory instead of SimpleFSDirectory ?

2021-10-23 Thread Rahul Goswami
Hello, I posted this question on the java-lucene group but not sure if my email was noticed/made it through. Hence posting here. Apologies if this is against the guidelines/best practices. I am running Solr on a Windows server and mmap doesn't quite work well there, so I have been using SimpleFS.

Re: Error solr

2021-10-20 Thread Rahul Goswami
Pasquale, Are you using any custom written components anywhere in your solrconfig? It's possible that in your custom code you are calling getCore() on a SolrQueryRequest object and then calling a close() on that core. Since getCore() doesn't increment the ref count of the core AND core.close() decr

Re: Upgrade Solr Segments: UpgradeIndexMergePolicy

2021-10-08 Thread Rahul Goswami
as possible upgrade of an older > index by only upgrading segments that are created by previous Lucene > versions. forceMerge does no longer really merge; it is just used to > "forceMerge" older segment versions away. > > > On 10/7/21 8:46 AM, Rahul Goswami wrote: > > W

Re: Is there an easy way to determine Lucene versions for segments?

2021-10-07 Thread Rahul Goswami
regularly run into multi-terabyte indexes. Easier said than done, given the unknown challenges in doing so, so the feasibility remains to be seen. I really wish there was a supported way to do this out of the box. On Thu, Oct 7, 2021 at 9:36 AM Michael Conrad wrote: > > > On 10/7/21 8:46

Re: Is there an easy way to determine Lucene versions for segments?

2021-10-07 Thread Rahul Goswami
*7.2.1 to 8.x (doesn’t matter anyway) On Thu, Oct 7, 2021 at 8:46 AM Rahul Goswami wrote: > > Won’t work. I have tried optimize on 7.7.2 to 8.x where several segments > were originally written in 5.x and 6.x. > We are scratching our heads to achieve this seamlessly since reindexing

Re: Is there an easy way to determine Lucene versions for segments?

2021-10-07 Thread Rahul Goswami
Won’t work. I have tried optimize on 7.7.2 to 8.x where several segments were originally written in 5.x and 6.x. We are scratching our heads to achieve this seamlessly since reindexing will take several weeks given the size of indexes for many of our customers. -Rahul On Thu, Oct 7, 2021 at 8:35

Re: Using collapse and expand results

2021-09-22 Thread Rahul Goswami
Scott, What is the data type of the field "author" in your schema? Collapse parser works only for string, int and float data types types -Rahul On Wed, Sep 22, 2021 at 1:33 PM Scott Derrick wrote: > > I was using result grouping but the docs recommend using collapse expand > > I tried this simp

CircuitBreaker for update requests

2021-09-04 Thread Rahul Goswami
Hello, The circuit breaker flow today hits only during searches. Curious to know if there was any specific reason why the update flow was omitted from invoking the circuit breaker? Bulk indexing jobs can also be a cause for high resource usage, potentially causing service degradation. Thanks, Rah

Re: Cpu 100%

2021-08-09 Thread Rahul Goswami
Jeff, Can you please share the output of this REST end point from the node where you are experiencing high CPU usage? Do this while you are seeing the high CPU usage. Preferably share 3 samples at intervals of 10 seconds . localhost:8983/solr/admin/info/threads Also, during which operation are yo

Re: facing issue on solr

2021-07-31 Thread Rahul Goswami
Do you mean Solr index backup? Are you using SolrCloud or standalone Solr? If index back up, refer to the documentation below. Change the version in the documentation link (8_9) to the Solr version you are using. https://solr.apache.org/guide/8_9/making-and-restoring-backups.html -Rahul On Sat,

Re: Quick Query Question: "body":""

2021-07-28 Thread Rahul Goswami
Minor edit: *if “body” field is indexed=true AND analyzed (i.e. Some text type; not of type “string”). On Wed, Jul 28, 2021 at 9:53 PM Rahul Goswami wrote: > If ‘body’ field is indexed=true, Shawn’s query should give you results > where body=“” as well as where body field doesn’t exist

Re: Quick Query Question: "body":""

2021-07-28 Thread Rahul Goswami
If ‘body’ field is indexed=true, Shawn’s query should give you results where body=“” as well as where body field doesn’t exist at all. Also, I agree that the format body:[* TO *] is much faster for high cardinality fields (which most likely “body” is). -Rahul On Wed, Jul 28, 2021 at 7:46 PM Shawn

Re: CVE-2021-27905 Apache Solr ReplicationHandler/SSRF vulnerability

2021-07-28 Thread Rahul Goswami
Digging out this old thread since I am looking for an answer to the same question. To Matthew's response above, since the /replication is an implicit handler, even if removed from solrconfig.xml, it would still work. I looked around (aka Googled) to find a way in which someone exploited this vulner

Re: Function query not working

2021-06-29 Thread Rahul Goswami
-- > Alessandro Benedetti > Apache Lucene/Solr Committer > Director, R&D Software Engineer, Search Consultant > > www.sease.io > > > On Fri, 25 Jun 2021 at 14:12, Rahul Goswami wrote: > > > Hello, > > I am using Solr 7.7.2. Bel

Re: Defense against deep paging?

2021-06-26 Thread Rahul Goswami
This begs a question...For anyone who has been burnt by the deep pagination issue in the past, what is a reasonable value of "start" param beyond which there is a noticeable performance degradation? Rahul On Fri, Jun 25, 2021 at 11:28 PM Walter Underwood wrote: > Cursors require keeping session

Re: Defense against deep paging?

2021-06-25 Thread Rahul Goswami
Is this deep pagination happening as a result of user action (eg: wanting to see what is on the 200th page of the result)? Or due to queries triggered in a loop trying to fetch results for some batch job ? If it's the latter, you could consider suggesting client code change to use streaming calls (

Function query not working

2021-06-25 Thread Rahul Goswami
Hello, I am using Solr 7.7.2. Below query is not returning expected results on the sample "techproducts" core. http://localhost:8983/solr/techproducts/select?debug=query&fq={!func}not(exists(popularity))&q=*:*&wt=json&rows=1 The query returns ALL documents in 'numFound' despite the 'not' function

Re: Troublshooting Solr lock up

2021-06-16 Thread Rahul Goswami
Forgot to mention, you need to collect the output while you observe the CPU spike. On Wed, Jun 16, 2021 at 11:38 AM Rahul Goswami wrote: > Also, can you please share the output of the below REST call from the > browser (any non-IE browser)? > > localhost:8983/solr/admin/info/thread

Re: Troublshooting Solr lock up

2021-06-16 Thread Rahul Goswami
Also, can you please share the output of the below REST call from the browser (any non-IE browser)? localhost:8983/solr/admin/info/threads Please replace the Solr port in the above call as per your environment. - Rahul On Wed, Jun 16, 2021 at 11:22 AM Dave wrote: > Out of curiosity do you hav

Re: Not getting results with escape character in search string

2021-06-04 Thread Rahul Goswami
Can you share the analyzer for the datatype you are using? If you are using WhitespaceTokenizerFactory as the tokenizer, "R-123" will be one whole token in which case you shouldn't need to escape the hyphen during search. -Rahul On Fri, Jun 4, 2021 at 9:05 AM yaswanth kumar wrote: > Can I get s

Re: update when no matching id

2021-05-17 Thread Rahul Goswami
ithin":1000,"overwrite":true},"timeout":1},"statusText":"Server > > Error","xhrStatus":"complete","resource":{"0":"[","1":"{","2":"\n","3":"\&q

  1   2   >