Upgrade from Solr 1.4 to Solr 8.8

[shacky]

Hi,
I have a very old Solr installation (VERY old: Solr 1.4.1 running on Ubuntu
12.04) which I would like to upgrade to the latest Solr version 8.8.2 on
Debian Buster.

Will I be able to migrate the index or should I reindex all documents?

Thank you very much!
Bye

Re: Upgrade from Solr 1.4 to Solr 8.8

[Subhajit Das]

That is a really old version. Migration is not possible. Officially, indexes 
are compatible across 1 major version change. Even the config files are not 
guaranteed to work directly.
Would suggest, create everything from scratch with old files as reference. Then 
re-index.

On Apr 21 2021, at 12:31 pm, shacky  wrote:
> Hi,
> I have a very old Solr installation (VERY old: Solr 1.4.1 running on Ubuntu
> 12.04) which I would like to upgrade to the latest Solr version 8.8.2 on
> Debian Buster.
>
> Will I be able to migrate the index or should I reindex all documents?
> Thank you very much!
> Bye
>

Re: Upgrade from Solr 1.4 to Solr 8.8

[Puneet Pawaia]

Hi,

You'll need to reindex.

Upgrade of index is supported from one major version below.

Regards
Puneet

On Wed, 21 Apr, 2021, 12:32 shacky,  wrote:

> Hi,
> I have a very old Solr installation (VERY old: Solr 1.4.1 running on Ubuntu
> 12.04) which I would like to upgrade to the latest Solr version 8.8.2 on
> Debian Buster.
>
> Will I be able to migrate the index or should I reindex all documents?
>
> Thank you very much!
> Bye
>

Re: Upgrade from Solr 1.4 to Solr 8.8

[shacky]

Thanks.
What about upgrading major to major 1 to 2, 2 to 3, ... , 7 to 8?
Bye

Il giorno mer 21 apr 2021 alle ore 09:25 Puneet Pawaia <
puneet.paw...@gmail.com> ha scritto:

> Hi,
>
> You'll need to reindex.
>
> Upgrade of index is supported from one major version below.
>
> Regards
> Puneet
>
> On Wed, 21 Apr, 2021, 12:32 shacky,  wrote:
>
> > Hi,
> > I have a very old Solr installation (VERY old: Solr 1.4.1 running on
> Ubuntu
> > 12.04) which I would like to upgrade to the latest Solr version 8.8.2 on
> > Debian Buster.
> >
> > Will I be able to migrate the index or should I reindex all documents?
> >
> > Thank you very much!
> > Bye
> >
>

Re: Upgrade from Solr 1.4 to Solr 8.8

[Daniel Exner]

Hi everyone,

this was true for Solr up until 7.x

For 8.x you *need* to reindex.


Am 21.04.21 um 09:24 schrieb Puneet Pawaia:

Hi,

You'll need to reindex.

Upgrade of index is supported from one major version below.

Regards
Puneet

On Wed, 21 Apr, 2021, 12:32 shacky,  wrote:


Hi,
I have a very old Solr installation (VERY old: Solr 1.4.1 running on Ubuntu
12.04) which I would like to upgrade to the latest Solr version 8.8.2 on
Debian Buster.

Will I be able to migrate the index or should I reindex all documents?

Thank you very much!
Bye







OpenPGP_signature
Description: OpenPGP digital signature

Re: Upgrade from Solr 1.4 to Solr 8.8

[Puneet Pawaia]

Solr 1.4 is very old. There may have been breaking changes in between. Plus
you would need to install each major version and then upgrade. Could be
time consuming.
If you have the data, best would be to reindex.

Regards
Puneet

On Wed, 21 Apr, 2021, 13:02 shacky,  wrote:

> Thanks.
> What about upgrading major to major 1 to 2, 2 to 3, ... , 7 to 8?
> Bye
>
> Il giorno mer 21 apr 2021 alle ore 09:25 Puneet Pawaia <
> puneet.paw...@gmail.com> ha scritto:
>
> > Hi,
> >
> > You'll need to reindex.
> >
> > Upgrade of index is supported from one major version below.
> >
> > Regards
> > Puneet
> >
> > On Wed, 21 Apr, 2021, 12:32 shacky,  wrote:
> >
> > > Hi,
> > > I have a very old Solr installation (VERY old: Solr 1.4.1 running on
> > Ubuntu
> > > 12.04) which I would like to upgrade to the latest Solr version 8.8.2
> on
> > > Debian Buster.
> > >
> > > Will I be able to migrate the index or should I reindex all documents?
> > >
> > > Thank you very much!
> > > Bye
> > >
> >
>

Re: Upgrade from Solr 1.4 to Solr 8.8

[Puneet Pawaia]

We did an upgrade from 6 to 8.2 by installing an in-between 7 instance and
then doing a 6 to 7 upgrade and then a 7 to 8 upgrade.

On Wed, 21 Apr, 2021, 13:04 Daniel Exner,  wrote:

> Hi everyone,
>
> this was true for Solr up until 7.x
>
> For 8.x you *need* to reindex.
>
>
> Am 21.04.21 um 09:24 schrieb Puneet Pawaia:
> > Hi,
> >
> > You'll need to reindex.
> >
> > Upgrade of index is supported from one major version below.
> >
> > Regards
> > Puneet
> >
> > On Wed, 21 Apr, 2021, 12:32 shacky,  wrote:
> >
> >> Hi,
> >> I have a very old Solr installation (VERY old: Solr 1.4.1 running on
> Ubuntu
> >> 12.04) which I would like to upgrade to the latest Solr version 8.8.2 on
> >> Debian Buster.
> >>
> >> Will I be able to migrate the index or should I reindex all documents?
> >>
> >> Thank you very much!
> >> Bye
> >>
> >
>
>

Re: Upgrade from Solr 1.4 to Solr 8.8

[Alessandro Benedetti]

Hi,
"What about upgrading major to major 1 to 2, 2 to 3, ... , 7 to 8?"
I wouldn't recommend this time consuming activity, I am in line with other
comments:

1) design your solution from scratch on Solr 8.x using your 1.4 config as a
reference.
2) re-index (from your data source or worst case scenario, extracting it
from your old Solr)

Cheers
--
Alessandro Benedetti
Apache Lucene/Solr Committer
Director, R&D Software Engineer, Search Consultant

www.sease.io


On Wed, 21 Apr 2021 at 08:02, shacky  wrote:

> Hi,
> I have a very old Solr installation (VERY old: Solr 1.4.1 running on Ubuntu
> 12.04) which I would like to upgrade to the latest Solr version 8.8.2 on
> Debian Buster.
>
> Will I be able to migrate the index or should I reindex all documents?
>
> Thank you very much!
> Bye
>

Re: Upgrade from Solr 1.4 to Solr 8.8

[shacky]

 Thanks Alessandro.

1) design your solution from scratch on Solr 8.x using your 1.4 config as a
> reference.
>

I cannot change the schema at the moment, but I will be able to reconfigure
Solr 8 from scratch if this is what you mean.


> 2) re-index (from your data source or worst case scenario, extracting it
> from your old Solr)


I have data into my RDBMS, so I am able to reindex.
The only problem is that it is time consuming as well.

Cheers from Italy!
Bye

Re: Queries on solr cloud 8.7 take more than twice as long than same query with same documents on solr cloud 6.5.1

[Charlie Hull]

Hi Russell,

I don't think there's necessarily a magic bullet here, as in one setting 
that will solve the performance issues you're seeing. You need to break 
down which part of a slow query is costing the most - and as Alessandro 
suggests you need to do this carefully, starting with a stripped down, 
basic query and adding the other parts in until you see what causes the 
problem. There's a *lot* of things that can impact performance in 
something as complex as Solr.


You might find this post useful - it's some tools and techniques we used 
to help a client with ecommerce performance issues last year (just 
before Black Friday!) 
https://opensourceconnections.com/blog/2021/01/05/a-solr-performance-debugging-toolkit/


Best

Charlie

On 20/04/2021 17:38, Russell Bahr wrote:

Hi Charlie and Alessandro,

Thank you for your responses, yes, it is a monster query, and unfortunately, 
this is one of our smaller and faster performing queries. We were able to run 
this in solr4.10.4 significantly faster than in solr6.5.1, and solr6.5.1 is as 
you see significantly faster than solr8.7.0.

Our application relies heavily on the grouping and ngroups as well as faceting 
and we have looked into other ways of getting the results that we get from 
those but it would be a major rewrite to try to get those in a different way 
and is not something we can try to do at this time.

I have gone through the configs and the schema and believe I have made the 
correct changes in those for the defaults that changed through the updated 
versions but am wondering if there is something that I missed, or configured 
incorrectly in there that could be contributing to the slowness.  Does anyone 
have any suggestions to try in the schema.xml or maybe in solrconfig.xml.

Thanks,
Russ


From: Charlie Hull 
Organization: OpenSource Connections
Reply-To: "users@solr.apache.org" 
Date: Tuesday, April 20, 2021 at 6:40 AM
To: "users@solr.apache.org" 
Subject: Re: Queries on solr cloud 8.7 take more than twice as long than same 
query with same documents on solr cloud 6.5.1

Hi Russell,

Your complex Boolean query and the amount of new documents you're adding
every day remind me of the media monitoring applications I've seen for
Solr. The way these were made performant (for people like Bloomberg) was
to run them as a reverse search: e.g. to test each documents against a
set of stored queries, rather than running a set of queries over an
index. We built Luwak (now contributed as the Lucene Monitor) for this
very purpose. 
https://www.flax.co.uk/index.html@p=3058.html
 has a short
writeup.

As Alessandro has suggested you'll need to break down and figure out
which part of your big query is causing the performance issue, which may
lead you to a recent change in Solr, but I feel you also need to
consider whether your overall architecture is suitable for your use
case. I also wonder how maintainable your giant queries are and if
anyone really understands the interplay of the various boosts and
Booleans. If you've got lots of queries like this and if they're
something users can generate/modify at will that won't help either.

Best

Charlie

On 20/04/2021 00:37, Russell Bahr wrote:

We are trying to upgrade our solr clusters and are running into
performance issues when moving to the newer version. We are finding
that queries are taking more than double the amount of time to return
when provided with the same query and both clusters are indexed with
the same documents. The queries that we run are using booting as well
as groups and ngroups. Our original clusters were solr 4.10.4 and we
experienced a similar degradation going from 4.10.4 to 6.5.1 and were
able to get around that by splitting the work from one 30 server
cluster to two clusters one with 30 servers and a second with 35
clusters. It is not acceptable for us to more than double our clusters
again. I have gone through the documentation and made changes to the
schema and solr config as needed to upgrade but am still not sure why
we are experiencing such a dramatic degradation in performance.

We are indexing about 55 new documents daily and updating around
55 documents daily as well.

The clusters are configured as follows with schema and configs and a
sample query that I am running against both cluster examples.

Old cluster:
Solr 6.5.1
31 GB heap
OpenJDK 64-Bit Server VM - 1.8.0_275-8u275-b01-1~deb9u1-b01

Shards -6 Replicas -5

/18643418 docs/

/Avg size/doc: 2.6Kb/

http://localhost:8983/solr/content/select?indent=on&q=*:*&wt=json

{

   "*responseHeader*":{

 "*zkConnected*":true,

 "*status*":0,

 "*QTime*":2665,

 "*params*":{

   "*q*":"*:*",

   "*indent*":"on",

   "*wt*":"json",

   "*_*":"1618866411985"}},

   "*grouped*":{

 "*fingerprint*":{

   "*matches*":18728759,

   "*ngroups*":14295515,

   "*groups*":[{

…



New cluster: - this query takes about *10* times as long.
Sol

Migrate from Solr to SolrCloud

[shacky]

Hi all,
I am about to migrate from Solr 1.4 to Solr 8.8 as I described in my
previous message here.

The definitive plan is to also migrate from standalone installation to
SolrCloud with at least 3 nodes up and running (preferable multi-master, or
single master + 2 read only nodes).

I don't wish to install SolrCloud immediately because I would like to test
the application on Solr 8.8 standalone to be sure that it is completely
supported from libraries and so on.

After a set of tests I would like to migrate from the standalone
installation to SolrCloud on 3 nodes.

Will I have some difficulties to make this migration or do you think it's
best to directly install SolrCloud now and avoid passing through the
standalone installation?

Thank you very much!
Bye

Solr equivalent of relational joins on nested documents

[Alain Rogister]

Hi,

Imagine that you have been tasked with designing a Solr-based system that 
indexes and queries data aggregated from several relational databases. All the 
data items are ultimately related to a common object type, so the logical data 
schema looks like :

Customer
 Profile
 Events
 Interactions
   Steps
 ActionPlans
   Objectives
Actions
(and quite a few more, with up to 3 or 4 levels of nesting)

Each of these has a number of attributes. What you need to do is the Solr 
equivalent of a SQL query that would find all Customers where 
Profile.item1='...' and there is some Event where event.item1='..' and 
event.item2='..' and there is some ActionPlan that has an Objective where 
Objective.item3='..' and this Objective has an Action where Action.item1='..' 
and Action.item2='..'

This is fairly trivial in a relational DB. But is it achievable in Solr ? 
Needless to say, there is no way a flat schema can work here. So I have assumed 
that nested documents were the only way to go. I have created a schema that 
includes these prerequisites, where "docType" is meant to contain the document 
type i.e. Profile, Event, Interaction etc :

 
 
 
 
 
  
  
 id 

and I have used naming conventions on the other fields to reflect the 
structure, e.g.










I have been able to index my documents correctly, i.e. retrieving a Customer by 
id with fl=*,[child limit=999] returns the customer and all dependent objects 
in the correct hierarchy.

I have also been able to express queries on two levels, such as: find all 
Customers that have an Interaction with code='..' and result='..' and that have 
an Event with type='..' and date before '...'. The syntax looks like this, 
using two filter queries and block joins :

q= docType:Customer AND ()
fq= {!parent which=docType:Customer} +docType:Interaction +interaction.code:C01 
+interaction.result:COMPLETE
fq= {!parent which=docType:Customer} +docType:Event +event.type:EVTTYPE1 
+event.date:[NOW-1YEAR TO NOW]

That seems to return correct results. However, I can't quite figure out the 
proper way to take it one level deeper e.g. add filtering conditions on the 
Steps of the Interactions selected by the first filter query.

For the record, I am able to do so using streaming expressions, but these bring 
their own complexities so I am trying to find out first if something simpler 
can save me.

Thanks !

Re: hl.preserveMulti returns the entire field even if no match

[abhi Abhishek]

Hi Mathew,
Did you try hl.requireFieldMatch=true; this would force Highlighter to
only generate highlights on matched field's


Thanks,
Abhishek

On Sat, Apr 17, 2021, 13:02 Roth, Matthew  wrote:

> Hi List,
>
> I am using 7.7.3 and I have a query where I would like to use the original
> highlighter to look for matches in the text of the document, the title, or
> a multivalued tag field.
>
> When I use hl.perserveMulti it returns the entire contents of the field
> because I also am using an hl.fragsize of zero. I can get around this by
> specifying f.TAG_NAME.hl.perserveMulti. This will only apply the
> perserveMutli rule to the multivalued TAG_NAME field. However, it will
> return the entire contents of the field even when no match. Is there a way
> to only return contents when there is a match to hl.q?
>
> Ultimately, I can just ignore this and always grab the results of the
> highlighted field from the response. But I'd prefer to only do that if it
> contains a match.
>
> Thanks,
> Matt
>

Re: Performance difference between json.facet and non-json.facet

[Furkan KAMACI]

Hi Jae,

You can check Yonik's blog post: https://yonik.com/facet-performance/

Kind Regards,
Furkan KAMACI

On Mon, Apr 19, 2021 at 6:08 PM Jae Joo  wrote:

> For the simple facet by field, is there any performance difference between
> two?
>
> Jae
>

Re: BUILD FAILED - Solr 8 on Mac OS Catalina

[Phill Campbell]

I just went through my “junk email folder” and found lots of responses from a 
few days ago. I do not know why my email let some though and filtered others 
with the same subject and “from”, but it did. 

Thank you for those responses. I was feeling like the community wasn’t that 
concerned. I am glad that you all are concerned. Thank you.

> On Apr 20, 2021, at 12:19 PM, Phill Campbell  
> wrote:
> 
>  [mvn] 
> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
> org.apache.solr.client.solrj.impl.Http2SolrClient
>  [mvn]at 
> org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
>  [mvn]at 
> org.apache.solr.client.solrj.impl.Http2SolrClient.(Http2SolrClient.java:161)
>  [mvn]at 
> org.apache.solr.client.solrj.impl.Http2SolrClient$Builder.build(Http2SolrClient.java:840)
>  [mvn]at 
> org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:319)
>  [mvn]at 
> org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:51)
>  [mvn]at 
> org.apache.solr.core.CoreContainer.load(CoreContainer.java:684)
>  [mvn]at org.apache.solr.util.TestHarness.(TestHarness.java:184)
>  [mvn]at org.apache.solr.util.TestHarness.(TestHarness.java:156)
>  [mvn]at org.apache.solr.util.TestHarness.(TestHarness.java:162)
>  [mvn]at org.apache.solr.util.TestHarness.(TestHarness.java:112)
>  [mvn]at 
> org.apache.solr.SolrTestCaseJ4.createCore(SolrTestCaseJ4.java:812)
>  [mvn]at 
> org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:802)
>  [mvn]at 
> org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:663)
>  [mvn]at 
> org.apache.solr.velocity.VelocityResponseWriterTest.beforeClass(VelocityResponseWriterTest.java:41)
>  [mvn]at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>  [mvn]at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>  [mvn]at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  [mvn]at java.lang.reflect.Method.invoke(Method.java:498)
>  [mvn]at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
>  [mvn]at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
>  [mvn]at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
>  [mvn]at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>  [mvn]at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>  [mvn]at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>  [mvn]at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>  [mvn]at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>  [mvn]at 
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
>  [mvn]at 
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>  [mvn]at 
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>  [mvn]at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>  [mvn]at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>  [mvn]at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>  [mvn]at 
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>  [mvn]at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>  [mvn]at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>  [mvn]at 
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
>  [mvn]at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>  [mvn]at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>  [mvn]at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
>  [mvn]at java.lang.Thread.run(Thread.java:748)
>  [mvn] 
>  [mvn] 
> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
> org.apache.http.impl.client.InternalHttpClient
>  [mvn]at 
> org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
>  [mvn]at 
> org.apa

Question regarding output field alias name

[yaswanth kumar]

Is there an option in solr 8 where I can rename (alias) a field that is
part of fl= parameter?

So example fl=firstname , it should show the solr results with "name"
instead of "firstname"

-- 
Thanks & Regards,
Yaswanth Kumar Konathala.
yaswanth...@gmail.com

Re: Question regarding output field alias name

[Alexandre Rafalovitch]

Yes, if you are using eDismax, including expanding alias to multiple
fields internally:
https://solr.apache.org/guide/8_8/the-extended-dismax-query-parser.html#field-aliasing-using-per-field-qf-overrides

f.name.qf=last_name first_name

Another example:
https://github.com/arafalov/solr-indexing-book/blob/master/published/languages/conf/solrconfig.xml#L20-L21

Regards,
   Alex.

On Wed, 21 Apr 2021 at 17:54, yaswanth kumar  wrote:
>
> Is there an option in solr 8 where I can rename (alias) a field that is
> part of fl= parameter?
>
> So example fl=firstname , it should show the solr results with "name"
> instead of "firstname"
>
> --
> Thanks & Regards,
> Yaswanth Kumar Konathala.
> yaswanth...@gmail.com

WordDelimiter does not generate expected token

[gnandre]

Hi,

I have a field value as bim.ClassUnderlying and a search query as
classunderlying does not return any results. If I search for
classUnderlying, it works.What can I change so that it works for
classunderlying query too? If I change splitOnCaseChange value from 1 to 0
in index time analyzer chain, then it works but I don't want to do it
because I want to extract class and underlying tokens too from
classUnderlying word.

Following is my field type definition.

 
  <
filter class="solr.KStemFilterFactory"/> 
  

subscribe solr releases

[Basheer K]

to subscribe Announcements of apache solr releases

Best Regards,
Basheer K