Issue with expand feature in collapse/expand result set workflow

[gnandre]

Hi,

I am using Solr 8.5.2 (non-SolrCloud) mode. It is a non-sharded scenario. I
am trying to collapse the resultset based on one of the string fields (say
X). Collapse query works as expected. It collapses the result set by
excluding the documents that have the same field X and retains only the top
raking document from each collapsed group.

Now, I want to expand the collapsed results to achieve similar
functionality like result grouping. However, when I add expand=true to
query, the returned expanded section doesn't include all collapsed groups.
It also doesn't include all collapsed documents for the collapsed groups
returned. Is there some issue with the 'expand' feature?

Re: Issue with expand feature in collapse/expand result set workflow

[Alexandre Rafalovitch]

Did you try expand.rows parameter for how many items to include in each group?
https://solr.apache.org/guide/8_7/collapse-and-expand-results.html

Regards,
   Alex.

On Thu, 20 May 2021 at 13:02, gnandre  wrote:
>
> Hi,
>
> I am using Solr 8.5.2 (non-SolrCloud) mode. It is a non-sharded scenario. I
> am trying to collapse the resultset based on one of the string fields (say
> X). Collapse query works as expected. It collapses the result set by
> excluding the documents that have the same field X and retains only the top
> raking document from each collapsed group.
>
> Now, I want to expand the collapsed results to achieve similar
> functionality like result grouping. However, when I add expand=true to
> query, the returned expanded section doesn't include all collapsed groups.
> It also doesn't include all collapsed documents for the collapsed groups
> returned. Is there some issue with the 'expand' feature?

Re: Issue with expand feature in collapse/expand result set workflow

[Michael Gibney]

Depending on what your goals/expectations are, it could also be worth
noting the `expand.q` and `expand.fq` params, which are applied when
fetching "expanded docs" (intersecting with the union of
`expand.field` result values). In cases where you want to "pivot" to
an unrestricted set of related clustered results, it may be helpful to
specify `expand.q=*:*`, or something similar.

On Thu, May 20, 2021 at 1:20 PM Alexandre Rafalovitch
 wrote:
>
> Did you try expand.rows parameter for how many items to include in each group?
> https://solr.apache.org/guide/8_7/collapse-and-expand-results.html
>
> Regards,
>Alex.
>
> On Thu, 20 May 2021 at 13:02, gnandre  wrote:
> >
> > Hi,
> >
> > I am using Solr 8.5.2 (non-SolrCloud) mode. It is a non-sharded scenario. I
> > am trying to collapse the resultset based on one of the string fields (say
> > X). Collapse query works as expected. It collapses the result set by
> > excluding the documents that have the same field X and retains only the top
> > raking document from each collapsed group.
> >
> > Now, I want to expand the collapsed results to achieve similar
> > functionality like result grouping. However, when I add expand=true to
> > query, the returned expanded section doesn't include all collapsed groups.
> > It also doesn't include all collapsed documents for the collapsed groups
> > returned. Is there some issue with the 'expand' feature?

geodist sort not working - release 8.8.2

[Adrián Cucó]

Hi ,

*Solr release 8.8.2, Ubuntu.
I am trying to run a query and getting  rows sorted by geodist and got the 
following error:

sort param could not be parsed as a query, and is not a field that exists in 
the index: geodist()

Quick example:

&q=*:*
&sfield=location
&pt=42.352455,-71.048069
&sort=geodist() asc

I am pretty sure I missed something but went throughout the docs and did not 
find any unusual.
Should I create a special index or something to use geodist()?

I guess I can run the same by using some condition like q=name:Peter , right?

Please kindly advise.

Thanks a lot

Regards
Adrian Cuco

Re: Issue with expand feature in collapse/expand result set workflow

[gnandre]

Alexandre, I did not because the default value of 5 was sufficient for the
number of results I am getting back. It is a result set of 7 documents
total and each group should have at max 2 documents in my case.
Groping feature (as opposed to collapse-expand feature) is working as
expected. I am not sure why collapse-expand feature is not working
correctly (especially expand, collapse is working fine)

Michael, I don't want to filter expand result set further. All I want is
basically un-collapse the collapsed result set. I tried expand.q=*:* just
for the experimentation.  There was definitely a change in 'expanded'
section but it was incorrect. There were only two documents with same y
value for X field in original result set but 'expanded' section shows 5
documents with value y grouped into one group. Some of those documents are
not even returned in original result set.

On Thu, May 20, 2021 at 1:46 PM Michael Gibney 
wrote:

> Depending on what your goals/expectations are, it could also be worth
> noting the `expand.q` and `expand.fq` params, which are applied when
> fetching "expanded docs" (intersecting with the union of
> `expand.field` result values). In cases where you want to "pivot" to
> an unrestricted set of related clustered results, it may be helpful to
> specify `expand.q=*:*`, or something similar.
>
> On Thu, May 20, 2021 at 1:20 PM Alexandre Rafalovitch
>  wrote:
> >
> > Did you try expand.rows parameter for how many items to include in each
> group?
> > https://solr.apache.org/guide/8_7/collapse-and-expand-results.html
> >
> > Regards,
> >Alex.
> >
> > On Thu, 20 May 2021 at 13:02, gnandre  wrote:
> > >
> > > Hi,
> > >
> > > I am using Solr 8.5.2 (non-SolrCloud) mode. It is a non-sharded
> scenario. I
> > > am trying to collapse the resultset based on one of the string fields
> (say
> > > X). Collapse query works as expected. It collapses the result set by
> > > excluding the documents that have the same field X and retains only
> the top
> > > raking document from each collapsed group.
> > >
> > > Now, I want to expand the collapsed results to achieve similar
> > > functionality like result grouping. However, when I add expand=true to
> > > query, the returned expanded section doesn't include all collapsed
> groups.
> > > It also doesn't include all collapsed documents for the collapsed
> groups
> > > returned. Is there some issue with the 'expand' feature?
>

SOLR Cel throws an error when parsing Images

[Charlie Hubbard]

Hi,

I'm using ExtractingRequestHandler and Solr CEL to extract data from
files.  And I'm getting an error when uploading images (jpeg, tiff,
png), but otherwise PDF, docs, xlsx etc have no problems.  So what's
happening is the server thinks the date (probably the archive_dt field
because that field is closest to the Date that is rejecting) isn't a
java.util.Date object so it falls into the DateMathParser because somewhere
it's converting the Date into a date string that can't be parsed by
DateMathParser, and kablowie.

I'm on Solr 6.6.6, and I'm using Solrj to communicate with the server.  I
believe I'm properly calling the server (ie using solr date format, etc).
Remember it's working just fine for PDFs and other document types without
issue.  And Dates are modeled the exactly same way for those documents.
The only thing different between the files that work and the files that
don't is the content of that file (doc vs image).  I'm not doing anything
fancy on the server either; this is pretty close to default configuration
when it comes to the ExtractingRequestHandler.

Any ideas on how to fix this would be very much appreciated.

thanks
Charlie

Here is the request and the exception that is happening in Solr:

2021-05-19 04:31:23.744 INFO  (qtp1543727556-21) [   x:igloo]
o.a.s.u.p.LogUpdateProcessorFactory [igloo]  webapp=/solr
path=/update/extract
params={literal.archiveDate_dt=2021-05-19T04:31:23.688Z&literal.categories=ONBEE&literal.categories=Uploaded&literal._type=document&literal._accountId=7&
literal.id=c308556b-e5f7-4d26-92d6-e6aa7648b2b2&version=2&literal._employeeNumber=PFC010001&literal.organization_id_i=114277&literal.org_level_1_name_s=XXX&literal.org_level_2_name_s=&literal._batchId=1038&literal.effectiveDate_dt=2019-01-29T05:00:00.000Z&literal._filename=015494_CKEQPQ0I8010.jpeg&wt=javabin{}
0 37

2021-05-19 04:31:23.746 ERROR (qtp1543727556-21) [   x:igloo]
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Invalid
Date String:'Wed May 19 04:31:23 +00:00 2021'
at
org.apache.solr.util.DateMathParser.parseMath(DateMathParser.java:234)
at org.apache.solr.schema.TrieField.createField(TrieField.java:644)
at org.apache.solr.schema.TrieField.createFields(TrieField.java:681)
at
org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:72)
at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:179)
at
org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:102)
at
org.apache.solr.update.DirectUpdateHandler2.updateDocument(DirectUpdateHandler2.java:922)
at
org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:913)
at
org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:302)
at
org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:239)
at
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:194)
at
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:67)
at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:979)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1192)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:748)
at
org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
at
org.apache.solr.handler.extraction.ExtractingDocumentLoader.doAdd(ExtractingDocumentLoader.java:126)
at
org.apache.solr.handler.extraction.ExtractingDocumentLoader.addDoc(ExtractingDocumentLoader.java:131)
at
org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:237)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)
at
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:724)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:530)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.se

TimeSeries Data

[Swapnil Saswade]

Hi,

We are using Routed Alias
. We had an
incident where there was a node in the cluster and we had to replace it.
After replacing the node we created the collections on the new node as
replicas. System is back operational but we are seeing the routed alias is
no longer creating a new collection based on the router.interval":"+1DAY"
and also the old collections are not getting dropped
-"router.autoDeleteAge":"/DAY-3DAYS". All the data is routed to the latest
collection. What would be the steps to check the logs and fix it
without losing the data.

Regards,
Swapnil.