Running Solr on AWS Lambda.

[Modassar Ather]

Hi,

I have a requirement of simple search and grouping of the result on the
index created using Solr-6.5.1 spread across 12 shards on a server outside
Solr context.
These searches can run on AWS Lambda using its resources to minimise the
load on the main Solr server with the index EBS volume attached to it.

I am trying to find the solution using either of the following ways:

   - By directly being able to call Solr APIs for searches and grouping on
   AWS Lambda.
   - By installing Solr on AWS Lambda where the same index can be attached
   and used for searching and grouping.

I think the simple solution will be if I can directly use the Solr APIs on
AWS Lambda but could not find Solr APIs to achieve it outside Solr context.
Please provide your comments if I can achieve it using any of the ways
mentioned above.

Thanks,
Modassar

Re: Running Solr on AWS Lambda.

[Gora Mohanty]

On Tue, 20 Jul 2021 at 15:05, Modassar Ather  wrote:

> Hi,
>
> I have a requirement of simple search and grouping of the result on the
> index created using Solr-6.5.1 spread across 12 shards on a server outside
> Solr context.
> These searches can run on AWS Lambda using its resources to minimise the
> load on the main Solr server with the index EBS volume attached to it.
>
> I am trying to find the solution using either of the following ways:
>
>- By directly being able to call Solr APIs for searches and grouping on
>AWS Lambda.
>

This should be possible. You will need to use one of the Solr client APIs,
installed outside of Lambda, and call the APIs from your Lambda functions.
Please see https://solr.apache.org/guide/8_9/client-apis.html


>- By installing Solr on AWS Lambda where the same index can be attached
>and used for searching and grouping.
>

Installing Solr on Lambda is probably not even possible, and certainly
inadvisable.

Regards,
Gora

Re: Running Solr on AWS Lambda.

[Modassar Ather]

Thanks Gora for your response.

The main reason for trying to get search and grouping run outside the
context of Solr is to avoid the load on Solr server.
Using a Solr client API will consume resources on the Solr server itself.
Please correct me if I'm wrong.

If it is possible to directly read an index from the EBSvolume and perform
a search and grouping on AWS Lambda using Lucene/Solr API then the load on
the Solr server will not increase.
I found Lucene APIs not supporting non-docValue fields for grouping.

Best,
Modassar






On Tue, Jul 20, 2021 at 4:00 PM Gora Mohanty  wrote:

> On Tue, 20 Jul 2021 at 15:05, Modassar Ather 
> wrote:
>
> > Hi,
> >
> > I have a requirement of simple search and grouping of the result on the
> > index created using Solr-6.5.1 spread across 12 shards on a server
> outside
> > Solr context.
> > These searches can run on AWS Lambda using its resources to minimise the
> > load on the main Solr server with the index EBS volume attached to it.
> >
> > I am trying to find the solution using either of the following ways:
> >
> >- By directly being able to call Solr APIs for searches and grouping
> on
> >AWS Lambda.
> >
>
> This should be possible. You will need to use one of the Solr client APIs,
> installed outside of Lambda, and call the APIs from your Lambda functions.
> Please see https://solr.apache.org/guide/8_9/client-apis.html
>
>
> >- By installing Solr on AWS Lambda where the same index can be
> attached
> >and used for searching and grouping.
> >
>
> Installing Solr on Lambda is probably not even possible, and certainly
> inadvisable.
>
> Regards,
> Gora
>

Re: Running Solr on AWS Lambda.

[Gora Mohanty]

On Tue, 20 Jul 2021 at 17:10, Modassar Ather  wrote:

> Thanks Gora for your response.
>
> The main reason for trying to get search and grouping run outside the
> context of Solr is to avoid the load on Solr server.
> Using a Solr client API will consume resources on the Solr server itself.
> Please correct me if I'm wrong.
>

Sorry, I seem not to have quite paid attention to that bit.


> If it is possible to directly read an index from the EBSvolume and perform
> a search and grouping on AWS Lambda using Lucene/Solr API then the load on
> the Solr server will not increase.
> I found Lucene APIs not supporting non-docValue fields for grouping.
>

This seems quite a bit more difficult to do, and while it should be
possible to do this I am afraid that you have gone beyond my familiarity
with Solr.

Regards,
Gora

Preserving Settings Across Upgrades?

[Scott Hollenbeck]

Can anyone share some thoughts on best practices for upgrading a solr node
and preserving my TLS settings and core configurations while doing so? My
existing installation has two cores with support for basic authentication
and TLS properly configured and working. In the past, I've done my upgrades
by running the service installation script (bin/install_solr_service.sh)
this way:

$ /opt/install_solr_service.sh /opt/solr-8.9.0.tgz -f -n

That installs the new release and re-creates a symlink to the new directory,
but I have to manually re-create the configurations for my cores, move the
keystore and security.json files, and re-edit bin/solr.in.sh to change the
TLS/SSL settings. There has to be a better way - what am I missing?

Scott

Re: Preserving Settings Across Upgrades?

[Eric Pugh]

Here are some of the things I do to facilitate upgrades:

1) Script out my steps using the various API’s available to Solr.   
2) Use SolrCloud, it makes life easier in decoupling where data is stored from 
the logical structure of my collections.
3) Adopts some of the “configuration as code” ideas in DevOps to manage all of 
those settings.

If you are open to Kubernetes, the https://solr.apache.org/operator/ 
 is also meant to make handling all those 
configuration values simpler to deal with.

Eric

> On Jul 20, 2021, at 8:09 AM, Scott Hollenbeck 
>  wrote:
> 
> Can anyone share some thoughts on best practices for upgrading a solr node
> and preserving my TLS settings and core configurations while doing so? Mya 
> existing installation has two cores with support for basic authentication
> and TLS properly configured and working. In the past, I've done my upgrades
> by running the service installation script (bin/install_solr_service.sh)
> this way:
> 
> $ /opt/install_solr_service.sh /opt/solr-8.9.0.tgz -f -n
> 
> That installs the new release and re-creates a symlink to the new directory,
> but I have to manually re-create the configurations for my cores, move the
> keystore and security.json files, and re-edit bin/solr.in.sh to change the
> TLS/SSL settings. There has to be a better way - what am I missing?
> 
> Scott
> 

___
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | 
http://www.opensourceconnections.com  | 
My Free/Busy   
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 


This e-mail and all contents, including attachments, is considered to be 
Company Confidential unless explicitly stated otherwise, regardless of whether 
attachments are marked as such.

Re: Multivalue field query to get first , second ... values of the field

[Natarajan, Rajeswari]

Thank you for your response.

-Rajeswari

On 7/19/21, 5:33 PM, "Yuval Paz"  wrote:

Hello Rajeswari,

As far as I know, Solr does not have a built in method to access a specific
index.

I can think in 3 ways to get the functionality you need:

1. Create a custom document transformer that return a specific index from a
multivalued field

2. Create a custom query function to do that

3. Store each value in its own field and add a copy field that unite them
all:

  
  
  
  
  

  
  
  
  


The third method is probably the best, as if you know the number of
elements, it is not really a multivalued field, but a search on a multiple
fields.


If you don't mind changing the score a bit, another solution is to create 4
fields(X_1/2/3/4), this time make them both indexed and stored, and create
a field alias that will combine them all, then you can search on the field
alias.

On Mon, Jul 19, 2021, 10:11 PM Natarajan, Rajeswari
 wrote:

> Hi,
>
> We have a multivakued field having always fixed number of values , 4 in
> each multivalued field
> For example
> 1st doc
> X = [“a” ,”ab
> “ , “abc” , abcd”]
>
> 2nd doc
> X=[“p”, “pq”, “pqr” , “pqrs”]
>
>
> Is there a way we can get  , “a” , “p”  as a query result , “ab” , “pq”
> as a query result   , based on the position in the list.
>
> Thanks,
> Rajeswari
>
>
>
>
>