Re: REST API

[Janvi T]

Hi, 
I think I misunderstood your suggestion. Definitely, solr works on documents 
and I have also put the data in documents. I got confused with "document". I 
meant we have tried putting JSON data into a text file and then indexed it. I 
was not taking in the context the document solr works upon.

Sorry for the confusion. What you were saying was right.

Thanks for your help.

On 2021/07/07 18:35:19, Christopher Schultz  
wrote: 
> Janvi,
> 
> On 7/7/21 10:08, Janvi T wrote:
> > We did that. Like get the data in the document and then index it in
> > solr. But the approach I was looking is for large data, because the
> > api I will be finally using will fetch a lot of data, it would not be
> > possible to put that data in document and then index it.
> 
> Solr works on "documents". If you can't put the data into a "document", 
> then Solr will not be able to index it, regardless of how large or small 
> it is.
> 
> If your "indexer program" cannot handle the size objects you intend to 
> handle, what makes you think Solr will be able to handle it?
> 
> -chris
> 
> > On 2021/07/07 13:43:14, Christopher Schultz  
> > wrote:
> >> Janvi,
> >>
> >> On 7/7/21 01:16, Janvi T wrote:
> >>> On 2021/07/07 04:37:28, Gora Mohanty  wrote:
>  On Wed, 7 Jul 2021 at 00:18, Janvi T  wrote:
> 
> > Hi everyone,
> > I wanted to know if we can fetch data from REST API can index the data 
> > in
> > solr. And if yes then how?
> >
> > I have an API that returns data in JSON format and I want to index that
> > data into solr. I read about Data import handler and how to use
> > URLDataSource but it does not seem to work with JSON data.
> >
> 
>  Please take a look at
>  https://solr.apache.org/guide/6_6/uploading-data-with-index-handlers.html
>  which has information on indexing from various formats. including JSON.
> >>   >
> >>> Hi, thanks for sharing but I am trying to get the JSON data from an
> >>> API. I have a REST API that returns JSON data and I want to use that
> >>> URL to get the data and then index it.
> >>
> >> Can you not write a small program to query your API and send the data as
> >> documents into Solr? You could even write it in Javscript and run it
> >> directly in a web browser.
> >>
> >> -chris
> >>
> 

Could not load collection from ZK (worked in solr4, failing in solr8) - Help Needed

[Reej M]

Hi All,

We are upgrading from solr 4 to solr 8. Updated the solr server to solr8.8.2, 
solrj8.8.2.
We are trying to resume the code and in one place where we are trying to get a 
collection from clusterstate. Code used is as below

Clusterstate clusterState = null;
For(SearchSolrZookeeper zk:livezks){
SolrZkClient zkClient = null;
ZKStateReader stateReader = null;
Try{
Zkclient = new SolrZkClient(zk.getUrl(), 3000);
stateReader = new ZKStateReader(Zkclient);
stateReader.createClusterStateWatchersAndUpdate();
clusterState = stateReader.getClusterState();
Connected = true;
Exception = null;
break;
}
Catch(){
}
Finally{
State reader.close();
Zkclient.close();
}

If(clusterstate!=null){
Doccollection dc = clusterstate.getCollection(“INVOICE”);
//here dc is null
}
}


Tried using getcollectionornull - returns null
Getcollectionornull(collection name, true) — returns null

When I debug I can see clusterstate has below values in it

Znodeversion:0
Live node:[192.123.33.44:3883_Solr, 
192.123.33.44:9883_Solr,192.123.33.45:3883_Solr,192.123.33.45:3883_Solr]
Collections:{ORDER=LazyCollectionRef(ORDER), INVOICE= 
LazyCollectionRef(INVOICE), PAYMENT= LazyCollectionRef(PAYMENT),etc..}

From the document collection we get the slices and iterate each replica to show 
the cores in a admin page. But this page worked fine with the code in solr4, 
now with solr8 it throws error. Kindly help. 

Regards
Reej

Microsoft.sqlserver.jdbc.SQLServerException on SOLR after upgrading SQL Server from 2012 --> 2019

[Paul, Lulu]

Hi Team,

Hi SOLR team,

Our .net project (currently running on solr-5.2.1) recently updated the DB end 
(from SQL Server 2012 to SQL Server 2019).
We made all the necessary changes with respect to app configs and SOLR configs 
- Changed data-config.xml, Restarted SOLR instance, re-started the server, 
Perform a full import on SOLR UI. But below is the error that gets flagged up 
on the Logging section of SOLR UI -

org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to 
execute query: select ID from

Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The driver could 
not establish a secure connection to SQL Server by using Secure Sockets Layer 
(SSL) encryption. Error: "SQL Server did not return a response. The connection 
has been closed.".
at 
com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:1368)
at com.microsoft.sqlserver.jdbc.TDSChannel.enableSSL(IOBuffer.java:1412)
at 
com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1058)
at 
com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:833)
at 
com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:716)
at 
com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:841)
at 
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:191)
at 
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:171)
at 
org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:440)
at 
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:308)


Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The driver could 
not establish a secure connection to SQL Server by using Secure Sockets Layer 
(SSL) encryption. Error: "SQL Server did not return a response. The connection 
has been closed.".

at 
com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:1368)

at com.microsoft.sqlserver.jdbc.TDSChannel.enableSSL(IOBuffer.java:1412)

at 
com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1058)

at 
com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:833)

at 
com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:716)

at 
com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:841)

at 
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:191)

at 
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:171)

at 
org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:440)

at 
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:308)

After some research it appeared that many users have faced TLS issues which can 
be the cause for such an error. I then tried to modify the config to force 
dataconfig.xml to accept TLSv1.
sslProtocol=TLSv1;encrypt=true;trustServerCertificate=true

The jdbc driver version is sqljdbc42

Unfortunately nothing seems to work. Any advice on what to do next ? Much 
appreciate, Thank you!

Best Wishes
Lulu



**
Experience the British Library online at www.bl.uk
The British Library's latest Annual Report and Accounts : 
www.bl.uk/aboutus/annrep/index.html
Help the British Library conserve the world's knowledge. Adopt a Book. 
www.bl.uk/adoptabook
The Library's St Pancras site is WiFi - enabled
*
The information contained in this e-mail is confidential and may be legally 
privileged. It is intended for the addressee(s) only. If you are not the 
intended recipient, please delete this e-mail and notify the 
postmas...@bl.uk : The contents of this e-mail must 
not be disclosed or copied without the sender's consent.
The statements and opinions expressed in this message are those of the author 
and do not necessarily reflect those of the British Library. The British 
Library does not take any responsibility for the views of the author.
*
Think before you print

Re: Could not load collection from ZK (worked in solr4, failing in solr8) - Help Needed

[Reej M]

> On 8 Jul 2021, at 7:00 PM, Reej M  wrote:
> 
> Hi All,
> 
> We are upgrading from solr 4 to solr 8. Updated the solr server to solr8.8.2, 
> solrj8.8.2.
> We are trying to resume the code and in one place where we are trying to get 
> a collection from clusterstate. Code used is as below
> 
> Clusterstate clusterState = null;
> For(SearchSolrZookeeper zk:livezks){
> SolrZkClient zkClient = null;
> ZKStateReader stateReader = null;
> Try{
>Zkclient = new SolrZkClient(zk.getUrl(), 3000);
>stateReader = new ZKStateReader(Zkclient);
>stateReader.createClusterStateWatchersAndUpdate();
>clusterState = stateReader.getClusterState();
>Connected = true;
>Exception = null;
>break;
> }
> Catch(){
> }
> Finally{
> State reader.close();
> Zkclient.close();
> }
> 
> If(clusterstate!=null){
> Doccollection dc = clusterstate.getCollection(“INVOICE”);
> //here dc is null
> }
> }
> 
> 
> Tried using getcollectionornull - returns null
> Getcollectionornull(collection name, true) — returns null
> 
> When I debug I can see clusterstate has below values in it
> 
Using SOLR8

> Znodeversion:0
> Live node:[192.123.33.44:3883_Solr, 
> 192.123.33.44:9883_Solr,192.123.33.45:3883_Solr,192.123.33.45:3883_Solr]
> Collections:{ORDER=LazyCollectionRef(ORDER), INVOICE= 
> LazyCollectionRef(INVOICE), PAYMENT= LazyCollectionRef(PAYMENT),etc..}

When Debugging with Solr4 clusterstate is as below
> Znodeversion:134
> Live node:[192.123.33.44:3883_Solr, 
> 192.123.33.44:9883_Solr,192.123.33.45:3883_Solr,192.123.33.45:3883_Solr]
> Collections:{ORDER(ORDER/clusterstate.json/134)={
“Replication factor”:”1”,
“Shards”:{“shard1”:{
Range:
“State”:”active”,
“Replicas”:{
“Core_node1”:{
“Core”:ORDER_shard1_replica1
“Base_url:
“Node_name”:”http://_solr ”
“State”:”active”
“Leader”:”true”},
“Core_node2”:{
“Core”:ORDER_shard1_replica1
“Base_url:
“Node_name”:”http://_solr ”
“State”:”active”
“Leader”:”true”}
}
}
> 
> From the document collection we get the slices and iterate each replica to 
> show the cores in a admin page. But this page worked fine with the code in 
> solr4, now with solr8 it throws error.




> Kindly help. 
> 
> Regards
> Reej

Re: Could not load collection from ZK (worked in solr4, failing in solr8) - Help Needed

[Yuval Paz]

What error do you see?

Also, what are you trying to do? If you are trying to get the cluster's
state from an outside service, and not in a Solr plugin, maybe you should
use CloudSolrClient instead:

```
var client = new CloudSolrClient
.builder(Arrays.asList("zkHost1:port1,zkHost2:port2,..."),
Optional.empty()).build();
```
And then use either the `ClusterStateProvider` from
`client.getClusterStateProvider()` or use the
`CollectionAdminRequest.clusterStatus` request.

On Thu, Jul 8, 2021, 2:01 PM Reej M  wrote:

> Hi All,
>
> We are upgrading from solr 4 to solr 8. Updated the solr server to
> solr8.8.2, solrj8.8.2.
> We are trying to resume the code and in one place where we are trying to
> get a collection from clusterstate. Code used is as below
>
> Clusterstate clusterState = null;
> For(SearchSolrZookeeper zk:livezks){
> SolrZkClient zkClient = null;
> ZKStateReader stateReader = null;
> Try{
> Zkclient = new SolrZkClient(zk.getUrl(), 3000);
> stateReader = new ZKStateReader(Zkclient);
> stateReader.createClusterStateWatchersAndUpdate();
> clusterState = stateReader.getClusterState();
> Connected = true;
> Exception = null;
> break;
> }
> Catch(){
> }
> Finally{
> State reader.close();
> Zkclient.close();
> }
>
> If(clusterstate!=null){
> Doccollection dc = clusterstate.getCollection(“INVOICE”);
> //here dc is null
> }
> }
>
>
> Tried using getcollectionornull - returns null
> Getcollectionornull(collection name, true) — returns null
>
> When I debug I can see clusterstate has below values in it
>
> Znodeversion:0
> Live node:[192.123.33.44:3883_Solr, 192.123.33.44:9883
> _Solr,192.123.33.45:3883_Solr,192.123.33.45:3883_Solr]
> Collections:{ORDER=LazyCollectionRef(ORDER), INVOICE=
> LazyCollectionRef(INVOICE), PAYMENT= LazyCollectionRef(PAYMENT),etc..}
>
> From the document collection we get the slices and iterate each replica to
> show the cores in a admin page. But this page worked fine with the code in
> solr4, now with solr8 it throws error. Kindly help.
>
> Regards
> Reej

Re: Could not load collection from ZK (worked in solr4, failing in solr8) - Help Needed

[sachin gk]

Have you upgraded the zookeeper too ? Sole 8.8.2 is compatible with
zookeeper 3.5.5

On Thu, Jul 8, 2021, 17:35 Yuval Paz  wrote:

> What error do you see?
>
> Also, what are you trying to do? If you are trying to get the cluster's
> state from an outside service, and not in a Solr plugin, maybe you should
> use CloudSolrClient instead:
>
> ```
> var client = new CloudSolrClient
> .builder(Arrays.asList("zkHost1:port1,zkHost2:port2,..."),
> Optional.empty()).build();
> ```
> And then use either the `ClusterStateProvider` from
> `client.getClusterStateProvider()` or use the
> `CollectionAdminRequest.clusterStatus` request.
>
> On Thu, Jul 8, 2021, 2:01 PM Reej M  wrote:
>
> > Hi All,
> >
> > We are upgrading from solr 4 to solr 8. Updated the solr server to
> > solr8.8.2, solrj8.8.2.
> > We are trying to resume the code and in one place where we are trying to
> > get a collection from clusterstate. Code used is as below
> >
> > Clusterstate clusterState = null;
> > For(SearchSolrZookeeper zk:livezks){
> > SolrZkClient zkClient = null;
> > ZKStateReader stateReader = null;
> > Try{
> > Zkclient = new SolrZkClient(zk.getUrl(), 3000);
> > stateReader = new ZKStateReader(Zkclient);
> > stateReader.createClusterStateWatchersAndUpdate();
> > clusterState = stateReader.getClusterState();
> > Connected = true;
> > Exception = null;
> > break;
> > }
> > Catch(){
> > }
> > Finally{
> > State reader.close();
> > Zkclient.close();
> > }
> >
> > If(clusterstate!=null){
> > Doccollection dc = clusterstate.getCollection(“INVOICE”);
> > //here dc is null
> > }
> > }
> >
> >
> > Tried using getcollectionornull - returns null
> > Getcollectionornull(collection name, true) — returns null
> >
> > When I debug I can see clusterstate has below values in it
> >
> > Znodeversion:0
> > Live node:[192.123.33.44:3883_Solr, 192.123.33.44:9883
> > _Solr,192.123.33.45:3883_Solr,192.123.33.45:3883_Solr]
> > Collections:{ORDER=LazyCollectionRef(ORDER), INVOICE=
> > LazyCollectionRef(INVOICE), PAYMENT= LazyCollectionRef(PAYMENT),etc..}
> >
> > From the document collection we get the slices and iterate each replica
> to
> > show the cores in a admin page. But this page worked fine with the code
> in
> > solr4, now with solr8 it throws error. Kindly help.
> >
> > Regards
> > Reej
>

Permission to edit wiki page on confluence

[Samuel Dupuis]

Hi,
I am writing here to ask for permission to edit the Solr wiki pages on
confluence. I want to add our company on the support page(
https://cwiki.apache.org/confluence/display/SOLR/Support) as we currently
offer Solr support.
If I understand correctly, I need to provide my username here for inclusion
in the contributor group in other to be able to edit the wiki page. My
username on confluence is "Samuel Dupuis".

Thank you,
Samuel

Re: Could not load collection from ZK (worked in solr4, failing in solr8) - Help Needed

[Reej Nayagam]

On Thu, 8 Jul 2021 at 8:10 PM, sachin gk  wrote:

> Have you upgraded the zookeeper too ? Sole 8.8.2 is compatible with
> zookeeper 3.5.5
>
zk version is 3.6.3

>
> On Thu, Jul 8, 2021, 17:35 Yuval Paz  wrote:
>
> > What error do you see?
> >
> > Also, what are you trying to do? If you are trying to get the cluster's
> > state from an outside service, and not in a Solr plugin, maybe you should
> > use CloudSolrClient instead:
> >
> > ```
> > var client = new CloudSolrClient
> > .builder(Arrays.asList("zkHost1:port1,zkHost2:port2,..."),
> > Optional.empty()).build();
> > ```
> > And then use either the `ClusterStateProvider` from
> > `client.getClusterStateProvider()` or use the
> > `CollectionAdminRequest.clusterStatus` request.


Thank you I tried with cloudsolrclient and it worked instead of
SolrZkClient. When I try to pass the zhkosts it throws host://2181 down? So
I replaced it with the cluster URL and it worked Thanks you.

>
> >
> > On Thu, Jul 8, 2021, 2:01 PM Reej M  wrote:
> >
> > > Hi All,
> > >
> > > We are upgrading from solr 4 to solr 8. Updated the solr server to
> > > solr8.8.2, solrj8.8.2.
> > > We are trying to resume the code and in one place where we are trying
> to
> > > get a collection from clusterstate. Code used is as below
> > >
> > > Clusterstate clusterState = null;
> > > For(SearchSolrZookeeper zk:livezks){
> > > SolrZkClient zkClient = null;
> > > ZKStateReader stateReader = null;
> > > Try{
> > > Zkclient = new SolrZkClient(zk.getUrl(), 3000);
> > > stateReader = new ZKStateReader(Zkclient);
> > > stateReader.createClusterStateWatchersAndUpdate();
> > > clusterState = stateReader.getClusterState();
> > > Connected = true;
> > > Exception = null;
> > > break;
> > > }
> > > Catch(){
> > > }
> > > Finally{
> > > State reader.close();
> > > Zkclient.close();
> > > }
> > >
> > > If(clusterstate!=null){
> > > Doccollection dc = clusterstate.getCollection(“INVOICE”);
> > > //here dc is null
> > > }
> > > }
> > >
> > >
> > > Tried using getcollectionornull - returns null
> > > Getcollectionornull(collection name, true) — returns null
> > >
> > > When I debug I can see clusterstate has below values in it
> > >
> > > Znodeversion:0
> > > Live node:[192.123.33.44:3883_Solr, 192.123.33.44:9883
> > > _Solr,192.123.33.45:3883_Solr,192.123.33.45:3883_Solr]
> > > Collections:{ORDER=LazyCollectionRef(ORDER), INVOICE=
> > > LazyCollectionRef(INVOICE), PAYMENT= LazyCollectionRef(PAYMENT),etc..}
> > >
> > > From the document collection we get the slices and iterate each replica
> > to
> > > show the cores in a admin page. But this page worked fine with the code
> > in
> > > solr4, now with solr8 it throws error. Kindly help.
> > >
> > > Regards
> > > Reej
> >
>
-- 
*Thanks,*
*Reej*