Great questions. Here are some of the answers.
" Exit condition `if Not rows:\n break` is not clear to me. Why should it
work? "
The exit condition is when the postgres_query fetch nothing then "if not
row" will breaks out from the while loop and close the cursor.
" Also, how many distinct ids
Ishan,
How do you multi-thread?
Secondly, could you please tell me what to look from the log that the
Indexing is committing 1000 documents at a time after executing the code
*solr.commit()?* Do you see anything that tells why it stops after 5000
rows, while there are about 15 rows fetched?
Oh, Mikhail. Thanks for your questions. There are just 5000 distinct ids.
I'll check during the office hours with the source side to fix their
problem and try again.
Thank you and you are a very wise man.
On Fri, Dec 15, 2023 at 4:32 AM Vince McMahon
wrote:
> Great questions. Here are some
Ishan, you are right. Doing multithreaded Indexing is going much faster.
I found out after the remote machine became unresponsive very quickly ; it
crashed. lol.
On Fri, Dec 15, 2023 at 4:35 AM Vince McMahon
wrote:
> Ishan,
>
> How do you multi-thread?
>
> Secondly, could you please tell me wh
On 12/15/23 05:41, Vince McMahon wrote:
Ishan, you are right. Doing multithreaded Indexing is going much faster.
I found out after the remote machine became unresponsive very quickly ; it
crashed. lol.
FWIW I got better results posting docs in batches from a single thread.
Work is in a "privat
Have you considered trying an existing document ingestion framework? I
wrote this one: https://github.com/nsoft/jesterj It already has a database
connector. If you do check it out and find difficulty please let me know by
leaving bug reports (if bug) or feedback (if confusion) in the discussions
se
Hello,
I am using Solr 8.11.1 in standalone mode and need to implement
authentication for API calls. I went over the below documentation page to
evaluate my options:
https://solr.apache.org/guide/8_11/securing-solr.html
JWTPlugin seems to be something which could be a good option. But it
mentions
FYI, providing the logs attached, the code already sends docs in 1K batch.
On Fri, Dec 15, 2023 at 7:11 PM Dmitri Maziuk
wrote:
> On 12/15/23 05:41, Vince McMahon wrote:
> > Ishan, you are right. Doing multithreaded Indexing is going much faster.
> > I found out after the remote machine became
On 12/15/23 13:26, Mikhail Khludnev wrote:
FYI, providing the logs attached, the code already sends docs in 1K batch.
That's not obvious to me: the docs are being fetched in 1K batches from
the DB and passed on to some "solr" library that may or may not be doing
"helpful" stuff under the hoo
I am impressed, Gus. Does it handle incremental changes from the source db
tables, such as insert, update, and delete.
On Fri, Dec 15, 2023 at 12:58 PM Gus Heck wrote:
> Have you considered trying an existing document ingestion framework? I
> wrote this one: https://github.com/nsoft/jesterj It
Nice insight, Dima. Happy Friday.
On Fri, Dec 15, 2023 at 11:11 AM Dmitri Maziuk
wrote:
> On 12/15/23 05:41, Vince McMahon wrote:
> > Ishan, you are right. Doing multithreaded Indexing is going much faster.
> > I found out after the remote machine became unresponsive very quickly ;
> it
> > cr
I guess one option would be running it behind a proxy (nginx, varnish,
haproxy etc) and adding the authentication at that level. This also means
you can use the proxy as an SSL terminator and so secure traffic.
On Sat, Dec 16, 2023 at 3:03 AM Rahul Goswami wrote:
> Hello,
> I am using Solr 8.11.
Isn’t jwt token just some http header? I wonder if it could be added easily by
constructing a custom http client instance for solrj to use.
Also docs say that it supports multiple authentication schemes. Maybe you can
use http basic and a service account just for solrj and jwt for other clients
13 matches
Mail list logo
