> On Sep 29, 2022, at 4:17 AM, Jan Høydahl <jan....@cominvent.com> wrote:
>
> * Index with multiple threads on the client, experiment to find a good number
> based on the number of CPUs on receiving side
That may also mean having multiple clients. We went from taking about 8 hours
to index our entire 42M rows to about 1.5 hours because we ran 10 indexer
clients at once. Each indexer takes roughly 1/10th of the data and churns away.
We don't have any of the clients do a commit. After the indexers are done, we
run one more time through the queue with a commit at the end.
As Jan says, make sure it's not your database that is the bottleneck, and
experiment with how many clients you want to have going at once.
Andy
