On 9/29/22 22:28, Gus Heck wrote:
* Do NOT commit during the bulk load, wait until the end
Unless something changed this is slightly risky. It can lead to very large
transaction logs and very long playback of the tx log on startup.
It is always good practice to have autoCommit configured with
openSearcher set to false and a relatively low maxTime value. I believe
the configs that Solr ships with set this to 15 seconds (actual value
being 15000 milliseconds), but I prefer making it 60 seconds just so
there is less overall stress on the system. That setting will eliminate
the problem with huge transaction logs. I believe this is discussed on
that Lucidworks article that you linked.
A commit that opens a new searcher should be done at the end of the
major indexing job. I would do this as a soft commit, but there's
nothing wrong with making it a hard commit that has openSearcher set to
true. On large indexing jobs there is likely to be little difference in
performance between the two types of commit.
Thanks,
Shawn
