>From the Solr UI, how can I tell the number of threads are set for indexing ?
On Sat, Nov 11, 2023, 5:31 PM Shawn Heisey <apa...@elyograg.org.invalid> wrote: > On 11/11/2023 12:32, Vince McMahon wrote: > > What is the fastest way to load and index this large and wide CSV file? > It > > is taking too long, 20+ hours, now. > > I am assuming here that you are sending the CSV data directly to Solr > and letting Solr parse it into documents. If that is incorrect, please > fully describe your indexing software. > > How many total documents are being indexed in those 20 hours? > > How many threads do you have indexing simultaneously? How many CSV > lines are you sending in each batch? > > When I was maintaining large-ish Solr installs, I was doing the indexing > single-threaded and it would do about 1000 docs per second. Indexing > with multiple threads is the secret to making Solr index quickly. > > Thanks, > Shawn > >
