Re: [EXTERNAL] Re: Slow HNSW creation times.

2024-05-01 Thread Krishnamurthy, Kannan
is considered bad. Will soon share insights from the profiler. Kannan From: Uwe Schindler Date: Monday, April 29, 2024 at 8:08 AM To: java-user@lucene.apache.org Subject: [EXTERNAL] Re: Slow HNSW creation times. Hi, how much physical RAM has the machine, because 30 GiB heap sounds a lot to me

Re: Slow HNSW creation times.

2024-04-29 Thread Uwe Schindler
ging bottleneck. On Fri, Apr 19, 2024 at 4:17 PM Krishnamurthy, Kannan wrote: Greetings, We are experiencing slow HNSW creation times during index merge. Specifically, we have noticed that the HNSW graph creation becomes progressively slow after reaching a certain size. Our indexing workflow cr

Re: Slow HNSW creation times.

2024-04-28 Thread Adrien Grand
from disk directly? If this doesn't sound right, running your application with a profiler would help identify your merging bottleneck. On Fri, Apr 19, 2024 at 4:17 PM Krishnamurthy, Kannan wrote: > Greetings, > > We are experiencing slow HNSW creation times during index merge. >

Slow HNSW creation times.

2024-04-19 Thread Krishnamurthy, Kannan
Greetings, We are experiencing slow HNSW creation times during index merge. Specifically, we have noticed that the HNSW graph creation becomes progressively slow after reaching a certain size. Our indexing workflow creates around 60 indices, each containing approximately 500k vectors. The