Found the bug. It was an issue due to the CUDA runtime library not being available before the first call to cuVS's CuVSServiceProvider. There is an initialization race between two code paths and the wrong one wins. When this happens, the GPU resource *seems unavailable* to the indexing code path and it falls back to using CPU based Lucene99HnswVectorsWriter instead.
Submitted a PR : https://github.com/apache/solr/pull/4328 -Rahul On Wed, Apr 22, 2026 at 2:16 AM Rahul Goswami <[email protected]> wrote: > Hi, > I was trying the GPU accelerated vector indexing in Solr 10.0 on an Nvidia > L40S GPU following the steps documented here: > https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html#gpu-acceleration > > I am running Solr in standalone mode and made the schema and solrconfig > changes to the techproducts core. I indexed 100 mock vectors and the > /update call succeeds. However, when I look closely at the logs, I notice > that it silently falls back to CPU based indexing. > > I think I have all the prerequisites that the documentation mentions. > Trying to debug this with Claude I burnt through 2 session limits while it > kept gaslighting me with being "almost there" for over 4 hours without any > luck. > > Help would be much appreciated! > > *Environment details:* > $ *nvcc --version* > nvcc: NVIDIA (R) Cuda compiler driver > Copyright (c) 2005-2026 NVIDIA Corporation > Built on Thu_Mar_19_11:12:51_PM_PDT_2026 > Cuda compilation tools, *release 13.2, V13.2.78* > Build cuda_13.2.r13.2/compiler.37668154_0 > > $*echo $LD_LIBRARY_PATH* (libcuvs is part of LD_LIBRARY_PATH and I > verified that libcuvs_c.so is present) > /home/ubuntu/Work/solr/libcuvs-env/lib/python3.10/site-packages/ > *libcuvs/lib64* > :/home/ubuntu/Work/solr/libcuvs-env/lib/python3.10/site-packages/librmm/lib64:/home/ubuntu/Work/solr/libcuvs-env/lib/python3.10/site-packages/rapids_logger/lib64:/usr/local/cuda-13/lib64:/usr/local/cuda-12.9/lib:/usr/local/cuda-12.9/lib64:/usr/local/cuda-12.9:/usr/local/cuda-12.9/targets/x86_64-linux/lib/:/usr/local/cuda-12.9/extras/CUPTI/lib64:/opt/amazon/openmpi/lib:/usr/local/lib:/usr/lib > > $ *java -version* > openjdk version "22.0.2" 2024-07-16 > OpenJDK Runtime Environment Temurin-22.0.2+9 (build 22.0.2+9) > OpenJDK 64-Bit Server VM Temurin-22.0.2+9 (build 22.0.2+9, mixed mode, > sharing) > > *solr.log:* > 2026-04-22 05:45:26.122 INFO (qtp858232531-60-null-3) [ x:techproducts > t:null-3] o.a.s.c.CuVSCodec Initializing > Lucene99AcceleratedHNSWVectorsFormat with parameter values: > cuvsWriterThreads 32, cuvsIntGraphDegree 128, cuvsGraphDegree 64, > cuvsHnswLayers 1, cuvsHnswM 16, cuvsHnswEfConstruction 100 > 2026-04-22 05:45:26.122 INFO (qtp858232531-60-null-3) [ x:techproducts > t:null-3] o.a.s.c.CuVSCodec Initializing > Lucene99AcceleratedHNSWVectorsFormat with parameter values: > cuvsWriterThreads 32, cuvsIntGraphDegree 128, cuvsGraphDegree 64, > cuvsHnswLayers 1, cuvsHnswM 16, cuvsHnswEfConstruction 100 > 2026-04-22 05:45:26.122 WARN (qtp858232531-60-null-3) [ x:techproducts > t:null-3] c.n.c.l.Lucene99AcceleratedHNSWVectorsFormat *GPU based > indexing not supported, falling back to using the Lucene99HnswVectorsWriter* > 2026-04-22 05:45:26.391 INFO (qtp858232531-60-null-3) [ x:techproducts > t:null-3] o.a.s.u.p.LogUpdateProcessorFactory path=/update > params={commit=true}{add=[mock_product_0001 (1863148346926956544), > mock_product_0002 (1863148346933248000), mock_product_0003 > (1863148346934296576), mock_product_0004 (1863148346936393728), > mock_product_0005 (1863148346937442304), mock_product_0006 > (1863148346939539456), mock_product_0007 (1863148346940588032), > mock_product_0008 (1863148346946879488), mock_product_0009 > (1863148346947928064), mock_product_0010 (1863148346950025216), ... (100 > adds)], commit=} 0 272 > > Thanks, > Rahul >
