Thanks for the information about the CUDA project!
I think the main reason why you have not heard anything about Lucene/Solr/ElasticSearch working together with GPUs is mainly the fact, that Apache Lucene and all search servers on top of Lucene (Apache Solr, ElasticSearch) are pure Java applications, highly optimized to run in the Oracle virtual machine. Currently there is no official support for GPUs from Java APIs, you can only use proprietary wrapper libraries to make use of CUDA (e.g. http://www.jcuda.org/). It would be great, if there would be a platform independent way (directly in the official Java API) to execute jobs on GPUs. It might be worth a try to maybe implement the Lucene block codecs (the abstraction of the underlying posting list formats) using a GPU. Because this is encapsulated in a public API, it could be a separate project, using the JNI-based CUDA wrappers to encode/decode PFOR postings lists. The query execution logic is harder to port, because there is a lot of abstraction included (posting lists are doc-id iterators), which would need to be short circuited. Uwe ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen <http://www.thetaphi.de/> http://www.thetaphi.de eMail: [email protected] From: Yanning Li [mailto:[email protected]] Sent: Tuesday, July 09, 2013 11:02 PM To: [email protected] Subject: Anyone interested about using GPU to improve the performance of Lucene? Hi all, I work for NVIDIA Tesla Accelerating Computing Group. Recently we are noticed that GPU can really accelerate the performance of search engines. There are proof points not only from Google, but also from others, such as Yandex, Baidu, Bing, etc. But not much around Solr/Lucene. So we are trying to engage with Lucene developers more actively. 1) If possible we could like to hear from your perspective, are there some opportunities for GPU in Lucene/Solr? 2) Wondering is anyone interested to use GPU to accelerate the performance of Lucene/Solr? If so please feel free to let me know, we can send out free GPUs to get project started. Attached a paper talking about using GPU to accelerate index compression in case you have interests. Looking forward to hear from some of you, Best Yanning _____ This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. _____
