Thanks to Koji & Alessandro & Matthias & Charlie for your quick
reactions to my relatively broad question.
I have listed your answers below so that you and I have a better
overview.
I agree to Koji that we are discussing RAG.
Thanks to Matthias and Charlie for opening ways of learning and
discussion.
And special thanks to Alessandro for clearly describing real-world
problems with implementing RAG.
Obviously (as we say in German) very thick boards have to be drilled
here.
Currently I'm working on the topic (as a freelancer) out of my own
interest and discussing it with potential customers. Lets see how it
goes on.
Walter
-------------------------------------------------------------------------
By chance I found this paper, which perhaps might be interesting for
you:
https://arxiv.org/abs/2308.14963
Vector Search with OpenAI Embeddings: Lucene Is All You Need
Jimmy Lin, Ronak Pradeep, Tommaso Teofili, Jasper Xian
We provide a reproducible, end-to-end demonstration of vector search
with OpenAI embeddings using Lucene on the popular MS MARCO passage
ranking test collection. The main goal of our work is to challenge the
prevailing narrative that a dedicated vector store is necessary to take
advantage of recent advances in deep neural networks as applied to
search. Quite the contrary, we show that hierarchical navigable
small-world network (HNSW) indexes in Lucene are adequate to provide
vector search capabilities in a standard bi-encoder architecture. This
suggests that, from a simple cost-benefit analysis, there does not
appear to be a compelling reason to introduce a dedicated vector store
into a modern "AI stack" for search, since such applications have
already received substantial investments in existing, widely deployed
infrastructure.
-------------------------------------------------------------------------------
Koji Sekiguchi 25.04.2024 03:45
Hi Walter,
Isn't it an application commonly known as RAG
https://en.wikipedia.org/wiki/Prompt_engineering#Retrieval-augmented_generation
?
--
Koji
-------------------------------------------------------------------------------
Matthias Krüger 25.04.2024 09:35
Hallo Walter,
es gibt viele dieser Startups zur Zeit, wenn Du konkrete Fragen oder
Ideen zu Retrieval-Augmented-Generation-Ansätzen und Architekturen mit
Solr hast, können wir gern mal sprechen.
Viele Grüße
Matthias Krüger
OpenSource Connections
-------------------------------------------------------------------------------
Alessandro Benedetti 25.04.2024 14:55
Hi Walter,
We've been doing many AI integrations with Solr and we drafted a roadmap
to get some funding to implement it directly in Solr:
https://sease.io/2023/10/apache-lucene-solr-ai-roadmap-do-you-want-to-make-it-happen.html
We made little progress so far but hopefully will attract some more
attention.
In terms of what you saw, as Koji correctly mentioned, it's very likely
it was Retrieval Augmented Generation:
Doing it is extra easy for a quick prototype, to just showcase some
cherry picked magic.
But bringing it to production offers many challenges:
- the retrieval phase is still an open problem, you can do lexical
(traditional keyword search), you can do vector or hybrid, but it's far
from being easy.
You may need to chunk if you go with most embedding models.
- the LLM choice is quite challenging as well, you can go quickly with
the latest GPT-X but more likely you need some days to assess the best
commercial/open solution for your use case and domain
- the way you prompt the LLM is also challenging, depending if you just
want a generated answer, citations etc
We presented already some talks and tutorials around, we should have
something recorded in our training section.
We'll also speak soon on these topics at upcoming conferences:
https://eu.communityovercode.org/schedule/
https://2024.berlinbuzzwords.de/sessions?id=7VSFFK
https://2024.berlinbuzzwords.de/sessions?id=NCPYUH
Hope it helps!
-------------------------------------------------------------------------------
Charlie Hull 25.04.2024 18:00
We've just had the Haystack conference here in Charlottesville with many
talks on RAG and AI on Lucene based engines (can't remember a particular
one on Solr but a lot is transferrable). Check out www.haystackconf.com
- the videos & slides of all the talks will be published over the next
few weeks. You should also join Relevance Slack where there's a lot of
discussions on these subjects among the 5000 (!) members.
www.opensourceconnections.com/slack
Charlie
On Wed, 24 Apr 2024, 22:56 , <s...@cid.is> wrote:
Hi all,
is anybody already using AI to support searching with Solr/Lucene?
I just had an interesting demo from a german start-up.
I gave them plain text data, which I usually feed into Solr, and they
did some AI magic with these data, so that we could ask
human-language-questions and got human-language-answers.
This can take information retrieval to the next level.
But I'd prefer to have a common production line for this.
And I'm looking for software stacks which would allow standard
production procedures.
Thank you for *every* hint on this.
Walter Claassen
