Re: vector search support

2023-05-29 Thread Giuseppe Broccolo
Hi Nathan, I noticed you implemented a closest_vector function which returns the closest vector to a given one using the Euclidean distance: would it make sense to change the implementation in order to include also different distance definitions rather than the Euclidean one (for instance, cosine

Re: vector search support

2023-05-29 Thread Giuseppe Broccolo
Hi Jonathan, On 5/26/23 3:38 PM, Jonathan S. Katz wrote: > On 4/26/23 9:31 AM, Giuseppe Broccolo wrote: > > We finally opted for ElasticSearch as search engine, considering that it > > was providing what we needed: > > > > * support to store dense vectors > > * support for kNN searches (last ver

Re: vector search support

2023-05-26 Thread Jonathan S. Katz
On 4/26/23 9:31 AM, Giuseppe Broccolo wrote: Hi Nathan, I find the patches really interesting. Personally, as Data/MLOps Engineer, I'm involved in a project where we use embedding techniques to generate vectors from documents, and use clustering and kNN searches to find similar documents basi

Re: vector search support

2023-05-26 Thread Jonathan S. Katz
On 5/25/23 1:48 PM, Oliver Rice wrote: A nice side effect of using the float8[] to represent vectors is that it allows for vectors of different sizes to coexist in the same column. We most frequently see (pgvector) vector columns being used for storing ML embeddings. Given that different mode

Re: vector search support

2023-05-26 Thread Jonathan S. Katz
Hi, On 4/21/23 8:07 PM, Nathan Bossart wrote: Attached is a proof-of-concept/work-in-progress patch set that adds functions for "vectors" repreѕented with one-dimensional float8 arrays. These functions may be used in a variety of applications, but I am proposing them with the AI/ML use-cases in

Re: vector search support

2023-05-25 Thread Oliver Rice
Hi Nathan, A nice side effect of using the float8[] to represent vectors is that it allows for vectors of different sizes to coexist in the same column. We most frequently see (pgvector) vector columns being used for storing ML embeddings. Given that different models produce embeddings with a d

Re: vector search support

2023-04-26 Thread Giuseppe Broccolo
Hi Nathan, I find the patches really interesting. Personally, as Data/MLOps Engineer, I'm involved in a project where we use embedding techniques to generate vectors from documents, and use clustering and kNN searches to find similar documents basing on spatial neighbourhood of generated vectors.

vector search support

2023-04-21 Thread Nathan Bossart
Attached is a proof-of-concept/work-in-progress patch set that adds functions for "vectors" repreѕented with one-dimensional float8 arrays. These functions may be used in a variety of applications, but I am proposing them with the AI/ML use-cases in mind. I am posting this early in the v17 cycle i