branch: externals/vecdb
commit b540da81b270d01b6d903ce31741ee132293efa2
Author: Andrew Hyatt <ahy...@gmail.com>
Commit: Andrew Hyatt <ahy...@gmail.com>
Rename package from embed-db to vecdb
---
README.org | 32 +++----
embed-chroma.el => vecdb-chroma.el | 173 +++++++++++++++++++------------------
embed-qdrant.el => vecdb-qdrant.el | 99 ++++++++++-----------
embed-db.el => vecdb.el | 44 +++++-----
4 files changed, 176 insertions(+), 172 deletions(-)
diff --git a/README.org b/README.org
index a89fbc74cb..1b022405e0 100644
--- a/README.org
+++ b/README.org
@@ -1,7 +1,7 @@
-#+TITLE: embed-db: Embedding Search Library for Emacs
+#+TITLE: vecdb: Embedding Search Library for Emacs
* Introduction
-The =embed-db= package provides an interface to embedding database. These
databases enable "semantic search", which is a powerful way to search over
meaning. This kind of search needs specialized storage and retrieval.
+The =vecdb= package provides an interface to embedding database. These
databases enable "semantic search", which is a powerful way to search over
meaning. This kind of search needs specialized storage and retrieval.
This package doesn't provide end-user functionality on its own; it is designed
to be used in other packages that need semantic search.
@@ -9,51 +9,51 @@ The package does not provide embeddings, that can be done
with the [[https://git
* Configuring the collection
There are two concepts that together define a collection database of
embeddings: the /provider/, and the /collection/. The provider is what kind of
backend we are using, right now either =chroma=, or =qdrant=. This is a struct
defined by the exact provider you want to use.
-The collection is, for that provider, what exact database is getting used,
with each collection having its own separate data. Collections must be created
before being used. The collection is defined by the struct
~embed-db-collection~ which has a ~name~ (used to identify the collection),
~vector-size~, and ~payload-fields~. The ~vector-size~ will be based on the
size of the embedding vector from your provider. 1536 is what Open AI uses.
~payload-fields~ is an alist of fields and the [...]
+The collection is, for that provider, what exact database is getting used,
with each collection having its own separate data. Collections must be created
before being used. The collection is defined by the struct ~vecdb-collection~
which has a ~name~ (used to identify the collection), ~vector-size~, and
~payload-fields~. The ~vector-size~ will be based on the size of the embedding
vector from your provider. 1536 is what Open AI uses. ~payload-fields~ is an
alist of fields and their [...]
An example, putting it all together, is:
#+begin_src emacs-lisp
(defvar my-embed-provider (make-embed-qdrant-provider :api-key
my-qdrant-api-key :url my-qdrant-url))
-(defvar my-embed-collection (make-embed-db-collection :name "my test
collection" :vector-size 1536 :payload-fields (('my-id . 'string))))
+(defvar my-embed-collection (make-vecdb-collection :name "my test collection"
:vector-size 1536 :payload-fields (('my-id . 'string))))
#+end_src
The provider will be supplied by the end-user, specifying how they want things
stored, and any data necessary for that storage and retrieval to function. The
collection is typically partially supplied by the application, with the
possible exception of embedding size, which may be dependent on the exact
embedding provider they are using.
-Collections must be created before they can be used with ~embed-db-create~,
and ~embed-db-exists~ can return whether the collection exists.
+Collections must be created before they can be used with ~vecdb-create~, and
~vecdb-exists~ can return whether the collection exists.
#+begin_src emacs-lisp
-(unless (embed-db-exists my-embed-provider my-embed-collection)
- (embed-db-create my-embed-provider my-embed-collection))
+(unless (vecdb-exists my-embed-provider my-embed-collection)
+ (vecdb-create my-embed-provider my-embed-collection))
#+end_src
-They can also be deleted with ~embed-db-delete~.
+They can also be deleted with ~vecdb-delete~.
* Adding and replacing data
Before data is queried, it must be added. This is done via a batch operation
on
-a group of data, ~embed-db-upsert-items~. This either creates an item in the
collection
+a group of data, ~vecdb-upsert-items~. This either creates an item in the
collection
or replaces it, based on the =id= of the item. Here's an example of adding or
replacing one item:
#+begin_src emacs-lisp
-(embed-db-upsert-items my-embed-provider my-embed-collection
- (list (make-embed-db-item
+(vecdb-upsert-items my-embed-provider my-embed-collection
+ (list (make-vecdb-item
:id "example-id"
:vector [0.1 0.2 0.3 0.4]
:payload '(:my-id "235913926"))))
#+end_src
-These can be deleted with ~embed-db-delete-item~ and retrieved by ID with
~embed-db-get-item~.
+These can be deleted with ~vecdb-delete-item~ and retrieved by ID with
~vecdb-get-item~.
-IDs used in =embed-db= *must* be =uint64= values. If you have another ID you
need to use to tie it together with other storage, that should go into the
=payload=.
+IDs used in =vecdb= *must* be =uint64= values. If you have another ID you
need to use to tie it together with other storage, that should go into the
=payload=.
* Querying data
-Querying the database can be done with ~embed-db-search-by-vector~, passing it
a vector and optionally a number of results to return (10 is the default).
+Querying the database can be done with ~vecdb-search-by-vector~, passing it a
vector and optionally a number of results to return (10 is the default).
#+begin_src emacs-lisp
-(embed-db-search-by-vector my-embed-provider my-embed-collection [0.3 0.1 0.5
-0.9] 20)
+(vecdb-search-by-vector my-embed-provider my-embed-collection [0.3 0.1 0.5
-0.9] 20)
#+end_src
-This will return the specifies number of =embed-db-item= structs, with the
payloads they were stored with.
+This will return the specifies number of =vecdb-item= structs, with the
payloads they were stored with.
* Providers
** qdrant
[[https://qdrant.tech/][qdrant]] is an open source vector database that
concentrates mostly on running in the cloud, but can be run locally with a
docker container. They provide a free tier for your database in the cloud that
may be garbage collected after a period of inactivity.
diff --git a/embed-chroma.el b/vecdb-chroma.el
similarity index 52%
rename from embed-chroma.el
rename to vecdb-chroma.el
index b14584b5ef..bf88301c4b 100644
--- a/embed-chroma.el
+++ b/vecdb-chroma.el
@@ -1,9 +1,9 @@
-;;; embed-chroma.el --- An interface to the chroma databases -*-
lexical-binding: t; -*-
+;;; vecdb-chroma.el --- An interface to the chroma databases -*-
lexical-binding: t; -*-
;; Copyright (c) 2025 Free Software Foundation, Inc.
;; Author: Andrew Hyatt <ahy...@gmail.com>
-;; Homepage: https://github.com/ahyatt/embed-db
+;; Homepage: https://github.com/ahyatt/vecdb
;; SPDX-License-Identifier: GPL-3.0-or-later
;;
;; This program is free software; you can redistribute it and/or
@@ -20,30 +20,30 @@
;; along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>.
;;; Commentary:
-;; This package provides an interface to the chroma databases for embed-db.
+;; This package provides an interface to the chroma databases for vecdb.
;;; Code:
-(require 'embed-db)
+(require 'vecdb)
(require 'plz)
-(defconst embed-chroma-collection-id-cache
+(defconst vecdb-chroma-collection-id-cache
(make-hash-table :test 'equal)
"Cache for chroma collection IDs.")
-(cl-defstruct (embed-chroma-provider (:include embed-db-provider
+(cl-defstruct (vecdb-chroma-provider (:include vecdb-provider
(name "chroma")))
(binary "chroma")
(url "http://localhost:8000";)
(tenant "default")
(database "default"))
-(defun embed-chroma-call (provider method url-suffix &optional body sync)
+(defun vecdb-chroma-call (provider method url-suffix &optional body sync)
"Make an HTTP request to the Chroma API.
If BODY is provided, it will be sent as the request body.
SYNC indicates whether the request should be synchronous."
- (let ((url (embed-chroma-provider-url provider)))
+ (let ((url (vecdb-chroma-provider-url provider)))
(unless url
(error "Chroma URL is not set"))
(if sync
@@ -57,27 +57,27 @@ SYNC indicates whether the request should be synchronous."
:body (if body (json-encode body) "")
:then #'ignore))))
-(defun embed-chroma-has-tenant-p (provider)
+(defun vecdb-chroma-has-tenant-p (provider)
"Check if the chroma PROVIDER has a tenant."
(condition-case err
- (embed-chroma-call
+ (vecdb-chroma-call
provider
'get
- (format "/api/v2/tenants/%s" (embed-chroma-provider-tenant provider))
+ (format "/api/v2/tenants/%s" (vecdb-chroma-provider-tenant provider))
nil
t)
(plz-error (if (eq 404 (plz-response-status (plz-error-response (nth 2
err))))
nil
(error "Error checking tenant: %s" (plz-error-message
err))))))
-(defun embed-chroma-has-database-p (provider database)
+(defun vecdb-chroma-has-database-p (provider database)
"Check if the chroma PROVIDER has a DATABASE."
(condition-case err
- (embed-chroma-call
+ (vecdb-chroma-call
provider
'get
(format "/api/v2/tenants/%s/databases/%s"
- (embed-chroma-provider-tenant provider)
+ (vecdb-chroma-provider-tenant provider)
database)
nil
t)
@@ -85,146 +85,147 @@ SYNC indicates whether the request should be synchronous."
nil
(error "Error checking database: %s" (plz-error-message
err))))))
-(cl-defmethod embed-db-create ((provider embed-chroma-provider)
- (collection embed-db-collection))
+(cl-defmethod vecdb-create ((provider vecdb-chroma-provider)
+ (collection vecdb-collection))
"Create a new chroma collection."
- (unless (embed-chroma-has-tenant-p provider)
- (embed-chroma-call
+ (unless (vecdb-chroma-has-tenant-p provider)
+ (vecdb-chroma-call
provider
'post
"/api/v2/tenants"
- `(("name" . ,(embed-chroma-provider-tenant provider))) t))
- (unless (embed-chroma-has-database-p provider
(embed-chroma-provider-database provider))
- (embed-chroma-call
+ `(("name" . ,(vecdb-chroma-provider-tenant provider))) t))
+ (unless (vecdb-chroma-has-database-p provider
(vecdb-chroma-provider-database provider))
+ (vecdb-chroma-call
provider
'post
(format "/api/v2/tenants/%s/databases"
- (embed-chroma-provider-tenant provider))
- `(:name ,(embed-chroma-provider-database provider)) t))
- (embed-chroma-call
+ (vecdb-chroma-provider-tenant provider))
+ `(:name ,(vecdb-chroma-provider-database provider)) t))
+ (vecdb-chroma-call
provider
'post
(format "/api/v2/tenants/%s/databases/%s/collections"
- (embed-chroma-provider-tenant provider)
- (embed-chroma-provider-database provider))
- `(:name ,(embed-db-collection-name collection))
+ (vecdb-chroma-provider-tenant provider)
+ (vecdb-chroma-provider-database provider))
+ `(:name ,(vecdb-collection-name collection))
t))
-(cl-defmethod embed-db-delete ((provider embed-chroma-provider)
- (collection embed-db-collection))
+(cl-defmethod vecdb-delete ((provider vecdb-chroma-provider)
+ (collection vecdb-collection))
"Delete a chroma collection."
- (embed-chroma-call
+ (vecdb-chroma-call
provider
'delete
(format "/api/v2/tenants/%s/databases/%s/collections/%s"
- (embed-chroma-provider-tenant provider)
- (embed-chroma-provider-database provider)
- (embed-db-collection-name collection))
+ (vecdb-chroma-provider-tenant provider)
+ (vecdb-chroma-provider-database provider)
+ (vecdb-collection-name collection))
nil
t))
-(defun embed-chroma-collection-id (provider collection)
+(defun vecdb-chroma-collection-id (provider collection)
"Get the ID of a chroma COLLECTION in PROVIDER."
- (or (gethash (embed-db-collection-name collection)
- embed-chroma-collection-id-cache)
+ (or (gethash (vecdb-collection-name collection)
+ vecdb-chroma-collection-id-cache)
(let* ((url (format "/api/v2/tenants/%s/databases/%s/collections/%s"
- (embed-chroma-provider-tenant provider)
- (embed-chroma-provider-database provider)
- (embed-db-collection-name collection)))
- (result (embed-chroma-call provider 'get url nil t)))
+ (vecdb-chroma-provider-tenant provider)
+ (vecdb-chroma-provider-database provider)
+ (vecdb-collection-name collection)))
+ (result (vecdb-chroma-call provider 'get url nil t)))
(let ((id (plist-get result :id)))
- (puthash (embed-db-collection-name collection) id
- embed-chroma-collection-id-cache)
+ (puthash (vecdb-collection-name collection) id
+ vecdb-chroma-collection-id-cache)
id))))
-(cl-defmethod embed-db-exists ((provider embed-chroma-provider)
- (collection embed-db-collection))
+(cl-defmethod vecdb-exists ((provider vecdb-chroma-provider)
+ (collection vecdb-collection))
"Check if a chroma collection exists."
- (and (embed-chroma-has-tenant-p provider)
- (embed-chroma-has-database-p provider
- (embed-chroma-provider-database provider))
+ (and (vecdb-chroma-has-tenant-p provider)
+ (vecdb-chroma-has-database-p provider
+ (vecdb-chroma-provider-database provider))
(condition-case nil
- (embed-chroma-call
+ (vecdb-chroma-call
provider
'get
(format "/api/v2/tenants/%s/databases/%s/collections/%s"
- (embed-chroma-provider-tenant provider)
- (embed-chroma-provider-database provider)
- (embed-chroma-collection-id provider collection))
+ (vecdb-chroma-provider-tenant provider)
+ (vecdb-chroma-provider-database provider)
+ (vecdb-chroma-collection-id provider collection))
nil
t)
(plz-error nil))))
-(cl-defmethod embed-db-upsert-items ((provider embed-chroma-provider)
- (collection embed-db-collection)
- items &optional sync)
+(cl-defmethod vecdb-upsert-items ((provider vecdb-chroma-provider)
+ (collection vecdb-collection)
+ items &optional sync)
"Upsert items into a chroma collection."
(let ((url (format "/api/v2/tenants/%s/databases/%s/collections/%s/upsert"
- (embed-chroma-provider-tenant provider)
- (embed-chroma-provider-database provider)
- (embed-chroma-collection-id provider collection))))
- (embed-chroma-call
+ (vecdb-chroma-provider-tenant provider)
+ (vecdb-chroma-provider-database provider)
+ (vecdb-chroma-collection-id provider collection))))
+ (vecdb-chroma-call
provider
'post
url
- `(:embeddings ,(apply #'vector (mapcar #'embed-db-item-vector items))
- :ids ,(apply #'vector (mapcar #'embed-db-item-id items))
- :metadatas ,(apply #'vector (mapcar #'embed-db-item-payload
items)))
+ `(:embeddings ,(apply #'vector (mapcar #'vecdb-item-vector items))
+ :ids ,(apply #'vector (mapcar #'vecdb-item-id items))
+ :metadatas ,(apply #'vector (mapcar #'vecdb-item-payload
items)))
sync)))
-(cl-defmethod embed-db-get-item ((provider embed-chroma-provider)
- (collection embed-db-collection)
- item-id)
+(cl-defmethod vecdb-get-item ((provider vecdb-chroma-provider)
+ (collection vecdb-collection)
+ item-id)
"Get a single item from a chroma collection by ITEM-ID."
(let* ((url (format "/api/v2/tenants/%s/databases/%s/collections/%s/get"
- (embed-chroma-provider-tenant provider)
- (embed-chroma-provider-database provider)
- (embed-chroma-collection-id provider collection)))
- (result (embed-chroma-call provider 'get url
+ (vecdb-chroma-provider-tenant provider)
+ (vecdb-chroma-provider-database provider)
+ (vecdb-chroma-collection-id provider collection)))
+ (result (vecdb-chroma-call provider 'get url
`(:ids ,(vector item-id)
:limit 1)
t)))
(unless (= (length (plist-get result :items)) 1)
(error "Expected exactly one item, got %d"
(length (plist-get result :items)))
- (make-embed-db-item
+ (make-vecdb-item
:id (aref (plist-get result :ids) 0)
:vector (aref (plist-get result :embeddings) 0)
:payload (aref (plist-get result :metadatas) 0)))))
-(cl-defmethod embed-db-delete-items ((provider embed-chroma-provider)
- (collection embed-db-collection)
- item-ids &optional sync)
+(cl-defmethod vecdb-delete-items ((provider vecdb-chroma-provider)
+ (collection vecdb-collection)
+ item-ids &optional sync)
"Delete items from a chroma collection by ITEM-IDS."
(let ((url (format "/api/v2/tenants/%s/databases/%s/collections/%s/delete"
- (embed-chroma-provider-tenant provider)
- (embed-chroma-provider-database provider)
- (embed-chroma-collection-id provider collection))))
- (embed-chroma-call
+ (vecdb-chroma-provider-tenant provider)
+ (vecdb-chroma-provider-database provider)
+ (vecdb-chroma-collection-id provider collection))))
+ (vecdb-chroma-call
provider
'post
url
`(:ids ,(apply #'vector item-ids))
sync)))
-(cl-defmethod embed-db-search-by-vector ((provider embed-chroma-provider)
- (collection embed-db-collection)
- vector &optional limit)
+(cl-defmethod vecdb-search-by-vector ((provider vecdb-chroma-provider)
+ (collection vecdb-collection)
+ vector &optional limit)
"Search for items in a chroma collection by VECTOR."
(let* ((url (format "/api/v2/tenants/%s/databases/%s/collections/%s/query"
- (embed-chroma-provider-tenant provider)
- (embed-chroma-provider-database provider)
- (embed-chroma-collection-id provider collection)))
- (result (embed-chroma-call provider 'post url
+ (vecdb-chroma-provider-tenant provider)
+ (vecdb-chroma-provider-database provider)
+ (vecdb-chroma-collection-id provider collection)))
+ (result (vecdb-chroma-call provider 'post url
`(:query_embeddings [,vector]
:n_results ,(or limit
10)
:include ["embeddings"
"metadatas" "distances"])
t)))
(cl-loop for i from 0 below (length (aref (plist-get result :ids) 0))
- collect (make-embed-db-item ;
+ collect (make-vecdb-item ;
:id (aref (aref (plist-get result :ids) 0) i)
:vector (aref (aref (plist-get result :embeddings) 0) i)
:payload (aref (aref (plist-get result :metadatas) 0)
i)))))
-(provide 'embed-chroma)
-;;; embed-chroma.el ends here
+(provide 'vecdb-chroma)
+
+;;; vecdb-chroma.el ends here
diff --git a/embed-qdrant.el b/vecdb-qdrant.el
similarity index 54%
rename from embed-qdrant.el
rename to vecdb-qdrant.el
index e5fb029f9c..4b263ec84d 100644
--- a/embed-qdrant.el
+++ b/vecdb-qdrant.el
@@ -1,9 +1,9 @@
-;;; embed-qdrant.el --- An interface to the qdrant databases -*-
lexical-binding: t; -*-
+;;; vecdb-qdrant.el --- An interface to the qdrant databases -*-
lexical-binding: t; -*-
;; Copyright (c) 2025 Free Software Foundation, Inc.
;; Author: Andrew Hyatt <ahy...@gmail.com>
-;; Homepage: https://github.com/ahyatt/embed-db
+;; Homepage: https://github.com/ahyatt/vecdb
;; SPDX-License-Identifier: GPL-3.0-or-later
;;
;; This program is free software; you can redistribute it and/or
@@ -20,27 +20,27 @@
;; along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>.
;;; Commentary:
-;; This package provides an implementation of embed-db for the qdrant db.
+;; This package provides an implementation of vecdb for the qdrant db.
;;; Code:
-(require 'embed-db)
+(require 'vecdb)
(require 'cl-lib)
(require 'json)
(require 'plz)
-(cl-defstruct (embed-qdrant-provider (:include embed-db-provider
+(cl-defstruct (vecdb-qdrant-provider (:include vecdb-provider
(name "qdrant")))
"Provider for the qdrant embedding database."
(url nil)
(api-key nil))
-(defun embed-qdrant-call (provider method url-suffix &optional body sync)
+(defun vecdb-qdrant-call (provider method url-suffix &optional body sync)
"Make an HTTP request to the Qdrant API.
If BODY is provided, it will be sent as the request body.
SYNC indicates whether the request should be synchronous."
- (let ((url (embed-qdrant-provider-url provider))
- (api-key (embed-qdrant-provider-api-key provider)))
+ (let ((url (vecdb-qdrant-provider-url provider))
+ (api-key (vecdb-qdrant-provider-api-key provider)))
(unless url
(error "Qdrant URL is not set"))
(unless api-key
@@ -62,78 +62,78 @@ SYNC indicates whether the request should be synchronous."
:body (if body (json-encode body) "")
:then #'ignore))))
-(cl-defmethod embed-db-create ((provider embed-qdrant-provider)
- (collection embed-db-collection))
+(cl-defmethod vecdb-create ((provider vecdb-qdrant-provider)
+ (collection vecdb-collection))
"Create a new collection in the qdrant database.
-COLLECTION is an `embed-db-collection' object that specifies the
+COLLECTION is an `vecdb-collection' object that specifies the
properties of the collection."
- (embed-qdrant-call provider 'put (concat "/collections/"
(embed-db-collection-name collection))
- `(:vectors (:size ,(embed-db-collection-vector-size
collection)
+ (vecdb-qdrant-call provider 'put (concat "/collections/"
(vecdb-collection-name collection))
+ `(:vectors (:size ,(vecdb-collection-vector-size
collection)
:distance "Cosine"))
t))
-(cl-defmethod embed-db-delete ((provider embed-qdrant-provider)
- (collection embed-db-collection))
+(cl-defmethod vecdb-delete ((provider vecdb-qdrant-provider)
+ (collection vecdb-collection))
"Delete a collection from the qdrant database."
- (embed-qdrant-call provider 'delete (concat "/collections/"
(embed-db-collection-name collection))
+ (vecdb-qdrant-call provider 'delete (concat "/collections/"
(vecdb-collection-name collection))
nil t))
-(cl-defmethod embed-db-exists ((provider embed-qdrant-provider)
- (collection embed-db-collection))
+(cl-defmethod vecdb-exists ((provider vecdb-qdrant-provider)
+ (collection vecdb-collection))
"Check if a collection exists in the qdrant database."
- (let ((response (embed-qdrant-call provider 'get (concat "/collections/"
(embed-db-collection-name collection)
+ (let ((response (vecdb-qdrant-call provider 'get (concat "/collections/"
(vecdb-collection-name collection)
"/exists") nil t)))
(not (eq :false (plist-get (plist-get response :result) :exists)))))
-(cl-defmethod embed-db-upsert-items ((provider embed-qdrant-provider)
- (collection embed-db-collection)
- items &optional sync)
+(cl-defmethod vecdb-upsert-items ((provider vecdb-qdrant-provider)
+ (collection vecdb-collection)
+ items &optional sync)
"Insert items into a collection in the qdrant database."
- (embed-qdrant-call provider 'put
- (concat "/collections/" (embed-db-collection-name
collection) "/points?wait="
+ (vecdb-qdrant-call provider 'put
+ (concat "/collections/" (vecdb-collection-name
collection) "/points?wait="
(if sync "true" "false"))
`(:points ,(apply #'vector
(mapcar (lambda (item)
(append
- `(:id ,(embed-db-item-id
item)
- :vector
,(embed-db-item-vector item))
- (when (embed-db-item-payload
item)
- `(:payload
,(embed-db-item-payload item))))
+ `(:id ,(vecdb-item-id item)
+ :vector
,(vecdb-item-vector item))
+ (when (vecdb-item-payload
item)
+ `(:payload
,(vecdb-item-payload item))))
)
items)))
t))
-(cl-defmethod embed-db-get-item ((provider embed-qdrant-provider)
- (collection embed-db-collection)
- id)
+(cl-defmethod vecdb-get-item ((provider vecdb-qdrant-provider)
+ (collection vecdb-collection)
+ id)
"Retrieve an item from a collection in the qdrant database by ID."
- (let ((result (embed-qdrant-call provider 'get
- (concat "/collections/"
(embed-db-collection-name collection) "/points/" id))))
+ (let ((result (vecdb-qdrant-call provider 'get
+ (concat "/collections/"
(vecdb-collection-name collection) "/points/" id))))
(when result
(let ((point (plist-get result :result)))
(when point
- (make-embed-db-item :id (plist-get point :id)
- :vector (plist-get point :vector)
- :payload (plist-get point :payload)))))))
+ (make-vecdb-item :id (plist-get point :id)
+ :vector (plist-get point :vector)
+ :payload (plist-get point :payload)))))))
-(cl-defmethod embed-db-delete-items ((provider embed-qdrant-provider)
- (collection embed-db-collection)
- ids &optional sync)
+(cl-defmethod vecdb-delete-items ((provider vecdb-qdrant-provider)
+ (collection vecdb-collection)
+ ids &optional sync)
"Delete items from a collection in the qdrant database."
- (embed-qdrant-call provider 'post
- (concat "/collections/" (embed-db-collection-name
collection) "/points/delete?wait="
+ (vecdb-qdrant-call provider 'post
+ (concat "/collections/" (vecdb-collection-name
collection) "/points/delete?wait="
(if sync "true" "false"))
`(:points ,(apply #'vector ids))
nil
sync))
-(cl-defmethod embed-db-search-by-vector ((provider embed-qdrant-provider)
- (collection embed-db-collection)
- vector &optional limit)
+(cl-defmethod vecdb-search-by-vector ((provider vecdb-qdrant-provider)
+ (collection vecdb-collection)
+ vector &optional limit)
"Search for items in a collection in the qdrant database."
- (let ((result (embed-qdrant-call provider 'post
- (concat "/collections/"
(embed-db-collection-name collection) "/points/query")
+ (let ((result (vecdb-qdrant-call provider 'post
+ (concat "/collections/"
(vecdb-collection-name collection) "/points/query")
`(:query (:nearest ,vector)
:limit ,(or limit 10)
:with_payload t)
@@ -142,8 +142,9 @@ properties of the collection."
(let ((id (plist-get point :id))
(vector (plist-get point :vector))
(payload (plist-get point :payload)))
- (make-embed-db-item :id id :vector vector :payload payload)))
+ (make-vecdb-item :id id :vector vector :payload payload)))
(plist-get (plist-get result :result) :points))))
-(provide 'embed-qdrant)
-;;; embed-qdrant.el ends here
+(provide 'vecdb-qdrant)
+
+;;; vecdb-qdrant.el ends here
diff --git a/embed-db.el b/vecdb.el
similarity index 62%
rename from embed-db.el
rename to vecdb.el
index d2ff59b32c..c501ea3dba 100644
--- a/embed-db.el
+++ b/vecdb.el
@@ -1,9 +1,9 @@
-;;; embed-db.el --- An interface to embedding databases -*- lexical-binding:
t; -*-
+;;; vecdb.el --- An interface to vector databases for embeddings -*-
lexical-binding: t; -*-
;; Copyright (c) 2025 Free Software Foundation, Inc.
;; Author: Andrew Hyatt <ahy...@gmail.com>
-;; Homepage: https://github.com/ahyatt/embed-db
+;; Homepage: https://github.com/ahyatt/vecdb
;; Package-Requires: ((emacs "28.1") (plz "0.8"))
;; Package-Version: 0.1
;; SPDX-License-Identifier: GPL-3.0-or-later
@@ -29,11 +29,11 @@
(require 'cl-lib)
-(cl-defstruct embed-db-provider
+(cl-defstruct vecdb-provider
"A structure representing an embedding database provider."
name)
-(cl-defstruct embed-db-collection
+(cl-defstruct vecdb-collection
"A structure representing a collection in an embedding database.
The NAME field is the name of the collection.
@@ -45,52 +45,54 @@ PAYLOAD-FIELDS is an alist of fields and types (`integer',
`string', etc.)
conses, which are sometimes necessary for an initial setup."
name vector-size payload-fields)
-(cl-defstruct embed-db-item
+(cl-defstruct vecdb-item
"A structure representing data to be stored in an embedding database.
An ID may be an integer or a string, and is used to uniquely identify the
item."
id vector payload)
-(cl-defgeneric embed-db-create (provider collection)
+(cl-defgeneric vecdb-create (provider collection)
"Create a new collection of embeddings."
(ignore collection)
(signal 'not-implemented
- (list "embed-db-create not implemented for" (embed-db-provider-name
provider))))
+ (list "vecdb-create not implemented for" (vecdb-provider-name
provider))))
-(cl-defgeneric embed-db-delete (provider collection)
+(cl-defgeneric vecdb-delete (provider collection)
"Delete a collection of embeddings. This should remove all data."
(ignore collection)
(signal 'not-implemented
- (list "embed-db-delete not implemented for" (embed-db-provider-name
provider))))
+ (list "vecdb-delete not implemented for" (vecdb-provider-name
provider))))
-(cl-defgeneric embed-db-exists (provider collection)
+(cl-defgeneric vecdb-exists (provider collection)
"Check if a collection exists, return non-nil if it does."
(ignore collection)
(signal 'not-implemented
- (list "embed-db-exists not implemented for" (embed-db-provider-name
provider))))
+ (list "vecdb-exists not implemented for" (vecdb-provider-name
provider))))
-(cl-defgeneric embed-db-upsert-items (provider collection data-list &optional
sync)
- "Upsert a list of `embed-db-item' objects into the COLLECTION with PROVIDER."
+(cl-defgeneric vecdb-upsert-items (provider collection data-list &optional
sync)
+ "Upsert a list of `vecdb-item' objects into the COLLECTION with PROVIDER."
(ignore collection data-list sync)
(signal 'not-implemented
- (list "embed-db-upsert not implemented for" (embed-db-provider-name
provider))))
+ (list "vecdb-upsert not implemented for" (vecdb-provider-name
provider))))
-(cl-defgeneric embed-db-get-item (provider collection id)
+(cl-defgeneric vecdb-get-item (provider collection id)
"Get items with ID from the COLLECTION with PROVIDER."
(ignore collection id)
(signal 'not-implemented
- (list "embed-db-get not implemented for" (embed-db-provider-name
provider))))
+ (list "vecdb-get not implemented for" (vecdb-provider-name
provider))))
-(cl-defgeneric embed-db-delete-items (provider collection ids &optional sync)
+(cl-defgeneric vecdb-delete-items (provider collection ids &optional sync)
"Delete items with IDs from the COLLECTION with PROVIDER."
(ignore collection ids sync)
(signal 'not-implemented
- (list "embed-db-delete not implemented for" (embed-db-provider-name
provider))))
+ (list "vecdb-delete not implemented for" (vecdb-provider-name
provider))))
-(cl-defgeneric embed-db-search-by-vector (provider collection vector &optional
limit)
+(cl-defgeneric vecdb-search-by-vector (provider collection vector &optional
limit)
"Search for items in the COLLECTION with PROVIDER that are similar to
VECTOR."
(ignore collection vector limit)
(signal 'not-implemented
- (list "embed-db-search not implemented for" (embed-db-provider-name
provider))))
+ (list "vecdb-search not implemented for" (vecdb-provider-name
provider))))
-(provide 'embed-db)
+(provide 'vecdb)
+
+;; vecdb.el ends here
