wenjin272 commented on code in PR #997:
URL: https://github.com/apache/flink-agents/pull/997#discussion_r3828732550
##########
integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java:
##########
@@ -336,23 +336,25 @@ public Map<String, Object> getStoreKwargs() {
/**
* Retrieve documents from the vector store.
*
- * <p>If ids is not provided, this method will retrieve documents
according to {@code limit},
- * {@code offset}, and {@code filter_query} in additional arguments. If
{@code limit} is null,
- * up to {@link ElasticsearchVectorStore#MAX_RESULT_WINDOW} documents are
returned (an
- * Elasticsearch ceiling).
+ * <p>When {@code ids} is non-empty, documents are retrieved directly by
ID and the filter,
+ * limit, offset, and {@code filter_query} arguments are not applied.
*
- * <p>The unified {@code filters} DSL parameter is not yet translated to
Elasticsearch's native
- * query DSL — callers needing structured filtering should pass a raw
{@code filter_query} via
- * {@code extraArgs}. TODO: implement equality-DSL translation parallel to
the Python Chroma
- * implementation.
+ * <p>Otherwise, {@code filters} provides equality-only matching against
document metadata. Each
+ * entry is translated to an Elasticsearch {@code term} query on {@code
+ * <metadataField>.<key>.keyword}, and multiple entries are combined with
AND semantics. A raw
+ * Elasticsearch JSON query may also be supplied as {@code filter_query}
in {@code extraArgs};
+ * when both forms are present, they are combined with AND semantics.
*
- * @param ids The ids of the documents.
- * @param collection The name of the collection to be retrieved. If is
null, retrieve the
- * default collection.
- * @param filters Unified filter DSL. Currently ignored — see method
Javadoc.
- * @param limit Maximum number of documents to return; falls back to {@link
- * ElasticsearchVectorStore#MAX_RESULT_WINDOW} when null.
- * @param extraArgs Additional arguments. (offset, filter_query, etc.)
+ * <p>The {@code limit} parameter takes precedence over a {@code limit}
value in {@code
+ * extraArgs}. If neither is provided, up to {@link
ElasticsearchVectorStore#MAX_RESULT_WINDOW}
Review Comment:
One nuance here: `MAX_RESULT_WINDOW` is only the request-size fallback used
by this implementation, while the actual result-window limit is controlled by
the per-index `index.max_result_window` setting. `10000` is only its default
value.
If the index setting is raised, a limit above `10000` may succeed; if it is
lowered, even the default request size may fail. Could we clarify this
distinction? For example:
> If neither is provided, the request size defaults to `MAX_RESULT_WINDOW`.
Elasticsearch rejects requests whose offset plus limit exceeds the index’s
`index.max_result_window` setting, which defaults to 10,000.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
