[
https://issues.apache.org/jira/browse/NIFI-10304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17936343#comment-17936343
]
Vijaya Gorla commented on NIFI-10304:
-------------------------------------
The use case is when you know that query will not match any new data once it
reaches the end of the result set (e.g. reading all logs from yesterday).
Problem with using {{ConsumeElasticsearch}} for this use case is
* Having to provide an additional range field
* Elastic API calls would continue to be made even after reaching the end of
the result set, putting load on elastic
* Processor does not yield after reaching the end of the result set, putting
load on NiFi
On a separate note, {{ConsumeElasticsearch}} requires range field to have
unique values. It should support multiple fields, one of them a tie breaker,
similar to {{search_after}}.
> SearchElasticsearch behaves differently to ScrollElasticsearchHttp, the
> processor it replaces
> ---------------------------------------------------------------------------------------------
>
> Key: NIFI-10304
> URL: https://issues.apache.org/jira/browse/NIFI-10304
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.16.3
> Reporter: Vijaya Gorla
> Priority: Major
> Labels: elasticsearch
> Time Spent: 20m
> Remaining Estimate: 0h
>
> ScrollElasticsearchHttp stops running once it hits the last page. This is a
> very useful feature for adhoc flows. SearchElasticsearch processor on the
> other hand restarts from the first page once it hits the end and there is no
> configuration to stop this.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
