I'm not sure the meaning of stateless aggregation for structured streaming.
The ground rule is that both batch query and streaming query should provide
same output if there's no late rows and no indeterministic
functions/operations - the query should not produce different outputs
depending on how the input is broken down to multiple batches. By the
definition of ground rule, streaming aggregation is required to be stateful.
Thanks,
Jungtaek Lim (HeartSaVioR)
On Thu, Nov 28, 2019 at 9:17 PM Chitral Verma <chitralve...@gmail.com>
wrote:
> Hi Devs,
> I have a query regarding stateless aggregations.
>
> I understand that its possible to do stateless aggregation using mapGroups
> and flatMapGroups API in Spark 2.x+. I want to use aggregate queries on
> a streaming registered temporary view. Is there any way to do the same
> using spark.sql(" ... ") ?
>
> Also posted here,
>
> https://stackoverflow.com/questions/59050663/is-it-possible-to-do-stateless-aggregations-using-spark-sql
>
> Any helps will be appreciated.
>
> Regards,
> Chitral Verma
>
