I am not familiar with all the details about Spark, however, the link you shared is for Spark SQL. I thought Spark SQL is for batch processing only?
Personally, I would be open to add more built-in aggregations next to count(). It did not come up in the community so far, so there was no investment yet. -Matthias On 4/28/21 10:30 AM, Parthasarathy, Mohan wrote: > Hi, > > Whenever the discussion about what streaming framework to use for > near-realtime analytics, there is normally a discussion about Spark vs Kafka > streaming. One of the points in favor of Spark streaming is the simple > aggregations that are built-in. See here: > https://sparkbyexamples.com/spark/spark-sql-aggregate-functions/. When it > comes to Kafka streams, there is boilerplate code for some of them. Is there > any reason why it is not provided as part of the library ? I am unable to > find any discussion on this topic. Are there any plans to provide such > features in the Kafka streaming library ? > > Thanks > Mohan >
