Hi
we (@ hurence) have released on open source middleware based on
SparkStreaming over Kafka to do CEP and log mining, called *logisland*
(https://github.com/Hurence/logisland/) it has been deployed into
production for 2 years now and does a great job. You should have a look.
bye
Thomas Bailet
CTO : hurence
Le 18/10/17 à 22:05, Mich Talebzadeh a écrit :
As you may be aware the granularity that Spark streaming has is
micro-batching and that is limited to 0.5 second. So if you have
continuous ingestion of data then Spark streaming may not be granular
enough for CEP. You may consider other products.
Worth looking at this old thread on mine "Spark support for Complex
Event Processing (CEP)
https://mail-archives.apache.org/mod_mbox/spark-user/201604.mbox/%3CCAJ3fcbB8eaf0JV84bA7XGUK5GajC1yGT3ZgTNCi8arJg56=l...@mail.gmail.com%3E
HTH
Dr Mich Talebzadeh
LinkedIn
/https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw/
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk.Any and all responsibility for
any loss, damage or destruction of data or any other property which
may arise from relying on this email's technical content is explicitly
disclaimed. The author will in no case be liable for any monetary
damages arising from such loss, damage or destruction.
On 18 October 2017 at 20:52, anna stax <annasta...@gmail.com
<mailto:annasta...@gmail.com>> wrote:
Hello all,
Has anyone used spark streaming for CEP (Complex Event
processing). Any CEP libraries that works well with spark. I have
a use case for CEP and trying to see if spark streaming is a good
fit.
Currently we have a data pipeline using Kafka, Spark streaming and
Cassandra for data ingestion and near real time dashboard.
Please share your experience.
Thanks much.
-Anna
