I’m no expert, but I think Kafka, Kafka Connect and Kafka Streams could be a great fit for your use case.
Use Kafka Connect to pull data from the various data sources, into a topic or topics and then use Kafka Streams to do the joins, enrichment, and / or analysis needed to determine a record of interest. Send those records to another topic and have a simple consumer app watching that topic and sending notifications. Sounds like a fun project! Dave > On Sep 2, 2020, at 11:01 AM, cedric sende lubuele <sende.ced...@hotmail.com> > wrote: > > Let me introduce myself, my name is Cedric and I am a network engineer > passionate about new technologies and as part of my new activity, I am > interested in Big Data. Currently, I live in Africa (Congo) and as everyone > knows, Africa is very late in terms of IT infrastructure (the Cloud is > difficult, we work a lot on premise). > > While doing some research, I came across Kai Waehner's article (Kafka replace > database?<https://www.kai-waehner.de/blog/2020/03/12/can-apache-kafka-replace-database-acid-storage-%20transactions-sql-nosql-data-lake%20/>) > and I would like to be able to get an idea about the possibilities of Kafka. > > Let me explain, I am working on a project for integrating several databases > (MySQL, MongoDB, Neo4j, ...) and I have to develop with my team, an alert > system which must detect anomalies on different criteria linked to a person > in the various departments of the company. > Would Kafka be a good solution in order to centralize all the data and create > several analysis scripts to detect an anomaly and send back an alert message > such as for example a suspect wanted by the police? > > Thank you in advance > > > > Sende Cedric / Network IT > sende.ced...@hotmail.com<mailto:sende.ced...@hotmail.com> / 082/8446954 > > UPSAIL GROUP > http://upsail.co/<https://htmlsig.com/t/000001BFBBXF> > > [http://upsail.co/wp-content/themes/upsail/images/logo.png]
