thanks
G
On Wed, Jan 15, 2020 at 6:19 PM Robin Moffatt wrote:
> If spooldir doesn't suit, there's also
> https://github.com/streamthoughts/kafka-connect-file-pulse to check out.
> Also bear in mind tools like filebeat from Elastic support Kafka as a
> target.
>
>
> --
>
> Robin Moffatt | Senior
If spooldir doesn't suit, there's also
https://github.com/streamthoughts/kafka-connect-file-pulse to check out.
Also bear in mind tools like filebeat from Elastic support Kafka as a
target.
--
Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff
On Wed, 15 Jan 2020 at 12:48
Hi Tom
will do. for now I have 4 specific file types I need to ingest.
1. reading apache web server log files, http.log's.
2. reading in our custom log files
3. reading in log4j log files
4. mysql connection as a source
5. cassandra connection, as a sink
I can not use NFS mounting the source fil
Hi Robin
Ok, been reading, been asking some more questions and as far as something
like Spooldir is concerned, as a connector, I will need to NFS mount the
directory where the logs files are, be that our own custom files, or apache
http.log files onto the Connect cluster.
Yes a connector like a C
Hi George,
Since you mentioned CDC specifically you might want to check out Debezium (
https://debezium.io/) which operates as a connector of the sort Robin
referred to and does CDC for MySQL and others.
Cheers,
Tom
On Wed, Jan 15, 2020 at 10:18 AM Robin Moffatt wrote:
> The integration part
The integration part of Apache Kafka that you're talking about is
called Kafka Connect. Kafka Connect runs as its own process, known as
a Kafka Connect Worker, either on its own or as part of a cluster. Kafka
Connect will usually be deployed on a separate instance from the Kafka
brokers.
Kafka Con
Hi all.
Please advise, a real noob here still, unpacking how the stack still
works...
if I have a mySQL server, or a web server, or a 2 node JBOSS cluster.
If I want to use the mysql connector to connect to the MySQL DB to pull
data using CDC... then I need to install the Kafka stack on the DB s
