I agree with Alex, that a very short retention is probably a good solution,
if you want to minimize data loss when Flink is restarting.
If some small degree of data loss is okay, you could also consider
implementing a systlog connector for Flink.
I think that it is probably quite easy to create a
Why not to use Kafka with very short retention time?
On Sat, 4 Mar 2017 at 14:16, trang anh (Joey) wrote:
> Hi Alex,
>
> That is one of solutions we are looking right now, but the problem with us
> is the our system will be processing the huge data, few TB daily basis,
>
> So that we don’t want
Hi Alex,
That is one of solutions we are looking right now, but the problem with us
is the our system will be processing the huge data, few TB daily basis,
So that we don’t want store the data in our KAFKA cluster, we just want get
event data, do some processing and emitting to downstream systems
Hi Alex,
That is one of solutions we are looking right now, but the problem with us is
the our system will be processing the huge data, few TB daily basis,
So that we don’t want store the data in our KAFKA cluster, we just want get
event data, do some processing and emitting to downstream syst
Why not to send syslog data to Kafka, via rsyslog, or something like? From
my point of view this could be more flexible solution
On Sat, Mar 4, 2017 at 1:39 AM, trang anh (Joey)
wrote:
> Hi,
>
> I am looking for the Flink syslog source connector which can source data
> directly from syslog eve
Hi,
I am looking for the Flink syslog source connector which can source data
directly from syslog event as input data stream to Flink
I have checked the source connectors list, but seems there no such thing
available.
Appreciate any advise?
Regards
Joey
