Re: Flink Cassandra Sink - Setting TTL for the record

if you dont have this option in the conector API you need to implement a richsinkfunction with your desire logic. https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/streaming/api/functions/sink/RichSinkFunction.html On Mon, 4 Jun 2018, 21:32 Rohan Thimmappa, wrot

Re: flink JPS result changes

Flink 6 changed the execution model compactly https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65147077 https://docs.google.com/document/d/1zwBP3LKnJ0LI1R4-9hLXBVjOXAkQS5jKrGYYoPz-xRk/edit#heading=h.giuwq6q8d23j On Wed, Jul 11, 2018 at 5:09 PM Will Du wrote: > Hi folks > Do we

Re: Hbase state backend in Flink

Did try to use rocksdb[1] as state backend? 1. https://ci.apache.org/projects/flink/flink-docs-stable/ops/state/state_backends.html#the-rocksdbstatebackend On Thu, 27 Dec 2018, 18:17 Naveen Kumar Hi, > > I am exploring if we can plugin hbase as state backend in Flink. We have > need for stream

Re: ElasticSearch Connector

You can use flink to manipulate the data by using TimeCharacteristic.EventTime[1] and set Watermark. Then if you have a lag or other issue the data will be insert to the correct Indexes in elastic. More specific way to implement it with kafka[2] 1. https://ci.apache.org/projects/flink/flink-d

Re: Flink Elasticsearch Sink Issue

Did you set some checkpoints configuration? On Fri, Jun 21, 2019, 13:17 Ramya Ramamurthy wrote: > Hi, > > We use Kafka->Flink->Elasticsearch in our project. > The data to the elasticsearch is not getting flushed, till the next batch > arrives. > E.g.: If the first batch contains 1000 packets, t

Re: Get Window Time

Can you post the code please On 1 Nov 2017 16:58, "Erdem erdfem" wrote: > Hello, > > I have a datastream with sliding windows. I want to ask how can i get > which window pattern matches? > ex window 3 : 11m-21m > > [image: Satır içi resim 1] >

Re: Long running Flink job on yarn is getting failed after some time.

Can tou eleborate more please. On 13 Dec 2017 9:24, "Shivam Sharma" <28shivamsha...@gmail.com> wrote: > Hi, > > Flink version: 1.3.2 > > -- > Shivam Sharma > Data Engineer @ Goibibo > Indian Institute Of Information Technology, Design and Manufacturing > Jabalpur > Mobile No- (+91) 8882114744 >

Redis command

I have some streamed data by window time . the result is sent to redis has an hset . my problem is that each window iteration i can get different data from the stream and the earlier keys wont overwrite. as you can see in the image i have some new and old values ... Can i delete the hash before

Re: Redis command

i think the correct way is the SET TTl for each hashset ? but how can i do it from flink ? On Thu, Dec 21, 2017 at 2:49 PM, miki haiat wrote: > I have some streamed data by window time . > the result is sent to redis has an hset . > > my problem is that each window iterati

aggregate stream of logs

Hi , i have this scenario of applications that streaming logs to kafka . I want to use flink in order to aggregate the stream and calculate some data i have a guid that i can correlate the stream to single event and i have a field that i can use to understand if is the last stream of the event

Adding expire option to redis connector

Hi , We have a use that we need to collect some Meta data window it and group it by some rules. Each window can create different set of keys and value ,therefor the ability to set expire could be very helpful . i started to change the code , https://github.com/miko-code/bahir-flink/tree/master