Unsubscribe

-- Regards, Harshvardhan

Re: Writing a custom Rocksdb statistics collector

native-metrics> > > Best > Yun Tang > -- > *From:* Harshvardhan Agrawal > *Sent:* Thursday, January 31, 2019 0:23 > *To:* user > *Subject:* Writing a custom Rocksdb statistics collector > > > Hi, > > I am currently trying to

Writing a custom Rocksdb statistics collector

eparate RocksDB options object for each of the slots. Is this a good way to approach this problem? Do you think this will work? Thanks in advance! :) -- *Regards,Harshvardhan Agrawal*

Take RocksDB state dump

Hello, We are currently using a RocksDBStateBackend for our Flink pipeline. We want to analyze the data that is stored in Rocksdb state.Is there a recommended process to do that? The sst_dump tool available from RocksDB isn’t working for us and we keep on getting errors like “Snappy not supported

Re: Between Checkpoints in Kafka 11

Hi, Can someone please help me understand how does the exactly once semantic work with Kafka 11 in Flink? Thanks, Harsh On Tue, Sep 11, 2018 at 10:54 AM Harshvardhan Agrawal < harshvardhan.ag...@gmail.com> wrote: > Hi, > > I was going through the blog post on how TwoPhaseComm

Between Checkpoints in Kafka 11

Hi, I was going through the blog post on how TwoPhaseCommitSink function works with Kafka 11. One of the things I don’t understand is: What is the behavior of the Kafka 11 Producer between two checkpoints? Say that the time interval between two checkpoints is set to 15 minutes. Will Flink buffer a

Re: Behaviour of Process Window Function

en you considerable flexibility, you can >> based on processing time / event time / timer / it's clear method / >> customized implementation, the specific design depends on your business >> logic, how long you need to save the cache. >> >> Thanks, vino. >

Behaviour of Process Window Function

the clear method will do the trick? -- *Regards,Harshvardhan Agrawal*

Re: Implement Joins with Lookup Data

> If your data is limited it can also be an All data cache. The All data > cache can be updated, say each 2h, according to our requirement. > > Adding a cache can not only simplify your pipeline but also improve the > job performance. > > Best, Hequn > > > On Mon,

Re: Implement Joins with Lookup Data

wrote: > > Yes, using Kafka which you initialize with the initial values and then > feed changes to the Kafka topic from which you consume could be a solution. > > On Tue, Jul 24, 2018 at 3:58 PM Harshvardhan Agrawal < > harshvardhan.ag...@gmail.com> wrote: > > Hi Till

Behavior of time based operators

Hello, I have recently started reading Stream Processing with Apache Flink by Fabian and Vasiliki. In Chapter 3 of the book there is a statement that says: None of the functions expose an API to set time stamps of emitted records, manipulate the event-time clock of a task, or emit watermarks. Inst

Re: Order of events in a Keyed Stream

ion* of >>> Kafka topic >>> >>> 2018-07-29 11:01 GMT+08:00 Hequn Cheng : >>> >>>> Hi harshvardhan, >>>> If 1.the messages exist on the same topic and 2.there are no rebalance >>>> and 3.keyby on the same field with same value,

Re: Order of events in a Keyed Stream

if you perform keyBy(), you should keyBy on a field the consecutive > two messages share the same value. > > Best, Hequn > > On Sat, Jul 28, 2018 at 12:11 AM, Harshvardhan Agrawal < > harshvardhan.ag...@gmail.com> wrote: > >> Hi, >> >> >> We are cur

Order of events in a Keyed Stream

Hi, We are currently using Flink to process financial data. We are getting position data from Kafka and we enrich the positions with account and product information. We are using Ingestion time while processing events. The question I have is: say I key the position datasream by account number. If

Re: Implement Joins with Lookup Data

; feed changes to the Kafka topic from which you consume could be a solution. > > On Tue, Jul 24, 2018 at 3:58 PM Harshvardhan Agrawal < > harshvardhan.ag...@gmail.com> wrote: > > Hi Till, > > How would we do the initial hydration of the Product and Account data > since

Re: Implement Joins with Lookup Data

it with the > incoming event. > > [1] > https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/operators/ > [2] > https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/streaming/api/functions/co/CoMapFunction.html > > Cheers, > Till >

Re: Implement Joins with Lookup Data

cron to enrich later since > your processing doesn’t seem to require absolute real time. > > > > Thanks > > Ankit > > > > *From: *Jörn Franke > *Date: *Monday, July 23, 2018 at 10:10 PM > *To: *Harshvardhan Agrawal > *Cc: * > *Subject: *Re: Implement Jo

Implement Joins with Lookup Data

I go with the first approach or the second one. If the second one, how can I implement the join? -- *Regards,Harshvardhan Agrawal*

Re: Behaviour of triggers in Flink

cts/flink/flink-docs-master/dev/stream/operators/windows.html#default-triggers-of-windowassigners > > On Sun, Jul 22, 2018 at 11:59 PM, Harshvardhan Agrawal < > harshvardhan.ag...@gmail.com> wrote: > >> Hi, >> >> I have been trying to understand how triggers work i

Triggers for late elements

understand the how this works? -- *Regards,Harshvardhan Agrawal*

Behaviour of triggers in Flink

uch a behaviour here? -- *Regards,Harshvardhan Agrawal* *267.991.6618 | LinkedIn <https://www.linkedin.com/in/harshvardhanagr/>*