Re: Flink Job Manager unable to recognize Task Manager Available slots

2022-05-24 Thread s_penakalap...@yahoo.com
Hi Team, Any inputs please badly stuck. Regards,Sunitha On Sunday, May 22, 2022, 12:34:22 AM GMT+5:30, s_penakalap...@yahoo.com wrote: Hi All, Help please! We have standalone Flink service installed in individual VM and clubed to form a cluster with HA and checkpoint in place. When

Flink Job Manager unable to recognize Task Manager Available slots

2022-05-21 Thread s_penakalap...@yahoo.com
Hi All, Help please! We have standalone Flink service installed in individual VM and clubed to form a cluster with HA and checkpoint in place. When cancelling Job, Flink cluster went down and its unable to start up normally as Job manager is continuously going down with the below error: 2022-05-

Flink Event specific window

2021-04-20 Thread s_penakalap...@yahoo.com
Hi All, I have one requirement where I need to calculate total amount of transactions done by each each user in last 1 hour.Say Customer1 has done 2 transactions one at 11:00am and other one at 11:20 am.Customer2 has done 1 transaction one at 10:00 am Customer3 has done 3 transactions one at 11:

Datastream Lag Windowing function

2021-02-21 Thread s_penakalap...@yahoo.com
Hi All, I am using Flink1.12, I am trying to read realtime data from Kafka topic and as per the requirement I need to implement windowing LAG function. Approach I followed is below: DataStream vData = env.addSource(...)vData.keyBy(Id) createTemperoryViewthen apply flink sql. My sample data is lik

Re: Realtime Data processing from HBase

2020-12-27 Thread s_penakalap...@yahoo.com
. ThanksDeepak On Mon, Dec 28, 2020 at 10:42 AM s_penakalap...@yahoo.com wrote: Hi Team, Kindly help me with some inputs.. I am using Flink 1.12. Regards,Sunitha. On Thursday, December 24, 2020, 08:34:00 PM GMT+5:30, s_penakalap...@yahoo.com wrote: Hi Team, I recently encountered one

Re: Realtime Data processing from HBase

2020-12-27 Thread s_penakalap...@yahoo.com
Hi Team, Kindly help me with some inputs.. I am using Flink 1.12. Regards,Sunitha. On Thursday, December 24, 2020, 08:34:00 PM GMT+5:30, s_penakalap...@yahoo.com wrote: Hi Team, I recently encountered one usecase in my project as described below: My data source is HBaseWe receive huge

Realtime Data processing from HBase

2020-12-24 Thread s_penakalap...@yahoo.com
Hi Team, I recently encountered one usecase in my project as described below: My data source is HBaseWe receive huge volume of data at very high speed to HBase tables from source system.Need to read from HBase, perform computation and insert to postgreSQL. I would like few inputs on the below poi

Re: Working with bounded Datastreams - Flink 1.11.1

2020-10-27 Thread s_penakalap...@yahoo.com
Hi All, Request your inputs please. Regards,Sunitha On Tuesday, October 27, 2020, 01:01:41 PM GMT+5:30, s_penakalap...@yahoo.com wrote: Hi Team, I want to use Flink Datastream for Batch operations which involves huge data, I did try to calculate count and average on the whole

Working with bounded Datastreams - Flink 1.11.1

2020-10-27 Thread s_penakalap...@yahoo.com
Hi Team, I want to use Flink Datastream for Batch operations which involves huge data, I did try to calculate count and average on the whole Datastream with out using window function.  Approach I tried to calculate count on the datastream:1> Read data from table (say past 2 days of data) as Data

Re: Flink Batch Processing

2020-09-29 Thread s_penakalap...@yahoo.com
could help here. Piotrek [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158866741 pon., 28 wrz 2020 o 15:14 s_penakalap...@yahoo.com napisał(a): Hi All, Need your help in Flink Batch processing: scenario described below: we have multiple vehicles, we get data from each vehicl

Flink Batch Processing

2020-09-28 Thread s_penakalap...@yahoo.com
Hi All, Need your help in Flink Batch processing: scenario described below: we have multiple vehicles, we get data from each vehicle at a very high speed, 1 record per minute.thresholds can be set by the owner for each vehicle.  Say: we have 3 vehicles, threshold is set for 2 vehicles. Vehicle 1,

Re: Flink alert after database lookUp

2020-09-15 Thread s_penakalap...@yahoo.com
a but this might be overkill for your usecase. The link that I've sent you to for streaming pattern slides should work after registration. Regards, Timo On 09.09.20 09:49, s_penakalap...@yahoo.com wrote: > > Hi Timo, > > Thank you for the suggestions. > > I see now both

Re: How to schedule Flink Batch Job periodically or daily

2020-09-15 Thread s_penakalap...@yahoo.com
m moving the dev@ list to bcc) Flink does not have facilities for scheduling batch jobs, and there are no plans to add such a feature (this is not in the scope of Flink, there are already a number of workflow management tools). On Fri, Sep 11, 2020 at 1:10 PM s_penakalap...@yahoo.com wrote: Hi

How to schedule Flink Batch Job periodically or daily

2020-09-11 Thread s_penakalap...@yahoo.com
Hi Team, We have Flink Batch Jobs which needs to be scheduled as listed below:Case 1 :    2.00 UTC time  dailyCase 2 :    Periodically 2 hours onceCase 3:     Schedule based on an event Request you to help me on this,  How to approach all the 3 use cases. Can we use Oozie workflows or any better

Re: Flink alert after database lookUp

2020-09-09 Thread s_penakalap...@yahoo.com
). Regards, Timo [1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/learn-flink/etl.html#connected-streams [2] https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/streaming/joins.html [3] https://training.ververica.com/decks/patterns/ On 07.09.20 17:25, s_penakalap

Flink alert after database lookUp

2020-09-07 Thread s_penakalap...@yahoo.com
Hi All, I am new to Flink, request your help!!! My scenario : 1> we receive Json messages at a very high frequency like 10,000 messages / second2> we need to raise an Alert for a particular user if there is any breach in threshold value against each attribute in Json.3> These threshold values ar