Hi Mans, is this what you are looking for [1][2]?
-- Rong [1] https://issues.apache.org/jira/browse/FLINK-11501 [2] https://github.com/apache/flink/pull/7679 On Mon, Nov 25, 2019 at 3:29 AM M Singh <mans2si...@yahoo.com> wrote: > Thanks Ciazhi & Thomas for your responses. > > I read the throttling example but want to see if that work with a > distributed broker like Kinesis and how to have throttling feedback to the > Kinesis source so that it can vary the rate without interfering with > watermarks, etc. > > Thanks again > > Mans > > > On Monday, November 25, 2019, 05:55:21 AM EST, Thomas Julian < > thomasjul...@zoho.com> wrote: > > > related > > https://issues.apache.org/jira/browse/FLINK-13792 > > Regards, > Julian. > > > ---- On Mon, 25 Nov 2019 15:25:14 +0530 *Caizhi Weng > <tsreape...@gmail.com <tsreape...@gmail.com>>* wrote ---- > > Hi, > > As far as I know, Flink currently doesn't have a built-in throttling > function. You can write your own user-defined function to achieve this. > Your function just gives out what it reads in and limits the speed it gives > out records at the same time. > > If you're not familiar with user-defined functions, see > https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/udfs.html > > Here is a throttling iterator example: > https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/utils/ThrottledIterator.java > > M Singh <mans2si...@yahoo.com> 于2019年11月25日周一 上午5:50写道： > > Hi: > > I have an Flink streaming application that invokes some other web > services. However the webservices have limited throughput. So I wanted to > find out if there any recommendations on how to throttle the Flink > datastream so that they don't overload the downstrream services. I am > using Kinesis as source and sink in my application. > > Please let me know if there any hooks available in Flink, what are the > patterns that can be used and what are the best practices/pitfalls for > using them. > > Thanks > > Mans > > > >
