Hi, > If I want to use the rate limiter in other connectors, such as Kafka sink, ES sink, I need to do some more work on these connectors. Yes, you can do this by changing Kafka/ES sink, actually, this is how we did internally.
> I'd like to know if the community has a plan to make a lower-level implementation for all connectors, also for table API and SQL? In my understanding, there is no on-going work on this. And usually we should leverage the back-pressure feature to do this. We can hear more from others whether this is a valid need. 王雷 <flink...@gmail.com> 于2020年5月17日周日 下午2:32写道： > Hi Benchao > > Thanks for your answer! > > According to your answer, I found `GuavaFlinkConnectorRateLimiter` which > is the implementation of the `FlinkConnectorRateLimiter`. > > If I want to use the rate limiter in other connectors, such as Kafka sink, > ES sink, I need to do some more work on these connectors. > > I'd like to know if the community has a plan to make a lower-level > implementation for all connectors, also for table API and SQL? > > Thanks > Ray > > Benchao Li <libenc...@gmail.com> 于2020年5月14日周四 下午5:49写道： > >> AFAIK, `FlinkKafkaConsumer010#setRateLimiter` can configure the kafka >> source to have a rate limiter. >> (I assume you uses Kafka) >> However it only exists in Kafka 0.10 DataStream Connector, not in other >> versions nor table api. >> >> 王雷 <flink...@gmail.com> 于2020年5月14日周四 下午5:31写道： >> >>> hi, All >>> >>> Does Flink support rate limitation? >>> How to limit the rate when the external database connected by the sink >>> operator has throughput limitation. >>> Instead of passive back pressure after reaching the limit of the >>> external database, we want to limit rate actively. >>> >>> Thanks >>> Ray >>> >> >> >> -- >> >> Benchao Li >> School of Electronics Engineering and Computer Science, Peking University >> Tel:+86-15650713730 >> Email: libenc...@gmail.com; libenc...@pku.edu.cn >> >> -- Benchao Li School of Electronics Engineering and Computer Science, Peking University Tel:+86-15650713730 Email: libenc...@gmail.com; libenc...@pku.edu.cn
