Thanks guys for the explanation, I will give it a try. After all buffers are filled, the back pressure did it’s job, it works so far so good, but I will defiantly give a try to control the latency.
Thanks again! Andrew > On 11 Apr 2016, at 18:19, Stephan Ewen <se...@apache.org> wrote: > > Hi! > > Ufuk's suggestion explains how to buffer less between Flink operators. > > Is that what you were looking for, or are you looking for a way to fetch more > fine grained in the source from the message queue? > What type of source are you using? > > Greetings, > Stephan > > > > > On Mon, Apr 11, 2016 at 5:02 PM, Ufuk Celebi <u...@apache.org > <mailto:u...@apache.org>> wrote: > Hey Andrew, > > take a look at this here: > https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/streaming/index.html#controlling-latency > > <https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/streaming/index.html#controlling-latency> > > Does this help? > > – Ufuk > > On Thu, Apr 7, 2016 at 3:04 PM, Andrew Ge Wu <andrew.ge...@eniro.com > <mailto:andrew.ge...@eniro.com>> wrote: > > Hi guys > > > > We have a prioritized queue, where high priority item can jump the queue > > and we do not want to cache too much record in the buffer. > > Is there a way to configure my streaming source to use less buffer? so > > source always fetch and get latest high prio records? > > > > Any suggestion? thanks! > > > > > > Andrew > > -- > > Confidentiality Notice: This e-mail transmission may contain confidential > > or legally privileged information that is intended only for the individual > > or entity named in the e-mail address. If you are not the intended > > recipient, you are hereby notified that any disclosure, copying, > > distribution, or reliance upon the contents of this e-mail is strictly > > prohibited and may be unlawful. If you have received this e-mail in error, > > please notify the sender immediately by return e-mail and delete all copies > > of this message. > -- Confidentiality Notice: This e-mail transmission may contain confidential or legally privileged information that is intended only for the individual or entity named in the e-mail address. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited and may be unlawful. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete all copies of this message.
