That's exactly what I found yesterday! Thank you Aljoscha for confirming it!
On Mon, Aug 28, 2017 at 2:57 AM, Aljoscha Krettek
wrote:
> Hi Bowen,
>
> There is not built-in TTL but you can use a ProcessFunction to set a timer
> that clears state.
>
> ProcessFunction docs: https://ci.apache.org/pr
Hi Bowen,
There is not built-in TTL but you can use a ProcessFunction to set a timer that
clears state.
ProcessFunction docs:
https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/process_function.html
Best,
Aljoscha
> On 27. Aug 2017, at 19:19, Bowen Li wrote:
>
> Hi Rober
Hi Robert,
Thank you for the suggestion, I'll try that.
On a second thought, I can actually reduce the amount of generated
output so there aren't that many records being sent to Kinesis.
What I want to do is to use Flink's state to keep track of the last
computation result of a window
Hi Bowen,
I don't know what kind of relationship your company has to AWS, maybe they
are willing to look into the issue from their side.
To throttle a stream, I would recommend just doing a map operation that is
calling "Thread.sleep()" every n records.
On Sat, Aug 26, 2017 at 4:11 AM, Bowen Li
Hi Robert,
We use kinesis sink (FlinkKinesisProducer). The main pain is the Kinesis
Producer Library (KPL) that FlinkKinesisProducer uses.
KPL is basically a java wrapper with a c++ core. It's slow, unstable, easy
to crash, memory-and-CPU-consuming (it sends traffic via HTTP), and can't
handle hig
Hi Bowen,
(very nice graphics :) )
I don't think you can do anything about the windows itself (unless you are
able to build the windows yourself using the ProcessFunction, playing some
tricks because you know your data), so I should focus on reducing the pain
in "burning down your sink".
Are ther
