That makes sense, thank you, Hequn. I can see the tradeoff between using
allowedLateness on a window to trigger multiple firings, versus a window
with a watermark lagging some amount of time (e.g. 3 hours) that has only a
single firing.
Thanks again,
--
Scott Kidder
On Fri, Oct 19, 2018 at 7:51
indow(Time.hours(1))
.allowedLateness(Time.hours(3))
.aggregate(new EnvironmentAggregateWatchTimeFunction())
.uid("env-watchtime-stats")
.name("Env Watch-Time Stats")
.addSink(new EnvironmentWatchTimeDBSink());
Thank you,
--
Scott Kidder
, Flink withl create a new Kafka transaction and processing will be
exactly-once.
--Scott Kidder
On Fri, Jul 27, 2018 at 5:09 AM Aljoscha Krettek
wrote:
> Hi,
>
> this has been in the back of my head for a while now. I finally created a
> Jira issue: https://issues.apache.org/jira/
point, the application is restarted and restores from a
checkpoint taken before the savepoint? It seems like this might lead to the
Kafka producer writing a message multiple times with different committed
Kafka transactions.
--
Scott Kidder
rther
off-list.
Best,
--Scott Kidder
On Mon, Jul 16, 2018 at 5:11 AM Sofer, Tovi wrote:
> Thank you Scott,
>
> Looks like a very elegant solution.
>
>
>
> How did you manage high availability in single data center?
>
>
>
> Thanks,
>
> Tovi
>
>
>
&g
d. It should be pretty easy to automate all of this. I
would not recommend setting up a multi-datacenter Zookeeper cluster; in my
experience, Consul is much easier to work with.
Best,
--
Scott Kidder
On Mon, Jul 9, 2018 at 4:48 AM Sofer, Tovi wrote:
> Hi all,
>
>
>
> We are now exam
I too was able to confirm that the issue was fixed in the 1.2
release-candidate 0 (zero) tag. I was unable to get it working with 1.1.4,
but with 1.2 it worked without any additional modifications.
Best,
--Scott Kidder
On Mon, Jan 9, 2017 at 7:03 AM, Aljoscha Krettek
wrote:
> I just verif
t,
> Stephan
>
>
> On Thu, Dec 29, 2016 at 10:36 PM, Scott Kidder
> wrote:
>
>> Are processing-time timers retained in checkpoint & savepoint state and
>> run if a job is restarted? It's possible that the time at which timer was
>> intended to trigger
Are processing-time timers retained in checkpoint & savepoint state and run
if a job is restarted? It's possible that the time at which timer was
intended to trigger at passes while the job is in the middle of being
restarted. Are processing-time timers persisted at all?
Thanks!
--Scott Kidder
triggered an alert. If it meets your alerting criteria, then send an alert
and update the accumulator state to reflect the most recent stock price &
timestamp.
Best,
--Scott Kidder
On Mon, Dec 19, 2016 at 7:15 AM, Rudra Tripathy wrote:
> Hi All,
>
> I have a use case where I am mon
mall 3-node
HDFS cluster in the same region as my Flink hosts but distributed across 3
AZs. The checkpointing is very fast and, most importantly, just works.
Is there a firm requirement to use S3, or could you use HDFS instead?
Best,
--Scott Kidder
On Tue, Nov 22, 2016 at 11:52 PM, Jonathan S
I'm curious how other Flink users are doing this. CRON job on the active
job-manager? Mostly just looking for a sanity-check.
Thanks!
--Scott Kidder
your cluster (avoid duplication of dependencies).
Best,
--Scott Kidder
On Mon, Nov 14, 2016 at 7:59 AM, Gyula Fóra wrote:
> Hi,
>
> I have been trying to use the -C flag to add external jars with user code
> and I have observed some strange behaviour.
>
> What I am trying to
customizations are visible in this commit on my fork:
https://github.com/apache/flink/commit/6d69f99d7cd52b3c2f039cb4d37518859e159b32
It might be worth testing with newer AWS SDK & KCL libraries to see if the
problem persists.
Best,
--Scott Kidder
On Thu, Nov 3, 2016 at 7:08 AM, Josh w
14 matches
Mail list logo
