Hi,
Thank Dawid and Piotr for sharing the problem. +1 to EndInput/Finish can be
called repeatedly.
Let me share some of my understanding of this problem, hoping to help
people with a similar problem.
At the beginning, what bothered me the most was: if the framework notified
a task `Finish & Snap
camilesing created FLINK-23465:
--
Summary: JobListener#onJobExecuted cannot be called when sql job
finished
Key: FLINK-23465
URL: https://issues.apache.org/jira/browse/FLINK-23465
Project: Flink
Chesnay Schepler created FLINK-23464:
Summary: Benchmarks aren't compiling
Key: FLINK-23464
URL: https://issues.apache.org/jira/browse/FLINK-23464
Project: Flink
Issue Type: Bug
Hi,
Thanks for driving this @Till Rohrmann . I would
give +1 on reducing the heartbeat timeout and interval, though I'm not sure
if 15s and 3s would be enough either.
IMO, except for the standalone cluster, where the heartbeat mechanism in
Flink is totally relied, reducing the heartbeat can also
Yangze Guo created FLINK-23463:
--
Summary: Replace the tag with ShortCodes in document
Key: FLINK-23463
URL: https://issues.apache.org/jira/browse/FLINK-23463
Project: Flink
Issue Type: Bug
Srinivasulu Punuru created FLINK-23462:
--
Summary: Translate the abfs documentation to chinese
Key: FLINK-23462
URL: https://issues.apache.org/jira/browse/FLINK-23462
Project: Flink
Issue
Hi Steven,
> I probably missed sth here. isn't this the case today already? Why is it
a concern for the proposed change?
The problem is with the newly added `finish()` method and the already
existing `endInput()` call. Currently on master there are no issues,
because we are not checkpointing any
> if a failure happens after sequence of finish() -> snapshotState(), but
before notifyCheckpointComplete(), we will restore such a state and we
might end up sending some more records to such an operator.
I probably missed sth here. isn't this the case today already? Why is it a
concern for the pr
Roman Khachatryan created FLINK-23461:
-
Summary: Consider disallowing in-memory state handles for
materialized state
Key: FLINK-23461
URL: https://issues.apache.org/jira/browse/FLINK-23461
Project
Dawid Wysakowicz created FLINK-23460:
Summary: Add a global flag for enabling/disabling final checkpoints
Key: FLINK-23460
URL: https://issues.apache.org/jira/browse/FLINK-23460
Project: Flink
Anton Kalashnikov created FLINK-23459:
-
Summary: New metrics for dynamic buffer size
Key: FLINK-23459
URL: https://issues.apache.org/jira/browse/FLINK-23459
Project: Flink
Issue Type: Sub
Anton Kalashnikov created FLINK-23458:
-
Summary: Document required number of buffers in the network stack
Key: FLINK-23458
URL: https://issues.apache.org/jira/browse/FLINK-23458
Project: Flink
Anton Kalashnikov created FLINK-23457:
-
Summary: Sending the buffer of the right size for broadcast
Key: FLINK-23457
URL: https://issues.apache.org/jira/browse/FLINK-23457
Project: Flink
Anton Kalashnikov created FLINK-23456:
-
Summary: Manually test on cluster
Key: FLINK-23456
URL: https://issues.apache.org/jira/browse/FLINK-23456
Project: Flink
Issue Type: Sub-task
Shengkai Fang created FLINK-23455:
-
Summary: Remove the usage of the yaml file in SQLJobSubmission
Key: FLINK-23455
URL: https://issues.apache.org/jira/browse/FLINK-23455
Project: Flink
Issue
Anton Kalashnikov created FLINK-23454:
-
Summary: Sending the buffer of the right size for unicast
Key: FLINK-23454
URL: https://issues.apache.org/jira/browse/FLINK-23454
Project: Flink
Is
Anton Kalashnikov created FLINK-23453:
-
Summary: Dynamic calculation of the buffer size
Key: FLINK-23453
URL: https://issues.apache.org/jira/browse/FLINK-23453
Project: Flink
Issue Type:
Anton Kalashnikov created FLINK-23452:
-
Summary: Measuring subtask throughput
Key: FLINK-23452
URL: https://issues.apache.org/jira/browse/FLINK-23452
Project: Flink
Issue Type: Sub-task
Anton Kalashnikov created FLINK-23451:
-
Summary: FLIP-183: Dynamic buffer size adjustment
Key: FLINK-23451
URL: https://issues.apache.org/jira/browse/FLINK-23451
Project: Flink
Issue Type
Timo Walther created FLINK-23450:
Summary: Properties map is not set in DebeziumAvroFormatFactory
Key: FLINK-23450
URL: https://issues.apache.org/jira/browse/FLINK-23450
Project: Flink
Issue
Hi everyone,
I would like to start a vote on the FLIP-182 [1] which was discussed in this
thread [2].
The vote will be open for at least 72 hours unless there is an objection
or not enough votes.
Best,
Piotrek
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-182%3A+Support+watermark+a
Hi,
> I would not fully advertise this before we have the second part
implemented as well.
I'm not sure, maybe we could advertise with a big warning about this
limitation. I mean it's not as if this change would break something. At
worst it just wouldn't fully solve the problem with multiple spl
+1 (binding)
Best
Yuan
On Wed, Jul 21, 2021 at 7:40 PM Piotr Nowojski wrote:
> +1 (binding)
>
> Piotrek
>
> śr., 21 lip 2021 o 13:21 Anton Kalashnikov
> napisał(a):
>
> > Hi everyone,
> >
> > I would like to start a vote on FLIP-183 [1] which was discussed in this
> > thread [2].
> > The vote
+1 (binding)
Piotrek
śr., 21 lip 2021 o 13:21 Anton Kalashnikov napisał(a):
> Hi everyone,
>
> I would like to start a vote on FLIP-183 [1] which was discussed in this
> thread [2].
> The vote will be open for at least 72 hours unless there is an objection
> or not enough votes.
>
>
> [1]
>
> h
Hi Dawid,
Thanks for writing down those concerns.
I think the first issue boils down what should be the contract of lifecycle
methods like open(), close(), initializeState() etc and especially the new
additions like finish() and endInput(). And what should be their relation
with the operator stat
Hey Chesnay,
I think I got what that method was designed for now. Basically the
motivation is to let the SourceOutput to report the eventTimeFetchLag for
users. At this point, the SourceOutput only has the EventTime, so this
method provides a way for the users to pass the FetchTime to the
SourceOu
Hi everyone,
I would like to start a vote on FLIP-183 [1] which was discussed in this
thread [2].
The vote will be open for at least 72 hours unless there is an objection
or not enough votes.
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-183%3A+Dynamic+buffer+size+adjustment
[2
Thanks everyone for sharing your opinion. I updated the FLIP according
to discussion and I'm going to start the vote on this FLIP
--
Best regards,
Anton Kalashnikov
16.07.2021 09:23, Till Rohrmann пишет:
I think this is a good idea. +1 for this approach. Are you gonna update the
FLIP according
Hey all,
To make the issues that were found transparent to the community, I want
to post an update:
*1. Committing side-effects*
We do want to make sure that all side effects are committed before
bringing tasks down. Side effects are committed when calling
notifyCheckpointComplete. For the final
Kai Chen created FLINK-23449:
Summary: YarnTaskExecutorRunner does not contains MapReduce
classes
Key: FLINK-23449
URL: https://issues.apache.org/jira/browse/FLINK-23449
Project: Flink
Issue Ty
Thanks for the update, Joe, Xintong and Dawid. This is very helpful.
Cheers,
Till
On Wed, Jul 21, 2021 at 9:27 AM Johannes Moser wrote:
> Dear Flink Community,
>
> here's the latest update from the bi-weekly.
> Short summary: still a lot of stuff is going on, the feature freeze has
> been
> pus
Great, thanks Godfrey.
Cheers,
Till
On Wed, Jul 21, 2021 at 7:42 AM godfrey he wrote:
> Hi Till,
>
> Sorry for the late reply. The previous period, I focused on another urgent
> work,
> and suspended the releasing work. I've recently restarted it.
>
> Best,
> Godfrey
>
> Till Rohrmann 于2021年7月
Thanks for sharing these insights.
I think it is no longer true that the ResourceManager notifies the
JobMaster about lost TaskExecutors. See FLINK-23216 [1] for more details.
Given the GC pauses, would you then be ok with decreasing the heartbeat
timeout to 20 seconds? This should give enough ti
+1 (binding)
Look forward to the production ready~
Best,
Jingsong
On Mon, Jul 19, 2021 at 3:25 PM Zhu Zhu wrote:
> +1 (binding)
>
> Thanks,
> Zhu
>
> XING JIN 于2021年7月19日周一 上午10:29写道：
>
> > +1 (non-binding)
> >
> > Best,
> > Jin
> >
> > Guowei Ma 于2021年7月19日周一 上午9:41写道：
> >
> > > +1(binding)
Daisy Tsang created FLINK-23448:
---
Summary: Update and fix up README
Key: FLINK-23448
URL: https://issues.apache.org/jira/browse/FLINK-23448
Project: Flink
Issue Type: Improvement
Comp
Dear Flink Community,
here's the latest update from the bi-weekly.
Short summary: still a lot of stuff is going on, the feature freeze has been
pushed by two weeks to the >>>16th of August<<< and we are working
on the release communication. Good stuff: the test and blockers are
looking very good.
|I'm happy to announce that we have unanimously approved this release.|
|There are 3 approving votes, 3 of which are binding:|
|* Timo|
|* Dawid|
|* Chesnay|
|There are no disapproving votes.|
|Thanks everyone!|
On 15/07/2021 09:02, Chesnay Schepler wrote:
Hi everyone,
Please review and vote on
+1
On 20/07/2021 19:54, Dawid Wysakowicz wrote:
+1 (binding)
* checked the website PR
* verified changed versions and NOTICE files since version 13.0
* verified the checksum and the signature
Best,
Dawid
On 19/07/2021 10:58, Timo Walther wrote:
+1 (binding)
I went through all commit
Chesnay Schepler created FLINK-23447:
Summary: Bump lz4-java to 1.8
Key: FLINK-23447
URL: https://issues.apache.org/jira/browse/FLINK-23447
Project: Flink
Issue Type: Improvement
39 matches
Mail list logo
