On Sat, 17 Jul 2021 at 6:37 AM, JING ZHANG (Jira) wrote:
> JING ZHANG created FLINK-23413:
> --
>
> Summary: Port RelTimeIndicatorConverter from SCALA to JAVA
> Key: FLINK-23413
> URL: https://issues.apache.org/jira/br
Aitozi created FLINK-23415:
--
Summary: Shade kryo library to reduce class conflict
Key: FLINK-23415
URL: https://issues.apache.org/jira/browse/FLINK-23415
Project: Flink
Issue Type: Improvement
JING ZHANG created FLINK-23414:
--
Summary: Split Match_ROWTIME return type converter into a
separator class
Key: FLINK-23414
URL: https://issues.apache.org/jira/browse/FLINK-23414
Project: Flink
JING ZHANG created FLINK-23413:
--
Summary: Port RelTimeIndicatorConverter from SCALA to JAVA
Key: FLINK-23413
URL: https://issues.apache.org/jira/browse/FLINK-23413
Project: Flink
Issue Type: Sub
Mans Singh created FLINK-23412:
--
Summary: Improve sourceSink description
Key: FLINK-23412
URL: https://issues.apache.org/jira/browse/FLINK-23412
Project: Flink
Issue Type: Improvement
Hi Flink Devs,
I am Srini, I work for stream processing team at LinkedIn. LinkedIn is
taking a big bet on Apache Flink and migrating all the existing streaming
SQL apps to Flink. You might have seen mails from some of our team members
past few months. Thanks a lot for your support!
I just wanted
Hi Till, Dawid
Very thanks for all the thoughts!
1) Regarding the event used to notify finish()
Logically there indeed two purposes to introduce the new event, one is for
working
around the limitations fo the unaligned checkpoint and one for notifying the
finish(),
and as Dawid has pointed ou
To avoid confusion, can we either rename "SourceMetricGroup" to "
SplitReaderMetricGroup" or add "Reader" to the setter method names?
Yes, we should add the "unassigned/pending splits" enumerator metric. I
tried to publish those metrics for IcebergSourceEnumerator and ran into an
issue [1]. I don
Hi everyone,
Since Flink 1.5 we have the same heartbeat timeout and interval default
values that are defined as heartbeat.timeout: 50s and heartbeat.interval:
10s. These values were mainly chosen to compensate for lengthy GC pauses
and blocking operations that were executed in the main threads of
Sure, thanks for the pointers.
Cheers,
Till
On Fri, Jul 16, 2021 at 6:19 PM Hausmann, Steffen
wrote:
> Hi Till,
>
> You are right, I’ve left out some implementation details, which have
> actually changed a couple of time as part of the ongoing discussion. You
> can find our current prototype he
Hi Till,
You are right, I’ve left out some implementation details, which have actually
changed a couple of time as part of the ongoing discussion. You can find our
current prototype here [1] and a sample implementation of the KPL free Kinesis
sink here [2].
I plan to update the FLIP. But I thi
Hi Steffen,
I've taken another look at the FLIP and I stumbled across a couple of
inconsistencies. I think it is mainly because of the lacking code. For
example, it is not fully clear to me based on the current FLIP how we
ensure that there are no in-flight requests when
AsyncSinkWriter.snapshotSt
Jun Qin created FLINK-23411:
---
Summary: Expose Flink checkpoint details metrics
Key: FLINK-23411
URL: https://issues.apache.org/jira/browse/FLINK-23411
Project: Flink
Issue Type: Improvement
Jun Qin created FLINK-23410:
---
Summary: Use a pool of KafkaProducers to commit Kafka Transactions
Key: FLINK-23410
URL: https://issues.apache.org/jira/browse/FLINK-23410
Project: Flink
Issue Type: I
Thanks for your responses Dawid and Yun,
I agree with you that the whole lifecycle section of the
StreamOperator/StreamTask/Task in the FLIP needs to be better fleshed out.
I have to admit that I am a bit confused by the new events you are
mentioning because I couldn't find them in the FLIP. I ass
Hi Till,
> Ok, so the plan is that finish() will flush all pending events and then send
> the MAX_WATERMARK.
> What I am missing is the connection between the lifecycle of the operator and
> signals the StreamTask and Task might receive (and also their life cycles).
> At the moment we do have
Hi all,
I must admit I think I agree with Till that the relation between
life-cycles of the (Stream)Task and the Operator is missing in the FLIP.
Moreover I am now a bit concerned that we missed the relation with
stopping with savepoint. I am still positive we can incorporate that in
the current e
Dawid Wysakowicz created FLINK-23409:
Summary: CrossITCase fails with "NoResourceAvailableException:
Slot request bulk is not fulfillable! Could not allocate the required slot
within slot request timeout"
Key: FLINK-23409
Dawid Wysakowicz created FLINK-23408:
Summary: Wait for a checkpoint completed after finishing a task
Key: FLINK-23408
URL: https://issues.apache.org/jira/browse/FLINK-23408
Project: Flink
Ingo Bürk created FLINK-23407:
-
Summary: Mechanism to document enums used for ConfigOptions
Key: FLINK-23407
URL: https://issues.apache.org/jira/browse/FLINK-23407
Project: Flink
Issue Type: Sub-
Hi all,
I'd like to start a vote on FLIP-184 [1] which was
discussed in [2] [3]. The vote will be open for at least 72 hours
until 7.21 unless there is an objection.
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-184%3A+Refine+ShuffleMaster+lifecycle+management+for+pluggable+shuffle+s
Ok, so the plan is that finish() will flush all pending events and then
send the MAX_WATERMARK.
What I am missing is the connection between the lifecycle of the operator
and signals the StreamTask and Task might receive (and also their life
cycles). At the moment we do have the EndOfPartitionEvent
Hi Till,
Sorry that I do not see the reply when sending the last mail, I think as a
whole we are on the same page for the
1. For normal stop-with-savepoint, we do not call finish() and do not emit
MAX_WATERMARK
2. For normal finish / stop-with-savepoint --drain, we would call finish() and
emit
Hi Till, Piotr
Very thanks for the comments!
> 1) Does endOfInput entail sending of the MAX_WATERMARK?
I also agree with Piotr that currently they are independent mechanisms, and
they are basically the same
for the event time.
For more details, first there are some difference among the three
I think we should try to sort this out because it might affect how and when
finish() will be called (or in general how the operator lifecycle looks
like).
To give an example let's take a look at the stop-with-savepoint w/ and w/o
--drain:
1) stop-with-savepoint w/o --drain: Conceptually what we w
I think this is a good idea. +1 for this approach. Are you gonna update the
FLIP accordingly?
Cheers,
Till
On Thu, Jul 15, 2021 at 9:33 PM Steven Wu wrote:
> I really like the new idea.
>
> On Thu, Jul 15, 2021 at 11:51 AM Piotr Nowojski
> wrote:
>
> > Hi Till,
> >
> > > I assume that buffer
26 matches
Mail list logo
