+1 (non-binding).
It would be a great improvement, thanks for the effort!
Regards,
Roman
On Tue, Oct 20, 2020 at 4:49 PM Steven Wu wrote:
> +1 (non-binding).
>
> Some of our users have asked for this tradeoff of consistency over
> availability for some cases.
>
> On Mon, Oct 19, 2020 at 8:02
27;m happy to propose a fix if someone is able to assign the ticket to me.
>
> Best,
> Austin
>
> On Mon, Oct 19, 2020 at 6:56 AM Khachatryan Roman <
> khachatryan.ro...@gmail.com> wrote:
>
>> Hey Austin,
>>
>> I think you are right. The problematic row c
Hi devs,
I'd like to start a discussion of FLIP-151: Incremental snapshots for
heap-based state backend [1]
Heap backend, while being limited state sizes fitting into memory, also has
some advantages compared to RocksDB backend:
1. Serialization once per checkpoint, not per state modification. Th
Hi,
I'd like FLINK-20079 [1] to be merged into 1.11 and included in 1.11.3.
[1] https://issues.apache.org/jira/browse/FLINK-20079
Regards,
Roman
On Tue, Nov 10, 2020 at 8:21 AM Xintong Song wrote:
> Thanks for the notice, Dian.
>
> Thank you~
>
> Xintong Song
>
>
>
> On Tue, Nov 10, 2020 at
g the versions on this event - this allows
> you to let go of old snapshots and saves you from writing a log of
> antimatter entries. Maybe the ideas are still useful to you.
>
> Best,
> Stefan
>
> On 2020/11/04 01:54:25, Khachatryan Roman wrote:
> > Hi devs,>
> >
>
+1 for the migration
(I agree with Dawid, for me the most important benefit is better support of
parameterized tests).
Regards,
Roman
On Mon, Nov 30, 2020 at 9:42 PM Arvid Heise wrote:
> Hi Till,
>
> immediate benefit would be mostly nested tests for a better test structure
> and new paramete
Thanks for starting this discussion (and sorry for probably duplicated
questions, I couldn't find them answered in FLIP or this thread).
1. Option 1 is said to be not preferable because it wastes resources and
adds complexity (new event).
However, the resources would be wasted for a relatively sho
tChannel.onBuffer() is called
> with
> EndOfPartition) and then taking snapshot for the input channels, as the
> normal unaligned checkpoints does for the InputChannel side. Then
> we would be able to ensure the finished tasks always have an empty state.
>
> I'll also optimi
Hi,
I'm not sure whether it's https://issues.apache.org/jira/browse/FLINK-20654 or
a new issue but I agree it shouldn't block 1.12.1.
Regards,
Roman
On Mon, Jan 11, 2021 at 10:30 AM Arvid Heise wrote:
> Hi Xingbo,
>
> This ticket is actually about
> https://issues.apache.org/jira/browse/FLINK-
Hi Yun,
> b) With unaligned checkpoint enabled, the slower cases might happen if
the downstream task processes very slowly.
I think UC will be the common case with multiple sources each with DoP > 1.
IIUC, waiting for EoP will be needed on each subtask each time one of it's
source subtask finishe
Thanks for doing this!
Some teams are currently doing (or already finalizing) the "usability
sprint". I was wondering whether it makes sense to release the implemented
features without waiting for the major changes. If it works well we could
decide to shorten the next release as well.
Regards,
Ro
Thanks Dawid, Yun and Piotr,
I think I know where the confusion comes from regarding arbitrarily
recovery histories: Both my counter-proposals were for "no-claim"
mode; I didn't mean to replace "claim" mode with them.
However, as Yun pointed out, it's impossible to guarantee that all the
files wil
Hi devs,
I'd like to start a discussion of FLIP-158: Generalized incremental
checkpoints [1]
FLIP motivation:
Low end-to-end latency is a much-demanded property in many Flink setups.
With exactly-once, this latency depends on checkpoint interval/duration
which in turn is defined by the slowest no
Hi Ingo,
Thanks a lot for this proposal!
We had a related discussion recently in the context of FLINK-19520
(randomizing tests configuration) [1].
I believe other scenarios will benefit as well.
For the end users, I think substitution in configuration files is
preferable over parsing env vars in
+1 (non-binding)
Thanks for driving the FLIP!
Regards,
Roman
On Tue, Jan 19, 2021 at 2:58 AM Guowei Ma wrote:
> +1 non-binding
> Best,
> Guowei
>
>
> On Fri, Jan 15, 2021 at 10:56 PM Yun Gao
> wrote:
>
> >
> > Hi all,
> >
> > I would like to start the vote for FLIP-147[1], which propose to s
Hey Dawid,
Replying to your penultimate message:
> We have never done something like this before
I thought releasing usability features can be such an experiment. My
reasoning is the following. As each release requires coordination of all
the participating teams, it will not scale beyond some poin
Hi,
I think having two Deprecated annotations (Flink and Java) may be confusing.
One alternative is to combine standard annotation with mandatory Javadocs
tags (checked with checkstyle).
And starting from Java 9 it has "since" and "forRemoval" arguments.
Regards,
Roman
On Wed, Jan 20, 2021 at
I agree with Till about $FLINK_PROPERTIES being only supported by a Flink
buildfile.
Besides that,
1. having different ways of configuring different applications doesn't seem
convenient to me. For example, Kafka and ZK configured via usual properties
and Flink via concatenated one.
2. It could also
@Chesnay
could you explain how underscores in user-defined properties would be
affected with transformation like STATE_BACKEND -> state.backend?
IIUC, this transformation happens in Flink and doesn't alter ENV vars, so
the user app can still parse the original configuration.
OTH, I'm a bit concerne
;t expect newline characters to appear within
>> > > DYNAMIC_VARIABLE, but I guess it would follow the same behavior as if
>> > > you would declare them on the command-line?
>> > >
>> > > One more downside I see is that from a users perspecti
>>
>> > In the end DYNAMIC_PROPERTIES behaves exactly like env.java.opts;
>> > meaning that the existing rules set by the JVM apply.
>> >
>> > Here's an example: export DYNAMIC_PROPERTIES='-Drest.port=1234
>> > -Dother.option="iContainAn=Sign&q
is thread forward:
>
> * Do you agree that approach 1 has been rejected?
> * Is there any objection to approach 2?
> * Did we consider whether approach 3 is good enough?
> * Is approach 4 a viable option after all? I think we should continue the
> discussion around the questions
n 28, 2021 at 10:46 AM Piotr Nowojski
> > wrote:
> >
> > > Hi Roman,
> > >
> > > +1 from my side on this proposal. Also big +1 for the recent changes in
> > > this FLIP in the motivation and high level overview sections.
> > >
> > >
Hi everyone,
I'd like to start a vote on FLIP-158 [1] which was discussed in [2].
The vote will be open for at least 72 hours. Unless there are any
objections,
I'll close it by February 5, 2021 if we have received sufficient votes.
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-158%3
Hi Konstantin,
I think we should try it out.
Even if tickets don't work well it can be a good step towards managing
technical debt in some other way, like wiki.
Thanks!
Regards,
Roman
On Tue, Mar 2, 2021 at 9:32 AM Dawid Wysakowicz
wrote:
> I'd be fine with dropping the "Trivial" priority in
Thanks for your reply Xingong Song,
Could you please elaborate on the following:
> The proposed changes break several good properties that we designed for
> managed memory.
> 1. It's isolated across slots
Just to clarify, any way to manage the memory efficiently while capping its
usage
will break
sted
> > > in many cases. This is probably a necessary price for this new feature,
> > but
> > > let's not break the concept / properties of managed memory for it.
> > >
> > > In your proposal, the fraction for the share managed memory is by
> d
gt; Thanks,
> Alexey
> --
> *From:* Alexey Trenikhun
> *Sent:* Friday, August 28, 2020 11:31 AM
> *To:* Khachatryan Roman
> *Cc:* Flink User Mail List
> *Subject:* Re: FileSystemHaServices and BlobStore
>
> Motivation is to have k8s HA setup without extra component - Zookeep
Hello,
Yesterday the machine executing Flink benchmarks was replaced due to
hardware problems.
The HW configuration is different, so the results may differ from what we
had previously.
Regards,
Roman
29 matches
Mail list logo
