Congratulations, Qingsheng!
Best regards,
Zakelly
On Mon, Apr 24, 2023 at 11:52 AM Matthias Pohl
wrote:
>
> Congratulations, Qingsheng! :)
>
> On Mon, Apr 24, 2023, 05:17 Yangze Guo wrote:
>
> > Congratulations, Qingsheng!
> >
> > Best,
> > Yangze Guo
> >
> > On Mon, Apr 24, 2023 at 10:05 AM Sh
Congratulations, Leonard!
Best regards,
Zakelly
On Mon, Apr 24, 2023 at 3:25 PM Jing Ge wrote:
>
> Congrats! Leonard!
>
>
>
> Best regards,
>
> Jing
>
> On Mon, Apr 24, 2023 at 5:53 AM Matthias Pohl
> wrote:
>
> > Congrats, Leonard :)
> >
> > On Mon, Apr 24, 2023, 05:17 Yangze Guo wrote:
> >
like to put some more effort in this
> direction if this really turns out working well.
>
> One more thing I want to mention is that the proposed design shaded all the
> code changes and complexities in the newly introduced File management in
> TM. That says without enabling File merging
te to follow up on this discussion.
Best regards,
Zakelly
On Fri, Apr 28, 2023 at 12:03 AM Zakelly Lan wrote:
>
> Hi Yuan,
>
> Thanks for sharing your thoughts. Like you said, the code changes and
> complexities are shaded in the newly introduced file management in TM,
> while
fault value. For jobs with
> many tasks in a TM, it may be useful for file merging. However,
> it doesn't work well for jobs with a small number of tasks in a TM.
>
> I prefer just adding the `max-file-pool-size`, and
> the `pool size = number of tasks / max-file-pool-size`. WDYT?
&g
well.
2. There is a corner case that the directory may be left over when the
job stops, so I added some content in section 4.8.
Best,
Zakelly
[1]
https://docs.google.com/document/d/1NJJQ30P27BmUvD7oa4FChvkYxMEgjRPTVdO1dHLl_9I/edit#
On Fri, May 5, 2023 at 11:19 AM Zakelly Lan wrote:
>
>
gt; Thanks for the clarification!
>
> Currently, I understand what you mean, and LGTM.
>
> Best,
> Rui Fan
>
> On Fri, May 5, 2023 at 12:27 PM Zakelly Lan wrote:
>
> > Hi all,
> >
> > @Yun Tang and I have an offline discussion, and we agreed that:
> >
Hi everyone,
(I'm resending this vote since the last email was blocked for some reason.)
Thanks for all the feedback for FLIP-306: Unified File Merging
Mechanism for Checkpoints[1] on the discussion thread[2].
I'd like to start a vote for it. The vote will be open for at least 72
hours (until Ma
Hi Xiaogang Zhou,
Thanks for driving this!
I'm pretty interested in your verification test, could you please
provide more details? AFAIK, the performance is related to the format
of user data and the state size of the RocksDB, as well as the memory
setup (determines the proportion of IO required
Hi everyone,
Thanks for all the feedback for FLIP-306: Unified File Merging Mechanism for
Checkpoints[1] on the discussion thread[2].
I'd like to start a vote for it. The vote will be open for at least 72 hours
(until May 12th, 10:00 AM GMT) unless there is an objection or an insufficient
numb
Hi everyone,
(I'm resending this vote since the last email was blocked for some reason.)
Thanks for all the feedback for FLIP-306: Unified File Merging
Mechanism for Checkpoints[1] on the discussion thread[2].
I'd like to start a vote for it. The vote will be open for at least 72
hours (until Ma
Hi Xiaogang Zhou,
Thanks for driving this!
I'm pretty interested in your verification test, could you please
provide more details? AFAIK, the performance is related to the format
of user data and the state size of the RocksDB, as well as the memory
setup (determines the proportion of IO required
Hi everyone,
Thanks for all the feedback for FLIP-306: Unified File Merging
Mechanism for Checkpoints[1] on the discussion thread[2].
I'd like to start a vote for it. The vote will be open for at least 72
hours (until May 11th, 12:00AM GMT) unless there is an objection or an
insufficient number o
ding time, 72 hours means to open the discussion until May 12th.
> >
> > Best regards,
> > Jing
> >
> > On Tue, May 9, 2023 at 8:24 PM Zakelly Lan wrote:
> >
> > > Hi everyone,
> > >
> > > Thanks for all the feedback for FLIP-306: Unified File
Hi everyone,
Happy to announce that FLIP-306[1] has been approved!
According to the vote thread[2], there are 7 approving votes, out of
which 5 are binding:
* Yanfei Lei
* Yuan Mei (binding)
* Hangxiang Yu
* Rui Fan (binding)
* Yu Li (binding)
* Yun Tang (binding)
* Piotr Nowojski (binding)
And
Congratulations, Hangxiang!
Best regards,
Zakelly
On Mon, Aug 7, 2023 at 9:03 PM Lincoln Lee wrote:
>
> Congratulations Hangxiang!
>
> Best,
> Lincoln Lee
>
>
> Feifan Wang 于2023年8月7日周一 20:09写道：
>
> > Congratulations Hangxiang! :)
> >
> >
> >
> >
> >
> > ——
> > Name: Feifan Wang
> >
Congratulations, Yanfei!
Best regards,
Zakelly
On Mon, Aug 7, 2023 at 9:04 PM Lincoln Lee wrote:
>
> Congratulations, Yanfei!
>
> Best,
> Lincoln Lee
>
>
> Weihua Hu 于2023年8月7日周一 20:43写道：
>
> > Congratulations Yanfei!
> >
> > Best,
> > Weihua
> >
> >
> > On Mon, Aug 7, 2023 at 8:08 PM Feifan Wa
Hi everyone,
I am working on rebuilding the benchmark pipeline and it's almost
done. However, due to the change in machines for benchmarking, I will
need a few more days to run tests and gather the baseline scores for
further comparison. Once the pipeline is fully ready, we will proceed
with the p
Hi everyone,
I would like to initiate a discussion on FLIP-368, which focuses on
reorganizing the exceptions thrown in state interfaces [1].
Currently, we have identified several problems with the exceptions
thrown by state-related interfaces:
1. The exception types thrown by each interface are
hould be good enough.
> >
> > Thanks again!
> >
> > Best
> > Yuan
> >
> > On Tue, Sep 19, 2023 at 4:26 PM Zakelly Lan wrote:
> >
> > > Hi everyone,
> > >
> > > I am working on rebuilding the benchmark pipeline and it
want to implement it in ? Since it has to break
> >changes for users who have catched the IOException, If we want to
> > implement
> >it in 1.19, we must mark it very clearly in the release note, or we
> > should
> >make it in 2.0.
> >
> >
&g
Zakelly Lan wrote:
>
> Hi Hangxiang,
>
> Thank you for your response! Here are my thoughts:
>
> 1. Regarding the exceptions thrown by internal interfaces, I suggest
> keeping them as checked exceptions. Since these exceptions will be
> handled by the internal callers, it is mea
used up. Now the fix-PR[2] is ready for merging, I
> > > hope to backport it to release-1.18.
> > >
> > > Please let me know if you have any concerns. Thanks.
> > >
> > > [1] https://issues.apache.org/jira/browse/FLINK-32974
> > > [2] http
of
> > using unchecked Exceptions.
> >
> > Looking forward to your thoughts.
> >
> > Best regards,
> > Jing
> >
> >
> > [1]
> > https://docs.oracle.com/javase/tutorial/essential/exceptions/runtime.html
> > [2] https://github.com/apa
Hi Yunfeng and Dong,
Thanks for this FLIP. I have reviewed it briefly and have a few questions:
1. Is this FLIP proposing to buffer the output in the state backend?
If so, what is the data format of this buffer (what type of state does
it use and what is the value)? Additionally, how does the ope
akelly,
>
> No benchmark tests currently are affected by this issue. We
> may add benchmarks to guard it later. Thanks.
>
> Best,
> Yuxin
>
>
> Zakelly Lan 于2023年9月21日周四 11:56写道：
>
> > Hi Jing,
> >
> > Sure, I will run the benchmark with this fix
d be available after the
> weekend and there should be no big surprise. WDYT?
>
> Best regards,
> Jing
>
> [1]
> https://nightlies.apache.org/flink/flink-docs-master/release-notes/flink-1.15/#jdk-upgrade
>
> On Fri, Sep 22, 2023 at 11:26 AM Zakelly Lan wrote:
>
> >
haven't introduced FileSystemStateBackend if file-based map
> could be better. Could you please provide more illustrations on the
> pros & cons of state backend v.s. memory/filesystem?
>
> Best regards,
> Yunfeng
>
> On Thu, Sep 21, 2023 at 4:10 PM Zakelly Lan wrote:
>
> >
> > Best regards,
> > Jing
> >
> > On Fri, Sep 22, 2023 at 1:01 PM Zakelly Lan wrote:
> >
> > > Hi Jing,
> > >
> > > I agree we could wait for the result with Java 11. And it should be
> > > available next Monday.
> &
ith Flink, does not mean users don't expect to do something with
> the expected exception. Anyway, I am open to hearing different opinions.
>
> Best regards,
> Jing
>
> On Thu, Sep 21, 2023 at 7:02 AM Zakelly Lan wrote:
>
> > Hi Martijn,
> >
> > Thank
It's quite intuitive to provide such a configuration for sql gateway.
Thanks Yangze for bringing this up and looking forward to it.
Best,
Zakelly
On Sat, Oct 7, 2023 at 4:35 PM xiangyu feng wrote:
>
> Thanks for initiating this discussion. Within the development towards
> Streaming Warehousi
Hi everyone,
It seems we're gradually reaching a consensus. So I would like to
start a vote after 72 hours if there are no further discussions.
Please let me know if you have any concerns, thanks!
Best,
Zakelly
On Sat, Oct 7, 2023 at 4:07 PM Zakelly Lan wrote:
>
> Hi Jing,
>
&
for those cases; and be part of the contract with the caller. Can
> we be sure that there is no possibility that the state will become available;
> if so then I agree that a runtime Exception is appropriate. What do you think?
>
>
>
> Kind regards, David.
>
>
> From: Za
+1(non-binding)
Thanks for driving this.
Best,
Zakelly
On Wed, Oct 11, 2023 at 11:22 AM Weihua Hu wrote:
>
> +1(binding)
>
> Best,
> Weihua
>
>
> On Wed, Oct 11, 2023 at 10:56 AM xiangyu feng wrote:
>
> > +1(non-binding)
> >
> > Regards,
> > Xiangyu
> >
> > Shammon FY 于2023年10月11日周三 10:30写道：
Hi Jane,
The fine-grained TTL management is extremely useful for performance
tuning, so +1 for the idea. I have a minor suggestion: would it be
possible to provide a simple hint that allows the omission of the key?
For example, something like "SELECT /+ STATE_TTL('1h')/", which would
specify the T
r to throw exceptions if the TTL configuration
> > is mistakenly used as it will affect the data correctness.
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-33230
> >
> > Best
> > Yun Tang
> >
> > From: ConradJam
&
hould not be trying to create code now for an implementation
> > consideration that is not there yet,
> >
> > +1 from me ,
> > Kind regards, David.
> >
> > From: Zakelly Lan
> > Date: Wednesday, 11 October 2023 at 04:25
> > To: dev@flink.apache.org
&
<https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/hints/#dynamic-table-options>
> [3]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-113%3A+Supports+Dynamic+Table+Options+for+Flink+SQL
> <https://cwiki.apache.org/confluence/display/FLINK/FLIP-113
gt; callers that expect this exception to be thrown break, since the exceptions
> are part of the method's contract?
>
> Best regards,
> Jing
>
> On Fri, Oct 13, 2023 at 11:01 AM Zakelly Lan wrote:
>
> > Hi Yuan,
> >
> > Thanks for sharing your thou
Congratulations, Jane!
Best,
Zakelly
On Mon, Oct 16, 2023 at 12:51 PM yuxia wrote:
>
> Congratulations Jane!
>
> Best regards,
> Yuxia
>
> - 原始邮件 -
> 发件人: "Feng Jin"
> 收件人: "dev"
> 发送时间: 星期一, 2023年 10 月 16日 上午 11:29:26
> 主题: Re: [ANNOUNCE] New Apache Flink Committer - Jane Chan
>
> Con
Congratulations, Ron!
Best,
Zakelly
On Mon, Oct 16, 2023 at 12:04 PM Weihua Hu wrote:
>
> Congrats, Ron!
>
> Best,
> Weihua
>
>
> On Mon, Oct 16, 2023 at 11:50 AM Yangze Guo wrote:
>
> > Congrats, Ron!
> >
> > Best,
> > Yangze Guo
> >
> > On Mon, Oct 16, 2023 at 11:48 AM Matt Wang wrote:
> > >
Hi everyone,
Flink benchmarks [1] generate daily performance reports in the Apache
Flink slack channel (#flink-dev-benchmarks) to detect performance
regression [2]. Those benchmarks previously were running on several
machines donated and maintained by Ververica. Unfortunately, those
machines were
+1(non-binding)
Best,
Zakelly
On Mon, Oct 23, 2023 at 1:15 PM Benchao Li wrote:
>
> +1 (binding)
>
> Feng Jin 于2023年10月23日周一 13:07写道：
> >
> > +1(non-binding)
> >
> >
> > Best,
> > Feng
> >
> >
> > On Mon, Oct 23, 2023 at 11:58 AM Xuyang wrote:
> >
> > > +1(non-binding)
> > >
> > >
> > >
> > >
Congratulations and thank you all!
Best,
Zakelly
On Fri, Oct 27, 2023 at 12:39 PM Jark Wu wrote:
>
> Congratulations and thanks release managers and everyone who has
> contributed!
>
> Best,
> Jark
>
> On Fri, 27 Oct 2023 at 12:25, Hang Ruan wrote:
>
> > Congratulations!
> >
> > Best,
> > Hang
Hi Piotr,
Happy to see the trace! Thanks for this proposal.
One minor question: It is mentioned in the interface of Span:
Currently we don't support traces with multiple spans. Each span is
> self-contained and represents things like a checkpoint or recovery.
Does it mean the inclusion and sub
#x27;m not sure if I understand the proposal - I don't know how traces could
> > be used for this purpose?
> > Traces are perfect for one of events (like checkpointing, recovery, etc),
> > not for continuous monitoring
> > (like processing records). That's what
+1(non-binding)
Best,
Zakelly
On Wed, Nov 22, 2023 at 3:04 PM Hangxiang Yu wrote:
> +1 (binding)
> Thanks for driving this again!
>
> On Wed, Nov 22, 2023 at 10:30 AM Rui Fan <1996fan...@gmail.com> wrote:
>
> > +1(binding)
> >
> > Best,
> > Rui
> >
> > On Wed, Nov 22, 2023 at 6:43 AM Jing Ge
>
Hi, Rodrigo
It appears that the configurations you mentioned in your first question are
related to the flink kubernetes operator. Are you using the flink
kubernetes operator?
In regards to the cleaning behavior when users restore a job from a
savepoint or retained checkpoint, you can find detaile
This makes sense. +1 (non-binding)
Best,
Zakelly
On Thu, Dec 21, 2023 at 1:50 PM Jiabao Sun
wrote:
> +1 (non-binding)
>
> Best,
> Jiabao
>
>
> > 2023年12月21日 13:25，weijie guo 写道：
> >
> > +1(binding)
> >
> > Best regards,
> >
> > Weijie
> >
> >
> > Xin Jiang 于2023年12月21日周四 11:21写道：
> >
> >> 1.
Hi devs,
I'd like to start a discussion on FLIP-406: Reorganize State &
Checkpointing & Recovery Configuration[1].
Currently, the configuration options pertaining to checkpointing, recovery,
and state management are primarily grouped under the following prefixes:
- state.backend.* : configura
> The new option execution.checkpointing.dir is fine for me.
> > However, execution.checkpointing.savepoint.dir is a little weird.
> > I don't know which name is better now. Let us think about it more.
> >
> > 4. How about execution.recovery.claim-mode instead of
>
; `ExecutionCheckpointingOptions` will introduce some dependency
> > problems. Some options would be used by flink-runtime module, but
> > flink-runtime should not depend on flink-streaming-java. e.g.
> > FLINK-28286[1].
> > So, I prefer to move configurations to `CheckpointingOptions`,
inting.mode`, which is used
> to choose EXACTLY_ONCE or AT_LEAST_ONCE. Maybe we need to use another name
> or merge these two options.
>
> Best,
> Lijie
>
> Zakelly Lan 于2023年12月27日周三 11:43写道：
>
> > Hi everyone,
> >
> > Thanks all for your comment
t; [2]
> https://github.com/apache/flink/blob/master/flink-runtime-web/web-dashboard/src/app/pages/job/checkpoints/detail/job-checkpoints-detail.component.html#L27
> >
> > --
> > Best,
> > Yanfei
> >
> > Zakelly Lan 于2023年12月27日周三 14:41写道：
> > >
>
Hi Sue Alen,
AFAIK, Flink encountered memory leak issues with RocksDB block cache. The
root cause is the memory fragmentation with glibc. You may get some
information in FLINK-18712[1].
Actually there are some efforts from the Flink side, such as using jemalloc
as default in image[2].
I have no id
ures or improvements are useful
> for users, especially a lot of new flink users. Out-of-the-box options
> are good for users.
>
> WDYT?
>
> Best,
> Rui
>
> On Fri, Dec 29, 2023 at 1:45 PM Zakelly Lan wrote:
>
> > Hi everyone,
> >
> > Thanks all for y
Hi Xintong,
Thanks for driving this.
I would like to ask if there is any plan to review and refactor the CLI in
Flink 2.0. I recently found that the CLI commands and parameters are
confusing in some ways (e.g.
https://github.com/apache/flink/pull/23253#discussion_r1405707256). It
would be benefic
mentioned
> in your example. My biggest concern is whether there are contributors with
> enough capacity to work on it.
>
>
> Feel free to pick it up if you'd like to.
>
>
> Best,
>
> Xintong
>
>
>
> On Wed, Jan 3, 2024 at 11:37 AM Zakelly Lan wrote
Hi Piotr,
Thanks for driving this! Generally I support enabling the alignment timeout
for aligned checkpoint. And I second Rui's opinion, 30s seems a reasonable
value.
However I'm worried if there are some operators that do not support the
unaligned CP, which may cause data accuracy problems (as
r trying to clean this up! I don't have strong opinions on the
> topics discussed here, so generally speaking +1 from my side!
>
> Best,
> Piotrek
>
> śr., 3 sty 2024 o 04:16 Rui Fan <1996fan...@gmail.com> napisał(a):
>
> > Thanks for the feedback!
lso an issue if
> current local-recovery from enabled to disabled. Maybe another ticket is
> needed.
> 4. +1 for enabling execution.checkpointing.incremental by default which is
> basically default configuration in our production environment.
>
>
> On Mon, Jan 8, 2024 at 6:06 PM
since it is more in line with current practice
and friendly for existing users. Also It reduces the length of
configuration names as much as possible.
Looking forward to your opinions! Thanks!
Best,
Zakelly
On Wed, Jan 10, 2024 at 3:30 PM Zakelly Lan wrote:
> Hi Hangxiang,
>
> Thanks
Hi devs,
I'd like to start a discussion on FLIP-416: Deprecate and remove the
RestoreMode#LEGACY[1].
The FLIP-193[2] introduced two modes of state file ownership during
checkpoint restoration: RestoreMode#CLAIM and RestoreMode#NO_CLAIM. The
LEGACY mode, which was how Flink worked until 1.15, has
Hi everyone,
I would like to open a discussion on providing a unified file merging
mechanism for checkpoints[1].
Currently, many files are uploaded to the DFS during checkpoints,
leading to the 'file flood' problem when running
intensive workloads in a cluster. To tackle this problem, various
so
mended defaults?
>
>
> [1] https://issues.apache.org/jira/browse/FLINK-26590
>
> Best,
> Yanfei
>
> Zakelly Lan 于2023年4月3日周一 18:36写道：
>
> >
> > Hi everyone,
> >
> > I would like to open a discussion on providing a unified file merging
> > m
for unaligned checkpoints, unaligned checkpoint state would have been
> merged with the operators state file, so all in all there would be no
> regression visible to a user? The limit is that we always have at least a
> single file per subtask, but in exchange we are getting a simpler thr
s
> > > > correct, we did not take the option-3 or option-5 in the design doc [1]
> > > > for
> > > > the code complexity when implements the 1st version of changelog
> > > > state-backend.
> > > >
> > > > Could you also compare the current FLIP with th
e
different. I'm not sure if I made myself clear.
Thanks!
Best regards,
Zakelly
On Fri, Apr 7, 2023 at 8:08 PM Zakelly Lan wrote:
>
> Hi Yun,
>
> Thanks for your suggestions!
>
> I have read the FLINK-23342 and its design doc as you provided. First
> of all the goal
13 PM Zakelly Lan wrote:
>
> Hi Piotr,
>
> Thanks for your comments!
>
> (1) Sorry for the misleading, let me make it more clear. It is a
> concurrent checkpoint senario. Yes, the assumption you said needs to
> be followed, but the state handles here refer to the original SS
and investigation.
Thanks.
Best regards,
Zakelly
On Fri, Apr 7, 2023 at 8:15 PM Zakelly Lan wrote:
>
> Hi @Piotr and @Jingsong Li
>
> I have read access to the document, but I'm not sure whether the owner
> of this document wants to make it public. Actually, the doc is for
the stability of large-scale
> flink production. Looking forward to its completion and
> eventual acceptance by the community.
>
> [1]
> https://github.com/apache/flink/blob/65710b437318364ec19c0369d038acc10498/flink-state-backends/flink-statebackend-rocksdb/src
better have a comparing table across different options just
> like the doc wrote. And we could also list some of them in your Rejected
> Alternatives part.
>
>
> Best
> Yun Tang
>
> From: Zakelly Lan
> Sent: Tuesday, April 11, 2023 17:5
iving.
> > > > > >
> > > > > > Danny,
> > > > > >
> > > > > > On Sat, 13 Jan 2024, 08:20 Yanfei Lei,
> > wrote:
> > > > > >
> > > > > > > Thanks Zakelly for starting this discussion.
> > >
> > LGTM, Let's create a new FLIP to do this.
> >
> > IIUC, there is no clear ownership of the local copy files from the
> previous
> > > job and it's better to define one. This needs more discussion so we
> could
> > > create another thread for th
; Best,
> Rui
>
> On Tue, Jan 16, 2024 at 2:29 PM Zakelly Lan wrote:
>
> > Hi everyone,
> >
> > Thanks all for joining the discussion! I'd like to speed this up since it
> > lasts for nearly a month. I made changes on this FLIP based on
> suggestions
> >
Hi everyone,
I'd like to start a vote on the FLIP-416: Deprecate and remove the
RestoreMode#LEGACY [1]. The discussion thread is here [2].
The vote will be open for at least 72 hours unless there is an objection or
insufficient votes.
[1] https://cwiki.apache.org/confluence/x/ookkEQ
[2] https://
writing down the breaking change in the FLIP and
> let the community decide. For me, it might not be an optimized solution to
> introduce breaking changes between minor releases, but I personally won't
> block your effort.
>
> Best regards,
> Jing
>
> On Fri, Oc
> >
> > Best,
> > Yanfei
> >
> > Hangxiang Yu 于2024年1月19日周五 12:13写道：
> > >
> > > +1 (binding)
> > >
> > > On Fri, Jan 19, 2024 at 12:10 PM Zakelly Lan
> > wrote:
> > >
> > > > Hi everyone,
> > >
Hi devs,
I'm glad to announce that the FLIP-416[1] has been accepted. The voting
thread is here[2].
The proposal received 5 approving votes, all of which are binding:
- Yuan Mei (binding)
- Hangxiang Yu (binding)
- Yanfei Lei (binding)
- Rui Fan (binding)
- Yun Tang (binding)
And there is
Hi everyone,
It has been 6 days since the last call for discussion. I'd like to start a
vote after another 2 days.
Please let me know if you have any concerns. Thanks!
Best,
Zakelly
On Tue, Jan 16, 2024 at 2:54 PM Zakelly Lan wrote:
> Thanks for the suggestion Rui! The type
e.org/confluence/display/FLINK/FLIP-321%3A+Introduce+an+API+deprecation+process
Best,
Zakelly
On Mon, Jan 22, 2024 at 6:31 PM Zakelly Lan wrote:
> Hi everyone,
>
> It has been 6 days since the last call for discussion. I'd like to start a
> vote after another 2 days.
>
> Please
Hi everyone,
I'd like to start a vote on the FLIP-406: Reorganize State & Checkpointing
& Recovery Configuration [1]. The discussion thread is here [2].
The vote will be open for at least 72 hours unless there is an objection or
insufficient votes.
[1]
https://cwiki.apache.org/confluence/pages/v
Hi Jinzhong,
Thanks for driving this! +1 for fixing the lack of annotation.
I'm wondering if we really need to annotate *RocksDBStateUploader* and
*RocksDBStateDownloader
*with @Internal, as they seem to be ordinary classes without interacting
with other modules.
Also, I have reservations about a
> > +1 (binding)
> > > >
> > > > Hangxiang Yu 于2024年1月25日周四 10:00写道：
> > > > >
> > > > > +1 (binding)
> > > > >
> > > > > On Thu, Jan 25, 2024 at 8:49 AM Rui Fan <1996fan...@gmail.com>
> > wrote:
>
Hi devs,
I'm glad to announce that the FLIP-406[1] has been accepted. The voting
thread is here[2].
The proposal received 6 approving votes, 5 of which are binding:
- Rui Fan (binding)
- Hangxiang Yu (binding)
- Yanfei Lei (binding)
- Lijie Wang (binding)
- Xuannan Su (non-binding)
- Yuan M
Congratulations, Jiabao!
Best,
Zakelly
On Mon, Feb 19, 2024 at 9:40 PM Jing Ge wrote:
> Congrats! Jiabao!
>
> Best regards,
> Jing
>
> On Mon, Feb 19, 2024 at 2:38 PM Jeyhun Karimov
> wrote:
>
> > Congratulations, Jiabao!
> > Well deserved!
> >
> > On Mon, Feb 19, 2024 at 2:26 PM gongzhongqia
Hi devs,
When working on the FLIP-406[1], I realized that moving all options of
ExecutionCheckpointingOptions(flink-streaming-java) to
CheckpointingOptions(flink-core) depends on relocating the
enum CheckpointingMode(flink-streaming-java) to flink-core module. However,
the CheckpointingMode is ann
>> Thanks for driving this.
> > >> Moving this class to flink-core makes sense to me which could make the
> > code
> > >> path and configs clearer.
> > >> It's marked as @Public from 1.0 and 1.20 should be the next long-term
> > >> vers
d I think considering the state as
> "available" for sync access if it's in local disks (not RAM) is probably
> good enough (comparable to RocksDB).
>
> Best,
> Piotrek
>
> czw., 29 lut 2024 o 07:17 Yuan Mei napisał(a):
>
> > Hi Devs,
> >
> &
size thresholds and timeouts? These could help with
> memory consumption and latency analysis.
>
> 6. Why do we need to record the hashcode of a record in its
> RecordContext? It seems not used.
>
> 7. In "timers can be stored on the JVM heap or RocksDB", the link
ers (e.g., window size, watermark
> strategy, etc)
>
> - DFS consistency guarantees. The proposal in FLIP-427 is DFS-agnostic.
> However, different cloud providers have different storage consistency
> models.
> How do we want to deal with them?
>
> Regards,
> Jeyhun
>
&
file system layer conceals many underlying
details, so I guess the file consistency of different DFS is not a big
thing in our implementation. Of course, there may be some optimization when
dealing with different DFS, we may achieve this later.
Thanks again & Best,
Zakelly
On Mon, Mar 4, 2024 a
On Wed, Feb 28, 2024 at 6:16 PM Junrui Lee wrote:
>
> > Hi Zakelly,
> >
> > +1 for option 1. I prefer to minimize unnecessary additional development
> > and discussions due to internal code relocations and to avoid imposing
> > migration costs on users.
> >
>
t
>
> 8. Should this FLIP introduce metrics that measure the time a Flink
> job is back-pressured by State IOs? Under the current design, this
> metric could measure the time when the blocking buffer is full and
> yield() cannot get callbacks to process, which means the operator is
>
Hi Weijie,
Thanks for proposing this!
Unifying and optimizing state definitions is a very good thing. I like the
idea of 'definition goes before using', so overall +1 for this proposal.
However, I think the current definition is somewhat unclear. From a user's
point of view, I believe that state
+1 non-binding
Thanks for proposing this.
Best,
Zakelly
On Thu, Mar 7, 2024 at 10:13 AM Yanfei Lei wrote:
> +1(binding) for this vote.
>
> Hangxiang Yu 于2024年3月7日周四 09:54写道：
> >
> > +1 (binding)
> >
> > On Thu, Mar 7, 2024 at 9:34 AM Yun Tang wrote:
> >
> > > > +1 for this FLIP.
> > > Sorry
Hi devs,
I'd like to start a discussion on a sub-FLIP of FLIP-423: Disaggregated
State Storage and Management[1], which is a joint work of Yuan Mei, Zakelly
Lan, Jinzhong Li, Hangxiang Yu, Yanfei Lei and Feng Wang:
- FLIP-424: Asynchronous State APIs [2]
This FLIP introduces new API
于2024年3月5日周二 09:25写道：
> > > > >
> > > > > Hi Zakelly,
> > > > >
> > > > > > 5. I'm not very sure ... revisiting this later since it is not
> > > > important.
> > > > >
> > > > > It seems that w
ore like a
> flag.
> > > Can
> > > >> we simply have an Optional instead?
> > > >>
> > > >>
> > > >> We actually define three types RedistributionMode instead of two
> > because
> > > >> we don't want to
akes while coding(e.g. import wrong package by mistake etc.) and
> ease the job development with state?
>
> Best regards,
> Jing
>
>
> [1]
>
> https://github.com/apache/flink/blob/d6a4eb966fbc47277e07b79e7c64939a62eb1d54/flink-architecture-tests/flink-architecture-tests-pr
1 - 100 of 394 matches
Mail list logo
