[jira] [Created] (FLINK-37955) StreamingJoinOperator should not OOM due to collecting all matching records in memory

2025-06-13 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-37955: - Summary: StreamingJoinOperator should not OOM due to collecting all matching records in memory Key: FLINK-37955 URL: https://issues.apache.org/jira/browse/FLINK-37955

[jira] [Created] (FLINK-37853) FLIP-530: Dynamic job configuration

2025-05-26 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-37853: - Summary: FLIP-530: Dynamic job configuration Key: FLINK-37853 URL: https://issues.apache.org/jira/browse/FLINK-37853 Project: Flink Issue Type

Re: [VOTE] FLIP-530: Dynamic job configuration

2025-05-26 Thread Roman Khachatryan
i Fan (binding) - Kartikey Pant (non-binding) - Gustavo de Morais (non-binding) There are no disapproving votes. Thanks everyone! [1] https://cwiki.apache.org/confluence/x/uglKFQ Regards, Roman On Wed, May 21, 2025 at 9:56 AM Gustavo de Morais wrote: > Hi, > > +1 (non-binding). > &g

[VOTE] FLIP-530: Dynamic job configuration

2025-05-20 Thread Roman Khachatryan
https://lists.apache.org/thread/w1m420jx6h5cjv4rfy229xs00mmn7pwg Regards, Roman

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-19 Thread Roman Khachatryan
Thanks everyone for the discussion! I'm going to start a voting thread soon unless there are other suggestions or objections. Regards, Roman On Sat, May 17, 2025 at 2:01 PM Roman Khachatryan wrote: > Thanks Chesnay, I like your idea of returning 403 for non-white-listed > optio

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-17 Thread Roman Khachatryan
#x27;s a possibility of breaking the job via this API after passing the validation. For example, checkpoint timeout of 1 second would be valid, but might cause the checkpoints to fail.In such a case, configuration change should be reverted via a new PUT request. Regards, Roman On Thu, May 15, 202

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-14 Thread Roman Khachatryan
9 would be more appropriate. I've updated the document. Regards, Roman On Sun, May 11, 2025 at 10:36 AM Rui Fan <1996fan...@gmail.com> wrote: > Thanks Roman for driving this valuable proposal, it uses the Adaptive > Scheduler to greatly reduce the downtime of configuration updates, &

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-10 Thread Roman Khachatryan
some delay; and both are stored in HA immediately. So configuration change request results also in restarting and applying why pending resource requirements changes Regards, Roman On Fri, May 9, 2025, 05:10 Hangxiang Yu wrote: > Hi, Roman. > > Thanks for the FLIP. > +1 for supporti

[DISCUSS] FLIP-530: Dynamic job configuration

2025-05-08 Thread Roman Khachatryan
-endpoint with the support to read full job configuration; and update it. Looking forward to feedback. [1] https://cwiki.apache.org/confluence/x/uglKFQ Regards, Roman

Re: [VOTE] FLIP-511: Support transaction id pooling in Kafka connector

2025-03-27 Thread Roman Khachatryan
+1 (binding) Regards, Roman On Thu, Mar 27, 2025 at 1:18 AM Efrat Levitan wrote: > +1 (non-binding) > > Thanks for addressing this issue! > Efrat > > > On Wed, 26 Mar 2025 at 17:35 Arvid Heise wrote: > > > Dear devs, > > > > I'd like to start th

Re: [VOTE] FLIP-513: Split-level watermark metrics

2025-03-17 Thread Roman Khachatryan
+1 (binding) Regards, Roman On Mon, Mar 17, 2025 at 11:06 AM Efrat Levitan wrote: > Hi everyone, > > I'd like to start a vote on FLIP-513: Split-level watermark metrics [1] > which has been discussed in this thread [2]. > The vote will be open for at least 72 hour

Re: [VOTE] FLIP-481: Introduce Event Reporting

2024-12-06 Thread Roman Khachatryan
+1 (binding) Regards, Roman On Fri, Dec 6, 2024 at 10:19 AM Piotr Nowojski wrote: > Hi all! > > I would like to open the vote for FLIP-481 [1]. It has been discussed here > [2]. > > The vote will remain open for at least 72 hours (excluding the weekend). > > Best,

Re: [VOTE] FLIP-483: Add support for children Spans

2024-12-06 Thread Roman Khachatryan
+1 (binding) Regards, Roman On Fri, Dec 6, 2024 at 10:21 AM Piotr Nowojski wrote: > Hi all! > > I would like to open the vote for FLIP-483 [1]. It has been discussed here > [2]. > > The vote will remain open for at least 72 hours (excluding the weekend). > > Best,

Re: [VOTE] FLIP-484: Add custom metric variables to operators

2024-12-06 Thread Roman Khachatryan
+1 (binding) Regards, Roman On Fri, Dec 6, 2024 at 10:21 AM Piotr Nowojski wrote: > Hi all! > > I would like to open the vote for FLIP-484 [1]. It has been discussed here > [2]. > > The vote will remain open for at least 72 hours (excluding the weekend). > > Best,

Re: [VOTE] FLIP-482: Add OpenTelemetryEventReporter

2024-12-06 Thread Roman Khachatryan
+1 (binding) Regards, Roman On Fri, Dec 6, 2024 at 10:18 AM Piotr Nowojski wrote: > Hi all! > > I would like to open the vote for FLIP-482 [1]. It has been discussed here > [2]. > > The vote will remain open for at least 72 hours (excluding the weekend). > > Best,

Re: [DISCUSS] FLIP-483: Add support for children Spans

2024-12-04 Thread Roman Khachatryan
Thanks for the pointers, I also took a look at the Otel API and also couldn't find the way to "compose" a context instead of propagating it (which IMO wouldn't be desirable in Flink). So your proposal makes sense to me - as it keeps it simple. Regards, Roman On Tue, Dec

Re: [DISCUSS] FLIP-481: Introduce Event Reporting

2024-12-04 Thread Roman Khachatryan
stency is more important. Regards, Roman On Tue, Dec 3, 2024 at 12:07 PM Piotr Nowojski wrote: > Hi Roman! > > > 1. Should it list the events that would be emitted in the first version? > > If not as part of the proposed change, then maybe as examples > > I've added a

Re: [DISCUSS] Flink 1.20.1 release

2024-11-25 Thread Roman Khachatryan
+1, Thanks for volunteering! Regards, Roman On Mon, Nov 25, 2024 at 5:35 AM Zakelly Lan wrote: > +1 for this > > Thanks for driving! > > Best, > Zakelly > > On Mon, Nov 25, 2024 at 12:15 PM weijie guo > wrote: > > > Thanks for driving this! > > &g

[jira] [Created] (FLINK-36779) topn.cache.size metric is incorrect and non-changing during the time

2024-11-22 Thread Roman Boyko (Jira)
Roman Boyko created FLINK-36779: --- Summary: topn.cache.size metric is incorrect and non-changing during the time Key: FLINK-36779 URL: https://issues.apache.org/jira/browse/FLINK-36779 Project: Flink

[jira] [Created] (FLINK-36733) Don't transition task to RUNNING until the inputs are recovered (UC)

2024-11-15 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-36733: - Summary: Don't transition task to RUNNING until the inputs are recovered (UC) Key: FLINK-36733 URL: https://issues.apache.org/jira/browse/FLINK-

Re: [DISCUSS] FLIP-482: Add OpenTelemetryEventReporter

2024-11-14 Thread Roman Khachatryan
Hi Piotr, Adding OTel implementation makes sense, +1 for the proposal. Thanks Regards, Roman On Thu, Nov 7, 2024 at 2:37 PM Piotr Nowojski wrote: > Hi all! > > I would like to open up for discussion a new FLIP-482 [1]. > > Motivation > FLIP-481 [2] is adding the Event

Re: FLIP-484: Add custom metric variables to operators

2024-11-14 Thread Roman Khachatryan
Hi Piotr, thanks for the proposal, Can you please clarify 1. The scope of the variables added - is it only the last transformation? Do I understand correctly, that chaining does NOT affect this scoping? 2. Is Python API going to be supported as well? Thanks Regards, Roman On Thu, Nov 7, 2024

Re: [DISCUSS] FLIP-483: Add support for children Spans

2024-11-14 Thread Roman Khachatryan
ction 2. If at some point we come up with a way to address (1), then having a reference from child to parent would be more flexible? And probably not in the form of object reference, but just as a (String) identifier? Regards, Roman On Thu, Nov 7, 2024 at 2:41 PM Piotr Nowojski wrote: >

[jira] [Created] (FLINK-36714) sstmerge/CompactionScheduler termination can be interrupted

2024-11-14 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-36714: - Summary: sstmerge/CompactionScheduler termination can be interrupted Key: FLINK-36714 URL: https://issues.apache.org/jira/browse/FLINK-36714 Project: Flink

Re: [DISCUSS] FLIP-481: Introduce Event Reporting

2024-11-14 Thread Roman Khachatryan
It also adds a barrier to rename or move classes. So I'd remove the mention of Class. 3. Interface Event - Body Can you explain the purpose of this property? In my opinion, most information about an event can be stored in its attributes, except for when it's used by Flink itself; or is ve

[jira] [Created] (FLINK-36710) JobIDLoggingITCase fails on SourceCoordinator logs

2024-11-13 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-36710: - Summary: JobIDLoggingITCase fails on SourceCoordinator logs Key: FLINK-36710 URL: https://issues.apache.org/jira/browse/FLINK-36710 Project: Flink

[jira] [Created] (FLINK-36623) Improve logging in DefaultStateTransitionManager

2024-10-29 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-36623: - Summary: Improve logging in DefaultStateTransitionManager Key: FLINK-36623 URL: https://issues.apache.org/jira/browse/FLINK-36623 Project: Flink

[jira] [Created] (FLINK-36030) Don't use negative values for changelog recovery tests

2024-08-11 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-36030: - Summary: Don't use negative values for changelog recovery tests Key: FLINK-36030 URL: https://issues.apache.org/jira/browse/FLINK-36030 Project:

[jira] [Created] (FLINK-35970) Update documentation about FLINK-26050 (small file compaction)

2024-08-02 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-35970: - Summary: Update documentation about FLINK-26050 (small file compaction) Key: FLINK-35970 URL: https://issues.apache.org/jira/browse/FLINK-35970 Project

Re: [VOTE] FLIP-471: Fixing watermark idleness timeout accounting

2024-08-02 Thread Roman Khachatryan
+1 (binding) Regards, Roman On Wed, Jul 31, 2024 at 2:39 PM Rui Fan <1996fan...@gmail.com> wrote: > +1(binding) > > Best, > Rui > > Timo Walther 于2024年7月31日 周三17:47写道: > > > +1 (binding) > > > > Thanks for fixing this critical bug. > > >

[jira] [Created] (FLINK-35933) Skip distributing maxAllowedWatermark if there are no subtasks

2024-07-30 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-35933: - Summary: Skip distributing maxAllowedWatermark if there are no subtasks Key: FLINK-35933 URL: https://issues.apache.org/jira/browse/FLINK-35933 Project

[jira] [Created] (FLINK-35787) DefaultSlotStatusSyncer might bring down JVM (exit code 239 instead of a proper shutdown)

2024-07-08 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-35787: - Summary: DefaultSlotStatusSyncer might bring down JVM (exit code 239 instead of a proper shutdown) Key: FLINK-35787 URL: https://issues.apache.org/jira/browse/FLINK

[jira] [Created] (FLINK-35786) NPE in BlobServer / shutdownHook

2024-07-08 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-35786: - Summary: NPE in BlobServer / shutdownHook Key: FLINK-35786 URL: https://issues.apache.org/jira/browse/FLINK-35786 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-35769) State files might not be deleted on task cancellation

2024-07-05 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-35769: - Summary: State files might not be deleted on task cancellation Key: FLINK-35769 URL: https://issues.apache.org/jira/browse/FLINK-35769 Project: Flink

[jira] [Created] (FLINK-35742) Don't create RocksDB CF if task cancellation is in progress

2024-07-02 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-35742: - Summary: Don't create RocksDB CF if task cancellation is in progress Key: FLINK-35742 URL: https://issues.apache.org/jira/browse/FLINK-35742 Project:

Re: [VOTE] FLIP-444: Native file copy support

2024-06-26 Thread Roman Khachatryan
+1 (binding) Thanks for pushing this and updating the FLIP Regards, Roman On Wed, Jun 26, 2024 at 9:27 AM Piotr Nowojski wrote: > Thanks for pointing this out Zakelly. After the discussion on the dev > mailing list, I have updated the `PathsCopyingFileSystem` to merge its > functi

[jira] [Created] (FLINK-35557) MemoryManager only reserves memory per consumer type once

2024-06-07 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-35557: - Summary: MemoryManager only reserves memory per consumer type once Key: FLINK-35557 URL: https://issues.apache.org/jira/browse/FLINK-35557 Project: Flink

[jira] [Created] (FLINK-35556) Wrong constant in RocksDBSharedResourcesFactory.SLOT_SHARED_MANAGED

2024-06-07 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-35556: - Summary: Wrong constant in RocksDBSharedResourcesFactory.SLOT_SHARED_MANAGED Key: FLINK-35556 URL: https://issues.apache.org/jira/browse/FLINK-35556

[jira] [Created] (FLINK-35501) Use common thread pools when transferring RocksDB state files

2024-05-31 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-35501: - Summary: Use common thread pools when transferring RocksDB state files Key: FLINK-35501 URL: https://issues.apache.org/jira/browse/FLINK-35501 Project

Re: [VOTE] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-05-06 Thread Roman Khachatryan
+1 (binding) Regards, Roman On Mon, May 6, 2024 at 11:56 AM gongzhongqiang wrote: > +1 (non-binding) > > Best, > Zhongqiang Gong > > yue ma 于2024年5月6日周一 10:54写道: > > > Hi everyone, > > > > Thanks for all the feedback, I'd like to start a vote on th

Re: [DISCUSS] FLIP-444: Native file copy support

2024-04-30 Thread Roman Khachatryan
Hi Piotr, +1 for the proposal, the recovery time improvements are significant IMO Thanks for pushing this Regards, Roman On Tue, Apr 30, 2024 at 3:15 PM Piotr Nowojski wrote: > Hi all! > > I would like to put under discussion: > > FLIP-444: Native file copy

Re: [DISCUSS] FLIP-443: Interruptible watermark processing

2024-04-30 Thread Roman Khachatryan
Thanks for the proposal, I definitely see the need for this improvement, +1. Regards, Roman On Tue, Apr 30, 2024 at 3:11 PM Piotr Nowojski wrote: > Hi Yanfei, > > Thanks for the feedback! > > > 1. Currently when AbstractStreamOperator or AbstractStreamOperatorV2 > > pr

Re: [ANNOUNCE] New Apache Flink Committer - Zakelly Lan

2024-04-26 Thread Roman Khachatryan
Congrats, well deserved! Regards, Roman On Thu, Apr 18, 2024 at 9:06 AM xiangyu feng wrote: > Congratulations, Zakelly! > > > Regards, > Xiangyu Feng > > yh z 于2024年4月18日周四 14:27写道: > > > Congratulations Zakelly! > > > > Best regards, > > Yunho

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-22 Thread Roman Khachatryan
took part in some steps of this effort). Regards, Roman On Mon, Apr 22, 2024, 08:11 yue ma wrote: > Hi Flink devs, > > I would like to start a discussion on FLIP-447: Upgrade FRocksDB from > 6.20.3 to 8.10.0 > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-44

[jira] [Created] (FLINK-35184) Hash collision inside MiniBatchStreamingJoin operator

2024-04-21 Thread Roman Boyko (Jira)
Roman Boyko created FLINK-35184: --- Summary: Hash collision inside MiniBatchStreamingJoin operator Key: FLINK-35184 URL: https://issues.apache.org/jira/browse/FLINK-35184 Project: Flink Issue

Re: Support minibatch for TopNFunction

2024-04-10 Thread Roman Boyko
found in document [2] (under Nexmark subtitle) Looking forward to your feedback. [1] https://github.com/rovboyko/flink/tree/feature/topn-output-buffer [2] https://docs.google.com/document/d/1YPHwxKfiGSUOUOa6bc68fIJHO_UojTwZEC29VVEa-Uk -- Best regards, Roman Boyko e.: ro.v.bo...@gmail.com m

[jira] [Created] (FLINK-34994) JobIDLoggingITCase fails because of "checkpoint confirmation for unknown task"

2024-04-03 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-34994: - Summary: JobIDLoggingITCase fails because of "checkpoint confirmation for unknown task" Key: FLINK-34994 URL: https://issues.apache.org/jira/browse/F

Re: [VOTE] FLIP-425: Asynchronous Execution Model

2024-03-30 Thread Roman Khachatryan
+1 (binding) Regards, Roman On Fri, Mar 29, 2024 at 8:08 AM yue ma wrote: > +1 (non-binding) > > Yanfei Lei 于2024年3月27日周三 18:28写道: > > > Hi everyone, > > > > Thanks for all the feedback about the FLIP-425: Asynchronous Execution > > Model [1]. The discus

Re: [VOTE] FLIP-424: Asynchronous State APIs

2024-03-30 Thread Roman Khachatryan
+1 (binding) Regards, Roman On Fri, Mar 29, 2024 at 7:01 AM Xintong Song wrote: > +1 (binding) > > Best, > > Xintong > > > > On Fri, Mar 29, 2024 at 12:51 PM Yuepeng Pan > wrote: > > > +1(non-binding) > > > > Best, > > Yuepeng

Re: Support minibatch for TopNFunction

2024-03-27 Thread Roman Boyko
9VVEa-Uk <https://docs.google.com/document/d/1YPHwxKfiGSUOUOa6bc68fIJHO_UojTwZEC29VVEa-Uk/edit?usp=sharing> -- Best regards, Roman Boyko e.: ro.v.bo...@gmail.com m.: +79059592443 telegram: @rboyko On Thu, 28 Mar 2024 at 09:41, shuai xu wrote: > Hi, Roman > > Thanks for your proposal. I think t

Re: Support minibatch for TopNFunction

2024-03-25 Thread Roman Boyko
google.com/document/d/1YPHwxKfiGSUOUOa6bc68fIJHO_UojTwZEC29VVEa-Uk <https://docs.google.com/document/d/1YPHwxKfiGSUOUOa6bc68fIJHO_UojTwZEC29VVEa-Uk/edit?usp=sharing> [2] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/topn/#no-ranking-output-optimization -- Bes

Re: Support minibatch for TopNFunction

2024-03-23 Thread Roman Boyko
Hi Flink Community, I tried to describe my idea about minibatch for TopNFunction in this doc - https://docs.google.com/document/d/1YPHwxKfiGSUOUOa6bc68fIJHO_UojTwZEC29VVEa-Uk/edit?usp=sharing Looking forward to your feedback, thank you On Tue, 19 Mar 2024 at 12:24, Roman Boyko wrote: > He

Support minibatch for TopNFunction

2024-03-18 Thread Roman Boyko
] https://cwiki.apache.org/confluence/display/FLINK/FLIP-415 %3A+Introduce+a+new+join+operator+to+support+minibatch -- Best regards, Roman Boyko e.: ro.v.bo...@gmail.com m.: +79059592443 telegram: @rboyko

[jira] [Created] (FLINK-34694) Delete num of associations for streaming outer join

2024-03-15 Thread Roman Boyko (Jira)
Roman Boyko created FLINK-34694: --- Summary: Delete num of associations for streaming outer join Key: FLINK-34694 URL: https://issues.apache.org/jira/browse/FLINK-34694 Project: Flink Issue Type

[jira] [Created] (FLINK-34559) TVF Window Aggregations might stuck

2024-03-01 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-34559: - Summary: TVF Window Aggregations might stuck Key: FLINK-34559 URL: https://issues.apache.org/jira/browse/FLINK-34559 Project: Flink Issue Type

[jira] [Created] (FLINK-34420) Various YARN tests fail after failing to download hadoop.tar.gz

2024-02-10 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-34420: - Summary: Various YARN tests fail after failing to download hadoop.tar.gz Key: FLINK-34420 URL: https://issues.apache.org/jira/browse/FLINK-34420 Project

[jira] [Created] (FLINK-34417) Add JobID to logging MDC

2024-02-08 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-34417: - Summary: Add JobID to logging MDC Key: FLINK-34417 URL: https://issues.apache.org/jira/browse/FLINK-34417 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-34344) Wrong JobID in CheckpointStatsTracker

2024-02-02 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-34344: - Summary: Wrong JobID in CheckpointStatsTracker Key: FLINK-34344 URL: https://issues.apache.org/jira/browse/FLINK-34344 Project: Flink Issue Type

Re: [VOTE] FLIP-386: Support adding custom metrics in Recovery Spans

2023-11-23 Thread Roman Khachatryan
+1 (binding) Regards, Roman On Wed, Nov 22, 2023 at 12:55 PM Rui Fan <1996fan...@gmail.com> wrote: > +1(binding) > > Thanks for driving this proposal! > > Best, > Rui > > On Wed, Nov 22, 2023 at 7:44 PM Piotr Nowojski > wrote: > > > Hi All, > &

Re: [VOTE] FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter

2023-11-22 Thread Roman Khachatryan
+1 (binding) Regards, Roman On Wed, Nov 22, 2023, 7:30 AM Hangxiang Yu wrote: > +1(binding) > > On Wed, Nov 22, 2023 at 10:29 AM Rui Fan <1996fan...@gmail.com> wrote: > > > +1(binding) > > > > Best, > > Rui > > > > On Wed, Nov 22, 2023

Re: [VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-22 Thread Roman Khachatryan
+1 (binding) Regards, Roman On Wed, Nov 22, 2023, 7:08 AM Zakelly Lan wrote: > +1(non-binding) > > Best, > Zakelly > > On Wed, Nov 22, 2023 at 3:04 PM Hangxiang Yu wrote: > > > +1 (binding) > > Thanks for driving this again! > > > > On W

Re: [VOTE] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-22 Thread Roman Khachatryan
+1 (binding) Thanks for the proposal Regards, Roman On Wed, Nov 22, 2023, 10:08 AM Piotr Nowojski wrote: > Thanks Rui! > > +1 (binding) > > Best, > Piotrek > > śr., 22 lis 2023 o 08:05 Hangxiang Yu napisał(a): > > > +1 (binding) > > Thanks for your e

Re: [DISCUSS] FLIP-386: Support adding custom metrics in Recovery Spans

2023-11-21 Thread Roman Khachatryan
hould be used. How about dropping it from the API and always using min, max, sum, avg? I think we're interested in these aggregations for all the metrics, and there is no penalty for reporting all of them because it's only for initialization. Regards, Roman On Mon, Nov 20, 2023, 8:42 AM

[jira] [Created] (FLINK-33590) CheckpointStatsTracker.totalNumberOfSubTasks not updated

2023-11-17 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-33590: - Summary: CheckpointStatsTracker.totalNumberOfSubTasks not updated Key: FLINK-33590 URL: https://issues.apache.org/jira/browse/FLINK-33590 Project: Flink

Re: [DISCUSS] FLIP-386: Support adding custom metrics in Recovery Spans

2023-11-16 Thread Roman Khachatryan
ike the idea of introducing parameter objects for backend creation. Regards, Roman On Tue, Nov 7, 2023, 1:20 PM Piotr Nowojski wrote: > (Fixing topic) > > wt., 7 lis 2023 o 09:40 Piotr Nowojski napisał(a): > > > Hi all! > > > > I would like to start a discussion on a follo

Re: [DISCUSS] FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter

2023-11-16 Thread Roman Khachatryan
Thanks Piotr, the proposal totally makes sense to me. Does it depend on FLIP-384 for voting? Otherwise, we could probably start the vote already as there're no counter proposals or objections. Regards, Roman On Tue, Nov 7, 2023, 1:19 PM Piotr Nowojski wrote: > Hey, sorry for the

Re: [DISCUSS] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-16 Thread Roman Khachatryan
Thanks for the proposal, Starting with the minimal functionality and expanding if necessary as the FLIP describes makes a lot of sense to me. Regards, Roman On Wed, Nov 15, 2023, 9:31 PM Jing Ge wrote: > Hi Piotr, > > Sorry for the late reply and thanks for the proposal, it look

[jira] [Created] (FLINK-33442) UnsupportedOperationException thrown from RocksDBIncrementalRestoreOperation

2023-11-02 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-33442: - Summary: UnsupportedOperationException thrown from RocksDBIncrementalRestoreOperation Key: FLINK-33442 URL: https://issues.apache.org/jira/browse/FLINK-33442

[jira] [Created] (FLINK-33110) Array content gets replaced with last element duplicates

2023-09-18 Thread Roman Lupiichuk (Jira)
Roman Lupiichuk created FLINK-33110: --- Summary: Array content gets replaced with last element duplicates Key: FLINK-33110 URL: https://issues.apache.org/jira/browse/FLINK-33110 Project: Flink

Re: [ANNOUNCE] New Apache Flink Committer - Yanfei Lei

2023-08-14 Thread Roman Khachatryan
Congratulations, Yanfey! Regards, Roman On Wed, Aug 9, 2023 at 12:49 PM Benchao Li wrote: > Congrats, YanFei! > > Jing Ge 于2023年8月8日周二 17:41写道: > > > Congrats, YanFei! > > > > Best regards, > > Jing > > > > On Tue, Aug 8, 2023 at 3:04 

Re: [ANNOUNCE] New Apache Flink Committer - Hangxiang Yu

2023-08-14 Thread Roman Khachatryan
Congratulations, Hangxiang! Regards, Roman On Wed, Aug 9, 2023 at 12:49 PM Benchao Li wrote: > Congrats, Hangxiang! > > Jing Ge 于2023年8月8日周二 17:44写道: > > > Congrats, Hangxiang! > > > > Best regards, > > Jing > > > > On Tue, Aug 8, 2023

Re: [VOTE] FLIP-304: Pluggable Failure Enrichers

2023-04-20 Thread Roman Khachatryan
+1 (binding) The FLIP LGTM, thanks Panos! Regards, Roman On Thu, Apr 20, 2023 at 1:33 PM Hong Teoh wrote: > +1 (non-binding) > > Thank you for driving this effort, Panagiotis. > > Regards, > Hong > > > > On 20 Apr 2023, at 12:16, David Morávek wrote:

[jira] [Created] (FLINK-31601) While waiting for resources, resources check might be scheduled unlimited number of times (Adaptive Scheduler)

2023-03-23 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-31601: - Summary: While waiting for resources, resources check might be scheduled unlimited number of times (Adaptive Scheduler) Key: FLINK-31601 URL: https://issues.apache.org

Re: [VOTE] FLIP-291: Externalized Declarative Resource Management

2023-03-01 Thread Roman Khachatryan
+1 (binding) Thanks David, and everyone involved :) Regards, Roman On Wed, Mar 1, 2023 at 8:01 AM Gyula Fóra wrote: > +1 (binding) > > Looking forward to this :) > > Gyula > > On Wed, 1 Mar 2023 at 04:02, feng xiangyu wrote: > > > +1 (non-binding) > >

[jira] [Created] (FLINK-31261) Make AdaptiveScheduler aware of the (local) state size

2023-02-28 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-31261: - Summary: Make AdaptiveScheduler aware of the (local) state size Key: FLINK-31261 URL: https://issues.apache.org/jira/browse/FLINK-31261 Project: Flink

Re: [DISCUSS] FLIP-291: Externalized Declarative Resource Management

2023-02-28 Thread Roman Khachatryan
other setups, where the job should not be stopped at all, the user can always set it to 0. Regards, Roman On Tue, Feb 28, 2023 at 12:58 PM Maximilian Michels wrote: > Hi David, > > Thanks for the update! We consider using the new declarative resource > API for autoscaling. Currently

Re: [ANNOUNCE] New Apache Flink Committer - Rui Fan

2023-02-21 Thread Roman Khachatryan
Congratulations Rui! Regards, Roman On Mon, Feb 20, 2023 at 5:58 PM Anton Kalashnikov wrote: > Congrats Rui! > > -- > Best regards, > Anton Kalashnikov > > On 20.02.23 17:53, Matthias Pohl wrote: > > Congratulations, Rui :) > > > > On Mon, F

Re: [ANNOUNCE] New Apache Flink Committer - Anton Kalashnikov

2023-02-21 Thread Roman Khachatryan
Congratulations Anton, well deserved! Regards, Roman On Tue, Feb 21, 2023 at 9:34 AM Martijn Visser wrote: > Congratulations Anton! > > On Tue, Feb 21, 2023 at 8:08 AM Lincoln Lee > wrote: > > > Congratulations, Anton! > > > > Best, > > Lincoln Lee >

Re: [DISCUSS] FLIP-266: Simplify network memory configurations for TaskManager

2022-12-28 Thread Roman Khachatryan
proposed changes? Regards, Roman On Wed, Dec 28, 2022 at 12:19 PM JasonLee <17610775...@163.com> wrote: > Hi Yuxin > > > Thanks for the proposal, big + 1 for this FLIP. > > > > It is difficult for users to calculate the size of network memory. If the > setting is too

Re: [DISCUSS] FLIP-266: Simplify network memory configurations for TaskManager

2022-12-27 Thread Roman Khachatryan
rent exchange types, WDYT? [1] https://issues.apache.org/jira/browse/FLINK-24035 Regards, Roman On Tue, Dec 27, 2022 at 4:12 AM Yuxin Tan wrote: > Hi, Weihua > > Thanks for your suggestions. > > > 1. How about reducing ExclusiveBuffersPerChannel to 1 first when the > total

[jira] [Created] (FLINK-30088) Excessive state updates for TtlMapState and TtlListState

2022-11-18 Thread Roman Boyko (Jira)
Roman Boyko created FLINK-30088: --- Summary: Excessive state updates for TtlMapState and TtlListState Key: FLINK-30088 URL: https://issues.apache.org/jira/browse/FLINK-30088 Project: Flink Issue

Re: [DISCUSS] Allow sharing (RocksDB) memory between slots

2022-11-17 Thread Khachatryan Roman
I agree, the current calculation logic is already complicated. I just think that not using managed memory complicates the memory model even further. But as I mentioned earlier, both approaches have their pros and cons, so I'll update the proposal to use unmanaged memory. Thanks! Regards,

[jira] [Created] (FLINK-30073) Managed memory can be wasted if rocksdb memory is fixed-per-slot

2022-11-17 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-30073: - Summary: Managed memory can be wasted if rocksdb memory is fixed-per-slot Key: FLINK-30073 URL: https://issues.apache.org/jira/browse/FLINK-30073 Project

Re: [DISCUSS] Allow sharing (RocksDB) memory between slots

2022-11-16 Thread Roman Khachatryan
e-1.16/docs/deployment/memory/mem_setup_tm/#managed-memory [2] https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/deployment/memory/mem_setup_tm/#detailed-memory-model Regards, Roman On Wed, Nov 16, 2022 at 4:01 AM Xintong Song wrote: > Concerning isolation, I think ideally we want everything to

Re: [DISCUSS] Allow sharing (RocksDB) memory between slots

2022-11-15 Thread Khachatryan Roman
do it rather than users. Both approaches need some help from TM for: - storing the shared resources (static field in a class might be too dangerous because if the backend is loaded by the user-class-loader then memory will leak silently). - reading the configuration Regards, Roman On Sun, Nov 13,

Re: [DISCUSS] Allow sharing (RocksDB) memory between slots

2022-11-11 Thread Roman Khachatryan
RocksDB instance might close the BlockCache. I checked that manually and it seems to work as expected. And I think that would contradict the sharing concept, as described in the documentation [1]. [1] https://github.com/facebook/rocksdb/wiki/Block-Cache Regards, Roman On Wed, Nov 9, 2022 at

[jira] [Created] (FLINK-29985) SlotTable not close on TM termination

2022-11-10 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-29985: - Summary: SlotTable not close on TM termination Key: FLINK-29985 URL: https://issues.apache.org/jira/browse/FLINK-29985 Project: Flink Issue Type

Re: [DISCUSS] Allow sharing (RocksDB) memory between slots

2022-11-09 Thread Roman Khachatryan
native. The existing users definitely shouldn't be affected, so there must be at least feature toggle ("shared-fraction"). "share-scope" could potentially be replaced by some inference logic, but having it explicit seems less error-prone. Regards, Roman On Wed, Nov 9, 202

[DISCUSS] Allow sharing (RocksDB) memory between slots

2022-11-08 Thread Roman Khachatryan
. [1] https://issues.apache.org/jira/browse/FLINK-29928 [2] https://issues.apache.org/jira/browse/FLINK-7289 Regards, Roman

[jira] [Created] (FLINK-29928) Allow sharing (RocksDB) memory between slots

2022-11-08 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-29928: - Summary: Allow sharing (RocksDB) memory between slots Key: FLINK-29928 URL: https://issues.apache.org/jira/browse/FLINK-29928 Project: Flink Issue

[jira] [Created] (FLINK-29158) Fix logging in DefaultCompletedCheckpointStore

2022-08-31 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-29158: - Summary: Fix logging in DefaultCompletedCheckpointStore Key: FLINK-29158 URL: https://issues.apache.org/jira/browse/FLINK-29158 Project: Flink

[jira] [Created] (FLINK-29157) Clarify the contract between CompletedCheckpointStore and SharedStateRegistry

2022-08-31 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-29157: - Summary: Clarify the contract between CompletedCheckpointStore and SharedStateRegistry Key: FLINK-29157 URL: https://issues.apache.org/jira/browse/FLINK-29157

FLIP-50 current status

2022-08-19 Thread Roman Boyko
this feature unfinished? Thank you in advance! [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-50%3A+Spill-able+Heap+Keyed+State+Backend [2] https://github.com/realtime-storage-engine/flink-spillable-statebackend -- Best regards, Roman Boyko e.: ro.v.bo...@gmail.com m.: +79059592443

[jira] [Created] (FLINK-28976) Changelog 1st materialization delayed unneccesarily

2022-08-15 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-28976: - Summary: Changelog 1st materialization delayed unneccesarily Key: FLINK-28976 URL: https://issues.apache.org/jira/browse/FLINK-28976 Project: Flink

[jira] [Created] (FLINK-28931) BlockingPartitionBenchmark doesn't compile

2022-08-11 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-28931: - Summary: BlockingPartitionBenchmark doesn't compile Key: FLINK-28931 URL: https://issues.apache.org/jira/browse/FLINK-28931 Project: Flink

[jira] [Created] (FLINK-28647) Remove separate error handling and adjust documentation for CLAIM mode + RocksDB native savepoint

2022-07-22 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-28647: - Summary: Remove separate error handling and adjust documentation for CLAIM mode + RocksDB native savepoint Key: FLINK-28647 URL: https://issues.apache.org/jira/browse

[jira] [Created] (FLINK-28597) Empty checkpoint folders not deleted on job cancellation if their shared state is still in use

2022-07-18 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-28597: - Summary: Empty checkpoint folders not deleted on job cancellation if their shared state is still in use Key: FLINK-28597 URL: https://issues.apache.org/jira/browse

Re: [DISCUSS] Introduce multi delete API to Flink's FileSystem class

2022-06-30 Thread Roman Khachatryan
ch all the file systems would benefit, even those without bulk-delete support. [1] https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html Regards, Roman On Thu, Jun 30, 2022 at 5:10 PM Piotr Nowojski wrote: > Hi, > > Yes, I know that you can not use recursive d

[jira] [Created] (FLINK-27571) Recognize "less is better" benchmarks in regression detection script

2022-05-11 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-27571: - Summary: Recognize "less is better" benchmarks in regression detection script Key: FLINK-27571 URL: https://issues.apache.org/jira/browse/F

[jira] [Created] (FLINK-27556) Performance regression in checkpointSingleInput.UNALIGNED on 29.04.2022

2022-05-09 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-27556: - Summary: Performance regression in checkpointSingleInput.UNALIGNED on 29.04.2022 Key: FLINK-27556 URL: https://issues.apache.org/jira/browse/FLINK-27556

  1   2   3   4   5   >