Functions in Flick

Hi, I want to use Stateful Functions in Flick. Could you please clarify on the below? 1. To use the stateful function in a serverless kubernetes , what I need to install in serverless and kubernetes env.? Looks like I have many options and looking for some detailed info. 2. How much data (or ho

RideCleansingToKafka Filter Parameter Type

Greetings, First, thanks to DataArtisans for putting together Apache Flink® Training documentation. Its proving to be a practical way to learn both Flink and Scala. In compiling our exercise writing to Kafka, I am getting following error "missing parameter type" where the parameter 'r' seems not

Re: RideCleansingToKafka Filter Parameter Type

ckage:  org.apache.flink.streaming.api.scala._ I imported the package and now it compiles without errors. Thanks for the second look Andrey. -daviD On Friday, November 4, 2016 4:29 AM, Andrey Melentyev wrote: Hi daviD, does it compile okay if you clone the https://github.com/ fhueske/flink-tra

Could Not Submit Job DataStream Because No Connection to a JobManager

would be appreciated. Thank you -daviD

Re: Could Not Submit Job DataStream Because No Connection to a JobManager

Ah! Thanks Till. Now I am better informed :) -daviD On Tuesday, November 8, 2016 2:14 AM, Till Rohrmann wrote: Hi David, the dev ML is intended for technical discussions about Flink. Problems related to the usage of Flink should go to the u...@flink.apache.org. Be aware that the ML

Looking for reviewer: FLINK-13127

Hi, I've prepared a small patch related to Yarn deployment. Would anyone please have a time to take a look at it? Thanks https://github.com/apache/flink/pull/9022 Regards, D.

[DISCUSS] CPU flame graph for a job vertex in web UI.

Hello, While looking into Flink internals, I've noticed that there is already a mechanism for stack-trace sampling of a particular job vertex. I think it may be really useful to allow user to easily render a cpu flamegraph in a new UI for a selected

Re: Looking for reviewer: FLINK-13127

Hi Till, the PR should be ready ;) Can you please do the final check? Best, D. On Tue, Jul 23, 2019 at 3:01 PM Till Rohrmann wrote: > I've assigned the issue to you David. I think this feature makes sense. The > only question I have is why we need to sort the values. But let'

Re: [DISCUSS] CPU flame graph for a job vertex in web UI.

roof of concept and I'm not frontend person, so it may look little clumsy :) D. On Thu, Aug 1, 2019 at 11:40 AM Till Rohrmann wrote: > Hi David, > > thanks for starting this discussion. I like the idea of improving insights > into Flink's execution and I believe that

Re: [DISCUSS] CPU flame graph for a job vertex in web UI.

Hi Paul, for now I only plan to add the one based on java stack traces. On Fri, Aug 2, 2019 at 9:34 AM Paul Lam wrote: > Hi David, > > Thanks for the new feature! I think the flame graph would be a useful tool > to understand the state of job executions, and it looks good too. +

Re: [DISCUSS] CPU flame graph for a job vertex in web UI.

I've created FLINK-13550 <https://issues.apache.org/jira/browse/FLINK-13550> to track the issue. Is there any committer who'd be willing to "shepherd this effort"? :) Thanks, D. On Fri, Aug 2, 2019 at 10:22 AM David Morávek wrote: > Hi Paul, for now I only plan

Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release

I've spent time exploring this playground and its accompanying documentation, and found it to be a big step forward in making it easy for folks to experience some of Flink's key features firsthand. From a training and educational perspective, I'd love to see this in 1.9.

Re: [DISCUSS] Repository split

+1 for the motivation, -1 for the solution as all of the problems mention above can be addressed with the mono-repo as well. Multiple repositories: 1) This creates a big pain in case of change that targets code base in multiple repositories. Change needs to be split in multiple PRs, that need to b

Re: [VOTE] Apache Flink 1.9.0, release candidate #3

CoProcessFunction to a KeyedCoProcessFunction (which it should have been). I also tried a few complex queries in the SQL console, and wrote a simple job using the State Processor API. Everything worked. David David Anderson | Training Coordinator Follow us @VervericaData -- Join Flink Forward - The Apache

Re: [DISCUSS] FLIP-55: Introduction of a Table API Java Expression DSL

but I wonder if we can find a way to make it look less alien at first glance. David On Wed, Aug 21, 2019 at 1:33 PM Timo Walther wrote: > > Hi everyone, > > some of you might remember the discussion I started end of March [1] > about introducing a new Java DSL for Table API that

Re: [DISCUSS] FLIP-55: Introduction of a Table API Java Expression DSL

") could be interpreted to mean the value of the "foo" column, or a literal string. David On Tue, Aug 27, 2019 at 5:45 PM Timo Walther wrote: > > Hi David, > > thanks for your feedback. With the current design, the DSL would be free > of any ambiguity but it is defi

Re: [VOTE] FLIP-62: Set default restart delay for FixedDelay- and FailureRateRestartStrategy to 1s

+1 On Wed, Sep 4, 2019 at 1:38 PM Till Rohrmann wrote: > +1 (binding) > > On Wed, Sep 4, 2019 at 12:43 PM Chesnay Schepler > wrote: > > > +1 (binding) > > > > On 04/09/2019 11:18, JingsongLee wrote: > > > +1 (non-binding) > > > > > > default 0 is really not user production friendly. > > > > > >

Fine grained batch recovery vs. native libraries

Hi, we're testing the newly released batch recovery and are running into class loading related issues. 1) We have a per-job flink cluster 2) We use BATCH execution mode + region failover strategy Point 1) should imply single user code class loader per task manager (because there is only single p

Re: Fine grained batch recovery vs. native libraries

Hi Chesnay, I've created FLINK-13958 <https://issues.apache.org/jira/browse/FLINK-13958> to track the issue. Thanks, D. On Wed, Sep 4, 2019 at 1:56 PM Chesnay Schepler wrote: > This sounds like a serious bug, please open a JIRA ticket. > > On 04/09/2019 13:41, David M

[DISCUSS] Supporting multiple Flink versions vs. tech debt

Hello, we currently have an opened PR for Flink 1.9 , which greatly improves the runner for batch use-case. In case the PR gets merged, we would be supporting 5 latest major versions of Flink, which obviously come with high maintenance price and makes futu

Re: [DISCUSS] Supporting multiple Flink versions vs. tech debt

sorry, wrong mailing list, I wanted to send this to the beam one. Sorry for the confusion. D. On Sat, Sep 7, 2019 at 12:32 PM David Morávek wrote: > Hello, > > we currently have an opened PR for Flink 1.9 > <https://github.com/apache/beam/pull/9296>, which greatly improve

[DISCUSS] Preventing Mockito usage for the new code with Checkstyle

Hi Everyone, A long time ago, the community decided not to use Mockito-based tests because those are hard to maintain. This is already baked in our Code Style and Quality Guide [1]. Because we still have Mockito imported into the code base, it's very easy for newcomers to unconsciously introduce

Re: [DISCUSS] FLINK-31873: Add setMaxParallelism to the DataStreamSink Class

Hi Eric, this sounds reasonable, there are definitely cases where you need to limit sink parallelism for example not to overload the storage or limit the number of output files +1 Best, D. On Sun, Apr 23, 2023 at 1:09 PM Weihua Hu wrote: > Hi, Eric > > Thanks for bringing this discussion. > I

Re: [DISCUSS] Planning Flink 2.0

Hi, Great to see this topic moving forward; I agree it's long overdue. I keep thinking about 2.0 as a chance to eliminate things that didn't work, make the feature set denser, and fix rough edges and APIs that hold us back. Some items in the doc (Key Features section) don't tick these boxes for

Re: [DISCUSS] Preventing Mockito usage for the new code with Checkstyle

the proposal. > > > >>>>> > > > >>>>> Can we also prevent Junit4 usage in new code by this way？Because > > > >>>> currently > > > >>>>> we are aiming to migrate our codebase to JUnit 5. > > > >

Re: [NOTICE] Flink master branch now uses Maven 3.8.6

Chesnay, thank you for all your hard work on this! David On Fri, May 12, 2023 at 4:03 PM Chesnay Schepler wrote: > > > What happened? > > I have just merged the last commits to properly support Maven 3.3+ on > the Flink master branch. > > mvnw and CI have been up

Call for help on the Web UI (In-Place Rescaling)

Hi Everyone, In FLINK-31471, we've introduced new "in-place rescaling features" to the Web UI that show up when the scheduler supports FLIP-291 REST endpoints. I expect this to be a significant feature for user education (they have an easy way to try out how rescaling behaves, especially in combi

Re: Call for help on the Web UI (In-Place Rescaling)

+1 these things already popped up during the initial review by Chesnay, but I wasn't able to visualize them in a way that didn't look completely awful :) I hope that someone will be able to pick this up. Best, D. On Fri, May 19, 2023 at 1:30 PM Jing Ge wrote: > Hi David, &

Re: [DISCUSS] FLIP-294: Support Customized Job Meta Data Listener

Hi, Thanks for the FLIP! Data lineage is an important problem to tackle. Can you please expand on how this is planned to be wired into the JobManager? As I understand, the listeners will be configured globally (per cluster), so this won't introduce a new code path for running per-job / per-sessio

Re: [DISCUSS] Flink REST API improvements

Hi Hong, Thanks for starting the discussion. seems to be using the cached version of the entire Execution graph (stale > data), when it could just use the CheckpointStatsCache directly CheckpointStatsCache is also populated using the "cached execution graph," so there is nothing to gain from th

Re: [VOTE] FLIP-322 Cooldown period for adaptive scheduler

> > The vote closes within 6 hours and, as for now, there was no vote. This > is a very short FLIP, that takes a few minutes to read. > Maybe because there should have been a dedicated voting thread (marked as [VOTE]), this one was hidden and hard to notice. We should restart the vote with proper

Re: [VOTE] FLIP-322 Cooldown period for adaptive scheduler

8:49 AM David Morávek wrote: > The vote closes within 6 hours and, as for now, there was no vote. This >> is a very short FLIP, that takes a few minutes to read. >> > > Maybe because there should have been a dedicated voting thread (marked as > [VOTE]), this one was h

Re: [DISCUSS] FLIP-322 Cooldown period for adaptive scheduler

The FLIP reads sane to me. I'm unsure about the default values, though; 5 minutes of wait time between rescales feels rather strict, and we should rethink it to provide a better out-of-the-box experience. I'd focus on newcomers trying AS / Reactive Mode out. They will struggle if they add new reso

Re: [DISCUSS] FLIP-322 Cooldown period for adaptive scheduler

4, 2023 at 9:11 AM David Morávek wrote: > The FLIP reads sane to me. I'm unsure about the default values, though; 5 > minutes of wait time between rescales feels rather strict, and we should > rethink it to provide a better out-of-the-box experience. > > I'd focus on n

Re: Working improving the REST API

I've left some comments on the PR. On Tue, Jul 4, 2023 at 9:20 AM Martijn Visser wrote: > Hi Hong, > > Given that this changes the REST API, which is a public interface, I'm > wondering if this shouldn't first have had a (small) FLIP if I follow the > guidelines from the overview page [1]. > > B

Re: [DISCUSS] FLIP-322 Cooldown period for adaptive scheduler

Rescale, yes, but you shouldn't do that in the first place. That sounds great; let's go ahead and outline this in the FLIP. Best, D. On Tue, Jul 4, 2023 at 12:30 PM Etienne Chauchot wrote: > Hi all, > > Thanks David for your feedback. My comments are inline > > Le 04/07/2

Re: [DISCUSS] FLIP-326: Enhance Watermark to Support Processing-Time Temporal Join

ingested by the hybrid source before watermarking begins, followed by an unbounded component. I think this alternative handles use cases like processing-time temporal join rather nicely, without requiring any changes to watermarks or the core runtime. David On Thu, Jun 29, 2023 at 1:39 AM Martijn

Re: [DISCUSS] FLIP-326: Enhance Watermark to Support Processing-Time Temporal Join

Dong, Thank you for the careful analysis of my proposal. Your conclusions make sense to me. David On Mon, Jul 24, 2023 at 8:37 PM Dong Lin wrote: > > Hi David, > > Thank you for the detailed comments and the suggestion of this alternative > approach. > > I agree with you

Re: log4j2 integration with flink 1.10

eeded. David [1] https://issues.apache.org/jira/browse/FLINK-15672 [2] https://issues.apache.org/jira/browse/FLINK-5339 On Mon, Aug 21, 2023 at 4:21 AM Y SREEKARA BHARGAVA REDDY wrote: > > Hi Team, > > Does any one did integration flink 1.10 with log4j2 instead of log4j1. > > I

Re: [DISCUSS][2.0] Deprecating Accumulator in favor of MetricsGroup

AFAIK Apache Beam also used acummulators for metric collection, which is indeed a major use case. I’m not convinced that MetricGroup is fuĺly replacing what acummulators have to offer though; OperatorCoordinators might be able to rplace remaining capabilities, but this need bit more thoughts, the

Re: Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

+1 Keeping the legacy implementation in place is confusing and encourages adoption of something that really shouldn't be used. Thanks for driving this, David On Fri, Sep 1, 2023 at 8:45 AM Jing Ge wrote: > > Hi Wencong, > > Thanks for your clarification! +1 > > Best reg

Re: Proposal for Implementing Keyed Watermarks in Apache Flink

Hi Tawfik, It's exciting to see any ongoing research that tries to push Flink forward! The get the discussion started, can you please your paper with the community? Assessing the proposal without further context is tough. Best, D. On Mon, Sep 4, 2023 at 4:42 PM Tawfek Yasser Tawfek wrote: > D

Re: Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

+1 since there is an alternative, more complete implementation available Best, D. On Sat, Sep 2, 2023 at 12:07 AM David Anderson wrote: > +1 > > Keeping the legacy implementation in place is confusing and encourages > adoption of something that really shouldn't be used. >

Inconsistent build Flink from source in readme and docs

the docs to make them consistent, Kind regards, David Unless otherwise stated above: IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU

RE: Inconsistent build Flink from source in readme and docs

it I deprecated. Error for java levels less than 8. If this ok I will raise a Jira to add this Maven and potentially java checking to the build, Kind regards, David. From: Sergey Nuyanzin Date: Thursday, 14 September 2023 at 16:07 To: dev@flink.apache.org Subject: [EXTERNAL] Re: Inconsistent

FW: RE: Inconsistent build Flink from source in readme and docs

Hi again, I have just checked and you already enforcer have the enforcer for this in master. I guess the only extra piece would be to put out a warning for java 8 indicating it is deprecated – I could look at that in a Jira, Kind regards, David. From: David Radley Date: Friday, 15

FW: FW: RE: Inconsistent build Flink from source in readme and docs

: [3.1.1,) ${target.java.version} Is there a reason to have these 2 versions of Maven enforcers or can we use one Maven enforcer that will apply to all cases? Kind regards, David. From: David Radley Date: Friday, 15 September 2023 at 11:47 To: dev

RE: [DISCUSS]clean up the savepoints compatibility table

Hi, I like the proposal from Martijn. Having an ‘archived’ area for the older material would make sense so users on the older versions can still access the content for upgrade, keeping maybe the latest 3 releases in active table, Kind regards, David. From: Martijn Visser Date: Friday, 15

RE: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

Hi, I notice that there is an experimental lineage integration for Flink with OpenLineage https://openlineage.io/docs/integrations/flink . I think this feature would allow for a superior Flink OpenLineage integration, Kind regards, David. From: XTransfer Date: Tuesday, 19 September

FW: RE: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

to have this asset -> process -> asset pattern for each of the steps in the job. If this is present, please could you point me to it, Kind regards, David. From: David Radley Date: Tuesday, 19 September 2023 at 16:11 To: dev@flink.apache.org Subject: [EXTERNAL] RE: [DISCUSS]

FLINK-33159

, so I can proceed, kind regards, David. Unless otherwise stated above: IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

Hello Yuepeng, The FLIP reads sane; nice work! To re-phrase my understanding: The problem you're trying to solve only exists in complex graphs with different per-vertex parallelism. If the parallelism is set globally (assuming the pipeline has roughly even data skew), the algorithm could make thi

Re: [Discuss] FLIP-362: Support minimum resource limitation

H Xiangyui, The sentiment of the FLIP makes sense, but I keep wondering whether this is the best way to think about the problem. I assume that "interactive session cluster" users always want to keep some spare resources around (up to a configured threshold) to reduce cold start instead of statical

Re: Support AWS SDK V2 for Flink's S3 FileSystem

Hi Maomao, I wonder whether it would make sense to take a stab at consolidating the S3 filesystems instead and introduce a native one. The whole Hadoop wrapper around the S3 client exists for legacy reasons, and it adds complexity and probably an unnecessary performance penalty. If you take a loo

Rest ports

High Availability you have to set the high-availability mode to zookeeper, configure a ZooKeeper quorum and set up a masters file with all JobManagers hosts and their web UI ports.” Have I missed anything here? What are your thoughts? Kind regards, David. Unless otherwise stated

RE: [Discuss] FLIP-366: Support standard YAML for FLINK configuration

errors and warnings if the content is such that the user would need to manually fix up the file. If there are minimal user fixups, we should consider automatically migrating the config file to the new format, Kind regards, David. From: Chesnay Schepler Date: Tuesday, 3 October 2023 at 11

RE: Close orphaned/stale PRs

need to agree with a committer to get the associated pr reviewed and merged. This triaged state would be a pool of issues that new contributors to choose from I am happy to help to improve – once we have consensus, Kind regards, David. From: Venkatakrishnan Sowrirajan Date: Wedn

FW: RE: Close orphaned/stale PRs

Hi, To add I agree with Martijn’s insights; I think we are saying similar things. To progress agreed upon work, and not blanket close all stale prs, Kind regards, David. From: David Radley Date: Wednesday, 4 October 2023 at 10:59 To: dev@flink.apache.org Subject: [EXTERNAL] RE: Close

Re: [Discuss] FLIP-362: Support minimum resource limitation

strong opinion here, it's a significant improvement either way. This was just the first thing that I thought about after reading the flip. Best, D. On Tue, Oct 3, 2023 at 2:10 PM xiangyu feng wrote: > Hi David, > > Thx for your feedback. > > First of all, for keeping some sp

RE: FW: RE: Close orphaned/stale PRs

– I would think that component ownership helps scope the subset of prs to review / merge. Kind regards, David. From: Ryan Skraba Date: Wednesday, 4 October 2023 at 15:09 To: dev@flink.apache.org Subject: [EXTERNAL] Re: FW: RE: Close orphaned/stale PRs Hey, this has been an

Kafka Connector

suggest we also remove the connector/kafka component from the core flink repo – so no new prs come in with it. What do you think? Kind regards, David. Unless otherwise stated above: IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41

RE: [ANNOUNCE] Release 1.18.0, release candidate #0

? Kind regards, David. From: Jing Ge Date: Wednesday, 27 September 2023 at 15:11 To: dev@flink.apache.org Subject: [EXTERNAL] Re: [ANNOUNCE] Release 1.18.0, release candidate #0 Hi Folks, @Ryan FYI: CI passed and the PR has been merged. Thanks! If there are no more other concerns, I will

RE: [ANNOUNCE] Release 1.18.0, release candidate #0

built outside of the Flink repo. We may want to not include the Flink core version in the connector – or we might end up wanting to release a Kafka connector when there are no changes just to have a match with the Flink core version. Kind regards, David. From: Jing Ge Date: Wednesday, 4

RE: [ANNOUNCE] Release 1.18.0, release candidate #0

regression like this, Kind regards, David. From: Martijn Visser Date: Thursday, 5 October 2023 at 21:39 To: dev@flink.apache.org Subject: [EXTERNAL] Re: [ANNOUNCE] Release 1.18.0, release candidate #0 Hi David, It’s a deliberate choice to decouple the connectors. We shouldn’t block Flink 1.18

RE: [DISCUSS] FLIP-368 Reorganize the exceptions thrown in state interfaces

be appropriate for those cases; and be part of the contract with the caller. Can we be sure that there is no possibility that the state will become available; if so then I agree that a runtime Exception is appropriate. What do you think? Kind regards, David. From: Zakelly Lan Date: Monday, 9

FW: RE: [DISCUSS] FLIP-368 Reorganize the exceptions thrown in state interfaces

a new class ValueState2– that is used internally with the cleaned up Exceptions, but still expose the old class and Exceptions for existing external applications. I guess new applications could use the new ValueState2 . What do you think? Kind regards, David. From: David Radley Date

Operator 1.6 to Olm

Hi, I notice that the latest version in olm of the operator is 1.5. I plan to run the scripts to publish the 1.6 Flink operator to olm, Kind regards, David. Unless otherwise stated above: IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO

RE: FW: RE: [DISCUSS] FLIP-368 Reorganize the exceptions thrown in state interfaces

(such as network access) – then there could be retries. I agree we should not be trying to create code now for an implementation consideration that is not there yet, +1 from me , Kind regards, David. From: Zakelly Lan Date: Wednesday, 11 October 2023 at 04:25 To: dev@flink.apache.org

RE: FW: RE: Close orphaned/stale PRs

using git actions, so that if an approved approver indicates a pr is good then the raiser can merge – this would give us granularity on write access – PyTorch follows this sort of process. kind regards, David. From: Martijn Visser Date: Thursday, 12 October 2023 at 10:32 To: dev

RE: FW: RE: [DISCUSS] FLIP-368 Reorganize the exceptions thrown in state interfaces

for them. On balance, as I am risk averse, I would suggest delaying this to v2 as Jing has proposed. This is a cleaner API, is there a demand for this in a dot version? If the community think this is too risk averse, then we could go with 1.19. WDYT? Kind regards, David. From: Jing Ge

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

d. `wall` is very useful to debug blocks on I/O or locks. Getting per-thread information is nice to drill down into specific parts of the Flink application, e.g. the flame graph lets me ignore the many other tasks running on TM & drill down into just the Source threads, when debugging a Sour

Re: [ANNOUNCE] New Apache Flink Committer - Ron Liu

Congratulations Ron! From: Jark Wu Date: Sunday, 15 October 2023 at 18:57 To: dev Cc: ron9@gmail.com Subject: [EXTERNAL] [ANNOUNCE] New Apache Flink Committer - Ron Liu Hi, everyone On behalf of the PMC, I'm very happy to announce Ron Liu as a new Flink Committer. Ron has been continuousl

Re: Support AWS SDK V2 for Flink's S3 FileSystem

as a follow-up. > > > > [1] > https://github.com/apache/flink/blob/d78d52b27af2550f50b44349d3ec6dc84b966a8a/flink-core/src/main/java/org/apache/flink/core/fs/FileSystem.java#L695 > > [2] > https://github.com/apache/flink/blob/d78d52b27af2550f50b44349d3ec6dc84b966a8a/flink

Maven and java version variables

could be extended to include links to this information. I am happy to do that as part of this pr , if needed, if I can be supplied the links. I think this pr should be merged asap, so subsequent pom file changes use the Maven variables. WDYT Kind regards, David. Unless otherwise stated above

Backport strategy

1.15 1.16 1.17. WDYT? Kind regards, David. Unless otherwise stated above: IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU

RE: Backport strategy

case it is misleading? Kind regards, David. From: Martijn Visser Date: Monday, 23 October 2023 at 16:18 To: dev@flink.apache.org Subject: [EXTERNAL] Re: Backport strategy Hi David, The policy is that the current and and previous minor release are supported, and it's documented at

FLIP-233

agreement in the discussion thread please, Kind regards, David. Unless otherwise stated above: IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU

RE: FLIP-233

Thanks Leonard, Hopefully this will be reopened, as we would very much like this capability and want to take over the FLIP, continue the discussion to get a consensus, then implement, Kind regards, David From: Leonard Xu Date: Tuesday, 24 October 2023 at 02:56 To: dev Cc: jd...@amazon.com

RE: FLIP-233

Hi Martjin, Yes I am happy to continue to improve the existing Flip. Hi jing, I was looking to continue the discussion and update the Flip content. Are we OK to reopen : FLIP-233? I will update it and the discussion thread there. Kind regards, David. From: Martijn Visser Date: Tuesday

RE: Operator 1.6 to Olm

Hi, Fyi with some expert direction from James Busche, I have published the 1.6 OLM and operatorhub.io versions of the Flink operator. When 1.6.1 is out I will do the same again, Kind regards, David. From: Gyula Fóra Date: Tuesday, 10 October 2023 at 13:27 To: dev@flink.apache.org

Re: [VOTE] Add JSON encoding to Avro serialization

Looks good to me +1 From: Ryan Skraba Date: Wednesday, 25 October 2023 at 17:19 To: dev@flink.apache.org Subject: [EXTERNAL] [VOTE] Add JSON encoding to Avro serialization Hello! I'm reviewing a new feature of another contributor (Dale Lane) on FLINK-33058 that adds JSON-encoding in addition to

RE: Maven and java version variables

Hi Matthias, That sounds reasonable, Kind regards, David From: Matthias Pohl Date: Monday, 23 October 2023 at 16:41 To: dev@flink.apache.org Subject: [EXTERNAL] Re: Maven and java version variables Hi David, The change that caused the conflict in your PR is caused by FLINK-33291 [1]. I was

FW: Maven and java version variables

resolve another conflict? Kind regards, David. From: David Radley Date: Monday, 23 October 2023 at 12:25 To: dev@flink.apache.org Subject: [EXTERNAL] Maven and java version variables Hi, I have an open pr in the backlog that improves the pom.xml by introducing some Maven variables. The pr is https

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.6.1, release candidate #1

Hi, I downloaded the artifacts. * I did an install of the operator and ran the basic sample * I checked the checksums * Checked the GPG signatures * Ran the UI * Ran a Twistlock scan * I installed 1.6 then did a helm upgrade * I have not managed to do the source build a

flink-sql-connector-jdbc new release

regards, David. Unless otherwise stated above: IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU

RE: flink-sql-connector-jdbc new release

try to move it along– could you assign it to me please, Kind regards, David. From: Jingsong Li Date: Friday, 27 October 2023 at 06:54 To: dev@flink.apache.org Subject: [EXTERNAL] Re: flink-sql-connector-jdbc new release Hi David, Thanks for driving this. I think https://issues.apache.org

RE: flink-sql-connector-jdbc new release

Hi Jing, I just spotted the mailing list that it is a regression – I agree it is a blocker, Kind regards, David. From: David Radley Date: Friday, 27 October 2023 at 10:33 To: dev@flink.apache.org Subject: [EXTERNAL] RE: flink-sql-connector-jdbc new release Hi Jing, thanks are there any

RE: flink-sql-connector-jdbc new release

Hi Martijn, Thanks for the link. I suspect I cannot be the release manager, as I do not have the required access, but am happy to help this progress, kind regards, David. From: Martijn Visser Date: Friday, 27 October 2023 at 12:16 To: dev@flink.apache.org Subject: [EXTERNAL] Re: flink

Re: Pointers to computational models of Flink CEP

The implementation of Flink CEP was largely based on Efficient Pattern Matching over Event Streams by Jagrati Agrawal, Yanlei Diao, Daniel Gyllstrom, and Neil Immerman from UMass Amherst [1]. [1] https://people.cs.umass.edu/~yanlei/publications/sase-sigmod08.pdf Cheers, David On Tue, Oct 31

How do I source debug the scala in the flink table planner?

slow and tedious. Ideally I would like to be able to source debug the scala in the flink table planner; any advice would be fab? Kind regards, David. Unless otherwise stated above: IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41

RE: How do I source debug the scala in the flink table planner?

Scala breakpoints do not spring. I wonder if it relates to the planner being loaded in a different class loader. Kind regards, David. From: Sergey Nuyanzin Date: Thursday, 2 November 2023 at 19:58 To: dev@flink.apache.org Subject: [EXTERNAL] Re: How do I source debug the scala in the

Re: [DISCUSS] Change the default restart-strategy to exponential-delay

red in the documentation. David On Wed, Nov 15, 2023 at 10:17 PM Rui Fan <1996fan...@gmail.com> wrote: > Hi dear flink users and devs: > > FLIP-364[1] intends to make some improvements to restart-strategy > and discuss updating some of the default values of exponential-dela

Re: [DISCUSS] REST API behaviour when user main method throws error

Hi Danny, > My current proposal is that the REST API should not leave the Flink cluster in an inconsistent state. Regarding consistency, Flink only cares about individual jobs, but I can see your point. For streaming, this is probably something we could address by book-keeping jobs submitted by

Re: Proposal for Implementing Keyed Watermarks in Apache Flink

s. > > > > Instead of using only one watermark (vanilla/global watermark), we > changed > > the API to allow moving the keyBy() before the > > assignTimestampsAndWatermarks() so the stream will be partitioned then > the > > TimestampsAndWatermarkOperator will handle t

Apicurio Avro format proposal

/getting-started/assembly-configuring-kafka-client-serdes.html#registry-serdes-types-avro_registry I am happy to work on this, Kind regards, David. Unless otherwise stated above: IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41, North

Re: [VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

+1 (binding) Best, D. On Wed, Nov 22, 2023 at 11:21 AM Roman Khachatryan wrote: > +1 (binding) > > Regards, > Roman > > On Wed, Nov 22, 2023, 7:08 AM Zakelly Lan wrote: > > > +1(non-binding) > > > > Best, > > Zakelly > > > > On Wed, Nov 22, 2023 at 3:04 PM Hangxiang Yu > wrote: > > > > > +1 (

RE: Apicurio Avro format proposal

Hi Martijn, Ok will do, Kind regards, David. From: Martijn Visser Date: Wednesday, 22 November 2023 at 21:47 To: dev@flink.apache.org Subject: [EXTERNAL] Re: Apicurio Avro format proposal Hi David, Can you create a small FLIP for this? Best regards, Martijn On Wed, Nov 22, 2023 at 6

RE: Apicurio Avro format proposal

a main feature they both do, that is relevant to Flink. Does this help? If I have misrepresented anything – please let me know, I am investigating further so I can create a well described FLIP for the proposed change, Kind regards, David. From: Ryan Skraba Date: Thursday, 23 November 2

RE: [VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

+1(non-binding) From: Stefan Richter Date: Wednesday, 29 November 2023 at 09:44 To: dev@flink.apache.org Subject: [EXTERNAL] Re: [VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces +1 (binding) Best, Stefan > On 22. Nov 2023, at 11:20, Roman Khacha

Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

feel that we need to provide replacements for these internal, experimental features before officially deprecating the legacy group window aggregations, and I'd rather not wait. However, I'd be delighted to see a proposal for what that might look like. Best, David On Mon, Dec 4, 2023

  1   2   3   4   5   6   7   8   >