The pull request has been merged. Thank you for the discussion and
reviewing.
The page is live now: https://flink.apache.org/what-is-flink/special-thanks/
Best,
Jark
On Tue, 12 Mar 2024 at 17:44, Jark Wu wrote:
> I have created a JIRA issue and opened a pull request for this:
> https://github.c
Hi, Jeyhun.
Thanks for the FLIP. Totally +1 for it.
I have a question about the part `Additional option to disable this
optimization`. Is this option a source configuration or a table
configuration?
Besides that, there is a little mistake if I do not understand wrongly.
Should `Check if upstream
Hi Jeyhun,
Thanks for your clarification.
> Once a new partition is detected, we add it to our existing mapping. Our
mapping looks like Map> subtaskToPartitionAssignment,
where it maps each source subtaskID to zero or more partitions.
I understand your point. **It would be better if you could s
Hi Yubin,
Thanks for leading the discussion. I'm +1 for the FLIP.
As Jark said, it's a good opportunity to enhance the syntax for Catalog
from a more comprehensive perspective. So, I suggest expanding the scope of
this FLIP by focusing on the mechanism instead of one use case to enhance
the overa
Jacky Lau created FLINK-34665:
-
Summary: Add streaming rule for union to Rand and it convert to
StreamExecDeduplicate finally
Key: FLINK-34665
URL: https://issues.apache.org/jira/browse/FLINK-34665
Projec
Hi Yun, Jing, Martijn and Lincoln,
I'm seeking guidance on whether merging the bugfix[1][2] at this stage is
appropriate. I want to ensure that the actions align with the current
release process and do not disrupt the ongoing preparations.
[1] https://issues.apache.org/jira/browse/FLINK-29114
[2]
Hongshun Wang created FLINK-34666:
-
Summary: Keep assigned splits in order to fix wrong meta group
calculation
Key: FLINK-34666
URL: https://issues.apache.org/jira/browse/FLINK-34666
Project: Flink
Yanfei Lei created FLINK-34667:
--
Summary: Changelog state backend support local rescaling
Key: FLINK-34667
URL: https://issues.apache.org/jira/browse/FLINK-34667
Project: Flink
Issue Type: Bug
Update on FLINK-34227 [1] which I mentioned above: Chesnay helped identify
a concurrency issue in the JobMaster shutdown logic which seems to be in
the code for quite some time. I created a PR fixing the issue hoping that
the test instability is resolved with it.
The concurrency issue doesn't real
Zakelly Lan created FLINK-34668:
---
Summary: Report State handle of file merging directory to JM
Key: FLINK-34668
URL: https://issues.apache.org/jira/browse/FLINK-34668
Project: Flink
Issue Type:
Hi Jane,
Thank you for raising this question. I saw the discussion in the Jira
(include Matthias' point)
and sought advice from several PMCs (including the previous RMs), the
majority of people
are in favor of merging the bugfix into the release branch even during the
release candidate
(RC) voting
Hi Matthias,
Thanks for the updating!
And once again, thank you for continuously tracking all unstable cases and
driving the resolution!
Best,
Lincoln Lee
Lincoln Lee 于2024年3月14日周四 18:00写道：
> Hi Jane,
>
> Thank you for raising this question. I saw the discussion in the Jira
> (include Matthia
Hi,
Gentle ping to see if there are any other concerns or things that seems missing
from the FLIP.
Best,
Ferenc
On Monday, March 11th, 2024 at 11:11, Ferenc Csaky
wrote:
>
>
> Hi Jing,
>
> Thank you for your comments! Updated the FLIP with reasoning on the proposed
> release versions
Nothing to add from my side. Thanks, Alex.
+1 (binding)
On Thu, Mar 7, 2024 at 4:09 PM Alex Nitavsky wrote:
> Hi everyone,
>
> I'd like to start a vote on FLIP-402 [1]. It introduces new configuration
> options for Apache Flink's ZooKeeper integration for high availability by
> reflecting exist
Hey everyone,
I don't see any immediate blockers, so I'm going to start the release process.
Thanks,
Max
On Tue, Feb 20, 2024 at 8:55 PM Maximilian Michels wrote:
>
> Hey Rui, hey Ryan,
>
> Good points. Non-committers can't directly release but they can assist
> with the release. It would be gr
Hi everyone,
I'm wondering whether anyone has objections against removing the Azure
Pipelines Tutorial to "set up CI for a fork of the Flink repository" in the
Flink wiki. Flink's GitHub Actions workflow seems to work fine for forks
(at least for 1.18+ changes). No need to guide contributors to the
Hi Matthias,
thanks for driving this
agree GHA seems working ok
however to be on the safe side what if we mark it for removal or deprecated
first
and then remove together with dropping support of 1.17 where GHA is not
supported IIUC?
On Thu, Mar 14, 2024 at 11:42 AM Matthias Pohl
wrote:
> Hi
+1 for the FLIP, thanks Yubin for driving it.
Also +1 to complete the whole story about Catalog in this FLIP as Jark
and Jane said above.
Jane Chan 于2024年3月14日周四 15:33写道：
>
> Hi Yubin,
>
> Thanks for leading the discussion. I'm +1 for the FLIP.
>
> As Jark said, it's a good opportunity to enhanc
Hi Yubin,
Thank you for initiating this discussion! +1 for the proposal.
I also think it makes sense to group the missing catalog related
SQL syntaxes under this FLIP.
Looking forward to these features!
Best,
Ferenc
On Thursday, March 14th, 2024 at 08:31, Jane Chan wrote:
>
>
> Hi Yubin
Hi Lincoln,
Thank you for the prompt response and the effort to provide clarity on this
matter.
Best,
Jane
On Thu, Mar 14, 2024 at 6:02 PM Lincoln Lee wrote:
> Hi Jane,
>
> Thank you for raising this question. I saw the discussion in the Jira
> (include Matthias' point)
> and sought advice fro
Hi, Yubin.
Thanks for leading the discussion. +1 for the FLIP.
Best,
Zhongqiang Gong
Ferenc Csaky 于2024年3月14日周四 18:58写道：
> Hi Yubin,
>
> Thank you for initiating this discussion! +1 for the proposal.
>
> I also think it makes sense to group the missing catalog related
> SQL syntaxes under th
Thanks Jeyhun for bringing up this discussion, it is really exiting,
+1 for the general idea.
We also introduced a similar concept in Flink Batch internally to cope
with bucketed tables in Hive, it is a very important improvement.
> One thing to note is that for join queries, the parallelism of e
Jane Chan created FLINK-34669:
-
Summary: Optimization of Arch Rules for Connector Constraints and
Violation File Updates
Key: FLINK-34669
URL: https://issues.apache.org/jira/browse/FLINK-34669
Project: Fl
+1 (non-binding)
- Verified checksum and signature
- Verified no binary in src
- Built from src
- Reviewed release note PR
- Reviewed web PR
- Tested a simple datagen query and insert to blackhole sink via SQL Gateway
Best,
Ferenc
On Thursday, March 14th, 2024 at 12:14, Jane Chan wrote:
>
+1 (non-binding)
- Verified no binary files in source code
- Verified signature and checksum
- Build source code and run a simple job successfully
- Reviewed the release announcement PR
Best,
Zhongqiang Gong
Ferenc Csaky 于2024年3月14日周四 20:07写道：
> +1 (non-binding)
>
> - Verified checksum and s
Jinzhong Li created FLINK-34670:
---
Summary: Use SynchronousQueue to create asyncOperationsThreadPool
for SubtaskCheckpointCoordinatorImpl
Key: FLINK-34670
URL: https://issues.apache.org/jira/browse/FLINK-34670
Hi, Yubin
Thank you for initiating this discussion! +1 for the proposal.
Best,
Yuepeng Pan
At 2024-03-14 18:57:35, "Ferenc Csaky" wrote:
>Hi Yubin,
>
>Thank you for initiating this discussion! +1 for the proposal.
>
>I also think it makes sense to group the missing catalog related
LvYanquan created FLINK-34671:
-
Summary: Update the content of README.md in FlinkCDC project
Key: FLINK-34671
URL: https://issues.apache.org/jira/browse/FLINK-34671
Project: Flink
Issue Type: Imp
+1 (binding)
- Validated hashes
- Verified signature
- Verified that no binaries exist in the source archive
- Build the source with Maven via mvn clean install -Pcheck-convergence
-Dflink.version=1.19.0
- Verified licenses
- Verified web PR
- Started a cluster and the Flink SQL client, successful
Good pointing. I guess, marking it as deprecated and pointing to GitHub
Actions as the new workaround would be a better way than removing it
entirely for now.
On Thu, Mar 14, 2024 at 11:47 AM Sergey Nuyanzin
wrote:
> Hi Matthias,
>
> thanks for driving this
> agree GHA seems working ok
>
> howe
+1 (non-binding)
- Checked the pre-built jars are generated with jdk8
- Verified signature and checksum
- Verified no binary in source
- Verified source code tag
- Reviewed release note
- Reviewed web PR
- Built from source
- Run a simple job successfully
On Thu, Mar 14, 2024 at 2:21 PM Martijn V
Chesnay Schepler created FLINK-34672:
Summary: HA deadlock between JobMasterServiceLeadershipRunner and
DefaultLeaderElectionService
Key: FLINK-34672
URL: https://issues.apache.org/jira/browse/FLINK-34672
Ryan Skraba created FLINK-34673:
---
Summary: SessionRelatedITCase#testTouchSession failure on GitHub
Actions
Key: FLINK-34673
URL: https://issues.apache.org/jira/browse/FLINK-34673
Project: Flink
+1 (not-binding)
- verified signature
- verified checksum
- verified that source distribution does not contain binaries
- built from source
- reviewed the web PR
- reviewed the release note
- checked the tag
- checked the repo
Best regards,
Jing
On Thu, Mar 14, 2024 at 2:37 PM Sergey Nuyanzin w
+1 (binding)
- verified signatures
- verified hashsums
- checked Github release tag
- started SQL Client, used MySQL CDC connector to read records from database ,
the result is expected
- checked Jira issues for 1.19.0 and discussed with RMs that FLINK-29114 won’t
block this RC
- checked rele
+1 (non-binding)
- Verified signatures and checksums
- Verified that source does not contain binaries
- Build source code successfully
- Reviewed the web PR
- Reviewed the release note
Best,
Jiabao
Leonard Xu 于2024年3月14日周四 23:01写道：
>
> +1 (binding)
>
> - verified signatures
> - verified hashsu
Hi Yubin,
Thanks for driving the discussion, generally +1 for the FLIP, big +1 to
finalize the whole catalog syntax story in one FLIP,
thus I want to jump into the discussion again after you completed the whole
catalog syntax story.
Best,
Leonard
> 2024年3月14日 下午8:39，Roc Marshal 写道：
>
> Hi
Dear Flink Community,
Do you know if we have somewhere a tested ARM based flink docker image? I
think we can already run locally on an ARM macbook. But we don't have a
ARM specified docker image yet.
Regards,
Yang LI
Hi ,
I am currently prototyping an Avro Apicurio format that I hope to raise as a
FLIP very soon (hopefully by early next week). In my prototyping , I am
passing through the Kafka headers content as a map to the DeserializationSchema
and have extended the SerializationSchema to pass back header
Hi everyone,
Please review and vote on the release candidate #1 for the version
1.8.0 of the Apache Flink Kubernetes Operator, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
**Release Overview**
As an overview, the release consists
Hi Jark,
Thanks for the pointers. It's very helpful.
1. Looks like `tumble`, `hopping` are keywords in calcite parser. And the
syntax `cumulate(Table my_table, ...)` needs to get table information from
catalog somewhere for type validation etc. Can you send me some pointers
where the function get
Hi folks,
Thank you all for your input, it really makes sense to introduce missing
catalog-related SQL syntaxes under this FLIP, and I have changed the
title of doc to "FLIP-436: Introduce Catalog-related Syntax".
After comprehensive consideration, the following syntaxes should be
introduced, mor
Thanks for your response Gyula. Yes I understand, it doesn't really fit
nicely into the Kubernetes Operator pattern.
I do still wonder about the idea of supporting a feature where upon first
deploy, Flink Operator optionally (flag/param enabled) finds the most
recent snapshot (in a specified objec
Hi all,
I was wondering if anyone has any ideas or advice when it comes to being
efficient about the use of disks with the RocksDB StateBackend.
In general not all operators in a Flink Job will be stateful and require
persistent disks to use with RocksDB, and so any stateless operators
running on
Hi David!
I think passing the headers as a map (as opposed to
ConsumerRecord/ProducerRecord) is a great idea that should work. That way
the core Flink package doesn't have Kafka dependencies, it seems like
they're meant to be decoupled anyway. The one bonus that using the Record
objects has is tha
Hi Hang,
Thanks for the comments.
I have a question about the part `Additional option to disable this
> optimization`. Is this option a source configuration or a table
> configuration?
- It is a source configuration.
Besides that, there is a little mistake if I do not understand wrongly.
> Sho
Sergey Nuyanzin created FLINK-34674:
---
Summary: Migrate CalcSnapshotTransposeRule
Key: FLINK-34674
URL: https://issues.apache.org/jira/browse/FLINK-34674
Project: Flink
Issue Type: Sub-task
Sergey Nuyanzin created FLINK-34675:
---
Summary: Migrate AggregateReduceGroupingRule
Key: FLINK-34675
URL: https://issues.apache.org/jira/browse/FLINK-34675
Project: Flink
Issue Type: Sub-tas
Hi Jane,
Thanks for your comments.
I understand your point. **It would be better if you could sync the
> content to the FLIP**.
- Sure thing. I added my above answer to the FLIP.
Another thing is I'm curious about what the physical plan looks like. Is
> there any specific info that will be a
Sergey Nuyanzin created FLINK-34676:
---
Summary: Migrate ConvertToNotInOrInRule
Key: FLINK-34676
URL: https://issues.apache.org/jira/browse/FLINK-34676
Project: Flink
Issue Type: Sub-task
Hi everyone,
I'm happy to announce that we have unanimously approved this release.
There are 24 approving votes, 4 of which are binding:
- Xintong Song (binding)
- Jean-Baptiste Onofré (non binding)
- Ahmed Hamdy (non binding)
- Samrat Deb (non binding)
- Jeyhun Karimov (non binding)
- Hangxiang
Hi, Yubin. Big +1 for this Flip. I just left one minor comment following.
I found that although flink has not supported syntax 'DESCRIBE CATALOG
catalog_name' currently, it was already
discussed in flip-69[1], do we need to restart discussing it?
I don't have a particular preference regarding th
Qingsheng Ren created FLINK-34677:
-
Summary: Refactor the structure of documentation for Flink CDC
Key: FLINK-34677
URL: https://issues.apache.org/jira/browse/FLINK-34677
Project: Flink
Issue
Qingsheng Ren created FLINK-34678:
-
Summary: "Introduction" Page for Flink CDC Documentation
Key: FLINK-34678
URL: https://issues.apache.org/jira/browse/FLINK-34678
Project: Flink
Issue Type:
Qingsheng Ren created FLINK-34679:
-
Summary: "Core Concept" Pages for Flink CDC Documentation
Key: FLINK-34679
URL: https://issues.apache.org/jira/browse/FLINK-34679
Project: Flink
Issue Type
Qingsheng Ren created FLINK-34680:
-
Summary: "Overview" Page for Connectors in Flink CDC Documentation
Key: FLINK-34680
URL: https://issues.apache.org/jira/browse/FLINK-34680
Project: Flink
I
Qingsheng Ren created FLINK-34681:
-
Summary: "Deployment" Pages for Flink CDC Documentation
Key: FLINK-34681
URL: https://issues.apache.org/jira/browse/FLINK-34681
Project: Flink
Issue Type:
Qingsheng Ren created FLINK-34682:
-
Summary: "Developer Guide - Understanding Flink CDC" Page for
Flink CDC Documentation
Key: FLINK-34682
URL: https://issues.apache.org/jira/browse/FLINK-34682
Projec
Qingsheng Ren created FLINK-34683:
-
Summary: "Developer Guide - Contribute to Flink CDC" Page for
Flink CDC Documentation
Key: FLINK-34683
URL: https://issues.apache.org/jira/browse/FLINK-34683
Projec
Qingsheng Ren created FLINK-34684:
-
Summary: "Developer Guide - Licenses" Page for Flink CDC
Documentation
Key: FLINK-34684
URL: https://issues.apache.org/jira/browse/FLINK-34684
Project: Flink
Qingsheng Ren created FLINK-34685:
-
Summary: "Deployment - Kubernetes" Page for Flink CDC Documentation
Key: FLINK-34685
URL: https://issues.apache.org/jira/browse/FLINK-34685
Project: Flink
Qingsheng Ren created FLINK-34686:
-
Summary: "Deployment - YARN" Page for Flink CDC Documentation
Key: FLINK-34686
URL: https://issues.apache.org/jira/browse/FLINK-34686
Project: Flink
Issue
Hi Hao,
> Can you send me some pointers
where the function gets the table information?
Here is the code of cumulate window type checking [1].
> Also is it possible to support in
window functions in addiction to table?
Yes. It is not allowed in TVF.
Thanks for the syntax links of other systems
Qingsheng Ren created FLINK-34687:
-
Summary: Home Page of Flink CDC Documentation
Key: FLINK-34687
URL: https://issues.apache.org/jira/browse/FLINK-34687
Project: Flink
Issue Type: Sub-task
Hongshun Wang created FLINK-34688:
-
Summary: CDC framework split snapshot chunks asynchronously
Key: FLINK-34688
URL: https://issues.apache.org/jira/browse/FLINK-34688
Project: Flink
Issue Ty
Lee SeungMin created FLINK-34689:
Summary: check binlog_row_value_optoins
Key: FLINK-34689
URL: https://issues.apache.org/jira/browse/FLINK-34689
Project: Flink
Issue Type: Improvement
Hi Xuyang,
Thank you for pointing this out, The parser part of `describe catalog`
syntax
has indeed been implemented in FLIP-69, and it is not actually available.
we can complete the syntax in this FLIP [1]. I have updated the doc :)
Best,
Yubin
[1]
https://cwiki.apache.org/confluence/display/F
Hi Jinzhong,
Batching state access is a reasonable way to reduce the amount of I/O compared
to per-record state access. But I have some questions:
- In my opinion, we need to reduce the times of fetching RocksDB SST from
remote to local. The FLIP seems to batch the RocksDB put/get requests. I a
Hongshun Wang created FLINK-34690:
-
Summary: If data from upstream is decimal and primary key ,
starrocks sink will not support.
Key: FLINK-34690
URL: https://issues.apache.org/jira/browse/FLINK-34690
Hi Rui,
Thanks for your comments.
> In my opinion, we need to reduce the times of fetching RocksDB
> SST from remote to local. The FLIP seems to batch the RocksDB
> put/get requests. I am not sure this will reduce the SST fetching times.
For multiple Get(read) requests, we will convert them into
Hi Jark,
Thanks for the pointer. Sorry for the confusion: I meant how the table name
in window TVF gets translated to `SqlCallingBinding`. Probably we need to
fetch the table definition from the catalog somewhere. Do we treat those
window TVF specially in parser/planner so that catalog is looked u
LvYanquan created FLINK-34691:
-
Summary: failed to convert time type when using mysql to doris
pipeline.
Key: FLINK-34691
URL: https://issues.apache.org/jira/browse/FLINK-34691
Project: Flink
Is
Hi Di
Thank you for initiating this FLIP, +1 for this.
Regarding the option `doris.filter.query` of doris source table
Can we directly implement the FilterPushDown capability of Flink Source
like Jdbc Source [1] instead of introducing an option?
Regarding two-phase commit,
> At the same time,
73 matches
Mail list logo
