Re: [VOTE] Apache Flink CDC Release 3.4.0, release candidate #2

Hi Yanquan, 1. Xiqian has found that significant performance regressions in FLINK-37104[1]. It's better to block until this is fixed. 2. FLINK-37738[2] n your release note is still not ready, maybe we can modify it to 3.4.1 or until it is merged. [1] https://issues.apache.org/jira/browse/FLINK-37

[jira] [Created] (FLINK-37738) [cdc-connector][postgres] Support read changelog as append only mode

Hongshun Wang created FLINK-37738: - Summary: [cdc-connector][postgres] Support read changelog as append only mode Key: FLINK-37738 URL: https://issues.apache.org/jira/browse/FLINK-37738 Project

[jira] [Created] (FLINK-37696) replace postgres image to official pg 14 and adjust test to use pgoutput plugin

Hongshun Wang created FLINK-37696: - Summary: replace postgres image to official pg 14 and adjust test to use pgoutput plugin Key: FLINK-37696 URL: https://issues.apache.org/jira/browse/FLINK-37696

Re: [ANNOUNCE] New Apache Flink Committer - Yanquan Lv

Congrats Yanquan ! Best, Hongshun On Wed, Apr 16, 2025 at 9:04 PM gongzhongqiang wrote: > Congrats Yanquan Lv! Well deserved. > > Best, > Zhongqiang Gong > > Leonard Xu 于2025年4月16日周三 15:24写道： > > > Hi everyone, > > > > On behalf of the PMC, I'm happy to let you know that Yanquan Lv has > becom

Re: [ANNOUNCE] New Apache Flink Committer - Xiqian Yu

Congratulations! Best Hongshun > 2025年4月16日 15:25，Leonard Xu 写道： > > becoming

Re: [VOTE] Release flink-connector-elasticsearch v3.1.0, release candidate #1

+1 (non-binding) - verified JIRA release notes - Verify hashes and verify signatures - verified release tag, CI, and staging repo - reviewed web PR Best Hongshun On Wed, Apr 9, 2025 at 10:49 PM Ferenc Csaky wrote: > +1 (non-binding) > > - verified checksum and signature > - checked no binarie

[jira] [Created] (FLINK-37641) Fix: SchemaEvolveE2eITCase.testLenientSchemaEvolution timeout

Hongshun Wang created FLINK-37641: - Summary: Fix: SchemaEvolveE2eITCase.testLenientSchemaEvolution timeout Key: FLINK-37641 URL: https://issues.apache.org/jira/browse/FLINK-37641 Project: Flink

Re: How to add flink bot in flink subprojects(such as flink-connector-kafka)

Hi Robert, It seems that flinkbot was added by you, would you like to give some introduction. Best Hongshun On Wed, Apr 9, 2025 at 1:51 PM Hongshun Wang wrote: > Dear Devs, > > Currently, flinkbot[1] is very helpful to rerun failed tests. Anyone know > how to add this to flink subp

How to add flink bot in flink subprojects(such as flink-connector-kafka)

Dear Devs, Currently, flinkbot[1] is very helpful to rerun failed tests. Anyone know how to add this to flink subprojects(such as flink-connector-kafka, flink cdc)? It's inconvenient for developers who are not a committer. After each modification or unstable fail test, we have to ping a committer

Re: [VOTE] Release flink-connector-elasticsearch v4.0.0, release candidate #1

Hi Weijie Should we update NOTICE[1] to 2025 before release? [1] https://github.com/apache/flink-connector-elasticsearch/blob/main/NOTICE Best Hongshun On Mon, Apr 7, 2025 at 8:06 PM Zakelly Lan wrote: > +1 (binding) > > I have verified: > > - Checksum and signature > - There are no binaries

Re: [VOTE] Release flink-connector-kafka v4.0.0, release candidate #1

Hi Heise Should we update NOTICE[1] to 2025 before release? [1] https://github.com/apache/flink-connector-kafka/blob/main/NOTICE Best Hongshun

Re: [DISCUSS] FLIP-511: Support transaction id pooling in Kafka connector

> prefix. But that is available earliest in Kafka 4.1, so we need the current > approach anyhow. > > Note that the FLIP is already approved, so I can't change anything on it > anymore. Both 1. and 3. can be discussed on the pull request however [1]. > 2. is clearly out of scop

Re: [DISCUSS] FLIP-511: Support transaction id pooling in Kafka connector

> > > > > From: Arvid Heise > > Date: Wednesday, 26 March 2025 at 06:49 > > To: dev@flink.apache.org > > Subject: [EXTERNAL] Re: [DISCUSS] FLIP-511: Support transaction id > pooling in Kafka connector > > Hi Hongshun, > > > > this thread is abou

Re: [DISCUSS] FLIP-515: Dynamic Kafka Sink

is updated with MetadataUpdateEvent. How is it going in Sink? I really hope you can draw a diagram like what FLIP-246 does. Best hongshun On Thu, Apr 3, 2025 at 1:54 PM Hongshun Wang wrote: > Hi Matyas, > > Sorry for the late reply. Dynamic Kafka Source has been introduced for a >

Re: [DISCUSS] FLIP-515: Dynamic Kafka Sink

Hi Matyas, Sorry for the late reply. Dynamic Kafka Source has been introduced for a long time, and it is a little confusing for developers like me that KafkaDynamicSource and DynamicKafkaSource have the same kafka connector repo for different things. Now DynamicKafkaSink will be introduced again.

Re: [DISCUSS] FLIP-511: Support transaction id pooling in Kafka connector

-34554 2. Too much unclosed producer ( if commit is lag too much from pre-commit: https://issues.apache.org/jira/browse/FLINK-36569 3. maybe Invalid state: https://issues.apache.org/jira/browse/FLINK-37356 I do hope this flip can show how to solve this problem. Best Hongshun Wang On Fri, Feb 28

[jira] [Created] (FLINK-37529) Add flink-cdc-commo to the package to all the incremental source framework

Hongshun Wang created FLINK-37529: - Summary: Add flink-cdc-commo to the package to all the incremental source framework Key: FLINK-37529 URL: https://issues.apache.org/jira/browse/FLINK-37529 Project

[jira] [Created] (FLINK-37509) Increment scan source no need to buffer data when enable skipping backfill.

Hongshun Wang created FLINK-37509: - Summary: Increment scan source no need to buffer data when enable skipping backfill. Key: FLINK-37509 URL: https://issues.apache.org/jira/browse/FLINK-37509

[jira] [Created] (FLINK-37508) Postgres CDC Jdbc query should use debezium.snapshot.fetch.size rather than debezium.query.fetch.size.

Hongshun Wang created FLINK-37508: - Summary: Postgres CDC Jdbc query should use debezium.snapshot.fetch.size rather than debezium.query.fetch.size. Key: FLINK-37508 URL: https://issues.apache.org/jira/browse

[jira] [Created] (FLINK-37484) unbounded-chunk-first.enabled no use in unevently chunk split

Hongshun Wang created FLINK-37484: - Summary: unbounded-chunk-first.enabled no use in unevently chunk split Key: FLINK-37484 URL: https://issues.apache.org/jira/browse/FLINK-37484 Project: Flink

[jira] [Created] (FLINK-37396) Kafka connector python test fail

Hongshun Wang created FLINK-37396: - Summary: Kafka connector python test fail Key: FLINK-37396 URL: https://issues.apache.org/jira/browse/FLINK-37396 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-37380) Change TransactionalIdPrefix to a required option if Exactly once in Kafka Connector

Hongshun Wang created FLINK-37380: - Summary: Change TransactionalIdPrefix to a required option if Exactly once in Kafka Connector Key: FLINK-37380 URL: https://issues.apache.org/jira/browse/FLINK-37380

Re: [ANNOUNCE] New Apache Flink Committer - Xuyang

Congratulations, Xuyang! You deserve it Best, Hongshun On Thu, Feb 20, 2025 at 11:26 AM Shawn Huang wrote: > Congratulations ! > > Best, > Shawn Huang > > > Yanquan Lv 于2025年2月20日周四 11:05写道： > > > Congratulations, Xuyang! > > > > Best, Yanquan. > > > > > 2025年2月20日 10:24，Zakelly Lan 写道： > > >

[jira] [Created] (FLINK-37356) Recycle use of kafka producer(which commit error) will send data without AddPartitionsToTxnRequest

Hongshun Wang created FLINK-37356: - Summary: Recycle use of kafka producer(which commit error) will send data without AddPartitionsToTxnRequest Key: FLINK-37356 URL: https://issues.apache.org/jira/browse/FLINK

Re: [Discuss] Kafka Enumerator assigns splits based on the current assignment situation.

ke just a round-robin > assignment for all topic/partitions instead of hash based topic assignment > + round-robin partition? > > I am sorry if the discussion is getting more detailed, happy to move it to > a FLIP if the community agrees as well. > > Best Regards > Ahmed Hamdy &

Re: [Discuss] Kafka Enumerator assigns splits based on the current assignment situation.

based on the current circumstances. Best, Hongshun On Mon, Dec 30, 2024 at 10:03 AM Hongshun Wang wrote: > Hi Ahmed, > > Thanks for your question. > > Do you intend to keep the current assignment in the enumerator > state or are you only going to use per session assignment statu

Re: [Discuss] Kafka Enumerator assigns splits based on the current assignment situation.

lism change? > > > 1- https://issues.apache.org/jira/browse/FLINK-31762 > Best Regards > Ahmed Hamdy > > > On Fri, 27 Dec 2024 at 06:38, Hongshun Wang > wrote: > > > Hi devs, > > > > I'd like to push a new FLIP xxx: Kafka Enumerator assigns spl

Re: [DISCUSS] FLIP-467: Introduce Generalized Watermarks

Hi Xu Hang, Sorry, it's late to join this. Thanks for your job, it make sense to me. I have several questions: > With this abstraction, the original event-time watermark can be seen as a built-in use case of it. I wonder whether to replace the older event-time watermark strategy in this FLIP or

[Discuss] Kafka Enumerator assigns splits based on the current assignment situation.

Hi devs, I'd like to push a new FLIP xxx: Kafka Enumerator assigns splits based on the current assignment situation. Currently, when the Kafka enumerator finds a new split, it will choose a reader using the following algorithm: ```java static int getSplitOwner(TopicPartition tp, int numReaders)

[jira] [Created] (FLINK-36844) Postgres CDC document add mapping from postgres timestamp_tz to flink timestamp ltz

Hongshun Wang created FLINK-36844: - Summary: Postgres CDC document add mapping from postgres timestamp_tz to flink timestamp ltz Key: FLINK-36844 URL: https://issues.apache.org/jira/browse/FLINK-36844

Re: FLIP-484: Add custom metric variables to operators

Hi Piotr, thanks for the proposal. > it would be helpful to group metrics from each source/sink instance together, Could you please give a more detailed example how to do it? Currently, the example only shows source. Thanks, Hongshun On Thu, Nov 14, 2024 at 8:36 PM Roman Khachatryan wrote:

[jira] [Created] (FLINK-36618) Improve PostgresDialect.discoverDataCollections to reduce the start time of Postgres CDC

Hongshun Wang created FLINK-36618: - Summary: Improve PostgresDialect.discoverDataCollections to reduce the start time of Postgres CDC Key: FLINK-36618 URL: https://issues.apache.org/jira/browse/FLINK-36618

[jira] [Created] (FLINK-36609) Protobuf Format not support proto3

Hongshun Wang created FLINK-36609: - Summary: Protobuf Format not support proto3 Key: FLINK-36609 URL: https://issues.apache.org/jira/browse/FLINK-36609 Project: Flink Issue Type: Improvement

Re: [DISCUSS] FLIP-480: Support to deploy script in application mode

Hi, ShengKai Thanks for pushing for this meaningful job. I have some problems: Firstly, the outline design show the overall design in k8s. I wonder what is the scope of this FLIP, only aim for k8s, not including yarn? Secondly, > Flink offers great flexibility by permitting users to pass custom

[jira] [Created] (FLINK-36174) CDC yaml without pipeline will throw NullPointException

Hongshun Wang created FLINK-36174: - Summary: CDC yaml without pipeline will throw NullPointException Key: FLINK-36174 URL: https://issues.apache.org/jira/browse/FLINK-36174 Project: Flink

[jira] [Created] (FLINK-36163) Kafka cdc pipeline source supports protobuf format.

Hongshun Wang created FLINK-36163: - Summary: Kafka cdc pipeline source supports protobuf format. Key: FLINK-36163 URL: https://issues.apache.org/jira/browse/FLINK-36163 Project: Flink Issue

Re: Flink CDC insert-only changelog mode

:45 AM Daniel Henneberger wrote: > Unfortunately no, this would not be helpful. The changelog type needs to be > insert-only to invoke the correct streaming planner rules. > > On Wed, Aug 21, 2024 at 9:01 PM Hongshun Wang > wrote: > > > Hi Daniel， > > > > >

[jira] [Created] (FLINK-36150) tables.exclude is not valid if scan.binlog.newly-added-table.enabled is true.

Hongshun Wang created FLINK-36150: - Summary: tables.exclude is not valid if scan.binlog.newly-added-table.enabled is true. Key: FLINK-36150 URL: https://issues.apache.org/jira/browse/FLINK-36150

Re: Flink CDC insert-only changelog mode

Hi Daniel， > > There's no way to convert a different changelog stream to an insertonly > stream so I'm pushing this upstream to the connector. I wonder whether you just want the insert type changelog and ignore other kinds of changelog, or just want to read all kinds of changelog as insert in fli

[jira] [Created] (FLINK-36094) CDC SchemaRegistryRequestHandler should throw exception which is not SchemaEvolveException

Hongshun Wang created FLINK-36094: - Summary: CDC SchemaRegistryRequestHandler should throw exception which is not SchemaEvolveException Key: FLINK-36094 URL: https://issues.apache.org/jira/browse/FLINK-36094

[jira] [Created] (FLINK-36076) Hotfix: Set isSchemaChangeApplying as isSchemaChangeApplying for thread safe.

Hongshun Wang created FLINK-36076: - Summary: Hotfix: Set isSchemaChangeApplying as isSchemaChangeApplying for thread safe. Key: FLINK-36076 URL: https://issues.apache.org/jira/browse/FLINK-36076

[jira] [Created] (FLINK-36036) CDC remove unused dependency jar from calcite-core

Hongshun Wang created FLINK-36036: - Summary: CDC remove unused dependency jar from calcite-core Key: FLINK-36036 URL: https://issues.apache.org/jira/browse/FLINK-36036 Project: Flink Issue

[jira] [Created] (FLINK-36023) Flink K8S Native Application Mode add wrong jar url.

Hongshun Wang created FLINK-36023: - Summary: Flink K8S Native Application Mode add wrong jar url. Key: FLINK-36023 URL: https://issues.apache.org/jira/browse/FLINK-36023 Project: Flink Issue

[jira] [Created] (FLINK-36007) cdc load and register factory in once search

Hongshun Wang created FLINK-36007: - Summary: cdc load and register factory in once search Key: FLINK-36007 URL: https://issues.apache.org/jira/browse/FLINK-36007 Project: Flink Issue Type

[jira] [Created] (FLINK-35994) CDC pipeline use SavepointRestoreSettings in flink config flile as a fallback strategy.

Hongshun Wang created FLINK-35994: - Summary: CDC pipeline use SavepointRestoreSettings in flink config flile as a fallback strategy. Key: FLINK-35994 URL: https://issues.apache.org/jira/browse/FLINK-35994

[jira] [Created] (FLINK-35968) Remove flink-cdc-runtime depedency from connectors

Hongshun Wang created FLINK-35968: - Summary: Remove flink-cdc-runtime depedency from connectors Key: FLINK-35968 URL: https://issues.apache.org/jira/browse/FLINK-35968 Project: Flink Issue

[jira] [Created] (FLINK-35859) [flink-cdc] Fix: The assigner is not ready to offer finished split information, this should not be called

Hongshun Wang created FLINK-35859: - Summary: [flink-cdc] Fix: The assigner is not ready to offer finished split information, this should not be called Key: FLINK-35859 URL: https://issues.apache.org/jira/browse

[jira] [Created] (FLINK-35849) [flink-cdc] Use expose_snapshot to read snapshot of postgres cdc connector.

Hongshun Wang created FLINK-35849: - Summary: [flink-cdc] Use expose_snapshot to read snapshot of postgres cdc connector. Key: FLINK-35849 URL: https://issues.apache.org/jira/browse/FLINK-35849

[jira] [Created] (FLINK-35760) [docs] Add scan.newly-added-table.enabled to docs.

Hongshun Wang created FLINK-35760: - Summary: [docs] Add scan.newly-added-table.enabled to docs. Key: FLINK-35760 URL: https://issues.apache.org/jira/browse/FLINK-35760 Project: Flink Issue

[jira] [Created] (FLINK-35759) [docs] Add 'scan.incremental.snapshot.backfill.skip' to docs.

Hongshun Wang created FLINK-35759: - Summary: [docs] Add 'scan.incremental.snapshot.backfill.skip' to docs. Key: FLINK-35759 URL: https://issues.apache.org/jira/browse/FLINK-35759 Proj

[jira] [Created] (FLINK-35715) Mysql Source support schema cache to deserialize record

Hongshun Wang created FLINK-35715: - Summary: Mysql Source support schema cache to deserialize record Key: FLINK-35715 URL: https://issues.apache.org/jira/browse/FLINK-35715 Project: Flink

[jira] [Created] (FLINK-35693) Change variable specificOffsetFile, specificOffsetPos and startupTimestampMillis to Offset in StartupOptions

Hongshun Wang created FLINK-35693: - Summary: Change variable specificOffsetFile, specificOffsetPos and startupTimestampMillis to Offset in StartupOptions Key: FLINK-35693 URL: https://issues.apache.org/jira

Re: [ANNOUNCE] New Apache Flink Committer - Hang Ruan

Congratulations Hang! Best, Hongshun On Mon, Jun 17, 2024 at 5:49 PM Ron Liu wrote: > Congratulations, Hang! > > Best, > Ron > > Geng Biao 于2024年6月17日周一 12:35写道： > > > Congrats, Hang! > > Best, > > Biao Geng > > > > 发送自 Outlook for iOS > > ___

Re: [ANNOUNCE] New Apache Flink Committer - Zhongqiang Gong

Congrats Zhongqiang ! Best, Hongshun On Mon, Jun 17, 2024 at 12:35 PM Geng Biao wrote: > Congratulations, Zhongqiang! > Best, > Biao Geng > > 发送自 Outlook for iOS > > 发件人: Zakelly Lan > 发送时间: Monday, June 17, 2024 12:11:47 PM > 收件人: dev@fl

[jira] [Created] (FLINK-35567) CDC BinaryWriter cast NullableSerializerWrapper error

Hongshun Wang created FLINK-35567: - Summary: CDC BinaryWriter cast NullableSerializerWrapper error Key: FLINK-35567 URL: https://issues.apache.org/jira/browse/FLINK-35567 Project: Flink

[jira] [Created] (FLINK-35524) Clear connections pools when reader exist.

Hongshun Wang created FLINK-35524: - Summary: Clear connections pools when reader exist. Key: FLINK-35524 URL: https://issues.apache.org/jira/browse/FLINK-35524 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-35387) PG CDC source support heart beat

Hongshun Wang created FLINK-35387: - Summary: PG CDC source support heart beat Key: FLINK-35387 URL: https://issues.apache.org/jira/browse/FLINK-35387 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-35385) upgrader flink dependency version to 1.19

Hongshun Wang created FLINK-35385: - Summary: upgrader flink dependency version to 1.19 Key: FLINK-35385 URL: https://issues.apache.org/jira/browse/FLINK-35385 Project: Flink Issue Type

[jira] [Created] (FLINK-35349) Use connection in openJdbcConnection of SqlServerDialect/Db2Dialect/OracleDialect

Hongshun Wang created FLINK-35349: - Summary: Use connection in openJdbcConnection of SqlServerDialect/Db2Dialect/OracleDialect Key: FLINK-35349 URL: https://issues.apache.org/jira/browse/FLINK-35349

[jira] [Created] (FLINK-35344) Move same code from multiple subclasses to JdbcSourceChunkSplitter

Hongshun Wang created FLINK-35344: - Summary: Move same code from multiple subclasses to JdbcSourceChunkSplitter Key: FLINK-35344 URL: https://issues.apache.org/jira/browse/FLINK-35344 Project: Flink

Re: [DISCUSS] Connector releases for Flink 1.19

Hello Danny, Thanks for pushing this forward. I am available to assist with the CDC connector[1]. [1] https://github.com/apache/flink-cdc Best Hongshun On Sun, May 12, 2024 at 8:48 PM Sergey Nuyanzin wrote: > I'm in a process of preparation of RC for OpenSearch connector > > however it seems

[jira] [Created] (FLINK-35327) SQL Explain show push down condition

Hongshun Wang created FLINK-35327: - Summary: SQL Explain show push down condition Key: FLINK-35327 URL: https://issues.apache.org/jira/browse/FLINK-35327 Project: Flink Issue Type: Bug

Re: [DISCUSS] Flink CDC 3.2 Release Planning

Thanks Qinsheng for driving, +1 from my side. Besi, Hongshun On Wed, May 8, 2024 at 11:41 PM Leonard Xu wrote: > +1 for the proposal code freeze date and RM candidate. > > Best, > Leonard > > > 2024年5月8日 下午10:27，gongzhongqiang 写道： > > > > Hi Qingsheng > > > > Thank you for driving the release.

Re: [DISCUSS] FLIP-453: Promote Unified Sink API V2 to Public and Deprecate SinkFunction

Hi Martijn, Thanks for the proposal +1 from me.Some sinks still use sinkfunction; it's time to take a step forward. Best, Hongshun On Mon, May 6, 2024 at 5:44 PM Leonard Xu wrote: > +1 from my side, thanks Martijn for the effort. > > Best, > Leonard > > > 2024年5月4日 下午7:41，Ahmed Hamdy 写道： > > >

[jira] [Created] (FLINK-35281) FlinkEnvironmentUtils#addJar add each jar only once

Hongshun Wang created FLINK-35281: - Summary: FlinkEnvironmentUtils#addJar add each jar only once Key: FLINK-35281 URL: https://issues.apache.org/jira/browse/FLINK-35281 Project: Flink Issue

[jira] [Created] (FLINK-35234) Fix NullPointerException of org.apache.flink.cdc.common.configuration.ConfigurationUtils#convertToString

Hongshun Wang created FLINK-35234: - Summary: Fix NullPointerException of org.apache.flink.cdc.common.configuration.ConfigurationUtils#convertToString Key: FLINK-35234 URL: https://issues.apache.org/jira/browse

[jira] [Created] (FLINK-35149) Fix DataSinkTranslator#sinkTo ignoring pre-write topology if not TwoPhaseCommittingSink

Hongshun Wang created FLINK-35149: - Summary: Fix DataSinkTranslator#sinkTo ignoring pre-write topology if not TwoPhaseCommittingSink Key: FLINK-35149 URL: https://issues.apache.org/jira/browse/FLINK-35149

[jira] [Created] (FLINK-35143) Expose newly added tables capture in mysql pipeline connector

Hongshun Wang created FLINK-35143: - Summary: Expose newly added tables capture in mysql pipeline connector Key: FLINK-35143 URL: https://issues.apache.org/jira/browse/FLINK-35143 Project: Flink

[jira] [Created] (FLINK-35129) Postgres source commits the offset after every multiple checkpoint cycles.

Hongshun Wang created FLINK-35129: - Summary: Postgres source commits the offset after every multiple checkpoint cycles. Key: FLINK-35129 URL: https://issues.apache.org/jira/browse/FLINK-35129 Project

[jira] [Created] (FLINK-35128) Re-calculate the starting binlog offset after the new table added

Hongshun Wang created FLINK-35128: - Summary: Re-calculate the starting binlog offset after the new table added Key: FLINK-35128 URL: https://issues.apache.org/jira/browse/FLINK-35128 Project: Flink

[jira] [Created] (FLINK-35121) CDC pipeline connector should verify requiredOptions and optionalOptions

Hongshun Wang created FLINK-35121: - Summary: CDC pipeline connector should verify requiredOptions and optionalOptions Key: FLINK-35121 URL: https://issues.apache.org/jira/browse/FLINK-35121 Project

[jira] [Created] (FLINK-35093) Postgres source connector support SPECIFIC_OFFSETS start up mode from an existed replication slot.

Hongshun Wang created FLINK-35093: - Summary: Postgres source connector support SPECIFIC_OFFSETS start up mode from an existed replication slot. Key: FLINK-35093 URL: https://issues.apache.org/jira/browse/FLINK

[jira] [Created] (FLINK-35071) Remove dependency on flink-shaded from cdc source connector

Hongshun Wang created FLINK-35071: - Summary: Remove dependency on flink-shaded from cdc source connector Key: FLINK-35071 URL: https://issues.apache.org/jira/browse/FLINK-35071 Project: Flink

[jira] [Created] (FLINK-35067) Support metadata 'op_type' virtual column for Postgres CDC Connector.

Hongshun Wang created FLINK-35067: - Summary: Support metadata 'op_type' virtual column for Postgres CDC Connector. Key: FLINK-35067 URL: https://issues.apache.org/jira/browse/FLINK-35067

[jira] [Created] (FLINK-34952) Flink CDC pipeline supports SourceFunction

Hongshun Wang created FLINK-34952: - Summary: Flink CDC pipeline supports SourceFunction Key: FLINK-34952 URL: https://issues.apache.org/jira/browse/FLINK-34952 Project: Flink Issue Type

Re: [DISCUSS] Flink Website Menu Adjustment

+1 for the proposal Best Regards, Hongshun Wang On Tue, Mar 26, 2024 at 11:37 AM gongzhongqiang wrote: > Hi Martijn, > > Thank you for your feedback. > > I agree with your point that we should make a one-time update to the menu, > rather than continuously updating it. This w

Re: [NOTICE] Flink CDC is importing GitHub issues into Apache Jira

Congratulations. Thanks to Qingsheng for driving this job forward. Best, Hongshun On Wed, Mar 20, 2024 at 9:08 PM Qingsheng Ren wrote: > FYI: The auto-import has completed. Totally 137 issues are migrated from > GitHub issue to Apache Jira. Sorry again for flushing your inbox! > > Best, > Qings

Re: [ANNOUNCE] Apache Flink 1.19.0 released

Congratulations! Best, Hongshun On Tue, Mar 19, 2024 at 3:12 PM Shawn Huang wrote: > Congratulations! > > Best, > Shawn Huang > > > Xuannan Su 于2024年3月19日周二 14:40写道： > > > Congratulations! Thanks for all the great work! > > > > Best regards, > > Xuannan > > > > On Tue, Mar 19, 2024 at 1:31 PM

[jira] [Created] (FLINK-34724) Fix broken web link in document of legacy Flink CDC Sources

Hongshun Wang created FLINK-34724: - Summary: Fix broken web link in document of legacy Flink CDC Sources Key: FLINK-34724 URL: https://issues.apache.org/jira/browse/FLINK-34724 Project: Flink

[jira] [Created] (FLINK-34690) If data from upstream is decimal and primary key , starrocks sink will not support.

Hongshun Wang created FLINK-34690: - Summary: If data from upstream is decimal and primary key , starrocks sink will not support. Key: FLINK-34690 URL: https://issues.apache.org/jira/browse/FLINK-34690

[jira] [Created] (FLINK-34688) CDC framework split snapshot chunks asynchronously

Hongshun Wang created FLINK-34688: - Summary: CDC framework split snapshot chunks asynchronously Key: FLINK-34688 URL: https://issues.apache.org/jira/browse/FLINK-34688 Project: Flink Issue

[jira] [Created] (FLINK-34666) Keep assigned splits in order to fix wrong meta group calculation

Hongshun Wang created FLINK-34666: - Summary: Keep assigned splits in order to fix wrong meta group calculation Key: FLINK-34666 URL: https://issues.apache.org/jira/browse/FLINK-34666 Project: Flink

[jira] [Created] (FLINK-34634) Restarting the job will not read the changelog anymore if it stops before the synchronization of meta information is complete and some table is removed

Hongshun Wang created FLINK-34634: - Summary: Restarting the job will not read the changelog anymore if it stops before the synchronization of meta information is complete and some table is removed Key: FLINK-34634

Re: [ANNOUNCE] New Apache Flink Committer - Jiabao Sun

Congratulations, Jiabao :) Congratulations Jiabao! Best, Hongshun Best regards, Weijie On Tue, Feb 20, 2024 at 2:19 PM Runkang He wrote: > Congratulations Jiabao! > > Best, > Runkang He > > Jane Chan 于2024年2月20日周二 14:18写道： > > > Congrats, Jiabao! > > > > Best, > > Jane > > > > On Tue, Feb 20,

Re: [DISCUSS] Release new version of Flink's Kafka connector

y was added in a > backwards compatible matter (meaning a new minor version would be > sufficient). I'm looping in Hongshun Wang and Leonard Xu to help clarify > this. > > There's also a discussion happening in an open PR [3] on dropping support > for Flink 1.18 afterwards (sinc

[jira] [Created] (FLINK-34196) FLIP-389 Annotate SingleThreadFetcherManager as PublicEvolving.

Hongshun Wang created FLINK-34196: - Summary: FLIP-389 Annotate SingleThreadFetcherManager as PublicEvolving. Key: FLINK-34196 URL: https://issues.apache.org/jira/browse/FLINK-34196 Project: Flink

[RESULT][VOTE] FLIP-389 Annotate SingleThreadFetcherManager as PublicEvolving.

oving one. Thanks to all participants for discussion and voting! [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=278465498 [2] https://lists.apache.org/thread/t1zff21z440pvv48jyhm8pgtqsyplchn Best, Hongshun Wang

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

ing > the queue sounds reasonable to me. > > Thanks, > > Jiangjie (Becket) Qin > > On Wed, Jan 10, 2024 at 10:39 PM Hongshun Wang > wrote: > >> Hi Qingsheng, >> >> >> I agree with you that it would be clearer to have a new interface that >>

[jira] [Created] (FLINK-34129) correct writing: MiniBatchGlobalGroupAggFunction will make -D as +I then make +I as -U when state expired

Hongshun Wang created FLINK-34129: - Summary: correct writing: MiniBatchGlobalGroupAggFunction will make -D as +I then make +I as -U when state expired Key: FLINK-34129 URL: https://issues.apache.org/jira/browse

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

ngBlockingQueue) instance into the SplitFetcherManager. This >> hides the element queue from the connector developers and simplifies the >> SourceReaderBase to consist of only SplitFetcherManager and RecordEmitter >> as major components. >> >> In short, the public interface

[VOTE] FLIP-389: Annotate SingleThreadFetcherManager as PublicEvolving

SplitFetcherManager and RecordEmitter as major components. Any additional questions regarding this FLIP? Looking forward to hearing from you. Thanks, Hongshun Wang [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=278465498 [2] https://lists.apache.org/thread

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

Hi Becket, It has been a long time since we last discussed. Are there any other problems with this Flip from your side? I am looking forward to hearing from you. Thanks, Hongshun Wang

Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink

So cool, Big +1 for this exciting work. Best Hongshun On Thu, Dec 7, 2023 at 12:20 PM Qingsheng Ren wrote: > Thanks for kicking this off, Leonard! > > As one of the contributors of the CDC project, I'm truly honored to be part > of the community and so excited to hear the news. CDC project was

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

Hi all, Any additional questions or concern regarding this FLIP-389[1].? Looking forward to hearing from you. Thanks, Hongshun Wang [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=278465498 On Wed, Nov 22, 2023 at 3:44 PM Hongshun Wang wrote: > Hi Becket, > > Tha

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

replace its usage in SourceReaderBase. - Mark SplitFetcher and SplitFetcherTask as PublicEvolving. Any additional questions regarding this FLIP? Looking forward to hearing from you. Thanks, Hongshun Wang [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=278465498 On Wed, Nov

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

> to the end users. Having an interface of SplitFetcher is also fine, but > might not be necessary in this case. I don't know how to make SplitFetcher as PublicEnvolving but not to expose the constructor of SplitFetcher to the end users? Thanks, Hongshun Wang On Tue, Nov 21, 2023 a

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

s a class rather than Interface. Therefore, I want to change SplitFetcher to a public Interface and moving its implementation details to an implement subclass . Thanks, Hongshun Wang On Fri, Nov 17, 2023 at 6:21 PM Becket Qin wrote: > Hi Hongshun, > > SplitFetcher.enqueueTask() returns

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

kowicz If not, SplitFetcher.pause() and > SplitFetcher.resume() can be removed. In fact, they seem no longer used > anywhere. It seems no use any more. CC, @Arvid Heise Thanks, Hongshun Wang On Fri, Nov 17, 2023 at 11:42 AM Becket Qin wrote: > Hi Hongshun, > > Thanks for updating the F

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

is > actually RecordsWithSplitIds. So it looks like we can just let > SplitFetcherManager.poll() return a RecordsWithSplitIds. > > Thanks, > > Jiangjie (Becket) Qin > > On Tue, Nov 14, 2023 at 8:11 PM Hongshun Wang > wrote: > > > Hi Becket, > > I agree

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

asy fix, but I > > > want to make sure that it's also the right fix. If we are missing good > > > abstractions, then I would rather invest in those. > > > > > > Best regards, > > > > > > Martijn > > > > > > [1] https://

  1   2   >