[GitHub] [inlong] woofyzhao commented on a diff in pull request #6299: [INLONG-6297][Manager] Refactor the PollerPlugin and SortPoller interfaces

woofyzhao commented on code in PR #6299: URL: https://github.com/apache/inlong/pull/6299#discussion_r1006474691 ## inlong-manager/manager-client/src/main/java/org/apache/inlong/manager/client/api/impl/InlongGroupImpl.java: ## @@ -294,17 +294,25 @@ private InlongGroupContext gene

[GitHub] [inlong] leezng merged pull request #6305: [INLONG-6273][Dashboard] Support asynchronous loading of sink plugins

leezng merged PR #6305: URL: https://github.com/apache/inlong/pull/6305 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apach

[inlong] branch master updated (612231f71 -> 4da0d6900)

This is an automated email from the ASF dual-hosted git repository. leezng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/inlong.git from 612231f71 [INLONG-6271][Sort] Unhandled update_before data in canal json in multiple sink scene (#6272) add 4da0

[GitHub] [inlong] healchow commented on a diff in pull request #6299: [INLONG-6297][Manager] Refactor the PollerPlugin and SortPoller interfaces

healchow commented on code in PR #6299: URL: https://github.com/apache/inlong/pull/6299#discussion_r1006496423 ## inlong-manager/manager-client/src/main/java/org/apache/inlong/manager/client/api/impl/InlongGroupImpl.java: ## @@ -294,17 +294,25 @@ private InlongGroupContext gener

[GitHub] [inlong] healchow merged pull request #6299: [INLONG-6297][Manager] Refactor the PollerPlugin and SortPoller interfaces

healchow merged PR #6299: URL: https://github.com/apache/inlong/pull/6299 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apa

[inlong] branch master updated: [INLONG-6297][Manager] Refactor the PollerPlugin and SortPoller interfaces (#6299)

This is an automated email from the ASF dual-hosted git repository. healchow pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/inlong.git The following commit(s) were added to refs/heads/master by this push: new 6b96a5b41 [INLONG-6297][Manager] Refactor the

[GitHub] [inlong] thesumery opened a new pull request, #6306: [INLONG-6300][Sort] Schema update policy unifie behavior for all column change type

thesumery opened a new pull request, #6306: URL: https://github.com/apache/inlong/pull/6306 [INLONG-6300][Sort] Schema update policy unifie behavior for all column change type ### Prepare a Pull Request - [INLONG-6300][Sort] Schema update policy unifie behavior for all column chan

[GitHub] [inlong] yunqingmoswu opened a new pull request, #6308: [INLONG-6307][Sort] Add whether to ignore single-table error policy processing for multiple sink of DorisLoadNode

yunqingmoswu opened a new pull request, #6308: URL: https://github.com/apache/inlong/pull/6308 ### Prepare a Pull Request *(Change the title refer to the following example)* Title: [INLONG-6307][Sort] Add whether to ignore single-table error policy processing for multiple sink of D

[GitHub] [inlong] thesumery commented on pull request #6308: [INLONG-6307][Sort] Add whether to ignore single-table error policy processing for multiple sink of DorisLoadNode

thesumery commented on PR #6308: URL: https://github.com/apache/inlong/pull/6308#issuecomment-1293202521 Is this scene different with with schema update scene? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [inlong] lucaspeng12138 opened a new pull request, #6310: [INLONG-6309][Docker][Audit] Add configuration for audit driver and url

lucaspeng12138 opened a new pull request, #6310: URL: https://github.com/apache/inlong/pull/6310 ### Prepare a Pull Request Add configuration for audit driver and url - Fixes #6309 ### Motivation Add configuration for audit driver and url ### Modifications

[GitHub] [inlong] yunqingmoswu commented on a diff in pull request #6298: [INLONG-6296][Sort] Split one record to multiple records when the physical data has more records for KafkaLoadNode

yunqingmoswu commented on code in PR #6298: URL: https://github.com/apache/inlong/pull/6298#discussion_r1006588475 ## inlong-sort/sort-connectors/kafka/src/main/java/org/apache/inlong/sort/kafka/DynamicKafkaSerializationSchema.java: ## @@ -172,6 +189,101 @@ public ProducerRecord

[GitHub] [inlong] thesumery commented on a diff in pull request #6308: [INLONG-6307][Sort] Add whether to ignore single-table error policy processing for multiple sink of DorisLoadNode

thesumery commented on code in PR #6308: URL: https://github.com/apache/inlong/pull/6308#discussion_r1006592055 ## inlong-sort/sort-connectors/doris/src/main/java/org/apache/inlong/sort/doris/table/DorisDynamicSchemaOutputFormat.java: ## @@ -130,26 +133,28 @@ public void open(in

[GitHub] [inlong] yunqingmoswu commented on a diff in pull request #6308: [INLONG-6307][Sort] Add whether to ignore single-table error policy processing for multiple sink of DorisLoadNode

yunqingmoswu commented on code in PR #6308: URL: https://github.com/apache/inlong/pull/6308#discussion_r1006604660 ## inlong-sort/sort-connectors/doris/src/main/java/org/apache/inlong/sort/doris/table/DorisDynamicSchemaOutputFormat.java: ## @@ -130,26 +133,28 @@ public void open

[GitHub] [inlong] yunqingmoswu commented on pull request #6308: [INLONG-6307][Sort] Add whether to ignore single-table error policy processing for multiple sink of DorisLoadNode

yunqingmoswu commented on PR #6308: URL: https://github.com/apache/inlong/pull/6308#issuecomment-1293221944 > Is this scene different with with schema update scene? This part should mainly solve the problem of whether a table error affects other tables during the synchronization proce

[GitHub] [inlong] healchow merged pull request #6298: [INLONG-6296][Sort] Split one record to multiple records when the physical data has more records for KafkaLoadNode

healchow merged PR #6298: URL: https://github.com/apache/inlong/pull/6298 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apa

[inlong] branch master updated: [INLONG-6296][Sort] Split one record to multiple records when the physical data has more records (#6298)

This is an automated email from the ASF dual-hosted git repository. healchow pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/inlong.git The following commit(s) were added to refs/heads/master by this push: new 8fddb1d56 [INLONG-6296][Sort] Split one record

[GitHub] [inlong] EMsnap merged pull request #6306: [INLONG-6300][Sort] Schema update policy unifie behavior for all column change type

EMsnap merged PR #6306: URL: https://github.com/apache/inlong/pull/6306 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apach

[inlong] branch master updated: [INLONG-6300][Sort] Schema update policy unifie behavior for all column change type (#6306)

This is an automated email from the ASF dual-hosted git repository. zirui pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/inlong.git The following commit(s) were added to refs/heads/master by this push: new 453902dd3 [INLONG-6300][Sort] Schema update polic

[GitHub] [inlong] yunqingmoswu closed pull request #6308: [INLONG-6307][Sort] Add whether to ignore single-table error policy processing for multiple sink of DorisLoadNode

yunqingmoswu closed pull request #6308: [INLONG-6307][Sort] Add whether to ignore single-table error policy processing for multiple sink of DorisLoadNode URL: https://github.com/apache/inlong/pull/6308 -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [inlong] healchow commented on a diff in pull request #6302: [INLONG-6301][Sort] Doris connector add metrics

healchow commented on code in PR #6302: URL: https://github.com/apache/inlong/pull/6302#discussion_r1006945561 ## inlong-sort/sort-connectors/doris/src/main/java/org/apache/inlong/sort/doris/table/DorisDynamicSchemaOutputFormat.java: ## @@ -17,6 +17,7 @@ package org.apache.in

[GitHub] [inlong] healchow merged pull request #6308: [INLONG-6307][Sort] Add whether to ignore single-table error policy processing for multiple sink of DorisLoadNode

healchow merged PR #6308: URL: https://github.com/apache/inlong/pull/6308 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apa

[inlong] branch master updated: [INLONG-6307][Sort] Add whether to ignore single-table error policy processing for multiple sink (#6308)

This is an automated email from the ASF dual-hosted git repository. healchow pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/inlong.git The following commit(s) were added to refs/heads/master by this push: new 501d5e63c [INLONG-6307][Sort] Add whether to i

[GitHub] [inlong] thesumery opened a new pull request, #6312: [INLONG-6311][Sort] Bugfix:Multiple sink scene canal json old data missing field cause job fail

thesumery opened a new pull request, #6312: URL: https://github.com/apache/inlong/pull/6312 [INLONG-6311][Sort] Bugfix:Multiple sink scene canal json old data missing field cause job fail ### Prepare a Pull Request - [INLONG-6311][Sort] Bugfix:Multiple sink scene canal json old

[GitHub] [inlong] EMsnap commented on a diff in pull request #6312: [INLONG-6311][Sort] Bugfix:Multiple sink scene canal json old data missing field cause job fail

EMsnap commented on code in PR #6312: URL: https://github.com/apache/inlong/pull/6312#discussion_r1007556081 ## inlong-sort/sort-connectors/base/src/main/java/org/apache/inlong/sort/base/format/JsonDynamicSchemaFormat.java: ## @@ -99,7 +99,7 @@ public abstract class JsonDynamicS

[GitHub] [inlong] thesumery commented on a diff in pull request #6312: [INLONG-6311][Sort] Bugfix:Multiple sink scene canal json old data missing field cause job fail

thesumery commented on code in PR #6312: URL: https://github.com/apache/inlong/pull/6312#discussion_r1007558564 ## inlong-sort/sort-connectors/base/src/main/java/org/apache/inlong/sort/base/format/JsonDynamicSchemaFormat.java: ## @@ -99,7 +99,7 @@ public abstract class JsonDynam

[GitHub] [inlong] thesumery commented on a diff in pull request #6312: [INLONG-6311][Sort] Bugfix:Multiple sink scene canal json old data missing field cause job fail

thesumery commented on code in PR #6312: URL: https://github.com/apache/inlong/pull/6312#discussion_r1007558564 ## inlong-sort/sort-connectors/base/src/main/java/org/apache/inlong/sort/base/format/JsonDynamicSchemaFormat.java: ## @@ -99,7 +99,7 @@ public abstract class JsonDynam

[GitHub] [inlong] thesumery commented on a diff in pull request #6312: [INLONG-6311][Sort] Bugfix:Multiple sink scene canal json old data missing field cause job fail

thesumery commented on code in PR #6312: URL: https://github.com/apache/inlong/pull/6312#discussion_r1007558564 ## inlong-sort/sort-connectors/base/src/main/java/org/apache/inlong/sort/base/format/JsonDynamicSchemaFormat.java: ## @@ -99,7 +99,7 @@ public abstract class JsonDynam

[GitHub] [inlong] thesumery commented on a diff in pull request #6312: [INLONG-6311][Sort] Bugfix:Multiple sink scene canal json old data missing field cause job fail

thesumery commented on code in PR #6312: URL: https://github.com/apache/inlong/pull/6312#discussion_r1007558564 ## inlong-sort/sort-connectors/base/src/main/java/org/apache/inlong/sort/base/format/JsonDynamicSchemaFormat.java: ## @@ -99,7 +99,7 @@ public abstract class JsonDynam

[GitHub] [inlong] healchow commented on a diff in pull request #6310: [INLONG-6309][Docker][Audit] Add configuration for audit driver and url

healchow commented on code in PR #6310: URL: https://github.com/apache/inlong/pull/6310#discussion_r1007565230 ## inlong-audit/audit-docker/audit-docker.sh: ## @@ -43,11 +43,10 @@ if [ "${MQ_TYPE}" = "tubemq" ]; then fi # replace the configuration for audit store -if [ -n "$

[GitHub] [inlong] healchow opened a new pull request, #6314: [INLONG-6085][Manager] Remove the unused data consumption related classes

healchow opened a new pull request, #6314: URL: https://github.com/apache/inlong/pull/6314 ### Prepare a Pull Request - Fixes #6085 ### Motivation Remove the unused data consumption related classes. ### Modifications Remove the unused data consumption relate

[GitHub] [inlong] healchow commented on issue #6000: [Feature][Manager] Abstract Topic info to facilitate the expansion of different MQ

healchow commented on issue #6000: URL: https://github.com/apache/inlong/issues/6000#issuecomment-1294404410 Already fixed by #6230. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [inlong] thesumery commented on pull request #6312: [INLONG-6311][Sort] Bugfix:Multiple sink scene canal json old data missing field cause job fail

thesumery commented on PR #6312: URL: https://github.com/apache/inlong/pull/6312#issuecomment-1294412113 it's recommanded to config TimestampFormat and ignoreParseErrors as a configurable parameter.But failOnMissingField should not be exported. Here are the reasons. failOnMissingFiel

[GitHub] [inlong] gong commented on pull request #6302: [INLONG-6301][Sort] Doris connector add metrics

gong commented on PR #6302: URL: https://github.com/apache/inlong/pull/6302#issuecomment-1294509390 Please refer to jdbc connector, `CheckpointFuntion` just be used with funtion. And here getRuntimeContext will throw null exception, because outputFormat can't get runtimeContext. -- This