Re: [PR] [Improve] improve avro format convert [seatunnel]

2024-01-05 Thread via GitHub
Hisoka-X merged PR #6082: URL: https://github.com/apache/seatunnel/pull/6082 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunn

Re: [PR] [Improve] improve avro format convert [seatunnel]

2024-01-05 Thread via GitHub
hailin0 commented on PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#issuecomment-1878271648 waiting for ci ok -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [Improve] improve avro format convert [seatunnel]

2024-01-04 Thread via GitHub
liunaijie commented on PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#issuecomment-1878270510 @Hisoka-X @zhilinli123 added another commit to fix bytes type convert. and add type check in fake source result https://github.com/apache/seatunnel/pull/6082/commits/7235aedd77914ec52

Re: [PR] [Improve] improve avro format convert [seatunnel]

2024-01-04 Thread via GitHub
liunaijie commented on code in PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#discussion_r1442576859 ## seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e/src/test/java/org/apache/seatunnel/e2e/connector/kafka/KafkaIT.java: ## @@ -294,27 +297,150 @@ publi

Re: [PR] [Improve] improve avro format convert [seatunnel]

2024-01-04 Thread via GitHub
Hisoka-X commented on code in PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#discussion_r1442567593 ## seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e/src/test/java/org/apache/seatunnel/e2e/connector/kafka/KafkaIT.java: ## @@ -294,27 +297,150 @@ public

Re: [PR] [Improve] improve avro format convert [seatunnel]

2024-01-04 Thread via GitHub
Hisoka-X commented on PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#issuecomment-1878236145 Could you give us more detail information? ![image](https://github.com/apache/seatunnel/assets/32387433/c8045f43-984f-4a35-a7e7-29085c71aa4c) -- This is an automated message

Re: [PR] [Improve] improve avro format convert [seatunnel]

2024-01-04 Thread via GitHub
Hisoka-X commented on code in PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#discussion_r1442547319 ## docs/en/connector-v2/formats/avro.md: ## @@ -0,0 +1,125 @@ +# Avro format + +Avro is very popular in streaming data pipeline. Now seatunnel supports Avro format

Re: [PR] [Improve] improve avro format convert [seatunnel]

2024-01-04 Thread via GitHub
zhilinli123 commented on PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#issuecomment-1877281722 PTAL: @hailin0 @Hisoka-X -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] [Improve] improve avro format convert [seatunnel]

2024-01-02 Thread via GitHub
zhilinli123 commented on PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#issuecomment-1873750119 Wait CI, looks good, can you add documentation about avro for this @liunaijie -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] [Improve] improve avro format convert [seatunnel]

2024-01-02 Thread via GitHub
liunaijie commented on code in PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#discussion_r1439223872 ## seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e/src/test/resources/avro/kafka_avro_to_assert.conf: ## @@ -82,7 +82,21 @@ sink { rul

Re: [PR] [Improve] improve avro format convert [seatunnel]

2024-01-01 Thread via GitHub
Hisoka-X commented on code in PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#discussion_r1439126660 ## seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e/src/test/resources/avro/kafka_avro_to_assert.conf: ## @@ -82,7 +82,21 @@ sink { rule

Re: [PR] [Improve] improve avro format convert [seatunnel]

2023-12-28 Thread via GitHub
liunaijie commented on code in PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#discussion_r1438021868 ## seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e/src/test/resources/avro/kafka_avro_to_assert.conf: ## @@ -82,7 +82,21 @@ sink { rul

Re: [PR] [Improve] improve avro format convert [seatunnel]

2023-12-28 Thread via GitHub
zhilinli123 commented on PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#issuecomment-1871705105 Add a reference to the description of the document information: https://github.com/apache/seatunnel/blob/dev/docs/en/connector-v2/formats/ogg-json.md -- This is an automated m

Re: [PR] [Improve] improve avro format convert [seatunnel]

2023-12-28 Thread via GitHub
zhilinli123 commented on code in PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#discussion_r1437967328 ## seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e/src/test/resources/avro/kafka_avro_to_assert.conf: ## @@ -82,7 +82,21 @@ sink { r

Re: [PR] [Improve] improve avro format convert [seatunnel]

2023-12-28 Thread via GitHub
liunaijie commented on code in PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#discussion_r1437654631 ## seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e/src/test/resources/avro/kafka_avro_to_assert.conf: ## @@ -82,7 +82,21 @@ sink { rul

Re: [PR] [Improve] improve avro format convert [seatunnel]

2023-12-28 Thread via GitHub
liunaijie commented on code in PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#discussion_r1437654001 ## seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e/src/test/java/org/apache/seatunnel/e2e/connector/kafka/KafkaIT.java: ## @@ -294,17 +296,100 @@ publi

Re: [PR] [Improve] improve avro format convert [seatunnel]

2023-12-28 Thread via GitHub
zhilinli123 commented on code in PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#discussion_r1437477521 ## seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e/src/test/java/org/apache/seatunnel/e2e/connector/kafka/KafkaIT.java: ## @@ -294,17 +296,100 @@ pub

Re: [PR] [Improve] improve avro format convert [seatunnel]

2023-12-28 Thread via GitHub
zhilinli123 commented on code in PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#discussion_r1437477221 ## seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e/src/test/resources/avro/kafka_avro_to_assert.conf: ## @@ -82,7 +82,21 @@ sink { r

Re: [PR] [Improve] improve avro format convert [seatunnel]

2023-12-25 Thread via GitHub
zhilinli123 commented on PR #6082: URL: https://github.com/apache/seatunnel/pull/6082#issuecomment-1869274972 Add end to end data End to end test , refer to checkDebeziumFormat() https://github.com/liunaijie/seatunnel/blob/avro-array/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-ka

[PR] [Improve] improve avro format convert [seatunnel]

2023-12-25 Thread via GitHub
liunaijie opened a new pull request, #6082: URL: https://github.com/apache/seatunnel/pull/6082 ### Purpose of this pull request improve the avro format array convert ### Does this PR introduce _any_ user-facing change? ### How was this patch tested?