[
https://issues.apache.org/jira/browse/HIVE-27475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17739690#comment-17739690
]
Ayush Saxena commented on HIVE-27475:
-------------------------------------
[~zabetak] isn't the fix version: 4.0.0-beta-1?
[https://github.com/apache/hive/blob/master/pom.xml#L24]
or atleast that is what I am using now
> Downgrade Kafka version to 2.5.0 to stabilise Kafka handler
> ------------------------------------------------------------
>
> Key: HIVE-27475
> URL: https://issues.apache.org/jira/browse/HIVE-27475
> Project: Hive
> Issue Type: Task
> Components: kafka integration
> Affects Versions: 4.0.0-alpha-2
> Reporter: Akshat Mathur
> Assignee: Akshat Mathur
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
> It was needed to shade a version of zstd-jni that is compatible with parquet
> in hive-ql after upgrading parquet version dowstream, otherwise Tez couldn't
> work with parquet and zstd encryption. Parquet and kafka-client both use
> zstd-jni. In hive-ql, zstd-jni is coming as a transitive dependency from
> kafka-client. The zstd-jni version in kafka-client 2.5.0 was not compatible
> with parquet, so in [PR-4082|https://github.com/apache/hive/pull/4082]
> kafka-client was upgraded to latest version which used zstd-jni version close
> to the version in parquet.
> While the upgrade fixed the zstd-ini issue, It introduced multiple
> compatibility issue in the Kafka-handler and due to lack to test cases and
> disabled tests these issues went unidentified.
> With the recent refactor in [HIVE-27402|http://example.com/], Kafka-client
> dependency was moved out of hive-ql, Which fixed the original zstd-ini
> issue.(confirmed by [~difin])
> Hence , It makes more sense to downgrade kafka versions back to 2.5.0,
> stabalize kafka-handler, enable test and then upgrade.
> Ref to discussion can be found here:
> [#4436|https://github.com/apache/hive/pull/4436]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
