[
https://issues.apache.org/jira/browse/HIVE-26591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pratik Malani updated HIVE-26591:
---------------------------------
Description:
libthrift:0.13.0 is affected with
[CVE-2020-13949|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13949]
Currently I am using Spark 3.3.0 and Hive 2.3.9 with Hadoop 3.3.4.
When we do an upgrade to use libthrift:0.14.0 and above jar, below exception is
thrown while starting the Spark Thriftserver.
{noformat}
org.apache.hive.service.ServiceException: Failed to Start HiveServer2
at
org.apache.hive.service.CompositeService.start(CompositeService.java:79)
at
org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:104)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.start(HiveThriftServer2.scala:154)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:64)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:104)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.NoSuchMethodError:
org.apache.thrift.server.TThreadPoolServer$Args.requestTimeout(I)Lorg/apache/thrift/server/TThreadPoolServer$Args;
at
org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.initializeServer(ThriftBinaryCLIService.java:101)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.start(ThriftCLIService.java:176)
at
org.apache.hive.service.CompositeService.start(CompositeService.java:69)
at
org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:104)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.start(HiveThriftServer2.scala:154)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:64)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:104)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
{noformat}
After detailed investigation, found out that since 0.14.0 version, in class
org.apache.thrift.server.TThreadPoolServer, requestTimeout property has been
removed.
[https://jar-download.com/artifacts/org.apache.thrift/libthrift/0.13.0/source-code/org/apache/thrift/server/TThreadPoolServer.java]
Below code snippet is from libthrift:0.13.0, but the below mentioned attributes
have been removed from libthrift:0.14.0 onwards.
!image-2022-10-03-19-55-16-030.png|width=428,height=88!
Even in latest hive release (3.1.3), it is still referencing to the
requestTimeout attribute has been removed.
[https://jar-download.com/artifacts/org.apache.hive/hive-service/3.1.3/source-code/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java]
!image-2022-10-03-19-51-20-052.png|width=769,height=98!
Can we have any alternative approach or any fix version for the above mentioned
issue
was:
libthrift:0.13.0 is affected with
[CVE-2020-13949|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13949]
Currently I am using Spark 3.3.0 and Hive 2.3.9 with Hadoop 3.3.4.
When we do an upgrade to use libthrift:0.14.0 and above jar, below exception is
thrown while starting the Spark Thriftserver.
{noformat}
org.apache.hive.service.ServiceException: Failed to Start HiveServer2
at
org.apache.hive.service.CompositeService.start(CompositeService.java:79)
at
org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:104)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.start(HiveThriftServer2.scala:154)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:64)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:104)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.NoSuchMethodError:
org.apache.thrift.server.TThreadPoolServer$Args.requestTimeout(I)Lorg/apache/thrift/server/TThreadPoolServer$Args;
at
org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.initializeServer(ThriftBinaryCLIService.java:101)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.start(ThriftCLIService.java:176)
at
org.apache.hive.service.CompositeService.start(CompositeService.java:69)
at
org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:104)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.start(HiveThriftServer2.scala:154)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:64)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:104)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
{noformat}
After detailed investigation, found out that since 0.14.0 version, in class
org.apache.thrift.server.TThreadPoolServer, requestTimeout property has been
removed.
Even in latest hive release (3.1.3), it is still referencing to the
requestTimeout attribute has been removed.
[https://jar-download.com/artifacts/org.apache.hive/hive-service/3.1.3/source-code/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java]
!image-2022-10-03-19-51-20-052.png|width=769,height=98!
Can we have any alternative approach or any fix version for the above mentioned
issue
> libthrift 0.14.0 onwards doesn't works with Hive (All versions)
> ---------------------------------------------------------------
>
> Key: HIVE-26591
> URL: https://issues.apache.org/jira/browse/HIVE-26591
> Project: Hive
> Issue Type: Bug
> Components: Hive, Metastore
> Affects Versions: 1.2.2, 2.3.7, 2.3.9
> Reporter: Pratik Malani
> Assignee: Navis Ryu
> Priority: Critical
> Fix For: 3.1.3, 4.0.0
>
> Attachments: image-2022-10-03-19-51-20-052.png,
> image-2022-10-03-19-55-16-030.png
>
>
> libthrift:0.13.0 is affected with
> [CVE-2020-13949|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13949]
> Currently I am using Spark 3.3.0 and Hive 2.3.9 with Hadoop 3.3.4.
> When we do an upgrade to use libthrift:0.14.0 and above jar, below exception
> is thrown while starting the Spark Thriftserver.
> {noformat}
> org.apache.hive.service.ServiceException: Failed to Start HiveServer2
> at
> org.apache.hive.service.CompositeService.start(CompositeService.java:79)
> at
> org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:104)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.start(HiveThriftServer2.scala:154)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:64)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:104)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958)
> at
> org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> Caused by: java.lang.NoSuchMethodError:
> org.apache.thrift.server.TThreadPoolServer$Args.requestTimeout(I)Lorg/apache/thrift/server/TThreadPoolServer$Args;
> at
> org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.initializeServer(ThriftBinaryCLIService.java:101)
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.start(ThriftCLIService.java:176)
> at
> org.apache.hive.service.CompositeService.start(CompositeService.java:69)
> at
> org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:104)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.start(HiveThriftServer2.scala:154)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:64)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:104)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958)
> at
> org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> {noformat}
> After detailed investigation, found out that since 0.14.0 version, in class
> org.apache.thrift.server.TThreadPoolServer, requestTimeout property has been
> removed.
>
>
> [https://jar-download.com/artifacts/org.apache.thrift/libthrift/0.13.0/source-code/org/apache/thrift/server/TThreadPoolServer.java]
> Below code snippet is from libthrift:0.13.0, but the below mentioned
> attributes have been removed from libthrift:0.14.0 onwards.
> !image-2022-10-03-19-55-16-030.png|width=428,height=88!
>
>
> Even in latest hive release (3.1.3), it is still referencing to the
> requestTimeout attribute has been removed.
> [https://jar-download.com/artifacts/org.apache.hive/hive-service/3.1.3/source-code/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java]
> !image-2022-10-03-19-51-20-052.png|width=769,height=98!
> Can we have any alternative approach or any fix version for the above
> mentioned issue
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
