Alternatively, does anyone know how it was done that made the code generated by protobuf libraries extend/implement the hadoop third-party JAR instead of it's own? I can perhaps regenerate the Hive code with the Hadoop 3P libraries and that might fix it.
org.apache.hadoop.thirdparty.protobuf.GeneratedMessageV3 On Wed, Dec 9, 2020 at 2:50 PM David <dam6...@gmail.com> wrote: > Hello, > > For additional context, the unit tests in Hive are using MiniDFSCluster. > I suspect that is what is loading the ProtobufRpcEngine2 class, while the > local code under test uses ProtobufRpcEngine. > > Is there a way to load MiniDFSCluster in its own class loader? > > Thanks. > > On Wed, Dec 9, 2020 at 2:34 PM David <dam6...@gmail.com> wrote: > >> Hello Gang, >> >> There seems to be a new RpcEngine created as part of Hadoop 3.3. I'm >> trying to upgrade Hive to use Hadoop 3.3 and it uses the RpcEngine >> mechanism. However, tests are failing for Hive complaining about: >> >> "ReRegistration of rpcKind" >> >> I believe this is because both classes register, statically, as >> RPC.RpcKind.RPC_PROTOCOL_BUFFER and interfere. This has broken backwards >> compatibility. Is it intended for the two engines to be prohibited from >> running at the same time? What is the workaround to allow for third-party >> libraries to use ProtobufRpcEngine (protobuf 2)? >> >> >> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine2.java >> >> >> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java >> >> Thanks. >> >
