Re: Remove protobuf 2.5.0 from Spark dependencies

2023-06-01 Thread Steve Loughran
we would cut it from the hadoop dependencies, but still allow IPC messages using it to be marshalled if the protoc-compiled classes were using the protobuf-2.5 JAR *and* that JAR was on the classpath. it'd be come the homework of those apps which need protobuf-2.5, here hbase, to set things up.

Re: Remove protobuf 2.5.0 from Spark dependencies

2023-05-27 Thread Duo Zhang
For hbase, the problem is we rely on protobuf 2.5 for our coprocessors. See HBASE-27436. Cheng Pan 于2023年5月24日周三 10:00写道: > +CC dev@hbase > > Thanks, > Cheng Pan > > On Fri, May 19, 2023 at 4:08 AM Steve Loughran > wrote: > > > > > > > > On Thu, 18 May 2023 at 03:45, Cheng Pan wrote: > >> > >

Re: Remove protobuf 2.5.0 from Spark dependencies

2023-05-23 Thread Cheng Pan
+CC dev@hbase Thanks, Cheng Pan On Fri, May 19, 2023 at 4:08 AM Steve Loughran wrote: > > > > On Thu, 18 May 2023 at 03:45, Cheng Pan wrote: >> >> Steve, thanks for the information, I think HADOOP-17046 should be fine for >> the Spark case. >> >> Hadoop put the protobuf 3 into the pre-shaded h

Re: Remove protobuf 2.5.0 from Spark dependencies

2023-05-23 Thread Cheng Pan
+CC dev@hbase From: Steve Loughran Date: Friday, May 19, 2023 at 04:08To: Cc: dev Subject: Re: Remove protobuf 2.5.0 from Spark dependencies  On Thu, 18 May 2023 at 03:45, Cheng Pan <cheng...@apache.org> wrote:Steve, thanks for the information, I think HADOOP-17046 should be fine for the

Re: Remove protobuf 2.5.0 from Spark dependencies

2023-05-18 Thread Steve Loughran
On Thu, 18 May 2023 at 03:45, Cheng Pan wrote: > Steve, thanks for the information, I think HADOOP-17046 should be fine for > the Spark case. > > Hadoop put the protobuf 3 into the pre-shaded hadoop-thirdparty, and the > hadoop-client-runtime shades protobuf 2 during the package, which results >

Re: Remove protobuf 2.5.0 from Spark dependencies

2023-05-17 Thread Cheng Pan
Steve, thanks for the information, I think HADOOP-17046 should be fine for the Spark case. Hadoop put the protobuf 3 into the pre-shaded hadoop-thirdparty, and the hadoop-client-runtime shades protobuf 2 during the package, which results in protobuf 2 and 3 co-exist in hadoop-client-runtime in dif

Re: Remove protobuf 2.5.0 from Spark dependencies

2023-05-16 Thread Dongjoon Hyun
Thank you for sharing, Steve. Dongjoon On Tue, May 16, 2023 at 11:44 AM Steve Loughran wrote: > I have some bad news here which is even though hadoop cut protobuf 2.5 > support, hbase team put it back in (HADOOP-17046). I don't know if the > shaded hadoop client has removed that dependency on p

Re: Remove protobuf 2.5.0 from Spark dependencies

2023-05-16 Thread Steve Loughran
I have some bad news here which is even though hadoop cut protobuf 2.5 support, hbase team put it back in (HADOOP-17046). I don't know if the shaded hadoop client has removed that dependency on protobuf 2.5. In HADOOP-18487 i want to allow hadoop to cut that dependency, with hbase having to add it