2020-08-20 13:42:10 UTC - Lari Hotari: Are there any plans to upgrade the bundled Protobuf version within Pulsar? It's currently 3.5.1 (defined in <https://github.com/apache/pulsar/blob/master/pom.xml#L125-L126>) which is pretty old. The reason why I'm asking is that I'm using Protobuf within Pulsar functions. The jar file includes protobuf protoc generated classes. There is a problem with protobuf 3.5.1 runtime since it cannot use protoc generated classes from a Protobuf version >= 3.8.0. error message is ```java.lang.NoSuchMethodError: 'com.google.protobuf.Descriptors$FileDescriptor com.google.protobuf.Descriptors$FileDescriptor.internalBuildGeneratedFileFrom(java.lang.String[], com.google.protobuf.Descriptors$FileDescriptor[])'``` It would be nice to be able to use a newer protobuf (protoc generated classes) in Pulsar Functions. Should I create a Github issue?
I'd suggest bumping protobuf to 3.11.4 since 3.11.x is used in Apache Avro (3.11.1 in <https://github.com/apache/avro/blob/master/lang/java/pom.xml#L50>) . Besides bumping protobuf, I'd assume that it makes sense to upgrade Apache Avro version used in Pulsar. The latest is 1.10.0 . Pulsar uses currently 1.9.1 (<https://github.com/apache/pulsar/blob/master/pom.xml#L137>) . heart : ckdarby ---- 2020-08-20 14:14:37 UTC - prem m: @prem m has joined the channel ---- 2020-08-20 14:55:34 UTC - Addison Higham: I think there might already be an issue tracking this. I think there are some issues with a protobuf upgrade at the moment white_check_mark : Lari Hotari ---- 2020-08-20 15:47:01 UTC - Lari Hotari: I didn't find an explicit issue about it. The search <https://github.com/apache/pulsar/issues?q=protobuf+is%3Aopen> shows that there has been activity in <https://github.com/apache/pulsar/issues/7674> and <https://github.com/apache/pulsar/issues/7642> recently but I didn't see any mention about the protobuf upgrade. ---- 2020-08-20 15:50:54 UTC - Lari Hotari: In #7674 issue comments @Sijie Guo mentions that "@hnail is working on a proposal for improving Protobuf schema". Does that also include upgrading Protobuf? ---- 2020-08-20 15:50:59 UTC - hnail: @hnail has joined the channel ---- 2020-08-20 17:34:59 UTC - Sijie Guo: I think #7674 is a different story. For upgrading the protobuf to a newer version, let’s create a separate issue to track it. ---- 2020-08-20 17:53:50 UTC - Lari Hotari: Ok. would it help if I create an issue? ---- 2020-08-21 00:02:51 UTC - Sijie Guo: Yes. Please do that :slightly_smiling_face: white_check_mark : Lari Hotari ---- 2020-08-21 06:03:52 UTC - Lari Hotari: @Sijie Guo I created <https://github.com/apache/pulsar/issues/7869> for the protobuf upgrade. ----
