Adding debian-go@, I think this topic deserves a wider audience On Mon, Jun 3, 2024 at 2:31 PM Reinhard Tartler <siret...@gmail.com> wrote:
> Basically notary insists on golang-github-golang-protobuf-1-3-dev, > whereas golang-github-grpc-ecosystem-grpc-gateway.v2-dev (which is coming > via the opentelemetry deps) is expecting > golang-github-golang-protobuf-1-5-dev. > > How can we resolve this situation? > so, I noticed that we have at least 12 packages in Debian that have alternative dependency declaration in the form of `golang-github-golang-protobuf-1-3-dev | golang-github-golang-protobuf-1-5-dev`. Most (if not all of them) contain generated *.pb.go files. I think I may have been under the false assumption that code generated with protoc-gen-go-1-3 would not be compatible with golang-github-golang-protobuf-1-5-dev. But is code generated with protoc-gen-go-1-5 actually compatible with golang-github-golang-protobuf-1-3-dev, that is, can it be linked with golang/protobuf @v1.3? However, in that case the alternative "Depends" on those 12+ packages still confuses me. Let me explain. As far as I can tell, there are several implementations of protobuf, each coming with their own code generator and runtime library. Let me put this down the four implementations of protobuf in Debian in the following table: source package | generator | notes -------------- | --------- | ----- golang-gogoprotobuf | gogoprotobuf | more performant, deprecated upstream golang-github-golang-protobuf-1-3 | protoc-gen-go-1-3 | implements protobuf APIv1 golang-github-golang-protobuf-1-5 | protoc-gen-go-1-5 | is a version of APIv1 implemented "in terms of APIv2" golang-google-protobuf | protoc-gen-go | APIv2, supersedes the above The above information is mostly taken out of my understanding from https://go.dev/blog/protobuf-apiv2 The alternative dependencies in the 12+ package seem to imply that the generated code in those packages can be freely used with the runtimes that come with either golang/protobuf@v1.3 or @v1.5. But is that actually a safe assumption? I'm particularly concerned that code generated with protoc-gen-go-1-5 is unlikely to work with the library that comes with golang-github-golang-protobuf-1-3. I can see the other way around (code generated with protoc-gen-go-1-3 working with golang/protobuf@v1.3), but that sounds like something we should set a policy for. I totally see how code generated with gogoprotobuf not being compatible with anything else, leading to issues such as #975431 / https://github.com/containerd/ttrpc/issues/62#issuecomment-686768932. So this is effectively prevented by the "Conflicts" relationship in golang-github-golang-protobuf-1-5-dev: Package: golang-github-golang-protobuf-1-5-dev Source: golang-github-golang-protobuf-1-5 Version: 1.5.4-1 Conflicts: golang-github-golang-protobuf-1-3-dev, golang-goprotobuf-dev Similarly, we have another relationship here: Package: golang-github-golang-protobuf-1-3-dev Source: golang-github-golang-protobuf-1-3 Version: 1.3.5-4 Conflicts: golang-github-golang-protobuf-1-5-dev (Isn't there a missing Conflicts relationship on golang-goprotobuf-dev? -- is that intentional or an oversight?) In a similar vein, I can see that code generated by protoc-gen-go might cause similar crashes when used with the runtime from golang-github-golang-protobuf-1-3 (or possibly golang-github-golang-protobuf-1-5, but I'm not sure). Note that neither package declares any conflicts to golang-google-protobuf-dev (which comes from google/protobuf). Is that intentional or an oversight? Does my rambling above make sense? What conclusions do we want to take from here? - It seems to me that we probably want to move packages away from src:golang-gogoprotobuf as much as possible, in particular because of the interoperability concerns. Is that agreeable? - It also seems to me that we probably want to avoid having both golang-github-golang-protobuf-1-3 and golang-github-golang-protobuf-1-5 at the same time. Both implement APIv1, and as such should be source-code compatible. We currently seem to have 19 packages build-depending on golang-github-golang-protobuf-1-3-dev. Can we lift-and-shift all of them in one transition to @v1.5? - As for packaging docker 2.4 (cf. #1033839, which spun off this conversation), which of the four library implementations above should it be linking against? -- We can savely rule out gogoprotobuf (it's deprecated), and I guess google-protobuf as well, because I don't think all dependencies have moved to the APIv2. That leaves two options: golang-protobuf @v1.3 and @v1.5. Given the compatibility concerns above, I guess @v1.5 is the better choice. That means that at least the 'notary' package needs to be updated (as Arnaud points separately) to pick up protobuf v1.5. - Currently, packages that use grpc (that is, build-depend on golang-google-grpc-dev or protoc-gen-go-grpc), contain code that is generated with golang/protobuf @v1.5. Wouldn't we expect applications that use grpc to (possibly silently) break when linked against any other library than protobuf @v1.5? Would it help to reflect this in package relationships (i.e., add appropriate Conflicts or Build-Conflicts relationships)? Please do share your thoughts and opinions, I'd find the whole topic rather confusing and would like to help solve it, and not make it worse. -- regards, Reinhard
