Right, since that variable is static I suspect that the compiler doesn't have to output a symbol for it and so "nm" won't reliably report its existence. I'm not sure if there is a great way to tell whether a given pb.cc file was linked into a given dylib, but the pb.cc file will store its descriptor as a serialized protobuf and you can look for that. If you call strings on the dylib and grep for the .proto file name or particular message names (e.g. "ContentMetaInfo" in your case) then you should be able to find the serialized descriptor if it's present.
On Thu, Jul 25, 2019 at 4:31 PM Peter Gusev <[email protected]> wrote: > I checked for "static_descriptor_initialized" variable using "nm" and it > shows only one dylib for that. You mentioned don't fully trust nm -- is > there a better tool to check that? > > 0000000000019de0 S > protobuf_content_2dmeta_2dinfo_2eproto::static_descriptor_initializer > > On Thu, 25 Jul 2019 at 16:21, Adam Cozzette <[email protected]> wrote: > >> If the generated code appears in two dylibs then there will be two of >> those static variables performing the same initialization. >> >> On Thu, Jul 25, 2019 at 4:20 PM Peter Gusev <[email protected]> wrote: >> >>> This makes me think of what could potentially cause this code to execute >>> twice?... >>> >>> On Thu, 25 Jul 2019 at 16:13, Adam Cozzette <[email protected]> >>> wrote: >>> >>>> Right, this registration will happen when that static variable's >>>> initialization occurs. Ordinarily this will happen pre-main, though if >>>> you're dynamically loading these dylibs then it will happen at that loading >>>> time as well. >>>> >>>> On Thu, Jul 25, 2019 at 4:01 PM Peter Gusev <[email protected]> wrote: >>>> >>>>> Thanks for your insights! >>>>> It may also gets complicated with the environment I'm testing it in. >>>>> Hence, I'd like to know when does protobuf invoke this method? Is it >>>>> just when static variable "static_descriptor_initializer" gets >>>>> initialized? >>>>> >>>>> Thanks, >>>>> >>>>> On Thu, 25 Jul 2019 at 14:19, Adam Cozzette <[email protected]> >>>>> wrote: >>>>> >>>>>> If you can't edit the .proto files then I think the only solution is >>>>>> to make sure the code for content-meta-info.proto only appears in one >>>>>> dylib >>>>>> (or in the main binary only). >>>>>> >>>>>> I'm not sure I would fully trust nm to report which dylibs are >>>>>> including the code, because I can imagine that some or all of the symbols >>>>>> could get stripped depending on how they're built. If the string >>>>>> "ContentMetaInfo" appears in multiple binaries then they might all be >>>>>> including the generated code for that .proto file. >>>>>> >>>>>> On Thu, Jul 25, 2019 at 2:06 PM Peter Gusev <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Thanks! I think I've seen this kind of solution somewhere online >>>>>>> before... >>>>>>> but what if I don't have any edit access to the .proto files? >>>>>>> >>>>>>> In my debugger, I noticed the crash originates in >>>>>>> >>>>>>> void AddDescriptors() { >>>>>>> >>>>>>> static ::google::protobuf::internal::once_flag once; >>>>>>> >>>>>>> ::google::protobuf::internal::call_once(once, AddDescriptorsImpl); >>>>>>> >>>>>>> } >>>>>>> >>>>>>> So I search all dylibs my plugin uses (and the plugin itself) for >>>>>>> this symbol (as ' nm <binary> | grep "AddDescriptorsImpl" ') and the >>>>>>> only >>>>>>> place I found this symbol was in one dylib: >>>>>>> >>>>>>> 0000000000000f70 T >>>>>>> __ZN38protobuf_content_2dmeta_2dinfo_2eproto18AddDescriptorsImplEv >>>>>>> >>>>>>> 0000000000013fc0 s >>>>>>> __ZZN38protobuf_content_2dmeta_2dinfo_2eproto18AddDescriptorsImplEvE10descriptor >>>>>>> I'm trying to understand, how this code might be called twice?... >>>>>>> However, I can see various "ContentMetaInfo" in three different >>>>>>> binaries (two dylibs and plugin binary). >>>>>>> >>>>>>> On Thu, 25 Jul 2019 at 13:39, Adam Cozzette <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Based on the error message I think you have the right diagnosis of >>>>>>>> the problem. There are a couple ways you could try to fix it: >>>>>>>> 1. Move the generated protobuf code into its own dylib so that the >>>>>>>> original two dylibs don't include it themselves. >>>>>>>> 2. Alternatively, if you don't rely on protobuf reflection then you >>>>>>>> can set option optimize_for = LITE_RUNTIME; in the .proto file. >>>>>>>> That will remove reflection support, and as a side effect the generated >>>>>>>> protos will not register themselves in the process-wide descriptor pool >>>>>>>> (which is where the crash came from). I think you would still have to >>>>>>>> be >>>>>>>> careful to avoid ODR violations, so you might need to carefully set up >>>>>>>> the >>>>>>>> dylibs to hide their generated protobuf symbols. >>>>>>>> >>>>>>>> On Thu, Jul 25, 2019 at 10:11 AM { peetonn } <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I'm getting a runtime error: >>>>>>>>> >>>>>>>>> *[libprotobuf ERROR google/protobuf/descriptor_database.cc:58] >>>>>>>>> File already exists in database: content-meta-info.proto* >>>>>>>>> >>>>>>>>> *[libprotobuf FATAL google/protobuf/descriptor.cc:1358] CHECK >>>>>>>>> failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): * >>>>>>>>> >>>>>>>>> *libc++abi.dylib: terminating with uncaught exception of type >>>>>>>>> google::protobuf::FatalException: CHECK failed: >>>>>>>>> GeneratedDatabase()->Add(encoded_file_descriptor, size): * >>>>>>>>> >>>>>>>>> Which, I believe, happens because my macOS plugin code links >>>>>>>>> against two .dylibs which use protobuf and, apparently, use same >>>>>>>>> protobuf >>>>>>>>> objects. (Is this assumption correct? How can I check it?) >>>>>>>>> >>>>>>>>> How one shall debug and fix this error (given dylibs are >>>>>>>>> third-party)? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "Protocol Buffers" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/protobuf/274860bb-6027-4324-9674-4ac9d21d1ca2%40googlegroups.com >>>>>>>>> <https://groups.google.com/d/msgid/protobuf/274860bb-6027-4324-9674-4ac9d21d1ca2%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Peter Gusev <https://www.linkedin.com/in/peter-gusev-8135441a/> >>>>>>> gusev.io >>>>>>> [email protected] >>>>>>> *+1 213 587-27-48* >>>>>>> >>>>>>> >>>>>>> *Research Scholar @ REMAP UCLA >>>>>>> <http://www.remap.ucla.edu/home/about>Video >>>>>>> streaming/ICN networks/Creative Coding/Interactive Media* >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> Peter Gusev <https://www.linkedin.com/in/peter-gusev-8135441a/> >>>>> gusev.io >>>>> [email protected] >>>>> *+1 213 587-27-48* >>>>> >>>>> >>>>> *Research Scholar @ REMAP UCLA >>>>> <http://www.remap.ucla.edu/home/about>Video >>>>> streaming/ICN networks/Creative Coding/Interactive Media* >>>>> >>>> >>> >>> -- >>> Peter Gusev <https://www.linkedin.com/in/peter-gusev-8135441a/> >>> gusev.io >>> [email protected] >>> *+1 213 587-27-48* >>> >>> >>> *Research Scholar @ REMAP UCLA >>> <http://www.remap.ucla.edu/home/about>Video >>> streaming/ICN networks/Creative Coding/Interactive Media* >>> >> > > -- > Peter Gusev <https://www.linkedin.com/in/peter-gusev-8135441a/> > gusev.io > [email protected] > *+1 213 587-27-48* > > > *Research Scholar @ REMAP UCLA <http://www.remap.ucla.edu/home/about>Video > streaming/ICN networks/Creative Coding/Interactive Media* > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/CADqAXr4ddxstpAPafcUf6NL-WXFh1vb%3DRbpqAPwoe5a7YVGfPQ%40mail.gmail.com.
