On Wednesday, June 10, 2015 at 9:49:03 PM UTC-7, Jan Kyjovský wrote: > > I did as you told me. And tried to compile my proto files with protoc. > Well it eat them up without any problem so I guess that clear that there is > no problem with them but in that case why is it causing such error to me. > > As I gave some hint before problem is happening when I am trying to do > following with phone.proto which is being added as second into descriptor > database > > compiler::Parser parser; > > parser.RecordErrorsTo(&errCollector); > bool bV = parser.Parse(&tokenizer, result); > std::string strErr = errCollector.GetErrorAndWarnings(); > > return bV; > > Parse returns false and error collector holds following error: > 1:1: ERROR: Multiple package definitions. > > Do you have nay idea what that could meant? > Could you post the whole content of your code? It would be easier for me to figure out what's wrong if I can run the code myself.
> > On Tuesday, 9 June 2015 20:31:38 UTC+2, Feng Xiao wrote: >> >> >> >> On Mon, Jun 8, 2015 at 10:23 PM, Jan Kyjovský <[email protected]> >> wrote: >> >>> I have remade code so I now add into descriptor database all protofiles >>> specified by directory. There is but little problem with error handling. >>> From nature how I am handling errors it seems that message is possible to >>> decode but there is one error in error collector which is bothering me: >>> >>> 1:1: ERROR: Multiple package definitions. >>> >>> *My protos are defined as follows:* >>> *addressbook_fragmented.proto:* >>> import "phone.proto"; >>> >>> package tutorial; >>> >>> option java_package = "com.example.tutorial"; >>> option java_outer_classname = "AddressBookProtos"; >>> >>> message Person { >>> required string name = 1; >>> required int32 id = 2; // Unique ID number for this person. >>> optional string email = 3; >>> >>> // enum PhoneType { >>> // MOBILE = 0; >>> // HOME = 1; >>> // WORK = 2; >>> // } >>> >>> message PhoneNumber { >>> required string number = 1; >>> optional PhoneType type = 2 [default = HOME]; >>> } >>> >>> repeated PhoneNumber phone = 4; >>> optional int32 age = 5; >>> } >>> >>> // Our address book file is just one of these. >>> message AddressBook { >>> repeated Person person = 1; >>> } >>> >>> *phone.proto:* >>> package tutorial; >>> >>> enum PhoneType { >>> MOBILE = 0; >>> HOME = 1; >>> WORK = 2; >>> } >>> >>> Even though there is error in error collector I can ignore it and decode >>> message successfully. Yet if I try to remove this package info from >>> phone.proto itis not detecting any error but FindMessageTypeByName fails to >>> find data type. I am suspecting since not all references for person data >>> type I am asking for are resolved it fails. >>> >>> So I am confused how exactly should be these package keywords be used? >>> >> I don't see a problem in your use of "package". You could try to run >> protoc on these proto files and see if it reports any error. I suspect it's >> not the .proto file's problem. >> >> >>> >>> On Wednesday, 3 June 2015 19:05:59 UTC+2, Feng Xiao wrote: >>> >>>> >>>> >>>> On Wed, Jun 3, 2015 at 12:40 AM, Jan Kyjovský <[email protected]> >>>> wrote: >>>> >>>>> HI, >>>>> >>>>> I somehow understand what you mean but still I lack in experience with >>>>> it. I tried to switch to descriptor database but still I got some >>>>> problems. >>>>> >>>>> C_ProtoDecoder::C_ProtoDecoder(std::string strFile) >>>>> { >>>>> m_strProtoFile = strFile; >>>>> >>>>> // Build the descriptor pool >>>>> m_bConstructed = ParseProtoFile(&m_ProtoFileDescr); >>>>> >>>>> if (m_bConstructed) >>>>> { >>>>> m_ProtoDescrDatabase.Add(m_ProtoFileDescr); >>>>> m_Pool = new DescriptorPool(&m_ProtoDescrDatabase); >>>>> } >>>>> } >>>>> >>>>> C_ProtoDecoder::~C_ProtoDecoder() >>>>> { >>>>> delete m_Pool; >>>>> } >>>>> >>>>> bool C_ProtoDecoder::DecodeDataAsTypeFromWireFormat(std::string >>>>> strWireFormat, std::string strType) >>>>> { >>>>> Descriptor *descriptor = (Descriptor >>>>> *)m_Pool->FindMessageTypeByName(strType); >>>>> Message *message = >>>>> m_MessageFactory.GetPrototype(descriptor)->New(); >>>>> >>>>> if (descriptor == NULL) >>>>> { >>>>> char szError[256]; >>>>> >>>>> sprintf(szError, "Unknown data type %s!", strType.c_str()); >>>>> m_InternalPrinter.AddErrorMessage(szError); >>>>> >>>>> return false; >>>>> } >>>>> >>>>> bool bFlag = message->ParseFromString(strWireFormat); >>>>> >>>>> if (!bFlag) >>>>> { >>>>> m_InternalPrinter.AddErrorMessage("Encoding error!"); >>>>> } >>>>> else >>>>> { >>>>> m_Transformator.MorphToStructs(*message); >>>>> } >>>>> >>>>> return true; >>>>> } >>>>> >>>>> bool C_ProtoDecoder::ParseProtoFile(FileDescriptorProto *result) >>>>> { >>>>> int file_descriptor = open(m_strProtoFile.c_str(), O_RDONLY); >>>>> >>>>> if (file_descriptor == -1) >>>>> { >>>>> char szError[256]; >>>>> >>>>> sprintf(szError, "Invalid proto file \"%s\"!", >>>>> m_strProtoFile.c_str()); >>>>> m_InternalPrinter.AddErrorMessage(szError); >>>>> >>>>> return false; >>>>> } >>>>> >>>>> C_ParserErrorCollector errCollector; >>>>> >>>>> io::FileInputStream stream(file_descriptor); >>>>> stream.SetCloseOnDelete(true); >>>>> io::Tokenizer tokenizer(&stream, &errCollector); >>>>> result->set_name(m_strProtoFile); >>>>> >>>>> compiler::Parser parser; >>>>> >>>>> return parser.Parse(&tokenizer, result); >>>>> } >>>>> >>>>> I have used SimpleDescriptorDatabase since it looked like it has >>>>> already implemented and its covering what I will be requiring from it (or >>>>> at least I think it does) >>>>> >>>> If you use SimpleDescriptorDatabase, you need to add all .proto files >>>> into the database explicitly. For example, if you have 2 .proto files >>>> foo.proto and bar.proto, where bar.proto imports foo.proto, you need to >>>> add >>>> both foo.proto and bar.proto to SimpleDescriptorDatabase. If you already >>>> have a list of all proto files, using SimpleDescriptorDatabase should be >>>> fine. >>>> >>>> >>>>> >>>>> Everything is fine until I try to get descriptor by >>>>> FindMessageTypeByName from pool. It returns null. Am I committing some >>>>> steps? >>>>> >>>> I don't see anything obviously wrong in your code. >>>> >>>> >>>> >>>> >>>>> >>>>> >>>>> On Tuesday, 2 June 2015 19:49:53 UTC+2, Feng Xiao wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Jun 1, 2015 at 9:55 PM, Jan Kyjovský <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi again, >>>>>>> >>>>>>> As I have stated before I am done with decoding, but now I am >>>>>>> solving different type of problem. As I have mentioned before imports >>>>>>> may >>>>>>> prove problematic to our implementation. >>>>>>> >>>>>>> Let me describe use-case how it will be used. There will be >>>>>>> configuration stating for which data (message number, specific field) >>>>>>> will >>>>>>> be applied which proto file and structure (data type). Therefore there >>>>>>> is >>>>>>> no knowledge about other proto files up until now. In run time data are >>>>>>> fetched and shipped to decoder. In current implementation is specified >>>>>>> proto file loaded and specific type used for decoding. That much is >>>>>>> clear >>>>>>> and works. But if some parts of structures are imported. It will fail. >>>>>>> That >>>>>>> much is clear that using just one file descriptor will be not enough as >>>>>>> you >>>>>>> have mentioned earlier. >>>>>>> >>>>>>> That leads to your proposition (posted earlier) to use >>>>>>> DescriptorDatabase. I have looked on implementation of classes derived >>>>>>> from >>>>>>> DscriptorDatabase. I am more or less able to provide directory for >>>>>>> SourceTree, but that doesn't answer question about import. I mean since >>>>>>> I >>>>>>> am aware of only one proto file. Is import done automatically or other >>>>>>> files has to be also included manually into database? >>>>>>> >>>>>> >>>>>>> Another question I have is bout order of operations. You mentioned >>>>>>> that I have to first call FindFileByName() before >>>>>>> callingFindMessageTypeByName(). That may be problem since I am not >>>>>>> aware in >>>>>>> which proto file may be located. >>>>>>> >>>>>>> Also I have noticed in code note that proto files with types used in >>>>>>> other proto files have to be "loaded" first. So is there any way how to >>>>>>> ensure right order of loading? >>>>>>> >>>>>> Most of the work is done by DescriptorPool. Basically when you call >>>>>> FindFileByName() on a .proto file that is not yet loaded into the pool, >>>>>> DescriptorPool will call FindFileByName() on its underlying >>>>>> DescriptorDatabase to get the FileDescriptorProto for file. Then >>>>>> DescriptorPool will check the imports of this FileDescriptorPool. If it >>>>>> finds unsatisfied imports (i.e., files imported but not yet in the >>>>>> pool), >>>>>> it will issue FindFileByName() to the DescriptorDatabase again for these >>>>>> files. >>>>>> >>>>>> So basically DescriptorDatabase only needs to know how to load a >>>>>> single .proto file, while DescriptorPool will take care of the imports >>>>>> and >>>>>> ordering. >>>>>> >>>>>> Depending on what DescriptorDatabase you are using, you may or may >>>>>> not need to call FindFileByName() first. If the DescriptorDatabase has >>>>>> implement FindFileContainingSymbol, then calling >>>>>> DescriptorPool::FindMessageTypeByName() directly will be able to find >>>>>> the >>>>>> file using this method. If not (the case with >>>>>> SourceTreeDescriptorDatabase), FindFileByName() must be called first. As >>>>>> DescriptorPool will be able to find imports by itself, you don't need to >>>>>> call FindFileByName() for every file, but just the file that you >>>>>> actually >>>>>> need to use. >>>>>> >>>>>> >>>>>>> >>>>>>> I hope my question are not too strange but I my knowledge about >>>>>>> google protocol buffers are a bit limited. >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Protocol Buffers" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To post to this group, send email to [email protected]. >>>>>>> Visit this group at http://groups.google.com/group/protobuf. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Protocol Buffers" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at http://groups.google.com/group/protobuf. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Protocol Buffers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at http://groups.google.com/group/protobuf. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
