On Mon, Jun 8, 2015 at 10:23 PM, Jan Kyjovský <[email protected]>
wrote:
> I have remade code so I now add into descriptor database all protofiles
> specified by directory. There is but little problem with error handling.
> From nature how I am handling errors it seems that message is possible to
> decode but there is one error in error collector which is bothering me:
>
> 1:1: ERROR: Multiple package definitions.
>
> *My protos are defined as follows:*
> *addressbook_fragmented.proto:*
> import "phone.proto";
>
> package tutorial;
>
> option java_package = "com.example.tutorial";
> option java_outer_classname = "AddressBookProtos";
>
> message Person {
> required string name = 1;
> required int32 id = 2; // Unique ID number for this person.
> optional string email = 3;
>
> // enum PhoneType {
> // MOBILE = 0;
> // HOME = 1;
> // WORK = 2;
> // }
>
> message PhoneNumber {
> required string number = 1;
> optional PhoneType type = 2 [default = HOME];
> }
>
> repeated PhoneNumber phone = 4;
> optional int32 age = 5;
> }
>
> // Our address book file is just one of these.
> message AddressBook {
> repeated Person person = 1;
> }
>
> *phone.proto:*
> package tutorial;
>
> enum PhoneType {
> MOBILE = 0;
> HOME = 1;
> WORK = 2;
> }
>
> Even though there is error in error collector I can ignore it and decode
> message successfully. Yet if I try to remove this package info from
> phone.proto itis not detecting any error but FindMessageTypeByName fails to
> find data type. I am suspecting since not all references for person data
> type I am asking for are resolved it fails.
>
> So I am confused how exactly should be these package keywords be used?
>
I don't see a problem in your use of "package". You could try to run protoc
on these proto files and see if it reports any error. I suspect it's not
the .proto file's problem.
>
> On Wednesday, 3 June 2015 19:05:59 UTC+2, Feng Xiao wrote:
>
>>
>>
>> On Wed, Jun 3, 2015 at 12:40 AM, Jan Kyjovský <[email protected]>
>> wrote:
>>
>>> HI,
>>>
>>> I somehow understand what you mean but still I lack in experience with
>>> it. I tried to switch to descriptor database but still I got some problems.
>>>
>>> C_ProtoDecoder::C_ProtoDecoder(std::string strFile)
>>> {
>>> m_strProtoFile = strFile;
>>>
>>> // Build the descriptor pool
>>> m_bConstructed = ParseProtoFile(&m_ProtoFileDescr);
>>>
>>> if (m_bConstructed)
>>> {
>>> m_ProtoDescrDatabase.Add(m_ProtoFileDescr);
>>> m_Pool = new DescriptorPool(&m_ProtoDescrDatabase);
>>> }
>>> }
>>>
>>> C_ProtoDecoder::~C_ProtoDecoder()
>>> {
>>> delete m_Pool;
>>> }
>>>
>>> bool C_ProtoDecoder::DecodeDataAsTypeFromWireFormat(std::string
>>> strWireFormat, std::string strType)
>>> {
>>> Descriptor *descriptor = (Descriptor
>>> *)m_Pool->FindMessageTypeByName(strType);
>>> Message *message = m_MessageFactory.GetPrototype(descriptor)->New();
>>>
>>> if (descriptor == NULL)
>>> {
>>> char szError[256];
>>>
>>> sprintf(szError, "Unknown data type %s!", strType.c_str());
>>> m_InternalPrinter.AddErrorMessage(szError);
>>>
>>> return false;
>>> }
>>>
>>> bool bFlag = message->ParseFromString(strWireFormat);
>>>
>>> if (!bFlag)
>>> {
>>> m_InternalPrinter.AddErrorMessage("Encoding error!");
>>> }
>>> else
>>> {
>>> m_Transformator.MorphToStructs(*message);
>>> }
>>>
>>> return true;
>>> }
>>>
>>> bool C_ProtoDecoder::ParseProtoFile(FileDescriptorProto *result)
>>> {
>>> int file_descriptor = open(m_strProtoFile.c_str(), O_RDONLY);
>>>
>>> if (file_descriptor == -1)
>>> {
>>> char szError[256];
>>>
>>> sprintf(szError, "Invalid proto file \"%s\"!",
>>> m_strProtoFile.c_str());
>>> m_InternalPrinter.AddErrorMessage(szError);
>>>
>>> return false;
>>> }
>>>
>>> C_ParserErrorCollector errCollector;
>>>
>>> io::FileInputStream stream(file_descriptor);
>>> stream.SetCloseOnDelete(true);
>>> io::Tokenizer tokenizer(&stream, &errCollector);
>>> result->set_name(m_strProtoFile);
>>>
>>> compiler::Parser parser;
>>>
>>> return parser.Parse(&tokenizer, result);
>>> }
>>>
>>> I have used SimpleDescriptorDatabase since it looked like it has already
>>> implemented and its covering what I will be requiring from it (or at least
>>> I think it does)
>>>
>> If you use SimpleDescriptorDatabase, you need to add all .proto files
>> into the database explicitly. For example, if you have 2 .proto files
>> foo.proto and bar.proto, where bar.proto imports foo.proto, you need to add
>> both foo.proto and bar.proto to SimpleDescriptorDatabase. If you already
>> have a list of all proto files, using SimpleDescriptorDatabase should be
>> fine.
>>
>>
>>>
>>> Everything is fine until I try to get descriptor by
>>> FindMessageTypeByName from pool. It returns null. Am I committing some
>>> steps?
>>>
>> I don't see anything obviously wrong in your code.
>>
>>
>>
>>
>>>
>>>
>>> On Tuesday, 2 June 2015 19:49:53 UTC+2, Feng Xiao wrote:
>>>>
>>>>
>>>>
>>>> On Mon, Jun 1, 2015 at 9:55 PM, Jan Kyjovský <[email protected]>
>>>> wrote:
>>>>
>>>>> Hi again,
>>>>>
>>>>> As I have stated before I am done with decoding, but now I am solving
>>>>> different type of problem. As I have mentioned before imports may prove
>>>>> problematic to our implementation.
>>>>>
>>>>> Let me describe use-case how it will be used. There will be
>>>>> configuration stating for which data (message number, specific field) will
>>>>> be applied which proto file and structure (data type). Therefore there is
>>>>> no knowledge about other proto files up until now. In run time data are
>>>>> fetched and shipped to decoder. In current implementation is specified
>>>>> proto file loaded and specific type used for decoding. That much is clear
>>>>> and works. But if some parts of structures are imported. It will fail.
>>>>> That
>>>>> much is clear that using just one file descriptor will be not enough as
>>>>> you
>>>>> have mentioned earlier.
>>>>>
>>>>> That leads to your proposition (posted earlier) to use
>>>>> DescriptorDatabase. I have looked on implementation of classes derived
>>>>> from
>>>>> DscriptorDatabase. I am more or less able to provide directory for
>>>>> SourceTree, but that doesn't answer question about import. I mean since I
>>>>> am aware of only one proto file. Is import done automatically or other
>>>>> files has to be also included manually into database?
>>>>>
>>>>
>>>>> Another question I have is bout order of operations. You mentioned
>>>>> that I have to first call FindFileByName() before
>>>>> callingFindMessageTypeByName(). That may be problem since I am not aware
>>>>> in
>>>>> which proto file may be located.
>>>>>
>>>>> Also I have noticed in code note that proto files with types used in
>>>>> other proto files have to be "loaded" first. So is there any way how to
>>>>> ensure right order of loading?
>>>>>
>>>> Most of the work is done by DescriptorPool. Basically when you call
>>>> FindFileByName() on a .proto file that is not yet loaded into the pool,
>>>> DescriptorPool will call FindFileByName() on its underlying
>>>> DescriptorDatabase to get the FileDescriptorProto for file. Then
>>>> DescriptorPool will check the imports of this FileDescriptorPool. If it
>>>> finds unsatisfied imports (i.e., files imported but not yet in the pool),
>>>> it will issue FindFileByName() to the DescriptorDatabase again for these
>>>> files.
>>>>
>>>> So basically DescriptorDatabase only needs to know how to load a single
>>>> .proto file, while DescriptorPool will take care of the imports and
>>>> ordering.
>>>>
>>>> Depending on what DescriptorDatabase you are using, you may or may not
>>>> need to call FindFileByName() first. If the DescriptorDatabase has
>>>> implement FindFileContainingSymbol, then calling
>>>> DescriptorPool::FindMessageTypeByName() directly will be able to find the
>>>> file using this method. If not (the case with
>>>> SourceTreeDescriptorDatabase), FindFileByName() must be called first. As
>>>> DescriptorPool will be able to find imports by itself, you don't need to
>>>> call FindFileByName() for every file, but just the file that you actually
>>>> need to use.
>>>>
>>>>
>>>>>
>>>>> I hope my question are not too strange but I my knowledge about google
>>>>> protocol buffers are a bit limited.
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Protocol Buffers" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> To post to this group, send email to [email protected].
>>>>> Visit this group at http://groups.google.com/group/protobuf.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Protocol Buffers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To post to this group, send email to [email protected].
>>> Visit this group at http://groups.google.com/group/protobuf.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at http://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
