Figured out this is because the protobuf version is different between
hadoop (2.5.0) and the one I used (3.6.1) to generate the java files.
Once I downgrade the protoc to re-generate the java files, the issue
resolved.
Thanks
David
On Wednesday, September 12, 2018 at 3:28:49 PM UTC-7, David Yu wrote:
>
> Hi,
> I am working on the project to count the record in the protobuf file
> but I get the exception
>
> Exception in thread "main" java.lang.NoSuchMethodError:
> com.google.protobuf.CodedInputStream.shouldDiscardUnknownFields()Z
>
> at
> com.google.protobuf.GeneratedMessageV3.parseUnknownField(GeneratedMessageV3.java:290)
>
>
> This only happens on some of the protobuf files, not all of them has the
> issue.
>
> Same code is working for files with different schema.
>
>
> My protobuf file is gzipped.
>
>
> //Here is the code
>
> Configuration conf = new Configuration();
>
> FileSystem fs = FileSystem.get(conf);
>
> Path path = new Path(<HDFS path to file>);
>
> InputStream input = new GZIPInputStream(fs.open(path));
>
> Message m;
>
> while ((m = defaultMsg.getParserForType().parseDelimitedFrom(input)) !=
> null) {
>
> recordCount++;
>
> }
>
>
> Does anyone have idea?
>
>
> Thanks
>
> David
>
>
>
>
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
