I use protobuf-java-util:3.0.0-beta-2.
I create a file which contains a lot of Protobuf messages which are written
with Message#writeDelimitedTo(). The code is something like this:
Iterable<SomeMessage> messages = getHugeDataSet();
OutputStream os = new FileOutputStream("a_lot_of_messages.protobuf");
for (SomeMessage msg : messages) msg.writeDelimitedTo(os);
And I read those files with Builder#mergeDelimitedFrom(). Something like
this:
InputStream is = new FileInputStream("a_lot_of_messages.protobuf");
SomeMessage msg1 = SomeMessage.newBuilder().mergeDelimitedFrom(is).build();
SomeMessage msg2 = SomeMessage.newBuilder().mergeDelimitedFrom(is).build();
... // This is simplified - it's implemented as an Iterator in the real code
With that way, I can read the vast majority of files without any problems
but I sometimes get exceptions like this:
com.google.protobuf.InvalidProtocolBufferException: Protocol message had
invalid UTF-8. ...
The code which writes those files run on mobile devices which get power
failure sometimes. In such cases, chances of getting that exceptions are
high. So, apparently my code creates some malformed files on such
occasions. My code can read some parts of the file, but it can't read the
parts after the malformed chunk due to the error. Now I need to rescue and
read the data after the malformed chunk, but I couldn't find any way to do
that. So, I'd like to know the following:
1. Is there any way to rescue and read the parts which are written by the
code above after the malformed chunk?
2. If there is no way to do that, how can I improve my code so my app can
cope with such problems? Is there any best practice which is power failure
tolerant?
Thanks,
Kohei
Full exception stacktrace is something like this:
com.google.protobuf.InvalidProtocolBufferException: Protocol message had
invalid UTF-8.
at
com.google.protobuf.InvalidProtocolBufferException.invalidUtf8(InvalidProtocolBufferException.java:120)
~[protobuf-java-3.0.0-beta-2.jar:na]
at
com.google.protobuf.CodedInputStream.readStringRequireUtf8(CodedInputStream.java:410)
~[protobuf-java-3.0.0-beta-2.jar:na]
at com.example.Model$SomeData.<init>(Model.java:14775)
~[my-app-1.0-SNAPSHOT.jar:na]
at com.example.Model$SomeData.<init>(Model.java:14717)
~[my-app-1.0-SNAPSHOT.jar:na]
at com.example.Model$SomeData$1.parsePartialFrom(Model.java:18240)
~[my-app-1.0-SNAPSHOT.jar:na]
at com.example.Model$SomeData$1.parsePartialFrom(Model.java:18234)
~[my-app-1.0-SNAPSHOT.jar:na]
at
com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:495)
~[protobuf-java-3.0.0-beta-2.jar:na]
at com.example.Model$SomeMessage.<init>(Model.java:27250)
~[my-app-1.0-SNAPSHOT.jar:na]
at com.example.Model$SomeMessage.<init>(Model.java:27197)
~[my-app-1.0-SNAPSHOT.jar:na]
at com.example.Model$SomeMessage$1.parsePartialFrom(Model.java:28678)
~[my-app-1.0-SNAPSHOT.jar:na]
at com.example.Model$SomeMessage$1.parsePartialFrom(Model.java:28672)
~[my-app-1.0-SNAPSHOT.jar:na]
at com.example.Model$SomeMessage$Builder.mergeFrom(Model.java:27802)
~[my-app-1.0-SNAPSHOT.jar:na]
at com.example.Model$SomeMessage$Builder.mergeFrom(Model.java:27653)
~[my-app-1.0-SNAPSHOT.jar:na]
at
com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:235)
~[protobuf-java-3.0.0-beta-2.jar:na]
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:516)
~[protobuf-java-3.0.0-beta-2.jar:na]
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:290)
~[protobuf-java-3.0.0-beta-2.jar:na]
at
com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:305)
~[protobuf-java-3.0.0-beta-2.jar:na]
at
com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:530)
~[protobuf-java-3.0.0-beta-2.jar:na]
at
com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:311)
~[protobuf-java-3.0.0-beta-2.jar:na]
at
com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:522)
~[protobuf-java-3.0.0-beta-2.jar:na]
at com.example.MyUtils.read(MyUtils.java:54) [my-app-1.0-SNAPSHOT.jar:na]
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
