Hi Florian, Not sure what I can do with this code. I only have a byte array representing the above protobuf, I don't have any object or message. At most I have a map with the list of the "UnknownFieldSet.Field" fields after parsing the byte array (this is java code).
I'm relatively new to protobuf, I read a lot on the Internet trying to find a solution but I did not find anything. That's why I tried to traverse the map of fields and write them to the new byte array. It's the only solution I could think of, but I'm doing something wrong. Not sure if you can help me a bit more to solve this. Anyways thanks. Joan. On Wednesday, September 20, 2023 at 11:56:19 AM UTC+2 Florian Enner wrote: > Messages are serialized with a length delimiter, so changing the content > produces a mismatch and invalid message. > > Your schema has no affected repeated fields, so appending a delta should > work. I've never used the C# API, but here is some hopefully understandable > pseudo code: > > var delta = Request.newInstance(); > delta.getMutableMeta().getMutableCutOffTime() > .setValue(value) > .setScale(TimeSpanScale.MINMAX) > byte[] output = append(unmodifiedInputBytes, delta.toByteArray()); > > If the server expects a length delimiter you'd need to update it to the > new combined length. > > - Florian > > On Wednesday, September 20, 2023 at 11:03:46 AM UTC+2 Joan Balagueró wrote: > >> Hi Florian, >> >> Thanks for your quick response. I'm stuck on this. >> >> 1) It's not working. When I send the protobuf to the backend server (it's >> not our api nor server) using the first method, I get a right response. But >> using the second method I receive this error: >> ProtoBuf.ProtoException: Invalid wire-type; this usually means you have >> over-written a file without truncating or setting the length; see >> https://stackoverflow.com/q/2152978/23354 >> at ProtoBuf.ProtoReader.StartSubItem(ProtoReader reader) in >> C:\code\protobuf-net\src\protobuf-net\ProtoReader.cs:line 637 >> at ProtoBuf.ProtoReader.ReadTypedObject(Object value, Int32 key, >> ProtoReader reader, Type type) in >> C:\code\protobuf-net\src\protobuf-net\ProtoReader.cs:line 584 >> at proto_40(Object , ProtoReader ) >> at ProtoBuf.Meta.TypeModel.DeserializeCore(ProtoReader reader, Type >> type, Object value, Boolean noAutoCreate) in >> C:\code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 722 >> at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, >> Type type, SerializationContext context) in >> C:\code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 599 >> at >> WebBeds.Connect.AspNetCore.Formatters.ProtobufInputFormatter.ReadRequestBodyAsync(InputFormatterContext >> >> context) >> at >> Microsoft.AspNetCore.Mvc.Formatters.InputFormatter.ReadAsync(InputFormatterContext >> >> context) >> at >> Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext >> >> bindingContext) >> at >> Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext >> >> actionContext, IModelBinder modelBinder, IValueProvider valueProvider, >> ParameterDescriptor parameter, ModelMetadata metadata, Object value, Object >> container) >> at >> Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<<CreateBinderDelegate>g__Bind|0>d.MoveNext() >> --- End of stack trace from previous location --- >> at >> Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker >> >> invoker, Task lastTask, State next, Scope scope, Object state, Boolean >> isCompleted) >> at >> Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextExceptionFilterAsync>g__Awaited|26_0(ResourceInvoker >> >> invoker, Task lastTask, State next, Scope scope, Object state, Boolean >> isCompleted >> >> >> 2) This is the proto: >> >> The request with the 'Meta' element: >> >> message Request { >> Meta Meta = 1; >> repeated int32 Hotels = 2 [packed = false]; >> Country Market = 3; >> repeated Room Rooms = 4; >> .bcl.DateTime CheckIn = 5; >> .bcl.DateTime CheckOut = 6; >> OptionalCriteria OptionalCriteria = 7; >> } >> >> The 'Meta' element that contains the 'CutOffTime' that we want to modify: >> >> message Meta { >> int32 Client = 1; >> int32 Brand = 2; >> bool UseCache = 3; >> .bcl.TimeSpan CutOffTime = 4; >> bool B2C = 5; >> Language Language = 6; >> Currency Currency = 7; >> bool IncludeProviderAuditData = 8; >> SalesChannel SalesChannel = 9; >> string AgentId = 10; >> } >> >> The 'CutOffTime': >> >> message TimeSpan { >> sint64 value = 1; // the size of the timespan (in units of the selected >> scale) >> TimeSpanScale scale = 2; // the scale of the timespan [default = DAYS] >> enum TimeSpanScale { >> DAYS = 0; >> HOURS = 1; >> MINUTES = 2; >> SECONDS = 3; >> MILLISECONDS = 4; >> TICKS = 5; >> >> MINMAX = 15; // dubious >> } >> } >> >> Thanks, >> >> Joan. >> >> >> >> >> >> >> On Wednesday, September 20, 2023 at 10:40:08 AM UTC+2 Florian Enner wrote: >> >>> 1) A "varint" is a "variable length integer". When you replace a large >>> number with a small one, it's entirely possible to lose some bytes and >>> still be valid. You need to check the actual output. >>> >>> 2) Can you provide the proto definition of the field you want to modify? >>> Scalar fields get set to the last encountered value, so the easiest option >>> may be to copy the original bytes and append a delta containing the >>> differences. >>> >>> >>> >>> >>> On Wednesday, September 20, 2023 at 10:19:07 AM UTC+2 Joan Balagueró >>> wrote: >>> >>>> Hello, >>>> >>>> I have a protobuf message like this into a byte array: >>>> >>>> 1: { // META element >>>> 1: 2 >>>> 2: 1 >>>> 3: 1 >>>> 4: { // CutOffTime element within META >>>> 1: 10 >>>> 2: 3 >>>> } >>>> 5: 1 >>>> 6: 40 >>>> } >>>> 2: 9836 // HOTEL element >>>> 3: 724 // MARKET element >>>> >>>> >>>> We need to traverse this message and write it to a 'CodeOutputStream', >>>> but changing the values of the 'cutoff' element to, for example to '4: { >>>> 1: >>>> 7, 2: 4 }'. I'm not able to do it, I need some help. >>>> >>>> A basic algorithm that writes the same protobuf to another byte array >>>> but without changing anything works. Here I try the special case of 'META' >>>> (key = 1) that contains the 'cutoff' element. >>>> >>>> Map<Integer, UnknownFieldSet.Field> rootFields = >>>> UnknownFieldSet.parseFrom(document).asMap(); >>>> >>>> for (Map.Entry<Integer, UnknownFieldSet.Field> entry : >>>> rootFields.entrySet()) { >>>> if (entry.getKey() == 1) { >>>> ByteString bs = >>>> entry.getValue().getLengthDelimitedList().get(0); >>>> output.writeByteArray(1, bs.toByteArray()); >>>> } >>>> else { >>>> entry.getValue().writeTo(entry.getKey(), output); >>>> } >>>> } >>>> >>>> >>>> Now I try to go a bit further, trying to read the cuotff element, >>>> change the values and rewrite them to the 'output'. And here is when I'm >>>> not able to solve the problem. Below my try that does not work, it >>>> generates a byte array >>>> of 69 bytes instead of 73 (I'm losing 4 bytes): >>>> >>>> Map<Integer, UnknownFieldSet.Field> rootFields = >>>> UnknownFieldSet.parseFrom(document).asMap(); >>>> >>>> for (Map.Entry<Integer, UnknownFieldSet.Field> entry : >>>> rootFields.entrySet()) { >>>> if (entry.getKey() == 1) { >>>> ByteString bs = >>>> entry.getValue().getLengthDelimitedList().get(0); >>>> Map<Integer, UnknownFieldSet.Field> ufs = >>>> UnknownFieldSet.parseFrom(bs).asMap(); >>>> >>>> for (Map.Entry<Integer, UnknownFieldSet.Field> item >>>> : ufs.entrySet()) { >>>> if (item.getKey() == 4) { >>>> Map<Integer, UnknownFieldSet.Field> cutoff >>>> = >>>> UnknownFieldSet.parseFrom(item.getValue().getLengthDelimitedList().get(0)).asMap(); >>>> cutoff.put(1, >>>> UnknownFieldSet.Field.newBuilder().addVarint(7).build()).writeTo(1, >>>> output); >>>> cutoff.put(2, >>>> UnknownFieldSet.Field.newBuilder().addVarint(4).build()).writeTo(1, >>>> output); >>>> } >>>> else { >>>> item.getValue().writeTo(item.getKey(), >>>> output); >>>> } >>>> } >>>> } >>>> else { >>>> entry.getValue().writeTo(entry.getKey(), output); >>>> } >>>> } >>>> >>>> Any help would be really appreciated. >>>> >>>> Thanks, >>>> >>>> Joan. >>>> >>> -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/f4b4a46f-8029-4f79-9b7a-1b8504686757n%40googlegroups.com.
