As the very first step you should compile your schema and work with the generated classes: https://protobuf.dev/getting-started/javatutorial/#compiling-protocol-buffers
Don't mess with the reflection API if you can avoid it. On Wednesday, September 20, 2023 at 12:19:38 PM UTC+2 Joan Balagueró wrote: > Hi Florian, > > Not sure what I can do with this code. I only have a byte array > representing the above protobuf, I don't have any object or message. At > most I have a map with the list of the "UnknownFieldSet.Field" fields after > parsing the byte array (this is java code). > > I'm relatively new to protobuf, I read a lot on the Internet trying to > find a solution but I did not find anything. > > That's why I tried to traverse the map of fields and write them to the new > byte array. It's the only solution I could think of, but I'm doing > something wrong. > > Not sure if you can help me a bit more to solve this. > > Anyways thanks. > > Joan. > > > > > > On Wednesday, September 20, 2023 at 11:56:19 AM UTC+2 Florian Enner wrote: > >> Messages are serialized with a length delimiter, so changing the content >> produces a mismatch and invalid message. >> >> Your schema has no affected repeated fields, so appending a delta should >> work. I've never used the C# API, but here is some hopefully understandable >> pseudo code: >> >> var delta = Request.newInstance(); >> delta.getMutableMeta().getMutableCutOffTime() >> .setValue(value) >> .setScale(TimeSpanScale.MINMAX) >> byte[] output = append(unmodifiedInputBytes, delta.toByteArray()); >> >> If the server expects a length delimiter you'd need to update it to the >> new combined length. >> >> - Florian >> >> On Wednesday, September 20, 2023 at 11:03:46 AM UTC+2 Joan Balagueró >> wrote: >> >>> Hi Florian, >>> >>> Thanks for your quick response. I'm stuck on this. >>> >>> 1) It's not working. When I send the protobuf to the backend server >>> (it's not our api nor server) using the first method, I get a right >>> response. But using the second method I receive this error: >>> ProtoBuf.ProtoException: Invalid wire-type; this usually means you have >>> over-written a file without truncating or setting the length; see >>> https://stackoverflow.com/q/2152978/23354 >>> at ProtoBuf.ProtoReader.StartSubItem(ProtoReader reader) in >>> C:\code\protobuf-net\src\protobuf-net\ProtoReader.cs:line 637 >>> at ProtoBuf.ProtoReader.ReadTypedObject(Object value, Int32 key, >>> ProtoReader reader, Type type) in >>> C:\code\protobuf-net\src\protobuf-net\ProtoReader.cs:line 584 >>> at proto_40(Object , ProtoReader ) >>> at ProtoBuf.Meta.TypeModel.DeserializeCore(ProtoReader reader, Type >>> type, Object value, Boolean noAutoCreate) in >>> C:\code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 722 >>> at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, >>> Type type, SerializationContext context) in >>> C:\code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 599 >>> at >>> WebBeds.Connect.AspNetCore.Formatters.ProtobufInputFormatter.ReadRequestBodyAsync(InputFormatterContext >>> >>> context) >>> at >>> Microsoft.AspNetCore.Mvc.Formatters.InputFormatter.ReadAsync(InputFormatterContext >>> >>> context) >>> at >>> Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext >>> >>> bindingContext) >>> at >>> Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext >>> >>> actionContext, IModelBinder modelBinder, IValueProvider valueProvider, >>> ParameterDescriptor parameter, ModelMetadata metadata, Object value, Object >>> container) >>> at >>> Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<<CreateBinderDelegate>g__Bind|0>d.MoveNext() >>> --- End of stack trace from previous location --- >>> at >>> Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker >>> >>> invoker, Task lastTask, State next, Scope scope, Object state, Boolean >>> isCompleted) >>> at >>> Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextExceptionFilterAsync>g__Awaited|26_0(ResourceInvoker >>> >>> invoker, Task lastTask, State next, Scope scope, Object state, Boolean >>> isCompleted >>> >>> >>> 2) This is the proto: >>> >>> The request with the 'Meta' element: >>> >>> message Request { >>> Meta Meta = 1; >>> repeated int32 Hotels = 2 [packed = false]; >>> Country Market = 3; >>> repeated Room Rooms = 4; >>> .bcl.DateTime CheckIn = 5; >>> .bcl.DateTime CheckOut = 6; >>> OptionalCriteria OptionalCriteria = 7; >>> } >>> >>> The 'Meta' element that contains the 'CutOffTime' that we want to modify: >>> >>> message Meta { >>> int32 Client = 1; >>> int32 Brand = 2; >>> bool UseCache = 3; >>> .bcl.TimeSpan CutOffTime = 4; >>> bool B2C = 5; >>> Language Language = 6; >>> Currency Currency = 7; >>> bool IncludeProviderAuditData = 8; >>> SalesChannel SalesChannel = 9; >>> string AgentId = 10; >>> } >>> >>> The 'CutOffTime': >>> >>> message TimeSpan { >>> sint64 value = 1; // the size of the timespan (in units of the >>> selected scale) >>> TimeSpanScale scale = 2; // the scale of the timespan [default = DAYS] >>> enum TimeSpanScale { >>> DAYS = 0; >>> HOURS = 1; >>> MINUTES = 2; >>> SECONDS = 3; >>> MILLISECONDS = 4; >>> TICKS = 5; >>> >>> MINMAX = 15; // dubious >>> } >>> } >>> >>> Thanks, >>> >>> Joan. >>> >>> >>> >>> >>> >>> >>> On Wednesday, September 20, 2023 at 10:40:08 AM UTC+2 Florian Enner >>> wrote: >>> >>>> 1) A "varint" is a "variable length integer". When you replace a large >>>> number with a small one, it's entirely possible to lose some bytes and >>>> still be valid. You need to check the actual output. >>>> >>>> 2) Can you provide the proto definition of the field you want to >>>> modify? Scalar fields get set to the last encountered value, so the >>>> easiest >>>> option may be to copy the original bytes and append a delta containing the >>>> differences. >>>> >>>> >>>> >>>> >>>> On Wednesday, September 20, 2023 at 10:19:07 AM UTC+2 Joan Balagueró >>>> wrote: >>>> >>>>> Hello, >>>>> >>>>> I have a protobuf message like this into a byte array: >>>>> >>>>> 1: { // META element >>>>> 1: 2 >>>>> 2: 1 >>>>> 3: 1 >>>>> 4: { // CutOffTime element within META >>>>> 1: 10 >>>>> 2: 3 >>>>> } >>>>> 5: 1 >>>>> 6: 40 >>>>> } >>>>> 2: 9836 // HOTEL element >>>>> 3: 724 // MARKET element >>>>> >>>>> >>>>> We need to traverse this message and write it to a 'CodeOutputStream', >>>>> but changing the values of the 'cutoff' element to, for example to '4: { >>>>> 1: >>>>> 7, 2: 4 }'. I'm not able to do it, I need some help. >>>>> >>>>> A basic algorithm that writes the same protobuf to another byte array >>>>> but without changing anything works. Here I try the special case of >>>>> 'META' >>>>> (key = 1) that contains the 'cutoff' element. >>>>> >>>>> Map<Integer, UnknownFieldSet.Field> rootFields = >>>>> UnknownFieldSet.parseFrom(document).asMap(); >>>>> >>>>> for (Map.Entry<Integer, UnknownFieldSet.Field> entry : >>>>> rootFields.entrySet()) { >>>>> if (entry.getKey() == 1) { >>>>> ByteString bs = >>>>> entry.getValue().getLengthDelimitedList().get(0); >>>>> output.writeByteArray(1, bs.toByteArray()); >>>>> } >>>>> else { >>>>> entry.getValue().writeTo(entry.getKey(), output); >>>>> } >>>>> } >>>>> >>>>> >>>>> Now I try to go a bit further, trying to read the cuotff element, >>>>> change the values and rewrite them to the 'output'. And here is when I'm >>>>> not able to solve the problem. Below my try that does not work, it >>>>> generates a byte array >>>>> of 69 bytes instead of 73 (I'm losing 4 bytes): >>>>> >>>>> Map<Integer, UnknownFieldSet.Field> rootFields = >>>>> UnknownFieldSet.parseFrom(document).asMap(); >>>>> >>>>> for (Map.Entry<Integer, UnknownFieldSet.Field> entry : >>>>> rootFields.entrySet()) { >>>>> if (entry.getKey() == 1) { >>>>> ByteString bs = >>>>> entry.getValue().getLengthDelimitedList().get(0); >>>>> Map<Integer, UnknownFieldSet.Field> ufs = >>>>> UnknownFieldSet.parseFrom(bs).asMap(); >>>>> >>>>> for (Map.Entry<Integer, UnknownFieldSet.Field> >>>>> item : ufs.entrySet()) { >>>>> if (item.getKey() == 4) { >>>>> Map<Integer, UnknownFieldSet.Field> cutoff >>>>> = >>>>> UnknownFieldSet.parseFrom(item.getValue().getLengthDelimitedList().get(0)).asMap(); >>>>> cutoff.put(1, >>>>> UnknownFieldSet.Field.newBuilder().addVarint(7).build()).writeTo(1, >>>>> output); >>>>> cutoff.put(2, >>>>> UnknownFieldSet.Field.newBuilder().addVarint(4).build()).writeTo(1, >>>>> output); >>>>> } >>>>> else { >>>>> item.getValue().writeTo(item.getKey(), >>>>> output); >>>>> } >>>>> } >>>>> } >>>>> else { >>>>> entry.getValue().writeTo(entry.getKey(), output); >>>>> } >>>>> } >>>>> >>>>> Any help would be really appreciated. >>>>> >>>>> Thanks, >>>>> >>>>> Joan. >>>>> >>>> -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/fc811f91-f99c-418f-ab7d-84f5ac0b4993n%40googlegroups.com.
