Re: Registering Protobuf schema

2020-08-19 Thread Luke Cwik
I posted a separate >>> message on the dev channel covering this. >>> >>> >>> >>> Kind regards, >>> >>> >>> >>> Rob >>> >>> >>> >>> *From:* Brian Hulette [mailto:bhule...@google.com]

Re: Registering Protobuf schema

2020-08-19 Thread Brian Hulette
t;> >> Kind regards, >> >> >> >> Rob >> >> >> >> *From:* Brian Hulette [mailto:bhule...@google.com] >> *Sent:* 18 August 2020 20:50 >> *To:* user >> *Subject:* Re: Registering Protobuf schema >> >> >> &g

Re: Registering Protobuf schema

2020-08-19 Thread Luke Cwik
d by > BigQueryIO.Write or allow for optional fields) but I posted a separate > message on the dev channel covering this. > > > > Kind regards, > > > > Rob > > > > *From:* Brian Hulette [mailto:bhule...@google.com] &

RE: Registering Protobuf schema

2020-08-19 Thread Robert.Butcher
optional fields) but I posted a separate message on the dev channel covering this. Kind regards, Rob From: Brian Hulette [mailto:bhule...@google.com] Sent: 18 August 2020 20:50 To: user Subject: Re: Registering Protobuf schema * "This is an external

Re: Registering Protobuf schema

2020-08-18 Thread Brian Hulette
Hi Robert, Sorry for the late reply on this. I think you should be able to do this by registering it in your pipeline's SchemaRegistry manually, like so: Pipeline p; p.getSchemaRegistry().registerSchemaProvider(Fx.class, ProtoMessageSchema.class); Of course this isn't quite as nice as just ad

Re: Registering Protobuf schema

2020-07-12 Thread Kaymak, Tobias
This sounds like it is related to the problem I'm trying to solve. (In my case having a Java POJO containing a protobuf backed-class and trying to generate a Beam Schema from it.) I would be very interested to a solution to this as well :) On Tue, Jul 7, 2020 at 2:22 PM wrote: > Hi All, > > > >

Registering Protobuf schema

2020-07-07 Thread Robert.Butcher
Hi All, I have a BEAM pipeline where I am reading data from some parquet files and converting them into a different format based on protobuf generated classes. I wish to associate a schema (derived from the protobuf classes) for my PCollections. What is the appropriate way to do this with prot