[Question] infer schema from proto in java?

2022-04-05 Thread Balogh , György
Hi, I'm using the java beam sdk. I have a PCollection where XYZ is a class generated from a proto2 file with protoc. Is it possible to infer schema and have a PCollection from this? Thank you, Gyorgy -- György Balogh CEO E gyorgy.bal...@ultinous.com M +36 30 270 8342 <+36%2030%20270%208342> A HU

Re: [Question] infer schema from proto in java?

2022-04-05 Thread Balogh , György
Hi Brian, Thank you it worked, now I have a schema of my PCollection. The next step is still not clear. I'd like to convert this to PCollection to be able to query with SQL. The doc has an example on how to assemble the row but I assume there should be a way to do this automatically. Thank you, Gyo

Breaking change for FileIO WriteDynamic in Beam 2.34?

2022-04-05 Thread Siyu Lin
Hi Beam community, We have a batch pipeline which does not run regularly. Recently we have upgraded to Beam 2.36 and this broke the FileIO WriteDynamic process. We are using Dataflow Runner, and the errors are like this when there are multiple workers: Error message from worker: java.lang.NoClas