Hi Tobias, You should be able to annotate the EnrichedArticle class with an4 @DefaultSchema annotation and Beam will infer a schema for it. You would need to make some tweaks to the class though to be compatible with the built-in schema providers: you could make the members public and use JavaFieldSchema, or add getters/setters and use the JavaBeanSchema, or make it into an AutoValue and use AutoValueSchema.
Once you do that you should be able to convert a PCollection<EnrichedArticle> to a PCollection<Row> with Convert.toRows [1]. Brian [1] https://beam.apache.org/releases/javadoc/2.22.0/org/apache/beam/sdk/schemas/transforms/Convert.html#toRows-- On Fri, Jun 26, 2020 at 3:19 AM Kaymak, Tobias <[email protected]> wrote: > I have the following class definition: > > public class EnrichedArticle implements Serializable { > > // ArticleEnvelope is generated via Protobuf > private ArticleProto.ArticleEnvelope article; > // Asset is a Java POJO > private List<Asset> assets; > > public EnrichedArticle(ArticleProto.ArticleEnvelope article, List<Asset> > assets) { > this.article = article; > this.assets = assets; > } > } > > I am trying to generate a SerializableFunction<EnrichedArticle, Row> and > a Schema for it so that I can pass it easily to my BigQueryIO at the end of > my pipeline. Transforming the article to a Row object is straightforward: > > First I get the toRow() function for it via the helper: > > new ProtoMessageSchema().toRowFunction(TypeDescriptor.of( > ArticleProto.ArticleEnvelope.class)); > > Then I just apply that function to the article field. > However I don't know how I can manually transform my list of assets (a > simple Java POJO annotated with: @DefaultSchema(JavaFieldSchema.class) > > in my EnrichedArticle container/composition class. What's the recommended > way of doing this? > > > >
