Good question - I know it will be enforced at runtime (I think you'll get a ClassCastException if things don't match) but I'd have to check to see if we enforce it at graph-submission time.
If we don't have this validation in place, adding it would be an improvement. On Mon, Sep 18, 2023 at 3:04 PM Varun Golani <vg.blazingde...@gmail.com> wrote: > I've been looking at the following session from the 2019 Beam Summit > covering Schema-aware PCollections (18:00 - > https://www.youtube.com/watch?v=aRIZXtQiCHw). The specific part of the > video mentions that for fields accessed via the @FieldAccess annotation, > Beam checks for both the existence of the field AND matches the type during > pipeline construction time. > > When trying this out on my application locally, I do see that Beam flags > up incorrectly specified fields that don't exist in the PCollection's > schema. However it seems as if it doesn't enforce/check the types of the > parameter variable match the types specified in the Schema. > > For instance if I have a PCollection containing an attribute "field1" with > type String, when referring to this attribute in my DoFn using > *@FieldAccess("field1") > Double field1 *it does NOT flag this String/Double type mismatch during > pipeline construction similar to how it would do if I refer to an > attribute that doesn't exist in the schema. > > Is this expected behaviour? If so, would there be any way in which I could > write a custom validator which strongly enforces @FieldAccess parameter > types against the types in the corresponding schema? >
