> Auto is hard, because it would involve
> querying the runner before pipeline construction, and we may not even
> know what the runner is at this point
At the point where pipeline construction will start, you should have access
to the pipeline arguments and be able to determine the runner. What s
patibility checks
> around between SDKs and Runners though. But in some cases it would be
> easier for the SDK, eg to ensure VersionA of a transform is used vs
> VersionB, based on the existing transforma used in the job being updated.
>
>
> On Fri, Oct 27, 2023, 7:50 AM Kelle
Yes, scio has a `materialize` which under the hood encodes each element
using its coder and serializes the sequence of byte arrays in a thin avro
wrapper. Some spotify-internal teams use this to pass data between a
sequence of jobs but it ends up just being a shortcut when you don't want
to permane
