Re: How to run WordCount/Starter at Flink cluster?

Thanks for the clarification, Pavel. Does it make sense adjust the code to overcome this inconvenience? I can try to work on draft-PR. On Mon, May 8, 2023 at 8:30 AM Pavel Solomin wrote: > Hello! > > What Beam aims to provide is keeping the same code of the pipeline itself > when you switch from

Re: Question about BeamSqlSeekableTable

Does anyone know this? Thanks On Tue, Apr 25, 2023 at 9:51 PM Jeff Zhang wrote: > > Hi all, > > I am a little confused about the implementation of BeamSqlSeekableTable, > it looks like the join condition is implemented in method > BeamSqlSeekableTable#seekRow, so does that mean whatever the join

How to identify what objects in your code have to be serialized

I am trying to create a pipeline where I query paginated data from some external service via a client and join them into a PCollectionList and flatten it to get the final collection of data items. The data class is encoded using a ProtoCoder Here is my code: ---

Re: How to identify what objects in your code have to be serialized

Hi Sachin, Can you post the error that you are getting? It should provide some additional information / path. If you are trying to use DataClient on the pipeline (inside a PTransform, DoFn, etc), you would have to initialize that client inside the DoFn itself (e.g., @Setup

Re: How to identify what objects in your code have to be serialized

I am not using DataClient inside the pipeline but passing it externally to a static method and creating a PCollection using pipeline.apply transformations (as shown in my previous email). For now I made my DataClient serializable to just progress and later on was planning to initialize that client