DeferredDataFrame not saved as feather file. Problem in both DirectRunner and DataFlowRunner

2022-11-09 Thread Duarte Oliveira e Carmo
Hi all! *Context: I posted this question before on stack overflow, but hoping to get more answers here. * I'm trying to compute a sentence-transformers

Re: [CFP] In-person Beam meetups

2022-11-09 Thread Danielle Syse via user
Hi everyone, I hope you're all having a great start to your week! I want to remind everyone to join us for our few remaining in-person meetups of the year. Come network and discuss all things data processing over food and drinks! Please remember if you share this message to your social channels, y

Re: DeferredDataFrame not saved as feather file. Problem in both DirectRunner and DataFlowRunner

2022-11-09 Thread Brian Hulette via user
Hi Duarte, I commented on the Stack Overflow question. It looks like your to_dataframe and to_feather calls are outside of the Pipeline context, so they are being created _after_ the pipeline has already run. Hopefully moving them inside the Pipeline context will resolve the issue. Brian On Wed,