Ya fair enough, makes sense. I’ll reach out to GCP. Thanks Luke!
- Evan
On Fri, Jul 8, 2022 at 11:24 Luke Cwik wrote:
> I was suggesting GCP support mainly because I don't think you want to
> share the 2.36 and 2.40 version of your job file publicly as someone
> familiar with the layout and for
I was suggesting GCP support mainly because I don't think you want to share
the 2.36 and 2.40 version of your job file publicly as someone familiar
with the layout and format may spot a meaningful difference.
Also, if it turns out that there is no meaningful difference between the
two then the int
Thanks for your response Luke :-)
Updating in 2.36.0 works as expected, but as you alluded to I'm attempting
to update to the latest SDK; in this case there are no code changes in the
user code, only the SDK version. Is GCP support the only tool when it
comes to deciphering the steps added by Dat
Does doing a pipeline update in 2.36 work or do you want to do an update to
get the latest version?
Feel free to share the job files with GCP support. It could be something
internal but the coders for ephemeral steps that Dataflow adds are based
upon existing coders within the graph.
On Tue, Jul
+dev@
Reviving this thread as it has hit me again on Dataflow. I am trying to
upgrade an active streaming pipeline from 2.36.0 to 2.40.0. Originally, I
received an error that the step "Flatten.pCollections" was missing from the
new job graph. I knew from the code that that wasn't true, so I dum
Thanks for the ideas Luke. I checked out the json graphs as per your
recommendation (thanks for that, was previously unaware), and the
"output_info" was identical for both the running pipeline and the pipeline
I was hoping to update it with. I ended up opting to just drain and submit
the updated p
