Hi Robert
Thank you for the answer and creating the Beam issue. The documentation has
been also fixed, very nice!
Do you know when the runner v2 will become the default runner for Java
pipelines? It is a matter of months or years?
BTW. For testing purposes I sucesfully deployed one of the Google
Reuven, thanks for the reply.
The input type is "KafkaRecord[Array[Byte], Array[Byte]]" and uses the
"KafkaRecordCoder.of(NullableCoder.of(ByteArrayCoder.of),
ByteArrayCoder.of)" coder.
I can't paste the code for DoFn due to company policy, but here's the
structure:
//
Thanks, I'll check it out.
I split inputAdaptor.adapt() into different DoFn for testing and it threw
the same exception for the new DoFn. So I guess it's because of
inputAdaptor.adapt().
On Fri, May 6, 2022 at 11:45 AM Reuven Lax wrote:
> I meant to say .equals() not compareTo.
>
> On Fri, May
Hi Reuven,
Do you mean we should have coder explicitly defined for all input types and
output types in chaining do fns? Do we also need to have comparedTo and equals
defined as well?
thanks again!
Siyu
> On May 6, 2022, at 12:23 PM, Reuven Lax wrote:
>
>
> Could be - I would check the impl
Hi Pablo,
No worries at all!
Was wondering if “add any dependencies” means to add a side container to the
dataflow runner? Like prometheus scraper?
If so, is it only working for dataflow runner v2? We have not upgraded to v2
yet so it might be hard to do that in v1.
Thanks so much!
Siyu
>
Hi Pablo,
Thanks so much for your explanation!
Also, for prom client in the do fn, do we need to initialize them in the setup
code? My concern is that if we have this initialization in each do fn, we might
overwhelm the memory. I think we can pass them as the parameter? Do you have a
recommis
sorry for typo “recommissioned way” should be “recommended way”
> On May 6, 2022, at 4:31 PM, Siyu Lin wrote:
>
>
> Hi Pablo,
>
> Thanks so much for your explanation!
>
> Also, for prom client in the do fn, do we need to initialize them in the
> setup code? My concern is that if we have t
Got it! Thanks so much, and will try it out!
-siyu
>
> On May 6, 2022, at 4:53 PM, Pablo Estrada wrote:
>
>
> Hi Siyu!
> Yeah, that's a good question!
>
> Depending on the properties of the client that you're using, you can create
> it as a class/static variable rather than an instance va
