Recommended way to set coder for JdbcIO with Apache Beam

2023-03-23 Thread Juan Cuzmar
Hello all, I hope this message finds you well. I am currently working with Apache Beam's JdbcIO and need some guidance regarding setting a coder for the input data without resorting to the deprecated withCoder method. I've been trying to resolve this issue and would appreciate any insights or r

Re: Recommended way to set coder for JdbcIO with Apache Beam

2023-03-23 Thread Juan Cuzmar
; POJO? > > — > Alexey > > > On 23 Mar 2023, at 08:16, Juan Cuzmar jcuz...@protonmail.com wrote: > > > > Hello all, > > > > I hope this message finds you well. I am currently working with Apache > > Beam's JdbcIO and need some guidance regarding s

Re: Recommended way to set coder for JdbcIO with Apache Beam

2023-03-23 Thread Juan Cuzmar
an be inferred from the provided RowMapper. Juan Cuzmar. --- Original Message --- On Thursday, March 23rd, 2023 at 2:17 PM, Alexey Romanenko wrote: > Well, perhaps it’s a bug (I’ll try to check it on my side later). > > Can you try to annotate InboundData class with &g

Re: Recommended way to set coder for JdbcIO with Apache Beam

2023-03-27 Thread Juan Cuzmar
Hello! i created this example: https://github.com/j1cs/coder-error-beam and the issue you told me: https://github.com/apache/beam/issues/26003 Juan Cuzmar. --- Original Message --- On Monday, March 27th, 2023 at 10:58 AM, Alexey Romanenko wrote: > Hmm, it worked fine for me o

Apache Beam pipeline stuck indefinitely using Wait.on transform with JdbcIO

2023-04-21 Thread Juan Cuzmar
ind the sample code at: https://github.com/j1cs/app-beam Thank you for your help and support. Best regards, Juan Cuzmar.

Re: Apache Beam pipeline stuck indefinitely using Wait.on transform with JdbcIO

2023-04-22 Thread Juan Cuzmar
s > > and identify where it's getting stuck, but I haven't been able to pinpoint > > the issue. I've searched for solutions online, but none of them provided a > > successful resolution for my problem. > > > > Can anyone provide any insights or suggestions on how to debug and resolve > > this issue involving Wait.on and JdbcIO in my Apache Beam pipeline? > > > > You can find the sample code at: https://github.com/j1cs/app-beam > > > > Thank you for your help and support. > > > > Best regards, > > > > Juan Cuzmar.

Re: Apache Beam pipeline stuck indefinitely using Wait.on transform with JdbcIO

2023-04-22 Thread Juan Cuzmar
window. Wait.on > waits for the end of the current window before triggering. The default Window > is the GlobalWindow, so as written Wait.on will wait for the end of time (or > until you drain the pipeline, which will also trigger the GlobalWindow). > Try adding a 1-minute fixed windo

Re: Apache Beam pipeline stuck indefinitely using Wait.on transform with JdbcIO

2023-04-22 Thread Juan Cuzmar
I'm developing with direct runner. but should go to dataflow when deployed. Original Message On Apr 22, 2023, 13:13, Reuven Lax via user wrote: > What runner are you using to run this pipeline? > > On Sat, Apr 22, 2023 at 9:47 AM Juan Cuzmar wrote: &g

Re: Apache Beam pipeline stuck indefinitely using Wait.on transform with JdbcIO

2023-04-22 Thread Juan Cuzmar
ntation of the pubsub source - especially the watermark. For a > direct-runner test, I recommend using TestStream (which allows you to advance > the watermark manually, so you can test windowing). > > On Sat, Apr 22, 2023 at 10:28 AM Juan Cuzmar wrote: > >> I'm developin

Re: Apache Beam pipeline stuck indefinitely using Wait.on transform with JdbcIO

2023-04-25 Thread Juan Cuzmar
ble that the problem may be the direct runner's > implementation of the pubsub source - especially the watermark. For a > direct-runner test, I recommend using TestStream (which allows you to advance > the watermark manually, so you can test windowing). > > On Sat, Apr 22, 2