Is there any example that use hbase connector in stream API

2023-09-21 Thread 碳酸君
hi community: I'm trying to write some data to hbase in a stream job ,with flink-connector-hbase-2.2 . I have no idea about instantiate org.apache.flink.connector.hbase.util.HBaseTableSchema and org.apache.flink.connector.hbase.sink.HBaseMutationConverter . Regards, Teii

Re: Window aggregation on two joined table

2023-09-21 Thread Eugenio Marotti
Thank you. I’ll try to configure a watermark on the second table. Eugenio > Il giorno 21 set 2023, alle ore 16:30, Schwalbe Matthias > ha scritto: > > … well yes and no: > If the second table is a small table used for enrichment, you can also mark > it as broadcast table, but I don’t know ho

Re: Zookeeper HA with Kubernetes: Possible to use the same Zookeeper cluster w/multiple Flink Operators?

2023-09-21 Thread Brian King
Gyula, Thanks, you've helped us move closer to migrating our application to the Flink Operator![0] Best, Brian King SRE, Data Platform/Search Platform Wikimedia Foundation IRC: inflatador [0] https://phabricator.wikimedia.org/T326409 > On Sep 21, 2023, at 12:16 AM, Gyula Fóra wrote: > >

Completable Future in RichSinkFunction with Retry

2023-09-21 Thread patricia lee
I initially used the genericbase sink / the richAsync function, but these two were not applicable to my use case. I implemented a completable future that sends data sendBatch() to vendor api. Is there a built in api supported for retry with custom method in rich sink function? Regards, Pat

RE: Window aggregation on two joined table

2023-09-21 Thread Schwalbe Matthias
… well yes and no: * If the second table is a small table used for enrichment, you can also mark it as broadcast table, but I don’t know how to do that on table API * If the second table has significant data and significant update, the you need to configure watermarking/event time semant

Re: Window aggregation on two joined table

2023-09-21 Thread Eugenio Marotti
Hi Matthias, No the second table doesn’t have an event time and a watermark specified. In order for the window to work do I need a watermark also on the second table? Thanks Eugenio > Il giorno 21 set 2023, alle ore 13:45, Schwalbe Matthias > ha scritto: > > Ciao Eugenio, > > I might be mi

RE: Window aggregation on two joined table

2023-09-21 Thread Schwalbe Matthias
Ciao Eugenio, I might be mistaken, but did you specify the event time for the second table like you did for the first table (watermark(….))? I am no so acquainted with table api (doing more straight data stream api work), but I assume this join and windowing should be by event time. What do you

Postgres Jdbc Exactly Once Sink Deadlock with UPSERT query

2023-09-21 Thread Pascal Rosenkranz
Hi everyone, In a streaming Job, we are using the JdbcSink.exactlyOnceSink() as described on https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/connectors/d atastream/jdbc/#jdbcsinkexactlyoncesink