Hi Karthik,

As an additional update, since Flink 1.15 we have introduced the asynchronous 
sink base, which allows easy writing of an asynchronous sink (you simply 
provide the client, request and response handling) and the sink base will 
handle the state management, retries and batching.

See the blogpost [1] for more details!

Hope the above helps.

Regards,
Hong

[1] https://flink.apache.org/2022/03/16/the-generic-asynchronous-base-sink/


On 13 Jun 2023, at 05:29, Karthik Deivasigamani <karthi...@gmail.com> wrote:


CAUTION: This email originated from outside of the organization. Do not click 
links or open attachments unless you can confirm the sender and know the 
content is safe.


Thanks Martijn, the documentation for Async IO was also indicating the same and 
that's what prompted me to post this question here.
~
Karthik

On Mon, Jun 12, 2023 at 7:45 PM Martijn Visser 
<martijnvis...@apache.org<mailto:martijnvis...@apache.org>> wrote:
Hi Karthik,

In my opinion, it makes more sense to use a sink to leverage Scylla over using 
Async IO. The primary use case for Async IO is enrichment, not for writing to a 
sync.

Best regards,

Martijn

On Mon, Jun 12, 2023 at 4:10 PM Karthik Deivasigamani 
<karthi...@gmail.com<mailto:karthi...@gmail.com>> wrote:
Thanks Martijn for your response.
One thing I did not mention was that we are in the process of moving away from 
Cassandra to Scylla and would like to use the Scylla Java Driver for the 
following reason :

The Scylla Java driver is shard aware and contains extensions for a 
tokenAwareHostPolicy. Using this policy, the driver can select a connection to 
a particular shard based on the shard’s token. As a result, latency is 
significantly reduced because there is no need to pass data between the shards.

We were considering writing our own Sink to leverage Scylla Java Driver once 
the migration is done.
~
Karthik


On Mon, Jun 12, 2023 at 4:56 PM Martijn Visser 
<martijnvis...@apache.org<mailto:martijnvis...@apache.org>> wrote:
Hi,

Why wouldn't you just use the Flink Kafka connector and the Flink Cassandra 
connector for your use case?

Best regards,

Martijn

On Mon, Jun 12, 2023 at 12:03 PM Karthik Deivasigamani 
<karthi...@gmail.com<mailto:karthi...@gmail.com>> wrote:
Hi,
   I have a use case where I need to read messages from a Kafka topic, parse it 
and write it to a database (Cassandra). Since Cassandra supports async APIs I 
was considering using Async IO operator for my writes. I do not need 
exactly-once semantics for my use-case.
Is it okay to leverage the Async IO operator as a Sink (writing data into a DB)?
~
Karthik

Reply via email to