Hi Karthik, As an additional update, since Flink 1.15 we have introduced the asynchronous sink base, which allows easy writing of an asynchronous sink (you simply provide the client, request and response handling) and the sink base will handle the state management, retries and batching.
See the blogpost [1] for more details! Hope the above helps. Regards, Hong [1] https://flink.apache.org/2022/03/16/the-generic-asynchronous-base-sink/ On 13 Jun 2023, at 05:29, Karthik Deivasigamani <karthi...@gmail.com> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Thanks Martijn, the documentation for Async IO was also indicating the same and that's what prompted me to post this question here. ~ Karthik On Mon, Jun 12, 2023 at 7:45 PM Martijn Visser <martijnvis...@apache.org<mailto:martijnvis...@apache.org>> wrote: Hi Karthik, In my opinion, it makes more sense to use a sink to leverage Scylla over using Async IO. The primary use case for Async IO is enrichment, not for writing to a sync. Best regards, Martijn On Mon, Jun 12, 2023 at 4:10 PM Karthik Deivasigamani <karthi...@gmail.com<mailto:karthi...@gmail.com>> wrote: Thanks Martijn for your response. One thing I did not mention was that we are in the process of moving away from Cassandra to Scylla and would like to use the Scylla Java Driver for the following reason : The Scylla Java driver is shard aware and contains extensions for a tokenAwareHostPolicy. Using this policy, the driver can select a connection to a particular shard based on the shard’s token. As a result, latency is significantly reduced because there is no need to pass data between the shards. We were considering writing our own Sink to leverage Scylla Java Driver once the migration is done. ~ Karthik On Mon, Jun 12, 2023 at 4:56 PM Martijn Visser <martijnvis...@apache.org<mailto:martijnvis...@apache.org>> wrote: Hi, Why wouldn't you just use the Flink Kafka connector and the Flink Cassandra connector for your use case? Best regards, Martijn On Mon, Jun 12, 2023 at 12:03 PM Karthik Deivasigamani <karthi...@gmail.com<mailto:karthi...@gmail.com>> wrote: Hi, I have a use case where I need to read messages from a Kafka topic, parse it and write it to a database (Cassandra). Since Cassandra supports async APIs I was considering using Async IO operator for my writes. I do not need exactly-once semantics for my use-case. Is it okay to leverage the Async IO operator as a Sink (writing data into a DB)? ~ Karthik
