Re: [DISCUSS] Migrate or deprecate the Spark Kinesis connector

2024-10-17 Thread Johnson Chen
ng the same fate. We just didn't make the whole class of DStream > to produce warning messages, as we made the entry class to produce warning > messages and thought it's sufficient. > > On Mon, Oct 14, 2024 at 5:03 PM Johnson Chen > wrote: > >> Hi Spark community, >

[DISCUSS] Migrate or deprecate the Spark Kinesis connector

2024-10-14 Thread Johnson Chen
Hi Spark community, A couple months ago, I raised a PR to upgrade the AWS SDK to v2 for the Spark Kinesis connector: https://github.com/apache/spark/pull/44211. Given that the 4.0 feature freeze is coming, I am following up to check whether we still want to have this change in the upcoming 4.0 re