[update] I have raised a PR[1] for the changes.
The change doesn't affect any other changes to S3 connectors. Bests, Samrat [1] https://github.com/apache/flink/pull/29159 On Fri, Sep 11, 2026 at 3:29 PM Samrat Deb <[email protected]> wrote: > Thank you @[email protected] <[email protected]> > > Yes, This totally aligned with making native-s3-fs mainline along with > existing connectors. > Looking forward to hearing from the community to move forward. > > Bests, > Samrat > > > On Tue, Sep 8, 2026 at 2:22 PM Gabor Somogyi <[email protected]> > wrote: > >> Hey Samrat, >> >> +1 to promote in short. >> >> I personally think that one to one feature parity was not a target. >> Hadoop is a general purpose FS with huge amount of tuning possibilities >> however the new native S3 connector is specific to our Flink needs >> with smaller amount of tuning. >> >> The feature set which allows users to use it in production is there >> both from data processing and checkpointing perspective. >> It's used in many production pipelines already so from my point of view >> we can promote it as the preferred way. It's important to say it out >> that the old connectors must remain from code and documentation >> perspective but as a fallback if something can't be done with the new >> native s3 connector. >> >> After the promotion we should gather experience from the community, fill >> those >> gaps if there are, fix bugs and that is going to allow us to sunset the >> old >> connectors. I'm pretty sure that won't be lightning fast since we should >> allow everybody time to do the migration and express the missing >> parts/bugfixes. >> >> Happy to hear other voices. >> >> BR, >> G >> >> >> On Mon, Sep 7, 2026 at 7:25 AM Samrat Deb <[email protected]> wrote: >> >> > Hi All, >> > >> > flink-s3-fs-native (the AWS-SDK-v2, Hadoop-free S3 filesystem for s3:// >> and >> > s3a://) landed experimentally in Flink 2.3[1]. Since then, the [Phase 2] >> > Native S3 Filesystem FLINK-39109 [2] has closed 19 subtasks, closing the >> > feature gap compared to flink-s3-fs-hadoop or flink-s3-fs-presto. >> > >> > Proposal >> > >> > 1. promote NativeS3FileSystem from Experimental to mainstream in an >> > upcoming 2.4 release[3]. What is the community view and overall >> consensus? >> > >> > 2. If you run flink-s3-fs-hadoop or flink-s3-fs-presto in production, >> > please reply with anything you rely on that flink-s3-fs-native doesn't >> > cover yet. I've checked all three plugins' sources for the above, but >> > deployment-specific setups (custom credential providers, particular >> > S3-compatible endpoints, etc.) may surface gaps I've missed. >> > >> > Bests, >> > Samrat >> > >> > [1] >> > >> > >> https://cwiki.apache.org/confluence/spaces/FLINK/pages/396790457/FLIP-555+Flink+Native+S3+FileSystem >> > [2] https://issues.apache.org/jira/browse/FLINK-39109 >> > [3] https://lists.apache.org/thread/14vym623dh08rry7jsxj2sm6ty4m6cho >> > >> >
