CalvinKirs opened a new pull request, #50320:
URL: https://github.com/apache/doris/pull/50320
### What problem does this PR solve?
Issue Number: #52038
#### Background
In the context of #52038, we are introducing a new set of storage parameters
aimed at improving flexibility, supporting additional object storage types, and
simplifying protocol management. This PR focuses on adapting the filesystem
layer to align with the new parameter set and unifying the usage of protocols
across supported storage systems.
#### Key Changes
##### Protocol Unification
Historically, to support object storage systems like S3, we reused the HDFS
protocol (e.g., s3a://) as a workaround. This approach has led to confusion and
increased implementation complexity.
In this PR, the use of the HDFS protocol to interact with S3 storage is
completely removed. From now on, S3-compatible storage only using the S3
protocol exclusively, improving clarity and protocol consistency.
##### HDFS-Compatible Parameter Adapter
To ease future extensibility, a dedicated parameter adaptation layer is
introduced for HDFS compatibility. This layer translates deprecated parameter
keys to the new ones internally and acts as a bridge during the migration phase.
#### Package Structure (Transitional Phase)
To ensure smooth transition and compatibility, we temporarily maintain two
filesystem implementation packages:
fs: legacy implementation with current logic
fsv2: new implementation adapted for the modern parameter and protocol system
Once the new system has been fully validated, the fsv2 package will be
renamed to fs and become the primary implementation.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
