I'm reviewing the jiras fixed in 2.1, and adding release notes if they look like user visible change.
Please help me add more detailed description from the following jira filter <https://issues.apache.org/jira/issues/?filter=-1&jql=project%20%3D%20HDDS%20AND%20fixVersion%20%3D%202.1.0%20AND%20component%20not%20in%20(Documentation%2C%20Test)%20AND%20priority%20%3E%20Minor%20ORDER%20BY%20created%20DESC> : Also, with the help from gemini cli, based on git diff between master and ozone2.0 branches, I managed to summarize configuration property changes in ozone-default.xml in between: (I know it's not 100% correct and it needs to be reviewed) Added Configurations (present in asf/master but not in asf/ozone-2.0) * hdds.raft.server.rpc.first-election.timeout: Ratis Minimum timeout for the first election of a leader. If not configured, fallback to hdds.ratis.leader.election.minimum.timeout.duration. (Default: None) * ozone.om.compaction.service.enabled: Enable or disable a background job that periodically compacts rocksdb tables flagged for compaction. (Default: false) * ozone.om.compaction.service.run.interval: A background job that periodically compacts rocksdb tables flagged for compaction. Unit could be defined with postfix (ns,ms,s,m,h,d) (Default: 6h) * ozone.om.compaction.service.timeout: A timeout value of compaction service. If this is set greater than 0, the service will stop waiting for compaction completion after this time. Unit could be defined with postfix (ns,ms,s,m,h,d) (Default: 10m) * ozone.om.compaction.service.columnfamilies: A comma separated, no spaces list of all the column families that are compacted by the compaction service. If this is empty, no column families are compacted. (Default: keyTable,fileTable,directoryTable,deletedTable,deletedDirectoryTable,multipartInfoTable) * ozone.om.snapshot.compact.non.snapshot.diff.tables: Enable or disable compaction of tables that are not tracked by snapshot diff when their snapshots are evicted from cache. (Default: false) * hdds.heartbeat.initial-interval: Heartbeat interval used during Datanode initialization for SCM. (Default: 2s) * hdds.heartbeat.recon.interval: The heartbeat interval from a Datanode to Recon. (Default: 60s) * hdds.heartbeat.recon.initial-interval: Heartbeat interval used during Datanode initialization for Recon. (Default: 60s) * ozone.om.s3.grpc.server_enabled: Property to enable or disable Ozone Manager gRPC endpoint for clients. Right now, it is used by S3 Gateway only. (Default: true) * ozone.om.snapshot.prune.compaction.backup.batch.size: Prune SST files in Compaction backup directory in batches every ozone.om.snapshot.compaction.dag.prune.daemon.run.interval. (Default: 2000) * ozone.scm.ha.raft.server.rpc.first-election.timeout: ratis timeout for the first election of a leader. If not configured, fallback to ozone.scm.ha.ratis.leader.election.timeout. (Default: None) * ozone.om.edekcacheloader.interval.ms: When KeyProvider is configured, the interval time of warming up edek cache on OM starts up. All edeks will be loaded from KMS into provider cache. The edek cache loader will try to warm up the cache until succeed or OM leaves active state. (Default: 1000) * ozone.om.edekcacheloader.initial.delay.ms: When KeyProvider is configured, the time delayed until the first attempt to warm up edek cache on OM start up. (Default: 3000) * ozone.om.edekcacheloader.max-retries: When KeyProvider is configured, the max retries allowed to attempt warm up edek cache if none of key successful on OM start up. (Default: 10) Modified Configurations (default value changed) * hdds.datanode.volume.choosing.policy: Changed from org.apache.hadoop.ozone.container.common.volume.RoundRobinVolumeChoosingPolicy to org.apache.hadoop.ozone.container.common.volume.CapacityVolumeChoosingPolicy * ozone.key.deleting.limit.per.task: Changed from 20000 to 50000 * ozone.om.ratis.segment.size: Changed from 4MB to 64MB * ozone.om.ratis.segment.preallocated.size: Changed from 4MB to 4MB * ozone.om.fs.snapshot.max.limit: Changed from 1000 to 10000 * fs.trash.classname: Changed from org.apache.hadoop.ozone.om.TrashPolicyOzone to org.apache.hadoop.fs.ozone.OzoneTrashPolicy * hdds.scm.safemode.min.datanode: Changed from 1 to 3 * ozone.s3g.https-address: Changed from None to 0.0.0.0:9879 * ozone.s3g.webadmin.https-address: Changed from None to 0.0.0.0:19879 * hdds.secret.key.expiry.duration: Changed from 7d to 9d * ozone.om.snapshot.compaction.dag.prune.daemon.run.interval: Changed from 3600s to 10m * ozone.scm.ha.ratis.segment.size: Changed from 4MB to 64MB * ozone.scm.ha.dbtransactionbuffer.flush.interval: Changed from 600s to 60s Removed Configurations (present in asf/ozone-2.0 but not in asf/master) * hdds.datanode.volume.min.free.space * hdds.container.ratis.replication.level * hdds.recon.heartbeat.interval * hdds.rest.http-address * hdds.scm.safemode.pipeline-availability.check * ozone.manager.db.checkpoint.transfer.bandwidthPerSec * ozone.om.keyname.character.check.enabled On Wed, Sep 3, 2025 at 10:21 AM Tsz-Wo Nicholas Sze <[email protected]> wrote: > Hi Chung-En, > > That's great! Thanks again for working on the release. > > Tsz-Wo > > > On Wed, Sep 3, 2025 at 1:59 AM Chung-En Lee <[email protected]> wrote: > > > Hi Tsz-Wo, > > > > The proposal sounds good. I agree with the timeline you've outlined. > > > > I also wanted to bring up the disk balancer. Since it hasn't been merged > > into master yet, I'd like to get it into this release if we can. If > that's > > not feasible, we can plan to include it in the next version. > > > > Let me know what you think. > > > > Best regards, > > Chung-En > > > > On 2025/08/27 18:44:26 Tsz-Wo Nicholas Sze wrote: > > > Hi chung en, > > > > > > > ~8/29 Freeze scope > > > > 8/30~9/19 Freeze code > > > > 9/20~ Start the RC (Release Candidate) process > > > > > > The proposal sounds good. Once the scope has been freezed, we should > > > create a branch for 2.1.0. Then, only bug fixes can be merged to the > > 2.1.0 > > > branch. Feature changes and other commits can still be merged to the > > > master branch. > > > > > > We should also fix the dates so everyone would have the same > expectation. > > > We may have > > > - Sep 6: feature freeze and branching out. > > > - Sep 19: code freeze > > > > > > What do you think? > > > > > > Tsz-Wo > > > > > > On Tue, Aug 26, 2025 at 11:58 PM Wei-Chiu Chuang <[email protected]> > > wrote: > > > > > > > In addition the few improvements mentioned above > > > > - Ozone Snapshot Phase 3 HDDS-12940 > > > > - Container and volume scanners phase II HDDS-8387 > > > > > > > > A few big improvements are also worth mentioning: > > > > > > > > > > > > 1. HDDS-12564 <https://issues.apache.org/jira/browse/HDDS-12564> > > > > Handling > > > > disk issues in Datanodes - Phase II > > > > 1. HDDS-12716 <https://issues.apache.org/jira/browse/HDDS-12716> > > Ozone > > > > S3 gateway Phase 4 > > > > > > > > > > > > > > > > > > > > On Wed, Aug 20, 2025 at 1:58 AM Chung En Lee <[email protected]> > > wrote: > > > > > > > > > Hello everyone, > > > > > > > > > > Thank you for all the discussion and feedback. Based on our ongoing > > > > > discussions and the consensus so far, I'd like to propose the > > following > > > > > schedule for the Apache Ozone 2.1.0 release: > > > > > > > > > > ~8/29 Freeze scope > > > > > 8/30~9/19 Freeze code > > > > > 9/20~ Start the RC (Release Candidate) process > > > > > > > > > > The features to be included in this release are expected to be: > > > > > > > > > > - Container Reconciliation HDDS-10239 > > > > > <https://issues.apache.org/jira/browse/HDDS-10239> > > > > > - Disk Balancer HDDS-5713 < > > > > https://issues.apache.org/jira/browse/HDDS-5713 > > > > > > > > > > > - Ozone Short Circuit Read HDDS-10685 > > > > > <https://issues.apache.org/jira/browse/HDDS-10685> > > > > > - Container and volume scanners phase II HDDS-8387 > > > > > <https://issues.apache.org/jira/browse/HDDS-8387> > > > > > - Ozone Snapshot Phase 3 HDDS-12940 > > > > > <https://issues.apache.org/jira/browse/HDDS-12940> > > > > > - Replace goofys with mountpoint-s3 HDDS-13356 > > > > > <https://issues.apache.org/jira/browse/HDDS-13356> > > > > > > > > > > I look forward to your feedback on this proposed timeline. > > > > > > > > > > Thanks, > > > > > chung en > > > > > > > > > > On 2025/08/11 18:20:15 Wei-Chiu Chuang wrote: > > > > > > Container Reconciliation is merged into master. A few leftover > > minor > > > > > items > > > > > > are being worked on. HDDS-10239 > > > > > > <https://issues.apache.org/jira/browse/HDDS-10239> > > > > > > Disk balancer branch is voted to merge into master HDDS-5713 > > > > > > <https://issues.apache.org/jira/browse/HDDS-5713> > > > > > > > > > > > > > > > > > > Ozone Snapshot Phase 3 HDDS-12940 > > > > > > <https://issues.apache.org/jira/browse/HDDS-12940> -- we > > finished 3 > > > > > out > > > > > > of the 4 subprojects. The last one Snapshot Defragmentation > > > > HDDS-13003 > > > > > is > > > > > > making good progress. Should reach dev completion in a month > or > > so. > > > > > Think > > > > > > it can make into Ozone 2.1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > S3 Lifecycle is making steady progress -- it's a big project so > > I'm not > > > > > > sure if it'll make it soon. > > > > > > > > > > > > Looks like HDDS-11523 > > > > > > <https://issues.apache.org/jira/browse/HDDS-11523> Listener > > > > > > OM support is picking up steam again. > > > > > > Maybe we can include HDDS-13356 > > > > > > <https://issues.apache.org/jira/browse/HDDS-13356> (replace > goofys > > > > with > > > > > > mountpoint-s3) too. Looks like a medium level effort. Peter is > > working > > > > on > > > > > > these two projects. > > > > > > > > > > > > On Mon, Jul 21, 2025 at 6:20 PM Wei-Chiu Chuang < > > [email protected]> > > > > > wrote: > > > > > > > > > > > > > > > > > > > > 1. I am aware of a few other projects being actively > developed > > > > > > > > > > > > > > Volume / Container scanner is being pursued by @Ethan Rose > > > > > > > <[email protected]> and team. HDDS-8387 > > > > > > > <https://issues.apache.org/jira/browse/HDDS-8387> > > > > > > > Ozone Short Circuit Read HDDS-10685 > > > > > > > <https://issues.apache.org/jira/browse/HDDS-10685> by > @Sammi > > Chen > > > > > > > <[email protected]> > > > > > > > > > > > > > > > > > > > > > Ozone Snapshot Phase 3 HDDS-12940 > > > > > > > <https://issues.apache.org/jira/browse/HDDS-12940> > > > > > > > > > > > > > > > > > > > > > These could be included in the 2.1 scope too. > > > > > > > > > > > > > > On Mon, Jul 21, 2025 at 5:58 PM Chung En Lee - Apache < > > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > >> Hi community, > > > > > > >> > > > > > > >> I would like to initiate a discussion about the upcoming > Apache > > > > Ozone > > > > > 2.1 > > > > > > >> release. My primary goal is to push for a more regular and > > > > predictable > > > > > > >> release schedule, as I believe this would accelerate the > > delivery of > > > > > > >> features more effectively. > > > > > > >> > > > > > > >> For the 2.1 release, I propose including the following new > > features > > > > or > > > > > > >> improvements: > > > > > > >> > > > > > > >> *- Container Reconciliation* > > > > > > >> > > > > > > >> *- S3 Lifecycle* > > > > > > >> > > > > > > >> > > > > > > >> *- Disk Balancer* > > > > > > >> > > > > > > >> Besides the usual binary distributions, I'm hoping we can also > > > > provide > > > > > > >> Ozone in the following ways: > > > > > > >> > > > > > > >> - *DEB/RPM packages* > > > > > > >> > > > > > > >> *- ARM architecture binaries* > > > > > > >> > > > > > > >> What are your thoughts? Looking forward to your feedback! > > > > > > >> > > > > > > >> Thanks, > > > > > > >> > > > > > > >> Chung En > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > >
