Re: [VOTE] Change the default branch for ozone-site from asf-site to master

2024-02-18 Thread Kaijie Chen
+1Best,Kaijie Chen On Sun, 18 Feb 2024 17:14:06 +0800 Szabolcs Gál wrote +1 Szabolcs Gál On Sat, Feb 17, 2024 at 5:57 AM Siyao Meng wrote: > +1 > > *Is there any standard among other ASF projects for which branch should be > &g

Re: [Announce] Apache Ozone 1.4.0 Release

2024-01-21 Thread Kaijie Chen
Great work, Yiyang, Thanks everyone contributed to this release. Best, Kaijie On Mon, 22 Jan 2024 11:48:08 +0800 Sammi Chen  wrote Yiyang, Thank you, for all you work to let this happen! Bests, Sammi

Re: [VOTE] Apache Ozone 1.4.0 RC1

2024-01-17 Thread Kaijie Chen
+1 (binding)  Thanks Yiyang for driving this release. * Verified download link, signature and checksums  * Verified files matching git tag * Build from source successfully * Start a cluster using binary package  * Run basic commands Best,Kaijie

Re: [ANNOUNCE] Janus Chow (a.k.a. Yiyang Zhou) joins Apache Ozone PMC

2023-11-22 Thread Kaijie Chen
Congrats, Yiyang! Best,Kaijie On Thu, 23 Nov 2023 12:04:12 +0800 Sammi Chen  wrote Congratulations to Yiyang! Sammi On Wed, 22 Nov 2023 at 16:18, Hui Fei wrote: > Congrats, Yiyang

Re: Cutting Ozone 1.3.1

2023-03-27 Thread Kaijie Chen
1) are included as dependencies. Best, Kaijie On 2023/03/14 07:03:23 Kaijie Chen wrote: > +1, sound good > > I would like to propose two more fixes to the list. > A FS contract bug caused by HDDS-7253 is fixed by them. > > https://issues.apache.org/jira/browse/HDDS-7871 > htt

Re: Cutting Ozone 1.3.1

2023-03-14 Thread Kaijie Chen
+1, sound good I would like to propose two more fixes to the list. A FS contract bug caused by HDDS-7253 is fixed by them. https://issues.apache.org/jira/browse/HDDS-7871 https://issues.apache.org/jira/browse/HDDS-7991 Best, Kaijie On Tue, 14 Mar 2023 14:52:38 +0800 Ritesh Shukla wrote

Re: [DISCUSS] How many days to wait if no response from reviewer?

2023-01-31 Thread Kaijie Chen
+1 Thanks Uma for bring up this discussion. Personally I think this should be a convention instead of a rule. On Wed, 01 Feb 2023 07:16:37 +0800 Ritesh Shukla wrote --- > +1 for this. > We can wait to gather a broader consensus for certain rare cases where we > add new APIs or public

Re: [VOTE] Apache Ozone 1.3.0 RC2

2022-12-13 Thread Kaijie Chen
+1 (non-binding) * Verified the checksums and signature * Commit hash matches the git tag * Build from source successful * Started a cluster using binary package * Tested RATIS and EC write/read/delete with freon ockg/ockv/ockr * Tested ozone fs -ls in legacy bucket when '/' in key name. * Tested

Re: [VOTE] Merging HDDS-4454 Ozone Streaming Write Pipeline into master

2022-12-08 Thread Kaijie Chen
+1 (non-binding) Kaijie On Fri, 09 Dec 2022 15:06:35 +0800 Tsz Wo Sze wrote --- > Hi Ozone Dev, > > This is the VOTE thread following to the discussion thread > https://lists.apache.org/thread/nt7jwrxklk79qvmjwpv8noor760p156t > > Here is my +1. The VOTE will close in 7 days. Th

Re: [VOTE] Apache Ozone 1.3.0 RC1

2022-12-06 Thread Kaijie Chen
> https://issues.apache.org/jira/browse/HDDS-7526 we merged, we > should cherry-pick it to 1.3 branch as well. Thanks Ritesh for the fix, I have cherry-picked a bunch of fixes related to quota to 1.3 branch: * HDDS-7346. Cannot set bucket args when the bucket has quota set (#3975) * HDDS-7

Re: [DISCUSS] Proposal to merge HDDS-4454 Ozone Streaming Write Pipeline into master

2022-12-05 Thread Kaijie Chen
+1 for merging. Hope it can be tested more in master branch and become stable soon. Kaijie On Tue, 06 Dec 2022 09:35:42 +0800 Arpit Agarwal wrote --- > +1 for merging this feature. This is an important performance improvement > for Ozone. > > Nicholas, what's the plan to have stre

Re: [VOTE] Apache Ozone 1.3.0 RC1

2022-12-01 Thread Kaijie Chen
ry for example. Sorry, this is not a problem because the LICENSE and NOTICE files for binary package are under hadoop-ozone/dist/src/main/license/bin/ Kaijie On Thu, 01 Dec 2022 15:00:29 +0800 Kaijie Chen wrote --- > Thanks Nicholas for finding the license issues, and thanks M

Re: [VOTE] Apache Ozone 1.3.0 RC1

2022-11-30 Thread Kaijie Chen
Thanks Nicholas for finding the license issues, and thanks Mingchao for hosting RC2 release. I have submitted 2 PRs for the problems Nicholas found: HDDS-7567 and HDDS-7568. Please take a look. There may be another issue: If there is additional software included in the binary package (which most

Re: [VOTE] Apache Ozone 1.3.0 RC1

2022-11-30 Thread Kaijie Chen
Hi community, I just checked LICENSE.txt, the license of glyphicons-halflings seems also incorrect. Glyphicons-halflings itself is licensed under a commercial license [1]. However, there is a rule for "GLYPHICONS Halflings in Bootstrap in previous version 1.9.2" [2]. > GLYPHICONS Halflings fon

Re: [VOTE] Apache Ozone 1.3.0 RC1

2022-11-30 Thread Kaijie Chen
Thanks MingChao for driving the release. +1 (non-binding) * Verified the checksums and signature * Commit hash matches the git tag * Built from source, regular CI passed * Started a cluster using binary package * Tested EC with freon ockg, ockv and ockr * Verified orphan block deletion Best, Kai

Re: [VOTE] Apache Ozone 1.3.0 RC0

2022-11-14 Thread Kaijie Chen
+1, Thanks mingchao for the work. Verified: 1. Checksum and signature matches. 2. Successfully built from source. 3. Commit hash of the binary matches. 4. Checked LICENSE and NOTICE. 5. Successfully deployed on a cluster. 6. Run basic tests with freon. PS: Should we wait for Ratis 2.4.1 release?

Re: [RFC] Proposal: Reserve Space for Allocated Blocks

2022-11-02 Thread Kaijie Chen
for Allocated Blocks) > > > On 25-Oct-2022, at 9:18 AM, Kaijie Chen c...@apache.org> wrote: > > > > Looking into the AllocateBlock interface, it assumes all blocks allocated > > are in the same size. > > > >    List allocateB

Re: Slack invite

2022-10-26 Thread Kaijie Chen
Hi Kirill, Please join ASF slack workspace and search the #ozone channel. https://join.slack.com/t/the-asf/shared_invite/zt-1fm9561yr-uzTpjqg3jf5nxSJV5AE3KQ Best, Kaijie On Wed, 26 Oct 2022 14:37:53 +0800 Kirill Sizov wrote --- > Hi Ozone devs, > > Could you please add me to the ozo

Re: [RFC] Proposal: Reserve Space for Allocated Blocks

2022-10-24 Thread Kaijie Chen
ve and tend to converge to your expected size, overall your > system is stable(r). > > > Thanks > Anu > > > > > On Wed, Sep 28, 2022 at 5:56 AM Kaijie Chen c...@apache.org> wrote: > > > Hi Anu, > > > > Thanks for your suggestions

Re: [RFC] Proposal: Reserve Space for Allocated Blocks

2022-09-28 Thread Kaijie Chen
Plus: If someday the QPS of OM and SCM has been improved, this (over-allocation) might be a more significant problem. Kaijie On Wed, 28 Sep 2022 20:54:17 +0800 Kaijie Chen wrote --- > Hi Anu, > > Thanks for your suggestions. These are indeed where we can > improve the

Re: [RFC] Proposal: Reserve Space for Allocated Blocks

2022-09-28 Thread Kaijie Chen
ximum limit ? It is not needed per se -- more like a question > that I am asking myself. > > Thanks > Anu > > > > > On Mon, Sep 26, 2022 at 2:42 AM Kaijie Chen c...@apache.org> wrote: > > > Hi everyone, > > > > I've impleme

Re: [RFC] Proposal: Reserve Space for Allocated Blocks

2022-09-26 Thread Kaijie Chen
Hi everyone, I've implemented a container lease POC [1], and the result looks good. Here's what's changed in the POC: 1. SCM will keep a LeaseExipreAt for each OPEN container. If SCM receives container close command, it will change the container state to CLOSING, but it will not send clo

Re: [RFC] Proposal: Reserve Space for Allocated Blocks

2022-09-14 Thread Kaijie Chen
I believe > > > helps to solve the problem with the least amount of disruption, but I > > > possibly did not thought through all the possible failure scenarios, and > > > haven't checked what is actually in the code at the moment, just wanted > > to > &g

Re: [RFC] Proposal: Reserve Space for Allocated Blocks

2022-09-13 Thread Kaijie Chen
Thanks Anu and Steven for the suggestion. Granting a lease to client sounds like a more controllable way. However, if I understand correctly, clients don't talk to SCM directly. Does it mean OM has to relay the renew lease request to SCM? Is there a better way to implement it? Regards Kaijie --

Re: [RFC] Proposal: Reserve Space for Allocated Blocks

2022-09-08 Thread Kaijie Chen
Thanks Ethan. Yes this could be a simpler solution. The main idea is allowing container size limit to be exceeded, to ensure all allocated blocks can be finished. We can change it to something like this: 1. Datanode notices the container is near full. 2. Datanode sends close container action to SC

Re: [RFC] Proposal: Reserve Space for Allocated Blocks

2022-09-08 Thread Kaijie Chen
> Are you seeing this for Ratis writes or only EC? Have you changed the EC > pipeline limit to a higher value than 5? I wonder if a lesser number of > open write pipelines could contribute to this problem too. This exception is reproducible in both RATIS and EC. https://paste.ubuntu.com/p/NjpQ

Re: [RFC] Proposal: Reserve Space for Allocated Blocks

2022-09-07 Thread Kaijie Chen
> I'm not sure, but we can see a lot of errors like this in the client log. > Please see the attachment for more details. Sorry, the attachment seems not working, please see the following link instead: https://paste.ubuntu.com/p/dnNF45n7Q2/plain/

Re: [RFC] Proposal: Reserve Space for Allocated Blocks

2022-09-07 Thread Kaijie Chen
Thanks Stephen for explaining, > I have a few thoughts on this, but my knowledge may be out-dated. > > 1. During putBlock, the DN notices the usage has gone beyond 90%, so it > sends a close command to SCM via its heartbeat. > > 2. SCM closes the container on the SCM side. At this point, S

[RFC] Proposal: Reserve Space for Allocated Blocks

2022-09-07 Thread Kaijie Chen
assembling containers in the background. Regards, Kaijie Chen - To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org For additional commands, e-mail: dev-h...@ozone.apache.org

Re: [DISCUSS] 1.3.0 release

2022-07-28 Thread Kaijie Chen
+1 Thanks Mingchao. P.S. I'm trying to figure out an issue related to block deleting, and I hope it could be fixed in this release. Best, Kaijie Chen

Re: [DISCUSS] Proposal to merge Ozone datanode side RocksDB merge feature branch (HDDS-3630) into master

2022-05-23 Thread Kaijie Chen
Besides Arpit's questions, will there be any backward compatibility issues if we enable this by default?ThanksKaijie On 星期一, 23 五月 2022 23:51:16 +0800 aagar...@cloudera.com.INVALID wrote +1 for proceeding with the merge. This work is crucial for improving the DataNode stability. Thank

Re: [DISCUSS] 1.3.0 release

2022-04-22 Thread Kaijie Chen
I think 1.2.2 sounds like a bug fix version.If we are going to release a new feature version, 1.3.0 would be the proper name. On 星期五, 22 四月 2022 19:13:18 +0800 captain...@apache.org wrote Thanks @Stephen for your feedback. Maybe we can de-emphasize the EC in this version. If EC recov