Re: Deleting unused 'master' branch

I am not getting any -1 so I assume everyone's okay with it. But just in case it went into the spam folder... On Sun, Apr 13, 2025 at 3:54 PM Wei-Chiu Chuang wrote: > >1. INFRA-26751 > >I realized the master branch appeared again. Opene

Re: [DISCUSS] Optimize the invoke retry of async repsonder when some nameservices are slow

+ hdfs-dev. Thanks Haobo for your proposal. As you mentioned above, this may be related to RBF and ARR features, right? IMO, it is necessary to improve responder performance, but I am a little confused about nameservice will slow down the whole system, The first glance is client or package size sh

Re: [DISCUSS] Optimize the invoke retry of async repsonder when some nameservices are slow

think you meant to send this to hdfs-dev, not the -subscribe list. On Wed, 16 Apr 2025 at 14:09, Zhanghaobo wrote: > Hello, everyone, so sorry to bother you and I would like to discuss how > to optimize the invoke retry logic of async repsonder when some > nameservices are slow. > Currently,

Re: HADOOP-19343 feature branch: GCS FileSystem

I like it, even if gcs want to be more agile at first 1. can help testing/debugging against a broadly used store 2. stops us accidentally breaking things (who would do that 🙂) 3. makes it easier to do changes across the modules Because Chris is involved, I'm not worried about this being

Re: HADOOP-19343 feature branch: GCS FileSystem

Hi Wei-Chiu, This is an important trade-off to consider. There is some discussion in the JIRA issue comments relevant to this topic. To summarize, we prefer closer integration with the ASF community as we continue to evolve this FileSystem. It's true that releases will operate at a different caden

Re: HADOOP-19343 feature branch: GCS FileSystem

Chris, I'm curious is having a GCS fs implementation in the main Hadoop repo the right choice? As mentioned by Steve, Hadoop release cadence tends to be slow. There's a lot of legacy code so release work is usually a multi month effort. Have you considered having it in a separate repo? On Mon, Apr

Re: [ANNOUNCE] Apache Hadoop 3.4.1 release

Hi Ayush, I see. Thanks for the info. Bests, Sammi On Tue, 8 Apr 2025 at 16:47, Ayush Saxena wrote: > Hi Sammi, > The aarch64 isn't released in 3.4.1 [1], it was decided to be part of 3.4.2 > > -Ayush > > [1] https://lists.apache.org/thread/zf2rzk5scnst9zp7r6z4lnlww790pgns > > On Tue, 8 Apr 20

Re: [ANNOUNCE] Apache Hadoop 3.4.1 release

Hi Sammi, The aarch64 isn't released in 3.4.1 [1], it was decided to be part of 3.4.2 -Ayush [1] https://lists.apache.org/thread/zf2rzk5scnst9zp7r6z4lnlww790pgns On Tue, 8 Apr 2025 at 14:12, Sammi Chen wrote: > > Thanks Mukund for making the 3.4.1 release happen. > > Today I tried downloading t

Re: [ANNOUNCE] Apache Hadoop 3.4.1 release

Thanks Mukund for making the 3.4.1 release happen. Today I tried downloading the 3.4.1 binary packages. There are both binary and binary-aarch64 links on page https://hadoop.apache.org/releases.html. But on the binary-aarch64 link page, I was not able to find where to download the file. Is aar

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

pushed up. thanks for spotting that. Added another 3p. target to the release-support build file. Really trying to automate all these steps 3p.git-tag-push: [git] To github.com:apache/hadoop-thirdparty.git [git] * [new tag] release-1.4.0-RC1 -> release-1.4.0-RC1 On Mon, 24 M

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

thanks. will kick off a new build. Did anyone else see problems with the gpg versions? I destroyed the entire .gpgconf dir on the host i was building on; did manual verify after importing KEYS, and all seemed good. On Thu, 20 Mar 2025 at 04:37, Chris Nauroth wrote: > This is committed to tru

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

The time is up, the votes are in and they are +1 (binding) Ayush Saxena (binding) Chris Nauroth (binding) Steve Loughran (binding) Wei-Chiu Chuang (binding) Xiaoqiao He (binding) = 0: none -1: none Accordingly, the RC is the release; I will do the next steps in the process ASAP Steve On Tue,

Re: [VOTE] Request to merge HADOOP-19236 into trunk.

Thanks all. There are 3 binding +1 and 2 non-binding +1. HADOOP-19236 will be merged into trunk shortly. Best Regards, - Jinglun On 2025/03/20 03:05:49 jinglun wrote: > Hi everyone, I would like to propose merging > HADOOP-19236（https://issues.apache.org/jira/browse/HADOOP-19236） into trunk. >

Re: JDK 17 (or beyond) upgrade

Hello Vincent, There isn't a firm timeline yet, but the community is rallying around a 3.5.0 release with full support for JDK 17. In addition to the issue you linked, there is also this one: https://issues.apache.org/jira/browse/HADOOP-17177 Known work related to JDK 17 compatibility is going t

Re: [VOTE] Request to merge HADOOP-19236 into trunk.

+1 (non-binding) It's really great to hear that we started this vote to merge the HADOOP-19236 into trunk, finally. Best, Zheng ! On 2025/03/26 02:54:43 Xin Sun wrote: > +1 (non-binding). > This feature is very useful for us, and I look forward to merge it into trunk. > > On 2025/03/20 03:05:4

Re: [VOTE] Request to merge HADOOP-19236 into trunk.

+1 (non-binding). This feature is very useful for us, and I look forward to merge it into trunk. On 2025/03/20 03:05:49 jinglun wrote: > Hi everyone, I would like to propose merging > HADOOP-19236（https://issues.apache.org/jira/browse/HADOOP-19236） into trunk. > > > How to merge > The original

Re: [VOTE] Request to merge HADOOP-19236 into trunk.

+1 from my side. Best Regards. On 2025/03/20 03:05:49 jinglun wrote: > Hi everyone, I would like to propose merging > HADOOP-19236（https://issues.apache.org/jira/browse/HADOOP-19236） into trunk. > > > How to merge > The original branch is > HADOOP-19236(https://github.com/apache/hadoop/pull/

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

Update: +1(binding). On Tue, Mar 25, 2025 at 7:10 PM Steve Loughran wrote: > > > pushed up. thanks for spotting that. Added another 3p. target to the > release-support build file. Really trying to automate all these steps > > 3p.git-tag-push: > [git] To github.com:apache/hadoop-thirdparty.

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

Thanks Steve for driving this release. +0 and will give my +1 while tag to be fixed. [Y] LICENSE files exist and NOTICE is included. [Y] Build the source code on Ubuntu and OpenJDK 11. [Y] Verified checksums and Signature. [Y] Skimmed over the contents of ReleaseNotes and ChangeLogs. [N] release

Re: [VOTE] Request to merge HADOOP-19236 into trunk.

+1. I give a +1 and look forward to a smooth progression. Best Regards, Shilun Fan. On Thu, Mar 20, 2025 at 11:06 AM jinglun wrote: > > Hi everyone, I would like to propose merging HADOOP-19236（ https://issues.apache.org/jira/browse/HADOOP-19236） into trunk. > > > How to merge > The original br

Re: [VOTE] Request to merge HADOOP-19236 into trunk.

+1. Best Regards, - He Xiaoqiao On Thu, Mar 20, 2025 at 11:06 AM jinglun wrote: > > Hi everyone, I would like to propose merging > HADOOP-19236（https://issues.apache.org/jira/browse/HADOOP-19236） into trunk. > > > How to merge > The original branch is > HADOOP-19236(https://github.com/apache/h

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

+1 * Verified signatures. * LICENSE-binary has the expected dependencies and versions. * Site looks good too. * Specifically confirmed inclusion of HADOOP-19510, previously identified during RC0. Thanks for the release, Steve! Chris Nauroth On Fri, Mar 21, 2025 at 10:43 AM Steve Loughran wrot

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

+1 (Binding) * Built from source * Verified checksums * Verified Signature * Verified no diff b/w branch-1.4 & src tar [1] * Skimmed over the contents of ReleaseNotes & ChangeLog * Checked the NOTICE & LICENSE files Thanx Steve for driving the release. Good Luck!!! -Ayush [1] I think releas

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

I am +1 too. * Verified signature. * Release notes and changelogs look good. On Fri, Mar 21, 2025 at 4:35 PM Chris Nauroth wrote: > +1 > > * Verified signatures. > * LICENSE-binary has the expected dependencies and versions. > * Site looks good too. > * Specifically confirmed inclusion of HADOO

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

I did try everything, no problem with the keys or so. -Ayush On Thu, 20 Mar 2025 at 23:21, Steve Loughran wrote: > thanks. will kick off a new build. > > Did anyone else see problems with the gpg versions? > > I destroyed the entire .gpgconf dir on the host i was building on; did > manual verif

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

This is committed to trunk and branch-1.4.0 now, ready for another RC. Thanks to Aayush for the code review. Chris Nauroth On Wed, Mar 19, 2025 at 1:51 PM Chris Nauroth wrote: > -1 > > Signatures look good, but I found a few spots that need an update for the > new protobuf version: > > https:/

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

-1 Signatures look good, but I found a few spots that need an update for the new protobuf version: https://issues.apache.org/jira/browse/HADOOP-19510 https://github.com/apache/hadoop-thirdparty/pull/44 Chris Nauroth On Tue, Mar 18, 2025 at 8:47 AM Wei-Chiu Chuang wrote: > I just did > gpg -

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

I just did gpg --verify .tar.gz.asc .tar.gz Ok after importing hadoop KEYS , this is the output which looks good to me: gpg: Signature made Fri Mar 14 08:56:01 2025 PDT gpg:using RSA key 38237EE425050285077DB57AD22CF846DBB162A0 gpg:

Re: Dropping support for builds on Centos 7

Thanks Steve. I agree that it's time to move off Java 8. The *trunk* and *branch-3.4* CIs are still broken because of the original issue, so there is also a short-term problem that needs to be fixed. If we're not ready to drop Centos7 from CI right now, my PR

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

Let me check that; getting signing set up on the transient vm was tricky as my yubikey one wasn't there so I used my other .. regarding the jersey -bigger piece off work to actually pick up, so not touching it. On Fri, 14 Mar 2025 at 21:46, Wei-Chiu Chuang wrote: > Looks like your GPG key expir

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

can you give me what command you ran to check the signature, and see if you can update all your keys from the central servers? My key thinks it is not expired, so maybe its the KEYS file i need to update or you need to refresh your keys. I will have a look at the keys file tomorrow On Fri, 14 Mar

Re: hadoop thirdparty 1.4.0

I believe it should be 1.4.0. We only did a x.x.1 kind of release once, when we couldn't release 1.1.0 [1] in the Hadoop 3.3.1 RC1 due to some issues. Instead, we issued 1.1.1 [2], which was ultimately used in 3.3.1. However, I can't exactly recall what was the issue which led to 1.1.1, but I think

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

Verified SHA-512 checksum, GPG signature -- all good except that the signing key expired. Confirmed git tag release-1.4.0-RC0 and branch name branch-1.4.0 both maps to commit hash 5595f1357eaad02e1cfb660bcce7fd34515197ff. Confirmed it is released as 1.4.0. Noticed HADOOP-19075

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

Looks like your GPG key expired? gpg: Signature made Fri Mar 14 08:56:00 2025 PDT gpg:using RSA key 38237EE425050285077DB57AD22CF846DBB162A0 gpg: Good signature from "Steve Loughran (ASF code sign key - 2018) < ste...@apache.org>" [expired] gpg: aka "[jpeg image of

Re: Dropping support for builds on Centos 7

I think we really need to push ahead on a 3.5 release for this stuff and for java 17. I'm doing the new thirdparty jar once I've got the docker building *and signing* it...I'm having to do the build in EC2 as it doesn't seem to work on my arm64 laptop right now. Then ahmar can start the 3.4.2 rel

Re: [DISCUSS] Integration of Volcano Engine TOS in Hadoop.

Thanks xiaoqiao for your nice suggestion, agree for a formal VOTE. About the jenkins report. The pr is too big (over 2 lines) for jenkins to apply the patch (https://github.com/apache/hadoop/pull/7194). My plan is to split it into 2 pull request. The first pr includes the core implementati

Re: JDK23/24 Support

CC-ing to common-dev@hadoop.apache.org for more exposure. On Fri, Mar 7, 2025 at 8:02 AM Istvan Toth wrote: > Hi! > > I've been working on JDK23 support for a few weeks, and now I have a > mostly working patch (a few test failures may still happen). > > The existing HADOOP-19212 JIRA and its ori

Re: [DISCUSS] Integration of Volcano Engine TOS in Hadoop.

Thanks Jinglun for your work. LGTM. +1 from my side. BTW, Please check Jenkins report, it is better to get +1 from Jenkins before check in. cc @PJ Fanning @Steve Loughran @slfan1989 would you mind taking another check? If there are no more other comments or concerns, I just suggest launching ano

Re: Dropping support for builds on Centos 7

Thanks everyone for contributing. To summarize the discussion on YARN-11794: We can fix the build on 3.4 by using an older plugin binary (which does not have any changes related to the generated code compared to the current one), or by committing the generated Java code. Shilun Fan and I prefer th

Re: Dropping support for builds on Centos 7

"Support for any OS SHOULD NOT be dropped without first being documented as deprecated for a full major release and MUST NOT be dropped without first being deprecated for at least a full minor release." I am not aware of any distro being explicitly deprecated by Hadoop, I don't think that the inte

Re: Dropping support for builds on Centos 7

Istvan, thank you for working on restoring build support on CentOS 7 on branch-3.4, let’s keep the high level discussion on the mailing list and implementation details on the PR directly. Thanks, Cheng Pan > On Mar 12, 2025, at 16:47, Istvan Toth wrote: > > "Support for any OS SHOULD NOT be

Re: Dropping support for builds on Centos 7

Hi Chris, thank you for detailed explanation, now I see the exact issue after reading the description of YARN-11794. I have full confidence in Java cross-platform nature, but given that Hadoop involves some native code, and requires some native libraries to build and run, I'm afraid I must buil

Re: Dropping support for builds on Centos 7

I'm adding Cheng Pan, who expressed concerns here: https://github.com/apache/hadoop/pull/7493#issuecomment-2710815286 Cheng, this is a build time issue. If it doesn't impact runtime, then is it still a problem for you? I can imagine it might still be an impact if you need to make a custom patched

Re: Dropping support for builds on Centos 7

CC to d...@hadoop.apache.org On Tue, Mar 11, 2025 at 5:51 AM Istvan Toth wrote: > Hi, > > As reported in YARN-11794 , building Hadoop on Centos 7 has been broken > for some time, and I don't see an easy way to fix it. > > I propose removing Centos 7 from the supported build platforms, and > remo

Re: [DISCUSS] Integration of Volcano Engine TOS in Hadoop.

**Update** Hello everyone, thanks a lot for your attention. I'm happy to share the progress here. 1. Kotlin is removed. The library ve-tos-java-sdk has been updated to ve-tos-java-sdk-hadoop (https://github.com/volcengine/ve-tos-java-sdk/tree/ve-tos-java-sdk-hadoop), which is developed based

Re: hadoop thirdparty 1.4.0

On Wed, 5 Mar 2025 at 20:20, PJ Fanning wrote: > Due to the jar name change for protobuf that is merged but unreleased, > I think that 1.4.0 is better. > https://github.com/apache/hadoop-thirdparty/pull/38 OK > > > Is there a reason to not use the pre-existing Avro PR which seems to > have the

Re: hadoop thirdparty 1.4.0

Due to the jar name change for protobuf that is merged but unreleased, I think that 1.4.0 is better. https://github.com/apache/hadoop-thirdparty/pull/38 Is there a reason to not use the pre-existing Avro PR which seems to have the same changes? https://github.com/apache/hadoop-thirdparty/pull/40

Re: Re: Fwd: [DISCUSS] Plans for JDK22 / SecurityManager removal

Hi, Nihal Thank you very much for proposing the Jetty upgrade plan! I’d also like to share the latest progress on Hadoop's JDK17 upgrade. As of now, the trunk branch can be successfully compiled with JDK17, though there are still 2-3 critical JIRA issues that need to be addressed, and we are activ

Re: Fwd: [DISCUSS] Plans for JDK22 / SecurityManager removal

> I think that the best course of action would be adding a separate Jetty11 module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both used/updated by the 2.x / 3.x branches. +1, I would like to volunteer for this work. Maintaining Jetty 9.4.x could be quite challenging from a CVE perspec

Re: [DISCUSS] Hadoop 3.4.2 release

+1, thanx Ahmar for driving it. I plan to c-pick YARN-11738 to branch-3.4, if folks are ok with it, that is need by Tez, or atleast makes life easier there -Ayush On Sun, 2 Mar 2025 at 01:30, Chao Sun wrote: > +1, thanks Ahmar! > > On Fri, Feb 28, 2025 at 7:21 AM larry mccay wrote: > > > +1 >

Re: [DISCUSS] Hadoop 3.4.2 release

+1, thanks Ahmar! On Fri, Feb 28, 2025 at 7:21 AM larry mccay wrote: > +1 > > Thanks, Ahmar! > > On Fri, Feb 28, 2025 at 7:37 AM slfan1989 wrote: > > > +1 > > > > I can provide assistance with the ARM release, as I originally planned to > > release Hadoop 3.5.0, which supports JDK17, and have b

Re: [DISCUSS] Hadoop 3.4.2 release

+1 Thanks, Ahmar! On Fri, Feb 28, 2025 at 7:37 AM slfan1989 wrote: > +1 > > I can provide assistance with the ARM release, as I originally planned to > release Hadoop 3.5.0, which supports JDK17, and have been conducting > related tests. > > My approach is a workaround, and I hope it helps. > >

Re: [DISCUSS] Hadoop 3.4.2 release

+1 I can provide assistance with the ARM release, as I originally planned to release Hadoop 3.5.0, which supports JDK17, and have been conducting related tests. My approach is a workaround, and I hope it helps. Best Regards, - Shilun Fan On Thu, Feb 27, 2025 at 9:08 PM Bhavik Patel wrote: > H

Re: [DISCUSS] Hadoop 3.4.2 release

If anyone can help with this, it'd be great. Mukund couldn't get the arm docker image to do a build On Thu, 27 Feb 2025 at 09:38, Cheng Pan wrote: > Thanks Ahmar for preparing a new release. > > The aarch64 binary distribution tarball is absent in 3.4.1, better to > restore in this release. > >

Re: [DISCUSS] Hadoop 3.4.2 release

Hello PMCs, Can we do a release for hadoop-thirdparty 1.4.0 or 1.3.1 with the fix for CVE-2024-7254(HADOOP-19289) and bump hadoop-thirdparty version in hadoop-3.4.2? On Thu, 27 Feb 2025, 5:15 pm Anuj Modi, wrote: > +1 > Thanks for initiative Ahmar. > > For ABFS we would like to highlight two ma

Re: [DISCUSS] Hadoop 3.4.2 release

+1 Thanks for initiative Ahmar. For ABFS we would like to highlight two major changes in the upcoming release. 1. Announcing Deprecation of WASB driver: Jira , Documentation

Re: [DISCUSS] Hadoop 3.4.2 release

Thanks Ahmar for preparing a new release. The aarch64 binary distribution tarball is absent in 3.4.1, better to restore in this release. Thanks, Cheng Pan On 2025/02/27 05:38:38 Muralikrishna Dmmkr wrote: > Thanks Ahmar for initating discussion, Can we do a release for > hadoop-thirdparty 1.4.0

Re: [DISCUSS] Hadoop 3.4.2 release

Thanks Ahmar for initating discussion, Can we do a release for hadoop-thirdparty 1.4.0 or 1.3.1 with the fix for CVE-2024-7254(HADOOP-19289) and bump hadoop-thirdparty version in hadoop-3.4? Thanks, Murali krishna On Thu, 27 Feb 2025 at 7:56 AM, Xiaoqiao He wrote: > +1. Thanks Ahmar. > > On Thu

Re: [DISCUSS] Hadoop 3.4.2 release

+1. Thanks Ahmar. On Thu, Feb 27, 2025 at 2:31 AM Chris Nauroth wrote: > +1 > > Thank you, Ahmar! > > Chris Nauroth > > > On Wed, Feb 26, 2025 at 8:31 AM Ahmar Suhail wrote: > > > Hey all, > > > > Would like to start a discussion for a Hadoop 3.4.2 release. > > > > For S3A, we would like to rel

Re: [DISCUSS] Hadoop 3.4.2 release

+1 Thank you, Ahmar! Chris Nauroth On Wed, Feb 26, 2025 at 8:31 AM Ahmar Suhail wrote: > Hey all, > > Would like to start a discussion for a Hadoop 3.4.2 release. > > For S3A, we would like to release support for conditional writes, and > integration of a new analytics input stream. > > I hav

Re: [VOTE] Request to merge branch HDFS-17531 into trunk.

; common-dev@hadoop.apache.org ; Hui Fei < > feihui.u...@gmail.com>; hfutzhan...@163.com > *Subject:* Re: [VOTE] Request to merge branch HDFS-17531 into trunk. > > +1 > > Best Regards, > - Shilun Fan > > On Fri, Feb 14, 2025 at 3:22 PM jian zhang wrote: > > > Hi, al

Re: [DISCUSS] Integration of Volcano Engine TOS in Hadoop.

**Update** I discussed with the main tos sdk developer xiang(evansxi...@126.com), he would be happy to provide a new sdk depends on apache http-client. The new sdk would be released at 3/15. Thanks xiang for your help. If there are any other questions, please feel free to comment. On 2025/0

Re: [VOTE] Request to merge branch HDFS-17531 into trunk.

; Hui Fei ; hfutzhan...@163.com Subject: Re: [VOTE] Request to merge branch HDFS-17531 into trunk. +1 Best Regards, - Shilun Fan On Fri, Feb 14, 2025 at 3:22 PM jian zhang wrote: > Hi, all, the development of the asynchronous router functionality has been > completed. Thanks to a

Re: [DISCUSS] Integration of Volcano Engine TOS in Hadoop.

Thanks PJ Fanning and slfan for your suggestions ! > Would it be possible to consider using a lightweight HTTP client instead? Thanks for reminding me, this makes sense to me. I'll try to solve it as soon as possible. > We are upgrading to Junit5 and need TOS's Unit Tests to be developed in the

Re: [VOTE] Request to merge branch HDFS-17531 into trunk.

t point > does not exist, etc.), it directly puts the response into the response > queue; otherwise, it sends the request to the asynchronous handler thread > pool. > - Async Handler: Puts the request into the call queue > (connection.calls) of the connection thread and returns immediately w

Re: [DISCUSS] Integration of Volcano Engine TOS in Hadoop.

Thanks to Jinglun for initiating the discussion on TOS. +1 from my personal perspective. However, considering that we are upgrading to Junit5 and need TOS's Unit Tests to be developed in the Junit5 way, we can discuss it together under the relevant PR. Regarding PJ Fanning's suggestion, I think

Re: [DISCUSS] Request to merge branch HDFS-17531 into trunk.

feature. >> >> >> >> Best Regards, >> >> Jian Zhang >> >> >> >> Zhanghaobo 于2025年1月22日周三 18:22写道： >> >> >> >>> @Hui Fei Hi, Sir: >> >>> For the first opinion, I have create an umbrella JIRA >&

Re: [DISCUSS] Request to merge branch HDFS-17531 into trunk.

>> >> >>> >> Best Regards, >>> >> Jian Zhang >>> >> >>> >> Zhanghaobo 于2025年1月22日周三 18:22写道： >>> >> >>> >>> @Hui Fei Hi, Sir: >>> >>> For the first opinion, I have create an umbr

Re: [VOTE] Request to merge branch HDFS-17531 into trunk.

+1 - Zhang Haobo Replied Message | From | Xiaoqiao He | | Date | 02/14/2025 19:38 | | To | Ayush Saxena | | Cc | Hui Fei , jian zhang , Hdfs-dev , , | | Subject | Re: [VOTE] Request to merge branch HDFS-17531 into trunk. | +1 Best Regards, - He Xiaoqiao On Fri, Feb 14, 2025

Re: [DISCUSS] Integration of Volcano Engine TOS in Hadoop.

Just one thing to note is that we recently removed or reduced the okhttp3 dependency in Hadoop because the kotlin dependency brings in big jars and more complicated management of transitive dependencies. Would it be possible to consider using a lightweight HTTP client instead? The built-in Java cli

Re: [DISCUSS] Integration of Volcano Engine TOS in Hadoop.

Thanks xiaoqiao and steve for your attention and comments. Let me answer the dependencies and tests. **Dependencies** Hadoop-tos involves a new dependency com.volcengine:ve-tos-java-sdk:2.8.6. It is an open source project with apache 2.0 license (https://github.com/volcengine/ve-tos-java-sdk/b

Re: [VOTE] Request to merge branch HDFS-17531 into trunk.

equests from the request queue for preliminary > >> processing. If there are exceptions in the request (such as the mount > point > >> does not exist, etc.), it directly puts the response into the response > >> queue; otherwise, it sends the request to the asynchr

Re: [VOTE] Request to merge branch HDFS-17531 into trunk.

response into the response >> queue; otherwise, it sends the request to the asynchronous handler thread >> pool. >> - Async Handler: Puts the request into the call queue >> (connection.calls) of the connection thread and returns immediately without >> blocking and waiting.

Re: [VOTE] Request to merge branch HDFS-17531 into trunk.

queue; otherwise, it sends the request to the asynchronous handler thread > pool. > - Async Handler: Puts the request into the call queue > (connection.calls) of the connection thread and returns immediately without > blocking and waiting. > - Async Responder: Is responsible for pr

Re: [DISCUSS] Request to merge branch HDFS-17531 into trunk.

t; Zhanghaobo 于2025年1月22日周三 18:22写道： >> >> >> >>> @Hui Fei Hi, Sir: >> >>> For the first opinion, I have create an umbrella JIRA >> >>> https://issues.apache.org/jira/browse/HDFS-17716 >> >>> and move non-core JIRA under i

Re: [DISCUSS] Integration of Volcano Engine TOS in Hadoop.

Sounds good, though expect no commitment from me to review anything. My main concerns are about dependency libraries (what are they?) and testing. On Tue, 11 Feb 2025 at 05:10, Xiaoqiao He wrote: > Thanks Jinglun for your work. Basically +1 from me to involve it into the > Hadoop codebase. > a.

Re: [DISCUSS] Request to merge branch HDFS-17531 into trunk.

16 > >>> and move non-core JIRA under it. > >>> > >>> Best Wishes > >>> Haobo Zhang > >>> > >>> Replied Message > >>> From Hui Fei > >>> Date 01/22/2025 17:37 > >>> To jian zhang > >>> Cc H

Re: [DISCUSS] Request to merge branch HDFS-17531 into trunk.

ve non-core JIRA under it. >>> >>> Best Wishes >>> Haobo Zhang >>> >>> Replied Message >>> From Hui Fei >>> Date 01/22/2025 17:37 >>> To jian zhang >>> Cc Hdfs-dev , >>> , >>>

Re: [DISCUSS] Integration of Volcano Engine TOS in Hadoop.

Thanks Jinglun for your work. Basically +1 from me to involve it into the Hadoop codebase. a. After a quick review of JIRA and PR, I think it is solid including document and code style. b. Contributors involved here are diverse who are from different projects and companies, and active enough. c. Co

Re: Apache Hadoop 3.4.1 docs landing page

Thanks! I sent this update for review: https://issues.apache.org/jira/browse/HADOOP-19407 https://github.com/apache/hadoop/pull/7338 Please reply on the PR if there are any other 3.4.1 changes we should highlight. Chris Nauroth On Mon, Jan 27, 2025 at 10:46 AM Wei-Chiu Chuang wrote: > I alwa

Re: Apache Hadoop 3.4.1 docs landing page

I always forget to update the doc myself. Noted. Thank you Ayush. On Mon, Jan 27, 2025 at 10:35 AM Ayush Saxena wrote: > Hi Chris, > This page requires a manual update before the release. an update was made > for the 3.4.1 release in this commit [1], Not sure why the older links > weren't remove

Re: Apache Hadoop 3.4.1 docs landing page

Hi Chris, This page requires a manual update before the release. an update was made for the 3.4.1 release in this commit [1], Not sure why the older links weren't removed though. In case you want to explore more, you can check how it was handled for the 3.4.0 release in HADOOP-19039. The process i

Re: Error while running notifications feature from refs/heads/trunk:.asf.yaml in hadoop!

Some indent issue in the yaml file. Haven't spent time on it, but maybe changing the indentation like in [1] may fix it Not sure what triggered it all of sudden [1] https://github.com/apache/commons-bsf/pull/206 -Ayush On Mon, 27 Jan 2025 at 17:32, Steve Loughran wrote: > Not sure why I g

Re: [DISCUSS] Request to merge branch HDFS-17531 into trunk.

, , Xiaoqiao He , | | Subject | Re: [DISCUSS] Request to merge branch HDFS-17531 into trunk. | Got your idea. Thank you! - How about removing unfinished tasks and placing them under a new task as subtasks, like ARR improvements? If this feature is completed but there are still some open tasks, it

Re: [DISCUSS] Request to merge branch HDFS-17531 into trunk.

ervices are >>>> not fully utilized, leading to unreasonable resource allocation. >>>> >>>> III. Design and Improvements of the Asynchronous Router >>>> >>>> The asynchronous router solves the above problems by redesigning >>>

Re: [DISCUSS] Request to merge branch HDFS-17531 into trunk.

process and introducing an asynchronous processing >>> mechanism. Its core improvements include: >>> >>> - Handler: Retrieves requests from the request queue for preliminary >>> processing. If there are exceptions in the request (such as the mount point >&

Re: [DISCUSS] Request to merge branch HDFS-17531 into trunk.

nt point >> does not exist, etc.), it directly puts the response into the response >> queue; otherwise, it sends the request to the asynchronous handler thread >> pool. >> - Async Handler: Puts the request into the call queue >> (connection.calls) of the connec

Re: [DISCUSS] Request to merge branch HDFS-17531 into trunk.

d > pool. > - Async Handler: Puts the request into the call queue > (connection.calls) of the connection thread and returns immediately without > blocking and waiting. > - Async Responder: Is responsible for processing the responses > received by the connection thread. I

Re: [ANNOUNCE] Apache Hadoop 3.4.1 release

Pushed the hadoop site changes for Hadoop 3.4.1 along with lean jars. again and it looks good to me. It would be great if some of you can take a look at it when you have time. Thanks, Mukund On Thu, Nov 7, 2024 at 2:29 PM Mukund Madhav Thakur wrote: > `hugo server` does change the links for me.

Re: [ANNOUNCE] Apache Hadoop 3.4.1 release

`hugo server` does change the links for me. I have verified it. Not sure why it doesn't happen for you! Other than that, looking at removing the aarch64 links and adding the links for new lean jar is getting complicated. It is already broken right now for 3.3.2 to 3.3.4 [1] . This commit [2] has th

Re: [ANNOUNCE] Apache Hadoop 3.4.1 release

It isn't happening for me, I tried `hugo server` & the links stay intact, the baseUrl is specified in the config file [1], so it should be picked up from there, I saw some similar stuff on Stackoverflow [2], check if it helps Was checking the last commit, I think you have to drop the links to aarc

Re: [ANNOUNCE] Apache Hadoop 3.4.1 release

Thanks Ayush. Ah.. the `hugo server` to test locally updated all the hadoop.org to localhost and thus broke this. I have reverted my commit for now and the site is working fine. I have my local changes now which looks good to me. Is there any way we can get this reviewed? The trick is to create the

Re: [ANNOUNCE] Apache Hadoop 3.4.1 release

Hi Mukund, I think your commit broke the website, you have changed the references to hadoop.apache.org in the website to localhost [1], try clicking on Release 3.4.1 available here [2] or any other link & it will lead to resolve to some link with localhost. And Hadoop logo also went missing on the

Re: [ANNOUNCE] Apache Hadoop 3.4.1 release

Missed the hugo site generation part. It is done now. Thanks Xiaoqiao for pointing that out. On Sun, Nov 3, 2024 at 10:46 PM Xiaoqiao He wrote: > Hi Steve and Muhund, Any progress about 3.4.1 release documents on site? > Thanks. > > Best Regards, > - He Xiaoqiao > > On Fri, Oct 25, 2024 at 9:2

Re: [ANNOUNCE] Apache Hadoop 3.4.1 release

Hi Steve and Muhund, Any progress about 3.4.1 release documents on site? Thanks. Best Regards, - He Xiaoqiao On Fri, Oct 25, 2024 at 9:21 PM Steve Loughran wrote: > I thought we'd done that. > > Mukund is away this week; it'll be something we can fix next week. > > On Thu, 24 Oct 2024 at 07:06,

Re: [ANNOUNCE] Apache Hadoop 3.4.1 release

I thought we'd done that. Mukund is away this week; it'll be something we can fix next week. On Thu, 24 Oct 2024 at 07:06, Takanobu Asanuma wrote: > Thank you for your hard work in releasing 3.4.1! > > Could you also create the release documents on the Hadoop site? > - https://hadoop.apache.org

Re: [ANNOUNCE] Apache Hadoop 3.4.1 release

Thank you for your hard work in releasing 3.4.1! Could you also create the release documents on the Hadoop site? - https://hadoop.apache.org/ - https://hadoop.apache.org/releases.html Regards, - Takanobu 2024年10月18日(金) 20:26 Mukund Madhav Thakur : > On behalf of the Apache Hadoop Project Manage

Re: [VOTE] Release Apache Hadoop 3.4.1 RC3

Vote summary: +1 Xiaoqiao He (binding) Steve (binding) Ahmar (binding) Wei-Chiu Chuang (binding) Mukund (binding) Syed Shameerur Rahman (non-binding) Cheng Pan (non-binding) Dongjoon Hyun (non-binding) Sneha Vijayarajan (non-binding) =0 Ayush Saxena -1 None With 5 binding +1 votes and 4 non-b

Re: [VOTE] Release Apache Hadoop 3.4.1 RC3

Thanks Ayush for finding this issue. Looks like there is a problem with the ARM binaries. The reason for that could be that the create-release script doesn't work in ARM docker container but works manually after logging inside the container. So I was creating the tars manually. Seems like it didn't

Re: [VOTE] Release Apache Hadoop 3.4.1 RC3

Thanx Mukund for sharing the details, I might have missed that discussion. Do mention the purpose on the website once we upload the lean jar, so that relevant people can use it. I am holding my vote due to [3], that in general looks blocker to me, If I haven't messed up. Would be great if someone

  1   2   3   4   5   6   7   8   9   10   >