Re: [DISCUSS] Hadoop 3.4.2 release

2025-03-01 Thread Chao Sun
+1, thanks Ahmar! On Fri, Feb 28, 2025 at 7:21 AM larry mccay wrote: > +1 > > Thanks, Ahmar! > > On Fri, Feb 28, 2025 at 7:37 AM slfan1989 wrote: > > > +1 > > > > I can provide assistance with the ARM release, as I originally planned to > > release Hadoop 3.5.0, which supports JDK17, and have b

Re: [VOTE] Hadoop 3.2.x EOL

2023-12-06 Thread Chao Sun
+1 On Wed, Dec 6, 2023 at 8:39 AM Akira Ajisaka wrote: > > +1 > > > > On Wed, Dec 6, 2023 at 1:10 PM Xiaoqiao He wrote: > > > Dear Hadoop devs, > > > > Given the feedback from the discussion thread [1], I'd like to start > > an official thread for the community to vote on release line 3.2 EOL. >

Re: [VOTE] Release Apache Hadoop 3.3.5

2023-01-03 Thread Chao Sun
Hmm I'm looking at HADOOP-11867 related stuff but couldn't find it mentioned anywhere in change log or release notes. Are they actually up-to-date? On Mon, Jan 2, 2023 at 7:48 AM Masatake Iwasaki wrote: > > >- building HBase 2.4.13 and Hive 3.1.3 against 3.3.5 failed due to > > dependency ch

[jira] [Resolved] (HDFS-16686) GetJournalEditServlet fails to authorize valid Kerberos request

2022-09-13 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-16686. - Fix Version/s: 3.3.9 Hadoop Flags: Reviewed Resolution: Fixed > GetJournalEditServlet fa

[jira] [Resolved] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.

2022-08-17 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-4043. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Namenode Kerberos Login d

[jira] [Resolved] (HDFS-16702) MiniDFSCluster should report cause of exception in assertion error

2022-08-11 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-16702. - Hadoop Flags: Reviewed Resolution: Fixed > MiniDFSCluster should report cause of exception

Re: [VOTE] Release Apache Hadoop 3.3.4

2022-08-04 Thread Chao Sun
I'm still trying to get a successful run from Spark side here: https://github.com/apache/spark/pull/37281. The last CI run failed, but most likely due to some issue not related to Hadoop. On Thu, Aug 4, 2022 at 10:27 AM Steve Loughran wrote: > > The vote passed with the following result. > > Bind

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

2022-05-16 Thread Chao Sun
+1 - Compiled from source - Verified checksums & signatures - Launched a pseudo HDFS cluster and ran some simple commands - Ran full Spark tests with the RC Thanks Steve! Chao On Mon, May 16, 2022 at 2:19 AM Ayush Saxena wrote: > > +1, > * Built from source. > * Successful native build on Ubun

Re: [VOTE] Release Apache Hadoop 3.3.3

2022-05-09 Thread Chao Sun
; > > > > Akira > > > > > > On Sat, May 7, 2022 at 10:52 AM Ayush Saxena wrote: > > > > > >> Hmm, I see the artifacts ideally should have got overwritten by the new > > >> RC, but they didn’t. The reason seems like the staging path shared &g

Re: [VOTE] Release Apache Hadoop 3.3.3

2022-05-06 Thread Chao Sun
mail? > > On Sat, 7 May 2022 at 06:18, Chao Sun wrote: >> >> Seems there are some issues with the shaded client as I was not able >> to compile Apache Spark with the RC >> (https://github.com/apache/spark/pull/36474). Looks like it's compiled >> with t

Re: [VOTE] Release Apache Hadoop 3.3.3

2022-05-06 Thread Chao Sun
Seems there are some issues with the shaded client as I was not able to compile Apache Spark with the RC (https://github.com/apache/spark/pull/36474). Looks like it's compiled with the `-DskipShade` option and the hadoop-client-api JAR doesn't contain any class: ➜ hadoop-client-api jar tf 3.3.3/h

[jira] [Resolved] (HDFS-16507) [SBN read] Avoid purging edit log which is in progress

2022-03-30 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-16507. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > [SBN read] Avoid purg

Re: [ANNOUNCE] New Hadoop PMC Sun Chao

2022-03-09 Thread Chao Sun
Thanks all! I really appreciate all the help I received while working on Hadoop, and looking forward to contributing more to this project! A small correction: my given name is actually Chao. I made a mistake when adding my full name to the apache account :). Best, Chao On Tue, Mar 8, 2022 at 11:

Re: [ANNOUNCE] Apache Hadoop 3.3.2 release

2022-03-08 Thread Chao Sun
s, > - He Xiaoqiao > > [1] https://cwiki.apache.org/confluence/display/HADOOP2/HowToRelease > > On Sun, Mar 6, 2022 at 10:14 AM Chao Sun wrote: > >> Ah I forgot to do `hugo` before committing the changes. Sorry. It should >> be >> good now. >> >> Chao

Re: [ANNOUNCE] Apache Hadoop 3.3.2 release

2022-03-05 Thread Chao Sun
Ah I forgot to do `hugo` before committing the changes. Sorry. It should be good now. Chao On Sat, Mar 5, 2022 at 9:28 AM Chao Sun wrote: > Hmm interesting. I followed the steps in that second link you posted, > and the results looked good when I ran `hugo serve` locally: I can see

Re: [ANNOUNCE] Apache Hadoop 3.3.2 release

2022-03-05 Thread Chao Sun
uncement > about release 3.3.2. If we missed any steps to publish following > guidelines[2]? Thanks. > > Best Regards, > - He Xiaoqiao > > [1] https://hadoop.apache.org/ > [2] https://cwiki.apache.org/confluence/display/HADOOP2/HowToRelease > > > On Fri, Mar 4, 2022 at

[ANNOUNCE] Apache Hadoop 3.3.2 release

2022-03-03 Thread Chao Sun
Hi All, It gives me great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop 3.3.2. This is the second stable release of Apache Hadoop 3.3 line. It contains 284 bug fixes, improvements and enhancements since 3.3.1. Users are encouraged to read the overview o

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC5

2022-03-02 Thread Chao Sun
nd skimmed the > contents. > > Thanks, > Masatake Iwasaki > > On 2022/02/22 14:17, Chao Sun wrote: > > Hi all, > > > > Here's Hadoop 3.3.2 release candidate #5: > > > > The RC is available at: > http://people.apache.org/~sunchao/hadoop-3.3.2-RC

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC4

2022-02-21 Thread Chao Sun
Hi all, I just started another voting thread for RC5. Please evaluate and vote there. Thanks! On Mon, Feb 21, 2022 at 10:46 AM Chao Sun wrote: > Ah I take it back: the CHANGELOG is part of the hadoop-3.3.2-site.tar.gz > too. I'll start a new RC with the only change being the CHANGE

[VOTE] Release Apache Hadoop 3.3.2 - RC5

2022-02-21 Thread Chao Sun
Hi all, Here's Hadoop 3.3.2 release candidate #5: The RC is available at: http://people.apache.org/~sunchao/hadoop-3.3.2-RC5 The RC tag is at: https://github.com/apache/hadoop/releases/tag/release-3.3.2-RC5 The Maven artifacts are staged at: https://repository.apache.org/content/repositories/orga

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC4

2022-02-21 Thread Chao Sun
s. >>> I am curious if this requires new RC or you would like to update >>> Changelog >>> in the existing RC. Either way, once Changelog is updated, and since >>> other >>> functional testing looks good anyways, my vote would be +1 (non-binding). >>> &g

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC4

2022-02-21 Thread Chao Sun
nd since other >> functional testing looks good anyways, my vote would be +1 (non-binding). >> >> >> On Tue, 15 Feb 2022 at 11:27 PM, Chao Sun wrote: >> >> > Thanks again Viraj! >> > >> > > 1. Incorrect fix versions: HADOOP-17988, H

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC4

2022-02-15 Thread Chao Sun
7728 > > 2. HADOOP-17873 is not yet resolved but the corresponding commit is > present. > > 3. Resolved with 3.3.2 fixversion but no no corresponding commit found on > 3.3.2: > HADOOP-17936 > HADOOP-18066 > > > > On Mon, Feb 14, 2022 at 11:49 PM Chao Sun wro

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC4

2022-02-14 Thread Chao Sun
HADOOP-17290 > HADOOP-17819 > YARN-9551 > > > > On Mon, Feb 14, 2022 at 11:10 AM Viraj Jasani wrote: > > > Thanks Chao. Yes contents are now present, I will start the testing. > > > > > > On Mon, 14 Feb 2022 at 2:13 AM, Chao Sun wrote: > > > >>

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC4

2022-02-13 Thread Chao Sun
On Sat, Feb 12, 2022 at 3:12 AM Chao Sun wrote: > >> Hi all, >> >> Sorry for the delay! was waiting for a few fixes that people have >> requested >> to backport. I've just put together Hadoop 3.3.2 RC4 below: >> >> The RC is available at:

[VOTE] Release Apache Hadoop 3.3.2 - RC4

2022-02-11 Thread Chao Sun
Hi all, Sorry for the delay! was waiting for a few fixes that people have requested to backport. I've just put together Hadoop 3.3.2 RC4 below: The RC is available at: http://people.apache.org/~sunchao/hadoop-3.3.2-RC4/ The RC tag is at: https://github.com/apache/hadoop/releases/tag/release-3.3.2

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC3

2022-01-28 Thread Chao Sun
t; > Thanks and regards, > > Akira > > > > On Fri, Jan 28, 2022 at 1:39 PM Masatake Iwasaki < > iwasak...@oss.nttdata.co.jp <mailto:iwasak...@oss.nttdata.co.jp>> wrote: > > > > Thanks for putting this up, Chao Sun. > > > > I got following erro

[VOTE] Release Apache Hadoop 3.3.2 - RC3

2022-01-26 Thread Chao Sun
Hi all, I've put together Hadoop 3.3.2 RC3 below: The RC is available at: http://people.apache.org/~sunchao/hadoop-3.3.2-RC3/ The RC tag is at: https://github.com/apache/hadoop/releases/tag/release-3.3.2-RC3 The Maven artifacts are staged at: https://repository.apache.org/content/repositories/org

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC2

2022-01-25 Thread Chao Sun
org.restlet:jar:2.3.0: Could not transfer artifact > > > org.restlet.jee:org.restlet:pom:2.3.0 from/to > maven-default-http-blocker > > ( > > > http://0.0.0.0/): Blocked mirror for repositories: [maven-restlet ( > > > http://maven.restlet.org, default, releases+snapshots),

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC2

2022-01-24 Thread Chao Sun
Thanks for the update Steve! Mukund: could you please share the command to get the error above? I tried a few approaches but couldn't reproduce it :( Thanks again! Best, Chao On Mon, Jan 24, 2022 at 7:16 AM Steve Loughran wrote: > > fix is in t disable auditing, which is now the default > htt

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC2

2022-01-22 Thread Chao Sun
luster w/ NN, 1 DN, RM, AHS, JHS, and > 3 > > NMs > > > > - Validated inter- and intra-queue preemption > > > > - Validated exclusive node labels > > > > Thanks a lot Chao for your diligence and hard work on this release. > > > > Eric > >

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC2

2022-01-21 Thread Chao Sun
=6 > > > > > > all happy > > > > > > > > > > > > *## binary* > > > downloaded KEYS and imported, so adding your key to my list (also > signed > > > this and updat

[VOTE] Release Apache Hadoop 3.3.2 - RC2

2022-01-19 Thread Chao Sun
Hi all, I've put together Hadoop 3.3.2 RC2 below: The RC is available at: http://people.apache.org/~sunchao/hadoop-3.3.2-RC2/ The RC tag is at: https://github.com/apache/hadoop/releases/tag/release-3.3.2-RC2 The Maven artifacts are staged at: https://repository.apache.org/content/repositories/org

[jira] [Resolved] (HDFS-16410) Insecure Xml parsing in OfflineEditsXmlLoader

2022-01-05 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-16410. - Fix Version/s: 3.4.0 3.3.2 Resolution: Fixed > Insecure Xml parsing

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC0

2022-01-04 Thread Chao Sun
't know how long that will take to stabilize. Sounds good. Hope everyone had a good holiday break! I'll address the above issues and create RC1 soon. Thanks, Chao On Thu, Dec 30, 2021 at 6:00 AM Steve Loughran wrote: > On Tue, 14 Dec 2021 at 22:56, Chao Sun wrote: > >

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC0

2021-12-14 Thread Chao Sun
e Loughran wrote: > I'll do my best to test this; I'm a bit broken right now. > > I think we should mention in a release notes that is the version of a > log4j included in this and all previous releases is not vulnerable. But > provide a list plus links to any that have bee

[VOTE] Release Apache Hadoop 3.3.2 - RC0

2021-12-09 Thread Chao Sun
Hi all, Sorry for the long delay. I've prepared RC0 for Hadoop 3.3.2 below: The RC is available at: http://people.apache.org/~sunchao/hadoop-3.3.2-RC0/ The RC tag is at: https://github.com/apache/hadoop/releases/tag/release-3.3.2-RC0 The Maven artifacts are staged at: https://repository.apache.or

Re: [DISCUSS] Hadoop 3.3.2 release?

2021-11-16 Thread Chao Sun
Hi, I just created a release branch branch-3.3.2 and updated the version of branch-3.3 to 3.3.3-SNAPSHOT. Will try to get the RC1 ready soon. On Thu, Oct 28, 2021 at 9:58 AM Chao Sun wrote: > Hi, sorry for the delay! I plan to start the release process within a few > days. > > On

Re: [DISCUSS] Hadoop 3.3.2 release?

2021-10-28 Thread Chao Sun
Hi, sorry for the delay! I plan to start the release process within a few days. On Thu, Oct 28, 2021 at 3:25 AM Hui Fei wrote: > Thank you, Chao! > Do you need any help? I can assist you. > > Chao Sun 于2021年9月8日周三 上午1:06写道: > >> Hi all, >> >> It has been almost

Re: [DISCUSS] Hadoop 3.3.2 release?

2021-09-09 Thread Chao Sun
t; > On Tue, 7 Sep 2021 at 10:36 PM, Chao Sun wrote: > >> Hi all, >> >> It has been almost 3 months since the 3.3.1 release and branch-3.3 has >> accumulated quite a few commits (118 atm). In particular, Spark community >> recently found an issue which prevents o

[DISCUSS] Hadoop 3.3.2 release?

2021-09-07 Thread Chao Sun
Hi all, It has been almost 3 months since the 3.3.1 release and branch-3.3 has accumulated quite a few commits (118 atm). In particular, Spark community recently found an issue which prevents one from using the shaded Hadoop client together with certain compression codecs such as lz4 and snappy co

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

2021-06-10 Thread Chao Sun
+1 (non-binding) - verified signature and checksum - launched a single docker based cluster and ran some simple HDFS commands - build Spark master with 3.3.1 RC and : 1) run full Spark test suites and all success; 2) tested simple Spark commands against a S3 endpoint; 3) tested Spark on YARN with

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

2021-06-03 Thread Chao Sun
Hi Wei-Chiu, It seems the Maven staging repository is still pointing at RC0: https://repository.apache.org/content/repositories/staging/org/apache/hadoop/hadoop-client/3.3.1/, and it's probably because you need to drop the old RCs in Apache Nexus server. Could you do that? Thanks. Chao On Thu, J

Re: [DISCUSS] Change project style guidelines to allow line length 100

2021-05-20 Thread Chao Sun
+1. I think this will be very helpful. On Thu, May 20, 2021 at 3:10 PM Sangjin Lee wrote: > +1 (binding). It's long overdue IMHO. > > On Thu, May 20, 2021 at 2:11 PM Gergely Pollak > wrote: > > > I really like this initiative, thank you! > > +1 for line length increase to 100 characters. > > >

[jira] [Resolved] (HDFS-15754) Create packet metrics for DataNode

2021-01-07 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-15754. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Create packet metrics

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC5

2021-01-07 Thread Chao Sun
+1 (non-binding) - verified checksums & signatures - started a single-node cluster in docker and did some simple tests, also checked NN web-ui - Did a full test run in Spark using the RC and it looks good On Thu, Jan 7, 2021 at 3:01 AM 孙立晟 wrote: > +1 (non-binding) > > - Build from source with

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC4

2020-12-16 Thread Chao Sun
mat#testMissingBlocks fails > (#2136) > > HDFS-13404. Addendum: RBF: > > TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fail. > > Contributed by Takanobu Asanuma. > > HADOOP-16080. hadoop-aws does not work with hadoop-client-api (#2510). > > Contributed

Re: Regarding HDFS-15567. HDFS should expose msync() API to allow downstream applications call it explicitly

2020-12-13 Thread Chao Sun
t's certainly in the 3.2 branch. Will >> check >> > further. If it's in the release I was thinking of adding a warning in >> the >> > notes "unstable API"; stable if invoked from DFSClient >> >> On Fri, 11 Dec 2020 at 18:21, Chao Sun wro

Re: Regarding HDFS-15567. HDFS should expose msync() API to allow downstream applications call it explicitly

2020-12-11 Thread Chao Sun
I'm just curious why this is included in the 3.2.2 release? HDFS-15567 is tagged with 3.2.3 and the corresponding HDFS-14272 on server side is tagged with 3.3.0. > If it goes into FS/FC, what does it do for a viewfs with >1 mounted HDFS? Should it take path, msync(path) so that viewFS knows where

[jira] [Resolved] (HDFS-15708) TestURLConnectionFactory fails by NoClassDefFoundError in branch-3.3 and branch-3.2

2020-12-04 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-15708. - Fix Version/s: 3.3.1 3.2.2 Hadoop Flags: Reviewed Resolution: Fixed

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC3

2020-12-03 Thread Chao Sun
Thanks Xiaoqiao. I just committed the fix to branch-3.2 (and will port it to other branches as well later). Could you cherry-pick it to branch-3.2.2 and start another RC? really appreciate it. Best, Chao On Wed, Dec 2, 2020 at 7:59 PM Xiaoqiao He wrote: > Thanks Chao Sun for your qu

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC3

2020-12-01 Thread Chao Sun
Thanks Xiaoqiao for the work! Unfortunately we just discovered an issue (HADOOP-16080) which prevents Spark from consuming this release. I'm working on a PR for this and wondering if we can include the fix in the release too. Thanks again and apologies for the late notice. Best, Chao On Mon, Nov

[jira] [Resolved] (HDFS-15690) Add lz4-java as hadoop-hdfs test dependency

2020-11-21 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-15690. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Committed to trunk. I&#x

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC2

2020-11-13 Thread Chao Sun
+1 (non-binding) from me as well: - download the source and build it on mac successfully - verified all md5, sha512 as well as signatures, however @Xiaoqiao I was not able to find your public key in http://svn.apache.org/repos/asf/hadoop/common/dist/KEYS so had to use the other link. - started a s

Re: Wire compatibility between Hadoop 3.x client and 2.x server

2020-10-20 Thread Chao Sun
Fri, 16 Oct 2020 at 19:59, Chao Sun wrote: > >> Thanks for the replies all! >> >> > But in the opposite case, it might have problem, because 2.x server may >> not understand new client calls added in 3.x >> >> Yes not expecting this to work. I'm

Re: Wire compatibility between Hadoop 3.x client and 2.x server

2020-10-16 Thread Chao Sun
; > > On Mon, Oct 12, 2020 at 8:59 PM Wu,Jianliang(vip.com) < > > jianliang...@vipshop.com> wrote: > > > >> In our case, when nn has upgraded to 3.1.3 and dn’s version was still > >> 2.6, we found hive to call getContentSummary method , the client and > >

Wire compatibility between Hadoop 3.x client and 2.x server

2020-10-12 Thread Chao Sun
Hi community, Does Hadoop 3.x provide wire compatibility between 3.x clients and 2.x server? There is a blog post from Cloudera [1] mentioning wire compatibility between 2.x clients and 3.x server, but not the other direction. Curious if someone knows this. Also it'd be good to know if someone is

[jira] [Created] (HDFS-15601) Batch listing: gracefully fallback to use non-batched listing when NameNode doesn't support the feature

2020-09-25 Thread Chao Sun (Jira)
Chao Sun created HDFS-15601: --- Summary: Batch listing: gracefully fallback to use non-batched listing when NameNode doesn't support the feature Key: HDFS-15601 URL: https://issues.apache.org/jira/browse/HDFS-

[jira] [Resolved] (HDFS-15014) RBF: WebHdfs chooseDatanode shouldn't call getDatanodeReport

2020-07-29 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-15014. - Resolution: Duplicate > RBF: WebHdfs chooseDatanode shouldn't call getDatano

[jira] [Resolved] (HDFS-15465) Support WebHDFS accesses to the data stored in secure Datanode through insecure Namenode

2020-07-27 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-15465. - Fix Version/s: 3.4.0 Resolution: Fixed > Support WebHDFS accesses to the data stored in sec

Re: [ANNOUNCE] New Apache Hadoop PMC member : Ayush Saxena

2020-07-24 Thread Chao Sun
Congratulations Ayush! On Fri, Jul 24, 2020 at 6:43 AM Rahul Gupta wrote: > Congratulations > > On Thu, Jul 23, 2020 at 8:02 PM HarshaKiran Reddy Boreddy < > bharsh...@gmail.com> wrote: > > > Congratulations Ayush Saxena..!!! > > > > On Thu, Jul 23, 2020, 8:39 AM Xiaoqiao He wrote: > > > > > Co

[jira] [Created] (HDFS-15423) RBF: WebHDFS create shouldn't choose DN from all sub-clusters

2020-06-19 Thread Chao Sun (Jira)
Chao Sun created HDFS-15423: --- Summary: RBF: WebHDFS create shouldn't choose DN from all sub-clusters Key: HDFS-15423 URL: https://issues.apache.org/jira/browse/HDFS-15423 Project: Hadoop

Re: [ANNOUNCE] New Apache Hadoop Committer - He Xiaoqiao

2020-06-11 Thread Chao Sun
Congrats Xiaoqiao! On Thu, Jun 11, 2020 at 9:36 AM Ayush Saxena wrote: > Congratulations He Xiaoqiao!!! > > -Ayush > > > On 11-Jun-2020, at 9:30 PM, Wei-Chiu Chuang wrote: > > > > In bcc: general@ > > > > It's my pleasure to announce that He Xiaoqiao has been elected as a > > committer on the

[jira] [Created] (HDFS-15335) Report top N metrics for files in get listing ops

2020-05-05 Thread Chao Sun (Jira)
Chao Sun created HDFS-15335: --- Summary: Report top N metrics for files in get listing ops Key: HDFS-15335 URL: https://issues.apache.org/jira/browse/HDFS-15335 Project: Hadoop HDFS Issue Type

Re: HDFS-13616 : batch listing of multiple directories

2020-03-03 Thread Chao Sun
Hi Steve, Thanks for your valuable feedback and apologies for overlooking the needs from object store and others! I'll look into the PR. > Then, after 3.3.0 is out, someone gets to do the FileSystem implementation, with specification, tests etc. Not me -you are the HDFS team -of course you can do

Re: Next Hadoop Storage Online Meetup (APAC Mandarin)

2019-12-20 Thread Chao Sun
We are also very interested to learn from this experience, and I can help on the translation too. Please let me know if help needed. Best, Chao On Fri, Dec 20, 2019 at 9:38 AM Chen Liang wrote: > Thanks again for driving the syncup Wei-Chiu! I am also super interested > in learning about this

[jira] [Created] (HDFS-15029) RBF: Supporting batched listing

2019-12-02 Thread Chao Sun (Jira)
Chao Sun created HDFS-15029: --- Summary: RBF: Supporting batched listing Key: HDFS-15029 URL: https://issues.apache.org/jira/browse/HDFS-15029 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-15015) Backport HDFS-5040 to branch-2

2019-11-26 Thread Chao Sun (Jira)
Chao Sun created HDFS-15015: --- Summary: Backport HDFS-5040 to branch-2 Key: HDFS-15015 URL: https://issues.apache.org/jira/browse/HDFS-15015 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-15014) [RBF] WebHdfs chooseDatanode shouldn't call getDatanodeReport

2019-11-26 Thread Chao Sun (Jira)
Chao Sun created HDFS-15014: --- Summary: [RBF] WebHdfs chooseDatanode shouldn't call getDatanodeReport Key: HDFS-15014 URL: https://issues.apache.org/jira/browse/HDFS-15014 Project: Hadoop

[jira] [Created] (HDFS-15005) Backport HDFS-12300 to branch-2

2019-11-21 Thread Chao Sun (Jira)
Chao Sun created HDFS-15005: --- Summary: Backport HDFS-12300 to branch-2 Key: HDFS-15005 URL: https://issues.apache.org/jira/browse/HDFS-15005 Project: Hadoop HDFS Issue Type: Bug

[jira] [Reopened] (HDFS-14034) Support getQuotaUsage API in WebHDFS

2019-08-07 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reopened HDFS-14034: - Re-opening this for backporting to branch-2. > Support getQuotaUsage API in WebH

[jira] [Resolved] (HDFS-14671) WebHDFS: Add erasureCodingPolicy to ContentSummary

2019-07-31 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-14671. - Resolution: Duplicate > WebHDFS: Add erasureCodingPolicy to ContentSumm

[jira] [Created] (HDFS-14671) WebHDFS: Add erasureCodingPolicy to ContentSummary

2019-07-25 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14671: --- Summary: WebHDFS: Add erasureCodingPolicy to ContentSummary Key: HDFS-14671 URL: https://issues.apache.org/jira/browse/HDFS-14671 Project: Hadoop HDFS Issue Type: Sub

[jira] [Resolved] (HDFS-14110) NPE when serving http requests while NameNode is starting up

2019-07-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-14110. - Resolution: Duplicate > NPE when serving http requests while NameNode is starting

[jira] [Created] (HDFS-14660) [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider

2019-07-17 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14660: --- Summary: [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider Key: HDFS-14660 URL: https://issues.apache.org/jira/browse/HDFS-14660

Re: HDFS Performance Improvement JIRAs: reviewers wanted

2019-06-24 Thread Chao Sun
Thanks Wei-Chiu for sharing this great list! I'll try to help on the reviewing as well. I think it may be worthwhile to also maintain a list like this in the long term. Some kind of dashboard showing what are the patches pending to be reviewed, what patches are stale, etc, in different categories.

[jira] [Resolved] (HDFS-13189) Standby NameNode should roll active edit log when checkpointing

2019-05-01 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-13189. - Resolution: Duplicate > Standby NameNode should roll active edit log when checkpoint

[jira] [Created] (HDFS-14415) Backport HDFS-13799 to branch-2

2019-04-04 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14415: --- Summary: Backport HDFS-13799 to branch-2 Key: HDFS-14415 URL: https://issues.apache.org/jira/browse/HDFS-14415 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-14399) Backport HDFS-10536 to branch-2

2019-03-29 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14399: --- Summary: Backport HDFS-10536 to branch-2 Key: HDFS-14399 URL: https://issues.apache.org/jira/browse/HDFS-14399 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-14397) Backport HADOOP-15684 to branch-2

2019-03-28 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14397: --- Summary: Backport HADOOP-15684 to branch-2 Key: HDFS-14397 URL: https://issues.apache.org/jira/browse/HDFS-14397 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-14392) Backport HDFS-9787 to branch-2

2019-03-27 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14392: --- Summary: Backport HDFS-9787 to branch-2 Key: HDFS-14392 URL: https://issues.apache.org/jira/browse/HDFS-14392 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-14391) Backport HDFS-9659 to branch-2

2019-03-27 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14391: --- Summary: Backport HDFS-9659 to branch-2 Key: HDFS-14391 URL: https://issues.apache.org/jira/browse/HDFS-14391 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-14366) Improve HDFS append performance

2019-03-12 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14366: --- Summary: Improve HDFS append performance Key: HDFS-14366 URL: https://issues.apache.org/jira/browse/HDFS-14366 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-14346) EditLogTailer loses precision for sub-second edit log tailing and rolling interval

2019-03-07 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14346: --- Summary: EditLogTailer loses precision for sub-second edit log tailing and rolling interval Key: HDFS-14346 URL: https://issues.apache.org/jira/browse/HDFS-14346 Project

[jira] [Created] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-21 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14305: --- Summary: Serial number in BlockTokenSecretManager could overlap between different namenodes Key: HDFS-14305 URL: https://issues.apache.org/jira/browse/HDFS-14305 Project

[jira] [Created] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

2019-01-31 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14250: --- Summary: [Standby Reads] msync should sync with active NameNode to fetch the latest stateID Key: HDFS-14250 URL: https://issues.apache.org/jira/browse/HDFS-14250 Project

[jira] [Resolved] (HDFS-14168) Fix TestWebHdfsTimeouts

2018-12-31 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-14168. - Resolution: Duplicate > Fix TestWebHdfsTimeouts > --- > >

[jira] [Created] (HDFS-14168) Fix TestWebHdfsTimeouts

2018-12-22 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14168: --- Summary: Fix TestWebHdfsTimeouts Key: HDFS-14168 URL: https://issues.apache.org/jira/browse/HDFS-14168 Project: Hadoop HDFS Issue Type: Test Components

Re: [VOTE - 2] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

2018-12-18 Thread Chao Sun
+1 (no-binding) At Uber we've deployed a subset of features covered in this JIRA in production at multiple data centers, and it's being running great for a while now. We are seeing huge benefits in terms of scaling our NameNode throughput and providing better SLA guarantees for applications such a

[jira] [Created] (HDFS-14154) Add explanation for dfs.ha.tail-edits.period in user guide.

2018-12-17 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14154: --- Summary: Add explanation for dfs.ha.tail-edits.period in user guide. Key: HDFS-14154 URL: https://issues.apache.org/jira/browse/HDFS-14154 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-14146) Handle exception from internalQueueCall

2018-12-12 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14146: --- Summary: Handle exception from internalQueueCall Key: HDFS-14146 URL: https://issues.apache.org/jira/browse/HDFS-14146 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-14110) NPE when serving http requests while NameNode is starting up

2018-11-28 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14110: --- Summary: NPE when serving http requests while NameNode is starting up Key: HDFS-14110 URL: https://issues.apache.org/jira/browse/HDFS-14110 Project: Hadoop HDFS

[jira] [Created] (HDFS-14067) Allow manual failover between standby and observer

2018-11-12 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14067: --- Summary: Allow manual failover between standby and observer Key: HDFS-14067 URL: https://issues.apache.org/jira/browse/HDFS-14067 Project: Hadoop HDFS Issue Type: Sub

[jira] [Created] (HDFS-13931) Backport HDFS-6440 to branch 2.8.2

2018-09-21 Thread Chao Sun (JIRA)
Chao Sun created HDFS-13931: --- Summary: Backport HDFS-6440 to branch 2.8.2 Key: HDFS-13931 URL: https://issues.apache.org/jira/browse/HDFS-13931 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-13924) Handle BlockMissingException when reading from observer

2018-09-17 Thread Chao Sun (JIRA)
Chao Sun created HDFS-13924: --- Summary: Handle BlockMissingException when reading from observer Key: HDFS-13924 URL: https://issues.apache.org/jira/browse/HDFS-13924 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-13923) Add a configuration to turn on/off observer reads

2018-09-17 Thread Chao Sun (JIRA)
Chao Sun created HDFS-13923: --- Summary: Add a configuration to turn on/off observer reads Key: HDFS-13923 URL: https://issues.apache.org/jira/browse/HDFS-13923 Project: Hadoop HDFS Issue Type: Sub

[jira] [Created] (HDFS-13898) Throw retriable exception for getBlockLocations when ObserverNameNode is in safemode

2018-09-05 Thread Chao Sun (JIRA)
Chao Sun created HDFS-13898: --- Summary: Throw retriable exception for getBlockLocations when ObserverNameNode is in safemode Key: HDFS-13898 URL: https://issues.apache.org/jira/browse/HDFS-13898 Project

[jira] [Created] (HDFS-13814) Remove super user privilege requirement for HAServiceProtocol#getServiceStatus

2018-08-09 Thread Chao Sun (JIRA)
Chao Sun created HDFS-13814: --- Summary: Remove super user privilege requirement for HAServiceProtocol#getServiceStatus Key: HDFS-13814 URL: https://issues.apache.org/jira/browse/HDFS-13814 Project: Hadoop

[jira] [Created] (HDFS-13792) Fix FSN read/write lock metrics name

2018-08-03 Thread Chao Sun (JIRA)
Chao Sun created HDFS-13792: --- Summary: Fix FSN read/write lock metrics name Key: HDFS-13792 URL: https://issues.apache.org/jira/browse/HDFS-13792 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-13749) Implement a new client protocol method to get NameNode state

2018-07-19 Thread Chao Sun (JIRA)
Chao Sun created HDFS-13749: --- Summary: Implement a new client protocol method to get NameNode state Key: HDFS-13749 URL: https://issues.apache.org/jira/browse/HDFS-13749 Project: Hadoop HDFS

  1   2   >