Re: HADOOP-14163 proposal for new hadoop.apache.org
Bumping this thread at last time. I have the following proposal: 1. I will request a new git repository hadoop-site.git and import the new site to there (which has exactly the same content as the existing site). 2. I will ask infra to use the new repository as the source of hadoop.apache.org 3. I will sync manually all of the changes in the next two months back to the svn site from the git (release announcements, new committers) IN CASE OF ANY PROBLEM we can switch back to the svn without any problem. If no-one objects within three days, I'll assume lazy consensus and start with this plan. Please comment if you have objections. Again: it allows immediate fallback at any time as svn repo will be kept as is (+ I will keep it up-to-date in the next 2 months) Thanks, Marton On 06/21/2018 09:00 PM, Elek, Marton wrote: Thank you very much to bump up this thread. About [2]: (Just for the clarification) the content of the proposed website is exactly the same as the old one. About [1]. I believe that the "mvn site" is perfect for the documentation but for website creation there are more simple and powerful tools. Hugo has more simple compared to jekyll. Just one binary, without dependencies, works everywhere (mac, linux, windows) Hugo has much more powerful compared to "mvn site". Easier to create/use more modern layout/theme, and easier to handle the content (for example new release announcements could be generated as part of the release process) I think it's very low risk to try out a new approach for the site (and easy to rollback in case of problems) Marton ps: I just updated the patch/preview site with the recent releases: *** * http://hadoop.anzix.net * *** On 06/21/2018 01:27 AM, Vinod Kumar Vavilapalli wrote: Got pinged about this offline. Thanks for keeping at it, Marton! I think there are two road-blocks here (1) Is the mechanism using which the website is built good enough - mvn-site / hugo etc? (2) Is the new website good enough? For (1), I just think we need more committer attention and get feedback rapidly and get it in. For (2), how about we do it in a different way in the interest of progress? - We create a hadoop.apache.org/new-site/ where this new site goes. - We then modify the existing web-site to say that there is a new site/experience that folks can click on a link and navigate to - As this new website matures and gets feedback & fixes, we finally pull the plug at a later point of time when we think we are good to go. Thoughts? +Vinod On Feb 16, 2018, at 3:10 AM, Elek, Marton wrote: Hi, I would like to bump this thread up. TLDR; There is a proposed version of a new hadoop site which is available from here: https://elek.github.io/hadoop-site-proposal/ and https://issues.apache.org/jira/browse/HADOOP-14163 Please let me know what you think about it. Longer version: This thread started long time ago to use a more modern hadoop site: Goals were: 1. To make it easier to manage it (the release entries could be created by a script as part of the release process) 2. To use a better look-and-feel 3. Move it out from svn to git I proposed to: 1. Move the existing site to git and generate it with hugo (which is a single, standalone binary) 2. Move both the rendered and source branches to git. 3. (Create a jenkins job to generate the site automatically) NOTE: this is just about forrest based hadoop.apache.org, NOT about the documentation which is generated by mvn-site (as before) I got multiple valuable feedback and I improved the proposed site according to the comments. Allen had some concerns about the used technologies (hugo vs. mvn-site) and I answered all the questions why I think mvn-site is the best for documentation and hugo is best for generating site. I would like to finish this effort/jira: I would like to start a discussion about using this proposed version and approach as a new site of Apache Hadoop. Please let me know what you think. Thanks a lot, Marton - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Re: HADOOP-14163 proposal for new hadoop.apache.org
> On 31 Aug 2018, at 09:07, Elek, Marton wrote: > > Bumping this thread at last time. > > I have the following proposal: > > 1. I will request a new git repository hadoop-site.git and import the new > site to there (which has exactly the same content as the existing site). > > 2. I will ask infra to use the new repository as the source of > hadoop.apache.org > > 3. I will sync manually all of the changes in the next two months back to the > svn site from the git (release announcements, new committers) > > IN CASE OF ANY PROBLEM we can switch back to the svn without any problem. > > If no-one objects within three days, I'll assume lazy consensus and start > with this plan. Please comment if you have objections. > > Again: it allows immediate fallback at any time as svn repo will be kept as > is (+ I will keep it up-to-date in the next 2 months) > > Thanks, > Marton sounds good to me +1 - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
review for IPC client change needed
Hi While we wait for Jenkins to return, there's a patch for IPC client shutdown which needs some review by people with experience of that IPC client code https://issues.apache.org/jira/browse/HADOOP-10219 the IPC code is a key area, which is why it's sensitive -yet its shutdown logic is known to be broken & a source of timeouts on shutdown hooks. Can anyone with experience in this area take a look, otherwise those of us will superficial experience will be doing that voting for you thanks -steve
[jira] [Created] (HDDS-388) Fix the name of the db profile configuration key
Elek, Marton created HDDS-388: - Summary: Fix the name of the db profile configuration key Key: HDDS-388 URL: https://issues.apache.org/jira/browse/HDDS-388 Project: Hadoop Distributed Data Store Issue Type: Bug Reporter: Elek, Marton Assignee: Elek, Marton HDDS-359 introduced a new configuration for db profiles but at the end the name of the configuration key in ozone-default (ozone.db.profile) is different from the one which is used in the constant (hdds.db.profile). (It's moved to the HddsConfigKeys at the last minute) As a result TestOzoneConfigurationFields is failing for precommit tests. Uploading the trivial fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Re: HADOOP-14163 proposal for new hadoop.apache.org
+1 from me On Fri, Aug 31, 2018, 5:30 AM Steve Loughran wrote: > > > > On 31 Aug 2018, at 09:07, Elek, Marton wrote: > > > > Bumping this thread at last time. > > > > I have the following proposal: > > > > 1. I will request a new git repository hadoop-site.git and import the > new site to there (which has exactly the same content as the existing site). > > > > 2. I will ask infra to use the new repository as the source of > hadoop.apache.org > > > > 3. I will sync manually all of the changes in the next two months back > to the svn site from the git (release announcements, new committers) > > > > IN CASE OF ANY PROBLEM we can switch back to the svn without any problem. > > > > If no-one objects within three days, I'll assume lazy consensus and > start with this plan. Please comment if you have objections. > > > > Again: it allows immediate fallback at any time as svn repo will be kept > as is (+ I will keep it up-to-date in the next 2 months) > > > > Thanks, > > Marton > > sounds good to me > > +1 > > > > - > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >
[jira] [Created] (HDDS-389) Remove XceiverServer and XceiverClient and related classes
Mukul Kumar Singh created HDDS-389: -- Summary: Remove XceiverServer and XceiverClient and related classes Key: HDDS-389 URL: https://issues.apache.org/jira/browse/HDDS-389 Project: Hadoop Distributed Data Store Issue Type: Bug Components: SCM Reporter: Mukul Kumar Singh Grpc is now the default protocol for datanode to client communication. This jira proposes to remove all the instances of the classes from the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Re: HADOOP-14163 proposal for new hadoop.apache.org
Is there no way to host the new site and the old site concurrently? And link back & forth? +Vinod > On Aug 31, 2018, at 1:07 AM, Elek, Marton wrote: > > Bumping this thread at last time. > > I have the following proposal: > > 1. I will request a new git repository hadoop-site.git and import the new > site to there (which has exactly the same content as the existing site). > > 2. I will ask infra to use the new repository as the source of > hadoop.apache.org > > 3. I will sync manually all of the changes in the next two months back to the > svn site from the git (release announcements, new committers) > > IN CASE OF ANY PROBLEM we can switch back to the svn without any problem. > > If no-one objects within three days, I'll assume lazy consensus and start > with this plan. Please comment if you have objections. > > Again: it allows immediate fallback at any time as svn repo will be kept as > is (+ I will keep it up-to-date in the next 2 months) > > Thanks, > Marton > > > On 06/21/2018 09:00 PM, Elek, Marton wrote: >> Thank you very much to bump up this thread. >> About [2]: (Just for the clarification) the content of the proposed website >> is exactly the same as the old one. >> About [1]. I believe that the "mvn site" is perfect for the documentation >> but for website creation there are more simple and powerful tools. >> Hugo has more simple compared to jekyll. Just one binary, without >> dependencies, works everywhere (mac, linux, windows) >> Hugo has much more powerful compared to "mvn site". Easier to create/use >> more modern layout/theme, and easier to handle the content (for example new >> release announcements could be generated as part of the release process) >> I think it's very low risk to try out a new approach for the site (and easy >> to rollback in case of problems) >> Marton >> ps: I just updated the patch/preview site with the recent releases: >> *** >> * http://hadoop.anzix.net * >> *** >> On 06/21/2018 01:27 AM, Vinod Kumar Vavilapalli wrote: >>> Got pinged about this offline. >>> >>> Thanks for keeping at it, Marton! >>> >>> I think there are two road-blocks here >>> (1) Is the mechanism using which the website is built good enough - >>> mvn-site / hugo etc? >>> (2) Is the new website good enough? >>> >>> For (1), I just think we need more committer attention and get feedback >>> rapidly and get it in. >>> >>> For (2), how about we do it in a different way in the interest of progress? >>> - We create a hadoop.apache.org/new-site/ where this new site goes. >>> - We then modify the existing web-site to say that there is a new >>> site/experience that folks can click on a link and navigate to >>> - As this new website matures and gets feedback & fixes, we finally pull >>> the plug at a later point of time when we think we are good to go. >>> >>> Thoughts? >>> >>> +Vinod >>> On Feb 16, 2018, at 3:10 AM, Elek, Marton wrote: Hi, I would like to bump this thread up. TLDR; There is a proposed version of a new hadoop site which is available from here: https://elek.github.io/hadoop-site-proposal/ and https://issues.apache.org/jira/browse/HADOOP-14163 Please let me know what you think about it. Longer version: This thread started long time ago to use a more modern hadoop site: Goals were: 1. To make it easier to manage it (the release entries could be created by a script as part of the release process) 2. To use a better look-and-feel 3. Move it out from svn to git I proposed to: 1. Move the existing site to git and generate it with hugo (which is a single, standalone binary) 2. Move both the rendered and source branches to git. 3. (Create a jenkins job to generate the site automatically) NOTE: this is just about forrest based hadoop.apache.org, NOT about the documentation which is generated by mvn-site (as before) I got multiple valuable feedback and I improved the proposed site according to the comments. Allen had some concerns about the used technologies (hugo vs. mvn-site) and I answered all the questions why I think mvn-site is the best for documentation and hugo is best for generating site. I would like to finish this effort/jira: I would like to start a discussion about using this proposed version and approach as a new site of Apache Hadoop. Please let me know what you think. Thanks a lot, Marton - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org >>> >>> >>> - >>> To unsubscribe, e-ma
Re: HADOOP-14163 proposal for new hadoop.apache.org
+1. Thanks for the work, Marton! On Fri, Aug 31, 2018 at 8:37 AM Vinod Kumar Vavilapalli wrote: > Is there no way to host the new site and the old site concurrently? And > link back & forth? > > +Vinod > > > > On Aug 31, 2018, at 1:07 AM, Elek, Marton wrote: > > > > Bumping this thread at last time. > > > > I have the following proposal: > > > > 1. I will request a new git repository hadoop-site.git and import the > new site to there (which has exactly the same content as the existing site). > > > > 2. I will ask infra to use the new repository as the source of > hadoop.apache.org > > > > 3. I will sync manually all of the changes in the next two months back > to the svn site from the git (release announcements, new committers) > > > > IN CASE OF ANY PROBLEM we can switch back to the svn without any problem. > > > > If no-one objects within three days, I'll assume lazy consensus and > start with this plan. Please comment if you have objections. > > > > Again: it allows immediate fallback at any time as svn repo will be kept > as is (+ I will keep it up-to-date in the next 2 months) > > > > Thanks, > > Marton > > > > > > On 06/21/2018 09:00 PM, Elek, Marton wrote: > >> Thank you very much to bump up this thread. > >> About [2]: (Just for the clarification) the content of the proposed > website is exactly the same as the old one. > >> About [1]. I believe that the "mvn site" is perfect for the > documentation but for website creation there are more simple and powerful > tools. > >> Hugo has more simple compared to jekyll. Just one binary, without > dependencies, works everywhere (mac, linux, windows) > >> Hugo has much more powerful compared to "mvn site". Easier to > create/use more modern layout/theme, and easier to handle the content (for > example new release announcements could be generated as part of the release > process) > >> I think it's very low risk to try out a new approach for the site (and > easy to rollback in case of problems) > >> Marton > >> ps: I just updated the patch/preview site with the recent releases: > >> *** > >> * http://hadoop.anzix.net * > >> *** > >> On 06/21/2018 01:27 AM, Vinod Kumar Vavilapalli wrote: > >>> Got pinged about this offline. > >>> > >>> Thanks for keeping at it, Marton! > >>> > >>> I think there are two road-blocks here > >>> (1) Is the mechanism using which the website is built good enough - > mvn-site / hugo etc? > >>> (2) Is the new website good enough? > >>> > >>> For (1), I just think we need more committer attention and get > feedback rapidly and get it in. > >>> > >>> For (2), how about we do it in a different way in the interest of > progress? > >>> - We create a hadoop.apache.org/new-site/ where this new site goes. > >>> - We then modify the existing web-site to say that there is a new > site/experience that folks can click on a link and navigate to > >>> - As this new website matures and gets feedback & fixes, we finally > pull the plug at a later point of time when we think we are good to go. > >>> > >>> Thoughts? > >>> > >>> +Vinod > >>> > On Feb 16, 2018, at 3:10 AM, Elek, Marton wrote: > > Hi, > > I would like to bump this thread up. > > TLDR; There is a proposed version of a new hadoop site which is > available from here: https://elek.github.io/hadoop-site-proposal/ and > https://issues.apache.org/jira/browse/HADOOP-14163 > > Please let me know what you think about it. > > > Longer version: > > This thread started long time ago to use a more modern hadoop site: > > Goals were: > > 1. To make it easier to manage it (the release entries could be > created by a script as part of the release process) > 2. To use a better look-and-feel > 3. Move it out from svn to git > > I proposed to: > > 1. Move the existing site to git and generate it with hugo (which is > a single, standalone binary) > 2. Move both the rendered and source branches to git. > 3. (Create a jenkins job to generate the site automatically) > > NOTE: this is just about forrest based hadoop.apache.org, NOT about > the documentation which is generated by mvn-site (as before) > > > I got multiple valuable feedback and I improved the proposed site > according to the comments. Allen had some concerns about the used > technologies (hugo vs. mvn-site) and I answered all the questions why I > think mvn-site is the best for documentation and hugo is best for > generating site. > > > I would like to finish this effort/jira: I would like to start a > discussion about using this proposed version and approach as a new site of > Apache Hadoop. Please let me know what you think. > > > Thanks a lot, > Marton > > - > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apac
Re: HADOOP-14163 proposal for new hadoop.apache.org
+1 It’s better to new version link in old version. Brahma Reddy Battula On Fri, Aug 31, 2018 at 9:59 PM, Sangjin Lee wrote: > +1. Thanks for the work, Marton! > > On Fri, Aug 31, 2018 at 8:37 AM Vinod Kumar Vavilapalli < > vino...@apache.org> > wrote: > > > Is there no way to host the new site and the old site concurrently? And > > link back & forth? > > > > +Vinod > > > > > > > On Aug 31, 2018, at 1:07 AM, Elek, Marton wrote: > > > > > > Bumping this thread at last time. > > > > > > I have the following proposal: > > > > > > 1. I will request a new git repository hadoop-site.git and import the > > new site to there (which has exactly the same content as the existing > site). > > > > > > 2. I will ask infra to use the new repository as the source of > > hadoop.apache.org > > > > > > 3. I will sync manually all of the changes in the next two months back > > to the svn site from the git (release announcements, new committers) > > > > > > IN CASE OF ANY PROBLEM we can switch back to the svn without any > problem. > > > > > > If no-one objects within three days, I'll assume lazy consensus and > > start with this plan. Please comment if you have objections. > > > > > > Again: it allows immediate fallback at any time as svn repo will be > kept > > as is (+ I will keep it up-to-date in the next 2 months) > > > > > > Thanks, > > > Marton > > > > > > > > > On 06/21/2018 09:00 PM, Elek, Marton wrote: > > >> Thank you very much to bump up this thread. > > >> About [2]: (Just for the clarification) the content of the proposed > > website is exactly the same as the old one. > > >> About [1]. I believe that the "mvn site" is perfect for the > > documentation but for website creation there are more simple and powerful > > tools. > > >> Hugo has more simple compared to jekyll. Just one binary, without > > dependencies, works everywhere (mac, linux, windows) > > >> Hugo has much more powerful compared to "mvn site". Easier to > > create/use more modern layout/theme, and easier to handle the content > (for > > example new release announcements could be generated as part of the > release > > process) > > >> I think it's very low risk to try out a new approach for the site (and > > easy to rollback in case of problems) > > >> Marton > > >> ps: I just updated the patch/preview site with the recent releases: > > >> *** > > >> * http://hadoop.anzix.net * > > >> *** > > >> On 06/21/2018 01:27 AM, Vinod Kumar Vavilapalli wrote: > > >>> Got pinged about this offline. > > >>> > > >>> Thanks for keeping at it, Marton! > > >>> > > >>> I think there are two road-blocks here > > >>> (1) Is the mechanism using which the website is built good enough - > > mvn-site / hugo etc? > > >>> (2) Is the new website good enough? > > >>> > > >>> For (1), I just think we need more committer attention and get > > feedback rapidly and get it in. > > >>> > > >>> For (2), how about we do it in a different way in the interest of > > progress? > > >>> - We create a hadoop.apache.org/new-site/ where this new site > goes. > > >>> - We then modify the existing web-site to say that there is a new > > site/experience that folks can click on a link and navigate to > > >>> - As this new website matures and gets feedback & fixes, we finally > > pull the plug at a later point of time when we think we are good to go. > > >>> > > >>> Thoughts? > > >>> > > >>> +Vinod > > >>> > > On Feb 16, 2018, at 3:10 AM, Elek, Marton wrote: > > > > Hi, > > > > I would like to bump this thread up. > > > > TLDR; There is a proposed version of a new hadoop site which is > > available from here: https://elek.github.io/hadoop-site-proposal/ and > > https://issues.apache.org/jira/browse/HADOOP-14163 > > > > Please let me know what you think about it. > > > > > > Longer version: > > > > This thread started long time ago to use a more modern hadoop site: > > > > Goals were: > > > > 1. To make it easier to manage it (the release entries could be > > created by a script as part of the release process) > > 2. To use a better look-and-feel > > 3. Move it out from svn to git > > > > I proposed to: > > > > 1. Move the existing site to git and generate it with hugo (which is > > a single, standalone binary) > > 2. Move both the rendered and source branches to git. > > 3. (Create a jenkins job to generate the site automatically) > > > > NOTE: this is just about forrest based hadoop.apache.org, NOT about > > the documentation which is generated by mvn-site (as before) > > > > > > I got multiple valuable feedback and I improved the proposed site > > according to the comments. Allen had some concerns about the used > > technologies (hugo vs. mvn-site) and I answered all the questions why I > > think mvn-site is the best for documentation and hugo is best for > > generating site. > >
Re: HADOOP-14163 proposal for new hadoop.apache.org
+1 Thanks for initiating this Marton. On 8/31/18, 1:07 AM, "Elek, Marton" wrote: Bumping this thread at last time. I have the following proposal: 1. I will request a new git repository hadoop-site.git and import the new site to there (which has exactly the same content as the existing site). 2. I will ask infra to use the new repository as the source of hadoop.apache.org 3. I will sync manually all of the changes in the next two months back to the svn site from the git (release announcements, new committers) IN CASE OF ANY PROBLEM we can switch back to the svn without any problem. If no-one objects within three days, I'll assume lazy consensus and start with this plan. Please comment if you have objections. Again: it allows immediate fallback at any time as svn repo will be kept as is (+ I will keep it up-to-date in the next 2 months) Thanks, Marton On 06/21/2018 09:00 PM, Elek, Marton wrote: > > Thank you very much to bump up this thread. > > > About [2]: (Just for the clarification) the content of the proposed > website is exactly the same as the old one. > > About [1]. I believe that the "mvn site" is perfect for the > documentation but for website creation there are more simple and > powerful tools. > > Hugo has more simple compared to jekyll. Just one binary, without > dependencies, works everywhere (mac, linux, windows) > > Hugo has much more powerful compared to "mvn site". Easier to create/use > more modern layout/theme, and easier to handle the content (for example > new release announcements could be generated as part of the release > process) > > I think it's very low risk to try out a new approach for the site (and > easy to rollback in case of problems) > > Marton > > ps: I just updated the patch/preview site with the recent releases: > > *** > * http://hadoop.anzix.net * > *** > > On 06/21/2018 01:27 AM, Vinod Kumar Vavilapalli wrote: >> Got pinged about this offline. >> >> Thanks for keeping at it, Marton! >> >> I think there are two road-blocks here >> (1) Is the mechanism using which the website is built good enough - >> mvn-site / hugo etc? >> (2) Is the new website good enough? >> >> For (1), I just think we need more committer attention and get >> feedback rapidly and get it in. >> >> For (2), how about we do it in a different way in the interest of >> progress? >> - We create a hadoop.apache.org/new-site/ where this new site goes. >> - We then modify the existing web-site to say that there is a new >> site/experience that folks can click on a link and navigate to >> - As this new website matures and gets feedback & fixes, we finally >> pull the plug at a later point of time when we think we are good to go. >> >> Thoughts? >> >> +Vinod >> >>> On Feb 16, 2018, at 3:10 AM, Elek, Marton wrote: >>> >>> Hi, >>> >>> I would like to bump this thread up. >>> >>> TLDR; There is a proposed version of a new hadoop site which is >>> available from here: https://elek.github.io/hadoop-site-proposal/ and >>> https://issues.apache.org/jira/browse/HADOOP-14163 >>> >>> Please let me know what you think about it. >>> >>> >>> Longer version: >>> >>> This thread started long time ago to use a more modern hadoop site: >>> >>> Goals were: >>> >>> 1. To make it easier to manage it (the release entries could be >>> created by a script as part of the release process) >>> 2. To use a better look-and-feel >>> 3. Move it out from svn to git >>> >>> I proposed to: >>> >>> 1. Move the existing site to git and generate it with hugo (which is >>> a single, standalone binary) >>> 2. Move both the rendered and source branches to git. >>> 3. (Create a jenkins job to generate the site automatically) >>> >>> NOTE: this is just about forrest based hadoop.apache.org, NOT about >>> the documentation which is generated by mvn-site (as before) >>> >>> >>> I got multiple valuable feedback and I improved the proposed site >>> according to the comments. Allen had some concerns about the used >>> technologies (hugo vs. mvn-site) and I answered all the questions why >>> I think mvn-site is the best for documentation and hugo is best for >>> generating site. >>> >>> >>> I would like to finish this effort/jira: I would like to start a >>> discussion about using this proposed version and approach as a new >>> site of Apache Hadoop. Please let me know what you think. >>> >>> >>> Thanks a lot, >>> Marton >>>
[jira] [Resolved] (HDFS-13780) Postpone NameNode state discovery in ObserverReadProxyProvider until the first real RPC call.
[
https://issues.apache.org/jira/browse/HDFS-13780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Shvachko resolved HDFS-13780.
Resolution: Duplicate
Fix Version/s: HDFS-12943
I think it was incorporated, indeed.
> Postpone NameNode state discovery in ObserverReadProxyProvider until the
> first real RPC call.
> -
>
> Key: HDFS-13780
> URL: https://issues.apache.org/jira/browse/HDFS-13780
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Reporter: Konstantin Shvachko
>Assignee: Chen Liang
>Priority: Major
> Fix For: HDFS-12943
>
>
> Currently {{ObserverReadProxyProvider}} during instantiation discovers
> Observers by poking known NameNodes and checking their states. This rather
> expensive process can be postponed until the first actual RPC call.
> This is an optimization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Re: HADOOP-14163 proposal for new hadoop.apache.org
+1, thanks for working on this, Marton! Best, Wangda On Fri, Aug 31, 2018 at 11:24 AM Arpit Agarwal wrote: > +1 > > Thanks for initiating this Marton. > > > On 8/31/18, 1:07 AM, "Elek, Marton" wrote: > > Bumping this thread at last time. > > I have the following proposal: > > 1. I will request a new git repository hadoop-site.git and import the > new site to there (which has exactly the same content as the existing > site). > > 2. I will ask infra to use the new repository as the source of > hadoop.apache.org > > 3. I will sync manually all of the changes in the next two months back > to the svn site from the git (release announcements, new committers) > > IN CASE OF ANY PROBLEM we can switch back to the svn without any > problem. > > If no-one objects within three days, I'll assume lazy consensus and > start with this plan. Please comment if you have objections. > > Again: it allows immediate fallback at any time as svn repo will be > kept > as is (+ I will keep it up-to-date in the next 2 months) > > Thanks, > Marton > > > On 06/21/2018 09:00 PM, Elek, Marton wrote: > > > > Thank you very much to bump up this thread. > > > > > > About [2]: (Just for the clarification) the content of the proposed > > website is exactly the same as the old one. > > > > About [1]. I believe that the "mvn site" is perfect for the > > documentation but for website creation there are more simple and > > powerful tools. > > > > Hugo has more simple compared to jekyll. Just one binary, without > > dependencies, works everywhere (mac, linux, windows) > > > > Hugo has much more powerful compared to "mvn site". Easier to > create/use > > more modern layout/theme, and easier to handle the content (for > example > > new release announcements could be generated as part of the release > > process) > > > > I think it's very low risk to try out a new approach for the site > (and > > easy to rollback in case of problems) > > > > Marton > > > > ps: I just updated the patch/preview site with the recent releases: > > > > *** > > * http://hadoop.anzix.net * > > *** > > > > On 06/21/2018 01:27 AM, Vinod Kumar Vavilapalli wrote: > >> Got pinged about this offline. > >> > >> Thanks for keeping at it, Marton! > >> > >> I think there are two road-blocks here > >> (1) Is the mechanism using which the website is built good enough > - > >> mvn-site / hugo etc? > >> (2) Is the new website good enough? > >> > >> For (1), I just think we need more committer attention and get > >> feedback rapidly and get it in. > >> > >> For (2), how about we do it in a different way in the interest of > >> progress? > >> - We create a hadoop.apache.org/new-site/ where this new site > goes. > >> - We then modify the existing web-site to say that there is a new > >> site/experience that folks can click on a link and navigate to > >> - As this new website matures and gets feedback & fixes, we > finally > >> pull the plug at a later point of time when we think we are good to > go. > >> > >> Thoughts? > >> > >> +Vinod > >> > >>> On Feb 16, 2018, at 3:10 AM, Elek, Marton wrote: > >>> > >>> Hi, > >>> > >>> I would like to bump this thread up. > >>> > >>> TLDR; There is a proposed version of a new hadoop site which is > >>> available from here: https://elek.github.io/hadoop-site-proposal/ > and > >>> https://issues.apache.org/jira/browse/HADOOP-14163 > >>> > >>> Please let me know what you think about it. > >>> > >>> > >>> Longer version: > >>> > >>> This thread started long time ago to use a more modern hadoop site: > >>> > >>> Goals were: > >>> > >>> 1. To make it easier to manage it (the release entries could be > >>> created by a script as part of the release process) > >>> 2. To use a better look-and-feel > >>> 3. Move it out from svn to git > >>> > >>> I proposed to: > >>> > >>> 1. Move the existing site to git and generate it with hugo (which > is > >>> a single, standalone binary) > >>> 2. Move both the rendered and source branches to git. > >>> 3. (Create a jenkins job to generate the site automatically) > >>> > >>> NOTE: this is just about forrest based hadoop.apache.org, NOT > about > >>> the documentation which is generated by mvn-site (as before) > >>> > >>> > >>> I got multiple valuable feedback and I improved the proposed site > >>> according to the comments. Allen had some concerns about the used > >>> technologies (hugo vs. mvn-site) and I answered all the questions > why > >>> I think mvn-site is the best for documentation and hugo is best > for > >>> generati
[jira] [Resolved] (HDDS-337) keys created with key name having special character/wildcard should not allowed
[
https://issues.apache.org/jira/browse/HDDS-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anu Engineer resolved HDDS-337.
---
Resolution: Information Provided
> keys created with key name having special character/wildcard should not
> allowed
> ---
>
> Key: HDDS-337
> URL: https://issues.apache.org/jira/browse/HDDS-337
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Client
>Reporter: Nilotpal Nandi
>Assignee: Dinesh Chitlangia
>Priority: Major
> Fix For: 0.2.1
>
>
> Please find the snippet of command execution. Here , the keys are created
> with wildcard special character in its key name.
> Expectation :
> wildcard special characters should not be allowed.
>
> {noformat}
> hadoop@1a1fa8a11332:~/bin$ ./ozone oz -putKey root-volume/root-bucket/d++
> -file /etc/services -v
> 2018-08-08 13:17:48 WARN NativeCodeLoader:60 - Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Volume Name : root-volume
> Bucket Name : root-bucket
> Key Name : d++
> File Hash : 567c100888518c1163b3462993de7d47
> Key Name : d++ does not exist, creating it
> 2018-08-08 13:17:48 INFO ConfUtils:41 - raft.rpc.type = GRPC (default)
> 2018-08-08 13:17:48 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
> (custom)
> 2018-08-08 13:17:48 INFO ConfUtils:41 - raft.client.rpc.retryInterval = 300
> ms (default)
> 2018-08-08 13:17:48 INFO ConfUtils:41 -
> raft.client.async.outstanding-requests.max = 100 (default)
> 2018-08-08 13:17:48 INFO ConfUtils:41 - raft.client.async.scheduler-threads =
> 3 (default)
> 2018-08-08 13:17:49 INFO ConfUtils:41 - raft.grpc.flow.control.window = 1MB
> (=1048576) (default)
> 2018-08-08 13:17:49 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
> (custom)
> 2018-08-08 13:17:49 INFO ConfUtils:41 - raft.client.rpc.request.timeout =
> 3000 ms (default)
> Aug 08, 2018 1:17:49 PM
> org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl detectProxy
> WARNING: Failed to construct URI for proxy lookup, proceeding without proxy
> java.net.URISyntaxException: Illegal character in hostname at index 13:
> https://ozone_datanode_1.ozone_default:9858
> at java.net.URI$Parser.fail(URI.java:2848)
> at java.net.URI$Parser.parseHostname(URI.java:3387)
> at java.net.URI$Parser.parseServer(URI.java:3236)
> at java.net.URI$Parser.parseAuthority(URI.java:3155)
> at java.net.URI$Parser.parseHierarchical(URI.java:3097)
> at java.net.URI$Parser.parse(URI.java:3053)
> at java.net.URI.(URI.java:673)
> at
> org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl.detectProxy(ProxyDetectorImpl.java:128)
> at
> org.apache.ratis.shaded.io.grpc.internal.ProxyDetectorImpl.proxyFor(ProxyDetectorImpl.java:118)
> at
> org.apache.ratis.shaded.io.grpc.internal.InternalSubchannel.startNewTransport(InternalSubchannel.java:207)
> at
> org.apache.ratis.shaded.io.grpc.internal.InternalSubchannel.obtainActiveTransport(InternalSubchannel.java:188)
> at
> org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$SubchannelImpl.requestConnection(ManagedChannelImpl.java:1130)
> at
> org.apache.ratis.shaded.io.grpc.PickFirstBalancerFactory$PickFirstBalancer.handleResolvedAddressGroups(PickFirstBalancerFactory.java:79)
> at
> org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$NameResolverListenerImpl$1NamesResolved.run(ManagedChannelImpl.java:1032)
> at
> org.apache.ratis.shaded.io.grpc.internal.ChannelExecutor.drain(ChannelExecutor.java:73)
> at
> org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$LbHelperImpl.runSerialized(ManagedChannelImpl.java:1000)
> at
> org.apache.ratis.shaded.io.grpc.internal.ManagedChannelImpl$NameResolverListenerImpl.onAddresses(ManagedChannelImpl.java:1044)
> at
> org.apache.ratis.shaded.io.grpc.internal.DnsNameResolver$1.run(DnsNameResolver.java:201)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> hadoop@1a1fa8a11332:~/bin$ ./ozone oz -putKey root-volume/root-bucket/d**
> -file /etc/passwd -v
> 2018-08-08 13:18:13 WARN NativeCodeLoader:60 - Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Volume Name : root-volume
> Bucket Name : root-bucket
> Key Name : d**
> File Hash : b056233571cc80d6879212911cb8e500
> Key Name : d** does not exist, creating it
> 2018-08-08 13:18:14 INFO ConfUtils:41 - raft.rpc.type = GRPC (default)
> 2018-08-08 13:18:14 INFO ConfUtils:41 - raft.grpc.message.size.max = 33554432
> (custom)
> 2018-08-08 13:18:14 INFO ConfUtils:41 - raft.client.rpc.retryInterval = 30
[jira] [Created] (HDDS-390) Add method to check for valid key name based on URI characters
Dinesh Chitlangia created HDDS-390: -- Summary: Add method to check for valid key name based on URI characters Key: HDDS-390 URL: https://issues.apache.org/jira/browse/HDDS-390 Project: Hadoop Distributed Data Store Issue Type: Improvement Reporter: Dinesh Chitlangia Assignee: Dinesh Chitlangia As per design, key names composed of all valid characters in URI set are treated as valid key name. For URI character set: [https://tools.ietf.org/html/rfc2396#appendix-A] This Jira proposes to define validateKeyName() similar to validateResourceName() that validates bucket/volume name -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-391) Simplify AuditMessage structure to make audit logging easier to use
Dinesh Chitlangia created HDDS-391: -- Summary: Simplify AuditMessage structure to make audit logging easier to use Key: HDDS-391 URL: https://issues.apache.org/jira/browse/HDDS-391 Project: Hadoop Distributed Data Store Issue Type: Improvement Components: Ozone Manager Reporter: Dinesh Chitlangia Assignee: Dinesh Chitlangia In HDDS-376 a customer AuditMessage structure was created for use in Audit Logging. This Jira proposes to incorporate suggestive improvements from [~ajayydv]. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-392) Incomplete description about auditMap#key in AuditLogging Framework
Dinesh Chitlangia created HDDS-392: -- Summary: Incomplete description about auditMap#key in AuditLogging Framework Key: HDDS-392 URL: https://issues.apache.org/jira/browse/HDDS-392 Project: Hadoop Distributed Data Store Issue Type: Improvement Reporter: Dinesh Chitlangia Assignee: Dinesh Chitlangia Trivial issue where the description about key in auditMap is incomplete and can lead to developers creating invalid audit keys for logging. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-393) Audit Parser tool for processing ozone audit logs
Dinesh Chitlangia created HDDS-393: -- Summary: Audit Parser tool for processing ozone audit logs Key: HDDS-393 URL: https://issues.apache.org/jira/browse/HDDS-393 Project: Hadoop Distributed Data Store Issue Type: New Feature Reporter: Dinesh Chitlangia Assignee: Dinesh Chitlangia Jira to create audit parser tool to process ozone audit logs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-394) Rename *Key Apis in DatanodeContainerProtocol to *Block apis
Mukul Kumar Singh created HDDS-394: -- Summary: Rename *Key Apis in DatanodeContainerProtocol to *Block apis Key: HDDS-394 URL: https://issues.apache.org/jira/browse/HDDS-394 Project: Hadoop Distributed Data Store Issue Type: Bug Components: Ozone Datanode Reporter: Mukul Kumar Singh All the block apis in client datanode interaction are named *key apis(e.g. PutKey), This can be renamed to *Block apis. (e.g. PutBlock). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
