[GitHub] [hadoop-ozone] cxorm edited a comment on issue #94: HDDS-2255. Improve Acl Handler Messages
cxorm edited a comment on issue #94: HDDS-2255. Improve Acl Handler Messages URL: https://github.com/apache/hadoop-ozone/pull/94#issuecomment-549252790 > Below are the find bugs that need to be fixed Jenkins reported. > M D DLS: Dead store to result in org.apache.hadoop.ozone.web.ozShell.bucket.SetAclBucketHandler.call() At SetAclBucketHandler.java:[line 91] > M D DLS: Dead store to result in org.apache.hadoop.ozone.web.ozShell.keys.SetAclKeyHandler.call() At SetAclKeyHandler.java:[line 93] > > Unused variable result in SetAclBucketHandler and SetAclKeyHandler. Thanks @bharatviswa504 for the comment. Fixed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] cxorm commented on issue #94: HDDS-2255. Improve Acl Handler Messages
cxorm commented on issue #94: HDDS-2255. Improve Acl Handler Messages URL: https://github.com/apache/hadoop-ozone/pull/94#issuecomment-549252790 > Below are the find bugs that need to be fixed Jenkins reported. > M D DLS: Dead store to result in org.apache.hadoop.ozone.web.ozShell.bucket.SetAclBucketHandler.call() At SetAclBucketHandler.java:[line 91] > M D DLS: Dead store to result in org.apache.hadoop.ozone.web.ozShell.keys.SetAclKeyHandler.call() At SetAclKeyHandler.java:[line 93] > > Unused variable result in SetAclBucketHandler and SetAclKeyHandler. Thanks @bharatviswa504 for the comment. Fixed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Re: [VOTE] create ozone-dev and ozone-issues mailing lists
Thanks Arpit. Notification rules has also been adjusted (thanks to INFRA). https://issues.apache.org/jira/browse/INFRA-19378 Marton On 11/1/19 4:46 PM, Arpit Agarwal wrote: Thanks for kicking this off Marton. Submitted INFRA requests to create the following. The lists should be live soon. - ozone-dev@h.a.o - ozone-issues@h.a.o - ozone-commits@h.a.o On Oct 31, 2019, at 3:32 AM, Elek, Marton wrote: Thanks for all the votes and feedback. The vote is passed with no -1 and with many +1 The mailing lists will be created soon and the notification settings will be updated. Thank you for your patience. Marton On 10/27/19 9:25 AM, Elek, Marton wrote: As discussed earlier in the thread of "Hadoop-Ozone repository mailing list configurations" [1] I suggested to solve the current misconfiguration problem with creating separated mailing lists (dev/issues) for Hadoop Ozone. It would have some additional benefit: for example it would make easier to follow the Ozone development and future plans. Here I am starting a new vote thread (open for at least 72 hours) to collect more feedback about this. Please express your opinion / vote. Thanks a lot, Marton [1] https://lists.apache.org/thread.html/dc66a30f48a744534e748c418bf7ab6275896166ca5ade11560ebaef@%3Chdfs-dev.hadoop.apache.org%3E - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-2399) Update mailing list information in CONTRIBUTION and README files
Marton Elek created HDDS-2399: - Summary: Update mailing list information in CONTRIBUTION and README files Key: HDDS-2399 URL: https://issues.apache.org/jira/browse/HDDS-2399 Project: Hadoop Distributed Data Store Issue Type: Improvement Reporter: Marton Elek We have new mailing lists: [ozone-...@hadoop.apache.org|mailto:ozone-...@hadoop.apache.org] [ozone-iss...@hadoop.apache.org|mailto:ozone-iss...@hadoop.apache.org] [ozone-comm...@hadoop.apache.org|mailto:ozone-comm...@hadoop.apache.org] We need to update CONTRIBUTION.md and README.md to use ozone-dev instead of hdfs-dev (optionally we can mention the issues/commits lists, but only in CONTRIBUTION.md) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-2400) Enable github actions based builds for oyone
Marton Elek created HDDS-2400:
-
Summary: Enable github actions based builds for oyone
Key: HDDS-2400
URL: https://issues.apache.org/jira/browse/HDDS-2400
Project: Hadoop Distributed Data Store
Issue Type: Improvement
Components: build
Reporter: Marton Elek
Assignee: Marton Elek
Current PR checks are executed in a private branch based on the scripts in
[https://github.com/elek/argo-ozone]
but the results are stored in a public repositories:
[https://github.com/elek/ozone-ci-q4|https://github.com/elek/ozone-ci-q3]
[https://github.com/elek/ozone-ci-03]
As we discussed during the community builds, it would be great to use github
actions (or any other cloud based build) to make all the build definitions more
accessible for the community.
[~vivekratnavel] checked CircleCI which has better reporting capabilities. But
INFRA has concerns about the permission model of circle-ci:
{quote}it is highly unlikley we will allow a bot to be able to commit code
(whether or not that is the intention, allowing circle-ci will make this
possible, and is a complete no)
{quote}
See:
https://issues.apache.org/jira/browse/INFRA-18131
[https://lists.apache.org/thread.html/af52e2a3e865c01596d46374e8b294f2740587dbd59d85e132429b6c@%3Cbuilds.apache.org%3E]
Fortunately we have a clear contract. Or build scripts are stored under
_hadoop-ozone/dev-support/checks_ (return code show the result, details are
printed out to the console output). It's very easy to experiment with different
build systems.
Github action seems to be an obvious choice: it's integrated well with GitHub
and it has more generous resource limitations.
With this Jira I propose to enable github actions based PR checks for a few
tests (author, rat, unit, acceptance, checkstyle, findbugs) as an experiment.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-2401) Add robot test file for ozone-hdfs
luhuachao created HDDS-2401: --- Summary: Add robot test file for ozone-hdfs Key: HDDS-2401 URL: https://issues.apache.org/jira/browse/HDDS-2401 Project: Hadoop Distributed Data Store Issue Type: Sub-task Reporter: luhuachao -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-14952) Skip safemode if blockSafe is 0 in new NN
Rajesh Balamohan created HDFS-14952:
---
Summary: Skip safemode if blockSafe is 0 in new NN
Key: HDFS-14952
URL: https://issues.apache.org/jira/browse/HDFS-14952
Project: Hadoop HDFS
Issue Type: Improvement
Components: namenode
Reporter: Rajesh Balamohan
When new NN is installed, it spends 30-45 seconds in Safemode. When
{{blockTotal}} is 0, it should be possible to short circuit safemode check in
{{BlockManagerSafeMode::areThresholdsMet}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-10867) [PROVIDED Phase 2] Block Bit Field Allocation of Provided Storage
[ https://issues.apache.org/jira/browse/HDFS-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs resolved HDFS-10867. --- Resolution: Later This work will be done in a future ticket when we handle bidirectional stores (where the external store can create new files and have them show up in HDFS). > [PROVIDED Phase 2] Block Bit Field Allocation of Provided Storage > - > > Key: HDFS-10867 > URL: https://issues.apache.org/jira/browse/HDFS-10867 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Priority: Major > Attachments: Block Bit Field Allocation of Provided Storage.pdf > > > We wish to design and implement the following related features for provided > storage: > # Dynamic mounting of provided storage within a Namenode (mount, unmount) > # Mount multiple provided storage systems on a single Namenode. > # Support updates to the provided storage system without having to regenerate > an fsimg. > A mount in the namespace addresses a corresponding set of block data. When > unmounted, any block data associated with the mount becomes invalid and > (eventually) unaddressable in HDFS. As with erasure-coded blocks, efficient > unmounting requires that all blocks with that attribute be identifiable by > the block management layer > In this subtask, we focus on changes and conventions to the block management > layer. Namespace operations are covered in a separate subtask. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-1515) Create ozone dev-support script to check hadolint violiations
[ https://issues.apache.org/jira/browse/HDDS-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek resolved HDDS-1515. --- Fix Version/s: 0.5.0 Resolution: Fixed Merged to the master. Thanks [~akkidx] the contribution. > Create ozone dev-support script to check hadolint violiations > - > > Key: HDDS-1515 > URL: https://issues.apache.org/jira/browse/HDDS-1515 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Marton Elek >Assignee: Akshesh Doshi >Priority: Major > Labels: newbie, pull-request-available > Fix For: 0.5.0 > > Time Spent: 20m > Remaining Estimate: 0h > > hadoop-ozone/dev-support/checks/ directory contains helper scripts to execute > different code quality checks locally. > They are different from yetus as they can be executed in an easy way and they > check _ALL_ the violation of the current code base. > We need to create a new script to check the > [hadolint|https://github.com/hadolint/hadolint] errors in the hadoop-ozone > and hadoop-hdds projects. > The contracts of the check scripts: > # Exit code should define the result (0: passed, <>0 failed) > # Violation should be printed out to the stdout > We can assume that the hadolint is part of the development environment. For > jenkins we can put it to the image of the dev builds. > As the check introduce zero-tolerance for the hadolint violations the biggest > issue here is to eliminate all of the existing issues. > Thanks to [~eyang] for reporting that it's still missing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-2402) Adapt hadolint check to improved CI framework
Attila Doroszlai created HDDS-2402:
--
Summary: Adapt hadolint check to improved CI framework
Key: HDDS-2402
URL: https://issues.apache.org/jira/browse/HDDS-2402
Project: Hadoop Distributed Data Store
Issue Type: Improvement
Components: build
Affects Versions: 0.5.0
Reporter: Attila Doroszlai
HDDS-1515 added a script to run [hadolint|https://github.com/hadolint/hadolint]
as a CI check. However, the CI check framework contract has been improved a
bit in HDDS-2030 since HDDS-1515 was filed. More details can be found in the
[README|https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/dev-support/checks/README.md].
The goal of this task is to update the script to create the extra output files:
* save list of failures to {{summary.txt}} (while also outputting everything
to stdout)
* output number of failures to {{failures}}
Refer to
[checkstyle.sh|https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/dev-support/checks/checkstyle.sh]
for example.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Topic for this Wednesday's APAC Hadoop storage community sync: Ozone
Hi! There have been a number of questions especially from the Chinese community regarding Ozone lately. I am happy to have @Xiaoyu Yao who graciously agreed to talk to us the current status of Ozone this Wednesday even (US) / Thursday (APAC). Time: US Pacific: Wednesday (Nov 6) 10pm Beijing: Thursday (Nov 7) 2pm Zoom: https://cloudera.zoom.us/j/880548968 Past community sync minutes: https://docs.google.com/document/d/1jXM5Ujvf-zhcyw_5kiQVx6g-HeKe-YGnFS_1-qFXomI/edit?usp=sharing Best, Weichiu
[jira] [Created] (HDDS-2403) Remove leftover reference to OUTPUT_FILE from shellcheck.sh
Attila Doroszlai created HDDS-2403:
--
Summary: Remove leftover reference to OUTPUT_FILE from
shellcheck.sh
Key: HDDS-2403
URL: https://issues.apache.org/jira/browse/HDDS-2403
Project: Hadoop Distributed Data Store
Issue Type: Bug
Affects Versions: 0.5.0
Reporter: Attila Doroszlai
{{shellcheck.sh}} gives the following error (but works fine otherwise):
{noformat}
$ hadoop-ozone/dev-support/checks/shellcheck.sh
hadoop-ozone/dev-support/checks/shellcheck.sh: line 23: : No such file or
directory
...
{noformat}
This happens because {{OUTPUT_FILE}} variable is undefined:
{code:title=https://github.com/apache/hadoop-ozone/blob/6b2cda125b3647870ef5b01cf64e3b3e4cdc55db/hadoop-ozone/dev-support/checks/shellcheck.sh#L23}
echo "" > "$OUTPUT_FILE"
{code}
The command can be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-2404) Add support for Registered id as service identifier for CSR.
Anu Engineer created HDDS-2404: -- Summary: Add support for Registered id as service identifier for CSR. Key: HDDS-2404 URL: https://issues.apache.org/jira/browse/HDDS-2404 Project: Hadoop Distributed Data Store Issue Type: Improvement Components: SCM Reporter: Anu Engineer The SCM HA needs the ability to represent a group as a single entity. So that Tokens for each of the OM which is part of an HA group can be honored by the data nodes. This patch adds the notion of a service group ID to the Certificate Infrastructure. In the next JIRAs, we will use this capability when issuing certificates to OM -- especially when they are in HA mode. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-2405) int2ByteString unnecessary byte array allocation
Attila Doroszlai created HDDS-2405:
--
Summary: int2ByteString unnecessary byte array allocation
Key: HDDS-2405
URL: https://issues.apache.org/jira/browse/HDDS-2405
Project: Hadoop Distributed Data Store
Issue Type: Bug
Affects Versions: 0.5.0
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai
{{int2ByteString}} implementations (currently duplicated in
[RatisHelper|https://github.com/apache/hadoop-ozone/blob/6b2cda125b3647870ef5b01cf64e3b3e4cdc55db/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/RatisHelper.java#L280-L289]
and
[Checksum|https://github.com/apache/hadoop-ozone/blob/6b2cda125b3647870ef5b01cf64e3b3e4cdc55db/hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/Checksum.java#L64-L73],
but the first one is being removed in HDDS-2375) result in unnecessary byte
array allocations:
# {{ByteString.Output}} creates 128-byte buffer by default, which is too large
for writing a single int
# {{DataOutputStream}} allocates an [extra 8-byte
array|https://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/io/DataOutputStream.java#l204],
used only for writing longs
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Reopened] (HDDS-2208) Propagate System Exceptions from OM transaction apply phase
[ https://issues.apache.org/jira/browse/HDDS-2208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal reopened HDDS-2208: - > Propagate System Exceptions from OM transaction apply phase > --- > > Key: HDDS-2208 > URL: https://issues.apache.org/jira/browse/HDDS-2208 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task > Components: Ozone Manager >Reporter: Supratim Deka >Assignee: Supratim Deka >Priority: Major > Labels: pull-request-available > Fix For: 0.5.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The change for HDDS-2206 tracks system exceptions during preExecute phase of > OM request handling. > The current jira is to implement exception propagation once the OM request is > submitted to Ratis - when the handler is running validateAndUpdateCache for > the request. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-2255) Improve Acl Handler Messages
[ https://issues.apache.org/jira/browse/HDDS-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham resolved HDDS-2255. -- Fix Version/s: 0.5.0 Resolution: Fixed > Improve Acl Handler Messages > > > Key: HDDS-2255 > URL: https://issues.apache.org/jira/browse/HDDS-2255 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: om >Reporter: Hanisha Koneru >Assignee: YiSheng Lien >Priority: Minor > Labels: newbie, pull-request-available > Fix For: 0.5.0 > > Time Spent: 20m > Remaining Estimate: 0h > > In Add/Remove/Set Acl Key/Bucket/Volume Handlers, we print a message about > whether the operation was successful or not. If we are trying to add an ACL > which is already existing, we convey the message that the operation failed. > It would be better if the message conveyed more clearly why the operation > failed i.e. the ACL already exists. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
