Hi Fengnan,
We had been testing cost based faire call queue internally. We also saw
latency increase, and we are trying to debug into this issue as well.
Current suspicion is that the way that the metrics were generated might be
introducing too much overhead. We are in the process of trying to rep
Chen Liang created HDFS-15404:
-
Summary: ShellCommandFencer should expose info about source
Key: HDFS-15404
URL: https://issues.apache.org/jira/browse/HDFS-15404
Project: Hadoop HDFS
Issue Type
Chen Liang created HDFS-15293:
-
Summary: Relax FSImage upload time delta check restriction
Key: HDFS-15293
URL: https://issues.apache.org/jira/browse/HDFS-15293
Project: Hadoop HDFS
Issue Type
Chen Liang created HDFS-15197:
-
Summary: Change ObserverRetryOnActiveException log to debug
Key: HDFS-15197
URL: https://issues.apache.org/jira/browse/HDFS-15197
Project: Hadoop HDFS
Issue Type
[
https://issues.apache.org/jira/browse/HDFS-15153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Liang resolved HDFS-15153.
---
Resolution: Duplicate
> TestDelegationTokensWithHA.testObserverReadProxyProviderWithDT fa
Chen Liang created HDFS-15153:
-
Summary:
TestDelegationTokensWithHA.testObserverReadProxyProviderWithDT fails
intermittently
Key: HDFS-15153
URL: https://issues.apache.org/jira/browse/HDFS-15153
Project
Chen Liang created HDFS-15148:
-
Summary: dfs.namenode.send.qop.enabled should not apply to primary
NN port
Key: HDFS-15148
URL: https://issues.apache.org/jira/browse/HDFS-15148
Project: Hadoop HDFS
Thanks again for driving the syncup Wei-Chiu! I am also super interested
in learning about this upgrade.
Moving forward, I am happy to help with the translation. Please feel free
to let me know if you need more hands on it :).
Chen
Wei-Chiu Chuang 于2019年12月19日周四 下午6:23写道：
> Yeah, for sure. I
Chen Liang created HDFS-14991:
-
Summary: Backport HDFS-14346 Better time precision in
getTimeDuration to branch-2
Key: HDFS-14991
URL: https://issues.apache.org/jira/browse/HDFS-14991
Project: Hadoop
Chen Liang created HDFS-14941:
-
Summary: Potential editlog race condition can cause corrupted file
Key: HDFS-14941
URL: https://issues.apache.org/jira/browse/HDFS-14941
Project: Hadoop HDFS
[
https://issues.apache.org/jira/browse/HDFS-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Liang reopened HDFS-12979:
---
Thanks for the catch [~shv]. I've committed to branch-3.2 and branch-3.1 as
there were only
Chen Liang created HDFS-14858:
-
Summary: [SBN read] Allow configurably enable/disable
AlignmentContext on NameNode
Key: HDFS-14858
URL: https://issues.apache.org/jira/browse/HDFS-14858
Project: Hadoop
Late +1. I am interested in the project too. Please include me.
Regards,
Chen
Varun Saxena 于2019年9月7日周六 上午10:35写道：
> Belated +1.
> I too am interested in the project. Please include me.
>
> Regards,
> Varun Saxena
>
> On Sat, Sep 7, 2019 at 9:35 PM Sunil Govindan wrote:
>
> > +1 to the propo
Chen Liang created HDFS-14822:
-
Summary: Revisit GlobalStateIdContext locking when getting server
state id
Key: HDFS-14822
URL: https://issues.apache.org/jira/browse/HDFS-14822
Project: Hadoop HDFS
Chen Liang created HDFS-14806:
-
Summary: Bootstrap standby may fail if used in-progress tailing
Key: HDFS-14806
URL: https://issues.apache.org/jira/browse/HDFS-14806
Project: Hadoop HDFS
Issue
Chen Liang created HDFS-14785:
-
Summary: [SBN read] Change client logging to be less aggressive
Key: HDFS-14785
URL: https://issues.apache.org/jira/browse/HDFS-14785
Project: Hadoop HDFS
Issue
Chen Liang created HDFS-14726:
-
Summary: Fix JN incompatibility issue in branch-2 due to backport
of HDFS-10519
Key: HDFS-14726
URL: https://issues.apache.org/jira/browse/HDFS-14726
Project: Hadoop HDFS
Chen Liang created HDFS-14611:
-
Summary: Move handshake secret field from Token to BlockAccessToken
Key: HDFS-14611
URL: https://issues.apache.org/jira/browse/HDFS-14611
Project: Hadoop HDFS
Chen Liang created HDFS-14573:
-
Summary: Backport Standby Read to branch-3
Key: HDFS-14573
URL: https://issues.apache.org/jira/browse/HDFS-14573
Project: Hadoop HDFS
Issue Type: Task
Chen Liang created HDFS-14205:
-
Summary: Backport HDFS-6440 to branch-2
Key: HDFS-14205
URL: https://issues.apache.org/jira/browse/HDFS-14205
Project: Hadoop HDFS
Issue Type: Improvement
Chen Liang created HDFS-14204:
-
Summary: Backport HDFS-12943 to branch-2
Key: HDFS-14204
URL: https://issues.apache.org/jira/browse/HDFS-14204
Project: Hadoop HDFS
Issue Type: Improvement
+1
Thanks Konstantin for driving the merge vote!
I have been working on the development and testing of the feature. It has
been running for several weeks on our cluster with ~100 nodes, which has HA
and Kerberos enabled. I have been able to run several different MapReduce
jobs and HDFS benchmarks
Hi Wei-Chiu,
HDFS-13566 and HDFS-13547 do allow different SASL configurations to coexist
on different NameNode ports, but for the whole feature to work, a upgrade
on the client jar is still required. I'm not too familiar with how HBase
accesses HDFS, based on your description, for this particular
Chen Liang created HDFS-14142:
-
Summary: Move ipfailover config key out of HdfsClientConfigKeys
Key: HDFS-14142
URL: https://issues.apache.org/jira/browse/HDFS-14142
Project: Hadoop HDFS
Issue
Hi Daryn,
This is an interesting and valid point to consider different implications
for security.
The purpose of the alignment context is to allow clients and servers sync
on their global state, so that when clients switch between ANN/SBN or
between SBNs, the reads are always consistent. One reas
Chen Liang created HDFS-14120:
-
Summary: ORFPP should also clone DT for the virtual IP
Key: HDFS-14120
URL: https://issues.apache.org/jira/browse/HDFS-14120
Project: Hadoop HDFS
Issue Type: Sub
[
https://issues.apache.org/jira/browse/HDFS-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Liang resolved HDFS-13547.
---
Resolution: Fixed
Fix Version/s: 3.1.1
> Add ingress port based sasl resol
Chen Liang created HDFS-14116:
-
Summary: Fix a potential class cast error in
ObserverReadProxyProvider
Key: HDFS-14116
URL: https://issues.apache.org/jira/browse/HDFS-14116
Project: Hadoop HDFS
Chen Liang created HDFS-14035:
-
Summary: NN status discovery does not leverage delegation token
Key: HDFS-14035
URL: https://issues.apache.org/jira/browse/HDFS-14035
Project: Hadoop HDFS
Issue
Chen Liang created HDFS-14017:
-
Summary: ObserverReadProxyProviderWithIPFailover does not quite
work
Key: HDFS-14017
URL: https://issues.apache.org/jira/browse/HDFS-14017
Project: Hadoop HDFS
Chen Liang created HDFS-14016:
-
Summary: ObserverReadProxyProvider can never enable observer read
except in tests
Key: HDFS-14016
URL: https://issues.apache.org/jira/browse/HDFS-14016
Project: Hadoop
Chen Liang created HDFS-13880:
-
Summary: Add mechanism to allow certain RPC calls to bypass sync
Key: HDFS-13880
URL: https://issues.apache.org/jira/browse/HDFS-13880
Project: Hadoop HDFS
Issue
Chen Liang created HDFS-13767:
-
Summary: Add msync server implementation.
Key: HDFS-13767
URL: https://issues.apache.org/jira/browse/HDFS-13767
Project: Hadoop HDFS
Issue Type: Sub-task
Chen Liang created HDFS-13699:
-
Summary: Add DFSClient sending handshake token to DataNode, and
allow DataNode overwrite downstream QOP
Key: HDFS-13699
URL: https://issues.apache.org/jira/browse/HDFS-13699
Chen Liang created HDFS-13688:
-
Summary: Introduce msync API call
Key: HDFS-13688
URL: https://issues.apache.org/jira/browse/HDFS-13688
Project: Hadoop HDFS
Issue Type: Sub-task
Chen Liang created HDFS-13617:
-
Summary: Allow wrapping NN QOP into token in encrypted message
Key: HDFS-13617
URL: https://issues.apache.org/jira/browse/HDFS-13617
Project: Hadoop HDFS
Issue
Chen Liang created HDFS-13566:
-
Summary: Add configurable additional RPC listener to NameNode
Key: HDFS-13566
URL: https://issues.apache.org/jira/browse/HDFS-13566
Project: Hadoop HDFS
Issue
Chen Liang created HDFS-13547:
-
Summary: Add ingress port based sasl resolver
Key: HDFS-13547
URL: https://issues.apache.org/jira/browse/HDFS-13547
Project: Hadoop HDFS
Issue Type: Sub-task
Chen Liang created HDFS-13541:
-
Summary: NameNode Port based selective encryption
Key: HDFS-13541
URL: https://issues.apache.org/jira/browse/HDFS-13541
Project: Hadoop HDFS
Issue Type
+1 (non-binding)
Thanks!
Chen
2018-04-22 14:04 GMT-07:00 Elek, Marton :
> +1 (non-binding)
>
> Tested the branch and everything worked well, even with turning on the
> optional maven profile. (Yarn wordcount, Hive, Spark, HBase)
>
> Marton
>
>
> On 04/17/2018 01:48 AM, Jitendra Pandey wrote:
>
Thanks for working on this Konstantin!
+1 (non-binding)
- verified checksum
- built from source
- started a single node HDFS cluster
- performed basic operations of ls/mkdir/put/get
- checked web UI
- ran the MR job pi
Regards,
Chen
> On Apr 9, 2018, at 4:14 PM, Konstantin Shvachko wrote:
>
+1 (non-binding).
Thanks,
Chen
> On Mar 20, 2018, at 11:20 AM, Owen O'Malley wrote:
>
> All,
>
> Following our discussions on the previous thread (Merging branch HDFS-7240
> to trunk), I'd like to propose the following:
>
> * HDSL become a subproject of Hadoop.
> * HDSL will release separate
Chen Liang created HDFS-12989:
-
Summary: Ozone: Container : Add key versioning support-4
Key: HDFS-12989
URL: https://issues.apache.org/jira/browse/HDFS-12989
Project: Hadoop HDFS
Issue Type
Chen Liang created HDFS-12958:
-
Summary: Ozone: remove setAllocatedBytes method in ContainerInfo
Key: HDFS-12958
URL: https://issues.apache.org/jira/browse/HDFS-12958
Project: Hadoop HDFS
Issue
Chen Liang created HDFS-12954:
-
Summary: Ozone: Container : Add key versioning support-3
Key: HDFS-12954
URL: https://issues.apache.org/jira/browse/HDFS-12954
Project: Hadoop HDFS
Issue Type
Chen Liang created HDFS-12925:
-
Summary: Ozone: Container : Add key versioning support-2
Key: HDFS-12925
URL: https://issues.apache.org/jira/browse/HDFS-12925
Project: Hadoop HDFS
Issue Type
[
https://issues.apache.org/jira/browse/HDFS-12265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Liang resolved HDFS-12265.
---
Resolution: Fixed
Release Note: Looks like this has been handled as part of HDFS-12387, close
Chen Liang created HDFS-12879:
-
Summary: Ozone : add scm init command to document.
Key: HDFS-12879
URL: https://issues.apache.org/jira/browse/HDFS-12879
Project: Hadoop HDFS
Issue Type
Chen Liang created HDFS-12793:
-
Summary: Ozone : TestSCMCli is failing consistently
Key: HDFS-12793
URL: https://issues.apache.org/jira/browse/HDFS-12793
Project: Hadoop HDFS
Issue Type: Bug
Chen Liang created HDFS-12626:
-
Summary: Ozone : delete open key entries that will no longer be
closed
Key: HDFS-12626
URL: https://issues.apache.org/jira/browse/HDFS-12626
Project: Hadoop HDFS
Chen Liang created HDFS-12543:
-
Summary: Ozone : allow create key without specifying size
Key: HDFS-12543
URL: https://issues.apache.org/jira/browse/HDFS-12543
Project: Hadoop HDFS
Issue Type
Chen Liang created HDFS-12475:
-
Summary: Ozone : add document for port sharing with WebHDFS
Key: HDFS-12475
URL: https://issues.apache.org/jira/browse/HDFS-12475
Project: Hadoop HDFS
Issue Type
[
https://issues.apache.org/jira/browse/HDFS-12268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Liang reopened HDFS-12268:
---
> Ozone: Add metrics for pending storage container reque
Chen Liang created HDFS-12454:
-
Summary: Ozone : the sample ozone-site.xml in OzoneGettingStarted
does not work
Key: HDFS-12454
URL: https://issues.apache.org/jira/browse/HDFS-12454
Project: Hadoop HDFS
Chen Liang created HDFS-12368:
-
Summary: [branch-2] Enable DFSNetworkTopology as default
Key: HDFS-12368
URL: https://issues.apache.org/jira/browse/HDFS-12368
Project: Hadoop HDFS
Issue Type
Chen Liang created HDFS-12352:
-
Summary: [branch-2] Use HDFS specific network topology to choose
datanode in BlockPlacementPolicyDefault
Key: HDFS-12352
URL: https://issues.apache.org/jira/browse/HDFS-12352
Chen Liang created HDFS-12346:
-
Summary: [branch-2] Combine the old and the new chooseRandom for
better performance
Key: HDFS-12346
URL: https://issues.apache.org/jira/browse/HDFS-12346
Project: Hadoop
Chen Liang created HDFS-12334:
-
Summary: [branch-2] Add storage type demand to into
DFSNetworkTopology#chooseRandom
Key: HDFS-12334
URL: https://issues.apache.org/jira/browse/HDFS-12334
Project: Hadoop
Chen Liang created HDFS-12325:
-
Summary: SFTPFileSystem operations should restore cwd
Key: HDFS-12325
URL: https://issues.apache.org/jira/browse/HDFS-12325
Project: Hadoop HDFS
Issue Type: Bug
Chen Liang created HDFS-12322:
-
Summary: [branch-2] Add storage type demand to into
DFSNetworkTopology#chooseRandom.
Key: HDFS-12322
URL: https://issues.apache.org/jira/browse/HDFS-12322
Project: Hadoop
Chen Liang created HDFS-12321:
-
Summary: Ozone : debug cli: add support to load user-provided SQL
query
Key: HDFS-12321
URL: https://issues.apache.org/jira/browse/HDFS-12321
Project: Hadoop HDFS
Chen Liang created HDFS-12311:
-
Summary: [branch-2] HDFS specific network topology classes with
storage type info included
Key: HDFS-12311
URL: https://issues.apache.org/jira/browse/HDFS-12311
Project
Chen Liang created HDFS-12306:
-
Summary: [branch-2]Separate class InnerNode from class
NetworkTopology and make it extendable
Key: HDFS-12306
URL: https://issues.apache.org/jira/browse/HDFS-12306
Project
Chen Liang created HDFS-12304:
-
Summary: Remove unused parameter from FsDatasetImpl#addVolume
Key: HDFS-12304
URL: https://issues.apache.org/jira/browse/HDFS-12304
Project: Hadoop HDFS
Issue
Chen Liang created HDFS-12287:
-
Summary: Remove a no-longer applicable TODO comment in
DatanodeManager
Key: HDFS-12287
URL: https://issues.apache.org/jira/browse/HDFS-12287
Project: Hadoop HDFS
Chen Liang created HDFS-12266:
-
Summary: Ozone : add debug cli to hdfs script
Key: HDFS-12266
URL: https://issues.apache.org/jira/browse/HDFS-12266
Project: Hadoop HDFS
Issue Type: Sub-task
Chen Liang created HDFS-12265:
-
Summary: Ozone : better handling of operation fail due to chill
mode
Key: HDFS-12265
URL: https://issues.apache.org/jira/browse/HDFS-12265
Project: Hadoop HDFS
Chen Liang created HDFS-12256:
-
Summary: Ozone : handle inactive containers on DataNode side
Key: HDFS-12256
URL: https://issues.apache.org/jira/browse/HDFS-12256
Project: Hadoop HDFS
Issue Type
Hi Konstantin,
Thanks for putting up 2.7.4-RC0 release!
+1 (non-binding).
With following checks:
- build the source on MacOS 10.12.5 with java version 1.8.0_91.
- deployed on a single node pseudo-custer
- tested HDFS basics: write/read/append/acl
- mapreduce job wordcount/bbp/grep
Thanks!
Chen
Chen Liang created HDFS-12187:
-
Summary: Ozone : add support to DEBUG CLI for ksm.db
Key: HDFS-12187
URL: https://issues.apache.org/jira/browse/HDFS-12187
Project: Hadoop HDFS
Issue Type: Sub
Chen Liang created HDFS-12155:
-
Summary: Ozone : add RocksDB support to DEBUG CLI
Key: HDFS-12155
URL: https://issues.apache.org/jira/browse/HDFS-12155
Project: Hadoop HDFS
Issue Type: Sub-task
Chen Liang created HDFS-12138:
-
Summary: Remove redundant 'public' modifiers from BlockCollection
Key: HDFS-12138
URL: https://issues.apache.org/jira/browse/HDFS-12138
Project: Hadoop HDFS
Chen Liang created HDFS-12130:
-
Summary: Optimizing permission check for getContentSummary
Key: HDFS-12130
URL: https://issues.apache.org/jira/browse/HDFS-12130
Project: Hadoop HDFS
Issue Type
[
https://issues.apache.org/jira/browse/HDFS-12041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Liang resolved HDFS-12041.
---
Resolution: Won't Fix
> Block Storage : make the server address config more
Chen Liang created HDFS-12043:
-
Summary: Add counters for block re-replication
Key: HDFS-12043
URL: https://issues.apache.org/jira/browse/HDFS-12043
Project: Hadoop HDFS
Issue Type: Bug
Chen Liang created HDFS-12041:
-
Summary: Block Storage : make the server address config more
concise
Key: HDFS-12041
URL: https://issues.apache.org/jira/browse/HDFS-12041
Project: Hadoop HDFS
Chen Liang created HDFS-12002:
-
Summary: Ozone : SCM cli misc fixes/improvements
Key: HDFS-12002
URL: https://issues.apache.org/jira/browse/HDFS-12002
Project: Hadoop HDFS
Issue Type: Sub-task
Chen Liang created HDFS-11998:
-
Summary: Enable DFSNetworkTopology as default
Key: HDFS-11998
URL: https://issues.apache.org/jira/browse/HDFS-11998
Project: Hadoop HDFS
Issue Type: Sub-task
Chen Liang created HDFS-11997:
-
Summary: ChunkManager functions do not use the argument keyName
Key: HDFS-11997
URL: https://issues.apache.org/jira/browse/HDFS-11997
Project: Hadoop HDFS
Issue
Chen Liang created HDFS-11996:
-
Summary: Ozone : add partial read of chunks
Key: HDFS-11996
URL: https://issues.apache.org/jira/browse/HDFS-11996
Project: Hadoop HDFS
Issue Type: Sub-task
Chen Liang created HDFS-11939:
-
Summary: Ozone : add read/write random access to Chunks of a key
Key: HDFS-11939
URL: https://issues.apache.org/jira/browse/HDFS-11939
Project: Hadoop HDFS
Issue
Chen Liang created HDFS-11932:
-
Summary: BPServiceActor thread name is not correctly set
Key: HDFS-11932
URL: https://issues.apache.org/jira/browse/HDFS-11932
Project: Hadoop HDFS
Issue Type
Chen Liang created HDFS-11923:
-
Summary: Stress test of DFSNetworkTopology
Key: HDFS-11923
URL: https://issues.apache.org/jira/browse/HDFS-11923
Project: Hadoop HDFS
Issue Type: Sub-task
Chen Liang created HDFS-11920:
-
Summary: Ozone : add key partition
Key: HDFS-11920
URL: https://issues.apache.org/jira/browse/HDFS-11920
Project: Hadoop HDFS
Issue Type: Sub-task
Chen Liang created HDFS-11907:
-
Summary: NameNodeResourceChecker should avoid calling
df.getAvailable too frequently
Key: HDFS-11907
URL: https://issues.apache.org/jira/browse/HDFS-11907
Project: Hadoop
Chen Liang created HDFS-11906:
-
Summary: Add log for NameNode#monitorHealth
Key: HDFS-11906
URL: https://issues.apache.org/jira/browse/HDFS-11906
Project: Hadoop HDFS
Issue Type: Improvement
Chen Liang created HDFS-11891:
-
Summary: DU#refresh should print the path of the directory when an
exception is caught
Key: HDFS-11891
URL: https://issues.apache.org/jira/browse/HDFS-11891
Project
Chen Liang created HDFS-11886:
-
Summary: Ozone : improving error handling for putkey operation
Key: HDFS-11886
URL: https://issues.apache.org/jira/browse/HDFS-11886
Project: Hadoop HDFS
Issue
[
https://issues.apache.org/jira/browse/HDFS-11872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Liang resolved HDFS-11872.
---
Resolution: Won't Fix
I misread {{getStorageContainerLocations}} as the lookup of container
Chen Liang created HDFS-11872:
-
Summary: Ozone : implement
StorageContainerManager#getStorageContainerLocations
Key: HDFS-11872
URL: https://issues.apache.org/jira/browse/HDFS-11872
Project: Hadoop HDFS
Chen Liang created HDFS-11859:
-
Summary: Ozone : separate blockLocationProtocol out of
containerLocationProtocol
Key: HDFS-11859
URL: https://issues.apache.org/jira/browse/HDFS-11859
Project: Hadoop HDFS
Chen Liang created HDFS-11857:
-
Summary: Ozone : need to refactor
StorageContainerLocationProtocolServerSideTranslatorPB
Key: HDFS-11857
URL: https://issues.apache.org/jira/browse/HDFS-11857
Project
Chen Liang created HDFS-11836:
-
Summary: Ozone : add sql debug CLI to hdfs script
Key: HDFS-11836
URL: https://issues.apache.org/jira/browse/HDFS-11836
Project: Hadoop HDFS
Issue Type: Sub-task
Chen Liang created HDFS-11802:
-
Summary: Ozone : add DEBUG CLI support for open container db file
Key: HDFS-11802
URL: https://issues.apache.org/jira/browse/HDFS-11802
Project: Hadoop HDFS
Issue
Chen Liang created HDFS-11788:
-
Summary: Ozone : add DEBUG CLI support for nodepool db file
Key: HDFS-11788
URL: https://issues.apache.org/jira/browse/HDFS-11788
Project: Hadoop HDFS
Issue Type
Chen Liang created HDFS-11759:
-
Summary: Ozone : SCMNodeManager#close() should also close node
pool manager object
Key: HDFS-11759
URL: https://issues.apache.org/jira/browse/HDFS-11759
Project: Hadoop
Chen Liang created HDFS-11756:
-
Summary: Ozone : add DEBUG CLI support of blockDB file
Key: HDFS-11756
URL: https://issues.apache.org/jira/browse/HDFS-11756
Project: Hadoop HDFS
Issue Type: Sub
Chen Liang created HDFS-11747:
-
Summary: Ozone : need to fix OZONE_SCM_DEFAULT_PORT
Key: HDFS-11747
URL: https://issues.apache.org/jira/browse/HDFS-11747
Project: Hadoop HDFS
Issue Type: Sub
Chen Liang created HDFS-11728:
-
Summary: Ozone: add the DB names to OzoneConsts
Key: HDFS-11728
URL: https://issues.apache.org/jira/browse/HDFS-11728
Project: Hadoop HDFS
Issue Type: Sub-task
Chen Liang created HDFS-11723:
-
Summary: Should log a warning message when users try to make
certain directories encryption zone
Key: HDFS-11723
URL: https://issues.apache.org/jira/browse/HDFS-11723
1 - 100 of 142 matches
Mail list logo
