David Tucker created HDFS-13392:
-----------------------------------
Summary: Incorrect length in Truncate CloseEvents
Key: HDFS-13392
URL: https://issues.apache.org/jira/browse/HDFS-13392
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 3.0.0
Reporter: David Tucker
Under stress (multiple clients truncating separate non-empty files in half
simultaneously), the CloseEvent triggered by a Truncate RPC may contain an
incorrect length. We're able to reproduce this reliably ~20% of the time (our
tests are somewhat randomized/fuzzy).
For example, given this Truncate request:
{noformat}
Request:
truncate {
src:
"/chai_test65c9a2a0-1188-439d-92e2-96a81c14a266-\357\254\200\357\272\217\357\255\217\343\203\276\324\262\342\204\200\342\213\251/chai_testbd968366-0016-4462-ac12-e48e0487bebd-\340\270\215\334\200\311\226\342\202\242\343\202\236\340\256\205\357\272\217/chai_testb5b155e8-b331-4f67-bdfa-546f82128b5d-\312\254\340\272\201\343\202\242\306\220\340\244\205\342\202\242\343\204\270a\334\240\337\213\340\244\240\343\200\243\342\202\243\343\203\276\313\225\346\206\250"
newLength: 2003855
clientName:
"\341\264\275\327\220\343\203\250\333\263\343\220\205\357\254\227\340\270\201\340\245\251\306\225\341\203\265\334\220\342\202\243\343\204\206!A\343\206\215\357\254\201\340\273\223\347\224\260"
}
Block Size: 1048576B
Old length: 4007711B (3.82205104828 blocks)
Truncation: 2003856B (1.91102600098 blocks)
New length: 2003855B (1.9110250473 blocks)
Response:
result: true
{noformat}
We see these INotify events:
{noformat}
TruncateEvent {
path:
/chai_test65c9a2a0-1188-439d-92e2-96a81c14a266-ffﺏﭏヾԲ℀⋩/chai_testbd968366-0016-4462-ac12-e48e0487bebd-ญ܀ɖ₢ゞஅﺏ/chai_testb5b155e8-b331-4f67-bdfa-546f82128b5d-ʬກアƐअ₢ㄸaܠߋठ〣₣ヾ˕憨
length: 2003855
timestamp: 1522716573143
}
{noformat}
{noformat}
CloseEvent {
path:
/chai_test65c9a2a0-1188-439d-92e2-96a81c14a266-ffﺏﭏヾԲ℀⋩/chai_testbd968366-0016-4462-ac12-e48e0487bebd-ญ܀ɖ₢ゞஅﺏ/chai_testb5b155e8-b331-4f67-bdfa-546f82128b5d-ʬກアƐअ₢ㄸaܠߋठ〣₣ヾ˕憨
length: -2
timestamp: 1522716575723
}
{noformat}
{{-2}} is not the only number that shows up as the length,
{{9223372036854775807}} is common too. These are detected by Python 2 tests,
and the latter is {{sys.maxint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
