[
https://issues.apache.org/jira/browse/HBASE-29696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18098094#comment-18098094
]
mazhengxuan commented on HBASE-29696:
-------------------------------------
I'd like to work on this issue. I plan to use the synthetic boundaries only for
calculating the intermediate split keys, then restore the original start and
end rows on the first and last generated splits. This keeps the final split
open-ended and prevents binary row keys such as 0xFFFF from being skipped. I
will also add regression tests for both empty-to-empty and non-empty-to-empty
ranges.
> TableInputFormatBase with NUM_MAPPERS_PER_REGION produces incorrect last
> InputSplit
> -----------------------------------------------------------------------------------
>
> Key: HBASE-29696
> URL: https://issues.apache.org/jira/browse/HBASE-29696
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 3.0.0, 2.6.3
> Reporter: Jasper van Maanen
> Priority: Major
>
> *How to reproduce*
> Create a table with a single region and add the row key
> {code:java}
> new byte[] {-1, -1}{code}
> Perform a MapReduce job with at least this setting:
> {code:java}
> "hbase.mapreduce.tableinput.mappers.per.region" = 2 {code}
> The row key is missing in the scan result.
>
> *Analysis*
> In TableInputF{color:#172b4d}ormatBase#createNInputSplitsUniform there is
> this code:
> {color}
> {code:java}
> // For special case: startRow or endRow is empty
> if (startRow.length == 0 && endRow.length == 0) {
> startRow = new byte[1];
> endRow = new byte[1];
> startRow[0] = 0;
> endRow[0] = -1;
> }
> if (startRow.length == 0 && endRow.length != 0) {
> startRow = new byte[1];
> startRow[0] = 0;
> }
> if (startRow.length != 0 && endRow.length == 0) {
> endRow = new byte[startRow.length];
> for (int k = 0; k < startRow.length; k++) {
> endRow[k] = -1;
> }
> } {code}
> Unfortunately, in the first 'if', the endRow is set to
> {code:java}
> new byte[] {-1} {code}
> But what if there is a row key with
> {code:java}
> new byte[] {-1, -1}{code}
> This row key is after the endRow and will be ignored by the scan.
> This is also an issue in the third 'if'. Since a row key can be of
> potentially of an infinite length, setting the end row in the third 'if' also
> prevents longer row keys (compared to the start row) to be ignored.
> Therefore, in both situations, the end row should stay empty. I can imagine
> the endRow is set for the next step:
> {code:java}
> // Split Region into n chunks evenly
> byte[][] splitKeys = Bytes.split(startRow, endRow, true, n - 1); {code}
> In that case, some compensation should be done later to return the correct
> input splits.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
