[
https://issues.apache.org/jira/browse/HBASE-30288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18105340#comment-18105340
]
mazhengxuan commented on HBASE-30288:
-------------------------------------
I looked into this issue.
WALPlayer parses the source tables and target table mappings correctly, but the
bulk-output path does not actually use the mappings consistently:
1. createSubmittableJob configures MultiTableHFileOutputFormat with the source
tables, so the generated HFiles use the source table descriptors and region
boundaries instead of those of the target tables.
2. WALKeyValueMapper only keeps a set of source tables and emits the source
table name in the composite output key. It does not read TABLE_MAP_KEY.
I propose to:
- continue using the source table names for filtering WAL entries;
- build the source-to-target mapping in WALKeyValueMapper and emit the target
table name in the composite key;
- configure the incremental-load TableInfo list using the mapped target tables;
- add an end-to-end TestWALPlayer case that generates HFiles for a
source-to-target mapping, bulk-loads them into the target table, and verifies
the data.
This should leave the non-bulk WALMapper path unchanged.
I also noticed that the current WALPlayer usage text still says that table
mappings are not allowed with bulk output. If bulk table mapping is intended to
be supported, I can update that documentation as part of the fix.
I would like to work on this issue. Could I take it?
> WALPlayer doesn't pass in the correct table mappings when using the bulk
> output option
> --------------------------------------------------------------------------------------
>
> Key: HBASE-30288
> URL: https://issues.apache.org/jira/browse/HBASE-30288
> Project: HBase
> Issue Type: Bug
> Components: walplayer
> Reporter: Hernan Romer
> Priority: Major
>
> WALPlayer doesn't pass in the correct table mappings when using the bulk
> output option. This means we won't bulkload to the correct locations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
