Hi,

We are using HBaseStorage intensively to load data from tables having more than 100 regions.

HBaseStorage generates 1 map par region, and our cluster having 50 map slots, it happens that our PIG scripts start 50 maps reading concurrently data from HBase.

The problem is that our HBase cluster has only 10 nodes, and thus the maps overload it (5 intensive readers per node is too much to bare).

So question: is there a way to say to PIG : limit the nb of maps to this maximum (ex: 10) ?
If not, how can I patch the code to do this ?

Thanks a lot for your help

Reply via email to