Now I see that it uses the distributed cache - but for some reason the TotalOrderPartitioner does not grab it. Ophir
On Thu, Aug 11, 2011 at 11:08, Ophir Cohen <[email protected]> wrote: > Hi, > I started to use bulk upload and encounter a strange problem. > I'm using Cloudera cdh3-u1. > > I'm using HFileOutputFormat.configureIncrementalLoad() to configure my > job. > This method create partition file for the TotalOrderPartitioner and save it > to HDFS. > > When the TotalOrderPartitioner initiated it tries to find the path for the > file in the configuration: > public static String getPartitionFile(Configuration conf) { > return conf.get(PARTITIONER_PATH, DEFAULT_PATH); > } > > The strange thing is that this parameter never assigned! > It looks to me that it should have configured > in HFileOutputFormat.configureIncrementalLoad() but it does not! > > Then it takes the default ("_part") or something similar and (of course) > does not find it... > > BTW > When I manually add this parameter it works great. > > Is that a bug or do I miss something? > Thanks, > Ophir > >
