Re: Task Serialization Error on DataFrame.foreachPartition

2015-06-22 Thread Ted Yu
private HTable table; You should declare table variable within apply() method. BTW which hbase release are you using ? I see you implement caching yourself. You can make use of the following HTable method: public void setWriteBufferSize(long writeBufferSize) throws IOExcep

Re: Task Serialization Error on DataFrame.foreachPartition

2015-06-21 Thread Ted Yu
Can you show us the code for loading Hive into hbase ? There shouldn't be 'return' statement in that code. Cheers > On Jun 20, 2015, at 10:10 PM, Nishant Patel wrote: > > Hi, > > I am loading data from Hive table to Hbase after doing some manipulation. > > I am getting error as 'Task not