----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30739/#review71597 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveKVResultCache.java <https://reviews.apache.org/r/30739/#comment117412> If I understand correctly, this can be renamed to something like IN_MEMORY_NUM_ROWS? ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveKVResultCache.java <https://reviews.apache.org/r/30739/#comment117419> Do we need a parameter here? Seems it can just use writeCursor? ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveKVResultCache.java <https://reviews.apache.org/r/30739/#comment117420> Also close input and output here? ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveKVResultCache.java <https://reviews.apache.org/r/30739/#comment117425> I suppose this is to avoid frequent switch buffer? But why the magic number 1? - Rui Li On Feb. 7, 2015, 3:09 a.m., Jimmy Xiang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/30739/ > ----------------------------------------------------------- > > (Updated Feb. 7, 2015, 3:09 a.m.) > > > Review request for hive, Rui Li and Xuefu Zhang. > > > Bugs: HIVE-9574 > https://issues.apache.org/jira/browse/HIVE-9574 > > > Repository: hive-git > > > Description > ------- > > Result KV cache doesn't use RowContainer any more since it has logic we don't > need, which is some overhead. We don't do lazy computing right away, instead > we wait a little till the cache is close to spill. > > > Diffs > ----- > > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveBaseFunctionResultList.java > 78ab680 > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveKVResultCache.java > 8ead0cb > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveMapFunction.java > 7a09b4d > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveMapFunctionResultList.java > e92e299 > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveReduceFunction.java > 070ea4d > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveReduceFunctionResultList.java > d4ff37c > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/KryoSerializer.java > 286816b > ql/src/test/org/apache/hadoop/hive/ql/exec/spark/TestHiveKVResultCache.java > 0df4598 > > Diff: https://reviews.apache.org/r/30739/diff/ > > > Testing > ------- > > Unit test, test on cluster > > > Thanks, > > Jimmy Xiang > >
