> On Nov. 22, 2016, 9:30 p.m., Illya Yalovyy wrote: > > ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java, line 2789 > > <https://reviews.apache.org/r/53966/diff/2/?file=1568277#file1568277line2789> > > > > Scalability concern: > > > > On some real datasets, it could be millions of elements in that list. > > If it happens in HS2 with many cocurrent connection this jvm can easily go > > down with OOM Exceptions. I would suggest reconsider that approach.
You're right, I did not see that case. Probably it would better to stick with the if (!exists && !rename) condition. This would be slow when doing many repeated INSERT INTO, but it will not have problems with memory. - Sergio ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53966/#review156629 ----------------------------------------------------------- On Nov. 21, 2016, 11:54 p.m., Sergio Pena wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/53966/ > ----------------------------------------------------------- > > (Updated Nov. 21, 2016, 11:54 p.m.) > > > Review request for hive. > > > Bugs: HIVE-15199 > https://issues.apache.org/jira/browse/HIVE-15199 > > > Repository: hive-git > > > Description > ------- > > The patch helps execute repeated INSERT INTO statements on S3 tables when the > scratch directory is on S3. > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/common/FileUtils.java > 1d8c04160c35e48781b20f8e6e14760c19df9ca5 > itests/hive-blobstore/src/test/queries/clientpositive/insert_into.q > 919ff7d9c7cb40062d68b876d6acbc8efb8a8cf1 > itests/hive-blobstore/src/test/results/clientpositive/insert_into.q.out > c25d0c4eec6983b6869e2eba711b39ba91a4c6e0 > ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java > 61b8bd0ac40cffcd6dca0fc874940066bc0aeffe > > Diff: https://reviews.apache.org/r/53966/diff/ > > > Testing > ------- > > > Thanks, > > Sergio Pena > >
