[
https://issues.apache.org/jira/browse/CASSANDRA-3945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246763#comment-13246763
]
Jonathan Ellis commented on CASSANDRA-3945:
-------------------------------------------
3859 has been committed.
> Support incremental/batch sizes for BulkRecordWriter, due to GC overhead
> issues
> -------------------------------------------------------------------------------
>
> Key: CASSANDRA-3945
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3945
> Project: Cassandra
> Issue Type: Bug
> Reporter: Chris Goffinet
> Assignee: Chris Goffinet
> Priority: Minor
> Fix For: 1.1.1
>
>
> When loading large amounts of data, currently the BulkRecordWriter will write
> out all the sstables, then stream them. This actually caused us GC overhead
> issues, due to our heap sizes for reducers. We ran into a problem where the
> number of SSTables on disk that had to be open would cause the jvm process to
> die. We also wanted a way to incrementally stream them as we created them. I
> created support for setting this, the default behavior is wait for them to be
> created. But if you increase to >= 1, you can determine the batch size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
