Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r14325602
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -87,9 +86,7 @@ private class MemoryStore(blockManager: BlockManager,
maxMemory: Long)
values: Iterator[Any],
level: StorageLevel,
returnValues: Boolean): PutResult = {
- val valueEntries = new ArrayBuffer[Any]()
- valueEntries ++= values
- putValues(blockId, valueEntries, level, returnValues)
+ putValues(blockId, values.toArray, level, returnValues)
--- End diff --
Yes, otherwise it would be a recursive call to itself?
Either way we should probably make this go through the safe unrolling code
path, because this might actually cause an OOM. Looks like we do this all over
the place.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
