[
https://issues.apache.org/jira/browse/SPARK-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234743#comment-16234743
]
Vinitha Reddy Gankidi commented on SPARK-22412:
-----------------------------------------------
Okay, thanks for letting me know. I also wanted to discuss if we should
consider other bin packing algorithms. According to this
http://www.math.unl.edu/~s-sjessie1/203Handouts/Bin%20Packing.pdf, next fit
decreasing is the least efficient of all but it is easiest to implement and has
O(N) run time.
[~marmbrus] ^
If we decide to continue using next fit decreasing, I'll fix the comment in
SPARK-22411 and resolve this as duplicate.
> Fix incorrect comment in DataSourceScanExec
> -------------------------------------------
>
> Key: SPARK-22412
> URL: https://issues.apache.org/jira/browse/SPARK-22412
> Project: Spark
> Issue Type: Documentation
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Vinitha Reddy Gankidi
>
> Next fit decreasing bin packing algorithm is used to combine splits in
> DataSourceScanExec but the comment incorrectly states that first fit
> decreasing algorithm is used. The current implementation doesn't go back to a
> previously used bin other than the bin that the last element was put into.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
