Github user mccheah commented on the pull request:
https://github.com/apache/spark/pull/4972#issuecomment-78420895
I wrote it in scala, extending AnyVal and writing a sealed trait that
extends Any to provide the extension point. However, I believe that using the
trait incurs the allocation overhead, which is what we were trying to avoid in
moving to Scala. From the scala docs
"A value class can only extend universal traits and cannot be extended
itself. A universal trait is a trait that extends Any, only has defs as
members, and does no initialization. Universal traits allow basic inheritance
of methods for value classes, but they incur the overhead of allocation."
http://docs.scala-lang.org/overviews/core/value-classes.html
That being said, I also didn't see any significant performance degradation
in moving to Scala. My most recent numbers:
Estimating the size of a 1000 x 1000 2D String array 50,000 times: 153.329
s, 153.1 s, 150.67 s
Estimating the size of a 1000 1D String array 1,000,000 times: 19.951 s,
20.75 s, 20.551 s
So we didn't lose any performance from moving from Java to Scala (compare
with numbers I've reported above in this discussion), but we gained a ton of
readability. It's odd to be using Java in this Scala-laden codebase anyways, so
it seems good to keep our things bound to Scala.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
