Hi Stephen,
I'm not sure which link you are referring to for the example code -- but
yes, the recommendation is that you create the enum in Java, eg. see
https://github.com/apache/spark/blob/v1.4.0/core/src/main/java/org/apache/spark/status/api/v1/StageStatus.java
Then nothing special is require
I am reviving an old thread here. The link for the example code for the
java enum based solution is now dead: would someone please post an updated
link showing the proper interop?
Specifically: it is my understanding that java enum's may not be created
within Scala. So is the proposed solution re
any update here? This is relevant for a currently open PR of mine -- I've
got a bunch of new public constants defined w/ format #4, but I'd gladly
switch to java enums. (Even if we are just going to postpone this
decision, I'm still inclined to switch to java enums ...)
just to be clear about th
well, perhaps I overstated things a little, I wouldn't call it the
"official" solution, just a recommendation in the never-ending debate (and
the recommendation from folks with their hands on scala itself).
Even if we do get this fixed in scaladoc eventually -- as its not in the
current versions,
If scaladoc can show the Java enum types, I do think the best way is then
just Java enum types.
On Mon, Mar 23, 2015 at 2:11 PM, Patrick Wendell wrote:
> If the official solution from the Scala community is to use Java
> enums, then it seems strange they aren't generated in scaldoc? Maybe
> we
If the official solution from the Scala community is to use Java
enums, then it seems strange they aren't generated in scaldoc? Maybe
we can just fix that w/ Typesafe's help and then we can use them.
On Mon, Mar 23, 2015 at 1:46 PM, Sean Owen wrote:
> Yeah the fully realized #4, which gets back t
The only issue I knew of with Java enums was that it does not appear in the
Scala documentation.
On Mon, Mar 23, 2015 at 1:46 PM, Sean Owen wrote:
> Yeah the fully realized #4, which gets back the ability to use it in
> switch statements (? in Scala but not Java?) does end up being kind of
> hug
Yeah the fully realized #4, which gets back the ability to use it in
switch statements (? in Scala but not Java?) does end up being kind of
huge.
I confess I'm swayed a bit back to Java enums, seeing what it
involves. The hashCode() issue can be 'solved' with the hash of the
String representation.
I've just switched some of my code over to the new format, and I just want
to make sure everyone realizes what we are getting into. I went from 10
lines as java enums
https://github.com/squito/spark/blob/fef66058612ebf225e58dd5f5fea6bae1afd5b31/core/src/main/java/org/apache/spark/status/api/Stage
Let me put a quick summary. #4 got majority vote with CamelCase but
not UPPERCASE. The following is a minimal implementation that works
for both Scala and Java. In Python, we use string for enums. This
proposal is only for new public APIs. We are not going to change
existing ones. -Xiangrui
~~~
se
It's unrelated to the proposal, but Enum#ordinal() should be much faster,
assuming it's not serialized to JVMs with different versions of the enum :)
On Mon, Mar 16, 2015 at 12:12 PM, Kevin Markey
wrote:
> In some applications, I have rather heavy use of Java enums which are
> needed for related
Hey Xiangrui,
Do you want to write up a straw man proposal based on this line of discussion?
- Patrick
On Mon, Mar 16, 2015 at 12:12 PM, Kevin Markey wrote:
> In some applications, I have rather heavy use of Java enums which are needed
> for related Java APIs that the application uses. And unf
In some applications, I have rather heavy use of Java enums which are
needed for related Java APIs that the application uses. And
unfortunately, they are also used as keys. As such, using the native
hashcodes makes any function over keys unstable and unpredictable, so we
now use Enum.name() a
In MLlib, we use strings for emu-like types in Python APIs, which is
quite common in Python and easy for py4j. On the JVM side, we
implement `fromString` to convert them back to enums. -Xiangrui
On Wed, Mar 11, 2015 at 12:56 PM, RJ Nowling wrote:
> How do these proposals affect PySpark? I think
How do these proposals affect PySpark? I think compatibility with PySpark
through Py4J should be considered.
On Mon, Mar 9, 2015 at 8:39 PM, Patrick Wendell wrote:
> Does this matter for our own internal types in Spark? I don't think
> any of these types are designed to be used in RDD records,
Does this matter for our own internal types in Spark? I don't think
any of these types are designed to be used in RDD records, for
instance.
On Mon, Mar 9, 2015 at 6:25 PM, Aaron Davidson wrote:
> Perhaps the problem with Java enums that was brought up was actually that
> their hashCode is not st
Perhaps the problem with Java enums that was brought up was actually that
their hashCode is not stable across JVMs, as it depends on the memory
location of the enum itself.
On Mon, Mar 9, 2015 at 6:15 PM, Imran Rashid wrote:
> Can you expand on the serde issues w/ java enum's at all? I haven't
Can you expand on the serde issues w/ java enum's at all? I haven't heard
of any problems specific to enums. The java object serialization rules
seem very clear and it doesn't seem like different jvms should have a
choice on what they do:
http://docs.oracle.com/javase/6/docs/platform/serializati
This has some disadvantage for Java, I think. You can't switch on an
object defined like this, but you can with an enum. And although the
scala compiler understands that the set of values is fixed because of
'sealed' and so can warn about missing cases, the JVM won't know this,
and can't do the sam
I have a strong dislike for java enum's due to the fact that they
are not stable across JVM's - if it undergoes serde, you end up with
unpredictable results at times [1].
One of the reasons why we prevent enum's from being key : though it is
highly possible users might depend on it internally and
For #4, my previous proposal may confuse the IDEs with additional
types generated by the case objects, and their toString contain the
underscore. The following works better:
sealed abstract class StorageLevel
object StorageLevel {
final val MemoryOnly: StorageLevel = {
case object MemoryOnl
I have a very strong dislike for #1 (scala enumerations). I'm ok with #4
(with Xiangrui's final suggestion, especially making it sealed & available
in Java), but I really think #2, java enums, are the best option.
Java enums actually have some very real advantages over the other
approaches -- yo
Yes - only new or internal API's. I doubt we'd break any exposed APIs for
the purpose of clean up.
Patrick
On Mar 5, 2015 12:16 AM, "Mridul Muralidharan" wrote:
> While I dont have any strong opinions about how we handle enum's
> either way in spark, I assume the discussion is targetted at (new)
While I dont have any strong opinions about how we handle enum's
either way in spark, I assume the discussion is targetted at (new) api
being designed in spark.
Rewiring what we already have exposed will lead to incompatible api
change (StorageLevel for example, is in 1.0).
Regards,
Mridul
On Wed
That's kinda annoying, but it's just a little extra boilerplate. Can you
call it as StorageLevel.DiskOnly() from Java? Would it also work if they
were case classes with empty constructors, without the field?
On Wed, Mar 4, 2015 at 11:35 PM, Xiangrui Meng wrote:
> `case object` inside an `object`
`case object` inside an `object` doesn't show up in Java. This is the
minimal code I found to make everything show up correctly in both
Scala and Java:
sealed abstract class StorageLevel // cannot be a trait
object StorageLevel {
private[this] case object _MemoryOnly extends StorageLevel
fina
I like #4 as well and agree with Aaron's suggestion.
- Patrick
On Wed, Mar 4, 2015 at 6:07 PM, Aaron Davidson wrote:
> I'm cool with #4 as well, but make sure we dictate that the values should
> be defined within an object with the same name as the enumeration (like we
> do for StorageLevel). Ot
I'm cool with #4 as well, but make sure we dictate that the values should
be defined within an object with the same name as the enumeration (like we
do for StorageLevel). Otherwise we may pollute a higher namespace.
e.g. we SHOULD do:
trait StorageLevel
object StorageLevel {
case object MemoryO
#4 with a preference for CamelCaseEnums
On Wed, Mar 4, 2015 at 5:29 PM, Joseph Bradley
wrote:
> another vote for #4
> People are already used to adding "()" in Java.
>
>
> On Wed, Mar 4, 2015 at 5:14 PM, Stephen Boesch wrote:
>
> > #4 but with MemoryOnly (more scala-like)
> >
> > http://docs.sc
another vote for #4
People are already used to adding "()" in Java.
On Wed, Mar 4, 2015 at 5:14 PM, Stephen Boesch wrote:
> #4 but with MemoryOnly (more scala-like)
>
> http://docs.scala-lang.org/style/naming-conventions.html
>
> Constants, Values, Variable and Methods
>
> Constant names should
#4 but with MemoryOnly (more scala-like)
http://docs.scala-lang.org/style/naming-conventions.html
Constants, Values, Variable and Methods
Constant names should be in upper camel case. That is, if the member is
final, immutable and it belongs to a package object or an object, it may be
considered
31 matches
Mail list logo
