Re: enum-like types in Spark

2015-07-02 Thread Imran Rashid
Hi Stephen, I'm not sure which link you are referring to for the example code -- but yes, the recommendation is that you create the enum in Java, eg. see https://github.com/apache/spark/blob/v1.4.0/core/src/main/java/org/apache/spark/status/api/v1/StageStatus.java Then nothing special is require

Re: enum-like types in Spark

2015-07-01 Thread Stephen Boesch
I am reviving an old thread here. The link for the example code for the java enum based solution is now dead: would someone please post an updated link showing the proper interop? Specifically: it is my understanding that java enum's may not be created within Scala. So is the proposed solution re

Re: enum-like types in Spark

2015-04-09 Thread Imran Rashid
any update here? This is relevant for a currently open PR of mine -- I've got a bunch of new public constants defined w/ format #4, but I'd gladly switch to java enums. (Even if we are just going to postpone this decision, I'm still inclined to switch to java enums ...) just to be clear about th

Re: enum-like types in Spark

2015-03-23 Thread Imran Rashid
well, perhaps I overstated things a little, I wouldn't call it the "official" solution, just a recommendation in the never-ending debate (and the recommendation from folks with their hands on scala itself). Even if we do get this fixed in scaladoc eventually -- as its not in the current versions,

Re: enum-like types in Spark

2015-03-23 Thread Reynold Xin
If scaladoc can show the Java enum types, I do think the best way is then just Java enum types. On Mon, Mar 23, 2015 at 2:11 PM, Patrick Wendell wrote: > If the official solution from the Scala community is to use Java > enums, then it seems strange they aren't generated in scaldoc? Maybe > we

Re: enum-like types in Spark

2015-03-23 Thread Patrick Wendell
If the official solution from the Scala community is to use Java enums, then it seems strange they aren't generated in scaldoc? Maybe we can just fix that w/ Typesafe's help and then we can use them. On Mon, Mar 23, 2015 at 1:46 PM, Sean Owen wrote: > Yeah the fully realized #4, which gets back t

Re: enum-like types in Spark

2015-03-23 Thread Aaron Davidson
The only issue I knew of with Java enums was that it does not appear in the Scala documentation. On Mon, Mar 23, 2015 at 1:46 PM, Sean Owen wrote: > Yeah the fully realized #4, which gets back the ability to use it in > switch statements (? in Scala but not Java?) does end up being kind of > hug

Re: enum-like types in Spark

2015-03-23 Thread Sean Owen
Yeah the fully realized #4, which gets back the ability to use it in switch statements (? in Scala but not Java?) does end up being kind of huge. I confess I'm swayed a bit back to Java enums, seeing what it involves. The hashCode() issue can be 'solved' with the hash of the String representation.

Re: enum-like types in Spark

2015-03-23 Thread Imran Rashid
>>> > >>>>>> be defined within an object with the same name as the enumeration > (like > >>>>>>> > >>>>>> we > >>>>> > >>>>>> do for StorageLevel). Otherwise we may pollute a

Re: enum-like types in Spark

2015-03-17 Thread Xiangrui Meng
gt;>>>>>> e.g. we SHOULD do: >>>>>>> >>>>>>> trait StorageLevel >>>>>>> object StorageLevel { >>>>>>>case object MemoryOnly extends StorageLevel >>>>>>>case object Di

Re: enum-like types in Spark

2015-03-16 Thread Aaron Davidson
atabricks.com> >>>> >>>>> wrote: >>>>>> >>>>>> #4 with a preference for CamelCaseEnums >>>>>>> >>>>>>> On Wed, Mar 4, 2015 at 5:29 PM, Joseph Bradley < >>>>>>> jos..

Re: enum-like types in Spark

2015-03-16 Thread Patrick Wendell
t;>>>case object MemoryOnly extends StorageLevel >>>>>>case object DiskOnly extends StorageLevel >>>>>> } >>>>>> >>>>>> On Wed, Mar 4, 2015 at 5:37 PM, Michael Armbrust < >>>> >>>> mich...@databricks.com&g

Re: enum-like types in Spark

2015-03-16 Thread Kevin Markey
t may be considered a constant (similar to Java'sstatic final members): 1. object Container { 2. val MyConstant = ... 3. } 2015-03-04 17:11 GMT-08:00 Xiangrui Meng : Hi all, There are many places where we use enum-like types in Spark, but in different ways. Every approac

Re: enum-like types in Spark

2015-03-16 Thread Xiangrui Meng
gt; > >> > >> >> > >> > - Patrick >> >> > >> > >> >> > >> > On Wed, Mar 4, 2015 at 6:07 PM, Aaron Davidson < >> ilike...@gmail.com> >> >> > >> wrote: >> >> > >> >> I'm cool with #4 as well,

Re: enum-like types in Spark

2015-03-11 Thread RJ Nowling
do for StorageLevel). Otherwise we may pollute a higher > namespace. > >> > >> >> > >> > >> >> e.g. we SHOULD do: > >> > >> >> > >> > >> >> trait StorageLevel > >> > >> >

Re: enum-like types in Spark

2015-03-09 Thread Patrick Wendell
gt; >> e.g. we SHOULD do: >> > >> >> >> > >> >> trait StorageLevel >> > >> >> object StorageLevel { >> > >> >> case object MemoryOnly extends StorageLevel >> > >> >> case object DiskOnly

Re: enum-like types in Spark

2015-03-09 Thread Aaron Davidson
el Armbrust < > > >> mich...@databricks.com> > > >> >> wrote: > > >> >> > > >> >>> #4 with a preference for CamelCaseEnums > > >> >>> > > >> >>> On Wed, Mar 4, 2015 at 5:29 PM, Joseph Bradley &

Re: enum-like types in Spark

2015-03-09 Thread Imran Rashid
> jos...@databricks.com> > >> >>> wrote: > >> >>> > >> >>> > another vote for #4 > >> >>> > People are already used to adding "()" in Java. > >> >>> > > >> >>> > > >> &

Re: enum-like types in Spark

2015-03-06 Thread Sean Owen
>>> >>> >>> > another vote for #4 >>> >>> > People are already used to adding "()" in Java. >>> >>> > >>> >>> > >>> >>> > On Wed, Mar 4, 2015 at 5:14 PM, Stephen Boesch >>

Re: enum-like types in Spark

2015-03-05 Thread Mridul Muralidharan
>> >>> > >> >>> > >> >>> > On Wed, Mar 4, 2015 at 5:14 PM, Stephen Boesch >> >>> wrote: >> >>> > >> >>> > > #4 but with MemoryOnly (more scala-like) >> >>> > > >&

Re: enum-like types in Spark

2015-03-05 Thread Xiangrui Meng
; >> >>> > On Wed, Mar 4, 2015 at 5:14 PM, Stephen Boesch >> >>> wrote: >> >>> > >> >>> > > #4 but with MemoryOnly (more scala-like) >> >>> > > >> >>> > > http://docs.scala-lang.org/style/nami

Re: enum-like types in Spark

2015-03-05 Thread Imran Rashid
;>> > > http://docs.scala-lang.org/style/naming-conventions.html > >>> > > > >>> > > Constants, Values, Variable and Methods > >>> > > > >>> > > Constant names should be in upper camel case. That is, if the > member is > >

Re: enum-like types in Spark

2015-03-05 Thread Patrick Wendell
t; > >> >>> wrote: > >> >>> > > >> >>> > > #4 but with MemoryOnly (more scala-like) > >> >>> > > > >> >>> > > http://docs.scala-lang.org/style/naming-conventions.html > >> &g

Re: enum-like types in Spark

2015-03-05 Thread Mridul Muralidharan
ed, Mar 4, 2015 at 5:14 PM, Stephen Boesch >> >>> wrote: >> >>> > >> >>> > > #4 but with MemoryOnly (more scala-like) >> >>> > > >> >>> > > http://docs.scala-lang.org/style/naming-conventio

Re: enum-like types in Spark

2015-03-04 Thread Aaron Davidson
t;> > > > >>> > > Constants, Values, Variable and Methods > >>> > > > >>> > > Constant names should be in upper camel case. That is, if the > member is > >>> > > final, immutable and it belongs to a package obje

Re: enum-like types in Spark

2015-03-04 Thread Xiangrui Meng
ming-conventions.html >>> > > >>> > > Constants, Values, Variable and Methods >>> > > >>> > > Constant names should be in upper camel case. That is, if the member is >>> > > final, immutable and it belongs to a package object or an obje

Re: enum-like types in Spark

2015-03-04 Thread Patrick Wendell
t (similar to Java'sstatic final members): >> > > >> > > >> > >1. object Container { >> > >2. val MyConstant = ... >> > > 3. } >> > > >> > > >> > > 2015-03-04 17:11 GMT-0

Re: enum-like types in Spark

2015-03-04 Thread Aaron Davidson
> considered a constant (similar to Java’sstatic final members): > > > > > > > > >1. object Container { > > > 2. val MyConstant = ... > > >3. } > > > > > > > > > 2015-03-04 17:11 GMT-08:00 Xiangrui Meng : > > >

Re: enum-like types in Spark

2015-03-04 Thread Michael Armbrust
ckage object or an object, it may > be > > considered a constant (similar to Java’sstatic final members): > > > > > >1. object Container { > >2. val MyConstant = ... > >3. } > > > > > > 2015-03-04 17:11 GMT-08:00 Xiangrui Meng : >

Re: enum-like types in Spark

2015-03-04 Thread Joseph Bradley
ant = ... >3. } > > > 2015-03-04 17:11 GMT-08:00 Xiangrui Meng : > > > Hi all, > > > > There are many places where we use enum-like types in Spark, but in > > different ways. Every approach has both pros and cons. I wonder > > whether there should b

Re: enum-like types in Spark

2015-03-04 Thread Stephen Boesch
considered a constant (similar to Java’sstatic final members): 1. object Container { 2. val MyConstant = ... 3. } 2015-03-04 17:11 GMT-08:00 Xiangrui Meng : > Hi all, > > There are many places where we use enum-like types in Spark, but in > different ways. Every approach has b

enum-like types in Spark

2015-03-04 Thread Xiangrui Meng
Hi all, There are many places where we use enum-like types in Spark, but in different ways. Every approach has both pros and cons. I wonder whether there should be an “official” approach for enum-like types in Spark. 1. Scala’s Enumeration (e.g., SchedulingMode, WorkerState, etc) * All types