+1 Tested MLlib on Mac OS X
On Thu, Sep 24, 2015 at 6:14 PM, Reynold Xin <r...@databricks.com> wrote:
> Krishna,
>
> Thanks for testing every release!
>
>
> On Thu, Sep 24, 2015 at 6:08 PM, Krishna Sankar <ksanka...@gmail.com>
> wrote:
>
>> +1 (non-binding, of course)
>>
>> 1. Compiled OSX 10.10 (Yosemite) OK Total time: 26:48 min
>> mvn clean package -Pyarn -Phadoop-2.6 -DskipTests
>> 2. Tested pyspark, mllib (iPython 4.0, FYI, notebook install is separate
>> “conda install python” and then “conda install jupyter”)
>> 2.1. statistics (min,max,mean,Pearson,Spearman) OK
>> 2.2. Linear/Ridge/Laso Regression OK
>> 2.3. Decision Tree, Naive Bayes OK
>> 2.4. KMeans OK
>> Center And Scale OK
>> 2.5. RDD operations OK
>> State of the Union Texts - MapReduce, Filter,sortByKey (word count)
>> 2.6. Recommendation (Movielens medium dataset ~1 M ratings) OK
>> Model evaluation/optimization (rank, numIter, lambda) with
>> itertools OK
>> 3. Scala - MLlib
>> 3.1. statistics (min,max,mean,Pearson,Spearman) OK
>> 3.2. LinearRegressionWithSGD OK
>> 3.3. Decision Tree OK
>> 3.4. KMeans OK
>> 3.5. Recommendation (Movielens medium dataset ~1 M ratings) OK
>> 3.6. saveAsParquetFile OK
>> 3.7. Read and verify the 4.3 save(above) - sqlContext.parquetFile,
>> registerTempTable, sql OK
>> 3.8. result = sqlContext.sql("SELECT
>> OrderDetails.OrderID,ShipCountry,UnitPrice,Qty,Discount FROM Orders INNER
>> JOIN OrderDetails ON Orders.OrderID = OrderDetails.OrderID") OK
>> 4.0. Spark SQL from Python OK
>> 4.1. result = sqlContext.sql("SELECT * from people WHERE State = 'WA'") OK
>> 5.0. Packages
>> 5.1. com.databricks.spark.csv - read/write OK (--packages
>> com.databricks:spark-csv_2.10:1.2.0)
>> 6.0. DataFrames
>> 6.1. cast,dtypes OK
>> 6.2. groupBy,avg,crosstab,corr,isNull,na.drop OK
>> 6.3. All joins,sql,set operations,udf OK
>> *Notes:*
>> 1. Speed improvement in DataFrame functions groupBy, avg,sum et al. *Good
>> work*. I am working on a project to reduce processing time from ~24 hrs
>> to ... Let us see what Spark does. The speedups would help a lot.
>> 2. FYI, UDFs getM and getY work now (Thanks). Slower; saturates the CPU.
>> A non-scientific snapshot below. I know that this really has to be done
>> more rigorously, on a bigger machine, with more cores et al..
>> [image: Inline image 1] [image: Inline image 2]
>>
>> On Thu, Sep 24, 2015 at 12:27 AM, Reynold Xin <r...@databricks.com>
>> wrote:
>>
>>> Please vote on releasing the following candidate as Apache Spark version
>>> 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if
>>> a majority of at least 3 +1 PMC votes are cast.
>>>
>>> [ ] +1 Release this package as Apache Spark 1.5.1
>>> [ ] -1 Do not release this package because ...
>>>
>>>
>>> The release fixes 81 known issues in Spark 1.5.0, listed here:
>>> http://s.apache.org/spark-1.5.1
>>>
>>> The tag to be voted on is v1.5.1-rc1:
>>>
>>> https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
>>>
>>> The release files, including signatures, digests, etc. can be found at:
>>> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>>>
>>> Release artifacts are signed with the following key:
>>> https://people.apache.org/keys/committer/pwendell.asc
>>>
>>> The staging repository for this release (1.5.1) can be found at:
>>> *https://repository.apache.org/content/repositories/orgapachespark-1148/
>>> <https://repository.apache.org/content/repositories/orgapachespark-1148/>*
>>>
>>> The documentation corresponding to this release can be found at:
>>> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>>>
>>>
>>> =======================================
>>> How can I help test this release?
>>> =======================================
>>> If you are a Spark user, you can help us test this release by taking an
>>> existing Spark workload and running on this release candidate, then
>>> reporting any regressions.
>>>
>>> ================================================
>>> What justifies a -1 vote for this release?
>>> ================================================
>>> -1 vote should occur for regressions from Spark 1.5.0. Bugs already
>>> present in 1.5.0 will not block this release.
>>>
>>> ===============================================================
>>> What should happen to JIRA tickets still targeting 1.5.1?
>>> ===============================================================
>>> Please target 1.5.2 or 1.6.0.
>>>
>>>
>>>
>>>
>>
>
