Bowen, What Andy is doing in the notebook is a slightly different thing. He’s using sbt to bring all spark jars (core, mllib, repl, what have you). You could use maven for that. He then creates a repl and submits all the spark code into it. Pretty sure spark unit tests cover similar uses cases. Maybe not mllib per se but this kind of submission.
Kind regards, Radek Gruchalski ra...@gruchalski.com (mailto:ra...@gruchalski.com) (mailto:ra...@gruchalski.com) de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/) Confidentiality: This communication is intended for the above-named person and may be confidential and/or legally privileged. If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. On Sunday, 22 November 2015 at 01:01, bowen zhang wrote: > Thanks Rad for info. I looked into the repo and see some .snb file using > spark mllib. Can you give me a more specific place to look for when invoking > the mllib functions? What if I just want to invoke some of the ML functions > in my HelloWorld.java? > > From: Rad Gruchalski <ra...@gruchalski.com (mailto:ra...@gruchalski.com)> > To: bowen zhang <bowenzhang...@yahoo.com (mailto:bowenzhang...@yahoo.com)> > Cc: "dev@spark.apache.org (mailto:dev@spark.apache.org)" > <dev@spark.apache.org (mailto:dev@spark.apache.org)> > Sent: Saturday, November 21, 2015 3:43 PM > Subject: Re: Using spark MLlib without installing Spark > > Bowen, > > One project to look at could be spark-notebook: > https://github.com/andypetrella/spark-notebook > It uses Spark you in the way you intend to use it. > > > > Kind regards, > Radek Gruchalski > ra...@gruchalski.com (mailto:ra...@gruchalski.com) > (mailto:ra...@gruchalski.com) > de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/) > > Confidentiality: > This communication is intended for the above-named person and may be > confidential and/or legally privileged. > If it has come to you in error you must take no action based on it, nor must > you copy or show it to anyone; please delete/destroy and inform the sender > immediately. > > > On Sunday, 22 November 2015 at 00:38, bowen zhang wrote: > > Hi folks, > > I am a big fan of Spark's Mllib package. I have a java web app where I want > > to run some ml jobs inside the web app. My question is: is there a way to > > just import spark-core and spark-mllib jars to invoke my ML jobs without > > installing the entire Spark package? All the tutorials related Spark seems > > to indicate installing Spark is a pre-condition for this. > > > > Thanks, > > Bowen > > >
