kafka + mysql filtering problem

spc.broadcast(x.split(",")(5).filterNot(toRemove)) } } var msg = rule_apply(1, mto, rules) var word = lines.map(x => msg) word.print() ssc.start() ssc.awaitTermination() } } The problem is that mto variable always returns to “0” value after mapping lines DStream. I

TF-IDF Question

0=>spark]²)?. Regards and thanks in advance. Franco Barrientos Data Scientist Málaga #115, Of. 1003, Las Condes. Santiago, Chile. (+562)-29699649 (+569)-76347893 franco.barrien...@exalitica.com <mailto:franco.barrien...@exalitica.com> www.exalitica.com <http://www.exalitica.com/>

null Error in ALS model predict

ply a ratings.first() I get the follow error: Why this happend? I need to use this second way. Thanks in advance, Franco Barrientos Data Scientist Málaga #115, Of. 1003, Las Condes. Santiago, Chile. (+562)-29699649 (+569)-76347893 <mailto:franco.barrien...@exalitica.com> franc

RE: Effects problems in logistic regression

Thanks again DB Tsai, LogisticRegressionWithLBFGS works for me! De: Franco Barrientos [mailto:franco.barrien...@exalitica.com] Enviado el: jueves, 18 de diciembre de 2014 16:42 Para: 'DB Tsai' CC: 'Sean Owen'; user@spark.apache.org Asunto: RE: Effects problems in

RE: Effects problems in logistic regression

Thanks I will try. De: DB Tsai [mailto:dbt...@dbtsai.com] Enviado el: jueves, 18 de diciembre de 2014 16:24 Para: Franco Barrientos CC: Sean Owen; user@spark.apache.org Asunto: Re: Effects problems in logistic regression Can you try LogisticRegressionWithLBFGS? I verified that this will

RE: Effects problems in logistic regression

Yes, without the “amounts” variables the results are similiar. When I put other variables its fine. De: Sean Owen [mailto:so...@cloudera.com] Enviado el: jueves, 18 de diciembre de 2014 14:22 Para: Franco Barrientos CC: user@spark.apache.org Asunto: Re: Effects problems in logistic

Effects problems in logistic regression

calculates exp(-1*(-0.4021+(-207.1749)*amount)) this is a big number, in fact infinity for spark. How can I treat this variable? or why this happened? Thanks , Franco Barrientos Data Scientist Málaga #115, Of. 1003, Las Condes. Santiago, Chile. (+562)-29699649 (+569)-76347893

Percentile

Hi folks!, Anyone known how can I calculate for each elements of a variable in a RDD its percentile? I tried to calculate trough Spark SQL with subqueries but I think that is imposible in Spark SQL. Any idea will be welcome. Thanks in advance, Franco Barrientos Data Scientist Málaga

join 2 tables

t an error: Franco Barrientos Data Scientist Málaga #115, Of. 1003, Las Condes. Santiago, Chile. (+562)-29699649 (+569)-76347893 <mailto:franco.barrien...@exalitica.com> franco.barrien...@exalitica.com <http://www.exalitica.com/> www.exalitica.com <http://exalitica.com/web/img/frim.png>

S3 table to spark sql

isterTempTable("trx_u3m") Now my problema i show can i transform string variable into date variables (fechau3m)? Franco Barrientos Data Scientist Málaga #115, Of. 1003, Las Condes. Santiago, Chile. (+562)-29699649 (+569)-76347893 <mailto:franco.barrien...@exalitica.com> f