Re: Inconsistent joinWith behavior?

2016-06-20 Thread Richard Marscher
expr, Outer).map({ case (t, u) => (Option(t), > Option(u)) })". We are trying to use null to create a "(Int, Int)" and > somehow it ended up with a tuple2 having default values. > > Can you create a jira? We will investigate the issue. > > Thanks! > > Yin >

Inconsistent joinWith behavior?

2016-06-20 Thread Richard Marscher
whereas the other case passes the nulls through. I have a small databricks notebook showing the case against 2.0 preview: https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/160347920874755/4268263383756277/673639177603143/latest.html -- *Richard

Re: Encoder Guide / Option[T] Encoder

2016-06-16 Thread Richard Marscher
pect? > > > https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/1023043053387187/1073771007111588/2840265927289860/latest.html > > On Thu, Jun 16, 2016 at 8:37 AM, Richard Marscher < > rmarsc...@localytics.com> wrote: > >> Are there any user or dev g

Encoder Guide / Option[T] Encoder

2016-06-16 Thread Richard Marscher
nd skip passing it into the Option.apply -- *Richard Marscher* Senior Software Engineer Localytics Localytics.com <http://localytics.com/> | Our Blog <http://localytics.com/blog> | Twitter <http://twitter.com/localytics> | Facebook <http://facebook.com/localytics> | Li

Re: Spark.Executor.Cores question

2015-10-27 Thread Richard Marscher
- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > For additional commands, e-mail: dev-h...@spark.apache.org > > -- *Richard Marscher* Software Engineer Localytics Localytics.com <http://localytics.com/> | Our Blog <http://

Re: Spark.Executor.Cores question

2015-10-27 Thread Richard Marscher
rs List mailing list archive at > Nabble.com. > > --------- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > For additional commands, e-mail: dev-h...@spark.apache.org > > -- *Richard Marscher* Software Eng

Trouble creating JIRA issue

2015-10-22 Thread Richard Marscher
ciated! Thanks -- *Richard Marscher* Software Engineer Localytics Localytics.com <http://localytics.com/> | Our Blog <http://localytics.com/blog> | Twitter <http://twitter.com/localytics> | Facebook <http://facebook.com/localytics> | LinkedIn <http://www.linkedin.com/company/1148792?trk=tyah>

Re: Worker memory leaks?

2015-07-20 Thread Richard Marscher
long-lived individual applications, but not tests > for long-lived clusters that run thousands of applications between > restarts. I'm going to create some tickets to add such tests. > > I've filed https://issues.apache.org/jira/browse/SPARK-9202 to follow up > on the finishedExe

Worker memory leaks?

2015-07-20 Thread Richard Marscher
lso have a larger and increasing workload over the past few weeks and possibly code changes to the application description that could be exacerbating these potential underlying issues. We run a lot of smaller applications per day, something in the range of hundreds to maybe 1000 applications per day

Re: If gmail, check sparm

2015-07-20 Thread Richard Marscher
unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > > For additional commands, e-mail: dev-h...@spark.apache.org > > > > ----- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > For additional commands, e-mail: