")
val dfCount = ctx.sql("select count(*) as cnt from training")
println(dfCount.first.getLong(0))
/Sim
Simeon Simeonov, Founder & CTO, Swoop<http://swoop.com/>
@simeons<http://twitter.com/simeons> |
blog.simeonov.com<http://blog.simeonov.com/> | 617.299.6746
I'd be happy to share the test file so you can reproduce this in your own
environment.
/Sim
Simeon Simeonov, Founder & CTO, Swoop<http://swoop.com/>
@simeons<http://twitter.com/simeons> |
blog.simeonov.com<http://blog.simeonov.com/> | 617.299.6746
From: Yin Huai mail
-memory 4g --executor-memory 4g
/Sim
Simeon Simeonov, Founder & CTO, Swoop<http://swoop.com/>
@simeons<http://twitter.com/simeons> |
blog.simeonov.com<http://blog.simeonov.com/> | 617.299.6746
From: Yin Huai mailto:yh...@databricks.com>>
Date: Monday, July 6, 2015 at
Yin, that did the trick.
I'm curious what was the effect of the environment variable, however, as the
behavior of the shell changed from hanging to quitting when the env var value
got to 1g.
/Sim
Simeon Simeonov, Founder & CTO, Swoop<http://swoop.com/>
@simeons<http://t
,
Sim
From: Michael Armbrust mailto:mich...@databricks.com>>
Date: Monday, August 10, 2015 at 2:36 PM
To: Simeon Simeonov mailto:s...@swoop.com>>
Cc: user mailto:user@spark.apache.org>>
Subject: Re: Spark inserting into parquet files with different schema
Older versions of Spark
/OVERWRITE
statement generates the same number of columns as its schema.
Is this behavior expected? Shall I create a JIRA issue if it is not?
From: Michael Armbrust mailto:mich...@databricks.com>>
Date: Monday, August 10, 2015 at 3:44 PM
To: Simeon Simeonov mailto:s...@swoop.com>&
