3000 filters don’t look like something reasonable. This is very difficult to
test and verify as well as impossible to maintain.
Could it be that your filters are another table that you should join with ?
The example is a little bit artificial to understand the underlying business
case. Can you pr
>From the information you provided I would tackle this as a batch problem,
because this way you have access to more sophisticated techniques and you
have more flexibility (maybe HDFS and a SparkJob, but also think about a
datastore offering good indexes for the kind of data types and values you
hav
Hi,
I am not familiar with ATNConfigSet, but some thoughts that might help.
How many distinct key1 (resp. key2) values do you have? Are these values
reasonably stable over time?
Are these records ingested in real-time or are they loaded from a datastore?
If the latter case the DB might be able t
Hi,
I need to run some queries on huge amount input records. Input rate for records
are 100K/seconds.
A record is like (key1,key2,value) and the application should report occurances
of kye1 = something && key2 == somethingElse.
The problem is there are too many filters in my query: more tha
ot;)
for (i <- 0 until numClusters) {
println(s"\nCLUSTER $i:")
some_tweets.foreach { t =>
if (model.predict(Utils.featurize(t)) == i) {
println(t)
}
}
}
}
}
From: lovelylavs [via Apache Spark User List]
[mailto:ml-node+s1001560n21956...@n3.nabble.com]
Sent: Sunday, March 08, 2015 2:34 AM
To: J
- BAS)
Subject: Re: Spark SQL Stackoverflow error
Hi,
how was this issue fixed?
If you reply to this email, your message will be added to the discussion below:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Stackoverflow-error-tp12086p21862.html
To
ln(s"\nCLUSTER $i:")
some_tweets.foreach { t =>
if (model.predict(Utils.featurize(t)) == i) {
println(t)
}
}
}
}
}
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Stackoverflow-error-tp12086p20605.html
Sent from the Apache Spark User
I couldn’t reproduce the exception, probably it’s solved in the latest code.
From: Vishal Vibhandik [mailto:vishal.vibhan...@gmail.com]
Sent: Thursday, August 14, 2014 11:17 AM
To: user@spark.apache.org
Subject: Spark SQL Stackoverflow error
Hi,
I tried running the sample sql code JavaSparkSQL
Hi,
I tried running the sample sql code JavaSparkSQL but keep getting this
error:-
the error comes on line
JavaSchemaRDD teenagers = sqlCtx.sql("SELECT name FROM people WHERE age >=
13 AND age <= 19");
C:\>spark-submit --class org.apache.spark.examples.sql.JavaSparkSQL
--master local
\spark-1.0.2
