This is what my Driver launch command looks like, it only contains 1 master
in -Dspark.master property whereas from Launcher I am passing 2 with port
6066.
Launch Command: "/path/to/java" "-cp" "" "-Xmx1024M"
"-Dspark.executor.extraJavaOptions=-Dlog4j.configuration=file:log4j-server.properties"
"-
Can anyone please help.
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
Googling doesn't find similar thread except
https://forums.databricks.com/questions/2095/sparkstreaming-to-process-http-rest-end-point-serv.html,
and it seems people doing this through custom receiver as well
(https://old.reddit.com/r/apachespark/comments/6ihdor/best_http_client_for_spark_to_rea
Can someone please help me out here, maybe point to some documentation for
the same? I couldn't find almost anything.
Thanks & Regards
Biplob Biswas
On Thu, Jun 28, 2018 at 11:13 AM Biplob Biswas
wrote:
> Hi,
>
> I had a few questions regarding the way *newApiHadoopRDD *accesses data
> from HB
Hi there,
I'm using *GBTClassifier* do some classification jobs and find the
performance of scoring stage is not quite satisfying. The trained model has
about 160 trees and the input feature vector is sparse and its size is
about 20+.
After some digging, I found the model will repeatedly and rand
I am currently working on a project in which we are dealing with TBs of CSV
files generating yearly. The main source of data is interval meter data, and
the others are Customer information, tariff and sites information. We might be
loading the data since 2010. One CVS(10 GBs min) file for each
I'm reading the source code of spark MLlib Kmean,
I find a clip of code
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
Hi, there
I think you should set "spark.streaming.backpressure.pid.minRate" as "no
set" like "spark.streaming.backpressure.initialRate".
The default value 100 is not good for my business.
It's better to explain it more detail in document, and let user make
decision by himself like "spark.stream
Hi,
I'm trying to debug an issue with Spark so I've set log level to DEBUG but
at the same time I'd like to avoid the httpclient.wire's verbose output by
setting it to WARN.
I tried the following log4.properties config but I'm still getting DEBUG
outputs for httpclient.wire:
log4j.rootCategory=DE
