Hi Hari
Meanwhile Iam trying out with different port. I need to confirm with you about
the installation for Spark and Flume.
For installation, I have just unzipped spark-1.1.0-bin-hadoop1.tar.gz and
apache-flume-1.4.0-bin.tar.gz for running spark streaming examples.
Is this the correct way or
Did you start a Flume agent to push data to the relevant port?
Thanks,
Hari
On Fri, Nov 7, 2014 at 2:05 PM, Jeniba Johnson
wrote:
> Hi,
> I have installed spark-1.1.0 and apache flume 1.4 for running streaming
> example FlumeEventCount. Previously the code was working fine. Now Iam facing
First, can you try a different port?
TIME_WAIT is basically a timeout for a socket to be completely decommissioned
for the port to be available for binding. Once you wait for a few minutes and
if you still see a startup issue, can you also send the error logs? From what I
can see, the port s
Hi Hari
Just to give you a background , I had installed spark-1.1.0 and apache flume
1.4 with basic configurations as needed. I just wanted to know that
Is this the correct way for running Spark streaming examples with Flume.
So As you had mentioned about the TIME_WAIT parameter, did not get e
Many methods are not required serialization EdgeRDD or VertexRDD(eg:
graph.edges.count), moreover , partitionsRDD(or targetStorageLevel) need
only in the driver. partitionsRDD (or targetStorageLevel) is not serialized no
effect.
-- Original --
From: "X
The socket may have been in TIME_WAIT. Can you try after a bit? The error
message definitely suggests that some other app is listening on that port.
Thanks,
Hari
On Mon, Nov 10, 2014 at 9:30 PM, Jeniba Johnson
wrote:
> Hi Hari
> Thanks for your kind reply
> Even after killing the process id
Hi Hari
Thanks for your kind reply
Even after killing the process id of the specific port. Still Iam facing with
the similar error.
The command I use is
sudo lsof -i -P | grep -i "listen"
Kill -9 PID
However If I try to work with the port which is available, still the error
remains the same
Nice, we currently encounter a stackoverflow error caused by this bug.
We also found that "val partitionsRDD: RDD[(PartitionID, EdgePartition[ED,
VD])],
val targetStorageLevel: StorageLevel = StorageLevel.MEMORY_ONLY)" will not
be serialized even without adding @transient.
However, transient can
Hi, all. I want to seek suggestions on how to do checkpoint more
efficiently, especially for iterative applications written by GraphX.
For iterative applications, the lineage of a job can be very long, which is
easy to cause statckoverflow error. A solution is to do checkpoint.
However, checkpoin
I have been trying to fix this bug.
The related PR:
https://github.com/apache/spark/pull/2631
-- Original --
From: "Xu Lijie";;
Date: Tue, Nov 11, 2014 10:19 AM
To: "user"; "dev";
Subject: Checkpoint bugs in GraphX
Hi, all. I'm not sure whether someone
Hey Sadhan,
I really don't think this is Spark log... Unlike Shark, Spark SQL
doesn't even provide a Hive mode to let you execute queries against
Hive. Would you please check whether there is an existing HiveServer2
running there? Spark SQL HiveThriftServer2 is just a Spark port of
HiveServer
Hi, all. I'm not sure whether someone has reported this bug:
There should be a checkpoint() method in EdgeRDD and VertexRDD as follows:
override def checkpoint(): Unit = { partitionsRDD.checkpoint() }
Current EdgeRDD and VertexRDD use *RDD.checkpoint()*, which only checkpoint
the edges/vertice
The sql run successfully? and what sql you running?
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/thrift-jdbc-server-probably-running-queries-as-hive-query-tp9267p9268.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com
I was testing out the spark thrift jdbc server by running a simple query in
the beeline client. The spark itself is running on a yarn cluster.
However, when I run a query in beeline -> I see no running jobs in the
spark UI(completely empty) and the yarn UI seem to indicate that the
submitted query
I tested 2 different implementations to generate the predicted ranked
list...The first version uses a cartesian of user and product features and
then generates a predicted value for each (user,product) key...
The second version does a collect on the skinny matrix (most likely
products) and then br
I reverted the patch locally, seems to be working for me.
On Mon, Nov 10, 2014 at 6:00 PM, Patrick Wendell wrote:
> I reverted that patch to see if it fixes it.
>
> On Mon, Nov 10, 2014 at 1:45 PM, Josh Rosen wrote:
> > It looks like the Jenkins maven builds are broken, too. Based on the
> > J
Looks like that port is not available because another app is using that port.
Can you take a look at netstat -a and use a port that is free?
Thanks,
Hari
On Fri, Nov 7, 2014 at 2:05 PM, Jeniba Johnson
wrote:
> Hi,
> I have installed spark-1.1.0 and apache flume 1.4 for running streaming
>
I reverted that patch to see if it fixes it.
On Mon, Nov 10, 2014 at 1:45 PM, Josh Rosen wrote:
> It looks like the Jenkins maven builds are broken, too. Based on the
> Jenkins logs, I think that this pull request may have broken things
> (although I'm not sure why):
>
> https://github.com/apach
Hey Andrew, your JIRA search link seems wrong, it's probably supposed to be
this:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%201.1.1%20ORDER%20BY%20priority%20DESC
Matei
> On Nov 10, 2014, at 2:18 PM, Andrew Or wro
(Tonight at midnight being in PST 12am on 11/11)
2014-11-10 14:17 GMT-08:00 Andrew Or :
> Hi everyone,
>
> I am the release manager for 1.1.1, and I am preparing to cut a release
> tonight at midnight. 1.1.1 is a maintenance release which will ship several
> important bug fixes to users of Spark
Hi everyone,
I am the release manager for 1.1.1, and I am preparing to cut a release
tonight at midnight. 1.1.1 is a maintenance release which will ship several
important bug fixes to users of Spark 1.1. Many users are waiting for
these fixes so I would like to release it as soon as possible.
At
I'm wondering why
https://issues.apache.org/jira/browse/SPARK-3638
only updated the version of http client for the kinesis-asl profile and
left the base dependencies unchanged.
Spark built without that profile still has the same
java.lang.NoSuchMethodError:
org.apache.http.impl.conn.DefaultClie
It looks like the Jenkins maven builds are broken, too. Based on the
Jenkins logs, I think that this pull request may have broken things
(although I'm not sure why):
https://github.com/apache/spark/pull/3030#issuecomment-62436181
On Mon, Nov 10, 2014 at 1:42 PM, Sadhan Sood wrote:
> Getting an
Getting an exception while trying to build spark in spark-core:
[ERROR]
while compiling:
/Users/dev/tellapart_spark/core/src/main/scala/org/apache/spark/ui/JettyUtils.scala
during phase: typer
library version: version 2.10.4
compiler version: version 2.10.4
reconstruct
Too bad Nick, I dont have anything immediately ready that tests Spark
Streaming with those extreme settings. :)
On Mon, Nov 10, 2014 at 9:56 AM, Nicholas Chammas
wrote:
> On Sun, Nov 9, 2014 at 1:51 AM, Tathagata Das
> wrote:
>>
>> This causes a scalability vs. latency tradeoff - if your limit i
On Sun, Nov 9, 2014 at 1:51 AM, Tathagata Das
wrote:
> This causes a scalability vs. latency tradeoff - if your limit is 1000
> tasks per second (simplifying from 1500), you could either configure
> it to use 100 receivers at 100 ms batches (10 blocks/sec), or 1000
> receivers at 1 second batches
Hi,
I have installed spark-1.1.0 and apache flume 1.4 for running streaming
example FlumeEventCount. Previously the code was working fine. Now Iam facing
with the below mentioned issues. My flume is running properly it is able to
write the file.
The command I use is
bin/run-example org.apac
27 matches
Mail list logo
