ot;, make sure I "close"
>> >> it else I had a ton of "too many files open" errors :)
>> >> - Use immutable objects as far as possible. If I use mutable objects
>> >> within a method/class then I turn them into immutable before passing
&g
ot;
Sent: Wednesday, September 17, 2014 4:10:12 PM
Subject: Re: Stable spark streaming app
Thanks :)
On Wed, Sep 17, 2014 at 2:10 PM, Paul Wais wrote:
> Thanks Tim, this is super helpful!
>
> Question about jars and spark-submit: why do you provide
> myawesomeapp.jar as the program jar
can make "logInfo"
>> calls from within other Objects/Methods/Classes and output goes to the
>> "stderr" file in the Yarn container logs. Good for debugging stream
>> processing logic.
>>
>> Currently, my processing delay is lower than my dStre
Nice write-up... very helpful!
-Original Message-
From: Tim Smith [mailto:secs...@gmail.com]
Sent: Wednesday, September 17, 2014 1:11 PM
Cc: spark users
Subject: Re: Stable spark streaming app
I don't have anything in production yet but I now at least have a stable
(running for
spark/log4j.properties' for logging
. Currently I cannot process > 1G file with this configuration. I tried various
things but could not succeed yet.
- Original Message -
From: "Tim Smith"
Cc: "spark users"
Sent: Wednesday, September 17, 2014 1:11:12 PM
Subject: Re:
es.apache.org/jira/browse/SPARK-2316
>
> Best I understand and have been told, this does not affect data
> integrity but may cause un-necessary recomputes.
>
> Hope this helps,
>
> Tim
>
>
> On Wed, Sep 17, 2014 at 8:30 AM, Soumitra Kumar
> wrote:
>> Hmm, no r
duct based on Spark Streaming.
>
> I am exploring Spark Streaming for enterprise software and am cautiously
> optimistic about it. I see huge potential to improve debuggability of Spark.
>
> - Original Message -
> From: "Tim Smith"
> To: "spark users&qu
.
- Original Message -
From: "Tim Smith"
To: "spark users"
Sent: Friday, September 12, 2014 10:09:53 AM
Subject: Stable spark streaming app
Hi,
Anyone have a stable streaming app running in "production"? Can you
share some overview of the app and setup like numb
Hi,
Anyone have a stable streaming app running in "production"? Can you
share some overview of the app and setup like number of nodes, events
per second, broad stream processing workflow, config highlights etc?
Thanks,
Tim
-
To
