[
https://issues.apache.org/jira/browse/SPARK-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-19077.
-------------------------------
Resolution: Invalid
Questions go to [email protected]
http://spark.apache.org/contributing.html
> no way to prevent spark streaming's eventLog inprogress file becoming too big?
> ------------------------------------------------------------------------------
>
> Key: SPARK-19077
> URL: https://issues.apache.org/jira/browse/SPARK-19077
> Project: Spark
> Issue Type: Question
> Affects Versions: 2.0.0
> Reporter: Zhong Chunlai
>
> My spark streaming job generates a large eventLog inprogess file after
> some days, which is more than 200G. And it is becoming even bigger and bigger
> as spark streaming continues running. I think it is not possible for
> history server is to load this eventLog file as it ends. is there any way to
> make it smaller, or make it a rolling log?
> Also I read spark code, I find eventLog is not log4j log, it can't set log
> level!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
