Thanks for your response Jonathan. Yes, this works. I also added another
way of achieving this to the Stackoverflow post. Thanks for the help.
On Tue, Feb 28, 2017 at 11:58 PM, Jonathan Kelly
wrote:
> Prithish,
>
> I saw you posted this on SO, so I responded there just now. See
> http://stackove
Prithish,
I saw you posted this on SO, so I responded there just now. See
http://stackoverflow.com/questions/42452622/custom-log4j-properties-on-aws-emr/42516161#42516161
In short, an hdfs:// path can't be used to configure log4j because log4j
knows nothing about hdfs. Instead, since you are usin
Steve, I tried that, but didn't work. Any other ideas?
On Mon, Feb 27, 2017 at 1:42 AM, Steve Loughran
wrote:
> try giving a resource of a file in the JAR, e.g add a file
> "log4j-debugging.properties into the jar, and give a config option of
> -Dlog4j.configuration=/log4j-debugging.properties
try giving a resource of a file in the JAR, e.g add a file
"log4j-debugging.properties into the jar, and give a config option of
-Dlog4j.configuration=/log4j-debugging.properties (maybe also try without the
"/")
On 26 Feb 2017, at 16:31, Prithish
mailto:prith...@gmail.com>> wrote:
Hoping s
Hoping someone can answer this.
I am unable to override and use a Custom log4j.properties on Amazon EMR. I
am running Spark on EMR (Yarn) and have tried all the below combinations in
the Spark-Submit to try and use the custom log4j.
In Client mode
--driver-java-options
"-Dlog4j.configuration=hdfs
