Re: Spark Metrics: custom source/sink configurations not getting recognized

2016-11-28 Thread Matthew Dailey
;>> >>>> If I initialize my own metrics system, it's picking up my file but then >>>> I'm missing master/executor level metrics/properties (eg. >>>> executor.sink.mySink.propName=myProp - can't read 'propName' from >>>&

Re: Spark Metrics: custom source/sink configurations not getting recognized

2016-09-08 Thread map reduced
>>> I'm missing master/executor level metrics/properties (eg. >>> executor.sink.mySink.propName=myProp - can't read 'propName' from >>> 'mySink') since they are initialized >>> <https://github.com/apache/spark/blob/master/cor

Re: Spark Metrics: custom source/sink configurations not getting recognized

2016-09-07 Thread map reduced
re/src/main/scala/org/apache/spark/deploy/master/Master.scala#L84> >> by >> Spark's metric system. >> >> Is there a (programmatic) way to have 'metrics.properties' shipped before >> executors initialize >> <https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkEnv.scala#L335> >> ? >> >> Here's my SO question >> <https://stackoverflow.com/questions/39340080/spark-metrics-custom-source-sink-configurations-not-getting-recognized> >> . >> >> Thanks, >> >> KP >> > > >

Re: Spark Metrics: custom source/sink configurations not getting recognized

2016-09-06 Thread Benjamin Kim
> > by Spark's metric system. > > Is there a (programmatic) way to have 'metrics.properties' shipped before > executors initialize > <https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkEnv.scala#L335> > ? > > Here's my SO question > <https://stackoverflow.com/questions/39340080/spark-metrics-custom-source-sink-configurations-not-getting-recognized>. > > Thanks, > > KP > >

Re: Spark Metrics: custom source/sink configurations not getting recognized

2016-09-06 Thread map reduced
github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/master/Master.scala#L84> > by > Spark's metric system. > > Is there a (programmatic) way to have 'metrics.properties' shipped before > executors initialize > <https://github.com/apach

Spark Metrics: custom source/sink configurations not getting recognized

2016-09-05 Thread map reduced
#x27; shipped before executors initialize <https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkEnv.scala#L335> ? Here's my SO question <https://stackoverflow.com/questions/39340080/spark-metrics-custom-source-sink-configurations-not-getting-recognized> . Thanks, KP