Ted,
That was a useful information. But i want it to use in development
cluster to learn the internals through the logs. On Using %C, actual FQCN
is not printed instead org.apache.spark.Logging overwrites. Is it a
intended change in Spark.
On Sun, Feb 28, 2016 at 3:46 AM, Ted Yu <yuzhih...@gmail.com> wrote:
> Looking at
>
> https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html
>
> *WARNING* Generating the caller class information is slow. Thus, use
> should be avoided unless execution speed is not an issue.
>
> On Sat, Feb 27, 2016 at 12:40 PM, Prabhu Joseph <
> prabhujose.ga...@gmail.com> wrote:
>
>> Hi All,
>>
>> When i change the spark log4j.properties conversion pattern to know
>> the fully qualified class name, all the logs has the FQCN as
>> org.apache.spark.Logging. The actual fully qualified class name is
>> overwritten.
>>
>> log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
>> *%C:* %m%n
>>
>>
>> 16/02/27 15:34:40 INFO org.apache.spark.Logging$class: Successfully
>> started service 'sparkDriver' on port 49482.
>> 16/02/27 15:34:40 DEBUG org.apache.spark.Logging$class: Using serializer:
>> class org.apache.spark.serializer.JavaSerializer
>> 16/02/27 15:34:40 INFO org.apache.spark.Logging$class: Registering
>> MapOutputTracker
>> 16/02/27 15:34:40 INFO org.apache.spark.Logging$class: Registering
>> BlockManagerMaster
>> 16/02/27 15:34:40 ERROR org.apache.spark.Logging$class: Failed to create
>> dir in maprfs:///var/mapr/local/prabhuFS2/spark. Ignoring this directory.
>> 16/02/27 15:34:40 ERROR org.apache.spark.Logging$class: Failed to create
>> any local dir.
>> 16/02/27 15:34:40 INFO org.apache.spark.Logging$class: Shutdown hook
>> called
>> 16/02/27 15:34:40 INFO org.apache.spark.Logging$class: Deleting directory
>> /tmp/spark-5544c349-0393-4bd0-8aab-c20331a9a1cf
>>
>>
>> Thanks,
>> Prabhu Joseph
>>
>>
>
