I have changed the properties file but it did not help.
Regards,
Vijay
Sent from my iPhone
> On Mar 23, 2016, at 5:39 AM, Aljoscha Krettek <aljos...@apache.org> wrote:
>
> Ok, then you should be able to change the log level to DEBUG in
> conf/log4j.properties.
>
>> On 23 Mar 2016, at 12:41, Vijay <vijikar...@yahoo.com> wrote:
>>
>> I think only the ERROR category gets displayed in the log file
>>
>> Regards,
>> Vijay
>>
>> Sent from my iPhone
>>
>>> On Mar 23, 2016, at 2:30 AM, Aljoscha Krettek <aljos...@apache.org> wrote:
>>>
>>> Hi,
>>> are you seeing the regular log output from the RollingSink in the
>>> TaskManager logs?
>>>
>>> Cheers,
>>> Aljoscha
>>>> On 22 Mar 2016, at 20:03, Vijay Srinivasaraghavan <vijikar...@yahoo.com>
>>>> wrote:
>>>>
>>>> I have tried both log4j logger as well as System.out.println option but
>>>> none of these worked.
>>>>
>>>> From what I have seen so far is the Filesystem streaming connector classes
>>>> are not packaged in the grand jar (flink-dist_2.10-1.1-SNAPSHOT.jar) that
>>>> is copied under <FLINK_HOME>/build-target/lib location as part of Flink
>>>> maven build step.
>>>>
>>>> So, I manually copied (overwrite) the compiled class files from
>>>> org.apache.flink.streaming.connectors.fs package to the my "Flink job"
>>>> distribution jar (otherwise it was using standard jars that are defined as
>>>> mvn dependency in Articatory) and then uploaded the jar to Job Manager.
>>>>
>>>> Am I missing something? How do I enable logging for the RollingSink class?
>>>>
>>>> <dependency>
>>>> <groupId>org.apache.flink</groupId>
>>>> <artifactId>flink-connector-filesystem_2.11</artifactId>
>>>> <version>${flink.version}</version>
>>>> <scope>provided</scope>
>>>> </dependency>
>>>>
>>>>
>>>> On Tuesday, March 22, 2016 3:04 AM, Aljoscha Krettek <aljos...@apache.org>
>>>> wrote:
>>>>
>>>>
>>>> Hi,
>>>> how are you printing the debug statements?
>>>>
>>>> But yeah all the logic of renaming in progress files and cleaning up after
>>>> a failed job happens in restoreState(BucketState state). The steps are
>>>> roughly these:
>>>>
>>>> 1. Move current in-progress file to final location
>>>> 2. truncate the file if necessary (if truncate is not available write a
>>>> .valid-length file)
>>>> 3. Move pending files to final location that where part of the checkpoint
>>>> 4. cleanup any leftover pending/in-progress files
>>>>
>>>> Cheers,
>>>> Aljoscha
>>>>
>>>>> On 22 Mar 2016, at 10:08, Vijay Srinivasaraghavan
>>>>> <vijikar...@yahoo.com.INVALID> wrote:
>>>>>
>>>>> Hello,
>>>>> I have enabled checkpoint and I am using RollingSink to sink the data to
>>>>> HDFS (2.7.x) from KafkaConsumer. To simulate failover/recovery, I stopped
>>>>> TaskManager and the job gets rescheduled to other Taskmanager instance.
>>>>> During this momemnt, the current "in-progress" gets closed and renamed to
>>>>> part-0-1 from _part-0-1_in-progress.
>>>>> I was hoping to see the debug statement that I have added to
>>>>> "restoreState" method but none of my debug statement gets printed. I am
>>>>> not sure if the restoreState() method gets invoked during this scenario.
>>>>> Could you please help me understand the flow during "failover" scenario?
>>>>> P.S: Functionally the code appears to be working fine but I am trying to
>>>>> understand the underlying implementation details. public void
>>>>> restoreState(BucketState state)
>>>>> Regards
>>>>> Vijay
>
