Hi David,
you are right, for some exceptions Flink only forwards to the
web-dashboard/application client but does not print to the log file. I
opened a Jira issue to track this: FLINK-3969
<https://issues.apache.org/jira/browse/FLINK-3969>.
Thanks for reporting!
Aljoscha
On Mon, 23 May 2016 at 22:08 David Kim <david....@braintreepayments.com>
wrote:
> Hi Max!
>
> Unfortunately, that's not the behavior I'm seeing.
>
> I verified my log4.properties is configured properly because I do see
> messages in the /log directory.
>
> However, for this stack trace (grabbed from the web dashboard), I do not
> see it in my log file:
>
> java.lang.RuntimeException: Could not forward element to next operator
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:354)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:337)
> at
> org.apache.flink.streaming.api.operators.StreamSource$NonTimestampContext.collect(StreamSource.java:158)
> at
> org.apache.flink.streaming.api.functions.source.FromElementsFunction.run(FromElementsFunction.java:127)
> at
> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:78)
> at
> org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:56)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:225)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
> at java.lang.Thread.run(Thread.java:745)
>
>
> Here's my sample program:
>
>
> val env = StreamExecutionEnvironment.getExecutionEnvironment
> env
> .fromCollection(List(1))
> .filter(element => {
> throw new RuntimeException("Throwing a runtime to test exception logging
> :)")
> })
> env.execute("MyTestJobName")
>
>
> The error is thrown in Task.java[1] and it was hard to track where flink
> would actually log to the log file.
>
>
> [1]
> https://github.com/apache/flink/blob/release-1.0.2-rc3/flink-runtime/src/main/java/org/apache/flink/runtime/taskmanager/Task.java#L559
>
>
> Thanks!
>
> David
>
>
> On Mon, May 23, 2016 at 12:01 PM Maximilian Michels <m...@apache.org>
> wrote:
>
>> Hi David,
>>
>> I'm afraid Flink logs all exceptions. You'll find the exceptions in the
>> /log directory.
>>
>> Cheers,
>> Max
>>
>> On Mon, May 23, 2016 at 6:18 PM, David Kim <
>> david....@braintreepayments.com> wrote:
>>
>>> Hello!
>>>
>>> Just wanted to check up on this. :)
>>>
>>> I grepped around for `log.error` and it *seems* that currently the only
>>> events for logging out exceptions are for non-application related errors.
>>>
>>> Thanks!
>>> David
>>>
>>> On Fri, May 20, 2016 at 12:35 PM David Kim <
>>> david....@braintreepayments.com> wrote:
>>>
>>>> Hello!
>>>>
>>>> Using flink 1.0.2, I noticed that exceptions thrown during a flink
>>>> program would show up on the flink dashboard in the 'Exceptions' tab.
>>>> That's great!
>>>>
>>>> However, I don't think flink currently logs this same exception. I was
>>>> hoping there would be an equivalent `log.error` call so that third party
>>>> logging frameworks can also act upon such errors.
>>>>
>>>> If this currently the known behavior, would it be troublesome to also
>>>> make a `log.error` call around the code that is responsible for sending the
>>>> exception to the dashboard?
>>>>
>>>> If there's a misconfiguration on my end, let me know!
>>>>
>>>> Thanks!
>>>> David
>>>>
>>>
>>
