the job name is straight up not accessible from within a function. Until
recently
it wasn't even accessible /anywhere/ on the TaskManager.
To get the job name in a function you would have to pass it from the
TaskManager
to the task, to the operator and (probably) to the RuntimeContext.
If you were content with using the job ID you would /only/ have to
modify the RuntimeContext.
Having these context information available in a UDF is imo a very useful
thing to have.
(which TaskManager am i on, which Job/Task/Operator do i belong to)
Note that this obviously only covers the case of modifying the logging
within a UDF. Anything
else would be unaffected.
Regards,
Chesnay
On 02.06.2016 22:39, Kostas Kloudas wrote:
Hello Paul,
If I understand correctly, your issues seem to be related to:
https://issues.apache.org/jira/browse/FLINK-1502?jql=assignee%20in%20(Zentol)%20AND%20text%20~%20%22metrics%22
<https://issues.apache.org/jira/browse/FLINK-1502?jql=assignee%20in%20%28Zentol%29%20AND%20text%20%7E%20%22metrics%22>
and
https://issues.apache.org/jira/browse/FLINK-2732?jql=assignee%20in%20(Zentol)%20AND%20text%20~%20%22metrics%22
<https://issues.apache.org/jira/browse/FLINK-2732?jql=assignee%20in%20%28Zentol%29%20AND%20text%20%7E%20%22metrics%22>
If this is correct, probably Chesnay (who worked on these issues) is
the right person to talk to.
Kostas
On Jun 2, 2016, at 6:53 PM, Hamilton, Paul <paul.hamilto...@sap.com
<mailto:paul.hamilto...@sap.com>> wrote:
Hi,
We are trying to rework the way our Flink jobs (and really it becomes
how the operators within the jobs) log. Specifically we are trying to
add contextual information to the log messages so that it is easier
to filter and search for a specific jobs logs after they have been
send through to our ELK stack. I am certainly open to suggestions,
and hopefully I am missing something easier, but these were my
current thoughts:
1)In the application and then also within individual functions that
log add the “job name” to the logging MDC in the open function so
that no matter where it is running on it can log the appropriate job
name. The problem here is that I haven’t found an easy way to get to
the job name within the RichFunction. What would be the best way to
get it?
2)Pass the job name (and any other contextual things we want to log)
in at construction time when building the flink job to each function
that needs them. Simple, but seems kind of ugly.
Thoughts?
Thanks,
Paul Hamilton
SAP Hybris
