for quite some usecases..
Best regards
Theo
Von: "Flavio Pompermaier"
An: "Aljoscha Krettek"
CC: "user" , "Till Rohrmann"
Gesendet: Mittwoch, 25. November 2020 10:44:57
Betreff: Re: Logs of JobExecutionListener
Thank you all for the clarific
Thank you all for the clarification..now things are much more clear. I hope
this discussion could be of clarification for other people having the same
doubts.
Best,
Flavio
On Wed, Nov 25, 2020 at 10:27 AM Aljoscha Krettek
wrote:
> One bigger problem here is that the code base is very inconsiste
One bigger problem here is that the code base is very inconsistent when
it comes to the @Public//@Internal annotations. Initially, only the
packages that were meant to be "public" had them. For example flink-core
has thorough annotations. Packages that were not meant to have any
user-facing cod
ok that's fine to me, just add an @internal annotation on the
RestClusterClient if it is intended only for internal use.. but wouldn't be
easier to provide some sort of client generation facility (e.g. swagger or
similar)?
Il mar 24 nov 2020, 11:38 Till Rohrmann ha scritto:
> I see the point in
I see the point in having a richer RestClusterClient. However, I think we
first have to make a decision whether the RestClusterClient is something
internal or not. If it is something internal, then only extending the
RestClusterClient and not adding these convenience methods to ClusterClient
could
For the sake of simplification (so everybody looking for missing methods
in RestClusterClient) I just shared the new methods at [1].
In this way you can add them to the RestClusterClient when you want (if you
want to).
I also had to change the visibility of some variables and methods in order
to ma
I don't know if they need to be added also to the ClusterClient but for
sure they are missing in the RestClusterClient
On Mon, Nov 23, 2020 at 4:31 PM Aljoscha Krettek
wrote:
> On 23.11.20 16:26, Flavio Pompermaier wrote:
> > Thank you Aljosha,.now that's more clear!
> > I didn't know that jobGr
On 23.11.20 16:26, Flavio Pompermaier wrote:
Thank you Aljosha,.now that's more clear!
I didn't know that jobGraph.getJobID() was the solution for my use case..I
was convinced that the job ID was assigned by the cluster!
And to me it's really weird that the job listener was not called by the
subm
Thank you Aljosha,.now that's more clear!
I didn't know that jobGraph.getJobID() was the solution for my use case..I
was convinced that the job ID was assigned by the cluster!
And to me it's really weird that the job listener was not called by the
submitJob...Probably this should be documented at l
On 20.11.20 22:09, Flavio Pompermaier wrote:
To achieve this, I was using the
RestClusterClient because with that I can use the
following code and retrieve the JobID:
(1) JobID flinkJobId =
client.submitJob(jobGraph).thenApply(DetachedJobExecutionResult::new).get().getJobID();
All you wan
My final analysis is that the RestClusterClient lack of many methods
(jarUpload, jarRun, getExceptions for example) and that the submitJob (and
the JobSubmitHandler endpoint) is bugged or should be deprecated (because
it does not call the job listeners).
Indeed, if the JarRunHandler endpoint is inv
I think that the problem is that my REST service submits the job to
the Flink standalone cluster and responds to the client with the
submitted job ID.
To achieve this, I was using the
RestClusterClient because with that I can use the
following code and retrieve the JobID:
(1) JobID flinkJobId
You're right..I removed my flink dir and I re-extracted it and now it
works. Unfortunately I didn't keep the old version to understand what
were the difference but the error was probably caused by the fact that
I had a previous version of the WordCount.jar (without the listener)
in the flink lib di
I also tried 1.11.0 and 1.11.2, both work for me.
On Thu, Nov 19, 2020 at 3:39 PM Aljoscha Krettek
wrote:
> Hmm, there was this issue:
> https://issues.apache.org/jira/browse/FLINK-17744 But it should be fixed
> in your version.
>
> On 19.11.20 12:58, Flavio Pompermaier wrote:
> > Which version
Hmm, there was this issue:
https://issues.apache.org/jira/browse/FLINK-17744 But it should be fixed
in your version.
On 19.11.20 12:58, Flavio Pompermaier wrote:
Which version are you using?
I used the exact same commands on Flink 1.11.0 and I didn't get the job
listener output..
Il gio 19 no
Which version are you using?
I used the exact same commands on Flink 1.11.0 and I didn't get the job
listener output..
Il gio 19 nov 2020, 12:53 Andrey Zagrebin ha scritto:
> Hi Flavio and Aljoscha,
>
> Sorry for the late heads up. I could not actually reproduce the reported
> problem with 'flin
Hi Aljoscha, in my main class, within the jar, I create the env and I call
env.execute(). The listener is not called if the job is ran by the CLI
client or FlinkRestClient, I don't see anything on the job manager or task
manager. To me this is a bug and you can verify it attaching a listener to
the
Hi Flavio and Aljoscha,
Sorry for the late heads up. I could not actually reproduce the reported
problem with 'flink run' and local standalone cluster on master.
I get the expected output with the suggested modification of WordCount
program:
$ bin/start-cluster.sh
$ rm -rf out; bin/flink run
fli
JobListener.onJobExecuted() is only invoked in
ExecutionEnvironment.execute() and ContextEnvironment.execute(). If none
of these is still in the call chain with that setup then the listener
will not be invoked.
Also, this would only happen on the client, not on the broker (in your
case) or th
I have a spring boot job server that act as a broker towards our
application and a Flink session cluster. To submit a job I use the
FlinkRestClient (that is also the one used in the CLI client when I use the
run action it if I'm not wrong). However both methods don't trigger the job
listener.
Il g
@Flavio, when you're saying you're using the RestClusterClient, you are
not actually using that manually, right? You're just submitting your job
via "bin/flink run ...", right?
What's the exact invocation of "bin/flink run" that you're using?
On 19.11.20 09:29, Andrey Zagrebin wrote:
Hi Flavi
Hi Flavio,
I think I can reproduce what you are reporting (assuming you also pass
'--output' to 'flink run').
I am not sure why it behaves like this. I would suggest filing a Jira
ticket for this.
Best,
Andrey
On Wed, Nov 18, 2020 at 9:45 AM Flavio Pompermaier
wrote:
> is this a bug or is it a
is this a bug or is it a documentation problem...?
Il sab 14 nov 2020, 18:44 Flavio Pompermaier ha
scritto:
> I've also verified that the problem persist also using a modified version
> of the WordCount class.
> If you add the code pasted at the end of this email at the end of its main
> method
I've also verified that the problem persist also using a modified version
of the WordCount class.
If you add the code pasted at the end of this email at the end of its main
method you can verify that the listener is called if you run the program
from the IDE, but it's not called if you submit the j
see inline
Il ven 13 nov 2020, 14:31 Matthias Pohl ha scritto:
> Hi Flavio,
> thanks for sharing this with the Flink community. Could you answer the
> following questions, please:
> - What's the code of your Job's main method?
>
it's actually very simple...the main class creates a batch executi
Hi Flavio,
thanks for sharing this with the Flink community. Could you answer the
following questions, please:
- What's the code of your Job's main method?
- What cluster backend and application do you use to execute the job?
- Is there anything suspicious you can find in the logs that might be
rel
Actually what I'm experiencing is that the JobListener is executed
successfully if I run my main class from the IDE, while the job listener is
not fired at all if I submit the JobGraph of the application to a cluster
using the RestClusterClient..
Am I doing something wrong?
My main class ends with
27 matches
Mail list logo
