Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/292#issuecomment-70416726
Thank you for the feedback. I'll address the inline comments.
Regarding the questions:
1. The separation between `flink-yarn` and `flink-yarn-tests` exists
because the `flink-yarn-tests` expect the `flink-dist` package to be build
before the `flink-yarn-tests` package.
The tests are really starting a YARN cluster and deploying a flink fat-jar
to it. Therefore, we need to run `flink-dist` first. I asked on the maven
mailing list if there is a way of simplifying this and it seems to be possible
to store the archetype descriptors (used to build the fat-jar) in a separate
maven module which is then accessed by `flink-dist` and a `prepare-tests` phase
of `flink-yarn`. But for that approach, we would need to create an additional
maven module (something like `flink-assemblies` for making the assembly
descriptor independent of the `flink-dist` package).
*tl;dr* maven is not flexible enough for a better solution.
2. Yep. That doesn't make much sense (I used a local yarn cluster for
testing it .. thats why I didn't really stumble across it). I'll fix it.
3. This is true for the old YARN client (before this pull request). As you
can see here:
https://github.com/apache/flink/pull/292/files#diff-37b2363833862d636afea47fab39a694L269
I removed the code that was computing the port. This new YARN client is
allocating ALL ports dynamically (web frontend, RPC). I'm using YARN to
transfer the RPC port of the AM to the client.
4. Probably not hard. Its a matter of taste I guess. I think my approach is
more flexible (imagine we want to have a `-m mesos-cluster` or a `-m
flink-local` at some point). Also, we would need to throw an exception if a
user is setting something like `-j -m myCluster:6123`. It would be good to get
some more opinions here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
