[
https://issues.apache.org/jira/browse/FLINK-1789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14948287#comment-14948287
]
ASF GitHub Bot commented on FLINK-1789:
---------------------------------------
Github user mxm commented on a diff in the pull request:
https://github.com/apache/flink/pull/593#discussion_r41488791
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/RemoteEnvironment.java ---
@@ -55,21 +59,8 @@
/** Optional shutdown hook, used in session mode to eagerly terminate
the last session */
private Thread shutdownHook;
- /**
- * Creates a new RemoteEnvironment that points to the master
(JobManager) described by the
- * given host name and port.
- *
- * <p>Each program execution will have all the given JAR files in its
classpath.
- *
- * @param host The host name or address of the master (JobManager),
where the program should be executed.
- * @param port The port of the master (JobManager), where the program
should be executed.
- * @param jarFiles The JAR files with code that needs to be shipped to
the cluster. If the program uses
- * user-defined functions, user-defined input formats,
or any libraries, those must be
- * provided in the JAR files.
- */
- public RemoteEnvironment(String host, int port, String... jarFiles) {
--- End diff --
Could we keep this constructor?
> Allow adding of URLs to the usercode class loader
> -------------------------------------------------
>
> Key: FLINK-1789
> URL: https://issues.apache.org/jira/browse/FLINK-1789
> Project: Flink
> Issue Type: Improvement
> Components: Distributed Runtime
> Reporter: Timo Walther
> Assignee: Timo Walther
> Priority: Minor
>
> Currently, there is no option to add customs classpath URLs to the
> FlinkUserCodeClassLoader. JARs always need to be shipped to the cluster even
> if they are already present on all nodes.
> It would be great if RemoteEnvironment also accepts valid classpaths URLs and
> forwards them to BlobLibraryCacheManager.
> The problem with the current approach is that the code loaded by the regular
> JVM class loader cannot refer to job specific types (which can be accessed
> only at the UserCodeClassLoader level). Unfortunately, this is the case if we
> use the classpath entry to generate the dataflows dynamically at runtime.
> Currently this functionality needs to be done by "hacks" (hardcode a
> filesystem path next to the list of jars when initializing the BlobManager
> entry). It makes sense to open an issue which makes this list parameterizable
> via an additional ExecutionEnvironment argument (this is basically the only
> main feature which prohibits the use of Emma project with "off-the-shelf"
> Flink).
> This, of course, would require that the folders are shared (e.g. via NFS)
> between client, master and workers. I think what made Stephan so excited is
> the idea of using the same URL mechanism in order to ship the code to all
> dependent parties (most probably by running a dedicated HTTP or FTP server on
> the client).
> We make the following assumptions for the use case where we need the global
> class path:
> - The URL is either a file path that points to a directory accessible to all
> nodes (NFS or so) and the client runs in the cluster as well.
> - The URL is an HTTP URL or so that points to a file server that serves the
> classes to work in non-shared directory settings.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
