dianfu commented on a change in pull request #10017: [FLINK-14019][python] add
support for managing environment and dependencies of Python UDF in Flink Python
API
URL: https://github.com/apache/flink/pull/10017#discussion_r348871126
##########
File path:
flink-python/src/main/java/org/apache/flink/python/AbstractPythonFunctionRunner.java
##########
@@ -283,74 +275,19 @@ public void processElement(IN element) {
@VisibleForTesting
public JobBundleFactory createJobBundleFactory(Struct pipelineOptions)
throws Exception {
return DefaultJobBundleFactory.create(
- JobInfo.create(taskName, taskName,
createEmptyRetrievalToken(), pipelineOptions));
- }
-
- private String createEmptyRetrievalToken() throws Exception {
- // try to find a unique file name for the retrieval token
- final Random rnd = new Random();
- for (int attempt = 0; attempt < 10; attempt++) {
- String directory =
tempDirs[rnd.nextInt(tempDirs.length)];
- retrievalToken = new File(directory, randomString(rnd)
+ ".json");
- if (retrievalToken.createNewFile()) {
- final DataOutputStream dos = new
DataOutputStream(new FileOutputStream(retrievalToken));
- dos.writeBytes("{\"manifest\": {}}");
- dos.flush();
- dos.close();
- return retrievalToken.getAbsolutePath();
- }
- }
-
- throw new IOException(
- "Could not find a unique file name in '" +
Arrays.toString(tempDirs) + "' for retrieval token.");
+ JobInfo.create(taskName, taskName,
createRetrievalToken(), pipelineOptions));
}
- private static String randomString(Random random) {
- final byte[] bytes = new byte[20];
- random.nextBytes(bytes);
- return StringUtils.byteToHexString(bytes);
+ private String createRetrievalToken() throws IOException {
Review comment:
remove this method and use environmentManager.createRetrievalToken()
directly?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
