[
https://issues.apache.org/jira/browse/FLINK-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16681430#comment-16681430
]
ASF GitHub Bot commented on FLINK-6046:
---------------------------------------
tillrohrmann closed pull request #7050: [hotfix] Follow up FLINK-6046 to clean
unused variable
URL: https://github.com/apache/flink/pull/7050
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/ExecutionJobVertex.java
b/flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/ExecutionJobVertex.java
index 2ab1d686410..ee735050976 100644
---
a/flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/ExecutionJobVertex.java
+++
b/flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/ExecutionJobVertex.java
@@ -56,7 +56,6 @@
import org.slf4j.Logger;
import javax.annotation.Nonnull;
-import javax.annotation.Nullable;
import java.io.IOException;
import java.util.ArrayList;
@@ -126,19 +125,10 @@
private int maxParallelism;
/**
- * Serialized task information which is for all sub tasks the same.
Thus, it avoids to
- * serialize the same information multiple times in order to create the
- * TaskDeploymentDescriptors.
+ * Either store a serialized task information, which is for all sub
tasks the same,
+ * or the permanent blob key of the offloaded task information BLOB
containing
+ * the serialized task information.
*/
- private SerializedValue<TaskInformation> serializedTaskInformation;
-
- /**
- * The key of the offloaded task information BLOB containing {@link
#serializedTaskInformation}
- * or <tt>null</tt> if not offloaded.
- */
- @Nullable
- private PermanentBlobKey taskInformationBlobKey = null;
-
private Either<SerializedValue<TaskInformation>, PermanentBlobKey>
taskInformationOrBlobKey = null;
private InputSplitAssigner splitAssigner;
@@ -193,8 +183,6 @@ public ExecutionJobVertex(
this.parallelism = numTaskVertices;
- this.serializedTaskInformation = null;
-
this.taskVertices = new ExecutionVertex[numTaskVertices];
this.operatorIDs =
Collections.unmodifiableList(jobVertex.getOperatorIDs());
this.userDefinedOperatorIds =
Collections.unmodifiableList(jobVertex.getUserDefinedOperatorIDs());
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add support for oversized messages during deployment
> ----------------------------------------------------
>
> Key: FLINK-6046
> URL: https://issues.apache.org/jira/browse/FLINK-6046
> Project: Flink
> Issue Type: New Feature
> Components: Distributed Coordination, Network
> Reporter: Nico Kruber
> Assignee: Nico Kruber
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.4.0
>
>
> This is the non-FLIP6 version of FLINK-4346, restricted to deployment
> messages:
> Currently, messages larger than the maximum Akka Framesize cause an error
> when being transported. We should add a way to pass messages that are larger
> than {{akka.framesize}} as may happen for task deployments via the
> {{TaskDeploymentDescriptor}}.
> We should use the {{BlobServer}} to offload big data items (if possible) and
> make use of any potential distributed file system behind. This way, not only
> do we avoid the akka framesize restriction, but may also be able to speed up
> deployment.
> I suggest the following changes:
> - the sender, i.e. the {{Execution}} class, tries to store the serialized
> job information and serialized task information (if oversized) from the
> {{TaskDeploymentDescriptor}} (tdd) on the {{BlobServer}} as a single
> {{NAME_ADDRESSABLE}} blob under its job ID (if this does not work, we send
> the whole tdd as usual via akka)
> - if stored in a blob, these data items are removed from the tdd
> - the receiver, i.e. the {{TaskManager}} class, tries to retrieve any
> offloaded data after receiving the {{TaskDeploymentDescriptor}} from akka; it
> re-assembles the original tdd
> - the stored blob may be deleted after re-assembly of the tdd
> Further (future) changes may include:
> - separating the serialized job information and serialized task information
> into two files and re-use the first one for all tasks
> - not re-deploying these two during job recovery (if possible)
> - then, as all other {{NAME_ADDRESSABLE}} blobs, these offloaded blobs may
> be removed when the job enters a final state instead
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
