[
https://issues.apache.org/jira/browse/FLINK-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15346600#comment-15346600
]
Geoffrey Mon edited comment on FLINK-4098 at 6/23/16 3:26 PM:
--------------------------------------------------------------
I wasn't sure how to put in print statements because they don't seem to appear
in the jobmanager {{.out}} when I add them into {{MapFunction._run()}}.
Here's a trimmed jobmanager log for a simple Python iteration program that
iterates each number in a Dataset:
{code}
INFO org.apache.flink.runtime.jobmanager.JobManager - Status of
job ac35083fec7c77cc1707245563aed90d (Flink Java Job at Thu Jun 23 11:14:51 EDT
2016) changed to RUNNING.
<snip>
INFO org.apache.flink.runtime.taskmanager.Task - Obtaining
local cache file for 'flink'
INFO org.apache.flink.runtime.taskmanager.Task -
PartialSolution (Bulk Iteration) (1/1) switched to RUNNING
INFO org.apache.flink.runtime.executiongraph.ExecutionGraph -
PartialSolution (Bulk Iteration) (1/1) (5c0ff0bae3952df909198ebf94f51dcf)
switched from DEPLOYING to RUNNING
INFO org.apache.flink.runtime.iterative.task.IterationHeadTask - starting
iteration \[1\]: PartialSolution (Bulk Iteration) (1/1)
INFO org.apache.flink.runtime.executiongraph.ExecutionGraph -
MapPartition (PythonMap) (1/1) (107be54ae5b60776dc35cc2ace83a2f0) switched from
CREATED to SCHEDULED
INFO org.apache.flink.runtime.executiongraph.ExecutionGraph -
MapPartition (PythonMap) (1/1) (107be54ae5b60776dc35cc2ace83a2f0) switched from
SCHEDULED to DEPLOYING
<snip>
INFO org.apache.flink.runtime.taskmanager.Task -
MapPartition (PythonMap) (1/1) switched to RUNNING
INFO org.apache.flink.runtime.executiongraph.ExecutionGraph -
MapPartition (PythonMap) (1/1) (107be54ae5b60776dc35cc2ace83a2f0) switched from
DEPLOYING to RUNNING
INFO org.apache.flink.runtime.iterative.task.IterationTailTask - starting
iteration \[1\]: MapPartition (PythonMap) (1/1)
{code}
The program does not proceed past that point and appears to hang without any
errors.
was (Author: hydronium):
Here's a jobmanager log for a simple Python iteration program that iterates
each number in a Dataset:
2016-06-23 11:14:51,900 INFO org.apache.flink.runtime.jobmanager.JobManager
- Status of job ac35083fec7c77cc1707245563aed90d (Flink Java Job at
Thu Jun 23 11:14:51 EDT 2016) changed to RUNNING.
<snip>
2016-06-23 11:14:52,266 INFO org.apache.flink.runtime.taskmanager.Task
- Obtaining local cache file for 'flink'
2016-06-23 11:14:52,267 INFO org.apache.flink.runtime.taskmanager.Task
- PartialSolution (Bulk Iteration) (1/1) switched to RUNNING
2016-06-23 11:14:52,268 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - PartialSolution
(Bulk Iteration) (1/1) (5c0ff0bae3952df909198ebf94f51dcf) switched from
DEPLOYING to RUNNING
2016-06-23 11:14:53,053 INFO
org.apache.flink.runtime.iterative.task.IterationHeadTask - starting
iteration [1]: PartialSolution (Bulk Iteration) (1/1)
2016-06-23 11:14:53,059 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - MapPartition
(PythonMap) (1/1) (107be54ae5b60776dc35cc2ace83a2f0) switched from CREATED to
SCHEDULED
2016-06-23 11:14:53,059 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - MapPartition
(PythonMap) (1/1) (107be54ae5b60776dc35cc2ace83a2f0) switched from SCHEDULED to
DEPLOYING
<snip>
2016-06-23 11:14:53,077 INFO org.apache.flink.runtime.taskmanager.Task
- MapPartition (PythonMap) (1/1) switched to RUNNING
2016-06-23 11:14:53,078 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - MapPartition
(PythonMap) (1/1) (107be54ae5b60776dc35cc2ace83a2f0) switched from DEPLOYING to
RUNNING
2016-06-23 11:14:53,155 INFO
org.apache.flink.runtime.iterative.task.IterationTailTask - starting
iteration [1]: MapPartition (PythonMap) (1/1)
The program does not proceed past that point and appears to hang without any
errors.
> Iteration support in Python API
> -------------------------------
>
> Key: FLINK-4098
> URL: https://issues.apache.org/jira/browse/FLINK-4098
> Project: Flink
> Issue Type: New Feature
> Components: Python API
> Affects Versions: 1.0.2
> Reporter: Geoffrey Mon
> Priority: Minor
>
> Bulk and delta iterations are not supported in the Python API. Currently
> working on this at https://github.com/GEOFBOT/flink
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
