[
https://issues.apache.org/jira/browse/FLINK-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316204#comment-14316204
]
Gyula Fora commented on FLINK-1493:
-----------------------------------
Hey,
I implemented a simple indexed iterator from which you can get the channel
index of the last record and also the total number of channels:
Please try it out at let me know if it is working properly I am not going to
push to the master until then (I need to take a flight soon and can only get
back to this on Friday).
Find the commit here:
https://github.com/mbalassi/flink/commit/ee3590ec4427aeca4bbde352199db972b47ee12d
The Jira issue:
https://issues.apache.org/jira/browse/FLINK-1517
On top of this you can implement the sorting operators I think by buffering up
the inputs. Lets not push this logic down to the AbstractRecordReader and keep
it at the operator. Thats the cleanest for now.
> Support for streaming jobs preserving global ordering of records
> ----------------------------------------------------------------
>
> Key: FLINK-1493
> URL: https://issues.apache.org/jira/browse/FLINK-1493
> Project: Flink
> Issue Type: New Feature
> Components: Streaming
> Reporter: Márton Balassi
>
> Distributed streaming jobs do not give total, global ordering guarantees for
> records only partial ordering is provided by the system: records travelling
> on the same exact route of the physical plan are ordered, but they aren't
> between routes.
> It turns out that although this feature can only be implemented via "merge
> sorting" in the input buffers on a timestamp field thus creating substantial
> latency is still desired for a number of applications.
> Just a heads up for the implementation: the sorting introduces back pressure
> in the buffers and might cause deadlocks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
