[
https://issues.apache.org/jira/browse/FLINK-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644170#comment-14644170
]
ASF GitHub Bot commented on FLINK-2324:
---------------------------------------
Github user gyfora commented on the pull request:
https://github.com/apache/flink/pull/937#issuecomment-125537856
I can rewrite this of course and create new test classes but in any way the
StreamCheckpointingIT case needs to be changed because it was not well designed
in the first place.
It had uncommented tests, and never checked checkpointing after shuffle
which was in fact broken for some reason I am still not sure (see some notes
above).
My only worry is that if we add new test classes for all the cases,
checkpointed, partitioned, non-partitioned (where the checkpointing actually
happens together with the same logic so I don't see a reason for having
separate test) then we end up with a lot of these pretty heavy tests.
> Rework partitioned state storage
> --------------------------------
>
> Key: FLINK-2324
> URL: https://issues.apache.org/jira/browse/FLINK-2324
> Project: Flink
> Issue Type: Improvement
> Reporter: Gyula Fora
> Assignee: Gyula Fora
>
> Partitioned states are currently stored per-key in statehandles. This is
> alright for in-memory storage but is very inefficient for HDFS.
> The logic behind the current mechanism is that this approach provides a way
> to repartition a state without fetching the data from the external storage
> and only manipulating handles.
> We should come up with a solution that can achieve both.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
