Re: Replacing Checkpointed interface with field annotations

2015-07-01 Thread Paris Carbone
+1 on offering both. This way we cover both simplicity and expressivity when needed. Annotations give a very clean and simple way for marking state imho. Perhaps most casual users will find it much better to just tag the fields that they want to persist. Paris > On 01 Jul 2015, at 15:55, Step

Re: Replacing Checkpointed interface with field annotations

2015-07-01 Thread Stephan Ewen
Actually, this is the first rework of the state interface. There is only one released version, yet. What we are doing here is trying to make sure that this first rework will most likely also be the last for the foreseeable future. >From the use cases I can think of, we need at least two different

Re: Replacing Checkpointed interface with field annotations

2015-07-01 Thread Robert Metzger
I agree, if we want to change the interface, now is the best time. So you are suggesting to change the methods in the Checkpointed interface from T snapshotState(long checkpointId, long checkpointTimestamp) throws Exception; void restoreState(T state); to void onSnapshot(id, ts) void onRestore

Re: Replacing Checkpointed interface with field annotations

2015-07-01 Thread Gyula Fóra
I understand your concerns Robert but I don't fully agree. The Checkpointed interface works indeed but there are so many use cases that it is not suitable for in the long run, and also the whole interface is slightly awkward in my opinion when returning simple fields which are already serializable

Re: Replacing Checkpointed interface with field annotations

2015-07-01 Thread Robert Metzger
Whats causing me the biggest headache here is that I don't see an end on all these "state interface" reworks. I think this is now the third big change to the interface. It is a horrible user experience to rework your old code with each new Flink release. I understand that there are always ways to

Re: Replacing Checkpointed interface with field annotations

2015-07-01 Thread Ufuk Celebi
On 01 Jul 2015, at 10:57, Gyula Fóra wrote: > Hey, > > Thanks for the feedback guys: > > @Max: You are right, this is not top priority to changes, I was just > mocking up some alternatives to try to make the state usage even simpler so > that the user can keep his current implementations and j

Re: Replacing Checkpointed interface with field annotations

2015-07-01 Thread Gyula Fóra
Hey, Thanks for the feedback guys: @Max: You are right, this is not top priority to changes, I was just mocking up some alternatives to try to make the state usage even simpler so that the user can keep his current implementations and just add 1-2 annotations. @Stephan, Robert: You are right tha

Re: Replacing Checkpointed interface with field annotations

2015-07-01 Thread Stephan Ewen
+1 for adding the annotation, but not removing the interface Robert is right, the nice thing about the current interface is that you can use it to commit the state yourself to a database and simply return a key to where the state is stored. That is quite nice. On Wed, Jul 1, 2015 at 10:14 AM, Rob

Re: Replacing Checkpointed interface with field annotations

2015-07-01 Thread Robert Metzger
I would certainly not replace the current Checkpointed interface by this implementation. The interface allows you to perform custom actions when creating a snapshot or restoring state. We could add the annotation variant for simple cases like in your example. On Wed, Jul 1, 2015 at 10:10 AM, Max

Re: Replacing Checkpointed interface with field annotations

2015-07-01 Thread Maximilian Michels
Hi Gyula, Looks like a neat feature you thought of; I like it. One problem I see is that with all the big changes and discussions in streaming, I wonder if we should get other things right first. Another problem could be the lack of clarity of this implementation. State annotations can now be very

Re: Replacing Checkpointed interface with field annotations

2015-06-30 Thread Hermann Gábor
Wow, this looks pretty concise. I really like it! On Mon, Jun 29, 2015 at 3:27 PM Gyula Fóra wrote: > Hey all! > > Just to add something new to the end of the discussion list. After some > discussion with Seif, and Paris, I have added a commit that replaces the > use of the Checkpointed interfac

Replacing Checkpointed interface with field annotations

2015-06-29 Thread Gyula Fóra
Hey all! Just to add something new to the end of the discussion list. After some discussion with Seif, and Paris, I have added a commit that replaces the use of the Checkpointed interface with field annotations. This is probably the most lightweight state declaration so far and it will probably w