Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3359
Because the old code is too outdated, I have updated the PR, reimplementing
RocksDBInternalTimerService from scratch. Some problems mentioned in the
previous comments may still exist, but I
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3359
@StefanRRichter Sorry for the delayed response. I am working on it and
shall update the PR by this weekend.
---
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3359
Very sorry for the delay. I was engaged at the work in the past months,
making flink capable of the terrible data flows in Singles Day.
RocksDBInternalTimerService is among the
Github user shixiaogang closed the pull request at:
https://github.com/apache/flink/pull/3859
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3859
I noticed that FLINK-6504 is also fixed in
https://github.com/apache/flink/pull/3870 , close this pr and let us address
all problems of incremental checkpointing there.
---
If your project is
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3870#discussion_r116172093
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/SharedStateRegistry.java
---
@@ -18,91 +18,137 @@
package
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3870#discussion_r116161754
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java
---
@@ -922,6
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3870#discussion_r116161318
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBIncrementalKeyedStateHandle.java
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3870#discussion_r116161230
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBIncrementalKeyedStateHandle.java
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3870#discussion_r116161117
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBIncrementalKeyedStateHandle.java
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3859#discussion_r116148624
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java
---
@@ -911,9
GitHub user shixiaogang opened a pull request:
https://github.com/apache/flink/pull/3859
[FLINK-6504] [FLINK-6467] [checkpoints] Add needed synchronization for
RocksDBIncrementalSnapshotOperation
This pull request adds missing synchronization for the access to the
following
Github user shixiaogang closed the pull request at:
https://github.com/apache/flink/pull/3801
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3801
Hi @StefanRRichter Thanks a lot for you pointing out the problem the
suggestion for the fix. I have updated the PR as suggested. A
`CloseableRegistry` is used to track opened i/o streams now
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3801
@StefanRRichter Thanks a lot for your review. I have updated the pull
request as suggested. The following changes are made
1. Remove the checkpoint type for incremental checkpoints. Now
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3801#discussion_r114703946
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java
---
@@ -808,6
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3801#discussion_r114703775
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java
---
@@ -621,6
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3801#discussion_r114565991
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java
---
@@ -769,9 +769,10 @@ public
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3801#discussion_r114565968
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateHandle.java
---
@@ -0,0
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3801#discussion_r114500597
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java
---
@@ -621,6
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3801
Hi @gyfora I am very happy to hear from you. The following are the answers
to your questions. Kindly let me know if you have any idea of them.
1. The incremental checkpoints supports
GitHub user shixiaogang opened a pull request:
https://github.com/apache/flink/pull/3801
[FLINK-6364] [checkpoints] Implement incremental checkpointing in
RocksDBKeyedStateBackend
This is the initial implementation of incremental checkpointing in
RocksDBKeyedStateBackend. Changes
Github user shixiaogang closed the pull request at:
https://github.com/apache/flink/pull/3524
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3359
@vpernin Thanks very much for your attention. The PR is supposed to work on
1.3-SNAPSHOT, but it's not testable now due to some known bugs.
Besides, i want to add suppor
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3524
@StephanEwen I have updated the PR, making the following changes:
1. Add a method called `discardSharedStatesOnFail()` in
`CompositeStateHandle`. This method is called when the pending
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3652#discussion_r108870133
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/test/java/org/apache/flink/contrib/streaming/state/benchmark/ListViaMergeSpeedMiniBenchmark.java
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3652#discussion_r108869236
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/test/java/org/apache/flink/contrib/streaming/state/benchmark/ListViaRangeSpeedMiniBenchmark.java
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3652#discussion_r108868849
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/test/java/org/apache/flink/contrib/streaming/state/benchmark/ListViaMergeSpeedMiniBenchmark.java
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3652#discussion_r108869459
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/test/java/org/apache/flink/contrib/streaming/state/benchmark/ListViaRangeSpeedMiniBenchmark.java
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3652#discussion_r108868469
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/test/java/org/apache/flink/contrib/streaming/state/benchmark/ListViaMergeSpeedMiniBenchmark.java
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3524
@StephanEwen Thanks very much for your valuable comments. The following
are some of my thoughts.
* Now the registration of shared states is put in `CheckpointCoordinator`
because it&#
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3524#discussion_r108826346
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/SharedStateRegistry.java
---
@@ -0,0 +1,192 @@
+/*
+ * Licensed to the
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3524#discussion_r108826017
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/SharedStateRegistry.java
---
@@ -0,0 +1,192 @@
+/*
+ * Licensed to the
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3558#discussion_r108615377
--- Diff:
flink-runtime/src/main/java/org/apache/flink/migration/v0/SavepointV0.java ---
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache
Github user shixiaogang closed the pull request at:
https://github.com/apache/flink/pull/3531
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3531
@StefanRRichter Thanks for your work. I will close the PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3531
@StefanRRichter I updated the PR as suggested. Very appreciated for your
hard work.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3524
@StephanEwen I have updated the PR as suggested. Changes include
1. Make `StateRegistry` to be `SharedStateRegistry` where only shared
states are registered. Now the `discardState()` method
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3531#discussion_r107704681
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/KeyGroupsStateHandle.java
---
@@ -91,10 +98,10 @@ public KeyGroupsStateHandle
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3531
@StefanRRichter Thanks a lot for your comments. I have updated the pull
request as suggested, making the type of raw keyed states to be
`KeyedStateHandle` as well.
---
If your project is set
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3531#discussion_r107646547
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/StateAssignmentOperation.java
---
@@ -306,6 +307,29 @@ private static void
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3531#discussion_r107646429
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/StateAssignmentOperation.java
---
@@ -306,6 +307,29 @@ private static void
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3524
Hi @StephanEwen The main reason is that we must have methods to delete
those unshared objects in failed `PendingCheckpoint`s. The `discardState()`
method is called when either the
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3521
@StephanEwen I added two tests to ensure that the checkpoints are not in
the store when exceptions are thrown. The methods to mock the exceptions of
subsuming may be a little tricky. Do you have
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3524
@StephanEwen Thanks a lot for your valuable comments. I will update the PR
as suggested.
* I think it's a good idea that we make the `StateRegistry` into
`SharedStateRegistry`. That
GitHub user shixiaogang opened a pull request:
https://github.com/apache/flink/pull/3558
[FLINK-6096][checkpoint] Refactor the migration of old-versioned savepoints
1. The migrated classes in `SavepointV0` are moved to the package
`org.apache.flink.migration.v0`. In the future
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3380
I prefer to throw more detailed exceptions e.g.
`IncompatibleTypeSerializerException`, `StateAccessException` and
`StateNotFoundException`. They all are extended from `FlinkRuntimeException
GitHub user shixiaogang opened a pull request:
https://github.com/apache/flink/pull/3531
[FLINK-6034][checkpoint] Add KeyedStateHandle for the snapshots in keyed
streams
## Changes
- Add `KeyedStateHandle` for the snapshots in keyed streams.
`KeyGroupsStateHandle` now is one
GitHub user shixiaogang opened a pull request:
https://github.com/apache/flink/pull/3524
[FLINK-6014][checkpoint] Allow the registration of state objects in
checkpoints
- Introduce `CompositeStateHandle` which is composed of a collection of
`StateObject`s and can register these
GitHub user shixiaogang opened a pull request:
https://github.com/apache/flink/pull/3521
[FLINK-6027][checkpoint] Ignore the exception thrown by the subsuming of
completed checkppoints
The exception thrown during the subsuming of old checkpoints now will be
ignored. Now
GitHub user shixiaogang opened a pull request:
https://github.com/apache/flink/pull/3462
[FLINK-5917][state] Remove size() method from MapState
The `size()` method is removed from `MapState` because its implementation
is costly in the backends.
You can merge this pull request into
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3334#discussion_r103612613
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
---
@@ -428,6 +450,9 @@ CheckpointTriggerResult
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3334#discussion_r103605788
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
---
@@ -428,6 +450,9 @@ CheckpointTriggerResult
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3334#discussion_r103605271
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
---
@@ -537,12 +562,27 @@ else if
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3334#discussion_r103604470
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
---
@@ -121,6 +121,8 @@
/** The
Github user shixiaogang closed the pull request at:
https://github.com/apache/flink/pull/3336
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3380
I think we may borrow some ideas from Java. For example, the methods in
`Map` do not throw any exception in their signatures. But the interfaces define
a set of specific `RuntimeException` that
Github user shixiaogang closed the pull request at:
https://github.com/apache/flink/pull/3387
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3387
The PR is just some cleaning of the code. Now that we are planning to
refactor the implementation, I agree to close the PR and I am very willing to
contribute to the FLIP.
---
If your project
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3380
I like the idea of find some more "specific" exceptions. Flink can define
some specific Exceptions like `StateAccessException`. That may help better
understand the code.
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3336
@aljoscha Thanks a lot for your hard work. I have fixed the typos in the
documentation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
GitHub user shixiaogang opened a pull request:
https://github.com/apache/flink/pull/3387
[FLINK-5863][queryable state] Add the serialization of list states in
KvStateRequestSerializer
1. Add `serializeList()` in `KvStateRequestSerialization`
2. Modify the unit tests of
GitHub user shixiaogang opened a pull request:
https://github.com/apache/flink/pull/3380
[FLINK-5865][state] Throw original exception in the states
The wrapping of `RuntimeException` is removed so that we can avoid
redundant stack printed in the log.
You can merge this pull
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3336
@StefanRRichter Very thanks for your work. I have rebased the pull
request and resolved the conflicts.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3336#discussion_r102225285
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBMapState.java
---
@@ -382,11 +342,26
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3336
I have added the documentation for `MapState`. You may take a look to see
if it's properly written.
---
If your project is set up for it, you can reply to this email and have your
reply a
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3336
@StefanRRichter I have updated the pull request as suggested. Now the map
serializer supports the serialization of null values.
---
If your project is set up for it, you can reply to this email
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3336#discussion_r102153318
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBMapState.java
---
@@ -0,0 +1,579
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3336#discussion_r102138099
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/query/netty/message/KvStateRequestSerializer.java
---
@@ -484,6 +487,71 @@ public static
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3336#discussion_r102135289
--- Diff:
flink-runtime/src/test/java/org/apache/flink/runtime/query/netty/message/KvStateRequestSerializerTest.java
---
@@ -410,6 +415,124 @@ public
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3336#discussion_r102129362
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java
---
@@ -834,7
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3336#discussion_r102128355
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/DefaultKeyedStateStore.java
---
@@ -93,6 +95,18 @@ public DefaultKeyedStateStore
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3336#discussion_r102127867
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBMapState.java
---
@@ -0,0 +1,579
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3336#discussion_r102127767
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBMapState.java
---
@@ -0,0 +1,579
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3336#discussion_r102126863
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBMapState.java
---
@@ -0,0 +1,579
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3336#discussion_r102125445
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBMapState.java
---
@@ -0,0 +1,579
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3336#discussion_r102125062
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBMapState.java
---
@@ -0,0 +1,579
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3336#discussion_r101987352
--- Diff:
flink-core/src/main/java/org/apache/flink/api/common/state/MapState.java ---
@@ -0,0 +1,111 @@
+/*
+ * Licensed to the Apache Software
GitHub user shixiaogang opened a pull request:
https://github.com/apache/flink/pull/3359
[FLINK-5544][streaming] Add InternalTimerService implemented in RocksDB
- Refactor the methods defined in `InternalTimerService`. Some common
implementation in `HeapInternalTimerService` now is
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3336#discussion_r101936792
--- Diff:
flink-core/src/main/java/org/apache/flink/api/common/state/MapState.java ---
@@ -0,0 +1,111 @@
+/*
+ * Licensed to the Apache Software
GitHub user shixiaogang opened a pull request:
https://github.com/apache/flink/pull/3336
[FLINK-4856][state] Add MapState in KeyedState
1. Add `MapState` and `MapStateDescriptor`
2. Implementation of `MapState` in `HeapKeyedStateBackend` and
`RocksDBKeyedStateBackend`.
3
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3305
@tillrohrmann Both `org.apache.flink.api.common.typeutils.base` and
`"org.apache.flink.api.common.typeinfo` are in the module `flink-core`.
---
If your project is set up for it, you can
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3305
@StephanEwen @tillrohrmann I found a problem that is the packages
`ListTypeInfo` and `ListTypeSerializer` locate. Now `ListTypeInfo` is put in
package "org.apache.flink.api.java.typeutils
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3305
@tillrohrmann Thanks for your review.
Sorry for the reformatted code. It seems that my IDE will automatically
reformat all the files I've edited. I will revert the reformatted
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3305
@StephanEwen Thanks a lot for your comments. I have updated the code as
suggested.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user shixiaogang closed the pull request at:
https://github.com/apache/flink/pull/2768
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/2768
Close the pull request because the state descriptor now is refactored with
the introduction of composited serializers (See
[FLINK-5790](https://issues.apache.org/jira/browse/FLINK-5790
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3305#discussion_r100967108
--- Diff:
flink-core/src/main/java/org/apache/flink/migration/util/MigrationInstantiationUtil.java
---
@@ -47,9 +47,16 @@ public
GitHub user shixiaogang opened a pull request:
https://github.com/apache/flink/pull/3305
[FLINK-5790][StateBackend] Use list types when ListStateDescriptor extends
StateDescriptor
1. Now the state serializer, instead of the element serializer, is stored
in `ListStateDescriptor
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/2768
@aljoscha That way, it's very confusing that a `ReadableState` is not a
`State`. Hence I made `State` read-only and introduced the `UpdatableState`
interface who extends `State` with the m
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/2768
@StephanEwen Thanks a lot for your comments.
**Removing `clear()` from `State`**
This change is suggested by @aljoscha who wants to let broadcast states
share the same
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/2768
Despite the changes in the state descriptors, the Flink jobs can restore
from old versioned snapshots now.
---
If your project is set up for it, you can reply to this email and have your
reply
GitHub user shixiaogang opened a pull request:
https://github.com/apache/flink/pull/3053
[FLINK-5400] Add accessor to folding states in RuntimeContext
- Add accessors in RuntimeContext and KeyedStateStore
- Fix errors in the comments for reducing states in RuntimeContext and
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/2768
I rebased the branch to resolve the conflicts with the master branch.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/2768
I moved default value from `SimpleStateDescriptor` to
`ValueStateDescriptor`. Now only `ValueStateDescriptor`s have default values.
The serialization methods may contain some duplicated code
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/2768
Oh... I added another field to make the code more clear, but I did not
notice the serialization problem. Thanks very much for your reminder.
Your solution does work though the concept of
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/2768
@aljoscha Thanks for your review. I have updated the PR according to your
suggestion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
GitHub user shixiaogang opened a pull request:
https://github.com/apache/flink/pull/2768
[FLINK-5023 & FLINK-5024] Add SimpleStateDescriptor to clarify the concepts
Changes in the definition of `State` and `StateDescriptor`:
- Add `get()` in the `State` interface.
- Re
Github user shixiaogang closed the pull request at:
https://github.com/apache/flink/pull/2377
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/2377#discussion_r75481109
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/rpc/jobmaster/JobMaster.java
---
@@ -248,4 +374,40 @@ void
1 - 100 of 102 matches
Mail list logo
