sjwiesman commented on a change in pull request #14981:
URL: https://github.com/apache/flink/pull/14981#discussion_r580593585
##########
File path: docs/content/docs/ops/state/savepoints.md
##########
@@ -46,9 +46,9 @@ In contrast to all this, Savepoints are created, owned, and
deleted by the user.
changing parallelism, forking a second job like for a red/blue deployment, and
so on. Of course, Savepoints must survive job termination. Conceptually,
Savepoints can be a bit more expensive to produce and restore and focus
more on portability and support for the previously mentioned changes to the
job.
-Those conceptual differences aside, the current implementations of Checkpoints
and Savepoints are basically using the same code and produce the same format.
However, there is currently one exception from this, and we might
-introduce more differences in the future. The exception is incremental
checkpoints with the RocksDB state backend. They are using some RocksDB
internal format instead of Flink’s native savepoint format. This makes them the
-first instance of a more lightweight checkpointing mechanism, compared to
Savepoints.
+In Flink 1.13 we unified the binary format of savepoints across all state
backends. That means you can take a savepoint and then restore from it using a
different state backend.
+All the state backends produce a common format only starting from the version
1.13. You cannot do that with checkpoints. It means the savepoint binary format
might be different from the one used for checkpoints. This is the case e.g. for
Heap state backend or
+if you use incremental checkpoints with the RocksDB state backend. The RocksDB
full snapshot share the format with unified savepoints, but it is not
guaranteed, and we might introduce further modifications in the future.
Review comment:
```suggestion
Flink's savepoint binary format is unified across all state backends. That
means you can take a savepoint with one state backend and then restore it using
another.
{{< hint info >}}
State backends did not start producing a common format until version 1.13.
Therefore, if you want to switch the state backend you should first upgrade
your Flink version than
take a savepoint with the new version, and only after that, you can restore
it with a different state backend.
{{< /hint >}}
```
##########
File path: docs/content/docs/ops/state/state_backends.md
##########
@@ -136,15 +136,16 @@ The total memory amount of RocksDB instance(s) per slot
can also be bounded, ple
# Choose The Right State Backend
-Currently, Flink's savepoint binary format is state backend specific.
-A savepoint taken with one state backend cannot be restored using another, and
you should carefully consider which backend you use before going to production.
-
In general, we recommend avoiding `MemoryStateBackend` in production because
it stores its snapshots inside the JobManager as opposed to persistent disk.
When deciding between `FsStateBackend` and `RocksDB`, it is a choice between
performance and scalability.
`FsStateBackend` is very fast as each state access and update operates on
objects on the Java heap; however, state size is limited by available memory
within the cluster.
On the other hand, `RocksDB` can scale based on available disk space and is
the only state backend to support incremental snapshots.
However, each state access and update requires (de-)serialization and
potentially reading from disk which leads to average performance that is an
order of magnitude slower than the memory state backends.
+In Flink 1.13 we unified the binary format of Flink's savepoints. That means
you can take a savepoint and then restore from it using a different state
backend.
+All the state backends produce a common format only starting from the version
1.13. Therefore, if you want to switch the state backend you should first
upgrade your Flink version than
+take a savepoint with the new version and only after that you can restore it
with a different state backend.
+
Review comment:
```suggestion
{{< hint info >}}
In Flink 1.13 we unified the binary format of Flink's savepoints. That means
you can take a savepoint and then restore from it using a different state
backend.
All the state backends produce a common format only starting from version
1.13. Therefore, if you want to switch the state backend you should first
upgrade your Flink version than
take a savepoint with the new version, and only after that you can restore
it with a different state backend.
{{< /hint >}}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
