Hi,
if you use the MemoryStateBackend, the state is shipped from the TMs to the
JM. The JM holds the state in its own memory (i.e., on the JVM heap).
This means that the MemoryStateBackend is not reliable by design and rather
meant for testing and running Flink in your IDE.
If you want to persist
Hi Team,
If the flink cluster is conainerized and managed through by a container
orchestrator,
1. the orchestrator allocates resources for each JM. TM etc., say if
the container (TM) needs to run with 2G RAM, how should this allocation be
honoured by the TM when its JVM starts. I'm thinki
Hi Flink users,
I try to cancel a job with savepoint via REST based on
https://issues.apache.org/jira/browse/FLINK-4787
I can call jobs/:jobid/cancel-with-savepoint/ in our 1.2.0 Flink cluster,
but in my test environment which is done by using:
org.apache.flink
flink-run
Hi Flink users,
When using `file:///mysavepoint` instead of `hdsf://mysavepoint for
ENVCONFIG.setString("state.checkpoints.dir", stateBackend);
savepoint will be transfered from task manager to job manager (same as using
memory as state backend), and stored in the disk of job manager?
Savepoint
I have prepared a small dummy dataset (data.txt) as follows:
Hello|5
Hi|15
WordsWithoutMeaning|25
AnotherWord|34
HelloWorld|46
HelloPlanet|67
HelloFlinkUsers|89
HelloProgrammers|98
DummyPhrase|105
AnotherDummy|123
And below is the code:
import org.apache.flink.api.java.io.TextInputFormat
import
We are currently looking into how we can keep the size of the code base
under control (because it is growing super large).
Part is moving libraries into a dedicated subrepository (there is a
separate mailing list thread on that) and some connectors to Bahir.
Connectors can move between Flink and B
Hi,
Another thought I had last night, maybe we could have another state for
recovering jobs in the future.
Deploying -> Recovering -> Running
This recovering state might only be applicable for state backends that have
to be restored before processing can start, lazy state backends (like
external d
No, we do not want to move all connectors to Apache Bahir or replace the
connectors by Bahir.
The Flink community aims to maintain the most important connectors within
Flink.
Maintaining all connectors would be a huge effort. So, we decided to move
some of the less frequently used connectors to Ba
Hi,
I was able to come up with a custom build of RocksDB yesterday that seems to
fix the problems. I still have to build the native code for different platforms
and then test it. I cannot make promises about the 1.2.1 release, but I would
be optimistic that this will make it in.
Best,
Stefan
Thank you Stephan for spotting the problem. In hindsight it’s obvious that this
can never work. I’ll figure something out :)
> Am 24.03.2017 um 10:28 schrieb Stephan Ewen :
>
> The code will not work properly, sorry.
>
> The value returned by the state is whatever is stored under the key for wh
10 matches
Mail list logo
