@vino Please take a deeper look at the question before answering them.
The benchmark implements custom operators that make use of several
non-Public APIs,
so API backwards compatibility simply doesn't apply here.
Updating the benchmark will require a bit of work. Below are some hints
i gathered, but given that we're jumping
5 versions at once (and 1.0 being really old) I can't guarantee anything
about the correctness.
The replacement for StateBackend is StateBinder. (apparently)
The OperatorState usage should be replaced with a ValueState usage.
The code in QueryableWindowOperatorEvicting#snapshotOperatorState can be
ported to a keyed ValueState,
have a look at the WindowOperator#open().
The StateHandle/StreamTaskState/CheckpointStateOutputView stuff should
become irrelevant after porting the operator to 1.5.
On 11.08.2018 04:14, vino yang wrote:
Hi Namu,
I don't think you need to pay attention to the internals of the Flink
API.
Its interface is backward compatible. If you update the dependent
version of the API and the corresponding version of the Flink system
so that their versions are consistent, there should be no problems.
Please ensure that the version of Flink is consistent with the version
of the API you are using.
Thanks, vino.
Naum Gjorgjeski <naum...@hotmail.com <mailto:naum...@hotmail.com>>
于2018年8月11日周六 上午4:25写道：
Hi all,
I still cannot resolve the problem. Can I please get any advice
for it? Thank you.
Best regards,
Naum Gjorgjeski
------------------------------------------------------------------------
*From:* Naum Gjorgjeski <naum...@hotmail.com
<mailto:naum...@hotmail.com>>
*Sent:* Wednesday, August 1, 2018 1:05 AM
*To:* user@flink.apache.org <mailto:user@flink.apache.org>
*Subject:* Yahoo Streaming Benchmark on a Flink 1.5 cluster
Hi,
I am trying to run the data Artisans version of the Yahoo
Streaming Benchmark. The benchmark applications are written for
Flink 1.0.1. However, I need them to run on a Flink 1.5 cluster.
When I try to build the benchmark applications with any version
of Flink from 1.3.0 or higher, I get many compile errors. The
compile errors state that some of the classes and methods cannot
be found (because part of the Flink API has changed in recent
versions).
The classes that cannot be found are:
org.apache.flink.streaming.connectors.kafka.partitioner.FixedPartitioner
org.apache.flink.api.common.state.OperatorState
org.apache.flink.api.common.state.StateBackend
org.apache.flink.runtime.state.AsynchronousStateHandle
org.apache.flink.runtime.state.StateHandle
org.apache.flink.streaming.runtime.tasks.StreamTaskState
org.apache.flink.runtime.state.AbstractStateBackend.CheckpointStateOutputView
The methods that cannot be found are:
org.apache.flink.streaming.api.operators.StreamOperator.snapshotOperatorState(long,long)
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getStateBackend()
org.apache.flink.runtime.state.AbstractStateBackend.createCheckpointStateOutputView(long,long)
I was able to find a substitution for few of them (e.g. using
FlinkFixedPartitioner instead of FlinkPartitioner), but for most
of them there are no straightforward substitutions. Could you
please give me an advice on how to resolve this problem? Thank
you in advance.
Best regards,
Naum
