The client was not using a config file, it is a stand-alone java app using the
flink-client jar file. Thanks for the clarification.
Michael
> On Apr 16, 2018, at 2:11 PM, Fabian Hueske wrote:
>
> The parallelism.default property that is configured in the flink-conf.yaml
> file is only consid
The parallelism.default property that is configured in the flink-conf.yaml
file is only considered if the config file belongs to the submitting client.
If you configured the property in the config file of your cluster setup and
used submitted from a client that used a different configuration file,
1.4.2. I have since set the parallelism explicitly after creating the env and
that is working. I also made the stream object serializable which may also be
involved in this. I will retest without the explicit parallelism when I get a
chance.
Michael
> On Apr 16, 2018, at 2:05 AM, Fabian Hue
(re-adding user mailing list)
A non-serializable function object should cause the job to fail, but not to
ignore a parallelism setting.
This might be a bug. Most users specify the parallelism directly in the
application code (via StreamExecutionEnvironment) or when submitting the
application.
Whi
I am pretty new to flink. I have a flink job that has 10 transforms (mostly
CoFlatMap with some simple filters and key extractrs as well. I have the
config set for 6 slots and default parallelism of 6, but all my stages show
paralellism of 1. Is that because there is only one task manager? S
g
> wrote:
> > Hello all,
> >
> > I have a question about parallelism and partitioning in the
> > DataStreams API. In Flink, a user can the parallelism of a data
> > source as well as operators. So when I set the parallelism of a data
> > source e.g.
I guess my previous question is also asking if the parallelism is set
for the operator or "data stream". Is there implied repartitioning
when the parallelism changes?
On Fri, Aug 18, 2017 at 2:08 PM, Jerry Peng wrote:
> Hello all,
>
> I have a question about parallelism and
Hello all,
I have a question about parallelism and partitioning in the
DataStreams API. In Flink, a user can the parallelism of a data
source as well as operators. So when I set the parallelism of a data
source e.g.
DataStream text =
env.readTextFile(params.get("input")).setPar
