nghao Chen
发件人: Krzysztof Chmielewski
发送时间: 2023年8月25日 17:17
收件人: Hang Ruan
抄送: liu ron ; user
主题: Re: How to use pipeline.jobvertex-parallelism-overrides property.
Hi,
thank you for replaying.
Hang Ruan, regarding:
" If they have different parallelisms, we cannot chain them togethe
Hi,
thank you for replaying.
Hang Ruan, regarding:
" If they have different parallelisms, we cannot chain them together." <-
that is clear for me.
What I'm still not sure about is if " JobVertexId#toHexString" that should
be used as key of pipeline.jobvertex-parallelism-overrides map is vertex
o
Hi, Krzysztof.
As liu ron said, the key of the map for this configuration is the value
from JobVertexId#toHexString. Maybe we could improve the docs to provide
more details.
The condition that two operators have the same parallelism is a must for
chaining them. If they have different parallelisms,
Hi, Krzysztof
As stated in the description section, this option is used to override the
parallelism of a JobVertex, where the key is JobVertex id, you can see [1]
for double check. A JobVertex may contain more than one operator, so we
cannot override the parallelism of a given operator alone. One
Hi,
have someone used pipeline.jobvertex-parallelism-overrides [1] property?
I wonder what actually should be a key here? Operator name?
What if my operators are chained and I want to override only one of its
elements. For example Source -> (Map1 chained with Map2) -> Sink. Can I
override Map2 on
