Parallelism under reactive scaling with slot sharing groups

2023-07-24 Thread Allen Wang
Hello, Our job has operators of source -> sink -> global committer. We have created two slot sharing groups, one for source and sink and one for global committer. The global committer has specified max parallelism of 1. No max parallelism set with the source/sink while there is a system level defa

About[jobmanager.memory.off-heap.size] not work on Flink(flink-session on yarn)

2023-07-24 Thread 傅文江
When I use ESSink, I find that I need to set Flink’s jobmanager.memory.off-heap.size to 256MB. The default 128MB does not meet the running requirements. It can be successful by modifying the configuration file, but I want to dynamically adjust the jobmanager.memory.off-heap.size through the c

Re: kafka sink

2023-07-24 Thread nick toker
Hi 1. we use flink 1.17.1 2. there is no traffic and data in the topics when it's happened 3 log part with the error: 2023-07-24 08:41:30,334 DEBUG org.apache.flink.connector.kafka.sink. FlinkKafkaInternalProducer [] - commitTransaction telephony-decoder- ORCHESTRATOR-MULTIMEDIA-IN-8-10 2023-07-