Scylla connector

Hi, i am new to Flink. Is there scylla connector equivalent to the cassandra connector: https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/connectors/cassandra.html? Or can Flink use Scylla as a sink via the cassandra connector? Thanks.

Re: Why Job Manager die/restarted when Task Manager die/restarted?

Another possibility is the JM is killed externally, e.g. K8s may kill JM/TM if it exceeds the resource limit. Thanks, Zhu Zhu Zhu Zhu 于2019年8月12日周一 下午1:45写道： > Hi Cam, > > Flink master should not die when getting disconnected with task managers. > It may exit for cases below: > 1. when the job

Re: Why Job Manager die/restarted when Task Manager die/restarted?

Hi Cam, Flink master should not die when getting disconnected with task managers. It may exit for cases below: 1. when the job terminated(FINISHED/FAILED/CANCELED). If you job is configured with no restart retry, a TM failure can cause the job to be FAILED. 2. JM lost HA leadership, e.g. lost conn

Re: Why available task slots are not leveraged for pipeline?

Hi Cam, This case is expected due to slot sharing. A slot can be shared by one instance of different tasks. So the used slot is count of your max parallelism of a task. You can specify the shared group with slotSharingGroup(String slotSharingGroup) on operators. Thanks, Zhu Zhu Abhishek Jain 于20

Re: Why available task slots are not leveraged for pipeline?

What you'se seeing is likely operator chaining. This is the default behaviour of grouping sub tasks to avoid transer overhead (from one slot to another). You can disable chaining if you need to. Please refer task and operator chains

Re: Kafka ProducerFencedException after checkpointing

Hi, I had the same exception recently. I want to confirm that if it is due to transaction timeout, then I will lose those data. Am I right? Can I make it fall back to at least once semantic in this situation? Best, Tony Wei Piotr Nowojski 於 2018年3月21日 週三 下午10:28寫道： > Hi, > > But that’s exactly

Why available task slots are not leveraged for pipeline?

Hello Flink expert, I have a cluster with 10 Task Managers, configured with 6 task slot each, and a pipeline that has 13 tasks/operators with parallelism of 5. But when running the pipeline I observer that only 5 slots are being used, the other 55 slots are available/free. It should use all of my

Why Job Manager die/restarted when Task Manager die/restarted?

Hello Flink experts, We are running Flink under Kubernetes and see that Job Manager die/restarted whenever Task Manager die/restarted or couldn't get connected each other. Is there any specific configurations/parameters that we need to turn on to stop this? Or this is expected? Thanks, Cam

[ANNOUNCE] Weekly Community Update 2019/32

Dear community, happy to share this week's community update. The first real release candidate of Apache Flink 1.9.0 has been published and the developer community has moved more towards the 1.10 development cycle with more discussion threads popping up again. As always, please feel free to add ad

Apache flink 1.7.2 security issues

Hello, We are using Apache Flink 1.7.2 version. During our security scans following issues are reported by our scan tool. Please let us know your comments on these issues. [1] 150085 Slow HTTP POST vulnerability Severity Potential Vulnerability - Level 3 Group Information Disclosure Threat The

Re: How many task managers can Flink efficiently scale to?

Hi Chad, We have (Blink) jobs each running with over 10 thousands of TMs. In our experience, the main regression caused by large scale TMs is the in TM allocation stage in ResourceManager, that some times it fails to allocate enough TMs before the allocation timeout. It does not deteriorate much o

Re: How many task managers can Flink efficiently scale to?

Hi Chad, In our cases, 1~2k TMs with up to ~10k TM slots are used in one job. In general, the CPU/memory of Job Manager should be increased with more TMs. Regards, Qi > On Aug 11, 2019, at 2:03 AM, Chad Dombrova wrote: > > Hi, > I'm still on my task management investigation, and I'm curious t