Thank you for having a look at this. I agree that the only way to really
gauge load is to look at lag. But the connector tasks should not crash and
die because of load. I will raise this with SF.
On Wed, Jul 3, 2024 at 7:14 PM Greg Harris
wrote:
> Hey Burton,
>
> Thanks for your question and bug
Hey Burton,
Thanks for your question and bug report.
The exception you included does not indicate that your connectors are
overloaded. The primary way of diagnosing an overloaded connector is the
consumer lag metric, and if you're seeing acceptable lag, that should
indicate that your connectors a
Hi,
I have 100+ sink connectors running with 100+ topics each with roughly 3
partitions per topic. There are running on K8s on 10 pods with 6 cpus and
32Gig mem. The connector in question is Snowflake's sink connector v2.2.0.
This worked in the mini batch mode SNOWPIPE, but once i switched over to
