Hi,
My Flink (1.14.0) job seems to output a lot of error messages with the
following text:
16:46:38,562 WARN
org.apache.flink.connector.kafka.source.metrics.KafkaSourceReaderMetrics []
- Error when getting Kafka consumer metric "records-lag" for partition
"lambada.events-0". Metric "pendingBytes"
There is a setting for dynamic topic discovery
https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/connectors/table/kafka/#topic-and-partition-discovery
Best,
Denis
On Fri, Oct 15, 2021 at 12:48 AM Denis Nutiu wrote:
> Hi,
>
> In my experience with the librdkafka client a
r
>>> KafkaSource is package protected, and the KafkaSourceBuilder does not
>>> supply a way to provide the `KafkaSubscriber`.
>>>
>>> How can I accomplish a period refresh of the topics to ingest?
>>>
>>> Thanks
>>>
>>>
>>>
--
Regards,
Denis Nutiu
m:* Rommel Holmes
*Sent:* Saturday, October 9, 2021 02:09
*To:* Denis Nutiu
*Cc:* user
*Subject:* Re: Flink S3 Presto Checkpointing Permission Forbidden
You already have s3 request ID, you can easily reach out to AWS tech
support to know what account was used to write to S3. I guess th
TOKEN_FILE: /var/run/secrets/
eks.amazonaws.com/serviceaccount/token
Has anyone managed to deploy Flink with IAM access to S3 for checkpointing
on AWS? Could you please share some working flink-s3-fs-presto or
flink-s3-fs-hadoop plugin configuration with IAM authentication to S3?
--
Best,
Denis Nutiu
r mount it in a volume.
b) Yes, you should be able to.
[1]
https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/resource-providers/native_kubernetes/
--
Best,
Denis Nutiu
tart if a fault occurs.
But I'd like to hear some other opinions.
Thank you!
--
Regards,
Denis Nutiu
