[
https://issues.apache.org/jira/browse/FLINK-7367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16121139#comment-16121139
]
ASF GitHub Bot commented on FLINK-7367:
---------------------------------------
Github user tzulitai commented on a diff in the pull request:
https://github.com/apache/flink/pull/4473#discussion_r132373023
--- Diff:
flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/config/ProducerConfigConstants.java
---
@@ -24,10 +24,28 @@
*/
public class ProducerConfigConstants extends AWSConfigConstants {
+ /** Deprecated key. **/
+ public static final String DEPRECATED_COLLECTION_MAX_COUNT =
"aws.producer.collectionMaxCount";
+
+ /** Deprecated key. **/
+ public static final String DEPRECATED_AGGREGATION_MAX_COUNT =
"aws.producer.aggregationMaxCount";
+
/** Maximum number of items to pack into an PutRecords request. **/
- public static final String COLLECTION_MAX_COUNT =
"aws.producer.collectionMaxCount";
+ public static final String COLLECTION_MAX_COUNT = "CollectionMaxCount";
--- End diff --
Do we really want to expose these config keys now?
AFAIK, if the user wants to tweak these, with this change they should just
simply refer to the KPL docs to see what keys are available. Explicitly
exposing partially some keys in Flink but not others is a bit weird, IMO.
> Parameterize more configs for FlinkKinesisProducer (RecordMaxBufferedTime,
> MaxConnections, RequestTimeout, etc)
> ---------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-7367
> URL: https://issues.apache.org/jira/browse/FLINK-7367
> Project: Flink
> Issue Type: Improvement
> Components: Kinesis Connector
> Affects Versions: 1.3.0
> Reporter: Bowen Li
> Assignee: Bowen Li
> Fix For: 1.3.3
>
>
> Right now, FlinkKinesisProducer only expose two configs for the underlying
> KinesisProducer:
> - AGGREGATION_MAX_COUNT
> - COLLECTION_MAX_COUNT
> Well, according to [AWS
> doc|http://docs.aws.amazon.com/streams/latest/dev/kinesis-kpl-config.html]
> and [their sample on
> github|https://github.com/awslabs/amazon-kinesis-producer/blob/master/java/amazon-kinesis-producer-sample/default_config.properties],
> developers can set more to make the max use of KinesisProducer, and make it
> fault-tolerant (e.g. by increasing timeout).
> I select a few more configs that we need when using Flink with Kinesis:
> - MAX_CONNECTIONS
> - RATE_LIMIT
> - RECORD_MAX_BUFFERED_TIME
> - RECORD_TIME_TO_LIVE
> - REQUEST_TIMEOUT
> We need to parameterize FlinkKinesisProducer to pass in the above params, in
> order to cater to our need
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
