+1 (binding) On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke <ghe...@cloudera.com> wrote:
> I would like to initiate the voting process for the "KIP-4 Create Topics > Schema changes". This is not a vote for all of KIP-4, but specifically for > the create topics changes. I have included the exact changes below for > clarity: > > > > Create Topics Request (KAFKA-2945 > > <https://issues.apache.org/jira/browse/KAFKA-2945>) > > > > CreateTopics Request (Version: 0) => [create_topic_requests] timeout > > create_topic_requests => topic num_partitions replication_factor > [replica_assignment] [configs] > > topic => STRING > > num_partitions => INT32 > > replication_factor => INT16 > > replica_assignment => partition_id [replicas] > > partition_id => INT32 > > replicas => INT32 > > configs => config_key config_value > > config_key => STRING > > config_value => STRING > > timeout => INT32 > > > > CreateTopicsRequest is a batch request to initiate topic creation with > > either predefined or automatic replica assignment and optionally topic > > configuration. > > > > Request semantics: > > > > 1. Must be sent to the controller broker > > 2. If there are multiple instructions for the same topic in one > > request an InvalidRequestException will be logged on the broker and > the > > client will be disconnected. > > - This is because the list of topics is modeled server side as a > > map with TopicName as the key > > 3. The principal must be authorized to the "Create" Operation on the > > "Cluster" resource to create topics. > > - Unauthorized requests will receive a > ClusterAuthorizationException > > 4. > > > > Only one from ReplicaAssignment or (num_partitions + > replication_factor > > ), can be defined in one instruction. > > - If both parameters are specified an InvalidRequestException will be > > logged on the broker and the client will be disconnected. > > - In the case ReplicaAssignment is defined number of partitions and > > replicas will be calculated from the supplied replica_assignment. > > - In the case of defined (num_partitions + replication_factor) > > replica assignment will be automatically generated by the server. > > - One or the other must be defined. The existing broker side auto > > create defaults will not be used > > (default.replication.factor, num.partitions). The client > implementation can > > have defaults for these options when generating the messages. > > - The first replica in [replicas] is assumed to be the preferred > > leader. This matches current behavior elsewhere. > > 5. Setting a timeout > 0 will allow the request to block until the > > topic metadata is "complete" on the controller node. > > - Complete means the local topic metadata cache been completely > > populated and all partitions have leaders > > - The topic metadata is updated when the controller sends out > > update metadata requests to the brokers > > - If a timeout error occurs, the topic could still be created > > successfully at a later time. Its up to the client to query for > the state > > at that point. > > 6. Setting a timeout <= 0 will validate arguments and trigger the > > create topics and return immediately. > > - This is essentially the fully asynchronous mode we have in the > > Zookeeper tools today. > > - The error code in the response will either contain an argument > > validation exception or a timeout exception. If you receive a > timeout > > exception, because you asked for 0 timeout, you can assume the > message was > > valid and the topic creation was triggered. > > 7. The request is not transactional. > > 1. If an error occurs on one topic, the others could still be > > created. > > 2. Errors are reported independently. > > > > QA: > > > > - Why is CreateTopicsRequest a batch request? > > - Scenarios where tools or admins want to create many topics should > > be able to with fewer requests > > - Example: MirrorMaker may want to create the topics downstream > > - What happens if some topics error immediately? Will it > > return immediately? > > - The request will block until all topics have either been created, > > errors, or the timeout has been hit > > - There is no "short circuiting" where 1 error stops the other > > topics from being created > > - Why implement "partial blocking" instead of fully async or fully > > consistent? > > - See Cluster Consistent Blocking > > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-cluster-consistent-blocking > > > > below > > - Why require the request to go to the controller? > > - The controller is responsible for the cluster metadata and its > > propagation > > - See Request Forwarding > > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-request > > > > below > > > > Create Topics Response > > > > > > > > CreateTopics Response (Version: 0) => [topic_error_codes] > > topic_error_codes => topic error_code > > topic => STRING > > error_code => INT16 > > > > CreateTopicsResponse contains a map between topic and topic creation > > result error code (see New Protocol Errors > > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-NewProtocolErrors > > > > ). > > > > The KIP is available here for reference (linked to the Create Topics schema > section): > * > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-CreateTopicsRequest(KAFKA-2945) > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-CreateTopicsRequest(KAFKA-2945) > >* > > A pull request is available implementing the proposed changes here: > https://github.com/apache/kafka/pull/1489 > > Here is a link to the past discussion on the mailing list: > * > http://search-hadoop.com/m/uyzND1rfG6v1oixmZ&subj=+DISCUSS+KIP+4+Create+Topic+Schema > < > http://search-hadoop.com/m/uyzND1rfG6v1oixmZ&subj=+DISCUSS+KIP+4+Create+Topic+Schema > >* > > Thank you, > Grant > -- > Grant Henke > Software Engineer | Cloudera > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke >
