The KIP-4 Create Topic Schema vote is close to passing so I would like to start the discussion for the DeleteTopic request/response and server side implementation.
Details for this implementation can be read here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946) I have included the exact content below for clarity: > Delete Topics Request (KAFKA-2946 > <https://issues.apache.org/jira/browse/KAFKA-2946>) > > > > DeleteTopics Request (Version: 0) => [topics] timeout > topics => STRING > timeout => INT32 > > DeleteTopicsRequest is a batch request to initiate topic deletion. > > Request semantics: > > 1. Must be sent to the controller broker > 2. If there are multiple instructions for the same topic in one > request the extra request will be ingnored > - This is because the list of topics is modeled server side as a set > - Multiple deletes results in the same end goal, so handling this > error for the user should be okay > 3. When requesting to delete a topic that does not exist, a an > InvalidTopic error will be returned for that topic. > 4. When requesting to delete a topic that is already marked for > deletion, the request will wait for the wait for the timeout and return as > usual. > - This is to avoid errors due to concurrent delete requests. The > end result is the same, the topic is deleted. > 5. The principal must be authorized to the "Delete" Operation on the > "Topic" resource to delete the topic. > - Unauthorized requests will receive a TopicAuthorizationException > 6. Setting a timeout > 0 will allow the request to block until the > delete is "complete" on the controller node. > - Complete means the local topic metadata cache no longer contains > the topic > - The topic metadata is updated when the controller sends out > update metadata requests to the brokers > - If a timeout error occurs, the topic could still be deleted > successfully at a later time. Its up to the client to query for the > state > at that point. > 7. Setting a timeout <= 0 will validate arguments and trigger the > delete topics and return immediately. > - This is essentially the fully asynchronous mode we have in the > Zookeeper tools today. > - The error code in the response will either contain an argument > validation exception or a timeout exception. If you receive a timeout > exception, because you asked for 0 timeout, you can assume the message > was > valid and the topic deletion was triggered. > 8. The request is not transactional. > 1. If an error occurs on one topic, the others could still be > deleted. > 2. Errors are reported independently. > > QA: > > - Why is DeleteTopicsRequest a batch request? > - Scenarios where tools or admins want to delete many topics should > be able to with fewer requests > - Example: Removing all cluster topics > - What happens if some topics error immediately? Will it > return immediately? > - The request will block until all topics have either been deleted, > errors, or the timeout has been hit > - There is no "short circuiting" where 1 error stops the other > topics from being deleted > - Why have a timeout at all? Deletes could take a while? > - True some deletes may take a while or never finish, however some > admin tools may want extended blocking regardless. > - If you don't want any blocking setting a timeout of 0 works. > - Future changes may make deletes much faster. See the Follow Up > Changes > > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-follow-up-changes> > section > above. > - Why implement "partial blocking" instead of fully async or fully > consistent? > - See Cluster Consistent Blocking > > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-cluster-consistent-blocking> > below > - Why require the request to go to the controller? > - The controller is responsible for the cluster metadata and > its propagation > - See Request Forwarding > > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-request> > below > > Delete Topics Response > > > > DeleteTopics Response (Version: 0) => [topic_error_codes] > topic_error_codes => topic error_code > topic => STRING > error_code => INT16 > > DeleteTopicsResponse is similar to CreateTopicsResponse. > A sample patch is on github ( https://github.com/granthenke/kafka/tree/delete-wire-new) though it could change drastically based on the feedback here. Note: This branch and patch is based on the CreateTopic request/response PR. I will open a PR once that patch is complete. Thank you, Grant -- Grant Henke Software Engineer | Cloudera gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
