Harsha, I understand the concern, but the solution of using this flag is a bit weird.
First, I can cause similar damage by setting retention.ms=1 on any topic. We also have APIs to delete data from topics directly. All without a "gating" flag. Because as a system we rely on access controls to make sure unauthorized users can't cause excessive damage. Second, I can think of no other system with a global flag for enabling/disabling deletes - no OS does that (deleting files from Linux is also a potential disaster) and no database. The fact that Kafka is rather unique in this makes the flag and its default seem like a trap for new users and an endless source of confusion. Keeping a feature flag around for deletion seems to me to have more downsides than upsides. If we want to add soft deletes to Kafka, we can discuss that proposal - at least it is something users will be familiar with. On Wed, Nov 25, 2020 at 11:54 AM Harsha Ch <harsha...@gmail.com> wrote: > > Hi Gwen, > Agree with you ACLs would be a better option. But not > all installations will be able to have secure setup and authorizer. > In a hosted service where they may not be ACLs or security setup this flag > helps prevent accidental topic deletion. > This config acted as a good gatekeeper to prevent accidents in deletion as > there is no confirmation or there is time delay in execution of the > deletion of a topic or a soft delete etc... > It's pretty much a catastrophic event in case of accidental deletion. We > also need to consider that not all installations will be secure and we need > to > provide a simpler way to gate keep certain functionalities such as delete > topic. > > Thanks, > Harsha > > > On Wed, Nov 25, 2020 at 10:25 AM Gwen Shapira <g...@confluent.io> wrote: > > > Thanks for bringing this up! > > > > I disagree that " there is a valid use-case to keep the flag > > "delete.topic.enable" as "false" in server.properties configs, so that > > someone doesn't accidentally delete a topic,". > > The flag was added and set to "false" back in the days when topic > > deletion was not a stable feature, and enabling it meant a risk of > > encountering serious bugs. Things changed in the last 4-5 years: > > - Topic deletion is now a stable and reliable feature. We have it > > enabled by default in Confluent Cloud and over a few years and > > hundreds of clusters, there was maybe one issue related to deletion of > > large number of partitions, and that was fixed. > > - We have authorization and ACLs, so the permissions to delete topics > > can be granted on an individual basis. This is the correct way to > > handle concerns about someone accidentally deleting data valuable > > data. Note that ACLs are dynamic. > > > > IMO, the best solution is to use the upcoming major release (3.0) to > > enable topic deletion by default and remove the flag entirely. > > > > Gwen > > > > On Wed, Nov 25, 2020 at 9:36 AM Prateek Agarwal <prat0...@gmail.com> > > wrote: > > > > > > Hi, > > > > > > I've created KIP-688 to enable support for dynamic update of > > > delete.topic.enable config. > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-688%3A+Support+dynamic+update+of+delete.topic.enable+config > > > > > > Please take a look and let me know if you have any feedback. > > > > > > Thanks > > > -- > > > Prateek Agarwal > > > > > > > > -- > > Gwen Shapira > > Engineering Manager | Confluent > > 650.450.2760 | @gwenshap > > Follow us: Twitter | blog > > -- Gwen Shapira Engineering Manager | Confluent 650.450.2760 | @gwenshap Follow us: Twitter | blog
