[
https://issues.apache.org/jira/browse/KAFKA-554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jay Kreps updated KAFKA-554:
----------------------------
Attachment: KAFKA-554-v2.patch
New patch v2. Addresses Jun's comments and rebases.
1. TopicCommand: Made it a bit more clear which options apply to which actions.
2. AdminUtils:
2.1 Fixed typo
2.2 Moved serialization to JSON. The APIs still take Properties instances as
that is a higherarchical map. However the data stored in ZK is now JSON. One
question is what the format of the change notifications should be. I added a
proposal to the kafka zk wiki, please take a look.
3. TopicConfigManager:
3.1 add the missing > in the following comment.
* /brokers/topics/<topic_name/config
3.2 Yeah but this component should stand alone ideally.
3.3.1 Done
3.3.2 Good call, added that optimization.
3.3.3 I wasn't sure if deleting all znodes would lead to the sequence starting
over. I am told it won't so removed this.
3.3.4 "It seems that sequential nodes under /brokers/config_changes are only
deleted when there is a new config change. So, they are not always deleted
after the configured expiration time." Yes. My rational is that I want to clean
up just to prevent notifications from piling up forever. But if no new
notifications come in then they can't keep piling up. I could also do this by
scheduling the deletion but the problem with that is knowing whether you have
already scheduled it (you don't want to schedule it over and over). I could
also add a background thread dedicated to cleanup, but it doesn't seem worth it
for the added complexity.
3.4. Nice call, sorted it
4. Hmm, that doesn't actually compile for me.
5. Not sure that helps. Config is a contract so we can't change these values in
the future so just having the value is more straight-forward (one fewer layers
of indirection)
6. Removed ModifyTopicKey
7. ZkUtils: removed imports
8. Yeah, that slipped in. Removed.
9. ReplicaFetchTest.logsMatch(): renamed tandp it to topicAndPart?
> Move all per-topic configuration into ZK and add to the CreateTopicCommand
> --------------------------------------------------------------------------
>
> Key: KAFKA-554
> URL: https://issues.apache.org/jira/browse/KAFKA-554
> Project: Kafka
> Issue Type: New Feature
> Reporter: Jay Kreps
> Labels: project
> Fix For: 0.8.1
>
> Attachments: KAFKA-554-v1.patch, KAFKA-554-v2.patch
>
>
> We have a number of per-topic configurations that control message retention
> and flush interval. Here is the list of properties I find in KafkaConfig that
> appear to be per-topic:
> topic.log.file.size
> topic.log.roll.hours
> topic.log.retention.hours
> topic.log.retention.size
> topic.flush.intervals.ms
> Currently we specify these in server.properties. This is not a good solution
> as it requires a rolling bounce of the cluster to make a change, which just
> doesn't scale to having hundreds of topics. Also the map encoded in a CSV
> string is kind of hacky.
> We should move these into ZK in some kind of JSON blob that allows easily
> adding new per-topic configs and we should remove these from
> server.properties.
> It would be good to start with a wiki design and get consensus on that first.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
