Re: Questions about Apache Kafka

2016-08-24 Thread Marko Bonaći
Hi Karin, regarding 5 (fsyncing to disk), take a look at the broker configuration parameters whose names start with log.flush. http://kafka.apache.org/documentation.html#brokerconfigs Marko Bonaći Monitoring | Alerting | Anomaly Detection | Centralized Log Management Solr & Elasticsearch Support

Re: Questions about Apache Kafka

2016-08-24 Thread Ian Wrigley
Actually, for 3 and 4 Kafka Streams is also a very good option — and comes as part of the standard Apache Kafka distribution. Ian. --- Ian Wrigley Director, Education Services Confluent, Inc > On Aug 24, 2016, at 2:37 PM, David Garcia wrote: > > Regarding 3 and 4: https://calcite.apache.org/d

Re: Questions about Apache Kafka

2016-08-24 Thread David Garcia
Regarding 3 and 4: https://calcite.apache.org/docs/stream.html (i.e. streaming SQL queries) On 8/24/16, 6:29 AM, "Herwerth Karin (CI/OSW3)" wrote: Dear Sir or Madam, I'm a beginner in Apache Kafka and have questions which I don't get it from the documentation. 1.

Re: Questions about Apache Kafka

2016-08-24 Thread Radoslaw Gruchalski
Hi Karin, 1. Kafka does not support TTL for individual messages, only the topic. From the documentation: The Kafka cluster retains all published messages—whether or not they have been consumed—for a configurable period of time. For example if the log retention is set to two days, then for the two