To ensure if messages are properly deserialized, you can look into writing
your own "formatter" for the ConsoleConsumer. Similarly, you might have to
wire in your own "line-reader" for the ConsoleProducer.
Thanks,
Neha
On Sat, Jan 12, 2013 at 11:12 AM, Ron Tsoref wrote:
> I have been experimen
I have been experimenting with Kafka for the last hour or so and it seems
like using a custom *tail* command with the producer is sending the written
lines of logs to Kafka.
However, there's no a clear separation between the messages that
are received by the ConsoleConsumer, so I can't be sure tha
Ron,
The best way of doing this would be to use the ConsoleProducer. Basically,
it reads data from the console and parses it using the message "reader"
which by default is the LineReader. In this case, you can either write your
own SquidMessageReader that understands the Squid access format [1] an
The following wiki describes the operational part of Kafka.
https://cwiki.apache.org/confluence/display/KAFKA/Operations
To get your log into Kafka, if this log4j data, you may consider adding a
KafkaLog4jAppender. Otherwise, you can probably use ConsoleProducer. You
will still need to deal with t
