Hi,
I've installed kafka 2.8.1,
created a topic using:
/opt/kafka/bin/kafka-topics.sh --create --topic "test" --zookeeper
"localhost:2381" --partitions 2 --replication-factor 2
Then opened a console producer and a console consumer.
I type a few lines on the producer and then the two kafka broker
Hi,
I have a consumer group with multiple threads (high-level consumers) which
read from a topic.
I am also using a SimpleConsumer to read messages given a start offset. I
am getting the offset as the last produced message using the below code.
How to get the last un-consumed message?
public
Sorry. I want the first un-consumed message offset.
On Tue, Jun 17, 2014 at 2:53 PM, Achanta Vamsi Subhash <
achanta.va...@flipkart.com> wrote:
> Hi,
>
> I have a consumer group with multiple threads (high-level consumers) which
> read from a topic.
>
> I am also using a SimpleConsumer to read m
It got stuck on this:
./gradlew jar
The TaskContainer.add() method has been deprecated and is scheduled to be
removed in Gradle 2.0. Please use the create() method instead.
Building project 'core' with Scala version 2.8.0
Building project 'perf' with Scala version 2.8.0
:clients:compileJava
:clie
Hi,
I was wondering if Kafka has been tested/certified under JRE 1.8?
Thanks!
Chris
We¹re about to start testing in the next month. As soon as I have 1.8 u5
on the systems to work with. I don¹t anticipate too much of an issue,
since 7 -> 8 is supposed to be less trouble than 6 -> 7, and we didn¹t
have any problems moving to 7.
-Todd
On 6/17/14, 6:22 AM, "Chris Neal" wrote:
>H
Perfect. Thanks very much!
Chris
On Tue, Jun 17, 2014 at 8:28 AM, Todd Palino
wrote:
> We¹re about to start testing in the next month. As soon as I have 1.8 u5
> on the systems to work with. I don¹t anticipate too much of an issue,
> since 7 -> 8 is supposed to be less trouble than 6 -> 7, an
Is your network stable?
Thanks,
Jun
On Tue, Jun 17, 2014 at 1:48 AM, Gerrit Jansen van Vuuren <
gerrit...@gmail.com> wrote:
> Hi,
>
> I've installed kafka 2.8.1,
> created a topic using:
>
> /opt/kafka/bin/kafka-topics.sh --create --topic "test" --zookeeper
> "localhost:2381" --partitions 2 --
Since the consumer returns the offset of each message, you can manage the
offset commit yourself in the application.
Thanks,
Jun
On Tue, Jun 17, 2014 at 2:24 AM, Achanta Vamsi Subhash <
achanta.va...@flipkart.com> wrote:
> Sorry. I want the first un-consumed message offset.
>
>
> On Tue, Jun 1
0.8.1 uses gradle for building jars. You can follow the README file in the
source code on how to use gradle.
Thanks,
Jun
On Tue, Jun 17, 2014 at 1:49 AM, 凌德聪 wrote:
> I want to build hadoop-consumer, the i follow this
> https://github.com/kafka-dev/kafka/tree/master/contrib/hadoop-consumer
>
The network is 10gig and so far has not given any issues I think its
extremely unlikely that it could be network (all ports are open and all
communication happens on an internal lan).
I'm running consumers and producers on the nodes where the brokers are
running and they are consuming and producin
Need help to run kafka 0.8.1 on windows 7. Currently kafka 0.8.1 can't be
run on windows using command prompt.
I have gone through below post which have steps to run kafka 0.8.0.
http://hocke.blogspot.in/2014/01/kafka-080-on-windows.html
But creating Topic and topic list commands are not support
Hi Shaikh,
I heard some throughput bottleneck of storm. It cannot really scale up with
kafka.
I recommend you to try DataTorrent platform(https://www.datatorrent.com/)
The platform itself is not open-source but it has a open-source library (
https://github.com/DataTorrent/Malhar) which contains a
we have been experimenting with Samza which is also worth a look. It's
basically a topic-to-topic node on Yarn.
On Jun 17, 2014, at 10:44 AM, hsy...@gmail.com wrote:
> Hi Shaikh,
>
> I heard some throughput bottleneck of storm. It cannot really scale up with
> kafka.
> I recommend you to try
My app is in scala and a quick search on serializing a scala class seems to
have potential issues with different versions of scala (I could be wrong as
I did a quick search).
Is it generally just a better idea to use plain old java classes for kafka
messages?
i.e. I simply use jackson like:
publ
Samza is an open source stream processing framework built on top of Kafka
and YARN. It is high throughput, scalable and has in built state management
and fault tolerance support. Though I may be biased, it is worth taking a
look :-)
Thanks,
Neha
On Tue, Jun 17, 2014 at 10:55 AM, Robert Rodgers
Yes, backwards incompatibility is a problem when using Java/Scala object
serialization. At LinkedIn, we use Avro as the serialization format for
Kafka data.
Thanks,
Neha
On Tue, Jun 17, 2014 at 11:58 AM, S Ahmed wrote:
> My app is in scala and a quick search on serializing a scala class seems
1. do you use a single producer for all message types/topics?
Not really. It is typical for a java service/process to have multiple
producers sending to different topics
2. For your pageview topic i.e. it is getting sent on a per page request
(albiet it is batched):
*What happens when your kafka
I almost got it to work but still stuck compiling scala
Any idea?
Jorge.
On Jun 17, 2014, at 7:22 AM, Jorge Marizan wrote:
> It got stuck on this:
>
> ./gradlew jar
> The TaskContainer.add() method has been deprecated and is scheduled to be
> removed in Gradle 2.0. Please use the create() me
There is some error log about failing leader election like that.
[2014-06-18 08:59:21,014] ERROR Controller 7 epoch 4 encountered error
while electing leader for partition [topicDEBUG,5] due to: Preferred
replica 1 for partition [topicDEBUG,5] is either not alive or not in the
isr. Current leader
What's the last line it's stuck on with debug flag on?
Tim
On Tue, Jun 17, 2014 at 4:46 PM, Jorge Marizan wrote:
> I almost got it to work but still stuck compiling scala
>
> Any idea?
>
> Jorge.
> On Jun 17, 2014, at 7:22 AM, Jorge Marizan wrote:
>
>> It got stuck on this:
>>
>> ./gradlew jar
:core:compileScala
Jorge.
On Jun 17, 2014, at 8:54 PM, Timothy Chen wrote:
> What's the last line it's stuck on with debug flag on?
>
> Tim
>
> On Tue, Jun 17, 2014 at 4:46 PM, Jorge Marizan
> wrote:
>> I almost got it to work but still stuck compiling scala
>>
>> Any idea?
>>
>> Jorge.
>
Did you have auto leader balancing enabled?
Thanks,
Jun
On Tue, Jun 17, 2014 at 5:06 PM, Bongyeon Kim
wrote:
> There is some error log about failing leader election like that.
>
>
> [2014-06-18 08:59:21,014] ERROR Controller 7 epoch 4 encountered error
> while electing leader for partition [t
Not sure what's wrong but I'm guessing there probably can be a gradle lock
somewhere.
Is there other gradle processes that is hanging around?
Tim
Sent from my iPhone
> On Jun 17, 2014, at 8:35 PM, Jorge Marizan wrote:
>
> :core:compileScala
>
> Jorge.
>
>> On Jun 17, 2014, at 8:54 PM, Timo
Not at all, I verified with ps aux and there is no Gradle processes left
behind when I cancel the compile job.
Jorge.
On Jun 17, 2014, at 11:45 PM, Timothy Chen wrote:
> Not sure what's wrong but I'm guessing there probably can be a gradle lock
> somewhere.
>
> Is there other gradle process
I don’t know what happened but I removed the folder were kafka resides on,
unzipped again, and magically everything seems to work. I will keep you updated.
Thanks man for your time. I appreciate it.
Jorge.
On Jun 17, 2014, at 11:54 PM, Jorge Marizan wrote:
> Not at all, I verified with ps au
No problem, let us know if it doesn't work again.
It does sounds like a file lock acquired somewhere though.
Tim
On Tue, Jun 17, 2014 at 8:59 PM, Jorge Marizan wrote:
> I don’t know what happened but I removed the folder were kafka resides on,
> unzipped again, and magically everything seems t
Now when I try to run, it fails finding the kafka.Kafka class:
kafka-server-start.sh /usr/local/etc/kafka/server.properties
Error: Could not find or load main class kafka.Kafka
Jorge
On Jun 17, 2014, at 11:54 PM, Jorge Marizan wrote:
> Not at all, I verified with ps aux and there is no Grad
So do you have the build folder generated in core/client and
classes/jars compiled in them?
Kafka-server-start.sh also sets the CLASSPATH to load the jar and
kafka.Kafka as well, so you want to make sure they're there.
Tim
On Tue, Jun 17, 2014 at 9:18 PM, Jorge Marizan wrote:
> Now when I try t
Hi
I was following the instructions for Kafka mirroring and had two
suggestions for improving the documentation at
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27846330:
1. Move "Note that the --zkconnect argument should point to the source
cluster's ZooKeeper...” above the con
Thanks Daniel for the findings, please feel free to update the wiki.
Guozhang
On Tue, Jun 17, 2014 at 9:56 PM, Daniel Compton
wrote:
> Hi
>
> I was following the instructions for Kafka mirroring and had two
> suggestions for improving the documentation at
> https://cwiki.apache.org/confluence/
Hi Guozhang
I feel pretty silly for asking this but how do I edit the Confluence Wiki? I
have an account and it says there is no edit restrictions defined for the page
but I can’t see an edit button anywhere. Do I need permission to edit the wiki?
Daniel.
On Wednesday, 18 June 2014 at 5:39
32 matches
Mail list logo
