Hi everyone,
Our setup: 3 node kafka (0.9.0.1) / 3 node zookeeper (3.4.6-1569965) setup on 6
separate VMware instances. We have around 10 topics with 32 partitions. Offsets
are managed by Kafka.
We have had a production issue and we suspect it was caused by VMWare snapshot
backups leading to
Hi Mike,
>From what you described it seems the socket on the producer was still
connected even after the broker instance is down. This is possible if the
broker instance went down in a sudden without closing the TCP connection
(e.g. lost power). Otherwise the producer should be able to detect the
That's great, thanks Jason.
We'll try and apply the patch in the meantime, and wait for the official
release for 0.10.1.1.
Please let us know if you need more details about the deadlocks on our side.
Thanks again!
Marcos
On Fri, Nov 4, 2016 at 1:02 PM, Jason Gustafson wrote:
> Hi Marcos,
>
>
Hi Marcos,
I think we'll try to get this into 0.10.1.1 (I updated the JIRA). Since
we're now seeing users hit this in practice, I'll definitely bump up the
priority on a fix. I can't say for sure when the release will be, but we'll
merge the fix into the 0.10.1 branch and you can build from there
Mike,
Did you ever figure this out?
We're considering using Kafka on Kubernetes and very interested in how it's
going for you.
On Thu, Oct 27, 2016 at 8:34 AM, Martin Gainty wrote:
> MG>can u write simpleConsumer to determine when lead broker times-out..
> then you'll need to tweak connection s
I understand the reason ...
I developed a POC for an AMQP - Kafka bridge
(https://github.com/rhiot/amqp-kafka-bridge) and the straightforward mapping
between AMQP "address" to Kafka "topic" isn't valid anymore.
In AMQP, the address is a simple string and a lot of systems (i.e. brokers) use
"/
We use topic names to generate directories on the file system, "/" is
not a valid character in directories (or if it is, it still makes
things like "ls" more complex), so we decided to simply do without it.
On Fri, Nov 4, 2016 at 8:46 AM, Paolo Patierno wrote:
> Hi all,
>
>
> why the "/" charac
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512
Robert,
you can use bin/kafka-consumer-groups.sh instead.
> bin/kafka-consumer-offset-checker.sh [2016-11-04 10:05:07,852] WARN
> WARNING: ConsumerOffsetChecker is deprecated and will be dropped in
> releases following 0.9.0. Use ConsumerGroupComma
Jason,
Thanks for that link. It does appear to be a very similar issue, if not
identical. In our case, the deadlock is reported as across 3 threads, one
of them being a group_metadata_manager thread. Otherwise, it looks the same.
On your questions:
- We did not see this in prior releases, but
Hi Becket,
I can confirm :
1. All messages are keyed
2. Generous memory settings already in place
3. The topic is not compacted
4. Luckily we haven't experienced this issue. We are testing 0.10.0 and
hope to make the jump shortly.
Thanks you very much for the support, very much appreciated :)
C
Hi all,
why the "/" character isn't allowed in a topic name ? If I try to use it I
receive an error on creation.
If I try to use the "\" character ... no error on creation but this character
is omitted from the name. Why ?
I.e. create "this\topic" ... it creates "thistopic"
Thanks,
Paolo
Hello expert,
I need help with the following error. I was tring to connect kafka to kerveros
server to verify SASL. I see this error when ZkClient tries to authenticate
with the zookeeper server. In the Kerberos logs I see tickets being exchanged.
I followed everything in documentation. Couldn
OK Thanks Damian.
Have a nice day.
Hamza
De : Damian Guy
Envoyé : vendredi 4 novembre 2016 00:58:36
À : users@kafka.apache.org
Objet : Re: Control results coming from windows
Hi Hamza,
I'm not sure what you mean in the first sentence? There are some breaking
A
Hi Hamza,
I'm not sure what you mean in the first sentence? There are some breaking
API changes form 0.10.0 -> 0.10.1, so you may need to change some code. I'd
also suggest you thoroughly test with the new version to ensure there are
no regressions.
There is a known issue with caching that may or
//server is using a JAAS-authenticated subject: determine service principal
name and hostname from kafka server's subject.
elevated privilege authentication implements thru kafka-subject so when you
establish your kerberos sasl server authenticator
SaslServerAuthenticator(String node, final S
Hi Damian,
If i would pass tp the 0.10.1. Is this possible without impacting the
applications that i've already done ?
If yes then which parameters are needed for deduplication with caching. Like
this you would save me time looking for it.
Thanks.
Hamza
De
Hi,
This is my server.properties (It works fine in SSL mode)
listeners=PLAINTEXT://10.160.32.151:9092,SSL://10.160.32.151:9093,SASL_SSL://10.160.32.151:9094
ssl.keystore.location=/root/server.keystore.jks
ssl.keystore.password=test1234
ssl.key.password=test1234
ssl.truststore.location=/root/ser
Hi Hamza,
If you are using version 0.10.0, then there is no way of controlling this.
In 0.10.1 we do some deduplication with caching, but you should still
expect multiple results for a window.
Thanks,
Damian
On Fri, 4 Nov 2016 at 08:42 Hamza HACHANI wrote:
> Hi,
>
>
> I implemented window usin
Are you specifying the option << —security-protocol SASL_PLAINTEXT >> in your
command as well?
> Le 4 nov. 2016 à 08:56, yassine chantit a écrit
> :
>
> Hi,Did you have in your jaas conf a section to configure zookeeper client to
> use kerberos as well ?Something like this : Client {
>
Hi,
Thanks for reply.
I configured kafka_server_jaas.conf as written in the documentation:
KafkaServer {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
useTicketCache=false
keyTab="/root/downloads/kafka_2.11-0.10.0.
Hi,Did you have in your jaas conf a section to configure zookeeper client to
use kerberos as well ?Something like this : Client {
com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true
keyTab="/path/to/client/keytab" storeKey=true useTicketCache=true
Hello expert,
I need help with the following error. I was tring to connect kafka to kerveros
server to verify SASL. I see this error when ZkClient tries to authenticate
with the zookeeper server. In the Kerberos logs I see tickets being exchanged.
I followed everything in documentation. Could
Hi,
I implemented window using Stream DSL.But results does get out from the window
anytime a new data come. Is there anyway to make result get out from the
window just at the end of the size of the window not like this way.
In other words, I do need to control the flow of data getting out of
Hi Eno
Thanks for the JIRA info
The change looks worth trying.Will let you know after i try it out.
Regards
Sai
On Wed, Nov 2, 2016 at 1:33 PM, Eno Thereska wrote:
> Hi Sai,
>
> For your second note on rebalancing taking a long time, we have just
> improved the situation in trunk after fixing
24 matches
Mail list logo
