Writing a Producer from Scratch

Hi, I am new to Kafka (but not new to computers). I want to write a kafka producer client for a Unisys OS 2200 mainframe. I need to write it in C, and since I have no access to Windows / Unix / Linux libraries, I have to develop the interface at the lowest level. So far, I have downloaded a kaf

[jira] [Created] (KAFKA-3332) Consumer can't consume messages from zookeeper chroot

Sergey Vergun created KAFKA-3332: Summary: Consumer can't consume messages from zookeeper chroot Key: KAFKA-3332 URL: https://issues.apache.org/jira/browse/KAFKA-3332 Project: Kafka Issue Typ

[jira] [Updated] (KAFKA-3332) Consumer can't consume messages from zookeeper chroot

[ https://issues.apache.org/jira/browse/KAFKA-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Vergun updated KAFKA-3332: - Description: I have faced issue when consumer can't consume messages from zookeeper chroot. It wa

[jira] [Updated] (KAFKA-3332) Consumer can't consume messages from zookeeper chroot

[ https://issues.apache.org/jira/browse/KAFKA-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Vergun updated KAFKA-3332: - Description: I have faced issue when consumer can't consume messages from zookeeper chroot. It wa

[jira] [Created] (KAFKA-3333) Client Partitioner - Round Robin

Stephen Powis created KAFKA-: Summary: Client Partitioner - Round Robin Key: KAFKA- URL: https://issues.apache.org/jira/browse/KAFKA- Project: Kafka Issue Type: New Feature

[jira] [Commented] (KAFKA-3333) Client Partitioner - Round Robin

[ https://issues.apache.org/jira/browse/KAFKA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15179984#comment-15179984 ] Stephen Powis commented on KAFKA-: -- Will link to a github pull request shortly.

[GitHub] kafka pull request: [KAFKA-3333] - Add RoundRobinPartitioner

GitHub user Crim opened a pull request: https://github.com/apache/kafka/pull/1012 [KAFKA-] - Add RoundRobinPartitioner https://issues.apache.org/jira/browse/KAFKA- The DefaultPartitioner typically distributes using the hash of the keybytes, and falls back to ro

[jira] [Commented] (KAFKA-3333) Client Partitioner - Round Robin

[ https://issues.apache.org/jira/browse/KAFKA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15179998#comment-15179998 ] ASF GitHub Bot commented on KAFKA-: --- GitHub user Crim opened a pull request:

[jira] [Updated] (KAFKA-3333) Client Partitioner - Round Robin

[ https://issues.apache.org/jira/browse/KAFKA-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Powis updated KAFKA-: - Fix Version/s: 0.10.0.0 Affects Version/s: 0.10.0.0 Status: Patch Available

[jira] [Created] (KAFKA-3334) First message on new topic not actually being sent, no exception thrown

Aleksandar Stojadinovic created KAFKA-3334: -- Summary: First message on new topic not actually being sent, no exception thrown Key: KAFKA-3334 URL: https://issues.apache.org/jira/browse/KAFKA-3334

[jira] [Commented] (KAFKA-3334) First message on new topic not actually being sent, no exception thrown

[ https://issues.apache.org/jira/browse/KAFKA-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180235#comment-15180235 ] Ashish K Singh commented on KAFKA-3334: --- Typically one should create topic before pr

Re: [DISCUSS] KIP-35 - Retrieve protocol version

Yeah here is my summary of my take: 1. Negotiating a per-connection protocol actually does add a lot of complexity to clients (many more failure states to get right). 2. Having the client configure the protocol version manually is doable now but probably a worse state. I suspect this will lead to

[jira] [Commented] (KAFKA-3334) First message on new topic not actually being sent, no exception thrown

[ https://issues.apache.org/jira/browse/KAFKA-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180267#comment-15180267 ] Aleksandar Stojadinovic commented on KAFKA-3334: Understood. To be honest,

[jira] [Assigned] (KAFKA-3334) First message on new topic not actually being sent, no exception thrown

[ https://issues.apache.org/jira/browse/KAFKA-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh reassigned KAFKA-3334: - Assignee: Ashish K Singh > First message on new topic not actually being sent, no excepti

[GitHub] kafka pull request: Generate javadocs for all Streams packages wit...

GitHub user miguno opened a pull request: https://github.com/apache/kafka/pull/1013 Generate javadocs for all Streams packages with the exception of internals You can merge this pull request into a Git repository by running: $ git pull https://github.com/miguno/kafka trunk-str

[jira] [Commented] (KAFKA-3334) First message on new topic not actually being sent, no exception thrown

[ https://issues.apache.org/jira/browse/KAFKA-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180279#comment-15180279 ] Ashish K Singh commented on KAFKA-3334: --- Documentation should help. Will post a PR s

[GitHub] kafka pull request: HOTFIX: Generate javadocs for all Streams pack...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1013 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3334) First message on new topic not actually being sent, no exception thrown

[ https://issues.apache.org/jira/browse/KAFKA-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180332#comment-15180332 ] James Cheng commented on KAFKA-3334: I believe part of the explanation for this is the

[jira] [Comment Edited] (KAFKA-3334) First message on new topic not actually being sent, no exception thrown

[ https://issues.apache.org/jira/browse/KAFKA-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180332#comment-15180332 ] James Cheng edited comment on KAFKA-3334 at 3/4/16 6:37 PM: I

[jira] [Commented] (KAFKA-3312) Add a offsets methods to ZkUtils and replace relevant usages

[ https://issues.apache.org/jira/browse/KAFKA-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180394#comment-15180394 ] Vahid Hashemian commented on KAFKA-3312: [~granthenke] Just to clarify, do you mea

[jira] [Comment Edited] (KAFKA-3312) Add a offsets methods to ZkUtils and replace relevant usages

[ https://issues.apache.org/jira/browse/KAFKA-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180394#comment-15180394 ] Vahid Hashemian edited comment on KAFKA-3312 at 3/4/16 7:12 PM:

[jira] [Commented] (KAFKA-3312) Add a offsets methods to ZkUtils and replace relevant usages

[ https://issues.apache.org/jira/browse/KAFKA-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180398#comment-15180398 ] Grant Henke commented on KAFKA-3312: [~vahid] Yes, but I was thinking no only building

Build failed in Jenkins: kafka-trunk-jdk7 #1092

See Changes: [cshapi] HOTFIX: Generate javadocs for all Streams packages with the exception of -- [...truncated 1517 lines...] kafka.consumer.ZookeeperConsumerConnectorTest > testConsumerRebal

[GitHub] kafka pull request: KAFKA-3331: Refactor TopicCommand to make desc...

GitHub user SinghAsDev opened a pull request: https://github.com/apache/kafka/pull/1014 KAFKA-3331: Refactor TopicCommand to make describeTopic testable and … …add unit tests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Si

[jira] [Commented] (KAFKA-3331) Refactor TopicCommand to make it testable and add unit tests

[ https://issues.apache.org/jira/browse/KAFKA-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180418#comment-15180418 ] ASF GitHub Bot commented on KAFKA-3331: --- GitHub user SinghAsDev opened a pull reques

[jira] [Created] (KAFKA-3335) Kafka Connect hangs in shutdown hook

Ben Kirwin created KAFKA-3335: - Summary: Kafka Connect hangs in shutdown hook Key: KAFKA-3335 URL: https://issues.apache.org/jira/browse/KAFKA-3335 Project: Kafka Issue Type: Bug Repo

Additional Kafka Connect schema logical types?

I’m working on a Kafka Connect connector that reads a MySQL binlog to provide near real-time change data capture, and I also plan connectors for other DBMSes. The problem is that I’m not able to map all of the MySQL data types — or even all of the standard JDBC types — to Kafka Connect Schemas w

[GitHub] kafka pull request: KAFKA-3299: Ensure that reading config log on ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/981 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Resolved] (KAFKA-3299) KafkaConnect: DistributedHerder shouldn't wait forever to read configs after rebalance

[ https://issues.apache.org/jira/browse/KAFKA-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-3299. -- Resolution: Fixed Fix Version/s: 0.10.0.0 Issue resolved by pull request

[jira] [Updated] (KAFKA-3299) KafkaConnect: DistributedHerder shouldn't wait forever to read configs after rebalance

[ https://issues.apache.org/jira/browse/KAFKA-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3299: - Assignee: Gwen Shapira > KafkaConnect: DistributedHerder shouldn't wait forever to

[jira] [Commented] (KAFKA-3299) KafkaConnect: DistributedHerder shouldn't wait forever to read configs after rebalance

[ https://issues.apache.org/jira/browse/KAFKA-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180642#comment-15180642 ] ASF GitHub Bot commented on KAFKA-3299: --- Github user asfgit closed the pull request

Build failed in Jenkins: kafka-trunk-jdk7 #1093

See Changes: [me] KAFKA-3299: Ensure that reading config log on rebalance doesn't hang -- [...truncated 1494 lines...] kafka.integration.AutoOffsetResetTest > testResetToEarliestWhenOffsetTooLo

Re: [DISCUSS] KIP-35 - Retrieve protocol version

Hello Jay, The overall approach sounds good. I do realize that this discussion has gotten too lengthy and is starting to shoot tangents. Maybe a KIP call will help us getting to a decision faster. I do have a few questions though. On Fri, Mar 4, 2016 at 9:52 AM, Jay Kreps wrote: > Yeah here is

[jira] [Commented] (KAFKA-3303) Pass partial record metadata to Interceptor onAcknowledgement in case of errors

[ https://issues.apache.org/jira/browse/KAFKA-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15181383#comment-15181383 ] ASF GitHub Bot commented on KAFKA-3303: --- GitHub user apovzner opened a pull request:

[GitHub] kafka pull request: KAFKA-3303: Pass partial record metadata to Pr...

GitHub user apovzner opened a pull request: https://github.com/apache/kafka/pull/1015 KAFKA-3303: Pass partial record metadata to ProducerInterceptor.onAcknowledgement on error This is a KIP-42 followup. Currently, If sending the record fails before it gets to the server,

Build failed in Jenkins: kafka-trunk-jdk8 #421

See Changes: [me] KAFKA-3299: Ensure that reading config log on rebalance doesn't hang -- [...truncated 3533 lines...] org.apache.kafka.clients.producer.RecordSendTest > testBlocking PASSED org

[jira] [Updated] (KAFKA-3303) Pass partial record metadata to Interceptor onAcknowledgement in case of errors

[ https://issues.apache.org/jira/browse/KAFKA-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Povzner updated KAFKA-3303: Status: Patch Available (was: In Progress) > Pass partial record metadata to Interceptor onAcknowle

Re: [DISCUSS] KIP-35 - Retrieve protocol version

Hey Ashish, Both good points. I think the issue with the general metadata request is the same as the issue with a version-specific metadata request from the other proposal--basically it's a chicken and egg problem, to find out anything about the cluster you have to be able to communicate somethin

[jira] [Updated] (KAFKA-3185) Allow users to cleanup internal data

[ https://issues.apache.org/jira/browse/KAFKA-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3185: - Priority: Blocker (was: Major) > Allow users to cleanup internal data > -

[jira] [Created] (KAFKA-3336) Unify ser/de pair classes into one serde class

Guozhang Wang created KAFKA-3336: Summary: Unify ser/de pair classes into one serde class Key: KAFKA-3336 URL: https://issues.apache.org/jira/browse/KAFKA-3336 Project: Kafka Issue Type: Sub-

[jira] [Updated] (KAFKA-3336) Unify ser/de pair classes into one serde class

[ https://issues.apache.org/jira/browse/KAFKA-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3336: - Description: Right now users must provide two separate classes for serializers and deserializers,

[jira] [Created] (KAFKA-3337) Extract selector as a separate groupBy operator for KTable aggregations

Guozhang Wang created KAFKA-3337: Summary: Extract selector as a separate groupBy operator for KTable aggregations Key: KAFKA-3337 URL: https://issues.apache.org/jira/browse/KAFKA-3337 Project: Kafka

[jira] [Created] (KAFKA-3338) Add print / writeAsTex / etc functions to the DSL

Guozhang Wang created KAFKA-3338: Summary: Add print / writeAsTex / etc functions to the DSL Key: KAFKA-3338 URL: https://issues.apache.org/jira/browse/KAFKA-3338 Project: Kafka Issue Type: S