.
Does Kafka Connect already have a mechanism for tasks to store and recover
arbitrary state? If not, then is there interest in adding this capability to
Kafka Connect? (If there is interest, then perhaps the dev list is a better
venue.)
Best regards,
Randall Hauch
config has to include much of the
same information included in the Kafka Connect workers configuration.
Am I just missing how a connector can see the worker configuration properties?
Or is there a way that Kafka Connect can help me create a Kafka consumer?
Best regards,
Randall Hauch
On January
On January 28, 2016 at 7:07:02 PM, Ewen Cheslack-Postava (e...@confluent.io)
wrote:
Randall,
Great question. Ideally you wouldn't need this type of state since it
should really be available in the source system. In your case, it might
actually make sense to be able to grab that information fro
On January 29, 2016 at 11:59:28 AM, Ewen Cheslack-Postava (e...@confluent.io)
wrote:
On Fri, Jan 29, 2016 at 7:06 AM, Randall Hauch wrote:
Luckily, MySQL includes the DDL statements in the log, so my connector parses
these as part of processing and builds up the schema state as it goes along
Hi, Paolo.
How are the Kafka Connect libraries loaded into your IntelliJ project? If
they are loaded as external libraries, where in the order of the external
libraries does the "org.apache.kafka:connector:connect-json" Maven module
appear? Or, is that module loaded as source?
Best regards,
Rand
>
> Paolo.
>
>
> Paolo Patierno
> Senior Software Engineer (IoT) @ Red Hat
> Microsoft MVP on Windows Embedded & IoT
> Microsoft Azure Advisor
>
> Twitter : @ppatierno<http://twitter.com/ppatierno>
> Linkedin : paolopatierno<http://it.linkedin.com/in/pao
o>
> Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno>
> Blog : DevExperience<http://paolopatierno.wordpress.com/>
>
>
>
> From: Randall Hauch
> Sent: Thursday, June 8, 2017 4:05 PM
> To: users@kafka.apache.org
> Subject: Re: ConnectSt
At any time, did your standalone worker config contain
`internal.value.converter.schemas.enable=false`?
On Thu, Jun 15, 2017 at 5:00 AM, Artem B. wrote:
> Hi all,
>
> Each time I start my source connector it fails to read the offset stored in
> a file with the following error:
>
> 21:05:01:519 |
Would you mind logging an issue against the Kafka Connect component in our
JIRA? https://issues.apache.org/jira/projects/KAFKA
Thanks!
On Sun, Jun 25, 2017 at 8:21 AM, 男Danngo wrote:
> Hi all,
>
> I'm trying to use the transformation for Kafka Connect and running into
> issues.
>
>
> The tranfo
Connect uses Kafka's group coordination mechanism and normal consumers to
balance/assign to the various sink tasks the topic partitions for the
subscribed topics. A task will continue to receive the records from its
assigned topic partitions until a rebalance occurs, which can happen when
any conne
On Tue, Oct 10, 2017 at 8:31 AM, Jehan Bruggeman
wrote:
> Hello,
>
> I'm trying to use a custom converter with Kafka Connect and I cannot seem
> to get it right. I'm hoping someone has experience with this and could help
> me figure it out !
>
>
> Initial situation
>
>
> - my cus
On Wed, Oct 4, 2017 at 9:44 AM, Jakub Scholz wrote:
> Hi,
>
> I have some questions about the configuration of the Kafka Connect REST
> interface.
>
> 1) What are the configuration options rest.advertised.host.name
> and rest.advertised.port useful for? The documentation says "If this is
> set, t
t;> that would help you figure it out ?
>>>
>>> Concerning the classpath: as described in my original email, I'm pretty
>>> sure the jars are correctly added to the classpath since the classes in
>>> the
>>> jar are recognized by Kafka Connect wh
Hi, Andrew.
The Converter is part of Connect's public API, so it certainly is
valid/encouraged to create new implementations when that makes sense for
users. I know of several Converter implementations that are outside of the
Apache Kafka project. The project's existing JSON converter is fairly
li
Connect is not only for sources that support reading from a specific point.
But unless the source system is tracking state for you, you connector's
likely to miss information in that source system if the connector stops.
This is probably okay in quite a few systems, so its good if you're fine
with
This is expected behavior, though there is already an enhancement request
to improve it: https://issues.apache.org/jira/browse/KAFKA-5505
Best regards,
Randall
On Fri, Jun 1, 2018 at 2:26 AM, Chintan Patel
wrote:
> Hello,
>
> I'm using Kafka connect in distributed mode. When I create new conne
Kafka Connect was designed explicitly to support this kind of use case.
Debezium's connectors implement the Kafka Connect API, and they just
released a MS SQL Server source connector that captures the changes from
SQL Server and writes them to Kafka. Debezium's MySQL source connector may
work with
Wang wrote:
> > > Hello all,
> > >
> > > The PMC of Apache Kafka is happy to announce another new committer
> > joining
> > > the project today: we have invited Randall Hauch as a project committer
> > and
> > > he has accepted.
> > >
/2.2/216/
/**
Thanks,
Randall Hauch
;
>
> On Fri, Oct 25, 2019 at 3:10 PM Randall Hauch wrote:
> >
> > Hello all, we identified around three dozen bug fixes, including an
> update
> > of a third party dependency, and wanted to release a patch release for
> the
> > Apache Kafka 2.2.0 release.
> &g
release!
A. Sophie Blee-Goldman, Matthias J. Sax, Bill Bejeck, Jason Gustafson,
Chris Egerton, Boyang Chen, Alex Diachenko, cpettitt-confluent, Magesh
Nandakumar, Randall Hauch, Ismael Juma, John Roesler, Konstantine
Karantasis, Mickael Maison, Nacho Muñoz Gómez, Nigel Liang, Paul, Rajini
Sivaram, Robert
id Hashemian <
> vahid.hashem...@gmail.com>
> > > wrote:
> > >
> > > > Awesome. Thanks for managing this release Randall!
> > > >
> > > > Regards,
> > > > --Vahid
> > > >
> > > > On Sun, Dec 1,
Ah, I merged https://issues.apache.org/jira/browse/KAFKA-9258 last night
and forgot to send an email to this thread.
Thanks!
Randall
On Wed, Dec 4, 2019 at 10:42 AM Eric Lalonde wrote:
> > I have also shared a pull request for the changes I have made to the test
> > script
> >
> > https://gith
CVE-2019-12399: Apache Kafka Connect REST API may expose plaintext secrets
in tasks endpoint
Severity: Medium
Vendor: The Apache Software Foundation
Versions Affected:
Apache Kafka 2.0.0, 2.0.1, 2.1.0, 2.1.1, 2.2.0, 2.2.1, 2.3.0
Description:
When Connect workers in Apache Kafka 2.0.0, 2.0.1,
rg/26/documentation.html
* Protocol:
https://kafka.apache.org/26/protocol.html
* Successful Jenkins builds for the 2.6 branch:
Unit/integration tests: https://builds.apache.org/job/kafka-2.6-jdk8/80/
System tests: (link to follow)
Thanks,
Randall Hauch
;m assuming that we'll
need to cut RC2.
But it'd be good for everyone else to double check this release.
Best regards,
Randall Hauch
On Mon, Jul 20, 2020 at 9:50 PM Randall Hauch wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the second candida
rg/26/documentation.html
* Protocol:
https://kafka.apache.org/26/protocol.html
* Successful Jenkins builds for the 2.6 branch:
Unit/integration tests: https://builds.apache.org/job/kafka-2.6-jdk8/91/ (one
flaky test)
System tests: (link to follow)
Thanks,
Randall Hauch
Any thoughts, Rajini?
On Mon, Jul 20, 2020 at 9:55 PM Randall Hauch wrote:
>
> When I was checking the documentation for RC1 after the tag was pushed, I
> noticed that the fix Rajini mentioned in the RC0 vote thread (
> https://github.com/apache/kafka/pull/8979
> <https:/
https://kafka.apache.org/26/documentation.html
* Protocol:
https://kafka.apache.org/26/protocol.html
* Successful Jenkins builds for the 2.6 branch:
Unit/integration tests: https://builds.apache.org/job/kafka-2.6-jdk8/101/
System tests: (link to follow)
Thanks,
Randall Hauch
gt; Thanks Randall, +1 (binding)
>
> Built from source and ran tests, had a quick look through some Javadoc
> changes, ran quickstart and some tests with Java 11 TLSv1.3 on the binary.
>
> Regards,
>
> Rajini
>
>
> On Tue, Jul 28, 2020 at 10:50 PM Randall Hauch wrote
at
>
> org.apache.kafka.trogdor.rest.JsonRestServer$HttpResponse.body(JsonRestServer.java:285)
> at
> org.apache.kafka.trogdor.agent.AgentClient.status(AgentClient.java:130)
> at
>
> org.apache.kafka.trogdor.agent.AgentTest.testAgentGetStatus(AgentTest.java:115)
>
g for at least one more binding +1 vote from PMC
members. Please download, test and vote by Monday, August 3, 9am PT,
Best regards!
Randall
On Fri, Jul 31, 2020 at 11:44 AM Randall Hauch wrote:
> Thanks, Rajini.
>
> Here's an update on the system tests. Unfortunately we've not
rce archive
> and Scala 2.13 binary archive.
>
> Thanks,
> Ismael
>
> On Tue, Jul 28, 2020, 2:52 PM Randall Hauch wrote:
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the third candidate for release of Apache Kafka 2.6.0. This is a
I've prepared a preliminary blog post about the upcoming Apache Kafka 2.6.0
release.
Please take a look and let me know via this thread if you want to
add/modify details.
Thanks to all who contributed to this blog post.
Unfortunately, the preview is not currently publicly visible at
https://blogs.
f the release contributors. It's an easy way to give
> some recognition. I know we have done that in the past.
>
> Thanks,
> Jason
>
> On Wed, Aug 5, 2020 at 8:25 AM Randall Hauch wrote:
>
> > I've prepared a preliminary blog post about the upcoming Apache Kafka
Thanks
> Maulin
>
>
> On Wed, Aug 5, 2020 at 9:49 AM Randall Hauch wrote:
>
> > Thanks, Jason. We haven't done that for a few releases, but I think it's
> a
> > great idea. I've updated the blog post draft and the Google doc to
> mention
> > th
I'ved incorporated this into the preview doc, but feedback is welcome.
Best regards,
Randall
On Wed, Aug 5, 2020 at 10:27 AM Randall Hauch wrote:
> I've prepared a preliminary blog post about the upcoming Apache Kafka
> 2.6.0 release.
> Please take a look and let me kn
, Manikumar Reddy, Mario Molina, Matthew Wong, Matthias J. Sax,
maulin-vasavada, Michael Viamari, Michal T, Mickael Maison, Mitch, Navina
Ramesh, Navinder Pal Singh Brar, nicolasguyomar, Nigel Liang, Nikolay,
Okada Haruki, Paul, Piotr Fras, Radai Rosenblatt, Rajini Sivaram, Randall
Hauch, Rens
> >
> > Adam Bellemare, Andras Katona, Andy Coates, Anna Povzner, A.
> > Sophie Blee-Goldman, Auston, belugabehr, Bill Bejeck, Boyang
> > Chen, Bruno Cadonna, Chia-Ping Tsai, Chris Egerton, David
> > Arthur, David Jacot, Dezhi “Andy” Fang, Dima Reznik, Ego,
> > E
rchenko, Jason Gustafson, Jeff
> Kim,
> > jeff kim, Jesse Gorzinski, jiameixie, Jim Galasyn, JoelWee, John Roesler,
> > John Thomas, Jorge Esteban Quilcate Otoya, Julien Jean Paul Sirocchi,
> > Justine Olshan, khairy, Konstantine Karantasis, Kowshik Prakasam, leah,
> Lee
&g
> Chia-Ping Tsai, Chris Egerton, dengziming, Greg Harris, Ismael Juma, Jim
> Galasyn, John Roesler, Julien Jean Paul Sirocchi, Lee Dongjin, Luke Chen,
> Manikumar Reddy, Matthias J. Sax, Michael G. Noll, Mickael Maison, Rajini
> Sivaram, Ramesh Krishnan M, Randall Hauch, Sharath Bha
Thanks, David.
I was able to successfully complete the following:
- Build release archive from the tag, installed locally, and ran a portion
of quickstart
- Installed 2.8.1 RC0 and performed quickstart for broker and Connect
- Verified signatures and checksums
- Verified the tag
- Compared the re
Thanks for generating a new RC1 with the corrected site docs, David.
I was able to successfully complete the following:
- Installed 2.8.1 RC0 and performed quickstart for broker and Connect
- Verified signatures and checksums
- Verified the tag
- Manually compared the release notes to JIRA
- Buil
r Iskuskov, Andras Katona, Bill Bejeck,
> Bruno Cadonna, Chris Egerton, Colin Patrick McCabe, David Arthur, David
> Jacot,
> Davor Poldrugo, Dejan Stojadinović, Geordie, Guozhang Wang, Ismael Juma,
> Jason Gustafson, Jeff Kim, John Gray, John Roesler, Justine Olshan,
> Konstantine
Severity: moderate
Description:
Some components in Apache Kafka use `Arrays.equals` to validate a password or
key, which is vulnerable to timing attacks that make brute force attacks for
such credentials more likely to be successful. Users should upgrade to 2.8.1 or
higher, or 3.0.0 or higher
Thanks for the RC, David.
I was able to successfully complete the following:
- Installed 3.1.0 RC1 and performed quickstart for broker and Connect
(using Java 17)
- Verified signatures and checksums
- Verified the tag
- Manually compared the release notes to JIRA
- Build release archive from the
Congratulations, Luke!
On Wed, Feb 9, 2022 at 11:02 PM Matthias J. Sax wrote:
> Congratulations! Glad to have you onboard, Luke!
>
> -Matthias
>
> On 2/9/22 16:37, Bill Bejeck wrote:
> > Congrats Luke! Well deserved.
> >
> > -Bill
> >
> > On Wed, Feb 9, 2022 at 7:25 PM Israel Ekpo wrote:
> >
>
Thanks for the RC, David.
I was able to successfully complete the following:
- Installed 3.2.1 RC3 and performed quickstart for broker and
Connect (using Java 17)
- Verified signatures and checksums
- Verified the tag
- Manually compared the release notes to JIRA
- Build release archive from the
48 matches
Mail list logo
