Re: Official Kafka Gitter Room?

[Harsha]

one of the benefits of keeping conversations in mailing lists is it can
be searchable later. It will benefit users who might seen the same error
which was answered on the mailing lists before. slack/glitter can do
that?


On Sat, Jul 18, 2015, at 09:59 PM, Gwen Shapira wrote:
> FYI:
> Joining Slack requires an invitation. We can take invite-requests in
> the mailing lists (like we do for KIP hangout).
> I wanted to point this out as this may be considered non-Apache
> (although I didn't find any specific wording against invite-only
> channels in addition to the mailing list).
> 
> So, if you want to try it out, yell for an invite :)
> 
> On Sat, Jul 18, 2015 at 6:26 PM, Gwen Shapira 
> wrote:
> > So, as an experiment, I created:
> > https://apachekafka.slack.com
> >
> > I figured we'll give it a whirl for a week or two for dev discussions,
> > see how it goes and if we have activity we can add this to the website
> > and announce on the lists.
> >
> > Gwen
> >
> > On Sat, Jul 18, 2015 at 6:18 PM, Gwen Shapira  wrote:
> >> Slack sounds good and I'm not a fan of gitter (we use it for Apache
> >> Sqoop, I dislike the GUI and the guilt-inducing emails about all the
> >> messages I missed...)
> >> Do they have free rooms for open source projects?
> >>
> >> On Sat, Jul 18, 2015 at 6:10 PM, Andrew Psaltis
> >>  wrote:
> >>> +1 for slack
> >>>
> >>> On Saturday, July 18, 2015, Neha Narkhede  wrote:
> >>>
>  FWIW, if the apache-kafka channel was on Slack, I'd hang out there :-)
> 
>  On Sat, Jul 18, 2015 at 7:20 AM, Ismael Juma   > wrote:
> 
>  > Hi Grant,
>  >
>  > I have been considering proposing the same thing myself for similar
>  > reasons, but I also have some doubts. More comments inline.
>  >
>  > On Fri, Jul 17, 2015 at 7:28 PM, Grant Henke   > wrote:
>  >
>  > > I think it could be useful to have a place to chat that is associated
>  > with
>  > > the Kafka repo.
>  >
>  >
>  > Agreed.
>  >
>  >
>  > > Note that we do currently have an IRC channel, but from my
>  > > experience its a ghost town.
>  > >
>  >
>  > I've recently joined #apache-kafka and activity is low, yes, but there 
>  > is
>  > some[1]. The bigger issue, perhaps, is that contributors are not there.
>  Do
>  > we think that this would be different with Gitter? And, if so, what is
>  the
>  > reason? Is it because no application needs to be installed? In theory,
>  > IRCCloud provides something similar for IRC (although the limitations 
>  > of
>  > the trial and the upgrade nagging can be annoying) and we could link to
>  the
>  > channel in GitHub's readme.
>  >
>  > Also, are there any Apache projects who use Gitter? There was a thread 
>  > in
>  > the Spark mailing list, but the response wasn't positive[2]. We would
>  need
>  > Apache Infra to set-up Gitter for us as they are the only ones that 
>  > have
>  > write access to the repository (the set-up is really easy though, which
>  is
>  > definitely a positive).
>  >
>  > Gitter may be the right answer, I just want to make sure it actually
>  helps
>  > us have a more active channel instead of splitting the existing 
>  > activity
>  > between two channels.
>  >
>  > Thoughts?
>  >
>  > Best,
>  > Ismael
>  >
>  > [1] 
>  > https://botbot.me/freenode/apache-kafka/2015-07-06/?tz=Europe/London
>  > [2]
>  >
>  >
>  http://apache-spark-developers-list.1001551.n3.nabble.com/Gitter-chat-room-for-Spark-td11636.html
>  >
> 
> 
> 
>  --
>  Thanks,
>  Neha
> 

Kafka Unit Test Failures on a Mac

[Grant Henke]

When running all Kafka tests I had been getting failures most every time.
Usually in the SocketServerTest class. However, when I would run individual
tests, there were no failures. After a bit of digging I found this is due
to the small default open files limit in Mac Yosemite. I am positing how to
increase the limit here in case anyone else has been running into the
issue. Let me know if this helped you too. If it is fairly common we can
put something on the wiki.

*Adjusting Open File Limits in Yosemite:*
Note: You can choose your own limits as appropriate

1. Write the following xml to /Library/LaunchDaemons/limit.maxfiles.plist:



http://www.apple.com/DTDs/PropertyList-1.0.dtd";>

  



  Label

limit.maxfiles

  ProgramArguments



  launchctl

  limit

  maxfiles

  65536

  65536



  RunAtLoad



  ServiceIPC





  


2. Then write the following to /Library/LaunchDaemons/limit.maxproc.plist:


http://www.apple.com/DTDs/PropertyList-1.0.dtd";>
  

  Label
limit.maxproc
  ProgramArguments

  launchctl
  limit
  maxproc
  2048
  2048

  RunAtLoad

  ServiceIPC


  


3. Add the following to your bashrc or bashprofile:

ulimit -n 65536

ulimit -u 2048


4. Restart your computer. After restart validate settings by executing:

launchctl limit



*Adjusting Open File Limits in Older Versions of OS X:*
Note: You can choose your own limits as appropriate

1. Add the following command to /etc/launchd.conf:

limit maxfiles 32768 65536


2. Restart your computer. After restart validate settings by executing:

launchctl limit

Re: Official Kafka Gitter Room?

[Edward Ribeiro]

Hi, Gwen,

I would like a Slack invite, *please*. :-)

Cheers,
Eddie
Em 19/07/2015 02:00, "Gwen Shapira"  escreveu:

> FYI:
> Joining Slack requires an invitation. We can take invite-requests in
> the mailing lists (like we do for KIP hangout).
> I wanted to point this out as this may be considered non-Apache
> (although I didn't find any specific wording against invite-only
> channels in addition to the mailing list).
>
> So, if you want to try it out, yell for an invite :)
>
> On Sat, Jul 18, 2015 at 6:26 PM, Gwen Shapira 
> wrote:
> > So, as an experiment, I created:
> > https://apachekafka.slack.com
> >
> > I figured we'll give it a whirl for a week or two for dev discussions,
> > see how it goes and if we have activity we can add this to the website
> > and announce on the lists.
> >
> > Gwen
> >
> > On Sat, Jul 18, 2015 at 6:18 PM, Gwen Shapira 
> wrote:
> >> Slack sounds good and I'm not a fan of gitter (we use it for Apache
> >> Sqoop, I dislike the GUI and the guilt-inducing emails about all the
> >> messages I missed...)
> >> Do they have free rooms for open source projects?
> >>
> >> On Sat, Jul 18, 2015 at 6:10 PM, Andrew Psaltis
> >>  wrote:
> >>> +1 for slack
> >>>
> >>> On Saturday, July 18, 2015, Neha Narkhede  wrote:
> >>>
>  FWIW, if the apache-kafka channel was on Slack, I'd hang out there :-)
> 
>  On Sat, Jul 18, 2015 at 7:20 AM, Ismael Juma   > wrote:
> 
>  > Hi Grant,
>  >
>  > I have been considering proposing the same thing myself for similar
>  > reasons, but I also have some doubts. More comments inline.
>  >
>  > On Fri, Jul 17, 2015 at 7:28 PM, Grant Henke   > wrote:
>  >
>  > > I think it could be useful to have a place to chat that is
> associated
>  > with
>  > > the Kafka repo.
>  >
>  >
>  > Agreed.
>  >
>  >
>  > > Note that we do currently have an IRC channel, but from my
>  > > experience its a ghost town.
>  > >
>  >
>  > I've recently joined #apache-kafka and activity is low, yes, but
> there is
>  > some[1]. The bigger issue, perhaps, is that contributors are not
> there.
>  Do
>  > we think that this would be different with Gitter? And, if so, what
> is
>  the
>  > reason? Is it because no application needs to be installed? In
> theory,
>  > IRCCloud provides something similar for IRC (although the
> limitations of
>  > the trial and the upgrade nagging can be annoying) and we could
> link to
>  the
>  > channel in GitHub's readme.
>  >
>  > Also, are there any Apache projects who use Gitter? There was a
> thread in
>  > the Spark mailing list, but the response wasn't positive[2]. We
> would
>  need
>  > Apache Infra to set-up Gitter for us as they are the only ones that
> have
>  > write access to the repository (the set-up is really easy though,
> which
>  is
>  > definitely a positive).
>  >
>  > Gitter may be the right answer, I just want to make sure it actually
>  helps
>  > us have a more active channel instead of splitting the existing
> activity
>  > between two channels.
>  >
>  > Thoughts?
>  >
>  > Best,
>  > Ismael
>  >
>  > [1]
> https://botbot.me/freenode/apache-kafka/2015-07-06/?tz=Europe/London
>  > [2]
>  >
>  >
> 
> http://apache-spark-developers-list.1001551.n3.nabble.com/Gitter-chat-room-for-Spark-td11636.html
>  >
> 
> 
> 
>  --
>  Thanks,
>  Neha
> 
>

Re: Official Kafka Gitter Room?

[Gwen Shapira]

Good point.

Slack has searchable archives, but I don't think it shows up in Google
like the mailing list does.

On Sun, Jul 19, 2015 at 12:37 AM, Harsha  wrote:
> one of the benefits of keeping conversations in mailing lists is it can
> be searchable later. It will benefit users who might seen the same error
> which was answered on the mailing lists before. slack/glitter can do
> that?
>
>
> On Sat, Jul 18, 2015, at 09:59 PM, Gwen Shapira wrote:
>> FYI:
>> Joining Slack requires an invitation. We can take invite-requests in
>> the mailing lists (like we do for KIP hangout).
>> I wanted to point this out as this may be considered non-Apache
>> (although I didn't find any specific wording against invite-only
>> channels in addition to the mailing list).
>>
>> So, if you want to try it out, yell for an invite :)
>>
>> On Sat, Jul 18, 2015 at 6:26 PM, Gwen Shapira 
>> wrote:
>> > So, as an experiment, I created:
>> > https://apachekafka.slack.com
>> >
>> > I figured we'll give it a whirl for a week or two for dev discussions,
>> > see how it goes and if we have activity we can add this to the website
>> > and announce on the lists.
>> >
>> > Gwen
>> >
>> > On Sat, Jul 18, 2015 at 6:18 PM, Gwen Shapira  
>> > wrote:
>> >> Slack sounds good and I'm not a fan of gitter (we use it for Apache
>> >> Sqoop, I dislike the GUI and the guilt-inducing emails about all the
>> >> messages I missed...)
>> >> Do they have free rooms for open source projects?
>> >>
>> >> On Sat, Jul 18, 2015 at 6:10 PM, Andrew Psaltis
>> >>  wrote:
>> >>> +1 for slack
>> >>>
>> >>> On Saturday, July 18, 2015, Neha Narkhede  wrote:
>> >>>
>>  FWIW, if the apache-kafka channel was on Slack, I'd hang out there :-)
>> 
>>  On Sat, Jul 18, 2015 at 7:20 AM, Ismael Juma >  > wrote:
>> 
>>  > Hi Grant,
>>  >
>>  > I have been considering proposing the same thing myself for similar
>>  > reasons, but I also have some doubts. More comments inline.
>>  >
>>  > On Fri, Jul 17, 2015 at 7:28 PM, Grant Henke >  > wrote:
>>  >
>>  > > I think it could be useful to have a place to chat that is 
>>  > > associated
>>  > with
>>  > > the Kafka repo.
>>  >
>>  >
>>  > Agreed.
>>  >
>>  >
>>  > > Note that we do currently have an IRC channel, but from my
>>  > > experience its a ghost town.
>>  > >
>>  >
>>  > I've recently joined #apache-kafka and activity is low, yes, but 
>>  > there is
>>  > some[1]. The bigger issue, perhaps, is that contributors are not 
>>  > there.
>>  Do
>>  > we think that this would be different with Gitter? And, if so, what is
>>  the
>>  > reason? Is it because no application needs to be installed? In theory,
>>  > IRCCloud provides something similar for IRC (although the limitations 
>>  > of
>>  > the trial and the upgrade nagging can be annoying) and we could link 
>>  > to
>>  the
>>  > channel in GitHub's readme.
>>  >
>>  > Also, are there any Apache projects who use Gitter? There was a 
>>  > thread in
>>  > the Spark mailing list, but the response wasn't positive[2]. We would
>>  need
>>  > Apache Infra to set-up Gitter for us as they are the only ones that 
>>  > have
>>  > write access to the repository (the set-up is really easy though, 
>>  > which
>>  is
>>  > definitely a positive).
>>  >
>>  > Gitter may be the right answer, I just want to make sure it actually
>>  helps
>>  > us have a more active channel instead of splitting the existing 
>>  > activity
>>  > between two channels.
>>  >
>>  > Thoughts?
>>  >
>>  > Best,
>>  > Ismael
>>  >
>>  > [1] 
>>  > https://botbot.me/freenode/apache-kafka/2015-07-06/?tz=Europe/London
>>  > [2]
>>  >
>>  >
>>  http://apache-spark-developers-list.1001551.n3.nabble.com/Gitter-chat-room-for-Spark-td11636.html
>>  >
>> 
>> 
>> 
>>  --
>>  Thanks,
>>  Neha
>> 

Re: Official Kafka Gitter Room?

[Gwen Shapira]

invited :)

btw. anyone already in Slack can invite others with: /invite_people 

On Sun, Jul 19, 2015 at 9:19 AM, Edward Ribeiro
 wrote:
> Hi, Gwen,
>
> I would like a Slack invite, *please*. :-)
>
> Cheers,
> Eddie
> Em 19/07/2015 02:00, "Gwen Shapira"  escreveu:
>
>> FYI:
>> Joining Slack requires an invitation. We can take invite-requests in
>> the mailing lists (like we do for KIP hangout).
>> I wanted to point this out as this may be considered non-Apache
>> (although I didn't find any specific wording against invite-only
>> channels in addition to the mailing list).
>>
>> So, if you want to try it out, yell for an invite :)
>>
>> On Sat, Jul 18, 2015 at 6:26 PM, Gwen Shapira 
>> wrote:
>> > So, as an experiment, I created:
>> > https://apachekafka.slack.com
>> >
>> > I figured we'll give it a whirl for a week or two for dev discussions,
>> > see how it goes and if we have activity we can add this to the website
>> > and announce on the lists.
>> >
>> > Gwen
>> >
>> > On Sat, Jul 18, 2015 at 6:18 PM, Gwen Shapira 
>> wrote:
>> >> Slack sounds good and I'm not a fan of gitter (we use it for Apache
>> >> Sqoop, I dislike the GUI and the guilt-inducing emails about all the
>> >> messages I missed...)
>> >> Do they have free rooms for open source projects?
>> >>
>> >> On Sat, Jul 18, 2015 at 6:10 PM, Andrew Psaltis
>> >>  wrote:
>> >>> +1 for slack
>> >>>
>> >>> On Saturday, July 18, 2015, Neha Narkhede  wrote:
>> >>>
>>  FWIW, if the apache-kafka channel was on Slack, I'd hang out there :-)
>> 
>>  On Sat, Jul 18, 2015 at 7:20 AM, Ismael Juma >  > wrote:
>> 
>>  > Hi Grant,
>>  >
>>  > I have been considering proposing the same thing myself for similar
>>  > reasons, but I also have some doubts. More comments inline.
>>  >
>>  > On Fri, Jul 17, 2015 at 7:28 PM, Grant Henke >  > wrote:
>>  >
>>  > > I think it could be useful to have a place to chat that is
>> associated
>>  > with
>>  > > the Kafka repo.
>>  >
>>  >
>>  > Agreed.
>>  >
>>  >
>>  > > Note that we do currently have an IRC channel, but from my
>>  > > experience its a ghost town.
>>  > >
>>  >
>>  > I've recently joined #apache-kafka and activity is low, yes, but
>> there is
>>  > some[1]. The bigger issue, perhaps, is that contributors are not
>> there.
>>  Do
>>  > we think that this would be different with Gitter? And, if so, what
>> is
>>  the
>>  > reason? Is it because no application needs to be installed? In
>> theory,
>>  > IRCCloud provides something similar for IRC (although the
>> limitations of
>>  > the trial and the upgrade nagging can be annoying) and we could
>> link to
>>  the
>>  > channel in GitHub's readme.
>>  >
>>  > Also, are there any Apache projects who use Gitter? There was a
>> thread in
>>  > the Spark mailing list, but the response wasn't positive[2]. We
>> would
>>  need
>>  > Apache Infra to set-up Gitter for us as they are the only ones that
>> have
>>  > write access to the repository (the set-up is really easy though,
>> which
>>  is
>>  > definitely a positive).
>>  >
>>  > Gitter may be the right answer, I just want to make sure it actually
>>  helps
>>  > us have a more active channel instead of splitting the existing
>> activity
>>  > between two channels.
>>  >
>>  > Thoughts?
>>  >
>>  > Best,
>>  > Ismael
>>  >
>>  > [1]
>> https://botbot.me/freenode/apache-kafka/2015-07-06/?tz=Europe/London
>>  > [2]
>>  >
>>  >
>> 
>> http://apache-spark-developers-list.1001551.n3.nabble.com/Gitter-chat-room-for-Spark-td11636.html
>>  >
>> 
>> 
>> 
>>  --
>>  Thanks,
>>  Neha
>> 
>>

Re: Official Kafka Gitter Room?

[Jonathan Natkins]

Slack results are only searchable to a point if you're using the free
version. I think they allow for 10K messages to be stored, but unlimited is
paid.

On Sunday, July 19, 2015, Gwen Shapira  wrote:

> Good point.
>
> Slack has searchable archives, but I don't think it shows up in Google
> like the mailing list does.
>
> On Sun, Jul 19, 2015 at 12:37 AM, Harsha >
> wrote:
> > one of the benefits of keeping conversations in mailing lists is it can
> > be searchable later. It will benefit users who might seen the same error
> > which was answered on the mailing lists before. slack/glitter can do
> > that?
> >
> >
> > On Sat, Jul 18, 2015, at 09:59 PM, Gwen Shapira wrote:
> >> FYI:
> >> Joining Slack requires an invitation. We can take invite-requests in
> >> the mailing lists (like we do for KIP hangout).
> >> I wanted to point this out as this may be considered non-Apache
> >> (although I didn't find any specific wording against invite-only
> >> channels in addition to the mailing list).
> >>
> >> So, if you want to try it out, yell for an invite :)
> >>
> >> On Sat, Jul 18, 2015 at 6:26 PM, Gwen Shapira  >
> >> wrote:
> >> > So, as an experiment, I created:
> >> > https://apachekafka.slack.com
> >> >
> >> > I figured we'll give it a whirl for a week or two for dev discussions,
> >> > see how it goes and if we have activity we can add this to the website
> >> > and announce on the lists.
> >> >
> >> > Gwen
> >> >
> >> > On Sat, Jul 18, 2015 at 6:18 PM, Gwen Shapira  > wrote:
> >> >> Slack sounds good and I'm not a fan of gitter (we use it for Apache
> >> >> Sqoop, I dislike the GUI and the guilt-inducing emails about all the
> >> >> messages I missed...)
> >> >> Do they have free rooms for open source projects?
> >> >>
> >> >> On Sat, Jul 18, 2015 at 6:10 PM, Andrew Psaltis
> >> >> > wrote:
> >> >>> +1 for slack
> >> >>>
> >> >>> On Saturday, July 18, 2015, Neha Narkhede  > wrote:
> >> >>>
> >>  FWIW, if the apache-kafka channel was on Slack, I'd hang out there
> :-)
> >> 
> >>  On Sat, Jul 18, 2015 at 7:20 AM, Ismael Juma  
> >>  > wrote:
> >> 
> >>  > Hi Grant,
> >>  >
> >>  > I have been considering proposing the same thing myself for
> similar
> >>  > reasons, but I also have some doubts. More comments inline.
> >>  >
> >>  > On Fri, Jul 17, 2015 at 7:28 PM, Grant Henke <
> ghe...@cloudera.com 
> >>  > wrote:
> >>  >
> >>  > > I think it could be useful to have a place to chat that is
> associated
> >>  > with
> >>  > > the Kafka repo.
> >>  >
> >>  >
> >>  > Agreed.
> >>  >
> >>  >
> >>  > > Note that we do currently have an IRC channel, but from my
> >>  > > experience its a ghost town.
> >>  > >
> >>  >
> >>  > I've recently joined #apache-kafka and activity is low, yes, but
> there is
> >>  > some[1]. The bigger issue, perhaps, is that contributors are not
> there.
> >>  Do
> >>  > we think that this would be different with Gitter? And, if so,
> what is
> >>  the
> >>  > reason? Is it because no application needs to be installed? In
> theory,
> >>  > IRCCloud provides something similar for IRC (although the
> limitations of
> >>  > the trial and the upgrade nagging can be annoying) and we could
> link to
> >>  the
> >>  > channel in GitHub's readme.
> >>  >
> >>  > Also, are there any Apache projects who use Gitter? There was a
> thread in
> >>  > the Spark mailing list, but the response wasn't positive[2]. We
> would
> >>  need
> >>  > Apache Infra to set-up Gitter for us as they are the only ones
> that have
> >>  > write access to the repository (the set-up is really easy
> though, which
> >>  is
> >>  > definitely a positive).
> >>  >
> >>  > Gitter may be the right answer, I just want to make sure it
> actually
> >>  helps
> >>  > us have a more active channel instead of splitting the existing
> activity
> >>  > between two channels.
> >>  >
> >>  > Thoughts?
> >>  >
> >>  > Best,
> >>  > Ismael
> >>  >
> >>  > [1]
> https://botbot.me/freenode/apache-kafka/2015-07-06/?tz=Europe/London
> >>  > [2]
> >>  >
> >>  >
> >> 
> http://apache-spark-developers-list.1001551.n3.nabble.com/Gitter-chat-room-for-Spark-td11636.html
> >>  >
> >> 
> >> 
> >> 
> >>  --
> >>  Thanks,
> >>  Neha
> >> 
>


-- 
Jonathan "Natty" Natkins
StreamSets | Customer Engagement Engineer
mobile: 609.577.1600 | linkedin 

Re: Official Kafka Gitter Room?

[Ismael Juma]

Hi Gwen,

I like Slack and I think it's great for companies. However, there are a few
things that make it less suitable for open-source projects, in my opinion.
More below.

On Sun, Jul 19, 2015 at 5:59 AM, Gwen Shapira  wrote:
>
> Joining Slack requires an invitation. We can take invite-requests in
> the mailing lists (like we do for KIP hangout).
> I wanted to point this out as this may be considered non-Apache
> (although I didn't find any specific wording against invite-only
> channels in addition to the mailing list).
>

This is indeed one of the two main issues. Requiring an invite to modify a
wiki or to participate in a hangout is not too bad. However, requiring an
invite to see and search present and past conversations is too much of a
barrier for casual users. If we are to adopt a text chat system, users
should be able to see and search the conversations seamlessly, IMO.

The second main issue is that one can only browse and search the 10k most
recent messages in the free version. Using the paid version fixes this, but
the fact that it's a per user license model makes it impractical unless we
were able to get a special deal.

As much as I like Slack, I don't think it provides enough over IRC or
Gitter to outweigh these issues.

What do others think?

Best,
Ismael

Re: Kafka Unit Test Failures on a Mac

[Ismael Juma]

Hello Grant,

Thanks for figuring this out. I have also run into this issue when running
the tests on OS X Yosemite.

Ideally the tests would fail in a way that would make it clear what the
issue is. That may be complicated, so we should at least document it as you
suggest.

I'll let you know if the issues goes away for me too with this change.

Best,
Ismael

On Sun, Jul 19, 2015 at 4:24 PM, Grant Henke  wrote:

> When running all Kafka tests I had been getting failures most every time.
> Usually in the SocketServerTest class. However, when I would run individual
> tests, there were no failures. After a bit of digging I found this is due
> to the small default open files limit in Mac Yosemite. I am positing how to
> increase the limit here in case anyone else has been running into the
> issue. Let me know if this helped you too. If it is fairly common we can
> put something on the wiki.
>
> *Adjusting Open File Limits in Yosemite:*
> Note: You can choose your own limits as appropriate
>
> 1. Write the following xml to /Library/LaunchDaemons/limit.maxfiles.plist:
>
> 
>
>  http://www.apple.com/DTDs/PropertyList-1.0.dtd";>
>
>   
>
> 
>
>   Label
>
> limit.maxfiles
>
>   ProgramArguments
>
> 
>
>   launchctl
>
>   limit
>
>   maxfiles
>
>   65536
>
>   65536
>
> 
>
>   RunAtLoad
>
> 
>
>   ServiceIPC
>
> 
>
> 
>
>   
>
>
> 2. Then write the following to /Library/LaunchDaemons/limit.maxproc.plist:
>
> 
>  http://www.apple.com/DTDs/PropertyList-1.0.dtd";>
>   
> 
>   Label
> limit.maxproc
>   ProgramArguments
> 
>   launchctl
>   limit
>   maxproc
>   2048
>   2048
> 
>   RunAtLoad
> 
>   ServiceIPC
> 
> 
>   
>
>
> 3. Add the following to your bashrc or bashprofile:
>
> ulimit -n 65536
>
> ulimit -u 2048
>
>
> 4. Restart your computer. After restart validate settings by executing:
>
> launchctl limit
>
>
>
> *Adjusting Open File Limits in Older Versions of OS X:*
> Note: You can choose your own limits as appropriate
>
> 1. Add the following command to /etc/launchd.conf:
>
> limit maxfiles 32768 65536
>
>
> 2. Restart your computer. After restart validate settings by executing:
>
> launchctl limit
>

Re: Review Request 36590: Patch for KAFKA-2275

[Ashish Singh]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36590/#review9
---



clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
(line 1069)


That is a good point and is actually true. The reason the test I added 
worked fine is because it never added any topic to metadata, so it sent a 
TopicMetadataRequest with no topics, in which case metadata for all topics is 
sent back by broker. However, if I add some topics to metadata by calling {{var 
mapTopicParts = this.consumers(0).partitionsFor(topic1, topic2)}} before 
calling {{var mapTopicParts = this.consumers(0).partitionsFor()}} the test 
fails. Will add the test and address the issues. Thanks!


- Ashish Singh


On July 18, 2015, 4:39 a.m., Ashish Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36590/
> ---
> 
> (Updated July 18, 2015, 4:39 a.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-2275
> https://issues.apache.org/jira/browse/KAFKA-2275
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> Return metadata for all topics if empty list is passed to partitionsFor
> 
> 
> KAFKA-2275: Add a "Map> partitionsFor(String... 
> topics)" API to the new consumer
> 
> 
> Diffs
> -
> 
>   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
> 252b759c0801f392e3526b0f31503b4b8fbf1c8a 
>   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
> bea3d737c51be77d5b5293cdd944d33b905422ba 
>   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
> c14eed1e95f2e682a235159a366046f00d1d90d6 
>   core/src/test/scala/integration/kafka/api/ConsumerTest.scala 
> 3eb5f95731a3f06f662b334ab2b3d0ad7fa9e1ca 
> 
> Diff: https://reviews.apache.org/r/36590/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ashish Singh
> 
>

Re: Review Request 33620: Patch for KAFKA-1690

[Sriharsha Chintalapani]

> On June 30, 2015, 5:27 p.m., Michael Herstine wrote:
> > clients/src/main/java/org/apache/kafka/common/network/TransportLayer.java, 
> > line 31
> > 
> >
> > This may be my lack of Kafak-specific knowledge, but I'm getting a bit 
> > lost, here. I see sockets, transport layers, channels, and now transport 
> > layers... it might be good to include some javadoc on what, exactly, this 
> > interface models & how that abstraction relates to other abstrations with 
> > similar-sounding (to me, at least) names.

Added comments. KafkaChannel should be the default network I/O across kafka 
brokers or clients. Since we are in middle of transition to client side network 
code and we've still BlockingChannel it can be confusing till we move this part 
of the code as well.


- Sriharsha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33620/#review89910
---


On June 23, 2015, 8:18 p.m., Sriharsha Chintalapani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33620/
> ---
> 
> (Updated June 23, 2015, 8:18 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1690
> https://issues.apache.org/jira/browse/KAFKA-1690
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. SSLFactory tests.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Added 
> PrincipalBuilder.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Addressing 
> reviews.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Addressing 
> reviews.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Addressing 
> reviews.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Fixed minor 
> issues with the patch.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Fixed minor 
> issues with the patch.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> Merge remote-tracking branch 'refs/remotes/origin/trunk' into KAFKA-1690-V1
> 
> 
> KAFKA-1690. Broker side ssl changes.
> 
> 
> KAFKA-1684. SSL for socketServer.
> 
> 
> KAFKA-1690. Added SSLProducerSendTest and fixes to get right port for SSL.
> 
> 
> Merge branch 'trunk' into KAFKA-1690-V1
> 
> 
> KAFKA-1690. Post merge fixes.
> 
> 
> KAFKA-1690. Added SSLProducerSendTest.
> 
> 
> KAFKA-1690. Minor fixes based on patch review comments.
> 
> 
> Diffs
> -
> 
>   build.gradle 30d1cf2f1ff9ed3f86a060da8099bb0774b4cf91 
>   checkstyle/import-control.xml f2e6cec267e67ce8e261341e373718e14a8e8e03 
>   clients/src/main/java/org/apache/kafka/clients/ClientUtils.java 
> 0d68bf1e1e90fe9d5d4397ddf817b9a9af8d9f7a 
>   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
> 2c421f42ed3fc5d61cf9c87a7eaa7bb23e26f63b 
>   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
> 48fe7961e2215372d8033ece4af739ea06c6457b 
>   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
> daff34db5bf2144e9dc274b23dc56b88f4efafdc 
>   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
> 951c34c92710fc4b38d656e99d2a41255c60aeb7 
>   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
> 5a37580ec69af08b97cf5b43b241790ba8c129dd 
>   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
> aa264202f2724907924985a5ecbe74afc4c6c04b 
>   clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java 
> bae528d31516679bed88ee61b408f209f185a8cc 
>   clients/src/main/java/org/apache/kafka/common/config/SSLConfigs.java 
> PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/network/Authenticator.java 
> PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/network/ByteBufferSend.java 
> df0e6d5105ca97b7e1cb4d334ffb7b443506bd0b 
>   clients/src/main/java/org/apache/kafka/common/network/Channel.java 
> PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/network/ChannelBuilder.java 
> PRE-CREATION 
>   
> clients/src/main/java/org/apache/kafka/common/network/DefaultAuthenticator.java
>  PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/network/NetworkReceive.java 
> 3ca0098b8ec8cfdf81158465b2d40afc47eb6f80 
>   
> clients/src/main/java/org/apache/kafka/common/network/PlainTextChannelBuilder.java

Re: Review Request 33620: Patch for KAFKA-1690

[Sriharsha Chintalapani]

> On June 30, 2015, 3:03 p.m., Jun Rao wrote:
> > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java,
> >  line 179
> > 
> >
> > Is this needed? It seems that the assumption is that if there is sth to 
> > write, SelectionKey.OP_WRITE should already be set and there is no need to 
> > set it again.

If the engine status is need_wrap than we need to write to the socketchannel.


> On June 30, 2015, 3:03 p.m., Jun Rao wrote:
> > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java,
> >  line 203
> > 
> >
> > Hmm, if write is false, we shouldn't write to the socket through flush, 
> > right?

The above condition is right but its confusing everyone. I added comments or if 
you I can just remove it.


> On June 30, 2015, 3:03 p.m., Jun Rao wrote:
> > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java,
> >  line 464
> > 
> >
> > Can write() be called if handshakeCompelete is false?

This is to guard during the renegotiation. But since we don't have 
renegotiation I can remove it.


> On June 30, 2015, 3:03 p.m., Jun Rao wrote:
> > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java,
> >  lines 473-477
> > 
> >
> > Can renegotiation happen in the middle of a write()? What happens to 
> > the byte that we haven't finished writing?

It can and we only append to netOutBuffer the unfinished bytes will stay intact 
and once the renegotiation finishes the writes will resume.


- Sriharsha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33620/#review89677
---


On June 23, 2015, 8:18 p.m., Sriharsha Chintalapani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33620/
> ---
> 
> (Updated June 23, 2015, 8:18 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1690
> https://issues.apache.org/jira/browse/KAFKA-1690
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. SSLFactory tests.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Added 
> PrincipalBuilder.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Addressing 
> reviews.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Addressing 
> reviews.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Addressing 
> reviews.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Fixed minor 
> issues with the patch.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client. Fixed minor 
> issues with the patch.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> KAFKA-1690. new java producer needs ssl support as a client.
> 
> 
> Merge remote-tracking branch 'refs/remotes/origin/trunk' into KAFKA-1690-V1
> 
> 
> KAFKA-1690. Broker side ssl changes.
> 
> 
> KAFKA-1684. SSL for socketServer.
> 
> 
> KAFKA-1690. Added SSLProducerSendTest and fixes to get right port for SSL.
> 
> 
> Merge branch 'trunk' into KAFKA-1690-V1
> 
> 
> KAFKA-1690. Post merge fixes.
> 
> 
> KAFKA-1690. Added SSLProducerSendTest.
> 
> 
> KAFKA-1690. Minor fixes based on patch review comments.
> 
> 
> Diffs
> -
> 
>   build.gradle 30d1cf2f1ff9ed3f86a060da8099bb0774b4cf91 
>   checkstyle/import-control.xml f2e6cec267e67ce8e261341e373718e14a8e8e03 
>   clients/src/main/java/org/apache/kafka/clients/ClientUtils.java 
> 0d68bf1e1e90fe9d5d4397ddf817b9a9af8d9f7a 
>   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
> 2c421f42ed3fc5d61cf9c87a7eaa7bb23e26f63b 
>   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
> 48fe7961e2215372d8033ece4af739ea06c6457b 
>   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
> daff34db5bf2144e9dc274b23dc56b88f4efafdc 
>   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
> 951c34c92710fc4b38d656e99d2a41255c60aeb7 
>   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
> 5a37580ec69af08b97cf5b43b241790ba8c129dd 
>   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
> aa264202f2724907924985

[jira] [Commented] (KAFKA-2100) Client Error doesn't preserve or display original server error code when it is an unknown code

[David Jacot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633066#comment-14633066
 ] 

David Jacot commented on KAFKA-2100:


[~gwenshap] Are you working on this one? I'd like to take it to familiarize 
myself with the contributing process.

> Client Error doesn't preserve or display original server error code when it 
> is an unknown code
> --
>
> Key: KAFKA-2100
> URL: https://issues.apache.org/jira/browse/KAFKA-2100
> Project: Kafka
>  Issue Type: Bug
>  Components: clients
>Reporter: Gwen Shapira
>Assignee: Gwen Shapira
>  Labels: newbie
>
> When the java client receives an unfamiliar error code, it translates it into 
> UNKNOWN(-1, new UnknownServerException("The server experienced an unexpected 
> error when processing the request"))
> This completely loses the original code, which makes troubleshooting from the 
> client impossible. 
> Will be better to preserve the original code and write it to the log when 
> logging the error.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)