Yup, depending on your Scala version ./sbt "++2.9.2 package"
2.8.0, 2.8.2, 2.9.1, 2.9.2 are available /* Joe Stein, Chief Architect http://www.medialets.com Twitter: @allthingshadoop Mobile: 917-597-9771 */ On Jun 17, 2013, at 8:41 PM, Jason Rosenberg <j...@squareup.com> wrote: > Joe, > > I there also a way to generate a sources jar via sbt? > > Thanks, > > Jason > > > On Mon, Jun 17, 2013 at 5:28 PM, Jason Rosenberg <j...@squareup.com> wrote: > >> Another thing I notice, is that the sample 'config/server.properties' in >> many cases seems to have no correlation to the default config properties. >> In some cases, it offers much greater allocation (e.g. for >> socket.send.buffer.bytes, and for others much less. >> >> Have all the sample settings in config/server.properties been vetted for >> the new config values? >> >> Jason >> >> >> On Mon, Jun 17, 2013 at 5:23 PM, Joe Stein <crypt...@gmail.com> wrote: >> >>> you can use this to build the binary distributable >>> >>> ./sbt release-tar >>> >>> >>> >>> >>> On Mon, Jun 17, 2013 at 8:17 PM, Jason Rosenberg <j...@squareup.com> >>> wrote: >>> >>>> Looking at the convenience script for running kafka server, it builds a >>>> classpath by looking in several directories (looking at >>>> kafka-run-class.sh). >>>> >>>> This script includes a jar under perf/target, should I care about that, >>>> when launching a mavenized version of the server? >>>> >>>> Also, it includes jars under the "libs/" directory, which doesn't exist >>>> (there is a "lib/') dir. Is that an oversight? >>>> >>>> Finally, it includes top-level jars that match kafka*.jar, which don't >>>> exist from what I can tell. >>>> >>>> Or is there an extra sbt command I need to run to build out more >>>> directories? >>>> >>>> Jason >>>> >>>> >>>> On Mon, Jun 17, 2013 at 4:48 PM, Jason Rosenberg <j...@squareup.com> >>> wrote: >>>> >>>>> Ok, >>>>> >>>>> So it seems the issue is related somehow to how I've wrapped the >>> server >>>>> (using a container app, using the maven pom from "./sbt make-pom"). >>> If >>>> I >>>>> start the server using the script "kafka-server-start.sh", it works >>> fine. >>>>> Still looking (but haven't been able to see anything obvious so far). >>>>> >>>>> Jason >>>>> >>>>> >>>>> On Sun, Jun 16, 2013 at 8:46 PM, Joe Stein <crypt...@gmail.com> >>> wrote: >>>>> >>>>>> Jason, >>>>>> >>>>>> I just uploaded a signed binary build (kafka_2.8.2-0.8.0-beta1.jar) >>> to >>>> the >>>>>> artifacts >>>>>> http://people.apache.org/~joestein/kafka-0.8.0-beta1-candidate1/ >>>>>> >>>>>> I also uploaded a 2.9.2 binary too kafka_2.9.2-0.8.0-beta1.jar >>>>>> >>>>>> >>>>>> On Sun, Jun 16, 2013 at 11:36 PM, Jason Rosenberg <j...@squareup.com> >>>>>> wrote: >>>>>> >>>>>>> Joe, >>>>>>> >>>>>>> So I am using the 2.8.2 build of the kafka jar, using that latest >>>>>>> beta1-candidate1 tag. >>>>>>> >>>>>>> The code above should be all you need to reproduce the issue. I'll >>>>>> create >>>>>>> a JIRA ticket. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Jason >>>>>>> >>>>>>> >>>>>>> On Sun, Jun 16, 2013 at 8:32 PM, Joe Stein <crypt...@gmail.com> >>>> wrote: >>>>>>> >>>>>>>> I have been running tests on 2.9.2 and 2.8.2 without any issues >>> off >>>>>> the >>>>>>>> beta1-candidate1 release tag >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >>> https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=tag;h=refs/tags/0.8.0-beta1 >>>>>>>> >>>>>>>> Binaries will be built from the source file available from >>> download >>>>>> here >>>>>>> : >>>>>>>> http://people.apache.org/~joestein/kafka-0.8.0-beta1-candidate1/ >>>>>>>> >>>>>>>> It could be that test failing is interacting with a local file >>>> already >>>>>>>> there and causing issues or such type of thing not reproducible >>>> maybe >>>>>>>> without failing the test first to cause the data not cleaned up >>> or >>>>>>>> something, dunno. >>>>>>>> >>>>>>>> If you have an issue can you put the steps to reproduce it >>> together >>>> or >>>>>>>> whatever amount of detailed information you can provide and then >>>>>> please >>>>>>>> open up a JIRA ticket >>> https://issues.apache.org/jira/browse/KAFKA >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Jun 16, 2013 at 11:14 PM, Jason Rosenberg < >>> j...@squareup.com >>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Yep, >>>>>>>>> >>>>>>>>> The configs are good. And my apps are working fine too. It >>> seems >>>>>> only >>>>>>>> to >>>>>>>>> be an issue with this test (and a few others like it). >>>>>>>>> >>>>>>>>> Jason >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Jun 16, 2013 at 3:02 PM, Eric Sites < >>>>>>> eric.si...@threattrack.com >>>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Jason, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Did you update your config file with the new name of the >>>> zookeeper >>>>>>>>>> settings: >>>>>>>>>> >>>>>>>>>> It was renamed from zk.connect to zookeeper.connect. >>>>>>>>>> >>>>>>>>>> You should check all of the settings because other setting >>> names >>>>>> have >>>>>>>>>> changed as well. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Eric Sites >>>>>>>>>> >>>>>>>>>> On 6/16/13 5:14 PM, "Jason Rosenberg" <j...@squareup.com> >>> wrote: >>>>>>>>>> >>>>>>>>>>> I've started having problems with the latest version of the >>> 0.8 >>>>>>>> branch. >>>>>>>>>>> The test below has started failing. It was working fine >>> with >>>> a >>>>>>> prior >>>>>>>>>>> version of 0.8, going back to Apr 30 >>>>>>>>>>> (sha 988d4d8e65a14390abd748318a64e281e4a37c19). >>>>>>>>>>> >>>>>>>>>>> I haven't figured out when exactly it started failing, but I >>>> saw >>>>>> it >>>>>>>>> with a >>>>>>>>>>> version on Jun 9 (sha >>> ddb7947c05583ea317e8f994f07b83bf6d5213c3) >>>>>> and >>>>>>>> now >>>>>>>>>>> also with the latest (sha >>>>>> 23acbd309f5e17de71db46cb6f1a60c8d38ea4e4). >>>>>>>>>>> >>>>>>>>>>> The test code is essentially this (assume a zk server is >>>> running >>>>>>> with >>>>>>>>>>> 'zkConnect', and a kafka broker running with a metadata >>> port at >>>>>>>> 'port': >>>>>>>>>>> >>>>>>>>>>> Properties pProps = new Properties(); >>>>>>>>>>> pProps.put("metadata.broker.list", "localhost:" + port); >>>>>>>>>>> pProps.put("serializer.class", >>>>>>> "kafka.serializer.StringEncoder"); >>>>>>>>>>> ProducerConfig pConfig = new ProducerConfig(pProps); >>>>>>>>>>> Producer<Integer, String> producer = new >>> Producer<Integer, >>>>>>>>>>> String>(pConfig); >>>>>>>>>>> KeyedMessage<Integer, String> data = >>>>>>>>>>> new KeyedMessage<Integer, String>("test-topic", >>>>>>>> "test-message"); >>>>>>>>>>> producer.send(data); >>>>>>>>>>> producer.close(); >>>>>>>>>>> >>>>>>>>>>> Properties cProps = new Properties(); >>>>>>>>>>> cProps.put("zookeeper.connect", zkConnect); >>>>>>>>>>> cProps.put("group.id", "group1"); >>>>>>>>>>> ConsumerConfig consumerConfig = new >>> ConsumerConfig(cProps); >>>>>>>>>>> ConsumerConnector consumerConnector = >>>>>>>>>>> Consumer.createJavaConsumerConnector(consumerConfig); >>>>>>>>>>> >>>>>>>>>>> Map<String, List<KafkaStream<byte[], byte[]>>> >>>>>>>> topicMessageStreams = >>>>>>>>>>> >>>>>>>>>> >>>>>>> consumerConnector.createMessageStreams(ImmutableMap.of("test-topic", >>>>>>>>> 1)); >>>>>>>>>>> List<KafkaStream<byte[], byte[]>> streams = >>>>>>>>>>> topicMessageStreams.get("test-topic"); >>>>>>>>>>> final KafkaStream<byte[], byte[]> stream = >>> streams.get(0); >>>>>>>>>>> final ConsumerIterator<byte[], byte[]> iter = >>>>>> stream.iterator(); >>>>>>>>>>> >>>>>>>>>>> // run in a separate thread >>>>>>>>>>> final AtomicBoolean success = new AtomicBoolean(false); >>>>>>>>>>> Thread consumeThread = new Thread(new Runnable() { >>>>>>>>>>> public void run() { >>>>>>>>>>> while (iter.hasNext()) { >>>>>>>>>>> byte[] msg = iter.next().message(); >>>>>>>>>>> String msgStr = new String(msg); >>>>>>>>>>> success.set(msgStr.equals("test-message")); >>>>>>>>>>> break; >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> }); >>>>>>>>>>> >>>>>>>>>>> consumeThread.start(); >>>>>>>>>>> // this now hangs with the latest code >>>>>>>>>>> consumeThread.join(); >>>>>>>>>>> >>>>>>>>>>> consumerConnector.shutdown(); >>>>>>>>>>> assertTrue(success.get()); >>>>>>>>>>> >>>>>>>>>>> The output looks like this: >>>>>>>>>>> >>>>>>>>>>> 912 [main] WARN kafka.producer.BrokerPartitionInfo - Error >>>> while >>>>>>>>> fetching >>>>>>>>>>> metadata [{TopicMetadata for topic test-topic -> >>>>>>>>>>> No partition metadata for topic test-topic due to >>>>>>>>>>> kafka.common.LeaderNotAvailableException}] for topic >>>>>> [test-topic]: >>>>>>>> class >>>>>>>>>>> kafka.common.LeaderNotAvailableException >>>>>>>>>>> 922 [main] WARN kafka.producer.BrokerPartitionInfo - Error >>>> while >>>>>>>>> fetching >>>>>>>>>>> metadata [{TopicMetadata for topic test-topic -> >>>>>>>>>>> No partition metadata for topic test-topic due to >>>>>>>>>>> kafka.common.LeaderNotAvailableException}] for topic >>>>>> [test-topic]: >>>>>>>> class >>>>>>>>>>> kafka.common.LeaderNotAvailableException >>>>>>>>>>> 923 [main] ERROR kafka.producer.async.DefaultEventHandler - >>>>>> Failed >>>>>>> to >>>>>>>>>>> collate messages by topic, partition due to: Failed to fetch >>>>>> topic >>>>>>>>>>> metadata >>>>>>>>>>> for topic: test-topic >>>>>>>>>>> 980 [kafka-request-handler-2] WARN >>>>>>>> kafka.server.HighwaterMarkCheckpoint >>>>>>>>> - >>>>>>>>>>> No highwatermark file is found. Returning 0 as the >>>> highwatermark >>>>>> for >>>>>>>>>>> partition [test-topic,0] >>>>>>>>>>> >>>>>>>>>>> The consumer never receives a message, and so the test >>>> hangs.... >>>>>>>>>>> >>>>>>>>>>> This test worked fine as I said with an older version of the >>>>>> branch, >>>>>>>> but >>>>>>>>>>> it >>>>>>>>>>> would output exceptions about LeaderNotAvailable, etc... >>>>>>>>>>> >>>>>>>>>>> Thoughts? >>>>>>>>>>> >>>>>>>>>>> Jason >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> /* >>>>>>>> Joe Stein >>>>>>>> http://www.linkedin.com/in/charmalloc >>>>>>>> Twitter: @allthingshadoop < >>> http://www.twitter.com/allthingshadoop> >>>>>>>> */ >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> /* >>>>>> Joe Stein >>>>>> http://www.linkedin.com/in/charmalloc >>>>>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> >>>>>> */ >>>>>> >>>>> >>>>> >>>> >>> >>> >>> >>> -- >>> >>> /* >>> Joe Stein >>> http://www.linkedin.com/in/charmalloc >>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> >>> */ >>> >> >>
