Hmmm....that's not working for me (no *.sources.jar files are generated). do I need to add a flag?
On Mon, Jun 17, 2013 at 5:47 PM, Joe Stein <crypt...@gmail.com> wrote: > Yup, depending on your Scala version > > ./sbt "++2.9.2 package" > > 2.8.0, 2.8.2, 2.9.1, 2.9.2 are available > > /* > Joe Stein, Chief Architect > http://www.medialets.com > Twitter: @allthingshadoop > Mobile: 917-597-9771 > */ > > On Jun 17, 2013, at 8:41 PM, Jason Rosenberg <j...@squareup.com> wrote: > > > Joe, > > > > I there also a way to generate a sources jar via sbt? > > > > Thanks, > > > > Jason > > > > > > On Mon, Jun 17, 2013 at 5:28 PM, Jason Rosenberg <j...@squareup.com> > wrote: > > > >> Another thing I notice, is that the sample 'config/server.properties' in > >> many cases seems to have no correlation to the default config > properties. > >> In some cases, it offers much greater allocation (e.g. for > >> socket.send.buffer.bytes, and for others much less. > >> > >> Have all the sample settings in config/server.properties been vetted for > >> the new config values? > >> > >> Jason > >> > >> > >> On Mon, Jun 17, 2013 at 5:23 PM, Joe Stein <crypt...@gmail.com> wrote: > >> > >>> you can use this to build the binary distributable > >>> > >>> ./sbt release-tar > >>> > >>> > >>> > >>> > >>> On Mon, Jun 17, 2013 at 8:17 PM, Jason Rosenberg <j...@squareup.com> > >>> wrote: > >>> > >>>> Looking at the convenience script for running kafka server, it builds > a > >>>> classpath by looking in several directories (looking at > >>>> kafka-run-class.sh). > >>>> > >>>> This script includes a jar under perf/target, should I care about > that, > >>>> when launching a mavenized version of the server? > >>>> > >>>> Also, it includes jars under the "libs/" directory, which doesn't > exist > >>>> (there is a "lib/') dir. Is that an oversight? > >>>> > >>>> Finally, it includes top-level jars that match kafka*.jar, which don't > >>>> exist from what I can tell. > >>>> > >>>> Or is there an extra sbt command I need to run to build out more > >>>> directories? > >>>> > >>>> Jason > >>>> > >>>> > >>>> On Mon, Jun 17, 2013 at 4:48 PM, Jason Rosenberg <j...@squareup.com> > >>> wrote: > >>>> > >>>>> Ok, > >>>>> > >>>>> So it seems the issue is related somehow to how I've wrapped the > >>> server > >>>>> (using a container app, using the maven pom from "./sbt make-pom"). > >>> If > >>>> I > >>>>> start the server using the script "kafka-server-start.sh", it works > >>> fine. > >>>>> Still looking (but haven't been able to see anything obvious so far). > >>>>> > >>>>> Jason > >>>>> > >>>>> > >>>>> On Sun, Jun 16, 2013 at 8:46 PM, Joe Stein <crypt...@gmail.com> > >>> wrote: > >>>>> > >>>>>> Jason, > >>>>>> > >>>>>> I just uploaded a signed binary build (kafka_2.8.2-0.8.0-beta1.jar) > >>> to > >>>> the > >>>>>> artifacts > >>>>>> http://people.apache.org/~joestein/kafka-0.8.0-beta1-candidate1/ > >>>>>> > >>>>>> I also uploaded a 2.9.2 binary too kafka_2.9.2-0.8.0-beta1.jar > >>>>>> > >>>>>> > >>>>>> On Sun, Jun 16, 2013 at 11:36 PM, Jason Rosenberg <j...@squareup.com > > > >>>>>> wrote: > >>>>>> > >>>>>>> Joe, > >>>>>>> > >>>>>>> So I am using the 2.8.2 build of the kafka jar, using that latest > >>>>>>> beta1-candidate1 tag. > >>>>>>> > >>>>>>> The code above should be all you need to reproduce the issue. I'll > >>>>>> create > >>>>>>> a JIRA ticket. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>>> Jason > >>>>>>> > >>>>>>> > >>>>>>> On Sun, Jun 16, 2013 at 8:32 PM, Joe Stein <crypt...@gmail.com> > >>>> wrote: > >>>>>>> > >>>>>>>> I have been running tests on 2.9.2 and 2.8.2 without any issues > >>> off > >>>>>> the > >>>>>>>> beta1-candidate1 release tag > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>> > >>> > https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=tag;h=refs/tags/0.8.0-beta1 > >>>>>>>> > >>>>>>>> Binaries will be built from the source file available from > >>> download > >>>>>> here > >>>>>>> : > >>>>>>>> http://people.apache.org/~joestein/kafka-0.8.0-beta1-candidate1/ > >>>>>>>> > >>>>>>>> It could be that test failing is interacting with a local file > >>>> already > >>>>>>>> there and causing issues or such type of thing not reproducible > >>>> maybe > >>>>>>>> without failing the test first to cause the data not cleaned up > >>> or > >>>>>>>> something, dunno. > >>>>>>>> > >>>>>>>> If you have an issue can you put the steps to reproduce it > >>> together > >>>> or > >>>>>>>> whatever amount of detailed information you can provide and then > >>>>>> please > >>>>>>>> open up a JIRA ticket > >>> https://issues.apache.org/jira/browse/KAFKA > >>>>>>>> > >>>>>>>> Thanks! > >>>>>>>> > >>>>>>>> > >>>>>>>> On Sun, Jun 16, 2013 at 11:14 PM, Jason Rosenberg < > >>> j...@squareup.com > >>>>> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Yep, > >>>>>>>>> > >>>>>>>>> The configs are good. And my apps are working fine too. It > >>> seems > >>>>>> only > >>>>>>>> to > >>>>>>>>> be an issue with this test (and a few others like it). > >>>>>>>>> > >>>>>>>>> Jason > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Sun, Jun 16, 2013 at 3:02 PM, Eric Sites < > >>>>>>> eric.si...@threattrack.com > >>>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Jason, > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Did you update your config file with the new name of the > >>>> zookeeper > >>>>>>>>>> settings: > >>>>>>>>>> > >>>>>>>>>> It was renamed from zk.connect to zookeeper.connect. > >>>>>>>>>> > >>>>>>>>>> You should check all of the settings because other setting > >>> names > >>>>>> have > >>>>>>>>>> changed as well. > >>>>>>>>>> > >>>>>>>>>> Cheers, > >>>>>>>>>> Eric Sites > >>>>>>>>>> > >>>>>>>>>> On 6/16/13 5:14 PM, "Jason Rosenberg" <j...@squareup.com> > >>> wrote: > >>>>>>>>>> > >>>>>>>>>>> I've started having problems with the latest version of the > >>> 0.8 > >>>>>>>> branch. > >>>>>>>>>>> The test below has started failing. It was working fine > >>> with > >>>> a > >>>>>>> prior > >>>>>>>>>>> version of 0.8, going back to Apr 30 > >>>>>>>>>>> (sha 988d4d8e65a14390abd748318a64e281e4a37c19). > >>>>>>>>>>> > >>>>>>>>>>> I haven't figured out when exactly it started failing, but I > >>>> saw > >>>>>> it > >>>>>>>>> with a > >>>>>>>>>>> version on Jun 9 (sha > >>> ddb7947c05583ea317e8f994f07b83bf6d5213c3) > >>>>>> and > >>>>>>>> now > >>>>>>>>>>> also with the latest (sha > >>>>>> 23acbd309f5e17de71db46cb6f1a60c8d38ea4e4). > >>>>>>>>>>> > >>>>>>>>>>> The test code is essentially this (assume a zk server is > >>>> running > >>>>>>> with > >>>>>>>>>>> 'zkConnect', and a kafka broker running with a metadata > >>> port at > >>>>>>>> 'port': > >>>>>>>>>>> > >>>>>>>>>>> Properties pProps = new Properties(); > >>>>>>>>>>> pProps.put("metadata.broker.list", "localhost:" + port); > >>>>>>>>>>> pProps.put("serializer.class", > >>>>>>> "kafka.serializer.StringEncoder"); > >>>>>>>>>>> ProducerConfig pConfig = new ProducerConfig(pProps); > >>>>>>>>>>> Producer<Integer, String> producer = new > >>> Producer<Integer, > >>>>>>>>>>> String>(pConfig); > >>>>>>>>>>> KeyedMessage<Integer, String> data = > >>>>>>>>>>> new KeyedMessage<Integer, String>("test-topic", > >>>>>>>> "test-message"); > >>>>>>>>>>> producer.send(data); > >>>>>>>>>>> producer.close(); > >>>>>>>>>>> > >>>>>>>>>>> Properties cProps = new Properties(); > >>>>>>>>>>> cProps.put("zookeeper.connect", zkConnect); > >>>>>>>>>>> cProps.put("group.id", "group1"); > >>>>>>>>>>> ConsumerConfig consumerConfig = new > >>> ConsumerConfig(cProps); > >>>>>>>>>>> ConsumerConnector consumerConnector = > >>>>>>>>>>> Consumer.createJavaConsumerConnector(consumerConfig); > >>>>>>>>>>> > >>>>>>>>>>> Map<String, List<KafkaStream<byte[], byte[]>>> > >>>>>>>> topicMessageStreams = > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>> > consumerConnector.createMessageStreams(ImmutableMap.of("test-topic", > >>>>>>>>> 1)); > >>>>>>>>>>> List<KafkaStream<byte[], byte[]>> streams = > >>>>>>>>>>> topicMessageStreams.get("test-topic"); > >>>>>>>>>>> final KafkaStream<byte[], byte[]> stream = > >>> streams.get(0); > >>>>>>>>>>> final ConsumerIterator<byte[], byte[]> iter = > >>>>>> stream.iterator(); > >>>>>>>>>>> > >>>>>>>>>>> // run in a separate thread > >>>>>>>>>>> final AtomicBoolean success = new AtomicBoolean(false); > >>>>>>>>>>> Thread consumeThread = new Thread(new Runnable() { > >>>>>>>>>>> public void run() { > >>>>>>>>>>> while (iter.hasNext()) { > >>>>>>>>>>> byte[] msg = iter.next().message(); > >>>>>>>>>>> String msgStr = new String(msg); > >>>>>>>>>>> success.set(msgStr.equals("test-message")); > >>>>>>>>>>> break; > >>>>>>>>>>> } > >>>>>>>>>>> } > >>>>>>>>>>> }); > >>>>>>>>>>> > >>>>>>>>>>> consumeThread.start(); > >>>>>>>>>>> // this now hangs with the latest code > >>>>>>>>>>> consumeThread.join(); > >>>>>>>>>>> > >>>>>>>>>>> consumerConnector.shutdown(); > >>>>>>>>>>> assertTrue(success.get()); > >>>>>>>>>>> > >>>>>>>>>>> The output looks like this: > >>>>>>>>>>> > >>>>>>>>>>> 912 [main] WARN kafka.producer.BrokerPartitionInfo - Error > >>>> while > >>>>>>>>> fetching > >>>>>>>>>>> metadata [{TopicMetadata for topic test-topic -> > >>>>>>>>>>> No partition metadata for topic test-topic due to > >>>>>>>>>>> kafka.common.LeaderNotAvailableException}] for topic > >>>>>> [test-topic]: > >>>>>>>> class > >>>>>>>>>>> kafka.common.LeaderNotAvailableException > >>>>>>>>>>> 922 [main] WARN kafka.producer.BrokerPartitionInfo - Error > >>>> while > >>>>>>>>> fetching > >>>>>>>>>>> metadata [{TopicMetadata for topic test-topic -> > >>>>>>>>>>> No partition metadata for topic test-topic due to > >>>>>>>>>>> kafka.common.LeaderNotAvailableException}] for topic > >>>>>> [test-topic]: > >>>>>>>> class > >>>>>>>>>>> kafka.common.LeaderNotAvailableException > >>>>>>>>>>> 923 [main] ERROR kafka.producer.async.DefaultEventHandler - > >>>>>> Failed > >>>>>>> to > >>>>>>>>>>> collate messages by topic, partition due to: Failed to fetch > >>>>>> topic > >>>>>>>>>>> metadata > >>>>>>>>>>> for topic: test-topic > >>>>>>>>>>> 980 [kafka-request-handler-2] WARN > >>>>>>>> kafka.server.HighwaterMarkCheckpoint > >>>>>>>>> - > >>>>>>>>>>> No highwatermark file is found. Returning 0 as the > >>>> highwatermark > >>>>>> for > >>>>>>>>>>> partition [test-topic,0] > >>>>>>>>>>> > >>>>>>>>>>> The consumer never receives a message, and so the test > >>>> hangs.... > >>>>>>>>>>> > >>>>>>>>>>> This test worked fine as I said with an older version of the > >>>>>> branch, > >>>>>>>> but > >>>>>>>>>>> it > >>>>>>>>>>> would output exceptions about LeaderNotAvailable, etc... > >>>>>>>>>>> > >>>>>>>>>>> Thoughts? > >>>>>>>>>>> > >>>>>>>>>>> Jason > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> > >>>>>>>> /* > >>>>>>>> Joe Stein > >>>>>>>> http://www.linkedin.com/in/charmalloc > >>>>>>>> Twitter: @allthingshadoop < > >>> http://www.twitter.com/allthingshadoop> > >>>>>>>> */ > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> > >>>>>> /* > >>>>>> Joe Stein > >>>>>> http://www.linkedin.com/in/charmalloc > >>>>>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > >>>>>> */ > >>>>>> > >>>>> > >>>>> > >>>> > >>> > >>> > >>> > >>> -- > >>> > >>> /* > >>> Joe Stein > >>> http://www.linkedin.com/in/charmalloc > >>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > >>> */ > >>> > >> > >> >
