Hi Tommy, Yi and I discussed about it and initially, we thought it could have something to do with the topic auto-creation setting on your Kafka server. Is it enabled or disabled in your case?
I kind of suspect that the request timeout is insufficient. However, we do have retries on Samza to fetch the metadata. So, even if topic does get auto-created and metadata fetch is delayed, it will try to fetch the metadata again. Not very clear why SAMZA-971 has anything to do with this. That JIRA just reduces the number of calls we make to the broker. Another question, are you able to reproduce this issue ? Thanks! Navina On Wed, Sep 14, 2016 at 1:33 PM, Tommy Becker <tobec...@tivo.com> wrote: > Thanks for the response, and done. > > https://issues.apache.org/jira/browse/SAMZA-1018 > > On 09/14/2016 01:14 PM, Yi Pan wrote: > > Hi, Tommy, > > Could you open a JIRA for this one? Also, could you include the Kafka > broker version in this test? > > Thanks! > > -Yi > > On Wed, Sep 14, 2016 at 6:06 AM, Tommy Becker <tobec...@tivo.com><mailto: > tobec...@tivo.com> wrote: > > > > We are testing an upgrade to 0.10.1 from 0.9.1 and noticed a regression. > When starting a stream job that consumes a topic that does not yet exist, > the job dies with the following exception: > > Exception in thread "main" java.lang.IllegalArgumentException: No tasks > found. Likely due to no input partitions. Can't run a job with no tasks. > at org.apache.samza.container.grouper.task.GroupByContainerCoun > t.validateTasks(GroupByContainerCount.java:193) > at org.apache.samza.container.grouper.task.GroupByContainerCoun > t.balance(GroupByContainerCount.java:86) > at org.apache.samza.coordinator.JobModelManager$.refreshJobMode > l(JobCoordinator.scala:278) > at org.apache.samza.coordinator.JobModelManager$.jobModelGenera > tor$1(JobCoordinator.scala:211) > at org.apache.samza.coordinator.JobModelManager$.initializeJobM > odel(JobCoordinator.scala:217) > at org.apache.samza.coordinator.JobModelManager$.getJobCoordina > tor(JobCoordinator.scala:122) > at org.apache.samza.coordinator.JobModelManager$.apply(JobCoord > inator.scala:106) > at org.apache.samza.coordinator.JobModelManager$.apply(JobCoord > inator.scala:112) > at org.apache.samza.job.local.ThreadJobFactory.getJob(ThreadJob > Factory.scala:40) > at org.apache.samza.job.JobRunner.run(JobRunner.scala:129) > at org.apache.samza.job.JobRunner$.main(JobRunner.scala:66) > at org.apache.samza.job.JobRunner.main(JobRunner.scala) > > > > > > The root cause seems to be commit 920f803a2e3dab809f4d7bb518259b0f4164407f > from SAMZA-971. From what I can tell passing partitionsMetadataOnly = true > to the StreamMetadataCache in JobModelManager#getInputStreamPartitions is > what's causing this this behavior. The input topic is still created, but > the proper partition metadata is not returned, resulting in an empty set > being returned. The behavior of Kafka here is screwy, but this still seems > like a regression. The old behavior is nice because it doesn't require that > producer systems come up before the stream processors. > > -- > Tommy Becker > Senior Software Engineer > > Digitalsmiths > A TiVo Company > > www.digitalsmiths.com<http://www.digitalsmiths.com><http://w > ww.digitalsmiths.com><http://www.digitalsmiths.com> > tobec...@tivo.com<mailto:tobec...@tivo.com><mailto:tobec...@tivo.com > ><mailto:tobec...@tivo.com> > > > ________________________________ > > This email and any attachments may contain confidential and privileged > material for the sole use of the intended recipient. Any review, copying, > or distribution of this email (or any attachments) by others is prohibited. > If you are not the intended recipient, please contact the sender > immediately and permanently delete this email and any attachments. No > employee or agent of TiVo Inc. is authorized to conclude any binding > agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo > Inc. may only be made by a signed written agreement. > > > > > > > -- > Tommy Becker > Senior Software Engineer > > Digitalsmiths > A TiVo Company > > www.digitalsmiths.com<http://www.digitalsmiths.com> > tobec...@tivo.com<mailto:tobec...@tivo.com> > > ________________________________ > > This email and any attachments may contain confidential and privileged > material for the sole use of the intended recipient. Any review, copying, > or distribution of this email (or any attachments) by others is prohibited. > If you are not the intended recipient, please contact the sender > immediately and permanently delete this email and any attachments. No > employee or agent of TiVo Inc. is authorized to conclude any binding > agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo > Inc. may only be made by a signed written agreement. > -- Navina R.
