Hi, Tommy, Could you open a JIRA for this one? Also, could you include the Kafka broker version in this test?
Thanks! -Yi On Wed, Sep 14, 2016 at 6:06 AM, Tommy Becker <tobec...@tivo.com> wrote: > We are testing an upgrade to 0.10.1 from 0.9.1 and noticed a regression. > When starting a stream job that consumes a topic that does not yet exist, > the job dies with the following exception: > > Exception in thread "main" java.lang.IllegalArgumentException: No tasks > found. Likely due to no input partitions. Can't run a job with no tasks. > at org.apache.samza.container.grouper.task.GroupByContainerCoun > t.validateTasks(GroupByContainerCount.java:193) > at org.apache.samza.container.grouper.task.GroupByContainerCoun > t.balance(GroupByContainerCount.java:86) > at org.apache.samza.coordinator.JobModelManager$.refreshJobMode > l(JobCoordinator.scala:278) > at org.apache.samza.coordinator.JobModelManager$.jobModelGenera > tor$1(JobCoordinator.scala:211) > at org.apache.samza.coordinator.JobModelManager$.initializeJobM > odel(JobCoordinator.scala:217) > at org.apache.samza.coordinator.JobModelManager$.getJobCoordina > tor(JobCoordinator.scala:122) > at org.apache.samza.coordinator.JobModelManager$.apply(JobCoord > inator.scala:106) > at org.apache.samza.coordinator.JobModelManager$.apply(JobCoord > inator.scala:112) > at org.apache.samza.job.local.ThreadJobFactory.getJob(ThreadJob > Factory.scala:40) > at org.apache.samza.job.JobRunner.run(JobRunner.scala:129) > at org.apache.samza.job.JobRunner$.main(JobRunner.scala:66) > at org.apache.samza.job.JobRunner.main(JobRunner.scala) > > > > > > The root cause seems to be commit 920f803a2e3dab809f4d7bb518259b0f4164407f > from SAMZA-971. From what I can tell passing partitionsMetadataOnly = true > to the StreamMetadataCache in JobModelManager#getInputStreamPartitions is > what's causing this this behavior. The input topic is still created, but > the proper partition metadata is not returned, resulting in an empty set > being returned. The behavior of Kafka here is screwy, but this still seems > like a regression. The old behavior is nice because it doesn't require that > producer systems come up before the stream processors. > > -- > Tommy Becker > Senior Software Engineer > > Digitalsmiths > A TiVo Company > > www.digitalsmiths.com<http://www.digitalsmiths.com> > tobec...@tivo.com<mailto:tobec...@tivo.com> > > ________________________________ > > This email and any attachments may contain confidential and privileged > material for the sole use of the intended recipient. Any review, copying, > or distribution of this email (or any attachments) by others is prohibited. > If you are not the intended recipient, please contact the sender > immediately and permanently delete this email and any attachments. No > employee or agent of TiVo Inc. is authorized to conclude any binding > agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo > Inc. may only be made by a signed written agreement. >
