Jeremiah, were you able to resolve this issue? - Prateek
On Wed, Mar 6, 2019 at 10:08 AM Prateek Maheshwari <prateek...@gmail.com> wrote: > Hi Jeremiah, > > The configuration you want to look for is: > 'job.systemstreampartition.grouper.factory'. It should default to: > 'org.apache.samza.container.grouper.stream.GroupByPartitionFactory'. > Can you check if you see this value in the configuration logged by > SamzaContainer during container start? You can grep for: "Using > configuration". > > For context, there are two groupers for a Samza job. One that groups input > partitions into tasks (this one), and one that groups tasks into containers > (the one you mentioned above). > > Thanks, > Prateek > > > > On Wed, Mar 6, 2019 at 8:14 AM Jeremiah Adams <jad...@helixeducation.com> > wrote: > >> It appears that the issue is related to the KafkaCheckpointLogKey.java >> constructor. grouperFactoryClassName here is null. THe documentation >> indicates that task.name.grouper.factory config setting has a default value >> of >> org.apache.samza.container.grouper.task.GroupByContainerCountFactory. I >> wouldn't expect it to be null here. >> >> If I specify GroupByContainerCountFactory for the >> task.name.grouper.factory in my properties file, I get a >> NoSuchMethodException: >> >> Exception in thread "main" java.lang.InstantiationException: >> org.apache.samza.container.grouper.task.GroupByContainerCount >> at java.lang.Class.newInstance(Class.java:427) >> at org.apache.samza.util.Util$.getObj(Util.scala:80) >> at >> org.apache.samza.coordinator.JobModelManager$.readJobModel(JobModelManager.scala:261) >> at >> org.apache.samza.coordinator.JobModelManager$.getJobModelManager(JobModelManager.scala:155) >> at >> org.apache.samza.coordinator.JobModelManager$.apply(JobModelManager.scala:117) >> at >> org.apache.samza.coordinator.JobModelManager.apply(JobModelManager.scala) >> at >> org.apache.samza.clustermanager.ClusterBasedJobCoordinator.buildJobModelManager(ClusterBasedJobCoordinator.java:241) >> at >> org.apache.samza.clustermanager.ClusterBasedJobCoordinator.<init>(ClusterBasedJobCoordinator.java:152) >> at >> org.apache.samza.clustermanager.ClusterBasedJobCoordinator.main(ClusterBasedJobCoordinator.java:297) >> Caused by: java.lang.NoSuchMethodException: >> org.apache.samza.container.grouper.task.GroupByContainerCount.<init>() >> at java.lang.Class.getConstructor0(Class.java:3082) >> at java.lang.Class.newInstance(Class.java:412) >> ... 8 more >> >> >> >> Jeremiah Adams >> Software Engineer >> www.helixeducation.com <http://www.helixeducation.com/> >> Blog <http://www.helixeducation.com/blog/> | Twitter < >> https://twitter.com/HelixEducation> | Facebook < >> https://www.facebook.com/HelixEducation> | LinkedIn < >> http://www.linkedin.com/company/3609946> >> >> >> On 3/4/19, 2:48 PM, "Jeremiah Adams" <jad...@helixeducation.com> wrote: >> >> I am updating dependencies and moving from Samza V0.13.0 to V0.14.0. >> I develop locally using the grid app in the hello-samza project to spin up >> local yarn/zookeeper/kafka instances. >> >> Grid is running these versions: >> kafka_2.11-0.10.2.1.tgz >> hadoop-2.6.1.tar.gz >> zookeeper-3.4.3.tar.gz >> >> >> My job is now failing with the NPE below. anyone have ideas on the >> cause of this error? >> >> >> 2019-03-04 14:13:49 AsyncRunLoop [ERROR] Task Partition 0 commit >> failed >> java.lang.NullPointerException >> at >> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:782) >> at >> org.apache.samza.checkpoint.kafka.KafkaCheckpointLogKey.<init>(KafkaCheckpointLogKey.java:46) >> at >> org.apache.samza.checkpoint.kafka.KafkaCheckpointManager.writeCheckpoint(KafkaCheckpointManager.scala:136) >> at >> org.apache.samza.checkpoint.OffsetManager.writeCheckpoint(OffsetManager.scala:259) >> at >> org.apache.samza.container.TaskInstance.commit(TaskInstance.scala:205) >> at >> org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker$5.run(AsyncRunLoop.java:494) >> at >> org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.commit(AsyncRunLoop.java:513) >> at >> org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.run(AsyncRunLoop.java:379) >> at >> org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.access$300(AsyncRunLoop.java:314) >> at >> org.apache.samza.task.AsyncRunLoop.runTasks(AsyncRunLoop.java:228) >> at >> org.apache.samza.task.AsyncRunLoop.run(AsyncRunLoop.java:157) >> at >> org.apache.samza.container.SamzaContainer.run(SamzaContainer.scala:728) >> at >> org.apache.samza.runtime.LocalContainerRunner.run(LocalContainerRunner.java:102) >> at >> org.apache.samza.runtime.LocalContainerRunner.main(LocalContainerRunner.java:147) >> 2019-03-04 14:13:49 AsyncRunLoop [ERROR] Caught throwable and >> stopping run loop >> >> >> >> Jeremiah Adams >> Software Engineer >> >> https://url.emailprotection.link/?ahfhEufaAWbezBrUFPG98ZJcterGfIerU3ZwsA3Gv_C0~ >> < >> https://url.emailprotection.link/?a49H2rNGIIBtQOw6md8OcHp-qKE3Xn2gNiZ3dlqAeSDA~ >> > >> Blog< >> https://url.emailprotection.link/?a49H2rNGIIBtQOw6md8OcHgFEZu-KYuiu8doY66NWwmmyWxz7kC-27Yfnbdgd2wyh5gjXUa6LMT_NRXsj1g1VVg~~> >> | Twitter< >> https://url.emailprotection.link/?a0Q7ct5_6cOdbJ86kpWB0zx6RbtgugTVC7lU_W7za50jLdZQGpLgVlR1V06zckSaM5oOKb6QBo46Qp9xt0Tt7Aw~~> >> | Facebook< >> https://url.emailprotection.link/?aAmyAO_nS_C1aDgBLeKyGTu0tksTt1_mn2PcS8KJXNJPM04iRHKgX96qGgENV-dMSER5wl8zDVRr3RsS0OmcF9A~~> >> | LinkedIn< >> https://url.emailprotection.link/?aanlcNI-cN74Gdz-TD332xAl6lHu7TRNICWoHUFjYf-KlBjrCGHoYR65b3rl-OyW10nWFv6hwYvUSoVHL4b3vGA~~ >> > >> >> >> >>
