SPARK-23221 fixes an issue specific to KafkaContinuousSourceStressForDontFailOnDataLossSuite; I don't think it could cause other suites to deadlock.
Do note that the previous hang issues we saw caused by SPARK-23055 were correctly marked as failures. On Thu, Jan 25, 2018 at 3:40 PM, Shixiong(Ryan) Zhu <[email protected] > wrote: > + Jose > > On Thu, Jan 25, 2018 at 2:18 PM, Dongjoon Hyun <[email protected]> > wrote: > >> SPARK-23221 is one of the reasons for Kafka-test-suite deadlock issue. >> >> For the hang issues, it seems not to be marked as a failure correctly in >> Apache Spark Jenkins history. >> >> >> On Thu, Jan 25, 2018 at 1:03 PM, Marcelo Vanzin <[email protected]> >> wrote: >> >>> On Thu, Jan 25, 2018 at 12:29 PM, Sean Owen <[email protected]> wrote: >>> > I am still seeing these tests fail or hang: >>> > >>> > - subscribing topic by name from earliest offsets (failOnDataLoss: >>> false) >>> > - subscribing topic by name from earliest offsets (failOnDataLoss: >>> true) >>> >>> This is something that we are seeing internally on a different version >>> Spark, and we're currently investigating with our Kafka people. Not >>> sure it's the same issue (we have a newer version of Kafka libraries), >>> but this is just another way of saying that I don't think those hangs >>> are new in 2.3, at least. >>> >>> -- >>> Marcelo >>> >>> --------------------------------------------------------------------- >>> To unsubscribe e-mail: [email protected] >>> >>> >> >
