A few other observations. 1. Spark 1.3.1 (custom built against HDP 2.2) was running fine against the same cluster and same hadoop configuration (hence seems like regression).
2. HA is enabled for YARN RM and HDFS (not sure if this would impact anything but wanted to share anyway). 3. Found this issue : https://issues.apache.org/jira/browse/SPARK-5837 and multiple references to other YARN issues to the same. Continuing to understand and explore the possibilities documented there. Regards, Nachiketa On Fri, Jun 26, 2015 at 12:52 AM, Nachiketa <nachiketa.shu...@gmail.com> wrote: > Spark 1.4.0 - Custom built from source against Hortonworks HDP 2.2 (hadoop > 2.6.0+) > HDP 2.2 Cluster (Secure, kerberos) > > spark-shell (--master yarn-client) launches fine and the prompt shows up. > Clicking on the Application Master url on the YARN RM UI, throws 500 > connect error. > > The same build works well against a non-secure cluster (same HDP > distribution). > > No debug logs or stack trace is easily visible. Where do I look for what > is going wrong ? And has anything changed in spark security that could be > contributing to this ? > > Thank you for your help with this. > > > Regards, > Nachiketa > -- Regards, -- Nachiketa
