Apache Livy seems to have a slow release cycle and activity. On the other hand few of the major Hadoop/Spark vendors seem to continue to support Livy in their products for example:
Cloudera added Livy to their latest CDP 7.x version and Livy was not available in prior versions like CDH 5.x/6.x: https://docs.cloudera.com/runtime/7.2.14/running-spark-applications/topics/spark-submitting-applications-using-livy.html Amazon EMR 6.x latest version has Livy: https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-livy.html Azure HDinsight new blog: https://docs.microsoft.com/en-us/azure/hdinsight/spark/apache-spark-livy-rest-interface Google Dataproc initialization: https://github.com/GoogleCloudDataproc/initialization-actions/tree/master/livy While Livy seems to be in competition with Databricks REST api for submitting Spark jobs, those vendors using Spark that don’t want to be dependent on Databricks may be continuing with Livy REST api. So Livy may still have some life at least for the time being imho. -----Original Message----- From: Mike Thomsen <[email protected]> Sent: Thursday, April 7, 2022 8:19 AM To: [email protected] Subject: Deprecate Livy support? https://github.com/apache/incubator-livy Looks like Livy hasn't had any updates in about two years. Maybe it's time to deprecate for it since it looks like it hasn't made any progress in moving out of the incubator.
