I have seen this before on another release but cannot recall 1. *Initial Setup with Spark 3.5.2*: - I was using *Spark 3.5.2* recent upgrade, *Beeline 2.3.9*, and *Hadoop 3.1.1*. - The issue encountered was that the schema list was not visible, specifically when querying the database. The `SHOW TABLES` or database-related commands did not return expected results.
2. *Issue Details*: - The error indicated that the schema `DS` could not be found, despite the schema existing. This suggests a case sensitivity issue or a compatibility mismatch between Spark 3.5.2 and Hive/Hadoop setup. 3. *Solution by Downgrading Spark:* - *Downgraded Spark from 3.5.2 to 3.4.0*. - After downgrading to Spark 3.4.0, the schema names worked when forced to lowercase (e.g., using `ds` instead of `DS`), and I was able to successfully query the databases and schemas. This suggests that Spark 3.5.2 may handle case sensitivity or schema querying differently, which was incompatible with my setup. By downgrading to 3.4.0 and switching to lowercase schema names, the issue was resolved. *Summary:* - *Spark 3.5.2* caused issues with recognizing schema names, possibly due to case sensitivity or Hive metastore compatibility. - *Downgrading to Spark 3.4.0 *and using *lowercase schema names* resolved the issue, allowing the database and tables to be accessed correctly. I guess it is a good reminder that version compatibility between Spark, Hive, and Hadoop can sometimes cause issues, particularly with metastore interactions and case sensitivity. HTH Mich Talebzadeh, Architect | Data Engineer | Data Science | Financial Crime PhD <https://en.wikipedia.org/wiki/Doctor_of_Philosophy> Imperial College London <https://en.wikipedia.org/wiki/Imperial_College_London> London, United Kingdom view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* The information provided is correct to the best of my knowledge but of course cannot be guaranteed . It is essential to note that, as with any advice, quote "one test result is worth one-thousand expert opinions (Werner <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun <https://en.wikipedia.org/wiki/Wernher_von_Braun>)".
