On 20 May 2017, at 01:44, Bajpai, Amit X. -ND <amit.x.bajpai....@disney.com<mailto:n...@disney.com>> wrote:
Hi, I have a hive external table with the S3 location having no files (but the S3 location directory does exists). When I am trying to use Spark SQL to count the number of records in the table it is throwing error saying “File s3n://data/xyz does not exist. null/0”. select * from tablex limit 10 Can someone let me know how we can fix this issue. Thanks There isn't really a "directory" in S3, just a set of objects whose paths begin with a string. Try creating an empty file with an _ prefix in the directory; it should be ignored by Spark SQL but will cause the "directory" to come into being
