I sent you a fake token and key From: Julien Laurenceau <julien.laurenc...@pepitedata.com> Sent: 23 August 2019 6:06 PM To: Krishna Chandran Nair <kcn...@qatarairways.com.qa> Cc: user@spark.apache.org Subject: [External]Re: error while connecting to azure blob storage
Hi Did you just publicly disclosed a real token to tour blob storage ? If so, please be aware that you share the data and maybe write permission... Refresh tout token ! Regards Le ven. 23 août 2019 à 08:33, Krishna Chandran Nair <kcn...@qatarairways.com.qa<mailto:kcn...@qatarairways.com.qa>> a écrit : Hi Team, I have written a small code to connect to azure blob storage but go error. I have attached the error log. Please help Calling command -- ./spark-submit stg.py --jars /home/citus/spark/spark-2.3.3-bin-hadoop2.7/jars/hadoop-azure-3.2.0.jar,/home/citus/spark/spark-2.3.3-bin-hadoop2.7/jars/azure-storage-8.4.0.jar Code vi ~/spark/spark-2.3.3-bin-hadoop2.7/bin/stg.py from pyspark import SparkContext from pyspark.sql import SparkSession from pyspark.sql import DataFrameReader from pyspark.sql import SparkSession session = SparkSession.builder.getOrCreate() #session.conf.set("fs.azure", "org.apache.hadoop.fs.azure.NativeAzureFileSystem") #session.conf.set("fs.hdfs.impl", "org.apache.hadoop.hdfs.DistributedFileSystem") #session.conf.set("fs.file.impl", "org.apache.hadoop.fs.LocalFileSystem") #session.conf.set( # "fs.azure.sas.snowflakestrg.blob.core.windows.net/test<http://fs.azure.sas.snowflakestrg.blob.core.windows.net/test>", # "?sv=2018-03-28&ss=bfqt&srt=sco&sp=rwdlacup&se=2020-01-01T16:37:05Z&st=2019-08-13T08:37:05Z&spr=https&sig=BgTl8mibE%2B%2BTTIMG4dKR17NnGinMWEVTtn888MD8PT4%3D" #) session.conf.set( "fs.azure.account.key.snowflakestrg.blob.core.windows.net<http://fs.azure.account.key.snowflakestrg.blob.core.windows.net>", "LIWCYzrJOS4hs0DiQH6fAzjuBnuj/F8myVmJImomEqOqlAV4pSt7KWfr24mj0saaOTVNZkGTKUn41k4e9hqKSA==") df=session.read.csv("wasbs://t...@snowflakestrg.blob.core.windows.net/users.csv<http://t...@snowflakestrg.blob.core.windows.net/users.csv>") df.show(5) Qatar Airways - Going Places Together [OW LOGO] Disclaimer:- This message (including attachments) is intended solely for the addressee named above. It may be confidential, privileged, subject to copyright, trade secret, or other legal rules and may not be forwarded without the author's permission. If you are not the addressee you must not read, copy or disseminate this message. If you have received it in error please notify the sender immediately and delete the message from all storage devices. Any opinions expressed in this message do not necessarily represent the official positions of Qatar Airways. Any agreements (including any warranties, representations, or offers) concluded with Qatar Airways by using electronic correspondence shall only come into existence if an authorized representative of Qatar Airways has explicitly approved such contract formation. To the fullest extent permissible by law, Qatar Airways disclaim all liability for loss or damage to person or property arising from this message being infected by computer virus or other contamination.
