Hello Team, I am pinging back on this thread to get a pair of eyes on this issue. Ticket: https://issues.apache.org/jira/browse/SPARK-48423
On Thu, 6 Jun 2024 at 00:19, Chhavi Bansal <meetchhavi1...@gmail.com> wrote: > Hello team, > I was exploring on how to save ML pipeline to azure blob storage, but was > setback by an issue where it complains of `fs.azure.account.key` not > being found in the configuration even when I have provided the values in > the pipelineModel.option(key1,value1) field. I considered raising a > ticket on spark https://issues.apache.org/jira/browse/SPARK-48423, where > I describe the entire scenario. I tried debugging the code and found that > this key is being explicitly asked for in the code. The only solution was > to again set it part of spark.conf which could result to a race condition > since we work on multi-tenant architecture. > > > > Since saving to Azure blob storage would be common, Can someone please > guide me if I am missing something in the `.option` clause? > > > > I would be happy to make a contribution to the code if someone can shed > some light on how this could be solved. > > -- > Thanks and Regards, > Chhavi Bansal > -- Thanks and Regards, Chhavi Bansal
