Re: Log4j 1.2.17 spark CVE

2021-12-12 Thread Holden Karau
My understanding is it only applies to log4j 2+ so we don’t need to do anything. On Sun, Dec 12, 2021 at 8:46 PM Pralabh Kumar wrote: > Hi developers, users > > Spark is built using log4j 1.2.17 . Is there a plan to upgrade based on > recent CVE detected ? > > > Regards > Pralabh kumar > -- Tw

Re: Log4j 1.2.17 spark CVE

2021-12-12 Thread Sean Owen
Check the CVE - the log4j vulnerability appears to affect log4j 2, not 1.x. There was mention that it could affect 1.x when used with JNDI or SMS handlers, but Spark does neither. (unless anyone can think of something I'm missing, but never heard or seen that come up at all in 7 years in Spark) Th

Log4j 1.2.17 spark CVE

2021-12-12 Thread Pralabh Kumar
Hi developers, users Spark is built using log4j 1.2.17 . Is there a plan to upgrade based on recent CVE detected ? Regards Pralabh kumar

Re: Hi Team, I put a UDF-Utils jar on Google Cloud Storage, but I can't run it

2021-12-12 Thread bo zhao
Update. My Spark version is 2.3. bo zhao 于2021年12月13日周一 11:24写道: > Hi Team, > I'm doing migration from onPerm to Google Cloud Platform. I have a > UDF-Uitls jar including several udf functions. when code on Perm running > by Spark just need "add jar hdfs://x/home/path/UDF-Uitls.jar" with > Spar

Hi Team, I put a UDF-Utils jar on Google Cloud Storage, but I can't run it

2021-12-12 Thread bo zhao
Hi Team, I'm doing migration from onPerm to Google Cloud Platform. I have a UDF-Uitls jar including several udf functions. when code on Perm running by Spark just need "add jar hdfs://x/home/path/UDF-Uitls.jar" with SparkSQL. Now I put this jar on cloud storage. when I run SparkSQL with "add jar g

Re: [Proposal] Deprecate Trigger.Once and replace with Trigger.AvailableNow

2021-12-12 Thread Jungtaek Lim
Friendly reminder. I'll submit the proposed change if there is no objection observed this week. On Wed, Dec 8, 2021 at 4:16 PM Jungtaek Lim wrote: > Hi dev, > > I would like to hear voices about deprecating Trigger.Once, and replacing > it with Trigger.AvailableNow [1] in Structured Streaming. >