RE: Usage of JDK Vector API in ML/MLLib

2020-12-16 Thread Ludovic Henry
3 To: Sean Owen<mailto:sro...@gmail.com> Cc: Ludovic Henry<mailto:luhe...@microsoft.com>; dev@spark.apache.org<mailto:dev@spark.apache.org>; Bernhard Urban-Forster<mailto:beu...@microsoft.com> Subject: Re: Usage of JDK Vector API in ML/MLLib Regarding selective compilation

Re: Usage of JDK Vector API in ML/MLLib

2020-12-16 Thread Sean Owen
ntations on my end. > > > > Thank you, > > Ludovic > > > > [1] https://github.com/luhenry/blas > > > > *From: *Erik Krogen > *Sent: *Tuesday, 15 December 2020 17:33 > *To: *Sean Owen > *Cc: *Ludovic Henry ; dev@spark.apache.org; Bernhard > Urban-

RE: Usage of JDK Vector API in ML/MLLib

2020-12-16 Thread Ludovic Henry
<mailto:luhe...@microsoft.com>; dev@spark.apache.org<mailto:dev@spark.apache.org>; Bernhard Urban-Forster<mailto:beu...@microsoft.com> Subject: Re: Usage of JDK Vector API in ML/MLLib Regarding selective compilation, you can hide sources behind a Maven profile such as `-Pvector

Re: Usage of JDK Vector API in ML/MLLib

2020-12-15 Thread Erik Krogen
Regarding selective compilation, you can hide sources behind a Maven profile such as `-Pvectorized`. Check out what we do to switch between the `hive-1.2` and `hive-2.3` profiles where different source directories are grabbed at compile-time (the hive-1.2 profile was recently removed so you might h

Re: Usage of JDK Vector API in ML/MLLib

2020-12-15 Thread Sean Owen
Yes it's intriguing, though as you say not readily available in the wild yet. I would also expect native BLAS to outperform f2j also, so yeah that's the interesting question, whether this is a win over native code or not. I suppose the upside is eventually, we may expect this API to be available in

Usage of JDK Vector API in ML/MLLib

2020-12-15 Thread Ludovic Henry
Hello, I’ve, over the past few days, looked into using the new Vector API [1] to accelerate some BLAS operations straight from Java. You can find a gist at [2] containing most of the changes in mllib-local/src/main/scala/org/apache/spark/ml/linalg/BLAS.scala. To measure performance, I’ve a