Hi All, I am reaching out to seek guidance from the community regarding a code integration puzzle.
The architecture that we are currently pursuing for the MATLAB interface to Arrow [1] involves dispatching to the Arrow C++ libraries using MEX (a MATLAB facility for calling C/C++ code [2]). A major challenge with this approach has been keeping Arrow C++ objects (e.g. arrow::Array) alive in memory for the appropriate amount of time and making it easy to interface with them from MATLAB. MATLAB has a recommended solution for this problem [3]. However, we've been pursuing a MEX-based solution due to the pervasiveness of MEX and its familiarity to MATLAB users. Our hope is that using MEX will make it easy for others to contribute to the MATLAB interface. To help maintain the connection between MATLAB objects and C++, we've been experimenting with a MEX-based object dispatch layer. The primary goal of this work is to unblock development of the MATLAB interface to Arrow. However, this object dispatch layer is non-trivial and ultimately unrelated to the Arrow project's core mission. Therefore, submitting this code to the Arrow project doesn't seem like the optimal code integration strategy. We’ve been considering the possibility of creating a new open-source repository under the MathWorks GitHub organization [4] to host the object dispatch layer (a side effect of this approach is that it may help encourage reuse of this infrastructure in future open-source MATLAB projects). However, this approach would come with notable tradeoffs: 1. We would need to follow the ASF IP Clearance Process [5] to integrate this code into the Arrow project (it's possible we are mistaken about this). 2. It's not obvious how we should keep the code in sync. Would it be possible to use a git submodule [6] to "symlink" to the external repo? 3. What about licensing? Does the code need to be Apache licensed, or would it be possible to use another Apache-compatible license [7], like BSD? BSD is the default choice for new projects hosted under the MathWorks GitHub organization. Admittedly, we aren't sure what the best path forward is, so we appreciate the community's guidance. We welcome any suggestions. [1] https://github.com/apache/arrow/tree/master/matlab [2] https://www.mathworks.com/help/matlab/call-mex-files-1.html [3] https://www.mathworks.com/help/matlab/build-matlab-interface-to-c-library.html [4] https://github.com/mathworks [5] https://incubator.apache.org/ip-clearance/ [6] https://github.blog/2016-02-01-working-with-submodules/ [7] https://www.apache.org/legal/resolved.html#category-a Thank you, Kevin Gurney
