Jeurgen, and other GNU APL experts. I am exploring neural nets, word2vec and some other AI related areas.
Right now, I want to tie in google's word2vec trained models (the billion word one GoogleNews-vectors-negative300.bin.gz) This is a binary file containing a lot of floating point data -- about 3.5GB of data. These are words, followed by cosine distances. I could attempt to feed this in slow way, and put it into an APL workspace. But... I also intend on attempting to feed the data to a GPU. So, what I am looking for is a modification to GNU APL (and yes, I am willing to do the work) -- to allow for the complete suppression of normal C++ allocations, etc. and allow the introduction of simple float/double vectors or matrices (helpful to allow "C"-ish or UTF-8-ish strings: the data is (C string containing word name) (fixed number of floating point)... repeated LOTs of times. The data set(s) may be compressed, so I don't want read them directly -- possibly from a shared memory region (64 bit system only, of course), or , perhaps using shared variables... but I don't think that would be fast enough. Anyway, this begins to allow the push into "big data" and AI applications. Just looking for some input and ideas here. Many thanks Fred Weigel
