@Linas Thanks for sharing your experiences on large data processing using Guile! I'm developing a framework to do the similar work for our product too. But what I'm planning is to do map/reduce for large data in a distributed system. Anyway, I don't think it's a good way to analysis massive data in just one very-strong machine. It requires too much for the language compiler.
I'm still very confident to use Guile on machine learning work. Please don't forget that no any dynamic language do the heavy computation directly, they wrapped BLAS or MKL, just like what numpy does. And I have chance to get a very strong Nvidia GPU cluster server for the work on the next stage. I don't think the computing speed will be the issue for me. We're using Python and C++14 at present, but I don't think Python is the idea glue language, no one wants to use Cython, trust me. The tail-recursive and delimited-continuation for lightweight task scheduling is preferred too. The key issue for me is the expressiveness, which could let us type less code to do more work. Actually, we use Guile to generate many C++ code in compiling time to save lot work coding work. This does matter, especially when you have few people but much work to finish in a tight deadline. There should be better paradigm to use Guile to glue C/C++ code. Implement algorithm in Guile directly is not the good way. And wrap C++ interface with FFI simply is not good too. Sometimes we need to do more abstract work to get both advantages of Guile and C/C++. This is about how we use Guile, not the compiler issue. Of course, I confess there're many problems in our current Guile compiler. The problem is that we have to use Guile a lot to get to know what is the problem exactly, and give compelling reasons for the improvement. @Jan Yes, that should be a way to go. And I have a new idea which is just an idea at present. Many we could find a way to read PyObject to Gulie, and call Python module directly (say, numpy). There should be a type-compatible abstract level between Guile and PyObject. If it works, we may implement Python3 on Guile. Although it seems a large work to implement complete Python3 frontend, we may save lot of work to write alternative Python modules for Guile. Julia language does in this idea, but it's backend is compatible with Python. My idea is not to convert all Python types to Guile, just wrap some types to a special object like <pyobject> is enough, then Guile could be glue language for Python too. Maybe people ask, why bother to glue Python? Python rocks! Rocks? I need complete lambda calculus, proper tail call, static scoping, and delimited-continuations (not the generators). In our product code, we use lot of lambdas in C++ code, so I expect the programming mind could be consistent between glue language and main language. But Python can't provide multi-lines lambda expression only because it uses whitespaces for delimiters, there's no any chance to delimit multi expression within a lambda unless introducing more complexity to the parser. Back to the topic, Guile lacks sufficient practical usage in industry to reveal its disadvantages. But its advantages are apparent enough for me so that it worth for me to go further with it. Best regards. On Sat, Jul 15, 2017 at 6:10 PM, Jan Wedekind <j...@wedesoft.de> wrote: > One could implement something like Theano+NumPy with GNU Guile. I am trying > to do something like that (github.com/wedesoft/aiscm) but I am doing it in my > spare time only. Theoretically GNU Guile is better suited for this than > Python because of macros. > > On July 14, 2017 10:54:45 PM GMT+01:00, Linas Vepstas > <linasveps...@gmail.com> wrote: >>On Mon, Feb 13, 2017 at 2:28 PM, Panicz Maciej Godek >><godek.mac...@gmail.com >>> wrote: >> >>> >>> someone >>> responded critically: "are there out of the box libraries to estimate >>a >>> zero inflated negative >>> binomial regression model in guile". Of course, if I knew what a >>> zero-inflated >>> negative binomial regression model, I could deliver an implementation >>by >>> just explaining >>> the notions used in that phrase. >> >> >>Caution: the message below sounds negative. Sorry, I use guile daily >>and >>almost exclusively now. So there ... >> >>Lack of decent science libraries for scheme is a major stumbling block, >>for >>me. Simply having sine and cosine is not enough. I got excited (a >>decade >>ago) when I realized that guile supported GnuMP, and then rapidly >>deflated >>when I realized it only supported integers and rationals in GnuMP .. I >>work >>with arbitrary-precision floats. Or, I did back then. >> >>Maybe more important is making guile work well with large-RAM setups. >>Currently, I do data analysis, every day, in guile, on datasets that >>take >>20GB or 40GB -- my current one is 110GB when loaded in RAM, and guile >>starts getting buggy, crashy and slow when working at that size. >>Sometimes, it starts calling GC half-a-dozen times per second, for no >>apparent reason, eating up 6 cores (or more!) doing nothing but GC. >>Why? >>Who knows? Who can tell? >> >>Yes, I have a machine with 256 GB RAM and a few dozen cores, and SSD's >>that >>hold the data, but every time guile crashes, I have to wait an hour for >>the >>data to reload. I can live with it, but its a dirty secret I would not >>share with guile wannabe users. >> >>String handling in guile is a disaster area: If I give it a >>10-megabyte-long string in utf8, it promptly tries to convert all of >>that >>string in utf32, for utterly pointless reasons. This just makes it >>slow. >> >>There are still bugs between GC and the compiler: if call (eval "(some >>stuff) (other stuff)") the compiler will try to compile that string >>(after >>it was converted ti utf32!) and if GC happens to run at just that >>moment, >>guile crashes or hangs. These bugs need to be fixed. >> >>So although its a good start, there's a lot of work left until it can >>get >>to "the next level". And that work can't happen until guile is more >>popular. So it's very much chicken-and-egg scenario. >> >>--linas > > -- > Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.
