Having Pharo playing there seems to be a good place to show its strengths. However, you mentioned too many acronyms of technologies I don't understand (but hear a lot about).
The only thing I can agree with is that the self-contained nature of Pharo is a true advantage when deploying extra nodes. It's not only fast, but also pretty lightweight compared with behemoths such as Java. Regards, Esteban A. Maringolo 2015-04-29 9:57 GMT-03:00 p...@highoctane.be <p...@highoctane.be>: > I am involved in some Hadoop deployments and there is a very interesting > possiblity for Pharo in that ecosystem. > > Namely, there is a YARN thing in there which is a scheduler for distributing > computing on a cluster of nodes. > > It is possible to deploy all kinds of technologies on the nodes (e.g. > Python, R, Java) and Pharo images and VM (in headless mode) could be > deployed as well. > > The deployed node can communicate back to what is called an > AppllicationManager via REST callbacks (easy game in Pharo). There is also a > C API (now, this is FFI or a plugin - > http://zookeeper.apache.org/doc/trunk/zookeeperProgrammers.html) > > There is also an Hadoop component named ZooKeeper that focuses on acting as > a distributed configuration repository. > > One can talk to it with REST too > (https://github.com/apache/zookeeper/tree/trunk/src/contrib/rest) > > Given the fact that we also can use some Java calls (using the JNI module > with 32-bits Java), we can integrate well enough on YARN I'd say. > > There is also another project which is very nice and this is SLIDER (on > YARN). > This is about deploying stuff in an elastic way, (see > http://slider.incubator.apache.org/) > > The next logical thing is to have docker containers (containing a pharo > stack) deployed dynamically on the cluster using Slider (like this: > http://www.slideshare.net/hortonworks/docker-on-slider-45493303) > > First step here would be to have a basic YARN-Pharo application and a PoC > for talking to ZooKeeper. > > This would open interesting gates for Pharo given its strengths. > Even more when we'll get a 64-bit VM. > > What is cool with Pharo is that an image can be very small and self > containing vs Java application (which have tons of Jar files attached). > > Access to the data on the HDFS thing can happen through NFSv3 so, we can go > that route. > There is also a REST API to it > (https://hadoop.apache.org/docs/r1.0.4/webhdfs.html) > > Tell me what you think! > > Phil >
