I am investigating how to configure a Storm topology so that certain
tasks execute on specific hosts or classes of hosts: for example, a
particular task needs access to a database which is stored on a
locally-connected SSD on a particular machine.
In the talk from 2012 at https://vimeo.com/40972420 this is raised
(50:24 - 51:30) and in the reply Nathan talks about both Apache Mesos
and a pluggable scheduler for custom policies.
I can see mesos/storm but I don't see how it can be used to implement a
placement policy.
From googling I was able to find this:
https://dcvan24.wordpress.com/2015/04/07/metadata-aware-custom-scheduler-in-storm/
which looks pretty much like what I want.
However I couldn't find any authoritative documentation on the scheduler
API by browsing
https://storm.apache.org/doc-index.html
(but I might have missed the right link)
At
https://xumingming.sinaapp.com/885/twitter-storm-how-to-develop-a-pluggable-scheduler/
I found a link to https://github.com/nathanmarz/storm/issues/164 but
that is dead.
Thanks,
Brian Candler.
