My personal feeling is the use cases are so different across organizations. It would be really hard to design a **standard** policy for such things. What is your major concern of customizing your own placement policy?
- Sijie On Tue, May 16, 2017 at 4:05 AM, Enrico Olivelli <eolive...@gmail.com> wrote: > Hi bookkeepers, > I'm using BookKeeper for serveral projects, every project has its own > workload characteristics and I would like to be able to assign bookies > depending of the client type. It is quite common to share a BookKeeper > cluster between different applications. > > For instance I am using Bookies to store Database logs, Task Brokers > logs and recently I have started to use BookKeeper as data storage. > > Within the cluster I would like to use specific Bookies for mid-term > storage, some bookies for logs...and so on, but current placement > policies are not able to "distinguish" bookies. > > Actually I can achieve my goal by using a custom policy + custom > metadata + out of band bookie metadata. > > I would like to introduce a first step, following the work of on > "Resource aware data placement" (1), and introduce a list of "labels" > to be assigned to every bookie. > > For instance: bookies for long term storage will have label > "long-term", bookies for transaction logs may have label "wals". > > Another use case is to be able to request BookKeeper to write ledger > data on specific sets of bookies depending on the "customer" who is > the owner of data (I have customers already grouped by labels/tags) > > I would like to have a simple "standard" policy which uses some > "standard" metadata to select bookies. > > Thinks to add: > - a set of "labels" configurable for bookies > - Enrich the API (getBookieInfo) to query for labels and BookKeeper > client to keep a local cache of label-to-bookie assignments > - add a standard "custom metadata field" which is a list of labels to > use to select bookies, a bookie would be used only of it currently > "has" all of the labels requested > > > [1] https://cwiki.apache.org/confluence/display/BOOKKEEPER/ > BP-2+-+Resource+aware+data+placement > > All comments are welcome > > -- Enrico >
