Hi Joe, For most use cases, there would be no limit to the number of buckets you can have on a level db cluster. (Aside from obvious limits of, eventually you'd run out of disk space for all the objects).
Riak essentially treats the bucket as merely a prefix for the key. (It basically concatenates the bucket type, the bucket, and the key, and then hashes /that/ to determine where to place the object on the ring). So, there isn't a data structure anywhere on disk or in memory that holds a list of buckets on the cluster, to impose a limit. The exception to that is when you set non-standard properties on specific buckets. For example, if you have siblings turned on for all of your buckets, but you have a 'logs' bucket that will only have immutable data, it makes sense to set allow_mult=false for just the logs bucket, using the Set Bucket Properties api call (for example, http://docs.basho.com/riak/latest/dev/references/http/set-bucket-props/ ). Setting custom properties for a handful of buckets is not a problem. But if you decide to do that for hundreds of buckets, it becomes an issue. Whereas before, buckets were just a "stateless" prefix to the key, now the custom properties do actually live in a data structure on the cluster, and have to be gossiped around the ring. And as you add more custom buckets, that gossiped structure becomes larger and larger. If you do need to set custom properties on hundreds of buckets (such as specifying a Search index on them, for example), consider using Bucket Types instead. Bucket Type custom settings and metadata use a different persistence mechanism than bucket settings, and there's a lot more room for custom types. In addition, once you set up a custom bucket type, all of the buckets /within/ that type get those settings essentially for free. See http://docs.basho.com/riak/latest/dev/advanced/bucket-types/ for more discussion. The other exception that the docs mention (about the file handles etc), comes into play when you use a custom Multi backend (see http://docs.basho.com/riak/latest/ops/advanced/backends/multi/ ) for individual buckets. You can, for example, say in the config that you want your 'users' bucket to live in a custom LevelDB backend, separate from the rest of your levelDB data. And it'll have a separate data directory (which you can house on a different disk, etc). Which is where the file handle limit comes in. If you intend to spin up multiple custom backends on your cluster, you want to be sure the file system can handle opening that many files. The cases where you'd want to do that are extremely rare, though. So, to summarize: In most cases, there is no theoretical limit on the number of buckets you can have. If you must have custom settings on individual buckets, think tens and not hundreds or thousands. Better yet, use custom settings on Bucket Types, instead. Does that answer your question? Dmitri On Fri, Aug 7, 2015 at 10:56 AM, Joe Olson <technol...@nododos.com> wrote: > From http://docs.basho.com/riak/latest/theory/concepts/: > > "In general, large numbers of buckets within a Riak cluster is not a > problem. In practice, there are two potential restrictions on the maximum > number of buckets a Riak cluster can handle" > > Is there any further, more specific documentation on this that I could > read? > > I am interested in understanding the upper limits of the number of buckets > for say, a 10 node, ring_size = 64 (or 128) LevelDB backed Riak cluster. > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
