Anthony, Default bucket properties are defined in riak_core, not riak_kv, so you can't really set the hash function as a result of setting something else. Furthermore, you really only have to set these once, so you could configure your application to check the bucket props on startup and set them appropriately.
"backend" is a property only used by the multi_backend (which isn't used that frequently), so it seems a little presumptuous to special-case that property (turning it into an atom). There are a few other technical reasons that you don't want to arbitrarily turn binaries into atoms. I agree that it's not especially intuitive but I think it's a small point of pain, especially if we fix the documentation. Sean Cribbs <s...@basho.com> Developer Advocate Basho Technologies, Inc. http://basho.com/ On Jan 24, 2011, at 2:01 PM, Anthony Molinaro wrote: > Sean, > > I'll fork and submit a pull request sometime soon. Will things continue > to be configured with binaries, or will it switch to atoms at some point? > > Also, Is there a way to configure specific buckets in the config, or to > specify default bucket properties for a particular bucket type in the config? > I see there is default_bucket_props, and you can have bucket specific > properties, but what I would want is some way to have a custom bucket type > which would require the hash function be different. This doesn't seem like > bucket specific config (ie, config passed to start/2), but instead feels like > per bucket type default config. I think I can work around it by specifyng > it at the same time I specify the custom backend. Like > > curl -X PUT -H "content-type: application/json" -d > '{"props":{"backend":"my_backend"},"chash_keyfun":{"mod":"my_backend","fun":"my_hash"}}' > http://127.0.0.1:8098/riak/mybucket > > but this feels risky, as it would be easy for someone to miss. > > I'd prefer being able to set that in default bucket props for a particular > backend. That seems cleaner, and less error-prone. > > Anyway, if it doesn't work in this way, any reason it couldn't? > > -Anthony > > On Mon, Jan 24, 2011 at 08:57:26AM -0500, Sean Cribbs wrote: >> Anthony, >> >> Thanks for noticing that documentation error. If you would, please file an >> issue on the wiki's issue tracker: https://github.com/basho/riak_wiki/issues >> and we'll get it corrected. If you're feeling enterprising, you can also >> fork, fix and send a pull-request for the page. >> >> Sean Cribbs <s...@basho.com> >> Developer Advocate >> Basho Technologies, Inc. >> http://basho.com/ >> >> On Jan 24, 2011, at 2:43 AM, Anthony Molinaro wrote: >> >>> Hi Sean, >>> >>> Thanks, that seems to work. Should the wiki be changed? Currently >>> this page >>> >>> http://wiki.basho.com/Configuration-Files.html >>> >>> has >>> >>> multi_backend: list of backends to provide >>> Format of each backend specification is {BackendName, BackendModule, >>> BackendConfig}, where BackendName is any atom, BackendModule is the name of >>> the Erlang module implementing the backend (the same values you would >>> provide as "storage_backend" settings), and BackendConfig is >>> a parameter that will be passed to the "start/2" function of the backend >>> module. >>> >>> and it sounds like BackendName has to be a binary. >>> >>> Also, it has >>> >>> Specify the backend to use for a bucket with >>> riak_client:set_bucket(BucketName,[{backend, BackendName}] in Erlang >>> >>> but I'm a little unclear how you would invoke this call, do you attach >>> with riak attach, then run the command there? Because when I try that >>> I get >>> >>> ** exception error: undefined function riak_client:set_bucket/2 >>> >>> Seems there might be a riak_client:set_bucket/3, so that documentation may >>> be out of date as well. >>> >>> Thanks, >>> >>> -Anthony >>> >>> On Sun, Jan 23, 2011 at 02:50:32PM -0500, Sean Cribbs wrote: >>>> Anthony, >>>> >>>> This is something I discovered a while back - define your backend names as >>>> binaries and you'll be able to set them properly from the REST interface. >>>> Example: >>>> >>>> {storage_backend, riak_kv_multi_backend}, >>>> {multi_backend_default, <<"bitcask">>}, >>>> {multi_backend, >>>> [ {<<"bitcask">>, riak_kv_bitcask_backend, >>>> [{data_root, "/var/lib/riak/bitcask"}]}, >>>> {<<"dets">>, riak_kv_dets_backend, >>>> [{riak_kv_dets_backend_root, "/var/lib/riak/dets"}]}, >>>> {<<"ets">>, riak_kv_ets_backend, []}, >>>> {<<"fs">>, riak_kv_fs_backend, >>>> [{riak_kv_fs_backend_root, "/var/lib/riak/fs"}]}, >>>> {<<"cache">>, riak_kv_cache_backend, >>>> [ {riak_kv_cache_backend_memory, 100}, >>>> {riak_kv_cache_backend_ttl, 600}, >>>> {riak_kv_cache_backend_max_ttl, 3600} >>>> ]}, >>>> {<<"my_backend">>, my_backend, []} >>>> ]}, >>>> >>>> Then you can set it like so >>>> >>>> curl -X PUT -H "content-type: application/json" -d >>>> '{"props":{"backend":"my_backend"}}' http://127.0.0.1:8098/riak/mybucket >>>> >>>> Also note that only allow_mult and n_val are supported bucket properties >>>> from the PB interface (something we'll be fixing soon). >>>> >>>> Sean Cribbs <s...@basho.com> >>>> Developer Advocate >>>> Basho Technologies, Inc. >>>> http://basho.com/ >>>> >>>> On Jan 23, 2011, at 12:37 PM, Anthony Molinaro wrote: >>>> >>>>> >>>>> Hi, >>>>> >>>>> So I wanted to play around with creating a custom backend, and using >>>>> the multi backend, but I am having problems getting anything to work. >>>>> >>>>> Here's what I tried so far >>>>> >>>>> in app.config >>>>> >>>>> {storage_backend, riak_kv_multi_backend}, >>>>> {multi_backend_default, bitcask}, >>>>> {multi_backend, >>>>> [ {bitcask, riak_kv_bitcask_backend, >>>>> [{data_root, "/var/lib/riak/bitcask"}]}, >>>>> {dets, riak_kv_dets_backend, >>>>> [{riak_kv_dets_backend_root, "/var/lib/riak/dets"}]}, >>>>> {ets, riak_kv_ets_backend, []}, >>>>> {fs, riak_kv_fs_backend, >>>>> [{riak_kv_fs_backend_root, "/var/lib/riak/fs"}]}, >>>>> {cache, riak_kv_cache_backend, >>>>> [ {riak_kv_cache_backend_memory, 100}, >>>>> {riak_kv_cache_backend_ttl, 600}, >>>>> {riak_kv_cache_backend_max_ttl, 3600} >>>>> ]}, >>>>> {my_backend, my_backend, []} >>>>> ]}, >>>>> >>>>> Then I restart, open a shell and do the following >>>>> >>>>> 1> {ok, Pid} = riakc_pb_socket:start_link("127.0.0.1", 8087). >>>>> {ok,<0.68.0>} >>>>> 2> riakc_pb_socket:set_bucket(Pid, <<"b">>, [{backend, my_backend}]). >>>>> ok >>>>> 3> riakc_pb_socket:get_bucket(Pid,<<"b">>). >>>>> {ok,[{n_val,3},{allow_mult,false}]} >>>>> >>>>> So I didn't see my backend there, thus tried the REST API >>>>> >>>>>> curl 'http://127.0.0.1:8098/riak/b' ; echo >>>>> {"props":{"name":"b","n_val":3,"allow_mult":false,"last_write_wins":false,"precommit":[],"postcommit":[],"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"old_vclock":86400,"young_vclock":20,"big_vclock":50,"small_vclock":10,"r":"quorum","w":"quorum","dw":"quorum","rw":"quorum"}} >>>>> >>>>> It's not there either. So I try to set it with REST >>>>> >>>>>> curl -X PUT -H "Content-Type: application/json" -d >>>>>> '{"props":{"backend":"my_backend"}}' http://127.0.0.1:8098/riak/b >>>>> >>>>> Which works, in that now I have >>>>> >>>>>> curl 'http://127.0.0.1:8098/riak/b' ; >>>>>> echo{"props":{"backend":"my_backend","name":"b","n_val":3,"allow_mult":false,"last_write_wins":false,"precommit":[],"postcommit":[],"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"old_vclock":86400,"young_vclock":20,"big_vclock":50,"small_vclock":10,"r":"quorum","w":"quorum","dw":"quorum","rw":"quorum"}} >>>>> >>>>> However, in the shell I still get >>>>> >>>>> 4> riakc_pb_socket:get_bucket(Pid,<<"b">>). >>>>> {ok,[{n_val,3},{allow_mult,false}]} >>>>> >>>>> Also, if I attempt to put something I get >>>>> >>>>> 5> riakc_pb_socket:put(Pid, riakc_obj:new (<<"b">>, <<"c">>, <<"d">>)). >>>>> >>>>> =ERROR REPORT==== 19-Jan-2011::02:21:04 === >>>>> ** Generic server <0.68.0> terminating >>>>> ** Last message in was {req_timeout,#Ref<0.0.0.186>} >>>>> ** When Server state == {state,"127.0.0.1",8087,false,false,undefined, >>>>> undefined, >>>>> {[],[]}, >>>>> 1,[],infinity,100} >>>>> ** Reason for termination == >>>>> ** disconnected >>>>> ** exception exit: disconnected >>>>> >>>>> =CRASH REPORT==== 19-Jan-2011::02:21:04 === >>>>> crasher: >>>>> initial call: riakc_pb_socket:init/1 >>>>> pid: <0.68.0> >>>>> registered_name: [] >>>>> exception exit: disconnected >>>>> in function gen_server:terminate/6 >>>>> ancestors: [<0.65.0>] >>>>> messages: [{tcp,#Port<0.816>, >>>>> [0|<<10,7,116,105,109,101,111,117,116,16,1>>]}] >>>>> links: [<0.65.0>] >>>>> dictionary: [] >>>>> trap_exit: false >>>>> status: running >>>>> heap_size: 987 >>>>> stack_size: 24 >>>>> reductions: 1541 >>>>> neighbours: >>>>> neighbour: [{pid,<0.65.0>}, >>>>> {registered_name,[]}, >>>>> {initial_call,{erlang,apply,2}}, >>>>> {current_function,{gen,do_call,4}}, >>>>> {ancestors,[]}, >>>>> {messages,[{#Ref<0.0.0.185>,{error,timeout}}]}, >>>>> {links,[<0.25.0>,<0.68.0>]}, >>>>> {dictionary,[]}, >>>>> {trap_exit,false}, >>>>> {status,runnable}, >>>>> {heap_size,1597}, >>>>> {stack_size,38}, >>>>> {reductions,17388}] >>>>> >>>>> And the attached sasl log which has many errors. So I'm trying to figure >>>>> out >>>>> whether the problem is configuration, or a bug or what, and wondering if >>>>> any >>>>> one else has gotten a custom backend, or multi backend to work? >>>>> >>>>> I'm using 0.14 on Centos 5, using the basho RPM. >>>>> >>>>> Thanks, >>>>> >>>>> -Anthony >>>>> >>>>> -- >>>>> ------------------------------------------------------------------------ >>>>> Anthony Molinaro <antho...@alumni.caltech.edu> >>>>> <sasl-error.log>_______________________________________________ >>>>> riak-users mailing list >>>>> riak-users@lists.basho.com >>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>> >>> >>> -- >>> ------------------------------------------------------------------------ >>> Anthony Molinaro <antho...@alumni.caltech.edu> >> > > -- > ------------------------------------------------------------------------ > Anthony Molinaro <antho...@alumni.caltech.edu> _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
