Re: store/delete of key possible in MR
yes, I do -- View this message in context: http://riak-users.197444.n3.nabble.com/store-delete-of-key-possible-in-MR-tp4032242p4032252.html Sent from the Riak Users mailing list archive at Nabble.com. ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
issues with bucket types and corrugatediron
https://github.com/basho-labs/CorrugatedIron/issues/212 I may not be understanding this correctly but I'm about to test this in a sec, maybe I can create the bucket with a shell script to make sure its created with the specified bucket type, but thats assuming that it won't maintain two different buckets with the same name "Sessions" when I try to put objects (IE if I don't put objects and specify the bucket type in Corrugated Iron.) As for the rest of the config stuff: Konsole output storage_backend = multi multi_backend.bitcask_mult.storage_backend = bitcask multi_backend.bitcask_mult.bitcask.data_root = ../data/riak/bitcask_mult multi_backend.leveldb_mult.storage_backend = leveldb multi_backend.leveldb_mult.leveldb.data_root = ../data/riak/leveldb_mult multi_backend.leveldb_mult.storage_backend = memory multi_backend.default = bitcask_mult Because of the amount of trouble I had to go through to get that configuration figured out in the first place, I want to say it's probably better if I just have 3 different riak servers, one for memory, one for bitcask, and one for leveldb. This way I wouldn't even have to mess with bucket types, but I can't even begin to tell you how unacceptable that is for someone trying to setup riak for development purposes; - yeah ok sure just start 3 docker instances because docker totally doesn't have problems either. - I guess I could also just have 3 instances of riak running all of which use different listening ports but thats... fuggly. I'm having logistical hangups about this in terms of what I have to support in order to facilitate development efforts led by other people. I could use some advice as maybe just reading the documentation isn't doing me any favors? I do have a purpose in in wanting to use all 3 of these backends, let me describe why I want these things, feel free to tell me where I'm wrong: Memory: session: self expiring objects, fast. Object footprint is predictably measurable (has a sane maximum size, small) Bitcask: user account: no secondary indexes required, items are always retrieved by the same key, object footprint is predictably measurable (has a sane maximum size, only needs to be accessed once and the name and location of the user's data bucket is cached in the session object.) Since keys are in memory this would mean that authentication should be pretty fast, and the overhead on the server should be pretty minimal since it's only needed once every time the user logs in. leveldb: user account data buckets: each user has it's own bucket, may have secondary indexes for searching, bucket/data size is somewhat invariable; I'm also planning to index each bucket (as per the documentation for solr, indexing can be done on bucket types and I would like to set up solr so that it finds all user data buckets by bucket type.) I've established creating a bucket per user is an ok way to manage stuff, though I'm wondering what sort of implications this will have in how partitioning works/doesn't work; will each buckets data be "partitioned" (seems like in the context of replicas partitioning just means that each node is responsible for queries to specific items in the bucket vs specific items only exist in those parititons.) Any advice appreciated, -Paige ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: issues with bucket types and corrugatediron
Just a heads up, we figured out I had some consistency/misconfiguration issues with my riak.conf and the names I had specified for the backends: Konsole output storage_backend = multi multi_backend.bitcask_mult.storage_backend = bitcask multi_backend.bitcask_mult.bitcask.data_root = ../data/riak/bitcask_mult multi_backend.leveldb_mult.storage_backend = leveldb multi_backend.leveldb_mult.leveldb.data_root = ../data/riak/leveldb_mult #was: multi_backend.leveldb_mult.storage_backend = memory multi_backend.memory_mult.storage_backend = memory multi_backend.default = bitcask_mult Also the names of the storage backends in the bucket type definitions were inconsistent with the riak.conf (were: memory, leveldb, and bitcask) (changed to memory_mult, leveldb_mult, and bitcask_mult) now it works fine. On 12/10/14 15:43, Paige Thompson wrote: > https://github.com/basho-labs/CorrugatedIron/issues/212 > > I may not be understanding this correctly but I'm about to test this in > a sec, maybe I can create the bucket with a shell script to make sure > its created with the specified bucket type, but thats assuming that it > won't maintain two different buckets with the same name "Sessions" when > I try to put objects (IE if I don't put objects and specify the bucket > type in Corrugated Iron.) > > As for the rest of the config stuff: > > Konsole output > storage_backend = multi > > multi_backend.bitcask_mult.storage_backend = bitcask > multi_backend.bitcask_mult.bitcask.data_root = ../data/riak/bitcask_mult > > multi_backend.leveldb_mult.storage_backend = leveldb > multi_backend.leveldb_mult.leveldb.data_root = ../data/riak/leveldb_mult > > multi_backend.leveldb_mult.storage_backend = memory > > multi_backend.default = bitcask_mult > > > Because of the amount of trouble I had to go through to get that > configuration figured out in the first place, I want to say it's > probably better if I just have 3 different riak servers, one for memory, > one for bitcask, and one for leveldb. This way I wouldn't even have to > mess with bucket types, but I can't even begin to tell you how > unacceptable that is for someone trying to setup riak for development > purposes; > > - yeah ok sure just start 3 docker instances because docker totally > doesn't have problems either. > > - I guess I could also just have 3 instances of riak running all of > which use different listening ports but thats... fuggly. > > I'm having logistical hangups about this in terms of what I have to > support in order to facilitate development efforts led by other people. > I could use some advice as maybe just reading the documentation isn't > doing me any favors? I do have a purpose in in wanting to use all 3 of > these backends, let me describe why I want these things, feel free to > tell me where I'm wrong: > > Memory: > session: self expiring objects, fast. Object footprint is predictably > measurable (has a sane maximum size, small) > > > Bitcask: > user account: no secondary indexes required, items are always retrieved > by the same key, object footprint is predictably measurable (has a sane > maximum size, only needs to be accessed once and the name and location > of the user's data bucket is cached in the session object.) Since keys > are in memory this would mean that authentication should be pretty fast, > and the overhead on the server should be pretty minimal since it's only > needed once every time the user logs in. > > > leveldb: > user account data buckets: each user has it's own bucket, may have > secondary indexes for searching, bucket/data size is somewhat > invariable; I'm also planning to index each bucket (as per the > documentation for solr, indexing can be done on bucket types and I would > like to set up solr so that it finds all user data buckets by bucket > type.) I've established creating a bucket per user is an ok way to > manage stuff, though I'm wondering what sort of implications this will > have in how partitioning works/doesn't work; will each buckets data be > "partitioned" (seems like in the context of replicas partitioning just > means that each node is responsible for queries to specific items in the > bucket vs specific items only exist in those parititons.) > > Any advice appreciated, > > -Paige > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Map Reduce with Data Types
I can't get map reduce to work with the new data types.
I have a bucket type as defined below:
riak-admin bucket-type create map '{"props":{"datatype":"map"}}
There seems to be a problem going from the input phase to the map/reduce.
I'm getting notfound errors for objects that exist.
While the normal list of input tuples might look like [{<<"bucket">>,
<<"key">>}],
for data-type lookups I'm guessing (based on the src and how bucket/key
pairs for data types are defined elsewhere) should look like:
[{{<<"type">>, <<"bucket">>}, <<"key">>}]?
mapred(Pid, [{{<<"map">>, <<"articles">>},
<<"7bdbd2da-1463-46f8-b750-6f813700b28b">>}], [{map, {modfun,
'riak_kv_mapreduce', 'map_object_value'}, none, true}]).
Anyone have any insight or have a working example of mapred with data types?
Thanks!
--
Kyle Patterson
Full Stack Maniac
https://sqor.com/users/kisp
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: system metrics monitoring
please somebody reply my query. RegardsPragya Jain On Tuesday, 9 December 2014 11:53 AM, pragya jain wrote: hello all! As mentioned at statistics and monitoring page of Riak Systems Metrics To Graph | Metric | | Available Disk Space | | IOWait | | Read Operations | | Write Operations | | Network Throughput | | Load Average | Can somebody suggest me some monitoring tools that monitor these metrics? Regards Pragya Jain ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
