Java Client Riak Builders...

2012-05-28 Thread Guido Medina
Hi, I'm looking for a work around @RiakIndex annotation to support multiple values per index name, since the annotation is limited to one single value per annotated property (no collection support), I would like to know if there is a way of using the DomainBucketBuilder, mutation & conflict

Re: Java Client Riak Builders...

2012-05-29 Thread Guido Medina
29, 2012 6:09 PM To: Guido Medina Cc: riak-users@lists.basho.com Subject: Re: Java Client Riak Builders... Guido - The real fix is to enhance the client to support a Collection, I'll add an issue for this in github. What you would need to do right now is write your own Converter (which wou

Re: Java Client Riak Builders...

2012-05-29 Thread Guido Medina
Brian Roach Sent: Tuesday, May 29, 2012 7:05 PM To: Guido Medina Cc: riak-users@lists.basho.com Subject: Re: Java Client Riak Builders... Guido - Thanks, looking forward to it. Also as an FYI, on Friday I fixed the bug that was causing the requirement of the @JsonIgnore for Riak annotated f

Re: Java Client Riak Builders...

2012-05-30 Thread Guido Medina
Give a quick review to git://github.com/guido-medina/riak-java-client.git And let me know if there are more changes to add/correct, I'm switching from work to home so I can continue in there, the modifications are fair simple, I thought about adding bean utils for reflection but manage

Java Client: Connection pool recovery.

2012-06-01 Thread Guido Medina
Hi, We have a 4 nodes cluster and it works like a charm, our issue on development (this is not happening in production but we are afraid it might) is that when we are doing several concurrent store, if one thread gives an exception (timeout getting connection from the pool or connection ref

Re: Are integer indexes in 2i 64-bit or 32-bit?

2012-06-07 Thread Guido Medina
All points to 32 bits, at least for the Java client side (indexes can be of type Integer, not Long which is the 64 bits) Look for RiakIndex.java, that will give you some answers. I don’t know the exact answer though. Regards, Guido. From: Alexander Sicular Sent: Thursday, June 07, 2012 10:43

Re: Are integer indexes in 2i 64-bit or 32-bit?

2012-06-08 Thread Guido Medina
Jun 2012, at 22:55, Guido Medina wrote: All points to 32 bits, at least for the Java client side (indexes can be of type Integer, not Long which is the 64 bits) Look for RiakIndex.java, that will give you some answers. That's a mistake on the part of the client developer at that time

Re: Are integer indexes in 2i 64-bit or 32-bit?

2012-06-08 Thread Guido Medina
interface / annotations allowing for any of the types to be passed in. - Roach On Jun 8, 2012, at 4:18 AM, Guido Medina wrote: I would say just to add long and Long for indexing (So it that it supports Long, Integer and their respective natives), using BigNumber subclasses have a different se

Re: Riak backups

2012-06-28 Thread Guido Medina
Hi, If I might add up to it, I would like to know specifically what would be a proper way to have a replication site of a cluster, it seems to me it is easier to backup the replicated site than the cluster itself. We replicate using a Java scheduled job but it requires us to mark each key wi

Re: Riak behind a Load Balancer

2012-08-31 Thread Guido Medina
Hi, We use HA proxy on a 4 nodes cluster for a J2EE application, before using HA proxy we delegated on the Java Riak cluster client config to handle the 4 nodes, but too many errors were generated, once we switched to HA proxy and a single Java Riak client config, the errors were reduced dr

Re: Strategies for testing against Riak

2012-09-07 Thread Guido Medina
Hi, You could index your test data, 2i search that indexed data and delete after the tests, that will reduce the deletes enormously and make the clean up way faster IMHO. Maybe this doesn't apply much to your environment, anyways, it is a very doable solution, the indexing though would only

Re: REST API: Deleting key vs. marking as deleted.

2012-09-11 Thread Guido Medina
Hi, Just an idea, you could have a batch number incrementing once a minute (Integers won't run out of range for at least 20 years) and have your marking be an index name "delete_batch_number", and have a scheduled job to delete once a minute or something where it will just delete the keys w

Re: Evolving data model

2012-09-19 Thread Guido Medina
We have done similar things, but it always depends on the available tools, your requirements and needs, I will give you a short example, our main application uses a standard SQL, for historical data we use Riak, data that is not changing, for example, audit trails, daily chunks from different s

Re: HTTP or PB cluster client issue

2012-09-21 Thread Guido Medina
Hi, It is the Java client which to be honest, doesn't handle well one node going down, so, for example, in my company we use HA proxy for that, here is a starting configuration: https://gist.github.com/1507077 Once we switched to HA proxy we just use a simple client without cluster confi

Re: HTTP Client bug

2012-09-24 Thread Guido Medina
Deepak, I think that was fixed on the following pull request but, you would have to compile your own client from github which is easy; or, did you get the new 1.0.6? I'm not sure if it is already available at maven repos, anyways, there is a lot going on in the riak client in java, I would

Re: HTTP Client bug

2012-09-24 Thread Guido Medina
, do a maven clean test and if tests pass then you are good to go with your own version requirements. Regards, Guido. On 24/09/12 20:05, Guido Medina wrote: Deepak, I think that was fixed on the following pull request but, you would have to compile your own client from github which is easy

Re: HTTP Client bug

2012-09-24 Thread Guido Medina
so* resolve the issue I dont see a change to pom.xml during this pull. I tried the riak-client 1.0.6 and it still depends on 4.1.x. So the upgrade would be necessary to fix the memory leak. Thanks Deepak Bala On Tue, Sep 25, 2012 at 12:35 AM, Guido Medina mailto:guido.med...@temetra.com>>

Re: Bug on riak-client 1.0.6 - Riak index property not serialized

2012-09-26 Thread Guido Medina
Deepak, That's why in our company we have index annotations at method level, for us we agreed that an index is not really a POJO property but a pseudo/computed property, it was a waste for us to add a field just because an index, so we have our own branch of the Riak java client where the m

Re: Rial Search and key expiry: a difficult relationship?

2012-09-28 Thread Guido Medina
Hi Alfonso, I'm afraid for expire functionality, you would be looking at some caching solution, in Java for example you could use Guava or EHCache if running in the same JVM, if you need a distributed cache then memcached would be the right thing to use, now, if you need search AND expire d

Riak java client - HTTP client...

2012-10-02 Thread Guido Medina
Hi, Is there any particular issue with the Java Riak HttpClient when using a version other than 4.1.1? I have a pull request that doesn't pass the Travis CI build, even though I have verified it shouldn't have issues except for the that specific difference, if so, could it be tried with Htt

Re: Riak JAVA Client Performance

2012-10-10 Thread Guido Medina
That question has been answered few times, here is my old answer: Hi, It is the Java client which to be honest, doesn't handle well one node going down, so, for example, in my company we use HA proxy for that, here is a starting configuration:https://gist.github.com/1507077 Once we switched

Re: Riak JAVA Client Performance

2012-10-10 Thread Guido Medina
13 PM, Guido Medina mailto:guido.med...@temetra.com>> wrote: That question has been answered few times, here is my old answer: Hi, It is the Java client which to be honest, doesn't handle well one node going down, so, for example, in my company we use HA proxy for tha

Re: Riak JAVA Client Performance

2012-10-10 Thread Guido Medina
ark single node. If I knew that I can load single node with N requests / sec, I could assume that after load balancing over 5 nodes my throughput limit will increase linearly. Pavel On Wed, Oct 10, 2012 at 2:51 PM, Guido Medina mailto:guido.med...@temetra.com>> wrote: The answ

Re: Riak JAVA Client Performance

2012-10-10 Thread Guido Medina
. On 10/10/12 16:28, Guido Medina wrote: That's why I keep pushing to one answer, Riak is not meant to be in one cluster, you are removing the external factors and CAP settings you will be using, and it won't be linear, you could get the same results with RW=2 with 3, 4 and 5 nodes,

Re: Riak JAVA Client Performance

2012-10-10 Thread Guido Medina
an be 100% sure he is not wasting time on something. this is maybe premature optimization but it maybe also good position to understand library and enter new world of riak pozdrawiam Paweł Kamiński kami...@gmail.com pkaminski@gmail.com __ On 10 October 2012 17:30, Guido M

Re: Riak JAVA Client Performance

2012-10-11 Thread Guido Medina
ction timeout if no requests would be done for some period? Pavel On Wed, Oct 10, 2012 at 8:57 PM, Guido Medina mailto:guido.med...@temetra.com>> wrote: From that perspective, for now it is better to treat the client as you would treat a JDBC DataSource pool, the tidy up

Riak java client mutation...

2012-10-14 Thread Guido Medina
Hi Brian, I know what you mean about https://github.com/basho/riak-java-client/issues/175 which by modifying the current *Mutation* interface will break existing code, what about adding a *CriteriaMutation* or *OptionalMutation* interface that *extends* the *Mutation* interface? That way th

Re: bad jackson performance in riak-java-client

2012-10-17 Thread Guido Medina
I see I'm not the only one that spot it, I have insisted several times to upgrade Jackson libraries to version 1.9.9 and later to 1.9.10 with no avail, I'm not sure how important it is for Riak java client team to keep the POM healthy. That's why we keep our Riak java client copy, merge from ma

Re: Is Riak suitable for s small-record write-intensive billion-records application?

2012-10-18 Thread Guido Medina
Hi, That's exactly what Riak is designed for, there is no better usage of Riak than the scenario you are describing, now take into account the consistency, availability and concurrency of your writes, you might want to implement/use a sort of locking mechanism combined with in-memory cache

Re: Is Riak suitable for s small-record write-intensive billion-records application?

2012-10-19 Thread Guido Medina
Riak is all about high availability, if eventually consistent data is not a problem OR, you can cover those aspects of the CAP concept with an in-memory caching system and a sort of a locking mechanism to emulate the core atomic action of your application (put-if-absent) then I would say, you a

Re: Is Riak suitable for s small-record write-intensive billion-records application?

2012-10-19 Thread Guido Medina
of this days I will built a distributed re-entrant locking mechanism base on REST for the sake of it) For the last quote, the locking mechanism if well designed will always take care of that. Regards, Guido. On 19/10/12 13:42, Les Mikesell wrote: On Fri, Oct 19, 2012 at 6:57 AM, Guido Medin

Re: Is Riak suitable for s small-record write-intensive billion-records application?

2012-10-19 Thread Guido Medina
7;m limited to a certain level... On 19/10/12 16:32, Les Mikesell wrote: On Fri, Oct 19, 2012 at 8:02 AM, Guido Medina wrote: It depends, if you have siblings enabled at the bucket, then you need to resolve the conflicts using the object vclock, How does that work for simultaneous initial inserts?

Re: Is Riak suitable for s small-record write-intensive billion-records application?

2012-10-19 Thread Guido Medina
About distributed locking mechanism, you might wanna take a look at Google services, something called Chubby? Ctrl + F on that link: http://en.wikipedia.org/wiki/Distributed_lock_manager Regards, Guido. On 19/10/12 16:47, Guido Medina wrote: Locking mechanism on a single server is easy, on

Re: Logging chat messages

2012-10-22 Thread Guido Medina
Don't overkill it with technology, you could use Riak with a simple 2i index (integer index MMDD for the message date so you can search day by day backward), and for the message sequence or identifier you could either user ANY SQL database sequence or a UUID generator. HTH, Guido. On 22/

Re: Logging chat messages

2012-10-23 Thread Guido Medina
ul for me to navigate via date. I will do this. An incremental message id would be helpful for me to get last 50 messages and so forth. I will use another db for this. Thanks for all your help. Shashwat On Mon, Oct 22, 2012 at 2:46 PM, Guido Medina

Re: Logging chat messages

2012-10-23 Thread Guido Medina
tal message id would be helpful for me to get last 50 messages and so forth. I will use another db for this. Thanks for all your help. Shashwat On Mon, Oct 22, 2012 at 2:46 PM, Guido Medina mailto:guido.med...@temetra.com&g

Re: Web doc buglet

2012-10-31 Thread Guido Medina
HA proxy + Riak + ElasticSearch are your friends, Solr lacks documentation (way outdated), hard to find stuff done and samples, so if you have your cluster well setup and your meaning to do only key-value retrieval with assist of text index search using ElasticSearch, you are good. *Note:* We

Re: Web doc buglet

2012-11-01 Thread Guido Medina
You will realize that one specific solution doesn't solve everything, so you will probably have to use a piece from each base on their strengths, Riak = *Key-Value*, HA proxy (not related) is a good *TCP balancer*, we use it and it works like a charm, here is a sample configuration https://gist

Re: pseudo-atomic operations

2012-11-18 Thread Guido Medina
Atomicity is always a delicate subject, but it could be resolved with either the following solutions: 1. Your application can have a re-entrant or synchronized lock: /lock(username) { create if not present then exit lock; if present then return to user asking for another username }/ 2. Or

Re: Don't store null-Attributes in riak-json-data via riak-java-client

2012-12-14 Thread Guido Medina
Thats more related to Jackson object mapper, annotate your POJO like this: @JsonSerialize(include=JsonSerialize.Inclusion.NON_NULL) public class MyPOJO{ ... ... } Hope that helps, Guido. On 14/12/12 17:16, Ingo Rockel wrote: Hi all, Is there a way to tell the riak-java-client to not store at

Re: Don't store null-Attributes in riak-json-data via riak-java-client

2012-12-17 Thread Guido Medina
17/12/12 09:52, Ingo Rockel wrote: Hi Guido, thanks, that's what I was looking for and I was looking at the wrong place. Ingo Am 14.12.2012 20:16, schrieb Guido Medina: Thats more related to Jackson object mapper, annotate your POJO like this: @JsonSerialize(in

Thanks to Riak Java client contributors...

2013-01-18 Thread Guido Medina
Hi all, Just dropping a short note; thanks for the hard work done at the Riak Java client, specially to /Brian Roach/, the client is becoming more and more stable and very professional. Have a nice weekend, Guido. ___ riak-users mailing list riak-u

Re: Multiple 'database' in Riak

2013-01-19 Thread Guido Medina
Neither or both mixed? You could have prefixes per buckets and have N applications per cluster and add a 5 nodes cluster as needed, so that you can host as many per cluster, that way you know each bucket prefix point to an specific app. Hope that helps, Guido. On 19/01/13 17:26, Jeremiah Pes

Re: Multiple 'database' in Riak

2013-01-19 Thread Guido Medina
I suppose a separate cluster would make good sense? ie, 2 x 10 node cluster, buckets prefixed by app ? Or would you have prefixes with + + ? Jimmy. On Sat, Jan 19, 2013 at 5:47 PM, Guido Medina mailto:guido.med...@temetra.com>> wrote: Neither or both mixed? You could hav

Re: blocking get call?

2013-01-28 Thread Guido Medina
That would defy the purpose of distributed and high availability; the short answer is no, you will have to implement at the application level a blocking policy for the keys, assuming you know your key before hand /(think if it of your primary key if it was an SQL DB)/, in Riak last write wins,

Re: Is JSON number stored in binary format?

2013-01-28 Thread Guido Medina
I developed a custom IntRangeCollection class in Java for that, we store list of IDs (That come from a DB sequence), like Sean said; you will have to do it yourself, I spent a couple of days looking for something like that and ended up implementing it myself: http://stackoverflow.com/questions

Java client 2i question...

2013-01-29 Thread Guido Medina
Hi, On our presentation of how Temetra uses Riak on Dublin, we were told there was a special index which can list me the keys on a bucket using a 2i search, if so, does anyone has any code sample using the Java client to do that? Something like $[bucket _name] index? If so, is there any dang

Java client question...

2013-01-31 Thread Guido Medina
Hi, I have doubts about /withoutFetch()/ and /returnBody(boolean)/, I will put some scenarios: 1. Store object (or overwrite) existing Riak object where I'm 100% I don't need to fetch from Riak (Last write wins and goes to memory cache) 2. Apply a mutation to an object but this time return

Re: Java client 2i question...

2013-01-31 Thread Guido Medina
ences help someone out there//,/ Guido. On 29/01/13 18:44, Guido Medina wrote: Hi, On our presentation of how Temetra uses Riak on Dublin, we were told there was a special index which can list me the keys on a bucket using a 2i search, if so, does anyone has any code sample using the Java client

Re: Riak node issue after fresh install...

2013-02-01 Thread Guido Medina
Fixed, some process unknown to me was running on that server on 8087, sorry for the trouble. Guido. On 01/02/13 11:42, Guido Medina wrote: I force removed a Riak node from our development cluster, then wajig purge riak to delete all remaining files, and after fresh install it doesn't wa

Re: Tune Riak for fast inserts - populate DB

2013-02-13 Thread Guido Medina
Are you transferring using a single thread? If so, I would recommend you to use a ThreaPoolExecutor and schedule each write as you, control the failures (if any) using either an AtomicInteger or a concurrent/synchronized list where you can track the keys that failed. No matter how much you do,

Re: Tune Riak for fast inserts - populate DB

2013-02-13 Thread Guido Medina
13/02/13 10:29, Guido Medina wrote: Are you transferring using a single thread? If so, I would recommend you to use a ThreaPoolExecutor and schedule each write as you, control the failures (if any) using either an AtomicInteger or a concurrent/synchronized list where you can track the keys th

Re: best practices to work with buckets in java-client

2013-02-15 Thread Guido Medina
I would say it is totally safe to treat them as singleton (static reference or just singleton pattern), we have been doing that for a year with no issues so far. Hope that helps, Guido. On 15/02/13 22:07, Mikhail Tyamin wrote: Hello guys, what is the best way to work with Bucket object in j

Re: [ANN] Riak 1.3 Released

2013-02-21 Thread Guido Medina
Getting drunk now !!! Cheers, Guido. On 21/02/13 14:26, Dmitry Demeshchuk wrote: HELL YEAH On Thu, Feb 21, 2013 at 6:24 PM, Tom Santero > wrote: Here Ye! Here Ye! The Basho Engineering team is thrilled to announced that Riak 1.3 is official and ready

Re: 2i streaming

2013-03-04 Thread Guido Medina
We have done so far up to 3.5m keys listing for the same purpose using 2i search over protocol buffers and it seems to be fast enough. Maybe it is fast because it streams and compress the keys list directly into protocol buffer I/O stream without leaving a big foot print in memory? I don't kno

Re: using links or list of ids

2013-03-07 Thread Guido Medina
This is only an idea: You could mark your related objects with a 2i, when you need them retrieve the list of IDs from that 2i, fetch them concurrently into some local memory cache, and then navigate through your object graph by fetching them one by one from your local cache. It will be hard to

Re: simulate a transaction

2013-04-04 Thread Guido Medina
At least on two-phase commit enabled environment you can implement the rollback to "undo" your action, you expect things to go right and a very small % to go wrong, so implementing a rollback policy isn't such a bad idea, I had to do the same years ago for a payment client, when things went wro

Re: simulate a transaction

2013-04-04 Thread Guido Medina
h the rollback policy implementation. I know I'm suggesting the opposite (assume things went OK and take action when they go wrong) If that makes sense for your application. Guido. On 04/04/13 15:33, Guido Medina wrote: At least on two-phase commit enabled environment you can imp

Re: The suitability of MapReduce

2013-04-09 Thread Guido Medina
Rohman, It is more complicated than that, most big data systems use more than one DB engine (Including Facebook that uses like 5 different engines), for example (And we are not as big as Facebook), we use a relational SQL, a text search engine and Riak, you will have to balance each weaknes

Re: Post commit hooks are a single process, so they are executed in the same order as the commits ?

2013-04-09 Thread Guido Medina
Simon, We use a similar approach, using 2i and batch numbers (every bucket/key we are interested is stamped with a 2i batch number which increases once a minute), a Java client that copies to two different clusters once a minute also, from "last batch number copied" to "current batch number

Re: Migration from memcachedb to riak

2013-07-10 Thread Guido Medina
Then you are better off with Bitcask, that will be the fastest in your case (no 2i, no searches, no M/R) HTH, Guido. On 10/07/13 09:49, Edgar Veiga wrote: Hello all! I have a couple of questions that I would like to address all of you guys, in order to start this migration the best as possi

Re: Migration from memcachedb to riak

2013-07-10 Thread Guido Medina
/choosing-a-backend/LevelDB/ Or multi, use one as default and then the other for specific buckets: http://docs.basho.com/riak/1.2.0/tutorials/choosing-a-backend/Multi/ HTH, Guido. On 10/07/13 09:53, Guido Medina wrote: Then you are better off with Bitcask, that will be the fastest in your case (no

Re: Migration from memcachedb to riak

2013-07-10 Thread Guido Medina
d a lot more of physical nodes so more money spent... Instead we're using less machines with SSD disks to improve elevelDB performance. Best regards On 10 July 2013 09:58, Guido Medina <mailto:guido.med...@temetra.com>> wrote: Well, I rushed my answer before, if you want

Re: Migration from memcachedb to riak

2013-07-10 Thread Guido Medina
If you are using Java you could store Riak keys as binaries using Jackson smile format, supposedly it will compress faster and better than default Java serialization, we use it for very large keys (say a key with a large collection of entries), the drawback is that you won't be able to easily r

Re: Migration from memcachedb to riak

2013-07-10 Thread Guido Medina
mance. Best regards On 10 July 2013 09:58, Guido Medina mailto:guido.med...@temetra.com>> wrote: Well, I rushed my answer before, if you want performance, you probably want Bitcask, if you want compression then LevelDB, the following links should help yo

Re: Upgrade path 1.4.0

2013-07-10 Thread Guido Medina
Release notes: https://github.com/basho/riak/blob/1.4/RELEASE-NOTES.md Maybe related to this? Known Issues leveldb 1.3 to 1.4 conversion The first execution of 1.4.0 leveldb using a 1.3.x or 1.

Re: Upgrade path 1.4.0

2013-07-10 Thread Guido Medina
ch the servers were being upgraded. You mention in the release notes that the init scripts have been adjusted - maybe that has something to do with it? Cheers, Toby - Original Message - From: "Guido Medina" To: "riak-users" Sent: Wednesday, 10 July, 2013 9:05:16 P

Re: Disable "snappy" compression within eLevelDB

2013-07-12 Thread Guido Medina
Hi Simon, With Riak 1.4 I think there thre reasons why you shouldn't avoid it: 1. Less I/O overhead compressing objects. 2. Better compression. 3. Cause of 1 and 2, better performance. Try Riak 1.4 and tail -f /var/log/riak/console.log and see the new features version. HTH, Guido. On 12/07

Re: Riak 1.3 YCSB Read Time Out

2013-07-12 Thread Guido Medina
Hi Veronika, Riak 1.4.0 was released yesterday, with the new streaming features nodes will stream keys to you instead of working with the whole result in memory, which will probably avoid the timeout. Though your RAM and specially using a VM could be a limiting factor, still worthy a try. On

+zdbbl recommended value

2013-07-15 Thread Guido Medina
Hi, We had an issue in 1.4 where 2i operations were timing out, after going through support we were suggested to use "+zdbbl 16384", on the "Configuring Riak Docs" it is strongly suggested (unless doc need to be re-phrased) it should be higher: *Source:* http://docs.basho.com/riakcs/latest/

Re: +zdbbl recommended value

2013-07-15 Thread Guido Medina
o much memory usage penalty. -Jared On Mon, Jul 15, 2013 at 8:38 AM, Guido Medina mailto:guido.med...@temetra.com>> wrote: Hi, We had an issue in 1.4 where 2i operations were timing out, after going through support we were suggested to use "+zdbbl 16384", on

Re: riak_kv_memory_backend in production

2013-07-18 Thread Guido Medina
Since the data he is requiring to store is only "transient", would it make sense to set N=2 for performance? Or will N=2 have the opposite effect due to amount of nodes having such replica? Guido. On 18/07/13 16:15, Jared Morrow wrote: Kumar, We have a few customers who use the memory backen

Re: riak_kv_memory_backend in production

2013-07-18 Thread Guido Medina
Forgot to mention, with N=2 should he be able to have only 4 nodes and focus on RAM per node rather than 5? I know is not recommended but shouldn't N=2 reduce the minimum recommended nodes to 4? Guido. On 18/07/13 16:21, Guido Medina wrote: Since the data he is requiring to store is

Re: TCP recv timeout and handoffs almost all the time

2013-07-18 Thread Guido Medina
If what you are describing is happening for 1.4, type riak-admin diag and see the new recommended kernel parameters, also, on vm.args uncomment the +zdbbl 32768 parameter, since what you are describing is similar to what happened to us when we upgraded to 1.4. HTH, Guido. On 18/07/13 19:21,

Re: TCP recv timeout and handoffs almost all the time

2013-07-18 Thread Guido Medina
ously 12 node cluster. So maybe it is because of a 18 node cluster? I'll try the zdbbl stuff. Any other hint would be cool (if the new kernel parameters are also good for 1.3.1.. could you provide them?). Cheers Simon On Thu, 18 Jul 2013 19:34:18 +0100 Guido Medina wrote: If what you are

Re: Tweaking app.config and vm.args for best performance

2013-07-23 Thread Guido Medina
Take a look at http://docs.basho.com/riak/latest/ops/tuning/linux/ On the left menu, there are more tuning links, check them out too. HTH, Guido. On 23/07/13 14:50, rsb wrote: Environment - 5x Riak 1.4 nodes - Bitcask as a backend - Dedicated HDDs for each node storage, no RAID - Virtualized

Re: Tweaking app.config and vm.args for best performance

2013-07-24 Thread Guido Medina
I'm not sure, I would suggest you to go through release notes, the link I sent you is for the latest version which is 1.4, depending on the version of Riak you want to use or required to use different settings/tuning apply, go chronologically from 1st relevant version to last and see what appli

Re: Setting up my Riak architecture

2013-07-24 Thread Guido Medina
Riak is a multi-master database, your client will create a connection pool to all nodes in the cluster. For production it is recommended at least 5 nodes in the cluster, to get started, I guess you should first read some documentation, here is some that can help you: http://docs.basho.com/riak

Re: Tweaking app.config and vm.args for best performance

2013-07-24 Thread Guido Medina
, [ %% Set TCP_NODELAY {disable_http_nagle, true} ]}, -- Dave Brady - Original Message - From: "Guido Medina" To: "riak-users" Sent: Wednesday, July 24, 2013 12:29:06 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: Re: Tweaking ap

Java client stats question

2013-07-25 Thread Guido Medina
Hi, Is there a way to get the JSON stats via PBC? This is how we are doing it now, we would like to get rid of any HTTP call, currently, this is the only call being made to HTTP: /private void collectNodeInfo(final PBClientConfig clientConfig)// // {// //...// //RiakClusterStats stat

Re: Java client stats question

2013-07-25 Thread Guido Medina
wrote: Guido - Right now, no. We've been having some internal discussions around that topic and whether it's really a "client library" operation or not. How are you using stats? Is it for a monitoring app or ... ? Thanks, Brian Roach On Thu, Jul 25, 2013 at 4:25 AM, Guid

Re: Querying multiple 2i via Intersections

2013-07-27 Thread Guido Medina
Rohman, I think the reason for this is that the cluster will have to do the whole intersection in memory, 2i only provides queries for 1 single index and then return that result to either the client streaming or not, intersection indeed will require a MapReduce job to get a hold of both lists

Re: Querying multiple 2i via Intersections

2013-07-27 Thread Guido Medina
cs. --- Jeremiah Peschka - Founder, Brent Ozar Unlimited MCITP: SQL Server 2008, MVP Cloudera Certified Developer for Apache Hadoop On Sat, Jul 27, 2013 at 11:16 AM, Guido Medina mailto:guido.med...@temetra.com>> wrote: Rohman, I think the reason for this is that the cluste

Re: Querying multiple 2i via Intersections

2013-07-27 Thread Guido Medina
r, Brent Ozar Unlimited MCITP: SQL Server 2008, MVP Cloudera Certified Developer for Apache Hadoop On Sat, Jul 27, 2013 at 11:44 AM, Guido Medina mailto:guido.med...@temetra.com>> wrote: Are you saying that you can join two 2i indexes? Let's say you have a 2i named "date&q

Re: Reuse of Buckets (Java Client)

2013-07-31 Thread Guido Medina
Yes, it is thread safe, you can treat them as singleton instances per bucket, the following order is the kind of the general usage pattern: * Fetch bucket. * Optional: If exists verify it has your application values (N value, etc) * If doesn't exist create it with your settings. * Cache it a

Re: Many-To-Many relationships

2013-08-04 Thread Guido Medina
Hi Massimiliano, I think your design is very thorough, I wouldn't worry about the cardinality of such index but its per index size (how many keys a single 2i index will return?) , in that case think of 2i as yet another relational DB (LevelDB), you should test it with many keys and check its

Re: Nailing down the bucket design?

2013-08-07 Thread Guido Medina
Whats the size of each unit JSON wise?, if it is too small, you could have the player's units inside a single key as a collection, that way when you fetch a player your key will contain the units and you could play around with mutations/locking of such player's key. And also, it will leverage y

Re: Nailing down the bucket design?

2013-08-07 Thread Guido Medina
As a 2nd thought, you could have a key per player on the player's bucket and a key with the collection of units per player on the unit's bucket. Guido. On 07/08/13 15:52, Guido Medina wrote: Whats the size of each unit JSON wise?, if it is too small, you could have the player's

Re: Java client - conflict resolver on both fetch() and store()?

2013-08-11 Thread Guido Medina
Hi Matt, Like Sean said, you should have a mutator if you are dealing with conflict resolution in domain objects; a good side effect of using a mutator is that Riak Java client will fetch-modify-write so your conflict resolver will be called once(?), if you don't use mutators, you get the eff

Re: Java client - conflict resolver on both fetch() and store()?

2013-08-11 Thread Guido Medina
ou haven't passed in a Resolver then the DefaultResolver is used which ... isn't really a "resolver" - it simply passes through an object if there's only one, or throws an exception if there's multiple (siblings) present. Thanks, - Roach On Sun, Aug 11, 2013

Re: Java client - conflict resolver on both fetch() and store()?

2013-08-11 Thread Guido Medina
. On 11/08/13 18:49, Guido Medina wrote: Hi Brian, I probably asked a similar question before, let's say you have an in-memory cache and a single writer (I know, not the best distributed design), if you do the following, take into account that we use mutations but we have no siblings enable

Re: Java client - conflict resolver on both fetch() and store()?

2013-08-11 Thread Guido Medina
the resolved object be the one passed? I'm doing some tests and if I do store a mutation returning the body without fetching, I get a new mutated object and not the one I passed + mutation. So I'm wondering if that was the original intention. Thanks,

Java client - without fetch and mutation?

2013-08-11 Thread Guido Medina
Hi Brian, */New thread for this, sorry for the hijacking./* Yes, without fetch should be indeed used without mutation or conflict resolution, originally we had mutations and siblings, but our application ended up creating too many siblings and made Riak fail miserably, so we disable the sibli

Re: different versions in upgrade

2013-08-13 Thread Guido Medina
Same here, except that Riak 1.3.2 did that for me automatically. As Jeremiah mentioned, you should go first to 1.3.2 on all nodes, per node the first time Riak starts it will take some time upgrading the 2i indexes storage format, if you see any weirdness then execute "riak-admin reformat-index

Re: different versions in upgrade

2013-08-13 Thread Guido Medina
ype "riak-admin reformat-indexes" and tail -f /var/log/riak/console.log which should be done really fast if there isn't anything to fix. 4. Do 1 to 3 per node. 5. Do 1 and 2 but for for Riak 1.4.1. HTH, Guido. On 13/08/13 13:50, Guido Medina wrote: Same here, except that Riak 1.3.2

Re: vm.args change for 15% to 80% improvement in leveldb

2013-08-14 Thread Guido Medina
For the following information should it be +S 4:4 or +S 4:8? root@somehost# lshw -C processor *-cpu description: CPU product: Intel(R) Core(TM) i7 CPU 930 @ 2.80GHz vendor: Intel Corp. physical id: 4 bus info: cpu@0 version: Intel(R) Core(TM) i

Re: vm.args change for 15% to 80% improvement in leveldb

2013-08-14 Thread Guido Medina
ical cores with no hyper threading, so total threads is also 8, would that still be "+S 4:4", "+S 8:8" or "+S 8:0" Thanks, Guido. On 14/08/13 15:41, Matthew Von-Maszewski wrote: "threads=8" is the key phrase … +S 4:4 On Aug 14, 2013, at 10:04 AM, Guido M

Java Client - 2i indexes streaming

2013-08-14 Thread Guido Medina
Hi, I have to say it is nice, we started using it today and it seems to leave a very low CPU and memory footprint at both cluster and application using the client, now I have a couple of questions: 1. This one is probably part of Riak 1.4.x but won't hurt to ask: Will reduce identity (to c

Re: Including @RiakKey and @RiakIndexes in the stored JSON

2013-08-22 Thread Guido Medina
Create a pseudo getters for the 2i indexes, valid return types are: String, Long (And Integer) and Set of any of the mentioned, the benefit of this is the fact that your 2i indexes are not actual properties, they are meant to be computation of something, example: /public class Postcode {// //

Re: use Siblings to implement a message queue?

2013-09-12 Thread Guido Medina
Alex, RabbitMQ which is a good high performer, developed in Erlang and scales just as Riak. The old saying, the right tool for the right job, I like how fast Riak is fetching/storing key values on a distributed environment, I don't like Riak for queues, is it because it wasn't designed for t

  1   2   >