Re: Massive json schema update

2016-06-13 Thread Damien Krotkine
My advice : use the fetch-update loop that you already know. You won't get better performance or reliability by using a MapReduce. I understand that your values are indexed in Search. So either use a search query and update them all, or you could also do a list_keys in stream mode, and updated

Re: Massive json schema update

2016-06-08 Thread Guillaume Boddaert
Well, We are talking about ~30 million items, in a single bucket. Those are standard tweets, I need to alter all of them because of an invalid structure. My cluster is not in production, but I am willing to start validating a procedure that will be used at some point on a production cluster,

Re: Massive json schema update

2016-06-08 Thread Damien Krotkine
Hi Guillaume, If I understand correctly you need to change all the values of your JSON data. How many keys are we talking about, how big are the data, and in how many buckets are the keys? Also, is your cluster in production yet? > Le 7 juin 2016 à 18:43, Guillaume Boddaert > a écrit : >