So I should expect {error, notfound} inputs to map jobs while ownership
handoff is in progress? Are the not found items actually unavailable during
handoff or is this just not found on the old node, but will be picked up by
the new node during the same mapreduce job?
--
Jeremy
On Thu, Feb 28, 2013 at 11:28 AM, Jeremy Raymond <jeraym...@gmail.com>wrote:
> Yesterday I added a new node to my cluster. During the time when ownership
> handoff was happening (several hours of work) mapreduce map functions were
> receiving {error, notfound} as inputs. My Erlang mapred functions weren't
> designed to handle this. They hadn't encountered this before during normal
> operation. After the ownership handoff process completed the {error,
> notfound} inputs have stopped.
>
> Any explanations for the {error, notfound} inputs during ownership
> handoff? Is this because a node is attempting to process an object now
> moved to another node? If this is the case would the notfound object be
> found on the other node in the same mapreduce job (i.e. still visible to
> the overall mapred process)? Should I assume {error, notfound} inputs for
> all mapred jobs as a valid possible input and always handle it?
>
> I've also accumulated about 50MB of "Pipe worker startup failed:fitting
> was gone before startup" on each node during the ownership_transfer
> process. These messages are benign?
>
> Thanks a lot.
>
> --
> Jeremy
>
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
