Have you every take on look on https://github.com/caolan/async ? There is
the async.mapSeries function which takes your items and run them serial.
The code then can look like:
async.mapSeries(items, function(item, callback) {
worker(items, item, callback);
}, function(err, results) {
// Your error or results are here
});
When you depend on process.nextTick you can simply wrap your iterator with
it.
Am Donnerstag, 12. April 2012 04:32:24 UTC+2 schrieb Tim Dickinson:
>
> Thanks for the snippet. I have taken your var result = new Array(length);
> to optimize the array.
>
> Why I use a function for the call back with the result is to play on the
> make call now get result latter. Really I'm using this function to update
> data in my database.
>
>
> Array.prototype.forLoop = function(worker, callBack) {
> var items = this;
> var length = items.length;
> var results = new Array(length);
> var errors = new Array(length);
>
> var index = 0;
>
> function doNext() {
> worker.call(items, items[index], function(err, result) {
> results[index] = result;
> errors[index] = err;
> index++;
> if(index === length)
> callback(errors, results);
> else
> process.nextTick(doNext);
> });
> }
>
> process.nextTick(doNext);
>
> };
>
> On Wednesday, April 11, 2012 5:13:52 PM UTC-4, Tim Caswell wrote:
>>
>> Ok, so you have a large array and want to do some work on it but break up
>> the work across several ticks? That's fine. I still don't understand why
>> your workers have a callback if they are sync. So assuming sync workers
>> you could do something like:
>>
>> var queue = [1,2,3,4,5];
>>
>> function processSlowly(items, work, callback) {
>> var length = items.length;
>> var results = new Array(length);
>> var index = 0;
>> function doNext() {
>> results[index] = work(items[index]);
>> index++;
>> if (index === length) return callback(results);
>> process.nextTick(doNext);
>> }
>> process.nextTick(doNext);
>> }
>>
>>
>> If your workers are async, you don't need nectTick because the callback
>> is already called on a new tick.
>>
>> function processSerialy(items, work, callback) {
>> var length = items.length;
>> var results = new Array(length);
>> var index = 0;
>> function doNext() {
>> work(items[index], function (result) {
>> results[index] = result;
>> index++;
>> if (index === length) return callback(results);
>> doNext();
>> });
>> }
>> process.nextTick(doNext);
>> }
>>
>> Also note that I don't have any error checking or handling here. I don't
>> know if the function should bail out on first error, if it should log the
>> error and continue or some other behavior.
>>
>> On Wed, Apr 11, 2012 at 3:33 PM, Tim Price <[email protected]> wrote:
>>
>>> OK good point.
>>>
>>> The stack would be the wrong word. What im trying to do is push eash
>>> loop around the array to be in a new cycle around node loop.
>>>
>>> I'm trying to keep node event loop running fast. The "asyncForEach" can
>>> take as long as it need, it would be bulk work. I dont want to through
>>> 1000+ function into one cycle around nodes event loop. Now 1000+ is
>>> an example as 100+ might be a better real work example.
>>>
>>> With your asyncForEach would be the same as the forEach that is already
>>> part of the language.
>>>
>>> Maybe you can see why i used the nextTick.
>>>
>>>
>>> On Wednesday, April 11, 2012 4:08:10 PM UTC-4, Tim Caswell wrote:
>>>>
>>>> My point being that you're showing your worker has a callback. If
>>>> there is a callback then I assume it's an async function. The callback
>>>> will
>>>> already be called in a new stack. There is no need to use nextTick to
>>>> break the stack. If the worker is a sync function then it should return
>>>> it's value instead of passing it via callback. Then you can use
>>>> Array.prototype.map as-is to get the resultant array.
>>>>
>>>> Assuming the worker is async, my parallel example was async and
>>>> parallel. If you want async and serial that can be done too similair to
>>>> your original example.
>>>>
>>>> Since you said "Your asyncForEach is at the end of the day synchronous
>>>> loop just like any other forloop" I'm confused and somewhere one of your
>>>> assumptions is wrong.
>>>>
>>>> On Wed, Apr 11, 2012 at 3:02 PM, Tim Caswell <[email protected]>wrote:
>>>>
>>>>> If you just want to keep the stack from growing a simple while or for
>>>>> loop is much easier.
>>>>>
>>>>> var items = [1,2,3,4];
>>>>>
>>>>> function forEach(items, worker) {
>>>>> for (var i = 0, l = items.length; i < l; i++) {
>>>>> worker(items[i], i, items);
>>>>> }
>>>>> }
>>>>>
>>>>> forEach(items, function (value, index, array) {
>>>>> });
>>>>>
>>>>> or just use the built-in Array.prototype.forEach
>>>>>
>>>>> items.forEach(function (value, index, array) {
>>>>> });
>>>>>
>>>>> I don't understand the goal. My example was assuming that worker was
>>>>> non-blocking and called the callback in a new tick. It was parallel and
>>>>> async.
>>>>>
>>>>>
>>>>> On Wed, Apr 11, 2012 at 2:49 PM, Tim Price <[email protected]>wrote:
>>>>>
>>>>>> I Wanted to keep the items in serially, as im trying to break the
>>>>>> stack. Your asyncForEach is at the end of the day synchronous loop just
>>>>>> like any other forloop.
>>>>>>
>>>>>> On Wednesday, April 11, 2012 3:33:18 PM UTC-4, Tim Caswell wrote:
>>>>>>>
>>>>>>> That depends on what you want it to do. This seems to execute each
>>>>>>> item serially. You can do them all parallel and abort on the first
>>>>>>> error
>>>>>>> like this:
>>>>>>>
>>>>>>> function asyncForEach(array, worker, callback) {
>>>>>>> var len = array.length;
>>>>>>> var results = new Array(l);
>>>>>>> if (!len) return callback(null, results);
>>>>>>> var left = len;
>>>>>>> for (var i = 0; i < len; i++) {
>>>>>>> start(i);
>>>>>>> }
>>>>>>> function start(i) {
>>>>>>> worker(array[i], i, array, function (err, result) {
>>>>>>> if (err) return callback(err);
>>>>>>> results[i] = result;
>>>>>>> if (!--left) callback(null, results);
>>>>>>> });
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Apr 11, 2012 at 2:07 PM, Tim Price <[email protected]>wrote:
>>>>>>>
>>>>>>>> How would one better write this function?
>>>>>>>>
>>>>>>>> Array.prototype.forLoop = function(worker, callBack) {
>>>>>>>> var self = this;
>>>>>>>> var returnData = [];
>>>>>>>> var loop = function(i) {
>>>>>>>> if(i === self.length) {
>>>>>>>> return callBack(returnData);
>>>>>>>> }
>>>>>>>> process.nextTick(function() {
>>>>>>>>
>>>>>>>> worker.call(self, self[i], function(d) {
>>>>>>>>
>>>>>>>> returnData.push(d);
>>>>>>>> loop(++i);
>>>>>>>> });
>>>>>>>> });
>>>>>>>> }
>>>>>>>> loop(0);
>>>>>>>> };
>>>>>>>>
>>>>>>>>
>>>>>>>> and you would use it like so
>>>>>>>>
>>>>>>>>
>>>>>>>> ['sdf', 'sdfsdf'].forLoop(function(**ite**m, done) {
>>>>>>>> console.log(item, done)
>>>>>>>> done(item)
>>>>>>>> }, function(result) {
>>>>>>>> console.log(result)
>>>>>>>> })
>>>>>>>>
>>>>>>>> --
>>>>>>>> Job Board: http://jobs.nodejs.org/
>>>>>>>> Posting guidelines: https://github.com/joyent/**node**
>>>>>>>> /wiki/Mailing-List-**Posting-**Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines>
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "nodejs" group.
>>>>>>>> To post to this group, send email to [email protected]
>>>>>>>> To unsubscribe from this group, send email to
>>>>>>>> nodejs+unsubscribe@**googlegroup**s.com<nodejs%[email protected]>
>>>>>>>> For more options, visit this group at
>>>>>>>> http://groups.google.com/**group**/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>> Job Board: http://jobs.nodejs.org/
>>>>>> Posting guidelines: https://github.com/joyent/**
>>>>>> node/wiki/Mailing-List-**Posting-Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines>
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "nodejs" group.
>>>>>> To post to this group, send email to [email protected]
>>>>>> To unsubscribe from this group, send email to
>>>>>> nodejs+unsubscribe@**googlegroups.com<nodejs%[email protected]>
>>>>>> For more options, visit this group at
>>>>>> http://groups.google.com/**group/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en>
>>>>>>
>>>>>
>>>>>
>>>> --
>>> Job Board: http://jobs.nodejs.org/
>>> Posting guidelines:
>>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>>> You received this message because you are subscribed to the Google
>>> Groups "nodejs" group.
>>> To post to this group, send email to [email protected]
>>> To unsubscribe from this group, send email to
>>> [email protected]
>>> For more options, visit this group at
>>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>>
>>
>>
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
