Ok, so you have a large array and want to do some work on it but break up
the work across several ticks? That's fine. I still don't understand why
your workers have a callback if they are sync. So assuming sync workers
you could do something like:
var queue = [1,2,3,4,5];
function processSlowly(items, work, callback) {
var length = items.length;
var results = new Array(length);
var index = 0;
function doNext() {
results[index] = work(items[index]);
index++;
if (index === length) return callback(results);
process.nextTick(doNext);
}
process.nextTick(doNext);
}
If your workers are async, you don't need nectTick because the callback is
already called on a new tick.
function processSerialy(items, work, callback) {
var length = items.length;
var results = new Array(length);
var index = 0;
function doNext() {
work(items[index], function (result) {
results[index] = result;
index++;
if (index === length) return callback(results);
doNext();
});
}
process.nextTick(doNext);
}
Also note that I don't have any error checking or handling here. I don't
know if the function should bail out on first error, if it should log the
error and continue or some other behavior.
On Wed, Apr 11, 2012 at 3:33 PM, Tim Price <[email protected]> wrote:
> OK good point.
>
> The stack would be the wrong word. What im trying to do is push eash loop
> around the array to be in a new cycle around node loop.
>
> I'm trying to keep node event loop running fast. The "asyncForEach" can
> take as long as it need, it would be bulk work. I dont want to through
> 1000+ function into one cycle around nodes event loop. Now 1000+ is
> an example as 100+ might be a better real work example.
>
> With your asyncForEach would be the same as the forEach that is already
> part of the language.
>
> Maybe you can see why i used the nextTick.
>
>
> On Wednesday, April 11, 2012 4:08:10 PM UTC-4, Tim Caswell wrote:
>>
>> My point being that you're showing your worker has a callback. If there
>> is a callback then I assume it's an async function. The callback will
>> already be called in a new stack. There is no need to use nextTick to
>> break the stack. If the worker is a sync function then it should return
>> it's value instead of passing it via callback. Then you can use
>> Array.prototype.map as-is to get the resultant array.
>>
>> Assuming the worker is async, my parallel example was async and parallel.
>> If you want async and serial that can be done too similair to your original
>> example.
>>
>> Since you said "Your asyncForEach is at the end of the day synchronous
>> loop just like any other forloop" I'm confused and somewhere one of your
>> assumptions is wrong.
>>
>> On Wed, Apr 11, 2012 at 3:02 PM, Tim Caswell <[email protected]> wrote:
>>
>>> If you just want to keep the stack from growing a simple while or for
>>> loop is much easier.
>>>
>>> var items = [1,2,3,4];
>>>
>>> function forEach(items, worker) {
>>> for (var i = 0, l = items.length; i < l; i++) {
>>> worker(items[i], i, items);
>>> }
>>> }
>>>
>>> forEach(items, function (value, index, array) {
>>> });
>>>
>>> or just use the built-in Array.prototype.forEach
>>>
>>> items.forEach(function (value, index, array) {
>>> });
>>>
>>> I don't understand the goal. My example was assuming that worker was
>>> non-blocking and called the callback in a new tick. It was parallel and
>>> async.
>>>
>>>
>>> On Wed, Apr 11, 2012 at 2:49 PM, Tim Price <[email protected]>wrote:
>>>
>>>> I Wanted to keep the items in serially, as im trying to break the
>>>> stack. Your asyncForEach is at the end of the day synchronous loop just
>>>> like any other forloop.
>>>>
>>>> On Wednesday, April 11, 2012 3:33:18 PM UTC-4, Tim Caswell wrote:
>>>>>
>>>>> That depends on what you want it to do. This seems to execute each
>>>>> item serially. You can do them all parallel and abort on the first error
>>>>> like this:
>>>>>
>>>>> function asyncForEach(array, worker, callback) {
>>>>> var len = array.length;
>>>>> var results = new Array(l);
>>>>> if (!len) return callback(null, results);
>>>>> var left = len;
>>>>> for (var i = 0; i < len; i++) {
>>>>> start(i);
>>>>> }
>>>>> function start(i) {
>>>>> worker(array[i], i, array, function (err, result) {
>>>>> if (err) return callback(err);
>>>>> results[i] = result;
>>>>> if (!--left) callback(null, results);
>>>>> });
>>>>> }
>>>>> }
>>>>>
>>>>>
>>>>> On Wed, Apr 11, 2012 at 2:07 PM, Tim Price <[email protected]>wrote:
>>>>>
>>>>>> How would one better write this function?
>>>>>>
>>>>>> Array.prototype.forLoop = function(worker, callBack) {
>>>>>> var self = this;
>>>>>> var returnData = [];
>>>>>> var loop = function(i) {
>>>>>> if(i === self.length) {
>>>>>> return callBack(returnData);
>>>>>> }
>>>>>> process.nextTick(function() {
>>>>>>
>>>>>> worker.call(self, self[i], function(d) {
>>>>>>
>>>>>> returnData.push(d);
>>>>>> loop(++i);
>>>>>> });
>>>>>> });
>>>>>> }
>>>>>> loop(0);
>>>>>> };
>>>>>>
>>>>>>
>>>>>> and you would use it like so
>>>>>>
>>>>>>
>>>>>> ['sdf', 'sdfsdf'].forLoop(function(**ite**m, done) {
>>>>>> console.log(item, done)
>>>>>> done(item)
>>>>>> }, function(result) {
>>>>>> console.log(result)
>>>>>> })
>>>>>>
>>>>>> --
>>>>>> Job Board: http://jobs.nodejs.org/
>>>>>> Posting guidelines: https://github.com/joyent/**node**
>>>>>> /wiki/Mailing-List-**Posting-**Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines>
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "nodejs" group.
>>>>>> To post to this group, send email to [email protected]
>>>>>> To unsubscribe from this group, send email to
>>>>>> nodejs+unsubscribe@**googlegroup**s.com<nodejs%[email protected]>
>>>>>> For more options, visit this group at
>>>>>> http://groups.google.com/**group**/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en>
>>>>>>
>>>>>
>>>>> --
>>>> Job Board: http://jobs.nodejs.org/
>>>> Posting guidelines: https://github.com/joyent/**node/wiki/Mailing-List-
>>>> **Posting-Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines>
>>>> You received this message because you are subscribed to the Google
>>>> Groups "nodejs" group.
>>>> To post to this group, send email to [email protected]
>>>> To unsubscribe from this group, send email to
>>>> nodejs+unsubscribe@**googlegroups.com<nodejs%[email protected]>
>>>> For more options, visit this group at
>>>> http://groups.google.com/**group/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en>
>>>>
>>>
>>>
>> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to [email protected]
> To unsubscribe from this group, send email to
> [email protected]
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
