And, what about: - Master process loads the files, and build the result you need - Then, it sends the result to every child process via a message
Unless the desired result is child-dependent On Fri, Nov 30, 2012 at 5:04 PM, spqr <[email protected]> wrote: > The child process paradigm is fine for what I'm doing and there should be > no problems with the message passing. > My main concern is "system startup" not message passing. Each process has > to have loaded (potentially) thousands of > JavaScript files and built some summary data off the loaded files. When it > does process the messages it receives > all of the JavaScript objects are part of the processes state and it can > execute the ones it needs to based on the messages it > receives. I don't really know if loading those thousands of small > JavaScripts will be slow or fast, maybe there is no problem. > It just seems wasteful to have each process go through the exact same > loading process and it seems like there should be > some way to do most of that work just once. I was hoping there might be > some clever and tricky way to do it ;-) > > > On Friday, November 30, 2012 2:35:21 PM UTC-5, ajlopez wrote: > >> Umm... maybe I misunderstood the problem, or the below proposal: use >> cluster (or child process >> http://nodejs.org/api/**child_process.html<http://nodejs.org/api/child_process.html>) >> and send a message from master to workers. I don't know the limitations of >> this kind of communitacion >> >> http://nodejs.org/api/cluster.**html <http://nodejs.org/api/cluster.html> >> >> example will echo back all messages from the master: >> >> if (cluster.isMaster) { >> var worker = cluster.fork(); >> worker.send('hi there'); >> } else if (cluster.isWorker) { >> process.on('message', function(msg) { >> process.send(msg); >> });} >> >> >> On Fri, Nov 30, 2012 at 4:21 PM, spqr <[email protected]> wrote: >> >>> Thanks for connecting the dots. I was missing the connection to the OS >>> exec environment. >>> Trying to use the "env" definitely won't be a good solution to my >>> particular problem. The thousands >>> of scripts I will be loading are variable sized and may vary quite a bit >>> depending on how much data >>> they contain. A lot of data structures have to be populated as the >>> scripts are loaded. It looks to me >>> like my only choice is to have each process perform the load. One other >>> thought I have is to maybe have >>> the master communication process load all the files and write out some >>> sort of compilation file that >>> all the child processes would load instead or re-executing the entire >>> process. >>> >>> On Friday, November 30, 2012 11:35:13 AM UTC-5, Ben Noordhuis wrote: >>> >>>> On Fri, Nov 30, 2012 at 5:16 PM, spqr <[email protected]> wrote: >>>> > So, I guess this is a fairly stupid question because I can pass >>>> everything >>>> > to the process as part of the "env" >>>> > Maybe a new question is that since I'm talking about "quite a lot of >>>> junk," >>>> > is there any negative consequence >>>> > of passing a really large env to the child process. >>>> >>>> Not intrinsically but most (all?) operating systems put some arbitrary >>>> upper limit on the environment size. On modern Linux, for example, >>>> the maximum length of arguments + environment is usually (but not >>>> always) 2 MB. >>>> >>>> In case of doubt, consult the limitations section of `man execve`. >>>> >>> -- >>> Job Board: http://jobs.nodejs.org/ >>> Posting guidelines: https://github.com/joyent/**node/wiki/Mailing-List-* >>> *Posting-Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines> >>> You received this message because you are subscribed to the Google >>> Groups "nodejs" group. >>> To post to this group, send email to [email protected] >>> >>> To unsubscribe from this group, send email to >>> nodejs+un...@**googlegroups.com >>> >>> For more options, visit this group at >>> http://groups.google.com/**group/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en> >>> >> >> -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
