> > > Before we go into the merits of such functionality, what is a > plausible multiple writers/single reader scenario? The only one I can > come up is logging, where you probably don't care about the exact > order of writes. > > By the way, this 'thundering writers' issue seems like it's > straightforward to solve by sticking in an in-between stream that > wakes up writers selectively. >
It is indeed a form of logging. Consider a scenario where node.js is a web crawler and it's logging the response data of 10+ parallel web requests to a single file using a file format that allows the responses to be muxed together. I don't care about the particular ordering of the requests, but each requests data needs to be written in order so it can be demuxed later. If I have a thundering writer problem, I'm always going to be reading more data from the requests before the muxing stream is ready to write it. I'm piping the http responses to a custom writable stream that adds the muxing details per response which is then piped to a file backed writable stream, I'd be interested how you'd implement this in-between stream. Ideally it would also account for things like one request monopolizing the drain events and it would round robin the other waiting streams so there isn't the ability for one response to monopolize the writable stream. Likewise if the decision to mux all responses into one file is a bad design, I'd be happy to hear that as well but it seems reasonable to me. Rusty -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
