Another solution might be leveldb (using the [levelup][2] driver). /cc dominic, he should have an opinion on whether leveldb is suited.
Each client (browser or node.js) that is a writer has a leveldb (or [levelidb][1] for browsers) You can then trivially take a tailable live stream of all the data in a client (using [level-livefeed][3] or [level-live-stream][4]) and send it down any stream (like [engine.io-stream][5] or TCP) and pipe it into [levelup.writeStream][6] levelup has an example of this ["copy" functionality][7] 0) Installable with npm very npm friendly 1) Very high bandwidth, my app can use as much as you give it leveldb is in process. so it handles as much bandwidth as node does. 2) Multi-writers, including browsers and other servers if multiple people write to the same key the last one wins. 3) One reader on a single server Nothing enforces this. But the way it would be set up above is all writers send all data to the single reader server 5) Ordered data delivery would be nice but I could handle order problems in my code if your transport is ordered then data is ordered. However leveldb doesnt store a timestamp anywhere. You can trivially store them in the data or keys 6) Reliable delivery would be nice not not critical. End use of data is data-mining to be read by humans and I can show the human an indication data is missing. It would make my app look bad though. Delivery is as reliable as your transport. Advantages are that it works with any streaming transport. and that leveldb is very lightweight and minimalist. Disadvantages is how do you want to handle getting the data from the client to server. Do you want to flush the entire client db to the server which means there is lots of wasted bandwidth and excessive setting of keys that already exist. Or do you want to just want to stream only changes to the client db (not implemented in libraries but its a 3 line addition). The problem with streaming changes is that if your transport goes down changes get buffer in memory and if the client disconnects before the transport comes up those are lost. A second issues is how do you handle the last few messages in flight before the client disconnects, what if they get lost. These could be handled by replication but then you have to implement replication for your needs without flushing the entire db. [1]: https://github.com/Raynos/levelidb [2]: https://github.com/rvagg/node-levelup [3]: https://github.com/Raynos/level-livefeed [4]: https://github.com/dominictarr/level-live-stream [5]: https://github.com/Raynos/engine.io-stream [6]: https://github.com/rvagg/node-levelup#writeStream [7]: https://github.com/rvagg/node-levelup#pipes-and-node-stream-compatibility On Tue, Dec 18, 2012 at 1:57 PM, Mark Hahn <[email protected]> wrote: > @Ciprian > > Thx. I will definitely check out zeromq. > > On Tue, Dec 18, 2012 at 1:54 PM, Mark Hahn <[email protected]> wrote: > > You analysis is spot-on. Thx. > > > >> If your browser clients are new enough browsers, > > > > Yes. Only developers will use the tool so chrome will be preferred. > > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
