degreane,
The performance of my implement is not good enough, the memory consumption
is pretty hurge. As I just used the fs.createReadStream api that said the
stream will auto close after end/error emitted, but the momory not freed.
Need your help. Are you implemented your lib with C++ of memory mapping or
just control the buffer? Or if it is okay for you to share your experiences
to me. Thanks a lot.
Regards,
Sun
On Thursday, October 31, 2013 11:42:45 PM UTC+8, degreane wrote:
>
> Sir ... All what you are trying to implement is fine and from one point
> of View is very much doable ...
>
> the question is how good is Node.js regarding streaming binary data files.
> I have been doing the same stuff that you are trying to solve now and i
> succeeded with node but when i put it into action it was not good enough in
> terms of performance ( The ammount of data cached was
> enourmous,functionality was great ) but response and latency and most of
> all memory usage due to buffers and streams when pushed to a server of
> 1000+ req/s the performance degraded so i ended up stitching everything in
> C++ using libevent .
>
> maybe i had some buffers initialized and never freed and maybe i was doing
> something wrong back then
> i had it on Node 0.8.22 ... and since then i just use node for normal MVC
> websites and heavey stuff i either code it with C/C++ or even Golang....
>
> Much Regards
>
> Faysal Banna
> Meteorological Services
> Rafic Harriri International Airport
> Beirut - Lebanon
> Mob: +961-3-258043
>
> On 10/31/2013 03:50 PM, yu sun wrote:
>
> Thanks for your time to look into this post.
>
> Basic I want to implement one file download cache server for local
> usage.
>
> The server should meet the following requirements:
> 1, Save target file/data into local server's disk.
> 2, Forwarding data stream.
> 3, Support resumable data cache.
> 4, Support partial download with Range header.
>
> *Scenario A (**Target File cached in local server's disk**):*
> * * For this scenario, i can download the target data from the
> local cache server. the server can simply server the data with pipe one
> readStream.
> *
> *
> *Scenario B (**Target File isnot in local server's disk**):*
> * *For this, i can pipe the source to multi target
> destinations,
> Like:
>
>> var req = request.get('
>>> http://nodejs.org/dist/v0.10.21/node-v0.10.21.tar.gz', function(err,
>>> response, body) {
>>
>> if (err) {...}
>>
>> });
>>
>> req.pipe(targetStream1);
>>
>> req.pipe(res);
>>
>> *
> *
> *Scenario C (**Target File partly cached in local server's disk due to
> network exception**):** *
> * * So when someone try to get the target data with partly
> cached, the server should do two things.
> 1st, response cached data with one stream quickly.
> 2nd, forward the remaining chunk under downloading.
>
> Any suggestion? Great thanks to you all.
>
> --
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to [email protected] <javascript:>
> To unsubscribe from this group, send email to
> [email protected] <javascript:>
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
> ---
> You received this message because you are subscribed to the Google Groups
> "nodejs" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected] <javascript:>.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
---
You received this message because you are subscribed to the Google Groups
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
