E Chalaron wrote:
> Hi all
>
> I am thinking of building a machine to do exclusively numbers munching .
> I need a lot of post processing (sometimes 1 to 2 TB of data) with
> colour correction through histogram, unsharp masking, stabilisation etc ...
>
> So ...
>
> 1. I am quite found of AMDs but eventually what I need is not a
> multitasking box, I need a calculator.
as long you don't processing in the temporal domain but only on single
frames, things are easily parallelizeable so multicore is an option
6x2ghz is still (way) more than 1x3ghz, the different stages of
processing can be even assigned to single cores (or do batch processing
in stipes, first only color correction, save to disk, then unsharp
masking). But caveat, bandwidth may become a limiting factor too, first
disk bandwidth (and latency) and next, when you do just simple not very
compute intensive transformations memory bandwidth might become a
limiting factor too.
For disk you certaily want some fast raid (raid 10) if you do the batch
rendering then possibly even 2 arrays that you can play ping-pong
rendering from one to the other back and forth, the 2nd one can then be
just raid0 on the expense that you may loose some (temporary) data in
case of a harddisk error, and be sure that you finally end with the
results on the array with redundance. Since this processing is highly
linear, much ram won't help much with caching, while when you have
enough ram you can tweak the disk accesses quite a bit (readahead, delay
writes, getting overall better performance because of less seeks) this
makes mostly sense when you have only one array where you read from and
write to. I highly recommend one of the newer/better filesystems which
can defer writes more aggressively and join writes (ext4) and some
hardcore kernel tuneing when you have to do this processing often.
For memory bandwidth it will only help to estimate (benchmark) what you
really need and then choose cpu/memory accordingly. IIRC AMD Opterons
are still best there because they have a NUMA architecture where each
socket has its own memory, but that of course requires a mainboard with
memory slots for each socket and equipped with sufficient memory
(dual/triple channel).
> 2. I wish I could use Cinelerra on a PS3 with Yellow Dog but not
> sure if that will ever be possible.
I somehow doubt that and even then, the plugins are not CELL aware, that
would need some serious coding efforts. We have some ideas for Lumiera
in that area but this isnt priority there either.
> 3. I don't know much about Intel these days.
iirc the current chips are damn fast (and cost a lot) the memory
bottleneck got improved but i think its still not as scalable as
opterons (which are pricey too).
When you really need a box to do this things (only) and often then it
makes sense to evaluate and benchmark your exact demands and choose the
hardware which fits it best.
>
> My idea is to have a core Cinelerra box in batch mode that will just go
> through the data located on other boxes as fast as possible.
> Eventually using rendering farm is an option too, but then the process
> will be as long as the slowest box...
> so ... not sure where from there...
> Any ideas? suggestions ? etc ...
dunno if i would really use cinelerra for it, it doesn't scale that well
and when something gets wrong you have to start over again. If its only
this batch rendering as you described i'd look into some commandline
tools to build the rendering pipe. ffmpeg, mencoder, gstreamer, AML.. if
you have enough disk bandwidth as i saied above then you may even do the
batch rendering with temporary files on disk, the advantage is that when
you do less at once you get overall better performance (due caching) and
can easily recover/restart when something broke in the middle. But this
really stands and fails with the disk bandwidth, if the cpu's start
waiting on the disks it will not work.
Christian
>
> Thanks a lot
> Edouard
>
_______________________________________________
Cinelerra mailing list
[email protected]
https://init.linpro.no/mailman/skolelinux.no/listinfo/cinelerra
