Thanks all for the reply. I think I need to think of other ways to approach
the problem.
Hanna
2016-01-24 17:45 GMT-05:00 William Dunlap <wdun...@tibco.com>:
> > 28 PiB. Storing such a large matrix even on file is not possible.
>
> The ads for Amazon Red Shift say it is possible. E.g.,
> Amazon Redshift is a fast, fully managed, petabyte-scale data
> warehouse that makes it simple and cost-effective to analyze
> all your data using your existing business intelligence tools.
> Start small for $0.25 per hour with no commitments and scale
> to petabytes for $1,000 per terabyte per year, less than a tenth
> the cost of traditional solutions. Customers typically see 3x
> compression, reducing their costs to $333 per uncompressed
> terabyte per year.
>
> Cost may be an issue:
>
> 28 petabytes * 1024 petabytes/terabyte * $333 terabyte/year ~= $9.5
> million/year
> or $26 thousand/day.
>
>
> Bill Dunlap
> TIBCO Software
> wdunlap tibco.com
>
> On Sun, Jan 24, 2016 at 1:29 PM, Henrik Bengtsson <
> henrik.bengts...@gmail.com> wrote:
>
>> FYI, the matrix you tried to allocate would hold
>> (3195*1290*495*35*35*35*15) * 3 = 3.936248e+15 values. Each value
>> would occupy 8 bytes of memory (for the double data type). In other
>> words, in order to keep this data matrix in memory you would require a
>> computer with at least 3.148998e+16 bytes of RAM, i.e. 29327331 GiB =
>> 28640 TiB = 28 PiB. Storing such a large matrix even on file is not
>> possible.
>>
>> In other words, you need to figure out how to approach your original
>> problem in a different way.
>>
>> /Henrik
>>
>> On Sun, Jan 24, 2016 at 8:46 AM, li li <hannah....@gmail.com> wrote:
>> > Hi all,
>> > I am doing some calculation with very large dimension. I need to
>> create a
>> > matrix
>> > with three columns and a very large number of rows
>> > (3195*1290*495*35*35*35*15=1.312083e+15) i
>> > n order to allocate calculation result from a for loop.
>> > R does not allow me to create such a matrix because of the large
>> dimension
>> > (see below). Is there a way to go around this?
>> > Thanks very much!!
>> > Hanna
>> >
>> >
>> >> matrix(0, 3195*1290*495*35*35*35*15, 3)
>> > Error in matrix(0, 3195 * 1290 * 495 * 35 * 35 * 35 * 15, 3) :
>> > invalid 'nrow' value (too large or NA)
>> > In addition: Warning message:
>> > In matrix(0, 3195 * 1290 * 495 * 35 * 35 * 35 * 15, 3) :
>> > NAs introduced by coercion
>> >>
>> >
>> > [[alternative HTML version deleted]]
>> >
>> > ______________________________________________
>> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> > https://stat.ethz.ch/mailman/listinfo/r-help
>> > PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> > and provide commented, minimal, self-contained, reproducible code.
>>
>> ______________________________________________
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
