[R] Eleaps in package subselect crashes when using include arguement

2012-07-26 Thread Nathan Stephens
I'm using eleaps to build a forward selection algorithm iteratively, but the program unexpectedly crashes. In fact, it completely closes my session in RStudio. The first 39 steps work fine, but on the 40th step, it unexpectedly stops with no errors. I've isolated the error to the code snippit below

[R] Optimization inconsistencies

2012-05-17 Thread Nathan Stephens
I have a very simple maximization problem where I'm solving for the vector x: objective function: w'x = value to maximize box constraints (for all elements of w): low < x < high equality constraint: sum(x) = 1 But I get inconsistent results depending on what starting values I. I've tried variou

[R] glmnet sparse matrix error: dim specifies too large an array

2012-04-23 Thread Nathan Stephens
I'm running into an unexpected error using the glmnet and Matrix packages. I have a matrix that is 8 million rows by 100 columns with 75% of the entries being zero. When I run a vanilla glmnet logistic model on my server with 300 GB of RAM, the task completes in 20 minutes: > x # 8 million x 100

[R] What is the largest in memory data object you've worked with in R?

2010-06-04 Thread Nathan Stephens
For me, I've found that I can easily work with 1 GB datasets. This includes linear models and aggregations. Working with 5 GB becomes cumbersome. Anything over that, and R croaks. I'm using a dual quad core Dell with 48 GB of RAM. I'm wondering if there is anyone out there running jobs in the 1

[R] Min hash

2010-01-25 Thread Nathan Stephens
Anyone know of a min hash algorithm written in R? --Nathan [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-

[R] Server hanging despite efforts to correct memory limits

2010-01-19 Thread Nathan Stephens
My group is working with datasets between 100 Mb and 1 GB in size, using multiple log ins. From the documentation, it appears that vsize is limited to 2^30-1, which tends to prove too restrictive for our use. When we drop that restriction (set vsize = NA) we end up hanging the server, which requi

[R] R jobs keep hanging linux server despite mem.limits modifcations

2010-01-18 Thread Nathan Stephens
My group is working with datasets between 100 Mb and 1 GB in size, using multiple log ins. From the documentation, it appears that vsize is limited to 2^30-1, which tends to prove too restrictive for our use. When we drop that restriction (set vsize = NA) we end up hanging the server, which requi