Re: [R] Runtime error using ncdf

[Camilo Mora]

Hi Dave,

I run into this problem with windows 7 and 8. I used the latest  
versions of r and ncdf as taken from icran.


I found my way around this problem breaking the files but it is a  
petty that ncdf has this bug as it is a quite useful package.


Thanks,

Camilo
Camilo Mora, Ph.D.
Department of Geography, University of Hawaii
Currently available in Colombia
Phone:   Country code: 57
 Provider code: 313
 Phone 776 2282
 From the USA or Canada you have to dial 011 57 313 776 2282
http://www.soc.hawaii.edu/mora/



Quoting "David W. Pierce" :


Hi Camilo,

you don't say what platform you are running on, but the version of the
underlying netcdf library installed on your machine must have been properly
compiled to enable access to files greater than 2GB. Otherwise,  the R
interface to the netcdf library cannot work with such files. You might
check how old your netcdf library is and/or whether it was compiled with
large file support.

If you are working on Linux, I suggest upgrading to ncdf4 instead of
continuing with ncdf; the ncdf package was deprecated many years ago and
does not get bug fixes.

Regards,

--Dave



On Sat, Aug 17, 2013 at 12:56 PM, Camilo Mora  wrote:


HI everyone,

I have encountered a problem while using ncdf to open nc files in R. I
found in the internet several comments in the past but no solution.

I could not find a direct solution but I found the source of the problem,
if anyone may know where the solution could be, and an indirect solution.

The problem occurs when opening large nc files with the ncdf package at
64bits. There may be a default setting on how big the files can be because
the file opens just fine in R at 32 bits (I could not use the 32 bit
version as very soon my process reached the RAM memory limit).

If of interest to anyone, I ended up splitting my files (making them
smaller than 2GB) using a software call NCO:
http://www.ncl.ucar.edu/**Support/talk_archives/2007/**1053.html
and then working in R at 64..

Cheers,

Camilo



Camilo Mora, Ph.D.
Department of Geography, University of Hawaii
Currently available in Colombia
Phone:   Country code: 57
 Provider code: 313
 Phone 776 2282
 From the USA or Canada you have to dial 011 57 313 776 2282
http://www.soc.hawaii.edu/**mora/ 

__**
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/**listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/**
posting-guide.html 
and provide commented, minimal, self-contained, reproducible code.





--
David W. Pierce
Division of Climate, Atmospheric Science, and Physical Oceanography
Scripps Institution of Oceanography, La Jolla, California, USA
(858) 534-8276 (voice)  /  (858) 534-8561 (fax)dpie...@ucsd.edu



__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] How can I make my functions run faster

[Laz]

Dear R users,

I have written a couple of R functions, some are through the help of the 
R group members. However, running them takes days instead of minutes or 
a few hours. I am wondering whether there is a quick way of doing that.


Here are all my R functions. The last one calls almost all of the 
previous functions. It is the one I am interested in most. It gives me 
the correct output but it takes several days to run only 1000 or 2000 
simulations!

e.g. system.time(test1<-finalF(designs=5,swaps=20));test1
will take about 20 minutes to run but 
system.time(test1<-finalF(designs=5,swaps=50));test1 takes about 10 
hours and system.time(test1<-finalF(designs=25,swaps=2000));test1 takes 
about 3 days to run


Here are my functions


#

ls() # list all existing objects
rm(list = ls()) # remove them all
rm(list = ls()[!grepl("global.var.A", ls())])
# refresh memory
gc()
ls()

### Define a function that requires useful input from the user
#b=4;g=seq(1,20,1);rb=5;cb=4;s2e=1; r=10;c=8

#

# function to calculate heritability
herit<-function(varG,varR=1)
{
  h<-4*varG/(varG+varR)
  return(c(heritability=h))
}

###
# function to calculate random error
varR<-function(varG,h2)
{
  varR<- varG*(4-h2)/h2
  return(c(random_error=varR))
}

##
# function to calculate treatment variance
varG<-function(varR=1,h2)
{
  varG<-varR*h2/(4-h2)
  return(c(treatment_variance=varG))
}


###

# calculating R inverse from spatial data
rspat<-function(rhox=0.6,rhoy=0.6)
{
  s2e<-1
  R<-s2e*eye(N)
  for(i in 1:N) {
for (j in i:N){
  y1<-y[i]
  y2<-y[j]
  x1<-x[i]
  x2<-x[j]
  R[i,j]<-s2e*(rhox^abs(x2-x1))*(rhoy^abs(y2-y1)) # Core AR(1)*AR(1)
  R[j,i]<-R[i,j]
}
  }
  IR<-solve(R)
  IR
}

ped<<-read.table("ped2new.txt",header=FALSE)
# Now work on the pedigree
## A function to return Zinverse from pedigree

ZGped<-function(ped)
{
  ped2<-data.frame(ped)
  lenp2<-length(unique(ped2$V1));lenp2 # how many Genotypes in total in 
the pedigree =40

  ln2<-length(g);ln2#ln2=nrow(matdf)=30
  # calculate the new Z
  Zped<-model.matrix(~ matdf$genotypes -1)# has order N*t = 180 by 30
  dif<-(lenp2-ln2);dif # 40-30=10
  #print(c(lenp2,ln2,dif))
  zeromatrix<-zeros(nrow(matdf),dif);zeromatrix # 180 by 10
  Z<-cbind(zeromatrix,Zped) # Design Matrix for random effect 
(Genotypes): 180 by 40

  # calculate the new G
  M<-matrix(0,lenp2,lenp2) # 40 by 40
  for (i in 1:nrow(ped2)) { M[ped2[i, 1], ped2[i, 2]] <- ped2[i, 3]  }
  G<-s2g*M # Genetic Variance covariance matrix for pedigree 2: 40 by 40
  IG<-solve(G)
  return(list(IG=IG, Z=Z))
}

##
##Required packages#

library(gmp)
library(knitr) # load this packages for publishing results
library(matlab)
library(Matrix)
library(psych)
library(foreach)
library(epicalc)
library(ggplot2)
library(xtable)
library(gdata)
library(gplots)

#b=6;g=seq(1,30,1);rb=5;cb=6;r=15;c=12;h2=0.3;rhox=0.6;rhoy=0.6;ped=0

setup<-function(b,g,rb,cb,r,c,h2,rhox=0.6,rhoy=0.6,ped="F")
  {
# where
# b   = number of blocks
# t   = number of treatments per block
# rb  = number of rows per block
# cb  = number of columns per block
# s2g = variance within genotypes
# h2  = heritability
# r   = total number of rows for the layout
# c   = total number of columns for the layout

### Check points
if(b==" ")
stop(paste(sQuote("block")," cannot be missing"))
if(!is.vector(g) | length(g)<3)
stop(paste(sQuote("treatments")," should be a vector and more 
than 2"))

if(!is.numeric(b))
stop(paste(sQuote("block"),"is not of class", sQuote("numeric")))
if(length(b)>1)
stop(paste(sQuote("block"),"has to be only 1 numeric value"))
if(!is.whole(b))
stop(paste(sQuote("block"),"has to be an", sQuote("integer")))

## Compatibility checks
if(rb*cb !=length(g))
   stop(paste(sQuote("rb x cb")," should be equal to number of 
treatment", sQuote("g")))

if(length(g) != rb*cb)
  stop(paste(sQuote("the number of treatments"), "is not equal to", 
sQuote("rb*cb")))


## Generate the design
g<<-g
genotypes<-times(b) %do% sample(g,length(g))
#genotypes<-rep(g,b)
block<-rep(1:b,each=length(g))
genotypes<-factor(genotypes)
block<-factor(block)

### generate the base design
k<-c/cb # number of blocks on the x-axis
x<<-rep(rep(1:r,each=cb),k)  # X-coordinate

#w<-rb
l<-cb
p<-r/rb
m<-l+1
d<-l*b/p
y<<-c(rep(1:l,r),rep(m:d,r)) # Y-coordinate

## compact
matdf<<-data.frame(x,y,block,genotypes)
N<<-nrow(matdf)
mm<-summ(matdf)
ss<-des(matdf)

## Identity matrices
X<<-model.matrix(~block-1)
h2<<-h2;rhox<<-rhox;rhoy<<-rhoy
s2g<<-varG

Re: [R] Adding additional points to ggplot2

[Chris89]

That worked great! 

Best regards, 
Chris



--
View this message in context: 
http://r.789695.n4.nabble.com/Adding-additional-points-to-ggplot2-tp4673928p4674059.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] How can I make my functions run faster

[Michael Dewey]

At 10:28 19/08/2013, Laz wrote:

Dear R users,

I have written a couple of R functions, some are through the help of 
the R group members. However, running them takes days instead of 
minutes or a few hours. I am wondering whether there is a quick way 
of doing that.


Your example code is rather long for humans to profile. Have you 
thought of getting R to tell where it is spending most time? The R 
extensions manual tells you how to do this.



Here are all my R functions. The last one calls almost all of the 
previous functions. It is the one I am interested in most. It gives 
me the correct output but it takes several days to run only 1000 or 
2000 simulations!

e.g. system.time(test1<-finalF(designs=5,swaps=20));test1
will take about 20 minutes to run but 
system.time(test1<-finalF(designs=5,swaps=50));test1 takes about 10 
hours and system.time(test1<-finalF(designs=25,swaps=2000));test1 
takes about 3 days to run


Here are my functions


#

ls() # list all existing objects
rm(list = ls()) # remove them all
rm(list = ls()[!grepl("global.var.A", ls())])
# refresh memory
gc()
ls()

### Define a function that requires useful input from the user
#b=4;g=seq(1,20,1);rb=5;cb=4;s2e=1; r=10;c=8

#

# function to calculate heritability
herit<-function(varG,varR=1)
{
  h<-4*varG/(varG+varR)
  return(c(heritability=h))
}

###
# function to calculate random error
varR<-function(varG,h2)
{
  varR<- varG*(4-h2)/h2
  return(c(random_error=varR))
}

##
# function to calculate treatment variance
varG<-function(varR=1,h2)
{
  varG<-varR*h2/(4-h2)
  return(c(treatment_variance=varG))
}


###

# calculating R inverse from spatial data
rspat<-function(rhox=0.6,rhoy=0.6)
{
  s2e<-1
  R<-s2e*eye(N)
  for(i in 1:N) {
for (j in i:N){
  y1<-y[i]
  y2<-y[j]
  x1<-x[i]
  x2<-x[j]
  R[i,j]<-s2e*(rhox^abs(x2-x1))*(rhoy^abs(y2-y1)) # Core AR(1)*AR(1)
  R[j,i]<-R[i,j]
}
  }
  IR<-solve(R)
  IR
}

ped<<-read.table("ped2new.txt",header=FALSE)
# Now work on the pedigree
## A function to return Zinverse from pedigree

ZGped<-function(ped)
{
  ped2<-data.frame(ped)
  lenp2<-length(unique(ped2$V1));lenp2 # how many Genotypes in 
total in the pedigree =40

  ln2<-length(g);ln2#ln2=nrow(matdf)=30
  # calculate the new Z
  Zped<-model.matrix(~ matdf$genotypes -1)# has order N*t = 180 by 30
  dif<-(lenp2-ln2);dif # 40-30=10
  #print(c(lenp2,ln2,dif))
  zeromatrix<-zeros(nrow(matdf),dif);zeromatrix # 180 by 10
  Z<-cbind(zeromatrix,Zped) # Design Matrix for random effect 
(Genotypes): 180 by 40

  # calculate the new G
  M<-matrix(0,lenp2,lenp2) # 40 by 40
  for (i in 1:nrow(ped2)) { M[ped2[i, 1], ped2[i, 2]] <- ped2[i, 3]  }
  G<-s2g*M # Genetic Variance covariance matrix for pedigree 2: 40 by 40
  IG<-solve(G)
  return(list(IG=IG, Z=Z))
}

##
##Required packages#

library(gmp)
library(knitr) # load this packages for publishing results
library(matlab)
library(Matrix)
library(psych)
library(foreach)
library(epicalc)
library(ggplot2)
library(xtable)
library(gdata)
library(gplots)

#b=6;g=seq(1,30,1);rb=5;cb=6;r=15;c=12;h2=0.3;rhox=0.6;rhoy=0.6;ped=0

setup<-function(b,g,rb,cb,r,c,h2,rhox=0.6,rhoy=0.6,ped="F")
  {
# where
# b   = number of blocks
# t   = number of treatments per block
# rb  = number of rows per block
# cb  = number of columns per block
# s2g = variance within genotypes
# h2  = heritability
# r   = total number of rows for the layout
# c   = total number of columns for the layout

### Check points
if(b==" ")
stop(paste(sQuote("block")," cannot be missing"))
if(!is.vector(g) | length(g)<3)
stop(paste(sQuote("treatments")," should be a vector and 
more than 2"))

if(!is.numeric(b))
stop(paste(sQuote("block"),"is not of class", sQuote("numeric")))
if(length(b)>1)
stop(paste(sQuote("block"),"has to be only 1 numeric value"))
if(!is.whole(b))
stop(paste(sQuote("block"),"has to be an", sQuote("integer")))

## Compatibility checks
if(rb*cb !=length(g))
   stop(paste(sQuote("rb x cb")," should be equal to number of 
treatment", sQuote("g")))

if(length(g) != rb*cb)
  stop(paste(sQuote("the number of treatments"), "is not equal 
to", sQuote("rb*cb")))


## Generate the design
g<<-g
genotypes<-times(b) %do% sample(g,length(g))
#genotypes<-rep(g,b)
block<-rep(1:b,each=length(g))
genotypes<-factor(genotypes)
block<-factor(block)

### generate the base design
k<-c/cb # number of blocks on the x-axis
x<<-rep(rep(1:r,each=cb),k)  # X-coordinate

#w<-rb
l<-cb
p<-r/rb
m<-l+1
d<-l*b/p
y<<-c(rep(1:l,r),rep(m:d,r)) # Y-coordinate

## com

Re: [R] How can I make my functions run faster

[Bert Gunter]

... and read the "R Language Definition" manual. I noticed unnecessary
constructs
(e.g., z <- f(something); return(z)) that suggest you have more basics
to learn to write efficient, well-structured R code.

-- Bert

On Mon, Aug 19, 2013 at 3:55 AM, Michael Dewey  wrote:
> At 10:28 19/08/2013, Laz wrote:
>>
>> Dear R users,
>>
>> I have written a couple of R functions, some are through the help of the R
>> group members. However, running them takes days instead of minutes or a few
>> hours. I am wondering whether there is a quick way of doing that.
>
>
> Your example code is rather long for humans to profile. Have you thought of
> getting R to tell where it is spending most time? The R extensions manual
> tells you how to do this.
>
>
>> Here are all my R functions. The last one calls almost all of the previous
>> functions. It is the one I am interested in most. It gives me the correct
>> output but it takes several days to run only 1000 or 2000 simulations!
>> e.g. system.time(test1<-finalF(designs=5,swaps=20));test1
>> will take about 20 minutes to run but
>> system.time(test1<-finalF(designs=5,swaps=50));test1 takes about 10 hours
>> and system.time(test1<-finalF(designs=25,swaps=2000));test1 takes about 3
>> days to run
>>
>> Here are my functions
>>
>>
>> #
>>
>> ls() # list all existing objects
>> rm(list = ls()) # remove them all
>> rm(list = ls()[!grepl("global.var.A", ls())])
>> # refresh memory
>> gc()
>> ls()
>>
>> ### Define a function that requires useful input from the user
>> #b=4;g=seq(1,20,1);rb=5;cb=4;s2e=1; r=10;c=8
>>
>> #
>> 
>> # function to calculate heritability
>> herit<-function(varG,varR=1)
>> {
>>   h<-4*varG/(varG+varR)
>>   return(c(heritability=h))
>> }
>>
>> ###
>> # function to calculate random error
>> varR<-function(varG,h2)
>> {
>>   varR<- varG*(4-h2)/h2
>>   return(c(random_error=varR))
>> }
>>
>> ##
>> # function to calculate treatment variance
>> varG<-function(varR=1,h2)
>> {
>>   varG<-varR*h2/(4-h2)
>>   return(c(treatment_variance=varG))
>> }
>>
>>
>> ###
>>
>> # calculating R inverse from spatial data
>> rspat<-function(rhox=0.6,rhoy=0.6)
>> {
>>   s2e<-1
>>   R<-s2e*eye(N)
>>   for(i in 1:N) {
>> for (j in i:N){
>>   y1<-y[i]
>>   y2<-y[j]
>>   x1<-x[i]
>>   x2<-x[j]
>>   R[i,j]<-s2e*(rhox^abs(x2-x1))*(rhoy^abs(y2-y1)) # Core AR(1)*AR(1)
>>   R[j,i]<-R[i,j]
>> }
>>   }
>>   IR<-solve(R)
>>   IR
>> }
>>
>> ped<<-read.table("ped2new.txt",header=FALSE)
>> # Now work on the pedigree
>> ## A function to return Zinverse from pedigree
>>
>> ZGped<-function(ped)
>> {
>>   ped2<-data.frame(ped)
>>   lenp2<-length(unique(ped2$V1));lenp2 # how many Genotypes in total in
>> the pedigree =40
>>   ln2<-length(g);ln2#ln2=nrow(matdf)=30
>>   # calculate the new Z
>>   Zped<-model.matrix(~ matdf$genotypes -1)# has order N*t = 180 by 30
>>   dif<-(lenp2-ln2);dif # 40-30=10
>>   #print(c(lenp2,ln2,dif))
>>   zeromatrix<-zeros(nrow(matdf),dif);zeromatrix # 180 by 10
>>   Z<-cbind(zeromatrix,Zped) # Design Matrix for random effect (Genotypes):
>> 180 by 40
>>   # calculate the new G
>>   M<-matrix(0,lenp2,lenp2) # 40 by 40
>>   for (i in 1:nrow(ped2)) { M[ped2[i, 1], ped2[i, 2]] <- ped2[i, 3]  }
>>   G<-s2g*M # Genetic Variance covariance matrix for pedigree 2: 40 by 40
>>   IG<-solve(G)
>>   return(list(IG=IG, Z=Z))
>> }
>>
>> ##
>> ##Required packages#
>> 
>> library(gmp)
>> library(knitr) # load this packages for publishing results
>> library(matlab)
>> library(Matrix)
>> library(psych)
>> library(foreach)
>> library(epicalc)
>> library(ggplot2)
>> library(xtable)
>> library(gdata)
>> library(gplots)
>>
>> #b=6;g=seq(1,30,1);rb=5;cb=6;r=15;c=12;h2=0.3;rhox=0.6;rhoy=0.6;ped=0
>>
>> setup<-function(b,g,rb,cb,r,c,h2,rhox=0.6,rhoy=0.6,ped="F")
>>   {
>> # where
>> # b   = number of blocks
>> # t   = number of treatments per block
>> # rb  = number of rows per block
>> # cb  = number of columns per block
>> # s2g = variance within genotypes
>> # h2  = heritability
>> # r   = total number of rows for the layout
>> # c   = total number of columns for the layout
>>
>> ### Check points
>> if(b==" ")
>> stop(paste(sQuote("block")," cannot be missing"))
>> if(!is.vector(g) | length(g)<3)
>> stop(paste(sQuote("treatments")," should be a vector and more than
>> 2"))
>> if(!is.numeric(b))
>> stop(paste(sQuote("block"),"is not of class", sQuote("numeric")))
>> if(length(b)>1)
>> stop(paste(sQuote("block"),"has to be only 1 numeric value"))
>> if(!is.whole(b))
>> stop(paste(sQuote("block"),"has to be an", sQuote("integer")))
>>
>> ## Compatibility checks
>> if(rb

Re: [R] How can I make my functions run faster

[Laz]

Yes Bert, I am a beginner in writing R functions. I just don't know what 
to avoid or what to use in order to make the R functions faster.


When I run the individual functions, they run quite well.
However, calling all of them using the final function it becomes too slow.

So I don't know how to make it faster.
I used system.time()

Regards,
Laz


On 8/19/2013 10:13 AM, Bert Gunter wrote:

... and read the "R Language Definition" manual. I noticed unnecessary
constructs
(e.g., z <- f(something); return(z)) that suggest you have more basics
to learn to write efficient, well-structured R code.

-- Bert

On Mon, Aug 19, 2013 at 3:55 AM, Michael Dewey  wrote:

At 10:28 19/08/2013, Laz wrote:

Dear R users,

I have written a couple of R functions, some are through the help of the R
group members. However, running them takes days instead of minutes or a few
hours. I am wondering whether there is a quick way of doing that.


Your example code is rather long for humans to profile. Have you thought of
getting R to tell where it is spending most time? The R extensions manual
tells you how to do this.



Here are all my R functions. The last one calls almost all of the previous
functions. It is the one I am interested in most. It gives me the correct
output but it takes several days to run only 1000 or 2000 simulations!
e.g. system.time(test1<-finalF(designs=5,swaps=20));test1
will take about 20 minutes to run but
system.time(test1<-finalF(designs=5,swaps=50));test1 takes about 10 hours
and system.time(test1<-finalF(designs=25,swaps=2000));test1 takes about 3
days to run

Here are my functions


#

ls() # list all existing objects
rm(list = ls()) # remove them all
rm(list = ls()[!grepl("global.var.A", ls())])
# refresh memory
gc()
ls()

### Define a function that requires useful input from the user
#b=4;g=seq(1,20,1);rb=5;cb=4;s2e=1; r=10;c=8

#

# function to calculate heritability
herit<-function(varG,varR=1)
{
   h<-4*varG/(varG+varR)
   return(c(heritability=h))
}

###
# function to calculate random error
varR<-function(varG,h2)
{
   varR<- varG*(4-h2)/h2
   return(c(random_error=varR))
}

##
# function to calculate treatment variance
varG<-function(varR=1,h2)
{
   varG<-varR*h2/(4-h2)
   return(c(treatment_variance=varG))
}


###

# calculating R inverse from spatial data
rspat<-function(rhox=0.6,rhoy=0.6)
{
   s2e<-1
   R<-s2e*eye(N)
   for(i in 1:N) {
 for (j in i:N){
   y1<-y[i]
   y2<-y[j]
   x1<-x[i]
   x2<-x[j]
   R[i,j]<-s2e*(rhox^abs(x2-x1))*(rhoy^abs(y2-y1)) # Core AR(1)*AR(1)
   R[j,i]<-R[i,j]
 }
   }
   IR<-solve(R)
   IR
}

ped<<-read.table("ped2new.txt",header=FALSE)
# Now work on the pedigree
## A function to return Zinverse from pedigree

ZGped<-function(ped)
{
   ped2<-data.frame(ped)
   lenp2<-length(unique(ped2$V1));lenp2 # how many Genotypes in total in
the pedigree =40
   ln2<-length(g);ln2#ln2=nrow(matdf)=30
   # calculate the new Z
   Zped<-model.matrix(~ matdf$genotypes -1)# has order N*t = 180 by 30
   dif<-(lenp2-ln2);dif # 40-30=10
   #print(c(lenp2,ln2,dif))
   zeromatrix<-zeros(nrow(matdf),dif);zeromatrix # 180 by 10
   Z<-cbind(zeromatrix,Zped) # Design Matrix for random effect (Genotypes):
180 by 40
   # calculate the new G
   M<-matrix(0,lenp2,lenp2) # 40 by 40
   for (i in 1:nrow(ped2)) { M[ped2[i, 1], ped2[i, 2]] <- ped2[i, 3]  }
   G<-s2g*M # Genetic Variance covariance matrix for pedigree 2: 40 by 40
   IG<-solve(G)
   return(list(IG=IG, Z=Z))
}

##
##Required packages#

library(gmp)
library(knitr) # load this packages for publishing results
library(matlab)
library(Matrix)
library(psych)
library(foreach)
library(epicalc)
library(ggplot2)
library(xtable)
library(gdata)
library(gplots)

#b=6;g=seq(1,30,1);rb=5;cb=6;r=15;c=12;h2=0.3;rhox=0.6;rhoy=0.6;ped=0

setup<-function(b,g,rb,cb,r,c,h2,rhox=0.6,rhoy=0.6,ped="F")
   {
 # where
 # b   = number of blocks
 # t   = number of treatments per block
 # rb  = number of rows per block
 # cb  = number of columns per block
 # s2g = variance within genotypes
 # h2  = heritability
 # r   = total number of rows for the layout
 # c   = total number of columns for the layout

 ### Check points
 if(b==" ")
 stop(paste(sQuote("block")," cannot be missing"))
 if(!is.vector(g) | length(g)<3)
 stop(paste(sQuote("treatments")," should be a vector and more than
2"))
 if(!is.numeric(b))
 stop(paste(sQuote("block"),"is not of class", sQuote("numeric")))
 if(length(b)>1)
 stop(paste(sQuote("block"),"has to be only 1 numeric value"))
 if(!is.whole(b))
 stop(paste(sQuote("block"),"has to be an", sQuote("integer")))

 ## Compati

Re: [R] How can I make my functions run faster

[Heramb Gadgil]

Greetings,

I am a newbie too. I will share what I do normally for speeding up the code.

1. Restrict defining too many variables (Global/ Local)
2. Use apply functions (apply,sapply,lapply,tapply, etc.) whenever feasible
3. Having multiple user defined functions doesn't help. Try to compact
everything in minimum number of functions
4. The in-memory of R is just 10% of your total RAM (Correct me if wrong).
Make sure most of it is used for processing and not storing

Hope this will help. Kindly suggest if I have misunderstood anything.

Thanks and Regards,

Heramb Gadgil


2013/8/19 Laz 

> Yes Bert, I am a beginner in writing R functions. I just don't know what
> to avoid or what to use in order to make the R functions faster.
>
> When I run the individual functions, they run quite well.
> However, calling all of them using the final function it becomes too slow.
>
> So I don't know how to make it faster.
> I used system.time()
>
> Regards,
> Laz
>
>
> On 8/19/2013 10:13 AM, Bert Gunter wrote:
>
>> ... and read the "R Language Definition" manual. I noticed unnecessary
>> constructs
>> (e.g., z <- f(something); return(z)) that suggest you have more basics
>> to learn to write efficient, well-structured R code.
>>
>> -- Bert
>>
>> On Mon, Aug 19, 2013 at 3:55 AM, Michael Dewey 
>> wrote:
>>
>>> At 10:28 19/08/2013, Laz wrote:
>>>
 Dear R users,

 I have written a couple of R functions, some are through the help of
 the R
 group members. However, running them takes days instead of minutes or a
 few
 hours. I am wondering whether there is a quick way of doing that.

>>>
>>> Your example code is rather long for humans to profile. Have you thought
>>> of
>>> getting R to tell where it is spending most time? The R extensions manual
>>> tells you how to do this.
>>>
>>>
>>>  Here are all my R functions. The last one calls almost all of the
 previous
 functions. It is the one I am interested in most. It gives me the
 correct
 output but it takes several days to run only 1000 or 2000 simulations!
 e.g. system.time(test1<-finalF(**designs=5,swaps=20));test1
 will take about 20 minutes to run but
 system.time(test1<-finalF(**designs=5,swaps=50));test1 takes about 10
 hours
 and system.time(test1<-finalF(**designs=25,swaps=2000));test1 takes
 about 3
 days to run

 Here are my functions


 ##**##**
 #

 ls() # list all existing objects
 rm(list = ls()) # remove them all
 rm(list = ls()[!grepl("global.var.A", ls())])
 # refresh memory
 gc()
 ls()

 ### Define a function that requires useful input from the user
 #b=4;g=seq(1,20,1);rb=5;cb=4;**s2e=1; r=10;c=8

 ##**###
 ##**##
 # function to calculate heritability
 herit<-function(varG,varR=1)
 {
h<-4*varG/(varG+varR)
return(c(heritability=h))
 }

 ##**#
 # function to calculate random error
 varR<-function(varG,h2)
 {
varR<- varG*(4-h2)/h2
return(c(random_error=varR))
 }

 ##**
 # function to calculate treatment variance
 varG<-function(varR=1,h2)
 {
varG<-varR*h2/(4-h2)
return(c(treatment_variance=**varG))
 }


 ##**#

 # calculating R inverse from spatial data
 rspat<-function(rhox=0.6,rhoy=**0.6)
 {
s2e<-1
R<-s2e*eye(N)
for(i in 1:N) {
  for (j in i:N){
y1<-y[i]
y2<-y[j]
x1<-x[i]
x2<-x[j]
R[i,j]<-s2e*(rhox^abs(x2-x1))***(rhoy^abs(y2-y1)) # Core
 AR(1)*AR(1)
R[j,i]<-R[i,j]
  }
}
IR<-solve(R)
IR
 }

 ped<<-read.table("ped2new.txt"**,header=FALSE)
 # Now work on the pedigree
 ## A function to return Zinverse from pedigree

 ZGped<-function(ped)
 {
ped2<-data.frame(ped)
lenp2<-length(unique(ped2$V1))**;lenp2 # how many Genotypes in
 total in
 the pedigree =40
ln2<-length(g);ln2#ln2=nrow(**matdf)=30
# calculate the new Z
Zped<-model.matrix(~ matdf$genotypes -1)# has order N*t = 180 by 30
dif<-(lenp2-ln2);dif # 40-30=10
#print(c(lenp2,ln2,dif))
zeromatrix<-zeros(nrow(matdf),**dif);zeromatrix # 180 by 10
Z<-cbind(zeromatrix,Zped) # Design Matrix for random effect
 (Genotypes):
 180 by 40
# calculate the new G
M<-matrix(0,lenp2,lenp2) # 40 by 40
for (i in 1:nrow(ped2)) { M[ped2[i, 1], ped2[i, 2]] <- ped2[i, 3]  }
G<-s2g*M # Genetic Variance covariance matrix for pedigree 2: 40 by
 40
IG<-solve(G)
return(list(IG=IG, Z=Z))
 }

 ##

Re: [R] A function taken out of its original environment performs more slowly.

[Duncan Murdoch]

On 13-08-17 7:05 PM, Xiao He wrote:

Hi dear R-users,

I encountered an interesting pattern. Take for example the function
combn(), I copied and pasted the function definition and saved it as a new
function named combn2() (see the end of this email). As it turned out,
combn2() seems to be substantially slower than the original function
combn() (see benchmark below),


Besides the difference Uwe pointed out, those functions likely have 
different environments, so searching for symbols will take a different 
amount of time.  Usually this will be longer from globalenv() than from 
the namespace of the package, but sometimes the reverse could be true.


Duncan Murdoch




system.time(combn(30, 5)); system.time(combn2(30, 5))

user  system elapsed
   0.304   0.003   0.308
user  system elapsed
   1.591   0.007   1.602


I wonder if there is any reason for this difference and if there is any way
to reduce the performance difference. Thanks!

combn2 <- function (x, m, FUN = NULL, simplify = TRUE, ...)
{
 stopifnot(length(m) == 1L)
 if (m < 0)
 stop("m < 0", domain = NA)
 if (is.numeric(x) && length(x) == 1L && x > 0 && trunc(x) ==
 x)
 x <- seq_len(x)
 n <- length(x)
 if (n < m)
 stop("n < m", domain = NA)
 m <- as.integer(m)
 e <- 0
 h <- m
 a <- seq_len(m)
 nofun <- is.null(FUN)
 if (!nofun && !is.function(FUN))
 stop("'FUN' must be a function or NULL")
 len.r <- length(r <- if (nofun) x[a] else FUN(x[a], ...))
 count <- as.integer(round(choose(n, m)))
 if (simplify) {
 dim.use <- if (nofun)
 c(m, count)
 else {
 d <- dim(r)
 if (length(d) > 1L)
 c(d, count)
 else if (len.r > 1L)
 c(len.r, count)
 else c(d, count)
 }
 }
 if (simplify) {
 out <- matrix(r, nrow = len.r, ncol = count)
 }
 else {
 out <- vector("list", count)
 out[[1L]] <- r
 }
 if (m > 0) {
 i <- 2L
 nmmp1 <- n - m + 1L
 while (a[1L] != nmmp1) {
 if (e < n - h) {
 h <- 1L
 e <- a[m]
 j <- 1L
 }
 else {
 e <- a[m - h]
 h <- h + 1L
 j <- 1L:h
 }
 a[m - h + j] <- e + j
 r <- if (nofun)
 x[a]
 else FUN(x[a], ...)
 if (simplify)
 out[, i] <- r
 else out[[i]] <- r
 i <- i + 1L
 }
 }
 if (simplify)
 array(out, dim.use)
 else out
}

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] escaping % in Rd files

[Duncan Murdoch]

On 13-08-18 7:57 AM, Jim Lemon wrote:

Hi all,
In trying to write an Rd file for a new package, I was stumped at
something that is probably quite simple. I have % characters in the
examples of one Rd file. Both my previous experience and some searching
argeed that one can escape % with \. This worked on this command:

fh_dates<-
 as.Date(paste(florida_hurr20$day,florida_hurr20$month,
 florida_hurr20$year,sep="-"),"\%d-\%b-\%Y")

which is okay in the HTML help page but it didn't work on another line
in the same file:

findval<-function(x,set) return(which(set \%in\% x))

The second line was cut off at the first % character in the HTML help
page. After lots of head scratching I noticed that the first line was
broken after the assignment so I changed the second line to:

findval<-
 function(x,set) return(which(set \%in\% x))

and it worked properly! I don't know whether this qualifies as something
interesting to the core team, but I thought I would let you know.


The syntax of Rd files is pretty complex (because of the way they grew). 
 Context affects how escaping works.  You can see the gruesome details 
in the document mentioned in the ?tools::parse_Rd help page.


Duncan Murdoch

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] How can I make my functions run faster

[Jeff Newmiller]

1. Keeping the number of variables down encourages you to structure your data, 
which allows you to re-use code more efficiently. However, I don't believe that 
the number of variables intrinsically slows down your code significantly.. e.g. 
storing a computed value in a local variable is almost always better than 
repeating the calculation.

2. Apply functions are not primarily intended for performance optimization; 
they are effective for compactly expressing the idea of your algorithms. Using 
vectorization and indexing is much more effective for enhancing performance 
(and in my mind is even better at expressing algorithms than using apply 
functions).

3. Agree that too many functions can slow things down, but that is normally 
less the fault of the functions than of the way some programmers handle data. 
It is more productive to focus on the positive idea of using vectors to compute 
as much as possible rather than the negative idea of eliminating functions.

4. I don't think I agree that this statement about memory usage generally 
applies to all uses of R. However, it can be crucial to learn to avoid mixing 
input/output with data processing if you are interested in performance.

To the OP: I will pass on trying to analyse your large code base... it is 
requested in the Posting Guide that you isolate your issues and ask specific 
questions.
---
Jeff NewmillerThe .   .  Go Live...
DCN:Basics: ##.#.   ##.#.  Live Go...
  Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/BatteriesO.O#.   #.O#.  with
/Software/Embedded Controllers)   .OO#.   .OO#.  rocks...1k
--- 
Sent from my phone. Please excuse my brevity.

Heramb Gadgil  wrote:
>Greetings,
>
>I am a newbie too. I will share what I do normally for speeding up the
>code.
>
>1. Restrict defining too many variables (Global/ Local)
>2. Use apply functions (apply,sapply,lapply,tapply, etc.) whenever
>feasible
>3. Having multiple user defined functions doesn't help. Try to compact
>everything in minimum number of functions
>4. The in-memory of R is just 10% of your total RAM (Correct me if
>wrong).
>Make sure most of it is used for processing and not storing
>
>Hope this will help. Kindly suggest if I have misunderstood anything.
>
>Thanks and Regards,
>
>Heramb Gadgil
>
>
>2013/8/19 Laz 
>
>> Yes Bert, I am a beginner in writing R functions. I just don't know
>what
>> to avoid or what to use in order to make the R functions faster.
>>
>> When I run the individual functions, they run quite well.
>> However, calling all of them using the final function it becomes too
>slow.
>>
>> So I don't know how to make it faster.
>> I used system.time()
>>
>> Regards,
>> Laz
>>
>>
>> On 8/19/2013 10:13 AM, Bert Gunter wrote:
>>
>>> ... and read the "R Language Definition" manual. I noticed
>unnecessary
>>> constructs
>>> (e.g., z <- f(something); return(z)) that suggest you have more
>basics
>>> to learn to write efficient, well-structured R code.
>>>
>>> -- Bert
>>>
>>> On Mon, Aug 19, 2013 at 3:55 AM, Michael Dewey
>
>>> wrote:
>>>
 At 10:28 19/08/2013, Laz wrote:

> Dear R users,
>
> I have written a couple of R functions, some are through the help
>of
> the R
> group members. However, running them takes days instead of minutes
>or a
> few
> hours. I am wondering whether there is a quick way of doing that.
>

 Your example code is rather long for humans to profile. Have you
>thought
 of
 getting R to tell where it is spending most time? The R extensions
>manual
 tells you how to do this.


  Here are all my R functions. The last one calls almost all of the
> previous
> functions. It is the one I am interested in most. It gives me the
> correct
> output but it takes several days to run only 1000 or 2000
>simulations!
> e.g. system.time(test1<-finalF(**designs=5,swaps=20));test1
> will take about 20 minutes to run but
> system.time(test1<-finalF(**designs=5,swaps=50));test1 takes about
>10
> hours
> and system.time(test1<-finalF(**designs=25,swaps=2000));test1
>takes
> about 3
> days to run
>
> Here are my functions
>
>
> ##**##**
> #
>
> ls() # list all existing objects
> rm(list = ls()) # remove them all
> rm(list = ls()[!grepl("global.var.A", ls())])
> # refresh memory
> gc()
> ls()
>
> ### Define a function that requires useful input from the user
> #b=4;g=seq(1,20,1);rb=5;cb=4;**s2e=1; r=10;c=8
>
> ##**###
> ##**##
> # function to calculate heritability
> herit<-function(

Re: [R] How can I make my functions run faster

[Heramb Gadgil]

Greetings,

Thanks Jeff. I appreciate your 'to the point' explanation. Will read into
it more.

Best,
Heramb Gadgil


2013/8/19 Jeff Newmiller 

> 1. Keeping the number of variables down encourages you to structure your
> data, which allows you to re-use code more efficiently. However, I don't
> believe that the number of variables intrinsically slows down your code
> significantly.. e.g. storing a computed value in a local variable is almost
> always better than repeating the calculation.
>
> 2. Apply functions are not primarily intended for performance
> optimization; they are effective for compactly expressing the idea of your
> algorithms. Using vectorization and indexing is much more effective for
> enhancing performance (and in my mind is even better at expressing
> algorithms than using apply functions).
>
> 3. Agree that too many functions can slow things down, but that is
> normally less the fault of the functions than of the way some programmers
> handle data. It is more productive to focus on the positive idea of using
> vectors to compute as much as possible rather than the negative idea of
> eliminating functions.
>
> 4. I don't think I agree that this statement about memory usage generally
> applies to all uses of R. However, it can be crucial to learn to avoid
> mixing input/output with data processing if you are interested in
> performance.
>
> To the OP: I will pass on trying to analyse your large code base... it is
> requested in the Posting Guide that you isolate your issues and ask
> specific questions.
> ---
> Jeff NewmillerThe .   .  Go Live...
> DCN:Basics: ##.#.   ##.#.  Live
> Go...
>   Live:   OO#.. Dead: OO#..  Playing
> Research Engineer (Solar/BatteriesO.O#.   #.O#.  with
> /Software/Embedded Controllers)   .OO#.   .OO#.  rocks...1k
> ---
> Sent from my phone. Please excuse my brevity.
>
> Heramb Gadgil  wrote:
> >Greetings,
> >
> >I am a newbie too. I will share what I do normally for speeding up the
> >code.
> >
> >1. Restrict defining too many variables (Global/ Local)
> >2. Use apply functions (apply,sapply,lapply,tapply, etc.) whenever
> >feasible
> >3. Having multiple user defined functions doesn't help. Try to compact
> >everything in minimum number of functions
> >4. The in-memory of R is just 10% of your total RAM (Correct me if
> >wrong).
> >Make sure most of it is used for processing and not storing
> >
> >Hope this will help. Kindly suggest if I have misunderstood anything.
> >
> >Thanks and Regards,
> >
> >Heramb Gadgil
> >
> >
> >2013/8/19 Laz 
> >
> >> Yes Bert, I am a beginner in writing R functions. I just don't know
> >what
> >> to avoid or what to use in order to make the R functions faster.
> >>
> >> When I run the individual functions, they run quite well.
> >> However, calling all of them using the final function it becomes too
> >slow.
> >>
> >> So I don't know how to make it faster.
> >> I used system.time()
> >>
> >> Regards,
> >> Laz
> >>
> >>
> >> On 8/19/2013 10:13 AM, Bert Gunter wrote:
> >>
> >>> ... and read the "R Language Definition" manual. I noticed
> >unnecessary
> >>> constructs
> >>> (e.g., z <- f(something); return(z)) that suggest you have more
> >basics
> >>> to learn to write efficient, well-structured R code.
> >>>
> >>> -- Bert
> >>>
> >>> On Mon, Aug 19, 2013 at 3:55 AM, Michael Dewey
> >
> >>> wrote:
> >>>
>  At 10:28 19/08/2013, Laz wrote:
> 
> > Dear R users,
> >
> > I have written a couple of R functions, some are through the help
> >of
> > the R
> > group members. However, running them takes days instead of minutes
> >or a
> > few
> > hours. I am wondering whether there is a quick way of doing that.
> >
> 
>  Your example code is rather long for humans to profile. Have you
> >thought
>  of
>  getting R to tell where it is spending most time? The R extensions
> >manual
>  tells you how to do this.
> 
> 
>   Here are all my R functions. The last one calls almost all of the
> > previous
> > functions. It is the one I am interested in most. It gives me the
> > correct
> > output but it takes several days to run only 1000 or 2000
> >simulations!
> > e.g. system.time(test1<-finalF(**designs=5,swaps=20));test1
> > will take about 20 minutes to run but
> > system.time(test1<-finalF(**designs=5,swaps=50));test1 takes about
> >10
> > hours
> > and system.time(test1<-finalF(**designs=25,swaps=2000));test1
> >takes
> > about 3
> > days to run
> >
> > Here are my functions
> >
> >
> > ##**##**
> > #
> >
> > ls() # list all existing objects
> > rm(list = ls()) # remove th

Re: [R] How can I make my functions run faster

[William Dunlap]

> I am a newbie too. I will share what I do normally for speeding up the code.
> 
> 1. Restrict defining too many variables (Global/ Local)
> 2. Use apply functions (apply,sapply,lapply,tapply, etc.) whenever feasible
> 3. Having multiple user defined functions doesn't help. Try to compact
> everything in minimum number of functions
> 4. The in-memory of R is just 10% of your total RAM (Correct me if wrong).
> Make sure most of it is used for processing and not storing

(2) The apply functions do not run a whole lot faster than the equivalent loop
and sometimes they run slower.  Their real advantage is that they make the
code more understandable - you don't have to wade through the boilerplate
of setting up a vector or matrix to return or worry about whether all the 
variables
defined in the loop are temporaries or need to be retained for the rest of the
function.  You can get big speedups by avoiding loops and apply functions and
trying to vectorize the code instead (although sometimes vectorization leads
to an unacceptable increase in memory usage).

(3) I think that splitting your problem up into several functions is helpful.  
You
can test each function on its own to check for correctness and speed.  You can
can reuse and share your tested functions.  There is some overhead involved
in calling a function but you have to make a lot of calls to notice.

 Make your functions "functional" - have almost all of the data come in via
the argument list and have all of the output go out via its return value.  
Without
this functions are hard to understand and to test.  The OP's functions had a lot
of <<-'s in them - get rid of them.  They also passed key variables like 'N' 
through
the global environment (or some other environment, depending on how they
were called).  Put that in the argument list so you can easily see how function 
using it
(rspat) behaves as a function of N (it will be quadratic, bad if N will be 
getting big).

   Know the type, shape, and size of you will want to work on.  Some algorithms 
are
fastest on matrices, some on datasets with many more  rows then columns and
some on other types or shapes of data.  Run your function on a variety of 
smallish
datasets to see how it behaves as the number of rows, columns, etc. grows.  It
is a waste of time to wait days for a result when you could know that its time 
is growing
as 0.0001 * nrow(X)^3.

   Then there are all the standard recommendations on speeding things up - don't
do an expensive computation if you don't use its result, don't recompute things
every time you pass through a loop, etc.

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com


> -Original Message-
> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On 
> Behalf
> Of Heramb Gadgil
> Sent: Monday, August 19, 2013 8:13 AM
> To: Laz
> Cc: r-help@r-project.org
> Subject: Re: [R] How can I make my functions run faster
> 
> Greetings,
> 
> I am a newbie too. I will share what I do normally for speeding up the code.
> 
> 1. Restrict defining too many variables (Global/ Local)
> 2. Use apply functions (apply,sapply,lapply,tapply, etc.) whenever feasible
> 3. Having multiple user defined functions doesn't help. Try to compact
> everything in minimum number of functions
> 4. The in-memory of R is just 10% of your total RAM (Correct me if wrong).
> Make sure most of it is used for processing and not storing
> 
> Hope this will help. Kindly suggest if I have misunderstood anything.
> 
> Thanks and Regards,
> 
> Heramb Gadgil
> 
> 
> 2013/8/19 Laz 
> 
> > Yes Bert, I am a beginner in writing R functions. I just don't know what
> > to avoid or what to use in order to make the R functions faster.
> >
> > When I run the individual functions, they run quite well.
> > However, calling all of them using the final function it becomes too slow.
> >
> > So I don't know how to make it faster.
> > I used system.time()
> >
> > Regards,
> > Laz
> >
> >
> > On 8/19/2013 10:13 AM, Bert Gunter wrote:
> >
> >> ... and read the "R Language Definition" manual. I noticed unnecessary
> >> constructs
> >> (e.g., z <- f(something); return(z)) that suggest you have more basics
> >> to learn to write efficient, well-structured R code.
> >>
> >> -- Bert
> >>
> >> On Mon, Aug 19, 2013 at 3:55 AM, Michael Dewey 
> >> wrote:
> >>
> >>> At 10:28 19/08/2013, Laz wrote:
> >>>
>  Dear R users,
> 
>  I have written a couple of R functions, some are through the help of
>  the R
>  group members. However, running them takes days instead of minutes or a
>  few
>  hours. I am wondering whether there is a quick way of doing that.
> 
> >>>
> >>> Your example code is rather long for humans to profile. Have you thought
> >>> of
> >>> getting R to tell where it is spending most time? The R extensions manual
> >>> tells you how to do this.
> >>>
> >>>
> >>>  Here are all my R functions. The last one calls almost all of the
>  previous
>  functions. It is t

Re: [R] Create new records based on event dates in a data frame

[gavinr]

Have used the suggested solution (the second method, as it seems a bit
leaner) on my real data, and it works perfectly. Thanks.



--
View this message in context: 
http://r.789695.n4.nabble.com/Create-new-records-based-on-event-dates-in-a-data-frame-tp4673839p4674082.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] model syntax processed --- probably common

[ivo welch]

dear R experts---I was programming a fama-macbeth panel regression (a
fama-macbeth regression is essentially T cross-sectional regressions, with
statistics then obtained from the time-series of coefficients), partly
because I wanted faster speed than plm, partly because I wanted some
additional features.

my function starts as

fama.macbeth <- function( formula, din ) {
   names <- terms( formula )
  ## omitted : I want an immediate check that the formula refers to
existing variables in the data frame with English error messages
   monthly.regressions <- by( din, as.factor(din$month), function(dd)
coef(lm(model.frame( formula, data=dd )))
   as.m <- do.call("rbind", monthly.regressions)
   colMeans(as.m)  ## or something like this.
}

say my data frame mydata has columns named month, r, laggedx and ... .  I
can call this function

   fama.macbeth( r ~ laggedx, din=mydata )

but it fails if I want to compute my x variables.  for example,

   myx <- d[,"laggedx"]
   fama.macbeth( r ~ myx)

I also wish that the computed myx still remembered that it was really
laggedx.  it's almost as if I should not create a vector myx but a data
frame myx to avoid losing the column name.  I wonder why such vectors don't
keep a name attribute of some sort.

there is probably an "R way" of doing this.  is there?

/iaw


Ivo Welch (ivo.we...@gmail.com)

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] problems with constrained optimization

[IliaN]

I tried ti run a simple example with function solnp() from Rsolnp: find
max(x+y) on a unit circle x*x+y*y=1. The answer should be x=y=1/sqrt(2)
with max=sqrt(2).

Here are my code and results



> # test Rsolnp

> library(Rsolnp)

> fn1 <- function(x)

+ {

+   x[1] + x[2]

+ }

> cond <- function(x)

+ {

+  z<- x[1]*x[1] + x[2]*x[2]

+ }

> x0 <- c(0,1)

> cval <- c(1)

> optim <-solnp(x0,fun=fn1,eqfun=cond,eqB=cval)



Iter: 1 fn: 1.   Pars:  -3.053e-11  1.000e+00

solnp--> Completed in 1 iterations

> optim

$pars

[1] -3.053113e-11  1.00e+00



$convergence

[1] 0



$values

[1] 1 1



$lagrange

 [,1]

[1,]  0.5



$hessian

  [,1] [,2]

[1,] 2e+100

[2,] 0e+001



$ineqx0

NULL



$nfuneval

[1] 13



$outer.iter

[1] 1



$elapsed

Time difference of 0.00500083 secs



> 

#

SOMETHING IS WRONG BUT I DO NOT SEE ANY MY MISTAKE. 


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Why the bdiag code doesn't work?

[arun]

Hi,
It is better to ?dput() your example dataset.
temp<- 
list(structure(c(16.91569, 190.89517, 169.70239, 162.14503, 190.8952, 
2573.2651, 1966.6202, 1829.3142, 169.7024, 1966.6202, 2335.3354, 
1676.7749, 162.145, 1829.314, 1676.775, 2106.543), .Dim = c(4L, 
4L), .Dimnames = list(NULL, NULL)), structure(c(29.27022, 329.11422, 
285.28642, 272.5794, 329.1142, 4454.2464, 3189.73, 3072.6324, 
285.2864, 3189.73, 3818.7995, 2776.0347, 272.5794, 3072.6324, 
2776.0347, 3379.8037), .Dim = c(4L, 4L), .Dimnames = list(NULL, 
    NULL)), structure(c(455.0825, 5496.1986, 4659.3058, 5236.8703, 
5496.199, 85142.057, 62349.821, 63831.882, 4659.306, 62349.821, 
59095.705, 57275.576, 5236.87, 63831.88, 57275.58, 70368.44), .Dim = c(4L, 
4L), .Dimnames = list(NULL, NULL)))


library(Matrix)

#I am not using R studio.  So, this may or may not help you.


A=matrix(0,18,18) 
 A[4:15,4:15] <- bdiag(temp) 
#Error in A[4:15, 4:15] <- bdiag(temp) : 
 # number of items to replace is not a multiple of replacement length


A1<-as(A,"sparseMatrix")
A1[4:15,4:15]<- bdiag(temp)

A1
18 x 18 sparse Matrix of class "dgCMatrix"
  
 [1,] . . .   .  . . .   .  . 
 [2,] . . .   .  . . .   .  . 
 [3,] . . .   .  . . .   .  . 
 [4,] . . .  16.91569  190.8952  169.7024  162.145   .  . 
 [5,] . . . 190.89517 2573.2651 1966.6202 1829.314   .  . 
 [6,] . . . 169.70239 1966.6202 2335.3354 1676.775   .  . 
 [7,] . . . 162.14503 1829.3142 1676.7749 2106.543   .  . 
 [8,] . . .   .  . . .  29.27022  329.1142
 [9,] . . .   .  . . . 329.11422 4454.2464
[10,] . . .   .  . . . 285.28642 3189.7300
[11,] . . .   .  . . . 272.57940 3072.6324
[12,] . . .   .  . . .   .  . 
[13,] . . .   .  . . .   .  . 
[14,] . . .   .  . . .   .  . 
[15,] . . .   .  . . .   .  . 
[16,] . . .   .  . . .   .  . 
[17,] . . .   .  . . .   .  . 
[18,] . . .   .  . . .   .  . 
  
 [1,]    . . .  . . .    . . .
 [2,]    . . .  . . .    . . .
 [3,]    . . .  . . .    . . .
 [4,]    . . .  . . .    . . .
 [5,]    . . .  . . .    . . .
 [6,]    . . .  . . .    . . .
 [7,]    . . .  . . .    . . .
 [8,]  285.2864  272.5794    .  . . .    . . .
 [9,] 3189.7300 3072.6324    .  . . .    . . .
[10,] 3818.7995 2776.0347    .  . . .    . . .
[11,] 2776.0347 3379.8037    .  . . .    . . .
[12,]    . .   455.0825  5496.199  4659.306  5236.87 . . .
[13,]    . .  5496.1986 85142.057 62349.821 63831.88 . . .
[14,]    . .  4659.3058 62349.821 59095.705 57275.58 . . .
[15,]    . .  5236.8703 63831.882 57275.576 70368.44 . . .
[16,]    . . .  . . .    . . .
[17,]    . . .  . . .    . . .
[18,]    . . .  . . .    . . .

A.K.




Why the following R code doesn't work in R studio??? 

temp <- vector("list",3) 
temp 
[[1]] 
          [,1]      [,2]      [,3]     [,4] 
[1,]  16.91569  190.8952  169.7024  162.145 
[2,] 190.89517 2573.2651 1966.6202 1829.314 
[3,] 169.70239 1966.6202 2335.3354 1676.775 
[4,] 162.14503 1829.3142 1676.7749 2106.543 

[[2]] 
          [,1]      [,2]      [,3]      [,4] 
[1,]  29.27022  329.1142  285.2864  272.5794 
[2,] 329.11422 4454.2464 3189.7300 3072.6324 
[3,] 285.28642 3189.7300 3818.7995 2776.0347 
[4,] 272.57940 3072.6324 2776.0347 3379.8037 

[[3]] 
          [,1]      [,2]      [,3]     [,4] 
[1,]  455.0825  5496.199  4659.306  5236.87 
[2,] 5496.1986 85142.057 62349.821 63831.88 
[3,] 4659.3058 62349.821 59095.705 57275.58 
[4,] 5236.8703 63831.882 57275.576 70368.44 

A=matrix(0,18,18) 
A[4:15,4:15] <- bdiag(temp) 

___ 

The returned error is: 

Error：as(x, "CsparseMatrix") : 
  no method or default for coercing “list” to “CsparseMatrix” 

Why the bdiag code doesn't work?

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE d

Re: [R] How can I make my functions run faster

[Berend Hasselman]

On 19-08-2013, at 18:24, William Dunlap  wrote:

>> I am a newbie too. I will share what I do normally for speeding up the code.
>> 
>> 1. Restrict defining too many variables (Global/ Local)
>> 2. Use apply functions (apply,sapply,lapply,tapply, etc.) whenever feasible
>> 3. Having multiple user defined functions doesn't help. Try to compact
>> everything in minimum number of functions
>> 4. The in-memory of R is just 10% of your total RAM (Correct me if wrong).
>> Make sure most of it is used for processing and not storing
> 
> (2) The apply functions do not run a whole lot faster than the equivalent loop
> and sometimes they run slower.  Their real advantage is that they make the
> code more understandable - you don't have to wade through the boilerplate
> of setting up a vector or matrix to return or worry about whether all the 
> variables
> defined in the loop are temporaries or need to be retained for the rest of the
> function.  You can get big speedups by avoiding loops and apply functions and
> trying to vectorize the code instead (although sometimes vectorization leads
> to an unacceptable increase in memory usage).
> 
> (3) I think that splitting your problem up into several functions is helpful. 
>  You
> can test each function on its own to check for correctness and speed.  You can
> can reuse and share your tested functions.  There is some overhead involved
> in calling a function but you have to make a lot of calls to notice.
> 
> Make your functions "functional" - have almost all of the data come in via
> the argument list and have all of the output go out via its return value.  
> Without
> this functions are hard to understand and to test.  The OP's functions had a 
> lot
> of <<-'s in them - get rid of them.  They also passed key variables like 'N' 
> through
> the global environment (or some other environment, depending on how they
> were called).  Put that in the argument list so you can easily see how 
> function using it
> (rspat) behaves as a function of N (it will be quadratic, bad if N will be 
> getting big).
> 
>   Know the type, shape, and size of you will want to work on.  Some 
> algorithms are
> fastest on matrices, some on datasets with many more  rows then columns and
> some on other types or shapes of data.  Run your function on a variety of 
> smallish
> datasets to see how it behaves as the number of rows, columns, etc. grows.  It
> is a waste of time to wait days for a result when you could know that its 
> time is growing
> as 0.0001 * nrow(X)^3.
> 
>   Then there are all the standard recommendations on speeding things up - 
> don't
> do an expensive computation if you don't use its result, don't recompute 
> things
> every time you pass through a loop, etc.


And when you have done all of the previous, consider byte compiling your 
functions with the compiler package, which is available in standard R. Often, 
but not always, this can lead to a speedup.

Berend

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] problems with constrained optimization

[Erich Neuwirth]

Bad starting value.
Try 
c(3/5,4/5)

On Aug 19, 2013, at 1:33 PM,  wrote:

> I tried ti run a simple example with function solnp() from Rsolnp: find
> max(x+y) on a unit circle x*x+y*y=1. The answer should be x=y=1/sqrt(2)
> with max=sqrt(2).
> 
> Here are my code and results
> 
> 
> 
>> # test Rsolnp
> 
>> library(Rsolnp)
> 
>> fn1 <- function(x)
> 
> + {
> 
> +   x[1] + x[2]
> 
> + }
> 
>> cond <- function(x)
> 
> + {
> 
> +  z<- x[1]*x[1] + x[2]*x[2]
> 
> + }
> 
>> x0 <- c(0,1)
> 
>> cval <- c(1)
> 
>> optim <-solnp(x0,fun=fn1,eqfun=cond,eqB=cval)
> 
> 
> 
> Iter: 1 fn: 1.   Pars:  -3.053e-11  1.000e+00
> 
> solnp--> Completed in 1 iterations
> 
>> optim
> 
> $pars
> 
> [1] -3.053113e-11  1.00e+00
> 
> 
> 
> $convergence
> 
> [1] 0
> 
> 
> 
> $values
> 
> [1] 1 1
> 
> 
> 
> $lagrange
> 
> [,1]
> 
> [1,]  0.5
> 
> 
> 
> $hessian
> 
>  [,1] [,2]
> 
> [1,] 2e+100
> 
> [2,] 0e+001
> 
> 
> 
> $ineqx0
> 
> NULL
> 
> 
> 
> $nfuneval
> 
> [1] 13
> 
> 
> 
> $outer.iter
> 
> [1] 1
> 
> 
> 
> $elapsed
> 
> Time difference of 0.00500083 secs
> 
> 
> 
>> 
> 
> #
> 
> SOMETHING IS WRONG BUT I DO NOT SEE ANY MY MISTAKE. 
> 
> 
>   [[alternative HTML version deleted]]
> 
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] lasso cross-validation

[Samuel Okoye]

Dear all,

I am using cv.glmnet in r and I have the following question: The default is 
10-fold cross-validation, but it is not clear to me how many times are 
repeated? Is it 50 repeats?

I am sore if me question will be very easy for some people!

Many thanks in advance,

Samuel

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] model syntax processed --- probably common

[Bert Gunter]

Ivo:

I may not get your question, but you seem to be confusing the name of
an object, which is essentially a pointer into memory and a language
construct -- (correction requested if I have misstated! --  and the
"names" attribute of (some) objects. You can, of course, attach a
"lab" or (whatever) attribute to an object that gives it a label and
that will be carried around with it. But without special code (if it's
at all possible, even) the label will know nothing about the name
assigned to the object -- why should it?! i.e.

> y <- structure(1:3,lab = "y")
> y
[1] 1 2 3
attr(,"lab")
[1] "y"
> z <- y
> z
[1] 1 2 3
attr(,"lab")
[1] "y

Feel free to ignore without response if my comment is irrelevant.

Cheers,
Bert


On Mon, Aug 19, 2013 at 9:45 AM, ivo welch  wrote:
> dear R experts---I was programming a fama-macbeth panel regression (a
> fama-macbeth regression is essentially T cross-sectional regressions, with
> statistics then obtained from the time-series of coefficients), partly
> because I wanted faster speed than plm, partly because I wanted some
> additional features.
>
> my function starts as
>
> fama.macbeth <- function( formula, din ) {
>names <- terms( formula )
>   ## omitted : I want an immediate check that the formula refers to
> existing variables in the data frame with English error messages
>monthly.regressions <- by( din, as.factor(din$month), function(dd)
> coef(lm(model.frame( formula, data=dd )))
>as.m <- do.call("rbind", monthly.regressions)
>colMeans(as.m)  ## or something like this.
> }
>
> say my data frame mydata has columns named month, r, laggedx and ... .  I
> can call this function
>
>fama.macbeth( r ~ laggedx, din=mydata )
>
> but it fails if I want to compute my x variables.  for example,
>
>myx <- d[,"laggedx"]
>fama.macbeth( r ~ myx)
>
> I also wish that the computed myx still remembered that it was really
> laggedx.  it's almost as if I should not create a vector myx but a data
> frame myx to avoid losing the column name.  I wonder why such vectors don't
> keep a name attribute of some sort.
>
> there is probably an "R way" of doing this.  is there?
>
> /iaw
>
> 
> Ivo Welch (ivo.we...@gmail.com)
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.



-- 

Bert Gunter
Genentech Nonclinical Biostatistics

Internal Contact Info:
Phone: 467-7374
Website:
http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biostatistics/pdb-ncb-home.htm

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] model syntax processed --- probably common

[David Winsemius]

On Aug 19, 2013, at 9:45 AM, ivo welch wrote:

> dear R experts---I was programming a fama-macbeth panel regression (a
> fama-macbeth regression is essentially T cross-sectional regressions, with
> statistics then obtained from the time-series of coefficients), partly
> because I wanted faster speed than plm, partly because I wanted some
> additional features.
> 
> my function starts as
> 
> fama.macbeth <- function( formula, din ) {
>   names <- terms( formula )
>  ## omitted : I want an immediate check that the formula refers to
> existing variables in the data frame with English error messages
>

Look the structure of a terms result from a formula argument with str():

 fama.macbeth <- function( formula, din ) {
   fnames <- terms( formula ) ; str(fnames)
 }

> fama.macbeth( x ~ y, data.frame(x=rnorm(10), y=rnorm(10) ) )
Classes 'terms', 'formula' length 3 x ~ y
  ..- attr(*, "variables")= language list(x, y)
  ..- attr(*, "factors")= int [1:2, 1] 0 1
  .. ..- attr(*, "dimnames")=List of 2
  .. .. ..$ : chr [1:2] "x" "y"
  .. .. ..$ : chr "y"
  ..- attr(*, "term.labels")= chr "y"
  ..- attr(*, "order")= int 1
  ..- attr(*, "intercept")= int 1
  ..- attr(*, "response")= int 1
  ..- attr(*, ".Environment")= 

Then extract the dimnames from the "factors" attribute to compare to the names 
in hte data-object:

> fama.macbeth <- function( formula, din ) {
  fnames <- terms( formula ) ;  dnames <- names( din)
  dimnames(attr(fnames, "factors"))[[1]] %in%  dnames
}
#[1] TRUE TRUE


I couldn't tell if this was the main thrust of you question. It seems to 
meander a bit.

-- 
David.

> monthly.regressions <- by( din, as.factor(din$month), function(dd)
> coef(lm(model.frame( formula, data=dd )))
>   as.m <- do.call("rbind", monthly.regressions)
>   colMeans(as.m)  ## or something like this.
> }
> say my data frame mydata has columns named month, r, laggedx and ... .  I
> can call this function
> 
>   fama.macbeth( r ~ laggedx, din=mydata )
> 
> but it fails

What fails?


> if I want to compute my x variables.  for example,
> 
>   myx <- d[,"laggedx"]
>   fama.macbeth( r ~ myx)
> 
> I also wish that the computed myx still remembered that it was really
> laggedx.  it's almost as if I should not create a vector myx but a data
> frame myx to avoid losing the column name.

I wouldn't say "almost"... rather that is exactly what you should do. R 
regression methods almost always work better when formulas are interpreted in 
the environment of the data argument.

>  I wonder why such vectors don't
> keep a name attribute of some sort.
> 
> there is probably an "R way" of doing this.  is there?
> 
> /iaw
> 
> 
> Ivo Welch (ivo.we...@gmail.com)
> 
>   [[alternative HTML version deleted]]

Still posting HTML?

> 
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

And do explain what the goal is.

-- 

David Winsemius
Alameda, CA, USA

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Windows "The system cannot find the path specified" error when submitting R in batch mode

[James Hackett]

This is an R and operating system related question. I am not able to submit an 
R program to be executed in batch mode by Windows. I have confirmed the 
location of the R.exe file. I am following the directions I found in Quick-R, 
but neither of the following work from a DOS prompt ...

D:\chip\Projects\Crescendo Bioscience\Studies\024-CL-01 - BRASS 
FM\biostat\programs\R\024-CL-01 - BRASS FM - Manuscript>"C:\Program 
Files\R\R-2.15.2\bin\R.exe" CMD BATCH "boxplotByMBDAAndCRP.R"
The system cannot find the path specified.

D:\chip\Projects\Crescendo Bioscience\Studies\024-CL-01 - BRASS 
FM\biostat\programs\R\024-CL-01 - BRASS FM - Manuscript>"C:\Program 
Files\R\R-2.15.2\bin\R.exe" CMD BATCH "D:\chip\Projects\Crescendo 
Bioscience\Studies\024-CL-01 - BRASS FM\biostat\programs\R\024-CL-01 - BRASS FM 
- Manuscript\boxplotByMBDAAndCRP.R"
The system cannot find the path specified.



Thank you for any help you can provide,
Chip

James "Chip" Hackett, Ph.D. | Statistical Consultant | Hackett & Associates, 
Inc. | c...@hackettassociates.net | 408.416.3747 (Office) | 805.509.0741 
(Mobile)

The material in this transmission may contain confidential information intended 
only for the  addressee. If you are not the addressee, any disclosure or use of 
this information by you is strictly prohibited. If you have received this 
transmission in error, please delete it, destroy all copies, and notify the 
sender immediately by reply or by calling 1-408-416-3747.



[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] model syntax processed --- probably common

[David Winsemius]

On Aug 19, 2013, at 12:48 PM, David Winsemius wrote:

> 
> On Aug 19, 2013, at 9:45 AM, ivo welch wrote:
> 
>> dear R experts---I was programming a fama-macbeth panel regression (a
>> fama-macbeth regression is essentially T cross-sectional regressions, with
>> statistics then obtained from the time-series of coefficients), partly
>> because I wanted faster speed than plm, partly because I wanted some
>> additional features.
>> 
>> my function starts as
>> 
>> fama.macbeth <- function( formula, din ) {
>>  names <- terms( formula )
>> ## omitted : I want an immediate check that the formula refers to
>> existing variables in the data frame with English error messages
>> 
> 
> Look the structure of a terms result from a formula argument with str():
> 
> fama.macbeth <- function( formula, din ) {
>   fnames <- terms( formula ) ; str(fnames)
> }
> 
>> fama.macbeth( x ~ y, data.frame(x=rnorm(10), y=rnorm(10) ) )
> Classes 'terms', 'formula' length 3 x ~ y
snipped output

> 
> Then extract the dimnames from the "factors" attribute to compare to the 
> names in hte data-object:
> 
>> fama.macbeth <- function( formula, din ) {
>  fnames <- terms( formula ) ;  dnames <- names( din)
>  dimnames(attr(fnames, "factors"))[[1]] %in%  dnames
> }
> #[1] TRUE TRUE
> 

This might be more economical"

?all.names
 fama.macbeth <- function( formula, din ) {
   fnames <- all.names( formula ) ; str(fnames)
   dnames <- names( din)
   fnames[-1] %in%  dnames
 }
 fama.macbeth( x ~ y, data.frame(x=rnorm(10), y=rnorm(10) ) )
 chr [1:3] "~" "x" "y"
[1] TRUE TRUE



> 
> I couldn't tell if this was the main thrust of you question. It seems to 
> meander a bit.
> 
> -- 
> David.
> 
>> monthly.regressions <- by( din, as.factor(din$month), function(dd)
>> coef(lm(model.frame( formula, data=dd )))
>>  as.m <- do.call("rbind", monthly.regressions)
>>  colMeans(as.m)  ## or something like this.
>> }
>> say my data frame mydata has columns named month, r, laggedx and ... .  I
>> can call this function
>> 
>>  fama.macbeth( r ~ laggedx, din=mydata )
>> 
>> but it fails
> 
> What fails?
> 
> 
>> if I want to compute my x variables.  for example,
>> 
>>  myx <- d[,"laggedx"]
>>  fama.macbeth( r ~ myx)
>> 
>> I also wish that the computed myx still remembered that it was really
>> laggedx.  it's almost as if I should not create a vector myx but a data
>> frame myx to avoid losing the column name.
> 
> I wouldn't say "almost"... rather that is exactly what you should do. R 
> regression methods almost always work better when formulas are interpreted in 
> the environment of the data argument.
> 
>> I wonder why such vectors don't
>> keep a name attribute of some sort.
>> 
>> there is probably an "R way" of doing this.  is there?
>> 
>> /iaw
>> 
>> 
>> Ivo Welch (ivo.we...@gmail.com)
>> 

David Winsemius
Alameda, CA, USA

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] model syntax processed --- probably common

[ivo welch]

thank you.  but uggh...sorry for my html post.  and sorry again for
having been obscure in my attempt to be brief.  here is a working
program.

fama.macbeth <- function( formula, din ) {
  fnames <- terms( formula )
  dnames <- names( din )
  stopifnot( all(dimnames(attr(fnames, "factors"))[[1]] %in%  dnames) )

  monthly.regressions <- by( din, as.factor(din$month), function(dd)
coef(lm(model.frame( formula, data=dd 
  as.m <- do.call("rbind", monthly.regressions)
  colMeans(as.m)
}

## a test data set
d <- data.frame( month=rep(1:5,10), y= rnorm(50), x= rnorm(50), z=rnorm(50) )

## this works beautifully, exactly how I want it.  the names are
there, the formula works.
print(fama.macbeth( y ~ x , din=d ))

## now I want something like the following statement to work, too
for (nm in c("x")) print(fama.macbeth( y ~ nm, din=d ))
   or
for (nm in c("x")) print(fama.macbeth( y ~ d[[nm]], din=d ))
  or whatever.

the output in both cases should be the same, preferably even knowing
that the name of the variable is really "x" and not nm.  is there a
standard common way to do this?

regards,

/iaw


Ivo Welch (ivo.we...@gmail.com)
http://www.ivo-welch.info/
J. Fred Weston Professor of Finance
Anderson School at UCLA, C519
Director, UCLA Anderson Fink Center for Finance and Investments
Free Finance Textbook, http://book.ivo-welch.info/
Editor, Critical Finance Review, http://www.critical-finance-review.org/



On Mon, Aug 19, 2013 at 12:48 PM, David Winsemius
 wrote:
>
> On Aug 19, 2013, at 9:45 AM, ivo welch wrote:
>
>> dear R experts---I was programming a fama-macbeth panel regression (a
>> fama-macbeth regression is essentially T cross-sectional regressions, with
>> statistics then obtained from the time-series of coefficients), partly
>> because I wanted faster speed than plm, partly because I wanted some
>> additional features.
>>
>> my function starts as
>>
>> fama.macbeth <- function( formula, din ) {
>>   names <- terms( formula )
>>  ## omitted : I want an immediate check that the formula refers to
>> existing variables in the data frame with English error messages
>>
>
> Look the structure of a terms result from a formula argument with str():
>
>  fama.macbeth <- function( formula, din ) {
>fnames <- terms( formula ) ; str(fnames)
>  }
>
>> fama.macbeth( x ~ y, data.frame(x=rnorm(10), y=rnorm(10) ) )
> Classes 'terms', 'formula' length 3 x ~ y
>   ..- attr(*, "variables")= language list(x, y)
>   ..- attr(*, "factors")= int [1:2, 1] 0 1
>   .. ..- attr(*, "dimnames")=List of 2
>   .. .. ..$ : chr [1:2] "x" "y"
>   .. .. ..$ : chr "y"
>   ..- attr(*, "term.labels")= chr "y"
>   ..- attr(*, "order")= int 1
>   ..- attr(*, "intercept")= int 1
>   ..- attr(*, "response")= int 1
>   ..- attr(*, ".Environment")=
>
> Then extract the dimnames from the "factors" attribute to compare to the 
> names in hte data-object:
>
>> fama.macbeth <- function( formula, din ) {
>   fnames <- terms( formula ) ;  dnames <- names( din)
>   dimnames(attr(fnames, "factors"))[[1]] %in%  dnames
> }
> #[1] TRUE TRUE
>
>
> I couldn't tell if this was the main thrust of you question. It seems to 
> meander a bit.
>
> --
> David.
>
>> monthly.regressions <- by( din, as.factor(din$month), function(dd)
>> coef(lm(model.frame( formula, data=dd )))
>>   as.m <- do.call("rbind", monthly.regressions)
>>   colMeans(as.m)  ## or something like this.
>> }
>> say my data frame mydata has columns named month, r, laggedx and ... .  I
>> can call this function
>>
>>   fama.macbeth( r ~ laggedx, din=mydata )
>>
>> but it fails
>
> What fails?
>
>
>> if I want to compute my x variables.  for example,
>>
>>   myx <- d[,"laggedx"]
>>   fama.macbeth( r ~ myx)
>>
>> I also wish that the computed myx still remembered that it was really
>> laggedx.  it's almost as if I should not create a vector myx but a data
>> frame myx to avoid losing the column name.
>
> I wouldn't say "almost"... rather that is exactly what you should do. R 
> regression methods almost always work better when formulas are interpreted in 
> the environment of the data argument.
>
>>  I wonder why such vectors don't
>> keep a name attribute of some sort.
>>
>> there is probably an "R way" of doing this.  is there?
>>
>> /iaw
>>
>> 
>> Ivo Welch (ivo.we...@gmail.com)
>>
>>   [[alternative HTML version deleted]]
>
> Still posting HTML?
>
>>
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> And do explain what the goal is.
>
> --
>
> David Winsemius
> Alameda, CA, USA
>

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Appropriateness of R functions for multicore

[Patrick Connolly]

On Sat, 17-Aug-2013 at 05:09PM -0700, Jeff Newmiller wrote:


|> In most threaded multitasking environments it is not safe to
|> perform IO in multiple threads. In general you will have difficulty
|> performing IO in parallel processing so it is best to let the
|> master hand out data to worker tasks and gather results from them
|> for storage. Keep in mind that just because you have eight cores
|> for processing doesn't mean you have eight hard disks, so if your
|> problem is IO bound in single processor operation then it will also
|> be IO bound in threaded operation.

For tasks which don't involve I/O but fail with mclapply, how does one
work out where the problem is?  The handy browser() function which
allows for interactive diagnosis won't work with parallel jobs.

What other approaches can one use?

Thanx




---



|> Jeff NewmillerThe .   .  Go Live...
|> DCN:Basics: ##.#.   ##.#.  Live Go...
|>   Live:   OO#.. Dead: OO#..  Playing
|> Research Engineer (Solar/BatteriesO.O#.   #.O#.  with
|> /Software/Embedded Controllers)   .OO#.   .OO#.  rocks...1k
|> --- 
|> Sent from my phone. Please excuse my brevity.
|> 
|> "Hopkins, Bill"  wrote:
|> >Has there been any systematic evaluation of which core R functions are
|> >safe for use with multicore? Of current interest, I have tried calling
|> >read.table() via mclapply() to more quickly read in hundreds of raw
|> >data files (I have a 24 core system with 72 GB running Ubuntu, a
|> >perfect platform for multicore). There was a 40% failure rate, which
|> >doesn't occur when I invoke read.table() serially from within a single
|> >thread. Another example was using pvec() to invoke
|> >sapply(strsplit(),...) on a huge character vector (to pull out fields
|> >from within a field). It looked like a perfect application for pvec(),
|> >but it fails when serial execution works.
|> >
|> >I thought I'd ask before taking on the task of digging into the
|> >underlying code to see what is might be causing failure in a multicore
|> >(well, multi-threaded) context.
|> >
|> >As an alternative, I could define multiple cluster nodes locally, but
|> >that shifts the tradeoff a bit in whether parallel execution is
|> >advantageous - the overhead is significantly more, and even with 72 GB,
|> >it does impose greater limits on how many cores can be used.
|> >
|> >Bill Hopkins
|> >
|> >__
|> >R-help@r-project.org mailing list
|> >https://stat.ethz.ch/mailman/listinfo/r-help
|> >PLEASE do read the posting guide
|> >http://www.R-project.org/posting-guide.html
|> >and provide commented, minimal, self-contained, reproducible code.
|> 
|> __
|> R-help@r-project.org mailing list
|> https://stat.ethz.ch/mailman/listinfo/r-help
|> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
|> and provide commented, minimal, self-contained, reproducible code.

-- 
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.   
   ___Patrick Connolly   
 {~._.~}   Great minds discuss ideas
 _( Y )_ Average minds discuss events 
(:_~*~_:)  Small minds discuss people  
 (_)-(_)  . Eleanor Roosevelt
  
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] model syntax processed --- probably common

[R. Michael Weylandt]

On Aug 19, 2013, at 16:05, ivo welch  wrote:

> thank you.  but uggh...sorry for my html post.  and sorry again for
> having been obscure in my attempt to be brief.  here is a working
> program.
> 
> fama.macbeth <- function( formula, din ) {

I think most users would expect 'din' to be 'data' here

>  fnames <- terms( formula )
>  dnames <- names( din )
>  stopifnot( all(dimnames(attr(fnames, "factors"))[[1]] %in%  dnames) )
> 
>  monthly.regressions <- by( din, as.factor(din$month), function(dd)
> coef(lm(model.frame( formula, data=dd 
>  as.m <- do.call("rbind", monthly.regressions)
>  colMeans(as.m)
> }
> 
> ## a test data set
> d <- data.frame( month=rep(1:5,10), y= rnorm(50), x= rnorm(50), z=rnorm(50) )
> 
> ## this works beautifully, exactly how I want it.  the names are
> there, the formula works.
> print(fama.macbeth( y ~ x , din=d ))
> 
> ## now I want something like the following statement to work, too
> for (nm in c("x")) print(fama.macbeth( y ~ nm, din=d ))
>   or
> for (nm in c("x")) print(fama.macbeth( y ~ d[[nm]], din=d ))
>  or whatever.
> 
> the output in both cases should be the same, preferably even knowing
> that the name of the variable is really "x" and not nm.  is there a
> standard common way to do this?

I don't think so -- most of the core modelling functions expect all the 
covariates to be passed in through a single data frame. 

Perhaps you're looking for programmatic construction of the formula using paste 
and as.formula:

fama.macbeth(as.formula(paste("y ~", nm)))

eval(parse(text = nm)) also works but you don't get nice names on the resulting 
object so I'd suggest as.formula()

If there's something better, I'd love to know -- I always have to use tricks 
here when doing similar looped regressions. 

Michael

> 
> regards,
> 
> /iaw
> 
> 
> Ivo Welch (ivo.we...@gmail.com)
> http://www.ivo-welch.info/
> J. Fred Weston Professor of Finance
> Anderson School at UCLA, C519
> Director, UCLA Anderson Fink Center for Finance and Investments
> Free Finance Textbook, http://book.ivo-welch.info/
> Editor, Critical Finance Review, http://www.critical-finance-review.org/
> 
> 
> 
> On Mon, Aug 19, 2013 at 12:48 PM, David Winsemius
>  wrote:
>> 
>> On Aug 19, 2013, at 9:45 AM, ivo welch wrote:
>> 
>>> dear R experts---I was programming a fama-macbeth panel regression (a
>>> fama-macbeth regression is essentially T cross-sectional regressions, with
>>> statistics then obtained from the time-series of coefficients), partly
>>> because I wanted faster speed than plm, partly because I wanted some
>>> additional features.
>>> 
>>> my function starts as
>>> 
>>> fama.macbeth <- function( formula, din ) {
>>>  names <- terms( formula )
>>> ## omitted : I want an immediate check that the formula refers to
>>> existing variables in the data frame with English error messages
>>> 
>> 
>> Look the structure of a terms result from a formula argument with str():
>> 
>> fama.macbeth <- function( formula, din ) {
>>   fnames <- terms( formula ) ; str(fnames)
>> }
>> 
>>> fama.macbeth( x ~ y, data.frame(x=rnorm(10), y=rnorm(10) ) )
>> Classes 'terms', 'formula' length 3 x ~ y
>>  ..- attr(*, "variables")= language list(x, y)
>>  ..- attr(*, "factors")= int [1:2, 1] 0 1
>>  .. ..- attr(*, "dimnames")=List of 2
>>  .. .. ..$ : chr [1:2] "x" "y"
>>  .. .. ..$ : chr "y"
>>  ..- attr(*, "term.labels")= chr "y"
>>  ..- attr(*, "order")= int 1
>>  ..- attr(*, "intercept")= int 1
>>  ..- attr(*, "response")= int 1
>>  ..- attr(*, ".Environment")=
>> 
>> Then extract the dimnames from the "factors" attribute to compare to the 
>> names in hte data-object:
>> 
>>> fama.macbeth <- function( formula, din ) {
>>  fnames <- terms( formula ) ;  dnames <- names( din)
>>  dimnames(attr(fnames, "factors"))[[1]] %in%  dnames
>> }
>> #[1] TRUE TRUE
>> 
>> 
>> I couldn't tell if this was the main thrust of you question. It seems to 
>> meander a bit.
>> 
>> --
>> David.
>> 
>>> monthly.regressions <- by( din, as.factor(din$month), function(dd)
>>> coef(lm(model.frame( formula, data=dd )))
>>>  as.m <- do.call("rbind", monthly.regressions)
>>>  colMeans(as.m)  ## or something like this.
>>> }
>>> say my data frame mydata has columns named month, r, laggedx and ... .  I
>>> can call this function
>>> 
>>>  fama.macbeth( r ~ laggedx, din=mydata )
>>> 
>>> but it fails
>> 
>> What fails?
>> 
>> 
>>> if I want to compute my x variables.  for example,
>>> 
>>>  myx <- d[,"laggedx"]
>>>  fama.macbeth( r ~ myx)
>>> 
>>> I also wish that the computed myx still remembered that it was really
>>> laggedx.  it's almost as if I should not create a vector myx but a data
>>> frame myx to avoid losing the column name.
>> 
>> I wouldn't say "almost"... rather that is exactly what you should do. R 
>> regression methods almost always work better when formulas are interpreted 
>> in the environment of the data argument.
>> 
>>> I wonder why such vectors don't
>>> keep a name attribute of some sort.
>>> 
>>> there is probably an "R w

Re: [R] model syntax processed --- probably common

[William Dunlap]

When I want to manipulate expressions, including formulae, I first
think of things like bquote() and substitute().   E.g.,

> for(nm in lapply(c("x","z"), as.name)) {
fmla <- formula( bquote( y ~ .(nm) )) 
print(fama.macbeth(fmla, din=d))
}
(Intercept)   x
-0.02384804  0.18151577
(Intercept)   z
 0.05562026  0.03174173

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com


> -Original Message-
> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On 
> Behalf
> Of ivo welch
> Sent: Monday, August 19, 2013 1:05 PM
> To: David Winsemius; r-help
> Subject: Re: [R] model syntax processed --- probably common
> 
> thank you.  but uggh...sorry for my html post.  and sorry again for
> having been obscure in my attempt to be brief.  here is a working
> program.
> 
> fama.macbeth <- function( formula, din ) {
>   fnames <- terms( formula )
>   dnames <- names( din )
>   stopifnot( all(dimnames(attr(fnames, "factors"))[[1]] %in%  dnames) )
> 
>   monthly.regressions <- by( din, as.factor(din$month), function(dd)
> coef(lm(model.frame( formula, data=dd 
>   as.m <- do.call("rbind", monthly.regressions)
>   colMeans(as.m)
> }
> 
> ## a test data set
> d <- data.frame( month=rep(1:5,10), y= rnorm(50), x= rnorm(50), z=rnorm(50) )
> 
> ## this works beautifully, exactly how I want it.  the names are
> there, the formula works.
> print(fama.macbeth( y ~ x , din=d ))
> 
> ## now I want something like the following statement to work, too
> for (nm in c("x")) print(fama.macbeth( y ~ nm, din=d ))
>or
> for (nm in c("x")) print(fama.macbeth( y ~ d[[nm]], din=d ))
>   or whatever.
> 
> the output in both cases should be the same, preferably even knowing
> that the name of the variable is really "x" and not nm.  is there a
> standard common way to do this?
> 
> regards,
> 
> /iaw
> 
> 
> Ivo Welch (ivo.we...@gmail.com)
> http://www.ivo-welch.info/
> J. Fred Weston Professor of Finance
> Anderson School at UCLA, C519
> Director, UCLA Anderson Fink Center for Finance and Investments
> Free Finance Textbook, http://book.ivo-welch.info/
> Editor, Critical Finance Review, http://www.critical-finance-review.org/
> 
> 
> 
> On Mon, Aug 19, 2013 at 12:48 PM, David Winsemius
>  wrote:
> >
> > On Aug 19, 2013, at 9:45 AM, ivo welch wrote:
> >
> >> dear R experts---I was programming a fama-macbeth panel regression (a
> >> fama-macbeth regression is essentially T cross-sectional regressions, with
> >> statistics then obtained from the time-series of coefficients), partly
> >> because I wanted faster speed than plm, partly because I wanted some
> >> additional features.
> >>
> >> my function starts as
> >>
> >> fama.macbeth <- function( formula, din ) {
> >>   names <- terms( formula )
> >>  ## omitted : I want an immediate check that the formula refers to
> >> existing variables in the data frame with English error messages
> >>
> >
> > Look the structure of a terms result from a formula argument with str():
> >
> >  fama.macbeth <- function( formula, din ) {
> >fnames <- terms( formula ) ; str(fnames)
> >  }
> >
> >> fama.macbeth( x ~ y, data.frame(x=rnorm(10), y=rnorm(10) ) )
> > Classes 'terms', 'formula' length 3 x ~ y
> >   ..- attr(*, "variables")= language list(x, y)
> >   ..- attr(*, "factors")= int [1:2, 1] 0 1
> >   .. ..- attr(*, "dimnames")=List of 2
> >   .. .. ..$ : chr [1:2] "x" "y"
> >   .. .. ..$ : chr "y"
> >   ..- attr(*, "term.labels")= chr "y"
> >   ..- attr(*, "order")= int 1
> >   ..- attr(*, "intercept")= int 1
> >   ..- attr(*, "response")= int 1
> >   ..- attr(*, ".Environment")=
> >
> > Then extract the dimnames from the "factors" attribute to compare to the 
> > names in
> hte data-object:
> >
> >> fama.macbeth <- function( formula, din ) {
> >   fnames <- terms( formula ) ;  dnames <- names( din)
> >   dimnames(attr(fnames, "factors"))[[1]] %in%  dnames
> > }
> > #[1] TRUE TRUE
> >
> >
> > I couldn't tell if this was the main thrust of you question. It seems to 
> > meander a bit.
> >
> > --
> > David.
> >
> >> monthly.regressions <- by( din, as.factor(din$month), function(dd)
> >> coef(lm(model.frame( formula, data=dd )))
> >>   as.m <- do.call("rbind", monthly.regressions)
> >>   colMeans(as.m)  ## or something like this.
> >> }
> >> say my data frame mydata has columns named month, r, laggedx and ... .  I
> >> can call this function
> >>
> >>   fama.macbeth( r ~ laggedx, din=mydata )
> >>
> >> but it fails
> >
> > What fails?
> >
> >
> >> if I want to compute my x variables.  for example,
> >>
> >>   myx <- d[,"laggedx"]
> >>   fama.macbeth( r ~ myx)
> >>
> >> I also wish that the computed myx still remembered that it was really
> >> laggedx.  it's almost as if I should not create a vector myx but a data
> >> frame myx to avoid losing the column name.
> >
> > I wouldn't say "almost"... rather that is exactly what you should do. R 
> > regression
> methods almost always work better when formulas are interpreted in the 
> environment

[R] How to rebuild an R package that has been removed from CRAN?

[Shang Zuofeng]

Dear All,

My project requires the use of a specific R package. However, this R
package has been removed from CRAN. But its older version can be found.
Unfortunately, the older version cannot be used. The thing is, after I
downloaded the older version and unzip it into the library folder of  R,
and I input library("package name"), it says that the package "is not a
valid installed package".

After an intensive search, I found a possible solution: to rebuild this R
package so that it may work properly. I have no idea how to make this
achieved because of my little experience on rebuilding an R package.

I highly appreciate your help.

Sincere thanks.
Zuofeng

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] How to rebuild an R package that has been removed from CRAN?

[Daniel Nordlund]

> -Original Message-
> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
> On Behalf Of Shang Zuofeng
> Sent: Monday, August 19, 2013 1:26 PM
> To: r-help@r-project.org
> Subject: [R] How to rebuild an R package that has been removed from CRAN?
> 
> Dear All,
> 
> My project requires the use of a specific R package. However, this R
> package has been removed from CRAN. But its older version can be found.
> Unfortunately, the older version cannot be used. The thing is, after I
> downloaded the older version and unzip it into the library folder of  R,
> and I input library("package name"), it says that the package "is not a
> valid installed package".
> 
> After an intensive search, I found a possible solution: to rebuild this R
> package so that it may work properly. I have no idea how to make this
> achieved because of my little experience on rebuilding an R package.
> 
> I highly appreciate your help.
> 
> Sincere thanks.
> Zuofeng
> 

Well, you haven't told us enough to let us help you.  Given that you have a zip 
file, I will assume for the moment that you are using some variant of MS 
Windows.  I don't think you want to unzip that file directly.  I think you want 
open R, go to the packages menu and choose install package from  local zip 
file.  Whether that will work depends on your version of R, your OS, the 
requirements of the package, why it was removed from CRAN, and a host of other 
things.  If you want more detailed help, you need to provide the "at a minimum" 
info requested in the posting guide.  It would also help if you told us what 
package you are trying to install.

Dan

Daniel Nordlund
Bothell, WA USA

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] How to rebuild an R package that has been removed from CRAN?

[Shang Zuofeng]

Thanks, Dan!

The package is "assist" which can be downloaded from the following link:

http://cran.r-project.org/src/contrib/Archive/assist/

The one I chose was assist_3.1.2.tar.gz

I have changed this file to .zip and installed it from local directory
through R. However, this method is still not working.

Thanks a lot for your kind help!

Best regards,
Zuofeng












2013/8/19 Daniel Nordlund 

> > -Original Message-
> > From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
> > On Behalf Of Shang Zuofeng
> > Sent: Monday, August 19, 2013 1:26 PM
> > To: r-help@r-project.org
> > Subject: [R] How to rebuild an R package that has been removed from CRAN?
> >
> > Dear All,
> >
> > My project requires the use of a specific R package. However, this R
> > package has been removed from CRAN. But its older version can be found.
> > Unfortunately, the older version cannot be used. The thing is, after I
> > downloaded the older version and unzip it into the library folder of  R,
> > and I input library("package name"), it says that the package "is not a
> > valid installed package".
> >
> > After an intensive search, I found a possible solution: to rebuild this R
> > package so that it may work properly. I have no idea how to make this
> > achieved because of my little experience on rebuilding an R package.
> >
> > I highly appreciate your help.
> >
> > Sincere thanks.
> > Zuofeng
> >
>
> Well, you haven't told us enough to let us help you.  Given that you have
> a zip file, I will assume for the moment that you are using some variant of
> MS Windows.  I don't think you want to unzip that file directly.  I think
> you want open R, go to the packages menu and choose install package from
>  local zip file.  Whether that will work depends on your version of R, your
> OS, the requirements of the package, why it was removed from CRAN, and a
> host of other things.  If you want more detailed help, you need to provide
> the "at a minimum" info requested in the posting guide.  It would also help
> if you told us what package you are trying to install.
>
> Dan
>
> Daniel Nordlund
> Bothell, WA USA
>
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] How to rebuild an R package that has been removed from CRAN?

[Daniel Nordlund]

The file you had, assist_3.1.2.tar.gz, was not a Windows binary zip file.  It 
was a source tarball.  That kind of file needs to be built and installed 
differently.  In order to do that, you need to have all the tools necessary for 
building packages.  This package does not have just pure R code in it, but it 
has code that needs compiling.  You need to determine why the package was 
removed from CRAN.  It may no longer work with current versions of R.  You 
probably need to contact the package maintainer to resolve this problem.  Sorry 
I can't provide more help.  

 

Dan

 

Daniel Nordlund

Bothell, WA USA

 

  _  

From: Shang Zuofeng [mailto:zuofengsh...@gmail.com] 
Sent: Monday, August 19, 2013 2:16 PM
To: Daniel Nordlund
Cc: r-help@r-project.org
Subject: Re: [R] How to rebuild an R package that has been removed from CRAN?

 

Thanks, Dan! 

The package is "assist" which can be downloaded from the following link:

http://cran.r-project.org/src/contrib/Archive/assist/

The one I chose was assist_3.1.2.tar.gz

I have changed this file to .zip and installed it from local directory through 
R. However, this method is still not working. 

Thanks a lot for your kind help!

Best regards,

Zuofeng


 

 

 

 

 

 


 

 

 

 

 

 

2013/8/19 Daniel Nordlund 

> -Original Message-
> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
> On Behalf Of Shang Zuofeng
> Sent: Monday, August 19, 2013 1:26 PM
> To: r-help@r-project.org
> Subject: [R] How to rebuild an R package that has been removed from CRAN?
>
> Dear All,
>
> My project requires the use of a specific R package. However, this R
> package has been removed from CRAN. But its older version can be found.
> Unfortunately, the older version cannot be used. The thing is, after I
> downloaded the older version and unzip it into the library folder of  R,
> and I input library("package name"), it says that the package "is not a
> valid installed package".
>
> After an intensive search, I found a possible solution: to rebuild this R
> package so that it may work properly. I have no idea how to make this
> achieved because of my little experience on rebuilding an R package.
>
> I highly appreciate your help.
>
> Sincere thanks.
> Zuofeng
>

Well, you haven't told us enough to let us help you.  Given that you have a zip 
file, I will assume for the moment that you are using some variant of MS 
Windows.  I don't think you want to unzip that file directly.  I think you want 
open R, go to the packages menu and choose install package from  local zip 
file.  Whether that will work depends on your version of R, your OS, the 
requirements of the package, why it was removed from CRAN, and a host of other 
things.  If you want more detailed help, you need to provide the "at a minimum" 
info requested in the posting guide.  It would also help if you told us what 
package you are trying to install.

Dan

Daniel Nordlund
Bothell, WA USA



 


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] How to rebuild an R package that has been removed from CRAN?

[Uwe Ligges]

On 20.08.2013 01:06, Daniel Nordlund wrote:

The file you had, assist_3.1.2.tar.gz, was not a Windows binary zip file.  It 
was a source tarball.  That kind of file needs to be built and installed 
differently.  In order to do that, you need to have all the tools necessary for 
building packages.  This package does not have just pure R code in it, but it 
has code that needs compiling.  You need to determine why the package was 
removed from CRAN.  It may no longer work with current versions of R.  You 
probably need to contact the package maintainer to resolve this problem.  Sorry 
I can't provide more help.



 and possible reasons for package archivals on CRAN are unresponsive 
maintainers of packages that do not pass the checks without problems any 
more.


Best,
Uwe Ligges





Dan



Daniel Nordlund

Bothell, WA USA



   _

From: Shang Zuofeng [mailto:zuofengsh...@gmail.com]
Sent: Monday, August 19, 2013 2:16 PM
To: Daniel Nordlund
Cc: r-help@r-project.org
Subject: Re: [R] How to rebuild an R package that has been removed from CRAN?



Thanks, Dan!

The package is "assist" which can be downloaded from the following link:

http://cran.r-project.org/src/contrib/Archive/assist/

The one I chose was assist_3.1.2.tar.gz

I have changed this file to .zip and installed it from local directory through 
R. However, this method is still not working.

Thanks a lot for your kind help!

Best regards,

Zuofeng



























2013/8/19 Daniel Nordlund 


-Original Message-
From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
On Behalf Of Shang Zuofeng
Sent: Monday, August 19, 2013 1:26 PM
To: r-help@r-project.org
Subject: [R] How to rebuild an R package that has been removed from CRAN?

Dear All,

My project requires the use of a specific R package. However, this R
package has been removed from CRAN. But its older version can be found.
Unfortunately, the older version cannot be used. The thing is, after I
downloaded the older version and unzip it into the library folder of  R,
and I input library("package name"), it says that the package "is not a
valid installed package".

After an intensive search, I found a possible solution: to rebuild this R
package so that it may work properly. I have no idea how to make this
achieved because of my little experience on rebuilding an R package.

I highly appreciate your help.

Sincere thanks.
Zuofeng



Well, you haven't told us enough to let us help you.  Given that you have a zip file, I 
will assume for the moment that you are using some variant of MS Windows.  I don't think 
you want to unzip that file directly.  I think you want open R, go to the packages menu 
and choose install package from  local zip file.  Whether that will work depends on your 
version of R, your OS, the requirements of the package, why it was removed from CRAN, and 
a host of other things.  If you want more detailed help, you need to provide the "at 
a minimum" info requested in the posting guide.  It would also help if you told us 
what package you are trying to install.

Dan

Daniel Nordlund
Bothell, WA USA






[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Appropriateness of R functions for multicore

[Jeff Newmiller]

I don't know... I suppose it depends how it fails. I recommend that you 
restrict yourself to using only the data that was passed as parameters to your 
parallel function. You may be able to tackle parts of the task and return only 
those partial results to confirm how far through the code you can get.
---
Jeff NewmillerThe .   .  Go Live...
DCN:Basics: ##.#.   ##.#.  Live Go...
  Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/BatteriesO.O#.   #.O#.  with
/Software/Embedded Controllers)   .OO#.   .OO#.  rocks...1k
--- 
Sent from my phone. Please excuse my brevity.

Patrick Connolly  wrote:
>On Sat, 17-Aug-2013 at 05:09PM -0700, Jeff Newmiller wrote:
>
>
>|> In most threaded multitasking environments it is not safe to
>|> perform IO in multiple threads. In general you will have difficulty
>|> performing IO in parallel processing so it is best to let the
>|> master hand out data to worker tasks and gather results from them
>|> for storage. Keep in mind that just because you have eight cores
>|> for processing doesn't mean you have eight hard disks, so if your
>|> problem is IO bound in single processor operation then it will also
>|> be IO bound in threaded operation.
>
>For tasks which don't involve I/O but fail with mclapply, how does one
>work out where the problem is?  The handy browser() function which
>allows for interactive diagnosis won't work with parallel jobs.
>
>What other approaches can one use?
>
>Thanx
>
>
>
>
>---
>
>
>
>|> Jeff NewmillerThe .   .  Go
>Live...
>|> DCN:Basics: ##.#.   ##.#. 
>Live Go...
>|>   Live:   OO#.. Dead: OO#.. 
>Playing
>|> Research Engineer (Solar/BatteriesO.O#.   #.O#. 
>with
>|> /Software/Embedded Controllers)   .OO#.   .OO#. 
>rocks...1k
>|>
>---
>
>|> Sent from my phone. Please excuse my brevity.
>|> 
>|> "Hopkins, Bill"  wrote:
>|> >Has there been any systematic evaluation of which core R functions
>are
>|> >safe for use with multicore? Of current interest, I have tried
>calling
>|> >read.table() via mclapply() to more quickly read in hundreds of raw
>|> >data files (I have a 24 core system with 72 GB running Ubuntu, a
>|> >perfect platform for multicore). There was a 40% failure rate,
>which
>|> >doesn't occur when I invoke read.table() serially from within a
>single
>|> >thread. Another example was using pvec() to invoke
>|> >sapply(strsplit(),...) on a huge character vector (to pull out
>fields
>|> >from within a field). It looked like a perfect application for
>pvec(),
>|> >but it fails when serial execution works.
>|> >
>|> >I thought I'd ask before taking on the task of digging into the
>|> >underlying code to see what is might be causing failure in a
>multicore
>|> >(well, multi-threaded) context.
>|> >
>|> >As an alternative, I could define multiple cluster nodes locally,
>but
>|> >that shifts the tradeoff a bit in whether parallel execution is
>|> >advantageous - the overhead is significantly more, and even with 72
>GB,
>|> >it does impose greater limits on how many cores can be used.
>|> >
>|> >Bill Hopkins
>|> >
>|> >__
>|> >R-help@r-project.org mailing list
>|> >https://stat.ethz.ch/mailman/listinfo/r-help
>|> >PLEASE do read the posting guide
>|> >http://www.R-project.org/posting-guide.html
>|> >and provide commented, minimal, self-contained, reproducible code.
>|> 
>|> __
>|> R-help@r-project.org mailing list
>|> https://stat.ethz.ch/mailman/listinfo/r-help
>|> PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>|> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] How to rebuild an R package that has been removed from CRAN?

[David Winsemius]

On Aug 19, 2013, at 4:12 PM, Uwe Ligges wrote:

> 
> On 20.08.2013 01:06, Daniel Nordlund wrote:
>> The file you had, assist_3.1.2.tar.gz, was not a Windows binary zip file.  
>> It was a source tarball.  That kind of file needs to be built and installed 
>> differently.  In order to do that, you need to have all the tools necessary 
>> for building packages.  This package does not have just pure R code in it, 
>> but it has code that needs compiling.  You need to determine why the package 
>> was removed from CRAN.  It may no longer work with current versions of R.  
>> You probably need to contact the package maintainer to resolve this problem. 
>>  Sorry I can't provide more help.
> 
> 
>  and possible reasons for package archivals on CRAN are unresponsive 
> maintainers of packages that do not pass the checks without problems any more.

I have a copy of 'assist' installed (for some mysterious reason my GUI package 
installer was able to find a binary copy for R 3.0.1 on my regular UC Berkeley 
CRAN repos):

> maintainer("assist")
[1] "Chunlei Ke "

>From the description file:

Package: assist
Version: 3.1.2
Title: A Suite of S-Plus Functions Implementing Smoothing Splines

Depends: R (>= 1.7.0), nlme

URL: http://www.pstat.ucsb.edu/faculty/yuedong/software.html
Packaged: 2013-03-12 15:29:39 UTC; ripley
Repository: CRAN
Date/Publication: 2013-03-12 16:30:20
NeedsCompilation: yes
Built: R 3.0.0; x86_64-apple-darwin10.8.0; 2013-03-16 08:53:21 UTC; unix
Archs: assist.so.dSYM


I also installed from source and here are the warning messages:


gfortran-4.2 -arch x86_64   -fPIC  -g -O2  -c rkpk1.f -o rkpk1.o
rkpk1.f:1972.72:

   10 ASSIGN 30 TO NEXT 
   1
Warning: Obsolete: ASSIGN statement at (1)
rkpk1.f:1977.19:

   20GO TO NEXT,(30, 50, 70, 110)   
  1
Warning: Obsolete: Assigned GOTO statement at (1)
rkpk1.f:1979.72:

  ASSIGN 50 TO NEXT 
   1
Warning: Obsolete: ASSIGN statement at (1)
rkpk1.f:1988.72:

  ASSIGN 70 TO NEXT 
   1
Warning: Obsolete: ASSIGN statement at (1)
rkpk1.f:1994.72:

  ASSIGN 110 TO NEXT
   1
Warning: Obsolete: ASSIGN statement at (1)

-
Good luck;

David.


> 
> Best,
> Uwe Ligges
> 
> 
>> 
>> 
>> Dan
>> 
>> 
>> 
>> Daniel Nordlund
>> 
>> Bothell, WA USA
>> 
>> 
>> 
>>   _
>> 
>> From: Shang Zuofeng [mailto:zuofengsh...@gmail.com]
>> Sent: Monday, August 19, 2013 2:16 PM
>> To: Daniel Nordlund
>> Cc: r-help@r-project.org
>> Subject: Re: [R] How to rebuild an R package that has been removed from CRAN?
>> 
>> 
>> 
>> Thanks, Dan!
>> 
>> The package is "assist" which can be downloaded from the following link:
>> 
>> http://cran.r-project.org/src/contrib/Archive/assist/
>> 
>> The one I chose was assist_3.1.2.tar.gz
>> 
>> I have changed this file to .zip and installed it from local directory 
>> through R. However, this method is still not working.
>> 
>> Thanks a lot for your kind help!
>> 
>> Best regards,
>> 
>> Zuofeng
>> 
>> 2013/8/19 Daniel Nordlund 
>> 
>>> -Original Message-
>>> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
>>> On Behalf Of Shang Zuofeng
>>> Sent: Monday, August 19, 2013 1:26 PM
>>> To: r-help@r-project.org
>>> Subject: [R] How to rebuild an R package that has been removed from CRAN?
>>> 
>>> Dear All,
>>> 
>>> My project requires the use of a specific R package. However, this R
>>> package has been removed from CRAN. But its older version can be found.
>>> Unfortunately, the older version cannot be used. The thing is, after I
>>> downloaded the older version and unzip it into the library folder of  R,
>>> and I input library("package name"), it says that the package "is not a
>>> valid installed package".
>>> 
>>> After an intensive search, I found a possible solution: to rebuild this R
>>> package so that it may work properly. I have no idea how to make this
>>> achieved because of my little experience on rebuilding an R package.
>>> 
>>> I highly appreciate your help.
>>> 
>>> Sincere thanks.
>>> Zuofeng
>>> 
>> 
>> Well, you haven't told us enough to let us help you.  Given that you have a 
>> zip file, I will assume for the moment that you are using some variant of MS 
>> Windows.  I don't think you want to unzip that file directly.  I think you 
>> want open R, go to the packages menu and choose install package from  local 
>> zip file.  Whether that will work depends on your version of R, your OS, the 
>> requirements of the package, why it was removed from CRAN, and a 

Re: [R] Appropriateness of R functions for multicore

[Daniel Nordlund]

The R high performance computing sig might be useful for some of these 
questions.
 
https://stat.ethz.ch/mailman/listinfo/r-sig-hpc


Dan

Daniel Nordlund
Bothell, WA USA
 

> -Original Message-
> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
> On Behalf Of Jeff Newmiller
> Sent: Monday, August 19, 2013 4:19 PM
> To: Patrick Connolly
> Cc: r-help@R-project.org; Hopkins,Bill
> Subject: Re: [R] Appropriateness of R functions for multicore
> 
> I don't know... I suppose it depends how it fails. I recommend that you
> restrict yourself to using only the data that was passed as parameters to
> your parallel function. You may be able to tackle parts of the task and
> return only those partial results to confirm how far through the code you
> can get.
> --
> -
> Jeff NewmillerThe .   .  Go
> Live...
> DCN:Basics: ##.#.   ##.#.  Live
> Go...
>   Live:   OO#.. Dead: OO#..  Playing
> Research Engineer (Solar/BatteriesO.O#.   #.O#.  with
> /Software/Embedded Controllers)   .OO#.   .OO#.
> rocks...1k
> --
> -
> Sent from my phone. Please excuse my brevity.
> 
> Patrick Connolly  wrote:
> >On Sat, 17-Aug-2013 at 05:09PM -0700, Jeff Newmiller wrote:
> >
> >
> >|> In most threaded multitasking environments it is not safe to
> >|> perform IO in multiple threads. In general you will have difficulty
> >|> performing IO in parallel processing so it is best to let the
> >|> master hand out data to worker tasks and gather results from them
> >|> for storage. Keep in mind that just because you have eight cores
> >|> for processing doesn't mean you have eight hard disks, so if your
> >|> problem is IO bound in single processor operation then it will also
> >|> be IO bound in threaded operation.
> >
> >For tasks which don't involve I/O but fail with mclapply, how does one
> >work out where the problem is?  The handy browser() function which
> >allows for interactive diagnosis won't work with parallel jobs.
> >
> >What other approaches can one use?
> >
> >Thanx
> >
> >
> >
> >
> >-
> --
> >
> >
> >
> >|> Jeff NewmillerThe .   .  Go
> >Live...
> >|> DCN:Basics: ##.#.   ##.#.
> >Live Go...
> >|>   Live:   OO#.. Dead: OO#..
> >Playing
> >|> Research Engineer (Solar/BatteriesO.O#.   #.O#.
> >with
> >|> /Software/Embedded Controllers)   .OO#.   .OO#.
> >rocks...1k
> >|>
> >-
> --
> >
> >|> Sent from my phone. Please excuse my brevity.
> >|>
> >|> "Hopkins, Bill"  wrote:
> >|> >Has there been any systematic evaluation of which core R functions
> >are
> >|> >safe for use with multicore? Of current interest, I have tried
> >calling
> >|> >read.table() via mclapply() to more quickly read in hundreds of raw
> >|> >data files (I have a 24 core system with 72 GB running Ubuntu, a
> >|> >perfect platform for multicore). There was a 40% failure rate,
> >which
> >|> >doesn't occur when I invoke read.table() serially from within a
> >single
> >|> >thread. Another example was using pvec() to invoke
> >|> >sapply(strsplit(),...) on a huge character vector (to pull out
> >fields
> >|> >from within a field). It looked like a perfect application for
> >pvec(),
> >|> >but it fails when serial execution works.
> >|> >
> >|> >I thought I'd ask before taking on the task of digging into the
> >|> >underlying code to see what is might be causing failure in a
> >multicore
> >|> >(well, multi-threaded) context.
> >|> >
> >|> >As an alternative, I could define multiple cluster nodes locally,
> >but
> >|> >that shifts the tradeoff a bit in whether parallel execution is
> >|> >advantageous - the overhead is significantly more, and even with 72
> >GB,
> >|> >it does impose greater limits on how many cores can be used.
> >|> >
> >|> >Bill Hopkins
> >|> >
> >|> >__
> >|> >R-help@r-project.org mailing list
> >|> >https://stat.ethz.ch/mailman/listinfo/r-help
> >|> >PLEASE do read the posting guide
> >|> >http://www.R-project.org/posting-guide.html
> >|> >and provide commented, minimal, self-contained, reproducible code.
> >|>
> >|> __
> >|> R-help@r-project.org mailing list
> >|> https://stat.ethz.ch/mailman/listinfo/r-help
> >|> PLEASE do read the posting guide
> >http://www.R-project.org/posting-guide.html
> >|> and provide commented, minimal, self-contained, reproducible code.
> 
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do rea

Re: [R] How to rebuild an R package that has been removed from CRAN?

[Daniel Nordlund]

Yeah, I tried building the package and got essentially the same warnings and 
decided that further assistance required someone above my pay grade. :-)

Daniel Nordlund
Bothell, WA USA
 

> -Original Message-
> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
> On Behalf Of David Winsemius
> Sent: Monday, August 19, 2013 4:43 PM
> To: Uwe Ligges
> Cc: r-help@r-project.org help
> Subject: Re: [R] How to rebuild an R package that has been removed from
> CRAN?
> 
> 
> On Aug 19, 2013, at 4:12 PM, Uwe Ligges wrote:
> 
> >
> > On 20.08.2013 01:06, Daniel Nordlund wrote:
> >> The file you had, assist_3.1.2.tar.gz, was not a Windows binary zip
> file.  It was a source tarball.  That kind of file needs to be built and
> installed differently.  In order to do that, you need to have all the
> tools necessary for building packages.  This package does not have just
> pure R code in it, but it has code that needs compiling.  You need to
> determine why the package was removed from CRAN.  It may no longer work
> with current versions of R.  You probably need to contact the package
> maintainer to resolve this problem.  Sorry I can't provide more help.
> >
> >
> >  and possible reasons for package archivals on CRAN are unresponsive
> maintainers of packages that do not pass the checks without problems any
> more.
> 
> I have a copy of 'assist' installed (for some mysterious reason my GUI
> package installer was able to find a binary copy for R 3.0.1 on my regular
> UC Berkeley CRAN repos):
> 
> > maintainer("assist")
> [1] "Chunlei Ke "
> 
> From the description file:
> 
> Package: assist
> Version: 3.1.2
> Title: A Suite of S-Plus Functions Implementing Smoothing Splines
> 
> Depends: R (>= 1.7.0), nlme
> 
> URL: http://www.pstat.ucsb.edu/faculty/yuedong/software.html
> Packaged: 2013-03-12 15:29:39 UTC; ripley
> Repository: CRAN
> Date/Publication: 2013-03-12 16:30:20
> NeedsCompilation: yes
> Built: R 3.0.0; x86_64-apple-darwin10.8.0; 2013-03-16 08:53:21 UTC; unix
> Archs: assist.so.dSYM
> 
> 
> I also installed from source and here are the warning messages:
> 
> 
> gfortran-4.2 -arch x86_64   -fPIC  -g -O2  -c rkpk1.f -o rkpk1.o
> rkpk1.f:1972.72:
> 
>10 ASSIGN 30 TO NEXT
>1
> Warning: Obsolete: ASSIGN statement at (1)
> rkpk1.f:1977.19:
> 
>20GO TO NEXT,(30, 50, 70, 110)
>   1
> Warning: Obsolete: Assigned GOTO statement at (1)
> rkpk1.f:1979.72:
> 
>   ASSIGN 50 TO NEXT
>1
> Warning: Obsolete: ASSIGN statement at (1)
> rkpk1.f:1988.72:
> 
>   ASSIGN 70 TO NEXT
>1
> Warning: Obsolete: ASSIGN statement at (1)
> rkpk1.f:1994.72:
> 
>   ASSIGN 110 TO NEXT
>1
> Warning: Obsolete: ASSIGN statement at (1)
> 
> -
> Good luck;
> 
> David.
> 
> 
> >
> > Best,
> > Uwe Ligges
> >
> >
> >>
> >>
> >> Dan
> >>
> >>
> >>
> >> Daniel Nordlund
> >>
> >> Bothell, WA USA
> >>
> >>
> >>
> >>   _
> >>
> >> From: Shang Zuofeng [mailto:zuofengsh...@gmail.com]
> >> Sent: Monday, August 19, 2013 2:16 PM
> >> To: Daniel Nordlund
> >> Cc: r-help@r-project.org
> >> Subject: Re: [R] How to rebuild an R package that has been removed from
> CRAN?
> >>
> >>
> >>
> >> Thanks, Dan!
> >>
> >> The package is "assist" which can be downloaded from the following
> link:
> >>
> >> http://cran.r-project.org/src/contrib/Archive/assist/
> >>
> >> The one I chose was assist_3.1.2.tar.gz
> >>
> >> I have changed this file to .zip and installed it from local directory
> through R. However, this method is still not working.
> >>
> >> Thanks a lot for your kind help!
> >>
> >> Best regards,
> >>
> >> Zuofeng
> >>
> >> 2013/8/19 Daniel Nordlund 
> >>
> >>> -Original Message-
> >>> From: r-help-boun...@r-project.org [mailto:r-help-bounces@r-
> project.org]
> >>> On Behalf Of Shang Zuofeng
> >>> Sent: Monday, August 19, 2013 1:26 PM
> >>> To: r-help@r-project.org
> >>> Subject: [R] How to rebuild an R package that has been removed from
> CRAN?
> >>>
> >>> Dear All,
> >>>
> >>> My project requires the use of a specific R package. However, this R
> >>> package has been removed from CRAN. But its older version can be
> found.
> >>> Unfortunately, the older version cannot be used. The thing is, after I
> >>> downloaded the older version and unzip it into the library folder of
> R,
> >>> and I input library("package name"), it says that the package "is not
> a
> >>> valid installed package".
> >>>
> >>> After an intensive search, I found a possible solution: to rebuild
> this R
> >>> package so that it may work properly. I have no idea how to make this
> >>> achieved because of my little experience on rebuilding an R package.
> >>>
> >>> I highly appreciate your help.
> >>>
> >>>

Re: [R] How to rebuild an R package that has been removed from CRAN?

[William Dunlap]

> further assistance required someone above my pay grade. :-)

Perhaps you just need someone above your age - that is FORTRAN IV syntax
(maybe I and II as well) which was finally removed from the standard for
Fortran 95.  If I recall that correctly from 1966 you can replace the
ASSIGN 30 TO NEXT
by
NEXT = 30
(and the same for the other ASSIGN's) and replace
GO TO NEXT,(30, 50, 70, 110)
by the sequence
IF (NEXT .EQ. 30) GO TO 30
IF (NEXT .EQ. 50) GO TO 50
IF (NEXT .EQ 70) GO TO 70
IF (NEXT .EQ. 110) GO TO 110
C SHOULD NEVER GET HERE - call some error reporting code to be careful

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com


> -Original Message-
> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On 
> Behalf
> Of Daniel Nordlund
> Sent: Monday, August 19, 2013 4:58 PM
> To: r-help@r-project.org
> Subject: Re: [R] How to rebuild an R package that has been removed from CRAN?
> 
> Yeah, I tried building the package and got essentially the same warnings and 
> decided that
> further assistance required someone above my pay grade. :-)
> 
> Daniel Nordlund
> Bothell, WA USA
> 
> 
> > -Original Message-
> > From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
> > On Behalf Of David Winsemius
> > Sent: Monday, August 19, 2013 4:43 PM
> > To: Uwe Ligges
> > Cc: r-help@r-project.org help
> > Subject: Re: [R] How to rebuild an R package that has been removed from
> > CRAN?
> >
> >
> > On Aug 19, 2013, at 4:12 PM, Uwe Ligges wrote:
> >
> > >
> > > On 20.08.2013 01:06, Daniel Nordlund wrote:
> > >> The file you had, assist_3.1.2.tar.gz, was not a Windows binary zip
> > file.  It was a source tarball.  That kind of file needs to be built and
> > installed differently.  In order to do that, you need to have all the
> > tools necessary for building packages.  This package does not have just
> > pure R code in it, but it has code that needs compiling.  You need to
> > determine why the package was removed from CRAN.  It may no longer work
> > with current versions of R.  You probably need to contact the package
> > maintainer to resolve this problem.  Sorry I can't provide more help.
> > >
> > >
> > >  and possible reasons for package archivals on CRAN are unresponsive
> > maintainers of packages that do not pass the checks without problems any
> > more.
> >
> > I have a copy of 'assist' installed (for some mysterious reason my GUI
> > package installer was able to find a binary copy for R 3.0.1 on my regular
> > UC Berkeley CRAN repos):
> >
> > > maintainer("assist")
> > [1] "Chunlei Ke "
> >
> > From the description file:
> >
> > Package: assist
> > Version: 3.1.2
> > Title: A Suite of S-Plus Functions Implementing Smoothing Splines
> >
> > Depends: R (>= 1.7.0), nlme
> >
> > URL: http://www.pstat.ucsb.edu/faculty/yuedong/software.html
> > Packaged: 2013-03-12 15:29:39 UTC; ripley
> > Repository: CRAN
> > Date/Publication: 2013-03-12 16:30:20
> > NeedsCompilation: yes
> > Built: R 3.0.0; x86_64-apple-darwin10.8.0; 2013-03-16 08:53:21 UTC; unix
> > Archs: assist.so.dSYM
> >
> >
> > I also installed from source and here are the warning messages:
> >
> >
> > gfortran-4.2 -arch x86_64   -fPIC  -g -O2  -c rkpk1.f -o rkpk1.o
> > rkpk1.f:1972.72:
> >
> >10 ASSIGN 30 TO NEXT
> >1
> > Warning: Obsolete: ASSIGN statement at (1)
> > rkpk1.f:1977.19:
> >
> >20GO TO NEXT,(30, 50, 70, 110)
> >   1
> > Warning: Obsolete: Assigned GOTO statement at (1)
> > rkpk1.f:1979.72:
> >
> >   ASSIGN 50 TO NEXT
> >1
> > Warning: Obsolete: ASSIGN statement at (1)
> > rkpk1.f:1988.72:
> >
> >   ASSIGN 70 TO NEXT
> >1
> > Warning: Obsolete: ASSIGN statement at (1)
> > rkpk1.f:1994.72:
> >
> >   ASSIGN 110 TO NEXT
> >1
> > Warning: Obsolete: ASSIGN statement at (1)
> >
> > -
> > Good luck;
> >
> > David.
> >
> >
> > >
> > > Best,
> > > Uwe Ligges
> > >
> > >
> > >>
> > >>
> > >> Dan
> > >>
> > >>
> > >>
> > >> Daniel Nordlund
> > >>
> > >> Bothell, WA USA
> > >>
> > >>
> > >>
> > >>   _
> > >>
> > >> From: Shang Zuofeng [mailto:zuofengsh...@gmail.com]
> > >> Sent: Monday, August 19, 2013 2:16 PM
> > >> To: Daniel Nordlund
> > >> Cc: r-help@r-project.org
> > >> Subject: Re: [R] How to rebuild an R package that has been removed from
> > CRAN?
> > >>
> > >>
> > >>
> > >> Thanks, Dan!
> > >>
> > >> The package is "assist" which can be downloaded from the following
> > link:
> > >>
> > >> http://cran.r-project.org/src/contrib/Archive/assist/
> > >>
> > >> The one I chose was assist_3.1.2.tar.gz
> > >>
> > >> I have changed this file to .zip and installed it from local directory
> > through R. However, this 

Re: [R] arules LHS & RHS AND Condition Filtering

[Kevin Shaney]

Hello - my message was bounced, but I cannot tell why.  Please let me know what 
I need to do to post / overcome bounce.

-Original Message-
From: kevin.shaney [mailto:kevin.sha...@rosetta.com]
Sent: Monday, August 19, 2013 8:35 PM
To: r-help@r-project.org
Subject: arules LHS & RHS AND Condition Filtering

I have been looking around rhelp and online for help on how to filter arules 
lists w.r.t. the following problem.  Any help is much appreciated!  Also, see 
code below.

- Have a data file with {rows = product orders}, {columns = product 
categories}, and {data = 1/0 yes/no flag indicating whether purchase-x included 
category-y}
- Want to select subset of RHS and LHS rules that ONLY include instances of 
purchase = 1 (e.g. I don’t care about whether a non-purchase event on LHS 
corresponded with a purchase event on RHS, and don’t care about non-purchase 
events at all on RHS)
- Code below works somewhat, but seems to be an OR condition (e.g. there are 
rules of form {V1=1, V2=0} => {V3=1} making it into my subset list.

Is there any way to limit both RHS and LHS so only subset rules with all 1’s 
are included?  Also, if below is generally correct, is there a more compact way 
to write all the subset rules (since all are intended to say “only want 
variable values = 1 in rules”)?

Appreciate help!



arulesinput <- read.csv("ARULES_EX.csv",stringsAsFactors=TRUE)
arulesinput_subset = cbind(  data.frame(lapply(arulesinput[,c(2:6)],factor))
)
rules.all <- apriori(arulesinput_subset, parameter =
list(minlen=2,maxlen=4,supp=0.01,conf=0.8))

rules_subset <- subset(rules.all,(lhs %in% c("V1=1", "V2=1", "V3=1", "V4=1",
"V5=1")) &
(rhs %in% c("V1=1",
"V2=1",
"V3=1",
"V4=1",
"V5=1"))
)

rules_subset_frame <- as(rules_subset,"data.frame")
write.csv(as(rules_subset,"data.frame"),"TEMP.csv")




--
View this message in context: 
http://r.789695.n4.nabble.com/arules-LHS-RHS-AND-Condition-Filtering-tp4674109.html
Sent from the R help mailing list archive at Nabble.com.


This e-mail message contains information that may be non-public, confidential 
or proprietary.
It is intended to be read only by the intended recipient(s).  Use, 
dissemination, distribution, or
reproduction of this message by unintended recipients is not authorized and may 
be unlawful.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] How to rebuild an R package that has been removed from CRAN?

[David Winsemius]

On Aug 19, 2013, at 5:07 PM, Shang Zuofeng wrote:

> So this is an alternative method. The package can be installed from source() 
> rather than rebuilt. Although the warnings exist, the package itself may 
> still be useful. Can you let me know how to installed from source?
> 
Quite a bit of effort has gon e into the manuals shipped with every 
installation of R. You should have gotten a copy of this;

http://www.cran.r-project.org/doc/manuals/R-admin.html


> Thanks a lot!
> Zuofeng
> 
> 
> 2013/8/19 David Winsemius 
> 
> On Aug 19, 2013, at 4:12 PM, Uwe Ligges wrote:
> 
> >
> > On 20.08.2013 01:06, Daniel Nordlund wrote:
> >> The file you had, assist_3.1.2.tar.gz, was not a Windows binary zip file.  
> >> It was a source tarball.  That kind of file needs to be built and 
> >> installed differently.  In order to do that, you need to have all the 
> >> tools necessary for building packages.  This package does not have just 
> >> pure R code in it, but it has code that needs compiling.  You need to 
> >> determine why the package was removed from CRAN.  It may no longer work 
> >> with current versions of R.  You probably need to contact the package 
> >> maintainer to resolve this problem.  Sorry I can't provide more help.
> >
> >
> >  and possible reasons for package archivals on CRAN are unresponsive 
> > maintainers of packages that do not pass the checks without problems any 
> > more.
> 
> I have a copy of 'assist' installed (for some mysterious reason my GUI 
> package installer was able to find a binary copy for R 3.0.1 on my regular UC 
> Berkeley CRAN repos):
> 
> > maintainer("assist")
> [1] "Chunlei Ke "
> 
> From the description file:
> 
> Package: assist
> Version: 3.1.2
> Title: A Suite of S-Plus Functions Implementing Smoothing Splines
> 
> Depends: R (>= 1.7.0), nlme
> 
> URL: http://www.pstat.ucsb.edu/faculty/yuedong/software.html
> Packaged: 2013-03-12 15:29:39 UTC; ripley
> Repository: CRAN
> Date/Publication: 2013-03-12 16:30:20
> NeedsCompilation: yes
> Built: R 3.0.0; x86_64-apple-darwin10.8.0; 2013-03-16 08:53:21 UTC; unix
> Archs: assist.so.dSYM
> 
> 
> I also installed from source and here are the warning messages:
> 
> 
> gfortran-4.2 -arch x86_64   -fPIC  -g -O2  -c rkpk1.f -o rkpk1.o
> rkpk1.f:1972.72:
> 
>10 ASSIGN 30 TO NEXT
>1
> Warning: Obsolete: ASSIGN statement at (1)
> rkpk1.f:1977.19:
> 
>20GO TO NEXT,(30, 50, 70, 110)
>   1
> Warning: Obsolete: Assigned GOTO statement at (1)
> rkpk1.f:1979.72:
> 
>   ASSIGN 50 TO NEXT
>1
> Warning: Obsolete: ASSIGN statement at (1)
> rkpk1.f:1988.72:
> 
>   ASSIGN 70 TO NEXT
>1
> Warning: Obsolete: ASSIGN statement at (1)
> rkpk1.f:1994.72:
> 
>   ASSIGN 110 TO NEXT
>1
> Warning: Obsolete: ASSIGN statement at (1)
> 
> -
> Good luck;
> 
> David.
> 
> 
> >
> > Best,
> > Uwe Ligges
> >
> >
> >>
> >>
> >> Dan
> >>
> >>
> >>
> >> Daniel Nordlund
> >>
> >> Bothell, WA USA
> >>
> >>
> >>
> >>   _
> >>
> >> From: Shang Zuofeng [mailto:zuofengsh...@gmail.com]
> >> Sent: Monday, August 19, 2013 2:16 PM
> >> To: Daniel Nordlund
> >> Cc: r-help@r-project.org
> >> Subject: Re: [R] How to rebuild an R package that has been removed from 
> >> CRAN?
> >>
> >>
> >>
> >> Thanks, Dan!
> >>
> >> The package is "assist" which can be downloaded from the following link:
> >>
> >> http://cran.r-project.org/src/contrib/Archive/assist/
> >>
> >> The one I chose was assist_3.1.2.tar.gz
> >>
> >> I have changed this file to .zip and installed it from local directory 
> >> through R. However, this method is still not working.
> >>
> >> Thanks a lot for your kind help!
> >>
> >> Best regards,
> >>
> >> Zuofeng
> >>
> >> 2013/8/19 Daniel Nordlund 
> >>
> >>> -Original Message-
> >>> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
> >>> On Behalf Of Shang Zuofeng
> >>> Sent: Monday, August 19, 2013 1:26 PM
> >>> To: r-help@r-project.org
> >>> Subject: [R] How to rebuild an R package that has been removed from CRAN?
> >>>
> >>> Dear All,
> >>>
> >>> My project requires the use of a specific R package. However, this R
> >>> package has been removed from CRAN. But its older version can be found.
> >>> Unfortunately, the older version cannot be used. The thing is, after I
> >>> downloaded the older version and unzip it into the library folder of  R,
> >>> and I input library("package name"), it says that the package "is not a
> >>> valid installed package".
> >>>
> >>> After an intensive search, I found a possible solution: to rebuild this R
> >>> package so that it may work properly. I have no idea how to make this
> >>> achieved because of my little experience on rebuilding an R pac

[R] How to run Bekk-MGarch analysis ?

[YuHong]

Hello all,

We would like to use R to run Bekk-Garch with 2-group to 4-group financial 
data sets.  However, the package "rmgarch" seems to only support DCC-GARCH, 
not BEKK-GARCH.  Any suggestions on using R to do BEKK-GARCH test?


Thanks a lot! & Best,

Hong Yu

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] can I calculate density for only non-negative value?

[Gallon Li]

 x=rchisq(100,1)
density(x)

the density plot will give density for negative part also. of course I can
truncate the plot to only view the non-negative part.

I wonder if there is a program to compute density for a user-specified
range, in this case, only [0, infinity).

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Creating the Spatial Weights

[Sebastian Kruk]

Dear R-users,

I have a shape file (.shp, sbx, .sbn, .shx and .dbf).

Can I create the Spatial Weights inside R?

Thanks,

Sebastián.

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Creating the Spatial Weights

[Pascal Oettli]

Hello,

Before asking, did you search on Internet using a web search engine?

Regards,
Pascal



2013/8/20 Sebastian Kruk 

> Dear R-users,
>
> I have a shape file (.shp, sbx, .sbn, .shx and .dbf).
>
> Can I create the Spatial Weights inside R?
>
> Thanks,
>
> Sebastián.
>
> [[alternative HTML version deleted]]
>
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Output from unix PS

[mohan . radhakrishnan]

Hi,

I am able to proceed like this.

Input :

4000  36  36   0 r-x--  java
40108000   8   8   8 rwx--  java
4010a000  12   0   0 -[ anon ]
4010d0001016  44  44 rwx--[ anon ]
4020b000  12   0   0 -[ anon ]

Ouput : There are a few problems but I am coding to solve them

   [,1]   [,2]  [,3]  [,4]
[,5]  [,6]  [,7]
[,8] [,9]   [,10] [,11]
  [1,] "4000" "36"  "36"  "0"   "r-x--"   "java"
NA  NA   NA   NA""
  [2,] "40108000" "8"   "8"   "8"   "rwx--"   "java"
NA  NA   NA   NA""



d <- do.call(rbind, lapply(strsplit(readLines("D:\\Log analysis\\Process
Maps\\19-08-2013\\node1-pmap.txt"), "\\s+"), function(fields) c(fields
[1:10], paste(fields[-(1:10)], collapse = " "
colnames(d) <- c("Address","Kbytes","RSS","Dirty
Mode","Mapping","Test6","Test7","Test8","Test9","Test10","Test11")


Type1 <- setNames(aggregate(as.numeric(Kbytes) ~Test7,data=d,sum),c
("AllocationType","Size"))

Type2 <- setNames(aggregate(as.numeric(Kbytes) ~Test6,data=d,sum),c
("AllocationType","Size"))


par(mfcol=c(1,3))
barplot(matrix(c(Type1[1,2],Type1[2,2])),beside=T, ylim = c(0, 1),
ylab="Kbytes",col=c("aquamarine3","coral","green","violet"))
barplot(Type2[,'Size'], width=10,ylim = c(0, 500), space=6, ylab="Kbytes")


The input is not cleanly separated into columns because there are spaces
and braces but this works.


Thanks,
Mohan





   Re: [R] Output from unix PS  


   mohan.radhakrishnan  
to: 
  Sarah Goslee  
16-08-2013 07:30 PM 




   Sent by: 
  r-help-boun...@r-project.org  
   Cc:  
  r-help









This is the output.

> str(data)
'data.frame':   78 obs. of  1 variable:
 $
USER...PID..CPU..MEMVSZ...RSS.TTY..STAT.START...TIME.COMMAND:
Factor w/ 78 levels "684504  0.0  0.0  31680  4596 ?Ss
Aug08   0:01 hald",..: 17 18 19 20 21 22 23 24 25 26 ...




> dput(head(data,20))
structure(list
(USER...PID..CPU..MEMVSZ...RSS.TTY..STAT.START...TIME.COMMAND =
structure(c(17L,
18L, 19L, 20L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 15L, 29L,
30L, 31L, 32L, 33L, 1L, 34L), .Label = c("684504  0.0  0.0  31680
4596 ?Ss   Aug08   0:01 hald",
"684513  0.0  0.0  12348   872 ?SAug08   0:00
hald-addon-acpi: listening on acpid socket /var/run/acpid.socket",
"684517  0.0  0.0  12348   864 ?SAug08   0:00
hald-addon-keyboard: listening on /dev/input/event1",
"684521  0.0  0.0  12348   860 ?SAug08   0:00
hald-addon-keyboard: listening on /dev/input/event0",
"apache   17344  0.0  0.0 174440  2372 ?SAug11
0:00 /usr/sbin/httpd",
"apache   17345  0.0  0.0 174440  2372 ?SAug11
0:00 /usr/sbin/httpd",
"apache   17346  0.0  0.0 174440  2372 ?SAug11
0:00 /usr/sbin/httpd",
"apache   17347  0.0  0.0 174440  2372 ?SAug11
0:00 /usr/sbin/httpd",
"apache   17348  0.0  0.0 174440  2372 ?SAug11
0:00 /usr/sbin/httpd",
"apache   17349  0.0  0.0 174440  2372 ?SAug11
0:00 /usr/sbin/httpd",
"apache   17350  0.0  0.0 174440  2372 ?SAug11
0:00 /usr/sbin/httpd",
"apache   17351  0.0  0.0 174440  2372 ?SAug11
0:00 /usr/sbin/httpd",
"avahi 4830  0.0  0.0  23296  1316 ?Ss   Aug08   

Re: [R] can I calculate density for only non-negative value?

[Prof Brian Ripley]

On 20/08/2013 03:35, Gallon Li wrote:

  x=rchisq(100,1)
density(x)

the density plot will give density for negative part also. of course I can
truncate the plot to only view the non-negative part.

I wonder if there is a program to compute density for a user-specified
range, in this case, only [0, infinity).


Yes, but it will not be using the same method as density.  One place to 
look is logspline() in package polspline, but please learn about the 
methodology of density estimation before you use any of these.




[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.




--
Brian D. Ripley,  rip...@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel:  +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax:  +44 1865 272595

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] can I calculate density for only non-negative value?

[David Winsemius]

On Aug 19, 2013, at 7:35 PM, Gallon Li wrote:

> x=rchisq(100,1)
> density(x)
> 
> the density plot will give density for negative part also. of course I can
> truncate the plot to only view the non-negative part.
> 
> I wonder if there is a program to compute density for a user-specified
> range, in this case, only [0, infinity).

This has been asked and answered several times on this list.

-- 

David Winsemius
Alameda, CA, USA

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] How to rebuild an R package that has been removed from CRAN?

[Shang Zuofeng]

So this is an alternative method. The package can be installed from
source() rather than rebuilt. Although the warnings exist, the package
itself may still be useful. Can you let me know how to installed from
source?

Thanks a lot!
Zuofeng


2013/8/19 David Winsemius 

>
> On Aug 19, 2013, at 4:12 PM, Uwe Ligges wrote:
>
> >
> > On 20.08.2013 01:06, Daniel Nordlund wrote:
> >> The file you had, assist_3.1.2.tar.gz, was not a Windows binary zip
> file.  It was a source tarball.  That kind of file needs to be built and
> installed differently.  In order to do that, you need to have all the tools
> necessary for building packages.  This package does not have just pure R
> code in it, but it has code that needs compiling.  You need to determine
> why the package was removed from CRAN.  It may no longer work with current
> versions of R.  You probably need to contact the package maintainer to
> resolve this problem.  Sorry I can't provide more help.
> >
> >
> >  and possible reasons for package archivals on CRAN are unresponsive
> maintainers of packages that do not pass the checks without problems any
> more.
>
> I have a copy of 'assist' installed (for some mysterious reason my GUI
> package installer was able to find a binary copy for R 3.0.1 on my regular
> UC Berkeley CRAN repos):
>
> > maintainer("assist")
> [1] "Chunlei Ke "
>
> From the description file:
>
> Package: assist
> Version: 3.1.2
> Title: A Suite of S-Plus Functions Implementing Smoothing Splines
>
> Depends: R (>= 1.7.0), nlme
>
> URL: http://www.pstat.ucsb.edu/faculty/yuedong/software.html
> Packaged: 2013-03-12 15:29:39 UTC; ripley
> Repository: CRAN
> Date/Publication: 2013-03-12 16:30:20
> NeedsCompilation: yes
> Built: R 3.0.0; x86_64-apple-darwin10.8.0; 2013-03-16 08:53:21 UTC; unix
> Archs: assist.so.dSYM
>
>
> I also installed from source and here are the warning messages:
>
>
> gfortran-4.2 -arch x86_64   -fPIC  -g -O2  -c rkpk1.f -o rkpk1.o
> rkpk1.f:1972.72:
>
>10 ASSIGN 30 TO NEXT
>1
> Warning: Obsolete: ASSIGN statement at (1)
> rkpk1.f:1977.19:
>
>20GO TO NEXT,(30, 50, 70, 110)
>   1
> Warning: Obsolete: Assigned GOTO statement at (1)
> rkpk1.f:1979.72:
>
>   ASSIGN 50 TO NEXT
>1
> Warning: Obsolete: ASSIGN statement at (1)
> rkpk1.f:1988.72:
>
>   ASSIGN 70 TO NEXT
>1
> Warning: Obsolete: ASSIGN statement at (1)
> rkpk1.f:1994.72:
>
>   ASSIGN 110 TO NEXT
>1
> Warning: Obsolete: ASSIGN statement at (1)
>
> -
> Good luck;
>
> David.
>
>
> >
> > Best,
> > Uwe Ligges
> >
> >
> >>
> >>
> >> Dan
> >>
> >>
> >>
> >> Daniel Nordlund
> >>
> >> Bothell, WA USA
> >>
> >>
> >>
> >>   _
> >>
> >> From: Shang Zuofeng [mailto:zuofengsh...@gmail.com]
> >> Sent: Monday, August 19, 2013 2:16 PM
> >> To: Daniel Nordlund
> >> Cc: r-help@r-project.org
> >> Subject: Re: [R] How to rebuild an R package that has been removed from
> CRAN?
> >>
> >>
> >>
> >> Thanks, Dan!
> >>
> >> The package is "assist" which can be downloaded from the following link:
> >>
> >> http://cran.r-project.org/src/contrib/Archive/assist/
> >>
> >> The one I chose was assist_3.1.2.tar.gz
> >>
> >> I have changed this file to .zip and installed it from local directory
> through R. However, this method is still not working.
> >>
> >> Thanks a lot for your kind help!
> >>
> >> Best regards,
> >>
> >> Zuofeng
> >>
> >> 2013/8/19 Daniel Nordlund 
> >>
> >>> -Original Message-
> >>> From: r-help-boun...@r-project.org [mailto:
> r-help-boun...@r-project.org]
> >>> On Behalf Of Shang Zuofeng
> >>> Sent: Monday, August 19, 2013 1:26 PM
> >>> To: r-help@r-project.org
> >>> Subject: [R] How to rebuild an R package that has been removed from
> CRAN?
> >>>
> >>> Dear All,
> >>>
> >>> My project requires the use of a specific R package. However, this R
> >>> package has been removed from CRAN. But its older version can be found.
> >>> Unfortunately, the older version cannot be used. The thing is, after I
> >>> downloaded the older version and unzip it into the library folder of
>  R,
> >>> and I input library("package name"), it says that the package "is not a
> >>> valid installed package".
> >>>
> >>> After an intensive search, I found a possible solution: to rebuild
> this R
> >>> package so that it may work properly. I have no idea how to make this
> >>> achieved because of my little experience on rebuilding an R package.
> >>>
> >>> I highly appreciate your help.
> >>>
> >>> Sincere thanks.
> >>> Zuofeng
> >>>
> >>
> >> Well, you haven't told us enough to let us help you.  Given that you
> have a zip file, I will assume for the moment that you are using some
> variant of MS Windows.  I don't think you want to unzip that f

Re: [R] Appropriateness of R functions for multicore

[Hopkins, Bill]

I wrap functions to run via multicore with tryCatch() to gather stats on 
failure rate and capture state.



I'm still interested in how/whether core fuctions were verified as being 
threadsafe.



Bill Hopkins



Written using a virtual Android keyboard...

-- Original message --
From: Jeff Newmiller
Date: 8/19/2013 5:18 PM
To: Patrick Connolly;
Cc: Hopkins, Bill;r-help@R-project.org;
Subject:Re: [R] Appropriateness of R functions for multicore

I don't know... I suppose it depends how it fails. I recommend that you 
restrict yourself to using only the data that was passed as parameters to your 
parallel function. You may be able to tackle parts of the task and return only 
those partial results to confirm how far through the code you can get.
---
Jeff NewmillerThe .   .  Go Live...
DCN:Basics: ##.#.   ##.#.  Live Go...
  Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/BatteriesO.O#.   #.O#.  with
/Software/Embedded Controllers)   .OO#.   .OO#.  rocks...1k
---
Sent from my phone. Please excuse my brevity.

Patrick Connolly  wrote:
>On Sat, 17-Aug-2013 at 05:09PM -0700, Jeff Newmiller wrote:
>
>
>|> In most threaded multitasking environments it is not safe to
>|> perform IO in multiple threads. In general you will have difficulty
>|> performing IO in parallel processing so it is best to let the
>|> master hand out data to worker tasks and gather results from them
>|> for storage. Keep in mind that just because you have eight cores
>|> for processing doesn't mean you have eight hard disks, so if your
>|> problem is IO bound in single processor operation then it will also
>|> be IO bound in threaded operation.
>
>For tasks which don't involve I/O but fail with mclapply, how does one
>work out where the problem is?  The handy browser() function which
>allows for interactive diagnosis won't work with parallel jobs.
>
>What other approaches can one use?
>
>Thanx
>
>
>
>
>---
>
>
>
>|> Jeff NewmillerThe .   .  Go
>Live...
>|> DCN:Basics: ##.#.   ##.#.
>Live Go...
>|>   Live:   OO#.. Dead: OO#..
>Playing
>|> Research Engineer (Solar/BatteriesO.O#.   #.O#.
>with
>|> /Software/Embedded Controllers)   .OO#.   .OO#.
>rocks...1k
>|>
>---
>
>|> Sent from my phone. Please excuse my brevity.
>|>
>|> "Hopkins, Bill"  wrote:
>|> >Has there been any systematic evaluation of which core R functions
>are
>|> >safe for use with multicore? Of current interest, I have tried
>calling
>|> >read.table() via mclapply() to more quickly read in hundreds of raw
>|> >data files (I have a 24 core system with 72 GB running Ubuntu, a
>|> >perfect platform for multicore). There was a 40% failure rate,
>which
>|> >doesn't occur when I invoke read.table() serially from within a
>single
>|> >thread. Another example was using pvec() to invoke
>|> >sapply(strsplit(),...) on a huge character vector (to pull out
>fields
>|> >from within a field). It looked like a perfect application for
>pvec(),
>|> >but it fails when serial execution works.
>|> >
>|> >I thought I'd ask before taking on the task of digging into the
>|> >underlying code to see what is might be causing failure in a
>multicore
>|> >(well, multi-threaded) context.
>|> >
>|> >As an alternative, I could define multiple cluster nodes locally,
>but
>|> >that shifts the tradeoff a bit in whether parallel execution is
>|> >advantageous - the overhead is significantly more, and even with 72
>GB,
>|> >it does impose greater limits on how many cores can be used.
>|> >
>|> >Bill Hopkins
>|> >
>|> >__
>|> >R-help@r-project.org mailing list
>|> >https://stat.ethz.ch/mailman/listinfo/r-help
>|> >PLEASE do read the posting guide
>|> >http://www.R-project.org/posting-guide.html
>|> >and provide commented, minimal, self-contained, reproducible code.
>|>
>|> __
>|> R-help@r-project.org mailing list
>|> https://stat.ethz.ch/mailman/listinfo/r-help
>|> PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>|> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Interpreting lmp() results

[Ann Marie Reinhold]

I am running permutation regressions in package lmPerm using lmp().  I
am getting what I find to be confusing results and I would like help
understanding what is going on.  To illustrate my problem, I created a
simple example and am running lmp() such that the results of the lmp()
models should be identical to that of lm().  I'm referring to the
notes section of the lmp() documentation where it says that the
"function will behave identically to lm() if the following parameters
are set: perm="", seqs=TRUE, center=FALSE."

Here is an example wherein I am unable to match my lmp() results to my
lm() results.

library(lmPerm)
library(lattice)

x1 <- c(rnorm(60, 150, 50),rnorm(60, 150, 50),rnorm(60, 150, 50))
y1 <- c(30-0.1*x1[1:60], rep(10, 60), 0.1*x1[121:180])
factor.levels1 <- c(rep("DOWN", 60), rep("FLAT", 60), rep("UP", 60))

xyplot(y1 ~ x1, groups = factor.levels1, auto.key = TRUE)

lmp.model.1 <- lmp(y1 ~ x1*factor.levels1 - 1,  perm = "", seqs =
TRUE, center = FALSE)
summary(lmp.model.1)
lm.model.1 <- lm(y1 ~ x1*factor.levels1 - 1)
summary(lm.model.1)

Here are the results:
> summary(lmp.model.1)
Call:
lmp(formula = y1 ~ x1 * factor.levels1 - 1, perm = "", seqs = TRUE,
center = FALSE)
Residuals:
   Min 1Q Median 3QMax
-1.509e-13 -1.700e-16  4.277e-17  9.558e-16  1.621e-14
Coefficients:
 Estimate Std. Errort value Pr(>|t|)
factor.levels1DOWN  3.000e+01  7.359e-15  4.077e+15   <2e-16 ***
factor.levels1FLAT  1.000e+01  4.952e-15  2.019e+15   <2e-16 ***
factor.levels1UP   -5.809e-16  5.095e-15 -1.140e-01   0.9094
x1  4.096e-17  2.137e-17  1.917e+00   0.0569 .
x1:factor.levels11 -1.000e-01  3.391e-17 -2.949e+15   <2e-16 ***
x1:factor.levels12 -4.500e-17  2.792e-17 -1.612e+00   0.1089
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 1.226e-14 on 174 degrees of freedom
Multiple R-Squared: 1,  Adjusted R-squared: 1
F-statistic: 3.721e+31 on 6 and 174 DF,  p-value: < 2.2e-16

> summary(lm.model.1)
Call:
lm(formula = y1 ~ x1 * factor.levels1 - 1)
Residuals:
   Min 1Q Median 3QMax
-3.141e-14 -3.190e-15 -9.880e-16  8.920e-16  1.905e-13
Coefficients:
Estimate Std. Errort value Pr(>|t|)
x1-1.000e-01  5.638e-17 -1.774e+15   <2e-16 ***
factor.levels1DOWN 3.000e+01  9.099e-15  3.297e+15   <2e-16 ***
factor.levels1FLAT 1.000e+01  6.123e-15  1.633e+15   <2e-16 ***
factor.levels1UP  -3.931e-15  6.300e-15 -6.240e-010.533
x1:factor.levels1FLAT  1.000e-01  6.826e-17  1.465e+15   <2e-16 ***
x1:factor.levels1UP2.000e-01  6.931e-17  2.886e+15   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 1.515e-14 on 174 degrees of freedom
Multiple R-squared:  1, Adjusted R-squared:  1


I thought that the results of summary(lmp.model.1) would be the same
as summary(lm.model.1).  However, I am concerned that I am
interpreting the results incorrectly because I can't get the results
to match.  Specifically, I simulated data with a slope for UP of 0.1,
the slope for FLAT of 0, and the slope for DOWN of -0.1. I can recover
these values in lm.model.1, but not lmp.model.1.  In the output for
the lmp.model.1, I am estimating the slope for DOWN to be
approximately -0.1 (4.096e-17-1.000e-01) and the slope of the FLAT to
be approximately 0 (4.096e-17-4.500e-17); however, the slope of UP
(what I think is equal to the reference level x1) is 4.096e-17.  Am I
interpreting the x1 term incorrectly?  Why are the lmp() results not
identical to the lm() results?

I ran a similar example using a modification of the above data wherein
factor level A is equal to FLAT, factor level B is equal to DOWN, and
factor level C is equal to UP.  Again, I was unable to match the
results from lm() and lmp().

x2 <- c(rnorm(60, 150, 50), rnorm(60, 150, 50),rnorm(60, 150, 50))
y2 <- c(rep(10, 60), 30-0.1*x2[61:120], 0.1*x2[121:180])
factor.levels2 <- c(rep("A", 60), rep("B", 60), rep("C", 60))

xyplot(y2 ~ x2, groups = factor.levels2, auto.key = TRUE)
lmp.model.2 <- lmp(y2 ~ x2*factor.levels2 - 1,  perm = "", seqs =
TRUE, center = FALSE)
summary(lmp.model.2)
lm.model.2 <- lm(y2 ~ x2*factor.levels2 - 1)
summary(lm.model.2)

Here are the results:
> summary(lmp.model.2)
Call:
lmp(formula = y2 ~ x2 * factor.levels2 - 1, perm = "", seqs = TRUE,
center = FALSE)
Residuals:
   Min 1Q Median 3QMax
-1.284e-13 -6.772e-16  1.439e-16  1.581e-15  4.323e-14
Coefficients:
 Estimate Std. Errort value Pr(>|t|)
factor.levels2A 1.000e+01  5.545e-15  1.803e+15  < 2e-16 ***
factor.levels2B 3.000e+01  4.707e-15  6.373e+15  < 2e-16 ***
factor.levels2C 1.556e-15  4.994e-15  3.120e-01 0.755688
x2  6.840e-17  1.860e-17  3.677e+00 0.000314 ***
x2:factor.levels21  1.030e-16  2.734e-17  3.767e+00 0.000226 ***
x2:factor.levels22

Re: [R] Appropriateness of R functions for multicore

[Prof Brian Ripley]

On 20/08/2013 03:13, Hopkins, Bill wrote:

I wrap functions to run via multicore with tryCatch() to gather stats on 
failure rate and capture state.



I'm still interested in how/whether core fuctions were verified as being 
threadsafe.


What does 'threads' have to do with this?  Multicore forks processes, 
not spawns threads.  And the manuals contain advice about what is known 
not to be safe usage in that case -- it is not the core functions but 
what you do with them that matters.


As for threads: see the manuals e.g. 
http://cran.r-project.org/doc/manuals/r-release/R-exts.html#OpenMP-support .






Bill Hopkins



Written using a virtual Android keyboard...

-- Original message --
From: Jeff Newmiller
Date: 8/19/2013 5:18 PM
To: Patrick Connolly;
Cc: Hopkins, Bill;r-help@R-project.org;
Subject:Re: [R] Appropriateness of R functions for multicore

I don't know... I suppose it depends how it fails. I recommend that you 
restrict yourself to using only the data that was passed as parameters to your 
parallel function. You may be able to tackle parts of the task and return only 
those partial results to confirm how far through the code you can get.
---
Jeff NewmillerThe .   .  Go Live...
DCN:Basics: ##.#.   ##.#.  Live Go...
   Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/BatteriesO.O#.   #.O#.  with
/Software/Embedded Controllers)   .OO#.   .OO#.  rocks...1k
---
Sent from my phone. Please excuse my brevity.

Patrick Connolly  wrote:

On Sat, 17-Aug-2013 at 05:09PM -0700, Jeff Newmiller wrote:


|> In most threaded multitasking environments it is not safe to
|> perform IO in multiple threads. In general you will have difficulty
|> performing IO in parallel processing so it is best to let the
|> master hand out data to worker tasks and gather results from them
|> for storage. Keep in mind that just because you have eight cores
|> for processing doesn't mean you have eight hard disks, so if your
|> problem is IO bound in single processor operation then it will also
|> be IO bound in threaded operation.

For tasks which don't involve I/O but fail with mclapply, how does one
work out where the problem is?  The handy browser() function which
allows for interactive diagnosis won't work with parallel jobs.

What other approaches can one use?

Thanx




---



|> Jeff NewmillerThe .   .  Go
Live...
|> DCN:Basics: ##.#.   ##.#.
Live Go...
|>   Live:   OO#.. Dead: OO#..
Playing
|> Research Engineer (Solar/BatteriesO.O#.   #.O#.
with
|> /Software/Embedded Controllers)   .OO#.   .OO#.
rocks...1k
|>
---

|> Sent from my phone. Please excuse my brevity.
|>
|> "Hopkins, Bill"  wrote:
|> >Has there been any systematic evaluation of which core R functions
are
|> >safe for use with multicore? Of current interest, I have tried
calling
|> >read.table() via mclapply() to more quickly read in hundreds of raw
|> >data files (I have a 24 core system with 72 GB running Ubuntu, a
|> >perfect platform for multicore). There was a 40% failure rate,
which
|> >doesn't occur when I invoke read.table() serially from within a
single
|> >thread. Another example was using pvec() to invoke
|> >sapply(strsplit(),...) on a huge character vector (to pull out
fields
|> >from within a field). It looked like a perfect application for
pvec(),
|> >but it fails when serial execution works.
|> >
|> >I thought I'd ask before taking on the task of digging into the
|> >underlying code to see what is might be causing failure in a
multicore
|> >(well, multi-threaded) context.
|> >
|> >As an alternative, I could define multiple cluster nodes locally,
but
|> >that shifts the tradeoff a bit in whether parallel execution is
|> >advantageous - the overhead is significantly more, and even with 72
GB,
|> >it does impose greater limits on how many cores can be used.
|> >
|> >Bill Hopkins
|> >
|> >__
|> >R-help@r-project.org mailing list
|> >https://stat.ethz.ch/mailman/listinfo/r-help
|> >PLEASE do read the posting guide
|> >http://www.R-project.org/posting-guide.html
|> >and provide commented, minimal, self-contained, reproducible code.
|>
|> __
|> R-help@r-project.org mailing list
|> https://stat.ethz.ch/mailman/listinfo/r-help
|> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
|> and provide commented, minimal, self-contained, reproducible code.


__
R-help@r