Thanks to all those who have replied to my query. I have decided do a thorough
reading on this subject and try to seek out a proper solution. I will stick to
the nnet package as mentioned by Jude and try and compare results with other
neural network software if possible.
Regards,
Indrajit
________________________________
From: "jude.r...@ubs.com" <jude.r...@ubs.com>
Sent: Thursday, May 28, 2009 10:49:36 PM
Subject: Re: [R] Neural Network resource
The package AMORE appears to be more flexible, but I got very poor results
using it when I tried to improve the predictive accuracy of a regression model.
I don't understand all the options well enough to be able to fine tune it to
get better predictions. However, using the nnet() function in package VR gave
me decent results and is pretty easy to use (see the Venables and Ripley book,
Modern Applied Statistics with S, pages 243 to 249, for more details). I tried
using package neuralnet as well but the neural net failed to converge. I could
not figure out how to set the threshold option (or other options) to get the
neural net to converge. I explored package neural as well. Of all these 4
packages, the nnet() function in package VR worked the best for me.
As another R user commented as well, you have too many hidden layers and too
many neurons. In general you do not need more than 1 hidden layer. One hidden
layer is sufficient for the "universal approximator" property of neural
networks to hold true. As you keep adding neurons to the one hidden layer, the
problem becomes more and more non-linear. If you add too many neurons you will
overfit. In general, you do not need to add more than 10 neurons. The
activation function in the hidden layer of Venables and Ripley's nnet()
function is logistic, and you can specify the activation function in the output
layer to be linear using linout = T in nnet(). Using one hidden layer, and
starting with one hidden neuron and working up to 10 hidden neurons, I built
several neural nets (4,000 records) and computed the training MSE. I also
computed the validation MSE on a holdout sample of over 1,000 records. I also
started with 2 variables and worked up to 15 variables in
a "for" loop, so in all, I built 140 neural nets using 2 "for" loops, and
stored the results in lists. I arranged my variables in the data frame based on
correlations and partial correlations so that I could easily add variables in a
"for" loop. This was my "crude" attempt to simulate variable selection since,
from what I have seen, neural networks do not have variable selection methods.
In my particular case, neural networks gave me marginally better results than
regression. It all depends on the problem. If the data has non-linear patterns,
neural networks will be better than linear regression.
My code is below. You can modify it to suit your needs if you find it useful.
There are probably lines in the code that are redundant which can be deleted.
HTH.
Jude Ryan
My code:
# set order in data frame train2 based on correlations and partial correlations
train2 <- train[, c(5,27,19,20,25,26,4,9,3,10,16,6,2,14,21,28)]
dim(train2)
names(train2)
library(nnet)
# skip = T
# train 10 neural networks in a loop and find the one with the minimum test and
validation error
# create various lists to store the results of the neural network running in
two for loops
# The Column List is for the outer for loop, which loops over variables
# The Row List is for the inner for loop, which loops over number of neurons in
the hidden layer
col_nn <- list() # stores the results of nnet() over variables - outer loop
row_nn <- list() # stores the results of nnet() over neurons - inner loop
col_mse <- list()
# row_mse <- list() # not needed because nn.mse is a data frame with rows
col_sum <- list()
row_sum <- list()
col_vars <- list()
row_vars <- list()
col_wts <- list()
row_wts <- list()
df_dim <- dim(train2)
df_dim[2] # number of variables
df_dim[2] - 1
num_of_neurons <- 10
# build data frame to store results of neural net for each run
nn.mse <- data.frame(Train_MSE=seq(1:num_of_neurons),
Valid_MSE=seq(1:num_of_neurons))
# open log file and redirect output to log file
sink("D:\\XXX\\YYY\\ Programs\\Neural_Network_v8_VR_log.txt")
# outer loop - loop over variables
for (i in 3:df_dim[2]) { # df_dim[2]
# inner loop - loop over number of hidden neurons
for (j in 1:num_of_neurons) { # upto 10 neurons in the hidden layer
# need to create a new data frame with just the predictor/input variables
needed
train3 <- train2[,c(1:i)]
coreaff.nn <- nnet(dep_var ~ ., train3, size = j, decay = 1e-3, linout = T,
skip = T, maxit = 1000, Hess = T)
# row_vars[[j]] <- coreaff.nn$call # not what we want
# row_vars[[j]] <- names(train3)[c(2:i)] # not needed in inner loop - same
number of variables for all neurons
row_sum[[j]] <- summary(coreaff.nn)
row_wts[[j]] <- coreaff.nn$wts
rownames(nn.mse)[j] <- paste("H", j, sep="")
nn.mse[j, "Train_MSE"] <- mean((train3$dep_var - predict(coreaff.nn))^2)
nn.mse[j, "Valid_MSE"] <- mean((valid$dep_var - predict(coreaff.nn,
valid))^2)
}
col_vars[[i-2]] <- names(train3)[c(2:i)]
col_sum[[i-2]] <- row_sum
col_wts[[i-2]] <- row_wts
col_mse[[i-2]] <- nn.mse
}
# cbind(col_vars[1],col_vars[2])
col_vars
col_sum
col_wts
sink()
cbind(col_mse[[1]],col_mse[[2]],col_mse[[3]],col_mse[[4]],col_mse[[5]],col_mse[[6]],col_mse[[7]],
col_mse[[8]],col_mse[[9]],col_mse[[10]],col_mse[[11]],col_mse[[12]],col_mse[[13]],col_mse[[14]])
# build a neural network with the same 9 variables as in regression
pred <- list()
summaries <- list()
wts <- list()
Indrajit wrote:
You are right there is a pdf file which describes the function. But let tell
you where I am coming from.
Just to test if a neural network will work better than a ordinary least square
regression, I created a dataset with one dependent variable and 6 other
independent variables. Now I had deliberately created the dataset in such
manner that we have an excellent regression model. Eg: Y = b0 + b1*x1 + b2*x2 +
b3*x3.. + b6*x6 + e
where e is normal random variable. Naturally any statistical analysis system
running regression would easily predict the values of b1, b2, b3, ..., b6 with
around 30-40 observations.
I fed this data into a Neural network (3 hidden layers with 6 neurons in each
layer) and trained the network. When I passed the input dataset and tried to
get the predictions, all the predicted values were identical! This confused me
a bit and was wondering whether my understanding of the Neural Network was
wrong.
Have you ever faced anything like it?
Regards,
Indrajit
___________________________________________
Jude Ryan
Director, Client Analytical Services
Strategy & Business Development
UBS Financial Services Inc.
1200 Harbor Boulevard , 4th Floor
Weehawken , NJ 07086-6791
Tel. 201-352-1935
Fax 201-272-2914
Email: jude.r...@ubs.com
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
