Thank you all for your input. This is an example of one data file (I have 74 data files):
2.90546E+11, threat, 1, 2, 1, 2, 1, death, stove, NA, NA, 205, 0, 394 2.90546E+11, threat, 2, 2, 2, 1, 1, emaciated, shortened, NA, NA, 205, 0, 502 2.90546E+11, threat, 3, 1, 1, 1, 2, mutilate, consider, NA, NA, 205, 1, 468 2.90546E+11, threat, 6, 1, 2, 2, 1, weep, shop, NA, NA, 203, 1, 345 2.90546E+11, threat, 9, 2, 1, 2, 2, tormented, easygoing, NA, NA, 205, 1, 373 2.90546E+11, threat, 10, 1, 2, 2, 2, snake, table, NA, NA, 205, 1, 343 2.90546E+11, threat, 11, 2, 2, 1, 1, crisis, faucet, NA, NA, 203, 1, 437 2.90546E+11, threat, 12, 1, 1, 1, 1, victim, utensil, NA, NA, 203, 1, 343 2.90546E+11, threat, 14, 1, 2, 2, 1, depressed, repentant, NA, NA, 203, 1, 441 2.90546E+11, threat, 15, 2, 2, 1, 2, scum, shoe, NA, NA, 205, 1, 475 Column 13 has values of 0s and 1s which my cognitive task outputted. Column 14 is the reaction time (ms) data. I want to get rid of the rows that contain zeros so I thought I'd first replace zeros with NAs then use complete.cases function to get rid of the NAs. I also wanted to apply other functions so I included them all in a loop. All work fine except for the one where I try to turn the zeros to NAs. Jim when I tried your mockdata example, it worked fine. But when I translated it to my data, I still get zeros in the output. Can you identify any mistranslations I'm doing? txt.files<-list.files(".",pattern="dotprobe") #all my data files are text files in one folder for(tf in txt.files) { d<-read.table(tf) d[,13][d[,13]==0]<-NA #column 13 contains zeros d<-d[ ,-c(10,11)] #get rid of columns 10 and 11 write.table(d,sub("[.]",".tlbs.",tf),quote=FALSE, row.names=FALSE) } That's an example of one of the output I get: V1 V2 V3 V4 V5 V6 V7 V8 V9 V12 V13 V14 2.90546E+11, threat, 1, 2, 1, 2, 1, death, stove, 205, 0, 394 2.90546E+11, threat, 2, 2, 2, 1, 1, emaciated, shortened, 205, 0, 502 2.90546E+11, threat, 3, 1, 1, 1, 2, mutilate, consider, 205, 1, 468 2.90546E+11, threat, 6, 1, 2, 2, 1, weep, shop, 203, 1, 345 2.90546E+11, threat, 9, 2, 1, 2, 2, tormented, easygoing, 205, 1, 373 2.90546E+11, threat, 10, 1, 2, 2, 2, snake, table, 205, 1, 343 Columns 10 and 11 were deleted. But zeros were not replaced by NAs. After all the data cleaning, the functions I'm interested in including in the loop are: get_tlbs and summarize_bias (and these also work fine in my loop). Thanks again 🙂 Sincerely Helen ________________________________ From: Jim Lemon <drjimle...@gmail.com> Sent: Tuesday, April 21, 2020 2:52 AM To: Rui Barradas <ruipbarra...@sapo.pt> Cc: Helen Sawaya <helensaw...@hotmail.com>; Michael Dewey <li...@dewey.myzen.co.uk>; r-help@R-project.org <r-help@r-project.org> Subject: Re: [R] NA command in a 'for' loop Hi Helen, Your problem may lie in using row.names=TRUE. I was puzzled when an extra column kept popping up in the output files. For reading in and replacing zeros with NAs, this seems to work: for(mockdata in 1:3) { mdf<-data.frame(sample(2:20,10),sample(2:20,10),sample(0:1,10,TRUE)) write.table(mdf,file=paste0("threat",mockdata,".txt"),quote=FALSE, row.names=FALSE,col.names=FALSE) } txt.files<-list.files(".",pattern="threat[1-3]") for(tf in txt.files) { d<-read.table(tf) d[,3][d[,3]==0]<-NA write.table(d,sub("[.]",".tbls.",tf),quote=FALSE,row.names=FALSE) } Jim On Tue, Apr 21, 2020 at 7:57 AM Rui Barradas <ruipbarra...@sapo.pt> wrote: > > Hello, > > I believe the only way we have to see what is happening is for you to > post the output of > > > dput(head(d, 20)) # or 30 > > > or, with d2 a subset of d that includes zeros, > > > dput(head(d2, 20)) > > > Hope this helps, > > Rui Barradas > > Às 17:48 de 20/04/20, Helen Sawaya escreveu: > > I have one column that represents correct response versus error (correct > > is coded as 1 and error is coded as 0). Nowhere else in the dataset are > > there values of 0. The vector is treated as an integer. > > ------------------------------------------------------------------------ > > *From:* Michael Dewey <li...@dewey.myzen.co.uk> > > *Sent:* Monday, April 20, 2020 7:35 PM > > *To:* Helen Sawaya <helensaw...@hotmail.com>; Rui Barradas > > <ruipbarra...@sapo.pt>; r-help@R-project.org <r-help@R-project.org> > > *Subject:* Re: [R] NA command in a 'for' loop > > Just a thought Helen but is x being treated as a real and what you think > > are zero and are printed as zero are in fact some very small number? If > > so you need to alter your test appropriately. > > > > Michael > > > > On 20/04/2020 17:25, Helen Sawaya wrote: > >> Thank you for your reply. > >> > >> I tried d[] <- lapply(d, function(x) {is.na(x) <- x == 0; x}) > >> but I am still getting zeros instead of NAs in my output.. > >> > >> I wonder if the problem is that some of my data files don't have any zeros > >> (participants made no errors).. > >> ________________________________ > >> From: Rui Barradas <ruipbarra...@sapo.pt> > >> Sent: Monday, April 20, 2020 9:05 AM > >> To: Helen Sawaya <helensaw...@hotmail.com>; r-help@R-project.org > >> <r-help@R-project.org> > >> Subject: Re: [R] NA command in a 'for' loop > >> > >> Hello, > >> > >> Instead of > >> > >> d[d == 0] <- NA > >> > >> try > >> > >> d[] <- lapply(d, function(x) {is.na(x) <- x == 0; x}) > >> > >> > >> Also, in the first for loop > >> > >> paste(i, sep = "") > >> > >> does nothing, it's the same as i. > >> And the same for > >> > >> (d2$V4 == 1) == TRUE > >> > >> Since (d2$V4 == 1) already is FALSE/TRUE there is no need for > >> > >> (.) == TRUE > >> > >> > >> Hope this helps, > >> > >> Rui Barradas > >> > >> > >> > >> Às 20:52 de 19/04/20, Helen Sawaya escreveu: > >>> Dear R experts, > >>> > >>> I am using a 'for' loop to apply commands to multiple datasets (each file > >>> is one participant). The only one not working is the command that > >>> identifies zeros in my datasets and changes them to NAs. But when I look > >>> at the output, zeros ("0") are still present. Surprisingly, the > >>> functions work fine when I apply them to a single > > dataset (outside the loop). I've tried: > >>> > >>> all.files <- list.files(".") > >>> txt.files <- grep("threat.txt",all.files,value=T) > >>> > >>> for(i in txt.files){ > >>> d <- read.table(paste(i,sep=""),header=F) > >>> d[d==0] <- NA #replace zeros with NA > >>> write.table(d, paste0(i,".tlbs.txt"), quote=FALSE, row.names=TRUE)} > >>> d<-d[ ,-c(10,11)] > >>> d2<-d[complete.cases(d), ] > >>> d2$V4<-as.numeric(d2$V4) > >>> congruent <- (d2$V4 == 1) == TRUE > >>> x <- get_tlbs(d2$V14, congruent, prior_weights = NULL, method = > >>> "weighted", fill_gaps = FALSE) > >>> write.table(x, paste0(i,".tlbs.txt"), quote=FALSE, row.names=TRUE)} > >>> > >>> I've also tried: > >>> > >>> for(i in txt.files){ > >>> d <- read.table(paste(i,sep=""),header=F) > >>> if (0 %in% d) > >>> {replace_with_na(d,replace = list(x = 0))} # replace zeros with NA > >>> d<-d[ ,-c(10,11)] > >>> d2<-d[complete.cases(d), ] > >>> d2$V4<-as.numeric(d2$V4) > >>> congruent <- (d2$V4 == 1) == TRUE > >>> x <- get_tlbs(d2$V14, congruent, prior_weights = NULL, method = > >>> "weighted", fill_gaps = FALSE) > >>> write.table(x, paste0(i,".summaryoutput.txt"), quote=FALSE, > >>> row.names=TRUE)} > >>> > >>> Thank you for your help. > >>> Sincerely > >>> Helen > >>> > >>> [[alternative HTML version deleted]] > >>> > >>> ______________________________________________ > >>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > >>> https://stat.ethz.ch/mailman/listinfo/r-help > >>> PLEASE do read the posting guide > >>> http://www.R-project.org/posting-guide.html > >>> and provide commented, minimal, self-contained, reproducible code. > >>> > >> > >> [[alternative HTML version deleted]] > >> > >> ______________________________________________ > >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > >> https://stat.ethz.ch/mailman/listinfo/r-help > >> PLEASE do read the posting guide > >> http://www.R-project.org/posting-guide.html > >> and provide commented, minimal, self-contained, reproducible code. > >> > >> > > > > -- > > Michael > > http://www.dewey.myzen.co.uk/home.html > > ______________________________________________ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.