Probably because you inadvertently ran different models. Without your code,
I haven't a clue.
On Sat, Aug 10, 2024, 12:29 Yuan Chun Ding <ycd...@coh.org> wrote:
> HI Bert and Ben,
>
>
>
> Yes, running lm.fit using the matrix format is much faster. I read a
> couple of online comments why it is faster.
>
>
>
> However, the residual values for three tested variables or genes from lm
> function and lm.fit function are different, with Pearson correlation of
> 0.55, 0.89, and 0.99.
>
>
>
> I have not found the reason.
>
>
>
> Thanks,
>
>
> Ding
>
>
>
> *From:* Bert Gunter <bgunter.4...@gmail.com>
> *Sent:* Friday, August 9, 2024 7:11 PM
> *To:* Ben Bolker <bbol...@gmail.com>
> *Cc:* Yuan Chun Ding <ycd...@coh.org>; r-help@r-project.org
> *Subject:* Re: [R] a fast way to do my job
>
>
>
> Better idea, Ben! It would work as you might expect it to to produce the
> same results as the above: ##first make sure your regressor is a matrix:
> pur2 <- matrix(purity2, ncol =1) ## convert the data frame variables into a
> matrix dat <-
>
> Better idea, Ben!
>
>
>
> It would work as you might expect it to to produce the same results as
>
> the above:
>
>
>
> ##first make sure your regressor is a matrix:
>
> pur2 <- matrix(purity2, ncol =1)
>
> ## convert the data frame variables into a matrix
>
> dat <- as.matrix(gem751be.rpkm[ , 74:35164])
>
> ##then
>
> result <- residuals(lm.fit( x= pur2, y = dat))
>
>
>
> Cheers,
>
> Bert
>
>
>
> On Fri, Aug 9, 2024 at 6:38 PM Ben Bolker <bbol...@gmail.com> wrote:
>
> >
>
> > You can also fit a linear model with a matrix-valued response
>
> > variable, which should be even faster (not sure off the top of my head
>
> > how to get the residuals and reshape them to the dimensions you want)
>
> >
>
> > On Fri, Aug 9, 2024 at 9:31 PM Bert Gunter <bgunter.4...@gmail.com> wrote:
>
> > >
>
> > > See ?lm.fit.
>
> > > I must be missing something, because:
>
> > >
>
> > > results <- sapply(74:35164, \(i) residuals(lm.fit(purity2,
>
> > > gem751be.rpkm[, i] )))
>
> > >
>
> > > would give you a 751 x 35091 matrix of the residuals from each of the
>
> > > regressions.
>
> > > I assume it will be considerably faster than all the overhead you are
>
> > > carrying in your current code, but of course you'll have to try it and
>
> > > see. ... Assuming that I have interpreted your request correctly.
>
> > > Ignore if not.
>
> > >
>
> > > Cheers,
>
> > > Bert
>
> > >
>
> > > On Fri, Aug 9, 2024 at 4:50 PM Yuan Chun Ding via R-help
>
> > > <r-help@r-project.org> wrote:
>
> > > >
>
> > > > Dear R users,
>
> > > >
>
> > > > I am running the following code below, the gem751be.rpkm is a
> > > > dataframe with dim of 751 samples by 35164 variables, 73 phenotypic
> > > > variables in the furst to 73rd column and 35091 genomic variables or
> > > > genes in the 74th to 35164th columns. What I need to do is to
> > > > calculate the residuals for each gene using the simple linear
> > > > regression model of genelist[i] ~ purity2;
>
> > > >
>
> > > > The following code is running, it takes long time, but I have an
> > > > expensive ThinkStation window computer.
>
> > > > Can you provide a fast way to do it?
>
> > > >
>
> > > > Thank you,
>
> > > >
>
> > > > Ding
>
> > > >
>
> > > > ---------------------------------------------------------------------------------
>
> > > >
>
> > > >
>
> > > > gem751be.rpkm <-merge(gem751be10, as.data.frame(t(rna849.fpkm2)),
>
> > > > + by.x="id2",by.y=0)
>
> > > > > row.names(gem751be.rpkm)<-gem751be.rpkm$id3
>
> > > > >
> > > > > colnames(gem751be.rpkm)<-gsub(colnames(gem751be.rpkm),pattern="-",replacement="_")
>
> > > > > genelist <- gem751be.rpkm %>% dplyr::select(74:35164)
>
> > > > > residuals <- NULL
>
> > > > > for (i in 1:length(genelist)) {
>
> > > > + #i=1
>
> > > > + formula <- reformulate("purity2", response=names(genelist)[i])
>
> > > > + model <- lm(formula, data = gem751be.rpkm)
>
> > > > + resi <- as.data.frame(residuals(model))
>
> > > > + colnames(resi)[1]<-names(genelist)[i]
>
> > > > + resi <-as.data.frame(t(resi))
>
> > > > + residuals <- rbind(residuals, resi)
>
> > > > + }
>
> > > >
>
> > > >
>
> > > >
>
> > > > ----------------------------------------------------------------------
>
> > > > ------------------------------------------------------------
>
> > > > -SECURITY/CONFIDENTIALITY WARNING-
>
> > > >
>
> > > > This message and any attachments are intended solely for the individual
> > > > or entity to which they are addressed. This communication may contain
> > > > information that is privileged, confidential, or exempt from disclosure
> > > > under applicable law (e.g., personal health information, research data,
> > > > financial information). Because this e-mail has been sent without
> > > > encryption, individuals other than the intended recipient may be able
> > > > to view the information, forward it to others or tamper with the
> > > > information without the knowledge or consent of the sender. If you are
> > > > not the intended recipient, or the employee or person responsible for
> > > > delivering the message to the intended recipient, any dissemination,
> > > > distribution or copying of the communication is strictly prohibited. If
> > > > you received the communication in error, please notify the sender
> > > > immediately by replying to this message and deleting the message and
> > > > any accompanying files from your system. If, due to the security risks,
> > > > you do not wish to rec
>
> > > > eive further communications via e-mail, please reply to this message
> > > > and inform the sender that you do not wish to receive further e-mail
> > > > from the sender. (LCP301)
>
> > > > ------------------------------------------------------------
>
> > > >
>
> > > > [[alternative HTML version deleted]]
>
> > > >
>
> > > > ______________________________________________
>
> > > > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>
> > > > https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX72PW30DQ$
> > > >
> > > > <https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX72PW30DQ$>
>
> > > > PLEASE do read the posting guide
> > > > https://urldefense.com/v3/__http://www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX66rfmKvA$
> > > >
> > > > <https://urldefense.com/v3/__http:/www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX66rfmKvA$>
>
> > > > and provide commented, minimal, self-contained, reproducible code.
>
> > >
>
> > > ______________________________________________
>
> > > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>
> > > https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX72PW30DQ$
> > >
> > > <https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-help__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX72PW30DQ$>
>
> > > PLEASE do read the posting guide
> > > https://urldefense.com/v3/__http://www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX66rfmKvA$
> > >
> > > <https://urldefense.com/v3/__http:/www.R-project.org/posting-guide.html__;!!Fou38LsQmgU!qcJ4z-vlMNzsa8XCsJUcuPOz8Vt12zsV_XaWpqXsyUYJBTlcNRonFPr7w7Ql3xqcDnZ9ZYC8JX66rfmKvA$>
>
> > > and provide commented, minimal, self-contained, reproducible code.
>
>
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
