Re: [R] read txt file - date - no space
Hi
I did not get through all answers you already got and you probably obtained
similar advice as mine.
# read data (if you have csv file just use read.csv)
> test<-read.table("clipboard", header=T, sep=",")
# control your object(s)
> str(test)
'data.frame': 8 obs. of 4 variables:
$ date: Factor w/ 8 levels "10/1/1998 0:00",..: 1 2 3 4 5 6 7 8
$ str1: num 0.6 0.2 0.6 0 0 0 0 0.2
$ str2: num 0 0.2 0.2 0 0 0 0 0
$ str3: num 0 0.2 0.4 0.6 0 0 0 0
#if it is OK change first column to real date by POSIXct
test$date<-as.POSIXct(test$date, format="%d/%m/%Y %H:%M")
#attach lubridate
> library(lubridate)
# aggregate your object(s) and use lubridate function
> aggregate(test[,-1], list(day(test$date)), mean)
Group.1 str1 str2 str3
1 10 0.2 0.05 0.15
# or format function
> aggregate(test[,-1], list(format(test$date, "%Y-%m-%d")), mean)
Group.1 str1 str2 str3
1 1998-01-10 0.2 0.05 0.15
If it does not work with your data you should post at least result of
str(yourdata)
or preferably
dput(yourdata[1:20,])
Cheers
Petr
> -Original Message-
> From: R-help On Behalf Of Diego Avesani
> Sent: Wednesday, August 1, 2018 8:55 AM
> To: Jeff Newmiller
> Cc: r-help mailing list
> Subject: Re: [R] read txt file - date - no space
>
> Dear all,
> I am sorry, I did a lot of confusion. I am sorry, I have to relax and stat
> all again
> in order to understand.
> If I could I would like to start again, without mixing strategy and waiting
> for
> your advice.
>
> I am really appreciate you help, really really.
> Here my new file, a *.csv file (buy the way, it is possible to attach it in
> the
> mailing list?)
>
> date,str1,str2,str3
> 10/1/1998 0:00,0.6,0,0
> 10/1/1998 1:00,0.2,0.2,0.2
> 10/1/1998 2:00,0.6,0.2,0.4
> 10/1/1998 3:00,0,0,0.6
> 10/1/1998 4:00,0,0,0
> 10/1/1998 5:00,0,0,0
> 10/1/1998 6:00,0,0,0
> 10/1/1998 7:00,0.2,0,0
>
>
> I read it as:
> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>
> at this point I would like to have the daily mean.
> What would you suggest?
>
> Really Really thanks,
> You are my lifesaver
>
> Thanks
>
>
>
> Diego
>
>
> On 1 August 2018 at 01:01, Jeff Newmiller wrote:
>
> > ... and the most common source of NA values in time data is wrong
> > timezones. You really need to make sure the timezone that is assumed
> > when the character data are converted to POSIXt agrees with the data.
> > In most cases the easiest way to insure this is to use
> >
> > Sys.setenv(TZ="US/Pacific")
> >
> > or whatever timezone from
> >
> > OlsonNames()
> >
> > corresponds with your data. Execute this setenv function before the
> > strptime or as.POSIXct() function call.
> >
> > You can use
> >
> > MyData[ is.na(MyData$datetime), ]
> >
> > to see which records are failing to convert time.
> >
> > [1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1
> >
> > On July 31, 2018 3:04:05 PM PDT, Jim Lemon
> wrote:
> > >Hi Diego,
> > >I think the error is due to NA values in your data file. If I extend
> > >your example and run it, I get no errors:
> > >
> > >MyData<-read.table(text="103001930 103001580 103001530
> > >1998-10-01 00:00:00 0.6 0 0
> > >1998-10-01 01:00:00 0.2 0.2 0.2
> > >1998-10-01 02:00:00 0.6 0.2 0.4
> > >1998-10-01 03:00:00 0 0 0.6
> > >1998-10-01 04:00:00 0 0 0
> > >1998-10-01 05:00:00 0 0 0
> > >1998-10-01 06:00:00 0 0 0
> > >1998-10-01 07:00:00 0.2 0 0
> > >1998-10-01 08:00:00 0.6 0 0
> > >1998-10-01 09:00:00 0.2 0.2 0.2
> > >1998-10-01 10:00:00 0.6 0.2 0.4
> > >1998-10-01 11:00:00 0 0 0.6
> > >1998-10-01 12:00:00 0 0 0
> > >1998-10-01 13:00:00 0 0 0
> > >1998-10-01 14:00:00 0 0 0
> > >1998-10-01 15:00:00 0.2 0 0
> > >1998-10-01 16:00:00 0.6 0 0
> > >1998-10-01 17:00:00 0.2 0.2 0.2
> > >1998-10-01 18:00:00 0.6 0.2 0.4
> > >1998-10-01 19:00:00 0 0 0.6
> > >1998-10-01 20:00:00 0 0 0
> > >1998-10-01 21:00:00 0 0 0
> > >1998-10-01 22:00:00 0 0 0
> > >1998-10-01 23:00:00 0.2 0 0
> > >1998-10-02 00:00:00 0.6 0 0
> > >1998-10-02 01:00:00 0.2 0.2 0.2
> > >1998-10-02 02:00:00 0.6 0.2 0.4
> > >1998-10-02 03:00:00 0 0 0.6
> > >1998-10-02 04:00:00 0 0 0
> > >1998-10-02 05:00:00 0 0 0
> > >1998-10-02 06:00:00 0 0 0
> > >1998-10-02 07:00:00 0.2 0 0
> > >1998-10-02 08:00:00 0.6 0 0
> > >1998-10-02 09:00:00 0.2 0.2 0.2
> > >1998-10-02 10:00:00 0.6 0.2 0.4
> > >1998-10-02 11:00:00 0 0 0.6
> > >1998-10-02 12:00:00 0 0 0
> > >1998-10-02 13:00:00 0 0 0
> > >1998-10-02 14:00:00 0 0 0
> > >1998-10-02 15:00:00 0.2 0 0
> > >1998-10-02 16:00:00 0.6 0 0
> > >1998-10-02 17:00:00 0.2 0.2 0.2
> > >1998-10-02 18:00:00 0.6 0.2 0.4
> > >1998-10-02 19:00:00 0 0 0.6
> > >1998-10-02 20:00:00 0 0 0
> > >1998-10-02 21:00:00 0 0 0
> > >1998-10-02 22:00:00 0 0 0
> > >1998-10-02 23:00:00 0.2 0 0",
> > >skip=1,stringsAsFactors=FALSE)
> > >names(MyData)<-c("date","time","st1","st2","st3")
> > >MyData$datetime<-strptime(paste(MyData$date,MyData$time),
> > > format="%Y-%m-%d %H:%M:%S")
> > >MyData$datetime
> > >st1_daily<-by(MyData$st1,MyData$date,mean)
> > >st2_daily<-by(
Re: [R] read txt file - date - no space
Dear Pikal, Deal all,
again really thank.
it seems not working.
Some specifications: My non data are -999, but I could change it.
My final results is:
11 -55.86242 -55.84764660 -277.4775
22 -55.47554 -94.58921682 -277.4845
33 -55.47095 -99.20239198 -277.4709
44 -55.46470 -55.45952932 -392.9071
55 -55.43335 -55.40171682 -388.4110
66 -55.40108 -55.37399691 -332.9068
77 -55.39201 -55.35156250 -332.8902
88 -110.87184 0.16136188 -281.8230
99 -110.95077 -55.63856096 -332.9564
10 10 -157.64430 -0.06602705 -315.3840
11 11 -105.06157 0.11507675 -315.4152
12 12 -70.08677 -52.54501096 -316.7247
So it is not correct.
For example for the first day in my csv I would have expected 0.167.
I am going to post what you have suggested:
for str(MyData)
'data.frame': 160008 obs. of 4 variables:
$ date: POSIXct, format: "1998-01-10 00:00:00" "1998-01-10 01:00:00"
"1998-01-10 02:00:00" "1998-01-10 03:00:00" ...
$ str1: num 0.6 0.2 0.6 0 0 0 0 0.2 0.6 0.2 ...
$ str2: num 0 0.2 0.2 0 0 0 0 0 0.2 0.4 ...
$ str3: num 0 0.2 0.4 0.6 0 0 0 0 0 0.4 ...
unfortunately, I am not able to post
dput(str[1:20,])
it gives me
Error in str[1:20, ] : object of type 'closure' is not subsettable
Thanks again,
I hope that what I posted could be enough in order to help me.
Diego
On 1 August 2018 at 13:58, PIKAL Petr wrote:
> Hi
>
> I did not get through all answers you already got and you probably
> obtained similar advice as mine.
>
> # read data (if you have csv file just use read.csv)
> > test<-read.table("clipboard", header=T, sep=",")
>
> # control your object(s)
> > str(test)
> 'data.frame': 8 obs. of 4 variables:
> $ date: Factor w/ 8 levels "10/1/1998 0:00",..: 1 2 3 4 5 6 7 8
> $ str1: num 0.6 0.2 0.6 0 0 0 0 0.2
> $ str2: num 0 0.2 0.2 0 0 0 0 0
> $ str3: num 0 0.2 0.4 0.6 0 0 0 0
>
> #if it is OK change first column to real date by POSIXct
> test$date<-as.POSIXct(test$date, format="%d/%m/%Y %H:%M")
>
> #attach lubridate
> > library(lubridate)
>
> # aggregate your object(s) and use lubridate function
>
> > aggregate(test[,-1], list(day(test$date)), mean)
> Group.1 str1 str2 str3
> 1 10 0.2 0.05 0.15
>
> # or format function
>
> > aggregate(test[,-1], list(format(test$date, "%Y-%m-%d")), mean)
> Group.1 str1 str2 str3
> 1 1998-01-10 0.2 0.05 0.15
>
> If it does not work with your data you should post at least result of
>
> str(yourdata)
>
> or preferably
>
> dput(yourdata[1:20,])
>
> Cheers
> Petr
>
>
> > -Original Message-
> > From: R-help On Behalf Of Diego Avesani
> > Sent: Wednesday, August 1, 2018 8:55 AM
> > To: Jeff Newmiller
> > Cc: r-help mailing list
> > Subject: Re: [R] read txt file - date - no space
> >
> > Dear all,
> > I am sorry, I did a lot of confusion. I am sorry, I have to relax and
> stat all again
> > in order to understand.
> > If I could I would like to start again, without mixing strategy and
> waiting for
> > your advice.
> >
> > I am really appreciate you help, really really.
> > Here my new file, a *.csv file (buy the way, it is possible to attach it
> in the
> > mailing list?)
> >
> > date,str1,str2,str3
> > 10/1/1998 0:00,0.6,0,0
> > 10/1/1998 1:00,0.2,0.2,0.2
> > 10/1/1998 2:00,0.6,0.2,0.4
> > 10/1/1998 3:00,0,0,0.6
> > 10/1/1998 4:00,0,0,0
> > 10/1/1998 5:00,0,0,0
> > 10/1/1998 6:00,0,0,0
> > 10/1/1998 7:00,0.2,0,0
> >
> >
> > I read it as:
> > MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
> >
> > at this point I would like to have the daily mean.
> > What would you suggest?
> >
> > Really Really thanks,
> > You are my lifesaver
> >
> > Thanks
> >
> >
> >
> > Diego
> >
> >
> > On 1 August 2018 at 01:01, Jeff Newmiller
> wrote:
> >
> > > ... and the most common source of NA values in time data is wrong
> > > timezones. You really need to make sure the timezone that is assumed
> > > when the character data are converted to POSIXt agrees with the data.
> > > In most cases the easiest way to insure this is to use
> > >
> > > Sys.setenv(TZ="US/Pacific")
> > >
> > > or whatever timezone from
> > >
> > > OlsonNames()
> > >
> > > corresponds with your data. Execute this setenv function before the
> > > strptime or as.POSIXct() function call.
> > >
> > > You can use
> > >
> > > MyData[ is.na(MyData$datetime), ]
> > >
> > > to see which records are failing to convert time.
> > >
> > > [1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1
> > >
> > > On July 31, 2018 3:04:05 PM PDT, Jim Lemon
> > wrote:
> > > >Hi Diego,
> > > >I think the error is due to NA values in your data file. If I extend
> > > >your example and run it, I get no errors:
> > > >
> > > >MyData<-read.table(text="103001930 103001580 103001530
> > > >1998-10-01 00:00:00 0.6 0 0
> > > >1998-10-01 01:00:00 0.2 0.2 0.2
> > > >1998-10-01 02:00:00 0.6 0.2 0.4
> > > >1998-10-01 03:00:00 0 0 0.6
> > > >1998-10-01 04:00:00 0 0 0
> > > >1998-10-01 05:00:00 0 0 0
> > > >
Re: [R] read txt file - date - no space
Dear Pikal, DEar all,
I do not if it could help:
if I print MyData%date, I get (at some point)
[281] "1998-12-10 16:00:00 CET" "1998-12-10 17:00:00 CET"
"1998-12-10 18:00:00 CET" "1998-12-10 19:00:00 CET"
[285] "1998-12-10 20:00:00 CET" "1998-12-10 21:00:00 CET"
"1998-12-10 22:00:00 CET" "1998-12-10 23:00:00 CET"
[289] NA NA NA
NA
[293] NA NA NA
NA
[297] NA NA NA
NA
Again Thanks
Diego
On 1 August 2018 at 14:29, Diego Avesani wrote:
> Dear Pikal, Deal all,
>
> again really thank.
>
> it seems not working.
> Some specifications: My non data are -999, but I could change it.
>
> My final results is:
>
> 11 -55.86242 -55.84764660 -277.4775
> 22 -55.47554 -94.58921682 -277.4845
> 33 -55.47095 -99.20239198 -277.4709
> 44 -55.46470 -55.45952932 -392.9071
> 55 -55.43335 -55.40171682 -388.4110
> 66 -55.40108 -55.37399691 -332.9068
> 77 -55.39201 -55.35156250 -332.8902
> 88 -110.87184 0.16136188 -281.8230
> 99 -110.95077 -55.63856096 -332.9564
> 10 10 -157.64430 -0.06602705 -315.3840
> 11 11 -105.06157 0.11507675 -315.4152
> 12 12 -70.08677 -52.54501096 -316.7247
>
>
> So it is not correct.
> For example for the first day in my csv I would have expected 0.167.
>
> I am going to post what you have suggested:
>
> for str(MyData)
>
> 'data.frame': 160008 obs. of 4 variables:
> $ date: POSIXct, format: "1998-01-10 00:00:00" "1998-01-10 01:00:00"
> "1998-01-10 02:00:00" "1998-01-10 03:00:00" ...
> $ str1: num 0.6 0.2 0.6 0 0 0 0 0.2 0.6 0.2 ...
> $ str2: num 0 0.2 0.2 0 0 0 0 0 0.2 0.4 ...
> $ str3: num 0 0.2 0.4 0.6 0 0 0 0 0 0.4 ...
>
> unfortunately, I am not able to post
>
> dput(str[1:20,])
>
> it gives me
>
> Error in str[1:20, ] : object of type 'closure' is not subsettable
>
>
> Thanks again,
> I hope that what I posted could be enough in order to help me.
>
>
>
> Diego
>
>
> On 1 August 2018 at 13:58, PIKAL Petr wrote:
>
>> Hi
>>
>> I did not get through all answers you already got and you probably
>> obtained similar advice as mine.
>>
>> # read data (if you have csv file just use read.csv)
>> > test<-read.table("clipboard", header=T, sep=",")
>>
>> # control your object(s)
>> > str(test)
>> 'data.frame': 8 obs. of 4 variables:
>> $ date: Factor w/ 8 levels "10/1/1998 0:00",..: 1 2 3 4 5 6 7 8
>> $ str1: num 0.6 0.2 0.6 0 0 0 0 0.2
>> $ str2: num 0 0.2 0.2 0 0 0 0 0
>> $ str3: num 0 0.2 0.4 0.6 0 0 0 0
>>
>> #if it is OK change first column to real date by POSIXct
>> test$date<-as.POSIXct(test$date, format="%d/%m/%Y %H:%M")
>>
>> #attach lubridate
>> > library(lubridate)
>>
>> # aggregate your object(s) and use lubridate function
>>
>> > aggregate(test[,-1], list(day(test$date)), mean)
>> Group.1 str1 str2 str3
>> 1 10 0.2 0.05 0.15
>>
>> # or format function
>>
>> > aggregate(test[,-1], list(format(test$date, "%Y-%m-%d")), mean)
>> Group.1 str1 str2 str3
>> 1 1998-01-10 0.2 0.05 0.15
>>
>> If it does not work with your data you should post at least result of
>>
>> str(yourdata)
>>
>> or preferably
>>
>> dput(yourdata[1:20,])
>>
>> Cheers
>> Petr
>>
>>
>> > -Original Message-
>> > From: R-help On Behalf Of Diego Avesani
>> > Sent: Wednesday, August 1, 2018 8:55 AM
>> > To: Jeff Newmiller
>> > Cc: r-help mailing list
>> > Subject: Re: [R] read txt file - date - no space
>> >
>> > Dear all,
>> > I am sorry, I did a lot of confusion. I am sorry, I have to relax and
>> stat all again
>> > in order to understand.
>> > If I could I would like to start again, without mixing strategy and
>> waiting for
>> > your advice.
>> >
>> > I am really appreciate you help, really really.
>> > Here my new file, a *.csv file (buy the way, it is possible to attach
>> it in the
>> > mailing list?)
>> >
>> > date,str1,str2,str3
>> > 10/1/1998 0:00,0.6,0,0
>> > 10/1/1998 1:00,0.2,0.2,0.2
>> > 10/1/1998 2:00,0.6,0.2,0.4
>> > 10/1/1998 3:00,0,0,0.6
>> > 10/1/1998 4:00,0,0,0
>> > 10/1/1998 5:00,0,0,0
>> > 10/1/1998 6:00,0,0,0
>> > 10/1/1998 7:00,0.2,0,0
>> >
>> >
>> > I read it as:
>> > MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>> >
>> > at this point I would like to have the daily mean.
>> > What would you suggest?
>> >
>> > Really Really thanks,
>> > You are my lifesaver
>> >
>> > Thanks
>> >
>> >
>> >
>> > Diego
>> >
>> >
>> > On 1 August 2018 at 01:01, Jeff Newmiller
>> wrote:
>> >
>> > > ... and the most common source of NA values in time data is wrong
>> > > timezones. You really need to make sure the timezone that is assumed
>> > > when the character data are converted to POSIXt agrees with the data.
>> > > In most cases the easiest way to insure this is to use
>> > >
>> > > Sys.setenv(TZ="US/Pacific")
>> > >
>> > > or whatever timezone from
>> > >
>> > > OlsonNa
Re: [R] read txt file - date - no space
Hi
see in line
From: Diego Avesani
Sent: Wednesday, August 1, 2018 2:30 PM
To: PIKAL Petr
Cc: r-help mailing list
Subject: Re: [R] read txt file - date - no space
Dear Pikal, Deal all,
again really thank.
it seems not working.
Some specifications: My non data are -999, but I could change it.
You must change it to NA. How the poor R should know that -999 is missing and
not a real value.
something like
Mydata[Mydata== -999] <- NA
should do it.
My final results is:
11 -55.86242 -55.84764660 -277.4775
22 -55.47554 -94.58921682 -277.4845
33 -55.47095 -99.20239198 -277.4709
44 -55.46470 -55.45952932 -392.9071
55 -55.43335 -55.40171682 -388.4110
66 -55.40108 -55.37399691 -332.9068
77 -55.39201 -55.35156250 -332.8902
88 -110.87184 0.16136188 -281.8230
99 -110.95077 -55.63856096 -332.9564
10 10 -157.64430 -0.06602705 -315.3840
11 11 -105.06157 0.11507675 -315.4152
12 12 -70.08677 -52.54501096 -316.7247
So it is not correct.
For example for the first day in my csv I would have expected 0.167.
I am going to post what you have suggested:
for str(MyData)
Your data seems to be OK, so after you change your -999 to NA everything should
be OK.
'data.frame': 160008 obs. of 4 variables:
$ date: POSIXct, format: "1998-01-10 00:00:00" "1998-01-10 01:00:00"
"1998-01-10 02:00:00" "1998-01-10 03:00:00" ...
$ str1: num 0.6 0.2 0.6 0 0 0 0 0.2 0.6 0.2 ...
$ str2: num 0 0.2 0.2 0 0 0 0 0 0.2 0.4 ...
$ str3: num 0 0.2 0.4 0.6 0 0 0 0 0 0.4 ...
unfortunately, I am not able to post
dput(str[1:20,])
it gives me
Error in str[1:20, ] : object of type 'closure' is not subsettable
dput(MyData[1:20,])
I would recommend you to spend some time reading R-intro which should be
located in doc/manual folder of your installation. It could help you in many
situations.
Cheers
Petr
Thanks again,
I hope that what I posted could be enough in order to help me.
Diego
On 1 August 2018 at 13:58, PIKAL Petr
mailto:petr.pi...@precheza.cz>> wrote:
Hi
I did not get through all answers you already got and you probably obtained
similar advice as mine.
# read data (if you have csv file just use read.csv)
> test<-read.table("clipboard", header=T, sep=",")
# control your object(s)
> str(test)
'data.frame': 8 obs. of 4 variables:
$ date: Factor w/ 8 levels "10/1/1998 0:00",..: 1 2 3 4 5 6 7 8
$ str1: num 0.6 0.2 0.6 0 0 0 0 0.2
$ str2: num 0 0.2 0.2 0 0 0 0 0
$ str3: num 0 0.2 0.4 0.6 0 0 0 0
#if it is OK change first column to real date by POSIXct
test$date<-as.POSIXct(test$date, format="%d/%m/%Y %H:%M")
#attach lubridate
> library(lubridate)
# aggregate your object(s) and use lubridate function
> aggregate(test[,-1], list(day(test$date)), mean)
Group.1 str1 str2 str3
1 10 0.2 0.05 0.15
# or format function
> aggregate(test[,-1], list(format(test$date, "%Y-%m-%d")), mean)
Group.1 str1 str2 str3
1 1998-01-10 0.2 0.05 0.15
If it does not work with your data you should post at least result of
str(yourdata)
or preferably
dput(yourdata[1:20,])
Cheers
Petr
> -Original Message-
> From: R-help
> mailto:r-help-boun...@r-project.org>> On Behalf
> Of Diego Avesani
> Sent: Wednesday, August 1, 2018 8:55 AM
> To: Jeff Newmiller mailto:jdnew...@dcn.davis.ca.us>>
> Cc: r-help mailing list mailto:r-help@r-project.org>>
> Subject: Re: [R] read txt file - date - no space
>
> Dear all,
> I am sorry, I did a lot of confusion. I am sorry, I have to relax and stat
> all again
> in order to understand.
> If I could I would like to start again, without mixing strategy and waiting
> for
> your advice.
>
> I am really appreciate you help, really really.
> Here my new file, a *.csv file (buy the way, it is possible to attach it in
> the
> mailing list?)
>
> date,str1,str2,str3
> 10/1/1998 0:00,0.6,0,0
> 10/1/1998 1:00,0.2,0.2,0.2
> 10/1/1998 2:00,0.6,0.2,0.4
> 10/1/1998 3:00,0,0,0.6
> 10/1/1998 4:00,0,0,0
> 10/1/1998 5:00,0,0,0
> 10/1/1998 6:00,0,0,0
> 10/1/1998 7:00,0.2,0,0
>
>
> I read it as:
> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>
> at this point I would like to have the daily mean.
> What would you suggest?
>
> Really Really thanks,
> You are my lifesaver
>
> Thanks
>
>
>
> Diego
>
>
> On 1 August 2018 at 01:01, Jeff Newmiller
> mailto:jdnew...@dcn.davis.ca.us>> wrote:
>
> > ... and the most common source of NA values in time data is wrong
> > timezones. You really need to make sure the timezone that is assumed
> > when the character data are converted to POSIXt agrees with the data.
> > In most cases the easiest way to insure this is to use
> >
> > Sys.setenv(TZ="US/Pacific")
> >
> > or whatever timezone from
> >
> > OlsonNames()
> >
> > corresponds with your data. Execute this setenv function before the
> > strptime or as.POSIXct() function call.
> >
> > You can use
> >
> > MyData[ is.na(MyData$datetime), ]
> >
> >
Re: [R] read txt file - date - no space
Try this:
> library(lubridate)
> library(tidyverse)
> input <- read.csv(text = "date,str1,str2,str3
+ 10/1/1998 0:00,0.6,0,0
+ 10/1/1998 1:00,0.2,0.2,0.2
+ 10/1/1998 2:00,0.6,0.2,0.4
+ 10/1/1998 3:00,0,0,0.6
+ 10/1/1998 4:00,0,0,0
+ 10/1/1998 5:00,0,0,0
+ 10/1/1998 6:00,0,0,0
+ 10/1/1998 7:00,0.2,0,0", as.is = TRUE)
> # convert the date and add the "day" so summarize
> input <- input %>%
+ mutate(date = mdy_hm(date),
+ day = floor_date(date, unit = 'day')
+ )
>
> by_day <- input %>%
+ group_by(day) %>%
+ summarise(m_s1 = mean(str1),
+ m_s2 = mean(str2),
+ m_s3 = mean(str3)
+ )
>
> by_day
# A tibble: 1 x 4
day m_s1 m_s2 m_s3
1 1998-10-01 00:00:00 0.200 0.0500 0.150
Jim Holtman
*Data Munger Guru*
*What is the problem that you are trying to solve?Tell me what you want to
do, not how you want to do it.*
On Tue, Jul 31, 2018 at 11:54 PM Diego Avesani
wrote:
> Dear all,
> I am sorry, I did a lot of confusion. I am sorry, I have to relax and stat
> all again in order to understand.
> If I could I would like to start again, without mixing strategy and waiting
> for your advice.
>
> I am really appreciate you help, really really.
> Here my new file, a *.csv file (buy the way, it is possible to attach it in
> the mailing list?)
>
> date,str1,str2,str3
> 10/1/1998 0:00,0.6,0,0
> 10/1/1998 1:00,0.2,0.2,0.2
> 10/1/1998 2:00,0.6,0.2,0.4
> 10/1/1998 3:00,0,0,0.6
> 10/1/1998 4:00,0,0,0
> 10/1/1998 5:00,0,0,0
> 10/1/1998 6:00,0,0,0
> 10/1/1998 7:00,0.2,0,0
>
>
> I read it as:
> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>
> at this point I would like to have the daily mean.
> What would you suggest?
>
> Really Really thanks,
> You are my lifesaver
>
> Thanks
>
>
>
> Diego
>
>
> On 1 August 2018 at 01:01, Jeff Newmiller
> wrote:
>
> > ... and the most common source of NA values in time data is wrong
> > timezones. You really need to make sure the timezone that is assumed when
> > the character data are converted to POSIXt agrees with the data. In most
> > cases the easiest way to insure this is to use
> >
> > Sys.setenv(TZ="US/Pacific")
> >
> > or whatever timezone from
> >
> > OlsonNames()
> >
> > corresponds with your data. Execute this setenv function before the
> > strptime or as.POSIXct() function call.
> >
> > You can use
> >
> > MyData[ is.na(MyData$datetime), ]
> >
> > to see which records are failing to convert time.
> >
> > [1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1
> >
> > On July 31, 2018 3:04:05 PM PDT, Jim Lemon wrote:
> > >Hi Diego,
> > >I think the error is due to NA values in your data file. If I extend
> > >your example and run it, I get no errors:
> > >
> > >MyData<-read.table(text="103001930 103001580 103001530
> > >1998-10-01 00:00:00 0.6 0 0
> > >1998-10-01 01:00:00 0.2 0.2 0.2
> > >1998-10-01 02:00:00 0.6 0.2 0.4
> > >1998-10-01 03:00:00 0 0 0.6
> > >1998-10-01 04:00:00 0 0 0
> > >1998-10-01 05:00:00 0 0 0
> > >1998-10-01 06:00:00 0 0 0
> > >1998-10-01 07:00:00 0.2 0 0
> > >1998-10-01 08:00:00 0.6 0 0
> > >1998-10-01 09:00:00 0.2 0.2 0.2
> > >1998-10-01 10:00:00 0.6 0.2 0.4
> > >1998-10-01 11:00:00 0 0 0.6
> > >1998-10-01 12:00:00 0 0 0
> > >1998-10-01 13:00:00 0 0 0
> > >1998-10-01 14:00:00 0 0 0
> > >1998-10-01 15:00:00 0.2 0 0
> > >1998-10-01 16:00:00 0.6 0 0
> > >1998-10-01 17:00:00 0.2 0.2 0.2
> > >1998-10-01 18:00:00 0.6 0.2 0.4
> > >1998-10-01 19:00:00 0 0 0.6
> > >1998-10-01 20:00:00 0 0 0
> > >1998-10-01 21:00:00 0 0 0
> > >1998-10-01 22:00:00 0 0 0
> > >1998-10-01 23:00:00 0.2 0 0
> > >1998-10-02 00:00:00 0.6 0 0
> > >1998-10-02 01:00:00 0.2 0.2 0.2
> > >1998-10-02 02:00:00 0.6 0.2 0.4
> > >1998-10-02 03:00:00 0 0 0.6
> > >1998-10-02 04:00:00 0 0 0
> > >1998-10-02 05:00:00 0 0 0
> > >1998-10-02 06:00:00 0 0 0
> > >1998-10-02 07:00:00 0.2 0 0
> > >1998-10-02 08:00:00 0.6 0 0
> > >1998-10-02 09:00:00 0.2 0.2 0.2
> > >1998-10-02 10:00:00 0.6 0.2 0.4
> > >1998-10-02 11:00:00 0 0 0.6
> > >1998-10-02 12:00:00 0 0 0
> > >1998-10-02 13:00:00 0 0 0
> > >1998-10-02 14:00:00 0 0 0
> > >1998-10-02 15:00:00 0.2 0 0
> > >1998-10-02 16:00:00 0.6 0 0
> > >1998-10-02 17:00:00 0.2 0.2 0.2
> > >1998-10-02 18:00:00 0.6 0.2 0.4
> > >1998-10-02 19:00:00 0 0 0.6
> > >1998-10-02 20:00:00 0 0 0
> > >1998-10-02 21:00:00 0 0 0
> > >1998-10-02 22:00:00 0 0 0
> > >1998-10-02 23:00:00 0.2 0 0",
> > >skip=1,stringsAsFactors=FALSE)
> > >names(MyData)<-c("date","time","st1","st2","st3")
> > >MyData$datetime<-strptime(paste(MyData$date,MyData$time),
> > > format="%Y-%m-%d %H:%M:%S")
> > >MyData$datetime
> > >st1_daily<-by(MyData$st1,MyData$date,mean)
> > >st2_daily<-by(MyData$st2,MyData$date,mean)
> > >st3_daily<-by(MyData$st3,MyData$date,mean)
> > >st1_daily
> > >st2_daily
> > >st3_daily
> > >
> > >Try adding na.rm=TRUE to the "by" calls:
> > >
> > >st1_daily<-by(
[R] New post for Rhelp
I have a database based on hourly data and I need to forecast next 24h of a single variable. I was thinking about applying an ARIMA model with some exogenous variables but I don't succeed to configure the hourly frequency, estimate ARIMA parameters, pdq ( exists some tests to check which parameters are better for the model?) and the structure of the model in its easy form because I would also like to introduce some seasonality form by analyzing some variables I highlighted some daily and weekly behaviours similar. I recognize that it could be quite difficult the problem but if you have also some useful links or some codes that can help me, please send me. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] New post for Rhelp
Statistics issues are generally off topic here; and we generally prefer posters to show us their own efforts rather than expecting us to solve the problem for them. However, this CRAN time series task view may be useful to you: https://cran.r-project.org/web/views/TimeSeries.html Cheers, Bert Bert Gunter "The trouble with having an open mind is that people keep coming along and sticking things into it." -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip ) On Wed, Aug 1, 2018 at 8:40 AM, Edoardo Silvestri < silvestri.cas...@gmail.com> wrote: > I have a database based on hourly data and I need to forecast next 24h of a > single variable. I was thinking about applying an ARIMA model with some > exogenous variables but I don't succeed to configure the hourly frequency, > estimate ARIMA parameters, pdq ( exists some tests to check which > parameters are better for the model?) and the structure of the model in its > easy form because I would also like to introduce some seasonality form by > analyzing some variables I highlighted some daily and weekly behaviours > similar. > > > > I recognize that it could be quite difficult the problem but if you have > also some useful links or some codes that can help me, please send me. > > [[alternative HTML version deleted]] > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/ > posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Combinations of true/false values where one pair is mutually exclusive
I have 6 variables, (A,B,C,D,E,F) that can either pass or fail (i.e., true
or false).
I can get a table of all pass/fail combinations with this:
scenarios <- expand.grid(A = c("pass", "fail"), B = c("pass", "fail"), C =
c("pass", "fail"), D = c("pass", "fail"), E = c("pass", "fail"), F =
c("pass", "fail"))
But I have the extra condition that if E is true, then F must be false, and
vice versa, so what I don't know is how to get all combinations when E and F
are mutually exclusive.
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] CODE HELP
Hello, I am interested to apply an econometric technique of Latent Variable framework on Environmental Kuznets Curve for 164 countries for a span of 25 years. The methodology and the code are from Simulation exercise from an unpublished paper "Two Examples of Convex-Programming-Based High-Dimensional Econometric Estimators" in R. Is it somehow possible to apply it to my data. I am attaching the codes Thanking You -- Saptorshee Chakraborty Personal Website: http://saptorshee.weebly.com/ __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] New post for Rhelp
On Wed, 1 Aug 2018 17:40:54 +0200 Edoardo Silvestri wrote: > I have a database based on hourly data and I need to forecast next > 24h of a single variable. I was thinking about applying an ARIMA > model with some exogenous variables but I don't succeed to configure > the hourly frequency, estimate ARIMA parameters, pdq ( exists some > tests to check which parameters are better for the model?) and the > structure of the model in its easy form because I would also like to > introduce some seasonality form by analyzing some variables I > highlighted some daily and weekly behaviours similar. > > > > I recognize that it could be quite difficult the problem but if you > have also some useful links or some codes that can help me, please > send me. > You are talking about analyzing data in a regular time series. R has wide ranging time series analysis packages. I would suggest starting with the basic package that comes with an R download and reading the basic information associated with it. You could also checkout Venables and Ripley, _Modern Applied Statistics with S [which R is a dialect of]. If this is a homework question, hit the books. JWDougherty __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Combinations of true/false values where one pair is mutually exclusive
Hi Rod,
How about this?
scenarios <- expand.grid(A = c("pass", "fail"), B = c("pass", "fail"), C =
c("pass", "fail"), D = c("pass", "fail"), E = c("pass", "fail"))
scenarios$F<-ifelse(scenarios$E=="pass","fail","pass")
Jim
On Thu, Aug 2, 2018 at 11:20 AM, R Stafford wrote:
> I have 6 variables, (A,B,C,D,E,F) that can either pass or fail (i.e., true
> or false).
> I can get a table of all pass/fail combinations with this:
>
> scenarios <- expand.grid(A = c("pass", "fail"), B = c("pass", "fail"), C =
> c("pass", "fail"), D = c("pass", "fail"), E = c("pass", "fail"), F =
> c("pass", "fail"))
>
> But I have the extra condition that if E is true, then F must be false, and
> vice versa, so what I don't know is how to get all combinations when E and F
> are mutually exclusive.
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] read txt file - date - no space
Dear I have check the one of the line that gives me problem. I mean, which give NA after R processing. I think that is similar to the others: 10/12/1998 10:00,0,0,0 10/12/1998 11:00,0,0,0 10/12/1998 12:00,0,0,0 10/12/1998 13:00,0,0,0 10/12/1998 14:00,0,0,0 10/12/1998 15:00,0,0,0 10/12/1998 16:00,0,0,0 10/12/1998 17:00,0,0,0 @jim: It seems that you suggestion is focus on reading data from the terminal. It is possible to apply it to a *.csv file? @Pikal: Could it be that there are some date conversion error? Thanks again, Diego Diego On 1 August 2018 at 17:01, jim holtman wrote: > > Try this: > > > library(lubridate) > > library(tidyverse) > > input <- read.csv(text = "date,str1,str2,str3 > + 10/1/1998 0:00,0.6,0,0 > + 10/1/1998 1:00,0.2,0.2,0.2 > + 10/1/1998 2:00,0.6,0.2,0.4 > + 10/1/1998 3:00,0,0,0.6 > + 10/1/1998 4:00,0,0,0 > + 10/1/1998 5:00,0,0,0 > + 10/1/1998 6:00,0,0,0 > + 10/1/1998 7:00,0.2,0,0", as.is = TRUE) > > # convert the date and add the "day" so summarize > > input <- input %>% > + mutate(date = mdy_hm(date), > + day = floor_date(date, unit = 'day') > + ) > > > > by_day <- input %>% > + group_by(day) %>% > + summarise(m_s1 = mean(str1), > + m_s2 = mean(str2), > + m_s3 = mean(str3) > + ) > > > > by_day > # A tibble: 1 x 4 > day m_s1 m_s2 m_s3 > > 1 1998-10-01 00:00:00 0.200 0.0500 0.150 > > Jim Holtman > *Data Munger Guru* > > > *What is the problem that you are trying to solve?Tell me what you want to > do, not how you want to do it.* > > > On Tue, Jul 31, 2018 at 11:54 PM Diego Avesani > wrote: > >> Dear all, >> I am sorry, I did a lot of confusion. I am sorry, I have to relax and stat >> all again in order to understand. >> If I could I would like to start again, without mixing strategy and >> waiting >> for your advice. >> >> I am really appreciate you help, really really. >> Here my new file, a *.csv file (buy the way, it is possible to attach it >> in >> the mailing list?) >> >> date,str1,str2,str3 >> 10/1/1998 0:00,0.6,0,0 >> 10/1/1998 1:00,0.2,0.2,0.2 >> 10/1/1998 2:00,0.6,0.2,0.4 >> 10/1/1998 3:00,0,0,0.6 >> 10/1/1998 4:00,0,0,0 >> 10/1/1998 5:00,0,0,0 >> 10/1/1998 6:00,0,0,0 >> 10/1/1998 7:00,0.2,0,0 >> >> >> I read it as: >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",") >> >> at this point I would like to have the daily mean. >> What would you suggest? >> >> Really Really thanks, >> You are my lifesaver >> >> Thanks >> >> >> >> Diego >> >> >> On 1 August 2018 at 01:01, Jeff Newmiller >> wrote: >> >> > ... and the most common source of NA values in time data is wrong >> > timezones. You really need to make sure the timezone that is assumed >> when >> > the character data are converted to POSIXt agrees with the data. In most >> > cases the easiest way to insure this is to use >> > >> > Sys.setenv(TZ="US/Pacific") >> > >> > or whatever timezone from >> > >> > OlsonNames() >> > >> > corresponds with your data. Execute this setenv function before the >> > strptime or as.POSIXct() function call. >> > >> > You can use >> > >> > MyData[ is.na(MyData$datetime), ] >> > >> > to see which records are failing to convert time. >> > >> > [1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1 >> > >> > On July 31, 2018 3:04:05 PM PDT, Jim Lemon >> wrote: >> > >Hi Diego, >> > >I think the error is due to NA values in your data file. If I extend >> > >your example and run it, I get no errors: >> > > >> > >MyData<-read.table(text="103001930 103001580 103001530 >> > >1998-10-01 00:00:00 0.6 0 0 >> > >1998-10-01 01:00:00 0.2 0.2 0.2 >> > >1998-10-01 02:00:00 0.6 0.2 0.4 >> > >1998-10-01 03:00:00 0 0 0.6 >> > >1998-10-01 04:00:00 0 0 0 >> > >1998-10-01 05:00:00 0 0 0 >> > >1998-10-01 06:00:00 0 0 0 >> > >1998-10-01 07:00:00 0.2 0 0 >> > >1998-10-01 08:00:00 0.6 0 0 >> > >1998-10-01 09:00:00 0.2 0.2 0.2 >> > >1998-10-01 10:00:00 0.6 0.2 0.4 >> > >1998-10-01 11:00:00 0 0 0.6 >> > >1998-10-01 12:00:00 0 0 0 >> > >1998-10-01 13:00:00 0 0 0 >> > >1998-10-01 14:00:00 0 0 0 >> > >1998-10-01 15:00:00 0.2 0 0 >> > >1998-10-01 16:00:00 0.6 0 0 >> > >1998-10-01 17:00:00 0.2 0.2 0.2 >> > >1998-10-01 18:00:00 0.6 0.2 0.4 >> > >1998-10-01 19:00:00 0 0 0.6 >> > >1998-10-01 20:00:00 0 0 0 >> > >1998-10-01 21:00:00 0 0 0 >> > >1998-10-01 22:00:00 0 0 0 >> > >1998-10-01 23:00:00 0.2 0 0 >> > >1998-10-02 00:00:00 0.6 0 0 >> > >1998-10-02 01:00:00 0.2 0.2 0.2 >> > >1998-10-02 02:00:00 0.6 0.2 0.4 >> > >1998-10-02 03:00:00 0 0 0.6 >> > >1998-10-02 04:00:00 0 0 0 >> > >1998-10-02 05:00:00 0 0 0 >> > >1998-10-02 06:00:00 0 0 0 >> > >1998-10-02 07:00:00 0.2 0 0 >> > >1998-10-02 08:00:00 0.6 0 0 >> > >1998-10-02 09:00:00 0.2 0.2 0.2 >> > >1998-10-02 10:00:00 0.6 0.2 0.4 >> > >1998-10-02 11:00:00 0 0 0.6 >> > >1998-10-02 12:00:00 0 0 0 >> > >1998-10-02 13:00:00 0 0 0 >> > >1998-10
