Re: [weewx-user] Re: Recommends for new station under 300

[Greg Troxel]

vince  writes:

> Agree - a wiki article on this would be great to see.
>
> The one detail I can find is (here) 
>  which 
> mentions the URL to fake out, but my recollection from back then is there's 
> also a ntp server it needs to hit periodically to prove Internet 
> connectivity (or else the watchdog reboots it).  The original problem 
> report mentioning ntp is (here) 
>  as well 
> in that long thread on wxforum.net.

Thanks; that's useful.


Two questions for the list:


1) Have people used the Wittboy/GW2000 combination, and:

  does it work with the gw1000 driver?

  does it actualy work, in terms of accurate rain measurements?  (I
  dimly remember that the weatherflow tempest needed some cloud-based AI
  to get valid measurements.)

>From looking, it seems the GW1101 is the one to get for budget-conscious
people, as it doesn't have a separate rain gauge that needs new
batteries (no solar) as in the GW1102 and GW1103 and doesn't have the
ultrasonic anemometer that needs power to work in the winter
(freezeups).

2) It seems one could get a wifi weather station and also a GW1100, to
be able to have  a display and local access.  This seems to be the
sensor suite in the GW1101 and I don't see why the GW1100 wouldn't
receive it:

  https://shop.ecowitt.com/products/ws2910

has anyone used a WS-series station and added a GW1100?
Or is the interceptor easy enough to deal with that there's no real
reason?

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/rmio7w6x9me.fsf%40s1.lexort.com.

[weewx-user] Re: Missing rainRate data

[Peter Cooper-Davis]

Hi Gary,

Thanks for the quick response. Your answer makes complete sense, but 
unfortunately it doesn't seem to work. Even after I have set the rainRate 
fields to NULL,  wee_database --calc-missing just seems to refill the 
rainRate with 0.0.

I've checked that I have definitely set the rain rate fields to NULL using 
the command you suggested above:

> SELECT dateTime,datetime(dateTime, 'unixepoch','localtime'),rain,rainRate 
FROM archive WHERE dateTime>=1661385600;
1661405100|2022-08-25 06:25:00|0.016904|
1661405400|2022-08-25 06:30:00|0.065687|
1661405700|2022-08-25 06:35:00|0.037562|
1661406000|2022-08-25 06:40:00|0.185201|
1661406300|2022-08-25 06:45:00|0.178290|
1661406600|2022-08-25 06:50:00|0.038078|
1661406900|2022-08-25 06:55:00|0.075650|
1661407200|2022-08-25 07:00:00|0.376993|
1661407500|2022-08-25 07:05:00|1.389899|
1661407800|2022-08-25 07:10:00|1.569504|
1661408100|2022-08-25 07:15:00|1.018610|

But after running wee_database --calc-missing --from=2022-08-25 the 
database looks like this

> SELECT dateTime,datetime(dateTime, 'unixepoch','localtime'),rain,rainRate 
FROM archive WHERE dateTime>=1661385600;
1661405100|2022-08-25 06:25:00|0.016904|0.0
1661405400|2022-08-25 06:30:00|0.065687|0.0
1661405700|2022-08-25 06:35:00|0.037562|0.0
1661406000|2022-08-25 06:40:00|0.185201|0.0
1661406300|2022-08-25 06:45:00|0.178290|0.0
1661406600|2022-08-25 06:50:00|0.038078|0.0
1661406900|2022-08-25 06:55:00|0.075650|0.0
1661407200|2022-08-25 07:00:00|0.376993|0.0
1661407500|2022-08-25 07:05:00|1.389899|0.0
1661407800|2022-08-25 07:10:00|1.569504|0.0
1661408100|2022-08-25 07:15:00|1.018610|0.0

Is there something obvious I am missing?

Pete

On Friday, 26 August 2022 at 03:40:09 UTC+1 gjr80 wrote:

> wee_database --calc-missing will only populate missing derived obs 
> provided (1) the missing derived obs field contains no data and (2) the 
> pre-requisites for the calculation exist. In your case you are failing on 
> the first test - your rainRate fields contain the value 0. You can try 
> setting the rainRate field to null in each affected record. Assuming you 
> are using a SQLite database this is most reliably done using the sqlite3 
> utility. To do this:
>
> 1. using something like Epoch Converter  
> to determine the epoch timestamp for the first and last archive records in 
> the block of records you wish to update, let's refer to these values as 
> start_timestamp and end_timestamp
>
> 2. install the sqlite3 utility if not already installed:
> $ sudo apt install sqlite3
>
> 3. stop WeeWX and make a backup of your WeeWX database, unless you have 
> changed it this will be /home/weewx/archive/weewx.sdb or 
> /var/lib/weewx/weewx.sdb depending on your WeeWX install type 
>
> 4. set the rainRate field for the records concerned to null:
> $ sqlite3 /home/weewx/archive/weewx.sdb
> > UPDATE archive SET rainRate=NULL WHERE dateTime>=start_timestamp AND 
> dateTime<=end_timestamp;
>
> replacing start_timestamp and end_timestamp with the actual epoch 
> timestamps. You can check the data has been cleared with the following 
> query:
> > SELECT dateTime,datetime(dateTime, 
> 'unixepoch','localtime'),rain,rainRate FROM archive 
> WHERE dateTime>=start_timestamp AND dateTime<=end_timestamp;
>
> 5. once happy the rainRate values have been nulled exit sqlite3 with .q
>
> 6. run wee_database --calc-missing to calculate the missing rainRate 
> data, use the --date or --from and --to 
>  to limit the 
> span of records that wee_database will operate on.
>
> 7. check the if rainRate has been correctly calculated:
> $ sqlite3 /home/weewx/archive/weewx.sdb
> > SELECT dateTime,datetime(dateTime, 
> 'unixepoch','localtime'),rain,rainRate FROM archive 
> WHERE dateTime>=start_timestamp AND dateTime<=end_timestamp;
> > .q
>
> If you are using MySQL/MariaDB the approach is the same, but you would use 
> mysql in lieu of sqlite3.
>
> Gary
> On Friday, 26 August 2022 at 05:50:12 UTC+10 peted...@gmail.com wrote:
>
>> I recently (25/08/2022) had to delete a 6 hour section of data from my 
>> database as the rain accumulation data contained in the LOOP packets from 
>> that period were incorrect. I refilled the database using the correct rain 
>> accumulation data from ARCHIVE packets over that 6 hours, and everything 
>> (in terms of the daily summaries etc.) now looks good. The only issue I 
>> have though is that there is no rainRate data in my database. Looking at a 
>> selection of records I can see rain accumulation, but no rainRate:
>>
>> dateTime|rain  |rainRate
>> 2022-08-25 06:15:00|1.018610|0.0
>> 2022-08-25 06:20:00|0.978673|0.0
>> 2022-08-25 06:25:00|1.367165|0.0
>> 2022-08-25 06:30:00|1.710911|0.0
>> 2022-08-25 06:35:00|1.588113|0.0
>> 2022-08-25 06:40:00|1.858461|0.0
>> 2022-08-25 06:45:00|1.654587|0.0
>> 2022-08-25 06:50:00|1.475450|0.0
>> 2022-08-25 06:55:00|1.

Re: [weewx-user] WDC Skin

[Montefiori Luca]

@david

Sorry for the late reply.

This morning I tested version 2.2.0
Great work.
I edited the configuration file:
 [[[inTemp]]]
 yScaleOffset = 0.5
 [[[inHumidity]]]
  yScaleOffset = 5
The "yScaleOffset = 0.5" setting for inTemp works correctly
The "yScaleOffset" setting for inHumidity seems not to work.

I also tried
 [[[inHumidity]]]
 yScaleMin = 60
 yScaleMax = 80
and this setting works correctly.
Where am I wrong?

Thanks for your help and your work.


Il giorno sab 20 ago 2022 alle ore 02:15 David Bätge 
ha scritto:

> @luca The new release 2.2.0
>  is out.
> Included is among other things (like a really pretty dark mode) three new
> options for charts customisation: yScaleMin, yScaleMax and yScaleOffset.
>
> Min and Max is for setting a static value, yScaleOffset is for setting a
> custom (dynamic) offset. you can read more about it in the wiki:
> https://github.com/Daveiano/weewx-wdc/wiki/Configuration#general
>
> I hope this will be a good addition and is what you were looking for,
> thank you for your feedback!
>
> David Bätge schrieb am Dienstag, 16. August 2022 um 03:23:58 UTC+2:
>
>> Yes from all that you have written, this is what makes sense and this is
>> what you want to change.
>>
>> I will probably release a new version (2.2.0) by the end of this week,
>> which will include your first proposed improvement, the configurable yScale
>> delta. The second improvement will hit in the next release. Thank you for
>> your feedback!
>> luca.mo...@gmail.com schrieb am Sonntag, 14. August 2022 um 09:51:09
>> UTC+2:
>>
>>> I read in https://www.weewx.com/docs/customizing.htm :
>>>
>>> *aggregate_interval*
>>>
>>> *The time period over which the data should be aggregated, in seconds.
>>> Required if aggregate_type has been set. Alternatively, the time can be
>>> specified by using one of the "shortcuts" (that is, hour, day, week, month,
>>> or year).*
>>> So, you would like to implement in your code the "aggregate_interval".
>>> At this moment the value of aggregate_interval is "coded" inside the
>>> software.
>>> You would like to let the user to be able to set this value for each
>>> contex/observation.
>>>
>>> This is probably the correct and most "elegant" solution.
>>>
>>> I agree with you.
>>>
>>>
>>>
>>>
>>> Il giorno sab 13 ago 2022 alle ore 16:22 David Bätge <
>>> david@gmail.com> ha scritto:
>>>
 I think we are talking about slightly different things. Let me explain:

 The values for




 *Today = last 24 hoursWeek = last 168 hoursMonth = last 31 daysYear =
 last 365 days*

 will not change and will not be changeable by the user because this
 would not make any sense. Last week will consist of the last 7 days, not 6,
 not 8. Every time period is a constant.

 *Currently each chart consists of 48 values (every 30 min per daily
 chart) (except for rain and evotranspiration)*

 Only the charts on the index page consist of 48 points:

 The index page, has a time span of 24h, aggregate_interval of 30min, so
 24/0.5 = *48 points*
 The weekly page, has a time span of 7 days (168h), aggregate_interval
 of 2h, so 168/2 = *84 points*
 The monthly pages, have a time span of 31 days (744h),
 aggregate_interval of 6h, so 744/6 = *124 points*
 As I said, the aggregate_interval for the stats page (alltime) gets
 calculated automatically to not make the graphs too heavy by producing too
 many points (currently the max is 100 points on the stats page).

 What you want to change is the aggregate_interval, that is what you
 call *resolution*.

 This should be changeable for
 1. The context, eg. index, weekly, monthly, yearly
 2. The observation (You are right, this should be changeable for every
 single graph (observation).)

 However, I tend to implement the aggregate_interval, and not a
 multiplier for multiple reasons:
 - weewx also uses this in the image generator so users should/could
 know about this
 - To accomplish a specific resolution, the user would need to calculate
 the specific multiplier, with a aggregate_interval it is much easier I
 think!?
 - with the aggregate_interval, the user would be really "free" in what
 to configure (like everything is possible)

 Please let me know what you think about that. And by the way, your
 English isn't bad at all ;)
 Best regards
 luca.mo...@gmail.com schrieb am Freitag, 12. August 2022 um 16:56:19
 UTC+2:

> > Currently, there are predefined aggregate_intervals for each page
> (index, week, month, year and stats) that should make sense. I
> thought about making it possible (opt-in) to change these values for every
> page.
>
> Probably this does not have much sense and needs much effort in

Re: [weewx-user] Re: Recommends for new station under 300

[Rainer Lang]

1. the GW1000, GW1100, WH2650, GW2000, WN19x0 consoles all share the 
same APi - the Ecowitt Gateway API aka GW1000 API
the Wittboy = GW2001 station = GW2000 console/gateway + WS90 7-in-1 
sensor array
The WS90 outdoor array can be connected to any of the above mentioned 
consoles with Ecowitt Gateway API (latest firmware needed)


The haptic rain sensor of the WS90 can be calibrated on five tiers 
depending on rain rate.
It has also already undergone several firmware updates to fix issues of 
rain rate, rain+wind, dew, non-rainfall "touches" etc. and its accuracy 
is growing.
In my case the WS90 readings are between a WH40 and a WS69 rain gauge 
readings.
It will need a larger number of different rain events to fine tune the 
calibration. It's not out-of-the box accurate yet (for rain), but looks 
much better than the "children's diseases" of the Tempest. It doesn't 
need a permanent  AI based crowd calibration.
One of the strong features of the WS90 is its stability, robustness and 
it doesn't have any moveable parts.
The GW2000 console can be connected via WiFi or via Ethernet (LAN cable) 
while all other recent Ecowitt consoles are WiFi only.


The above mentioned consoles GWyx00, WN191x, WH2650 can receive all 
existing recent Ecowitt sensors/arrays.
My GW1000, GW1100, GW2000 etc. can all receive the WS90 - so can the 
HP25x0 consoles (only they don't have an API) - alone or in parallel
You can factually combine +/- all consoles with all sensors - for model 
limitations see https://www.wxforum.net/index.php?topic=40730.0

Also beyond the offered packages/stations.

On 27.08.2022 12:29, Greg Troxel wrote:

vince  writes:


Agree - a wiki article on this would be great to see.

The one detail I can find is (here)
 which
mentions the URL to fake out, but my recollection from back then is there's
also a ntp server it needs to hit periodically to prove Internet
connectivity (or else the watchdog reboots it).  The original problem
report mentioning ntp is (here)
 as well
in that long thread on wxforum.net.

Thanks; that's useful.


Two questions for the list:


1) Have people used the Wittboy/GW2000 combination, and:

   does it work with the gw1000 driver?

   does it actualy work, in terms of accurate rain measurements?  (I
   dimly remember that the weatherflow tempest needed some cloud-based AI
   to get valid measurements.)

 From looking, it seems the GW1101 is the one to get for budget-conscious
people, as it doesn't have a separate rain gauge that needs new
batteries (no solar) as in the GW1102 and GW1103 and doesn't have the
ultrasonic anemometer that needs power to work in the winter
(freezeups).

2) It seems one could get a wifi weather station and also a GW1100, to
be able to have  a display and local access.  This seems to be the
sensor suite in the GW1101 and I don't see why the GW1100 wouldn't
receive it:

   https://shop.ecowitt.com/products/ws2910

has anyone used a WS-series station and added a GW1100?
Or is the interceptor easy enough to deal with that there's no real
reason?



--
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/9c37bf88-38d9-d5c8-5a3a-cdd8df9b38e7%40gmail.com.

Re: [weewx-user] WDC Skin

[David Bätge]

@luca You got me. Before making this configurable for the user, I set some 
of the values programmatically. I forgot to move some of these static set 
values from code to skin.conf. 

For example I have set a static

yScaleMin = 0
yScaleMax = 103

via Code. With this, it is not possible to set a custom yScaleOffset 
because it's not used if a yScaleMin or yScaleMax is set (see 
https://github.com/Daveiano/weewx-wdc/wiki/Configuration#general)

I will move all these programmatically set default values to the skin.conf 
in the next release to give the user full control. Thank you for pointing 
this out!



luca.mo...@gmail.com schrieb am Samstag, 27. August 2022 um 13:09:32 UTC+2:

> @david
>
> Sorry for the late reply.
>
> This morning I tested version 2.2.0
> Great work.
> I edited the configuration file:
>  [[[inTemp]]]
>  yScaleOffset = 0.5
>  [[[inHumidity]]]
>   yScaleOffset = 5
> The "yScaleOffset = 0.5" setting for inTemp works correctly
> The "yScaleOffset" setting for inHumidity seems not to work.
>
> I also tried
>  [[[inHumidity]]]
>  yScaleMin = 60
>  yScaleMax = 80
> and this setting works correctly.
> Where am I wrong?
>
> Thanks for your help and your work.
>
>
> Il giorno sab 20 ago 2022 alle ore 02:15 David Bätge  
> ha scritto:
>
>> @luca The new release 2.2.0 
>>  is out. 
>> Included is among other things (like a really pretty dark mode) three new 
>> options for charts customisation: yScaleMin, yScaleMax and yScaleOffset.
>>
>> Min and Max is for setting a static value, yScaleOffset is for setting a 
>> custom (dynamic) offset. you can read more about it in the wiki: 
>> https://github.com/Daveiano/weewx-wdc/wiki/Configuration#general
>>
>> I hope this will be a good addition and is what you were looking for, 
>> thank you for your feedback!
>>
>> David Bätge schrieb am Dienstag, 16. August 2022 um 03:23:58 UTC+2:
>>
>>> Yes from all that you have written, this is what makes sense and this is 
>>> what you want to change. 
>>>
>>> I will probably release a new version (2.2.0) by the end of this week, 
>>> which will include your first proposed improvement, the configurable yScale 
>>> delta. The second improvement will hit in the next release. Thank you for 
>>> your feedback!
>>> luca.mo...@gmail.com schrieb am Sonntag, 14. August 2022 um 09:51:09 
>>> UTC+2:
>>>
 I read in https://www.weewx.com/docs/customizing.htm :

 *aggregate_interval*

 *The time period over which the data should be aggregated, in seconds. 
 Required if aggregate_type has been set. Alternatively, the time can be 
 specified by using one of the "shortcuts" (that is, hour, day, week, 
 month, 
 or year).*
 So, you would like to implement in your code the "aggregate_interval".
 At this moment the value of aggregate_interval is "coded" inside the 
 software.
 You would like to let the user to be able to set this value for each 
 contex/observation.

 This is probably the correct and most "elegant" solution.

 I agree with you. 




 Il giorno sab 13 ago 2022 alle ore 16:22 David Bätge <
 david@gmail.com> ha scritto:

> I think we are talking about slightly different things. Let me explain:
>
> The values for
>
>
>
>
> *Today = last 24 hoursWeek = last 168 hoursMonth = last 31 daysYear = 
> last 365 days*
>
> will not change and will not be changeable by the user because this 
> would not make any sense. Last week will consist of the last 7 days, not 
> 6, 
> not 8. Every time period is a constant.
>
> *Currently each chart consists of 48 values (every 30 min per daily 
> chart) (except for rain and evotranspiration)*
>
> Only the charts on the index page consist of 48 points:
>
> The index page, has a time span of 24h, aggregate_interval of 30min, 
> so 24/0.5 = *48 points*
> The weekly page, has a time span of 7 days (168h), aggregate_interval 
> of 2h, so 168/2 = *84 points*
> The monthly pages, have a time span of 31 days (744h), 
> aggregate_interval of 6h, so 744/6 = *124 points*
> As I said, the aggregate_interval for the stats page (alltime) gets 
> calculated automatically to not make the graphs too heavy by producing 
> too 
> many points (currently the max is 100 points on the stats page).
>
> What you want to change is the aggregate_interval, that is what you 
> call *resolution*. 
>
> This should be changeable for 
> 1. The context, eg. index, weekly, monthly, yearly
> 2. The observation (You are right, this should be changeable for every 
> single graph (observation).)
>
> However, I tend to implement the aggregate_interval, and not a 
> multiplier for multiple reasons:
> - weewx also uses this in the image g

Re: [weewx-user] WDC Skin

[David Bätge]

Sorry, just for clarification: the yScaleMin = 0, yScaleMax = 103 values 
are set for humidity.



David Bätge schrieb am Samstag, 27. August 2022 um 17:02:55 UTC+2:

> @luca You got me. Before making this configurable for the user, I set some 
> of the values programmatically. I forgot to move some of these static set 
> values from code to skin.conf. 
>
> For example I have set a static
>
> yScaleMin = 0
> yScaleMax = 103
>
> via Code. With this, it is not possible to set a custom yScaleOffset 
> because it's not used if a yScaleMin or yScaleMax is set (see 
> https://github.com/Daveiano/weewx-wdc/wiki/Configuration#general)
>
> I will move all these programmatically set default values to the skin.conf 
> in the next release to give the user full control. Thank you for pointing 
> this out!
>
>
>
> luca.mo...@gmail.com schrieb am Samstag, 27. August 2022 um 13:09:32 
> UTC+2:
>
>> @david
>>
>> Sorry for the late reply.
>>
>> This morning I tested version 2.2.0
>> Great work.
>> I edited the configuration file:
>>  [[[inTemp]]]
>>  yScaleOffset = 0.5
>>  [[[inHumidity]]]
>>   yScaleOffset = 5
>> The "yScaleOffset = 0.5" setting for inTemp works correctly
>> The "yScaleOffset" setting for inHumidity seems not to work.
>>
>> I also tried
>>  [[[inHumidity]]]
>>  yScaleMin = 60
>>  yScaleMax = 80
>> and this setting works correctly.
>> Where am I wrong?
>>
>> Thanks for your help and your work.
>>
>>
>> Il giorno sab 20 ago 2022 alle ore 02:15 David Bätge  
>> ha scritto:
>>
>>> @luca The new release 2.2.0 
>>>  is out. 
>>> Included is among other things (like a really pretty dark mode) three new 
>>> options for charts customisation: yScaleMin, yScaleMax and yScaleOffset.
>>>
>>> Min and Max is for setting a static value, yScaleOffset is for setting 
>>> a custom (dynamic) offset. you can read more about it in the wiki: 
>>> https://github.com/Daveiano/weewx-wdc/wiki/Configuration#general
>>>
>>> I hope this will be a good addition and is what you were looking for, 
>>> thank you for your feedback!
>>>
>>> David Bätge schrieb am Dienstag, 16. August 2022 um 03:23:58 UTC+2:
>>>
 Yes from all that you have written, this is what makes sense and this 
 is what you want to change. 

 I will probably release a new version (2.2.0) by the end of this week, 
 which will include your first proposed improvement, the configurable 
 yScale 
 delta. The second improvement will hit in the next release. Thank you for 
 your feedback!
 luca.mo...@gmail.com schrieb am Sonntag, 14. August 2022 um 09:51:09 
 UTC+2:

> I read in https://www.weewx.com/docs/customizing.htm :
>
> *aggregate_interval*
>
> *The time period over which the data should be aggregated, in seconds. 
> Required if aggregate_type has been set. Alternatively, the time can be 
> specified by using one of the "shortcuts" (that is, hour, day, week, 
> month, 
> or year).*
> So, you would like to implement in your code the "aggregate_interval".
> At this moment the value of aggregate_interval is "coded" inside the 
> software.
> You would like to let the user to be able to set this value for each 
> contex/observation.
>
> This is probably the correct and most "elegant" solution.
>
> I agree with you. 
>
>
>
>
> Il giorno sab 13 ago 2022 alle ore 16:22 David Bätge <
> david@gmail.com> ha scritto:
>
>> I think we are talking about slightly different things. Let me 
>> explain:
>>
>> The values for
>>
>>
>>
>>
>> *Today = last 24 hoursWeek = last 168 hoursMonth = last 31 daysYear = 
>> last 365 days*
>>
>> will not change and will not be changeable by the user because this 
>> would not make any sense. Last week will consist of the last 7 days, not 
>> 6, 
>> not 8. Every time period is a constant.
>>
>> *Currently each chart consists of 48 values (every 30 min per daily 
>> chart) (except for rain and evotranspiration)*
>>
>> Only the charts on the index page consist of 48 points:
>>
>> The index page, has a time span of 24h, aggregate_interval of 30min, 
>> so 24/0.5 = *48 points*
>> The weekly page, has a time span of 7 days (168h), aggregate_interval 
>> of 2h, so 168/2 = *84 points*
>> The monthly pages, have a time span of 31 days (744h), 
>> aggregate_interval of 6h, so 744/6 = *124 points*
>> As I said, the aggregate_interval for the stats page (alltime) gets 
>> calculated automatically to not make the graphs too heavy by producing 
>> too 
>> many points (currently the max is 100 points on the stats page).
>>
>> What you want to change is the aggregate_interval, that is what you 
>> call *resolution*. 
>>
>> This should be changeable for 

[weewx-user] Re: WeatherCat import Vs Daylight Savings Time

[vhel...@gmail.com]

Hello,

I have succesfully imported Weathercat data from about 7 years. That was 
about year and half ago. I remember, that I didn't have any problems with 
daylight saving. And In Finland we have summer time , which is +1 hour. 

I have data logged in 1 minute intervals, and I used X minutes option in 
import config.

torstai 25. elokuuta 2022 klo 16.54.11 UTC+3 kurand kirjoitti:

> Thanks for confirming what I understood from weeimport.py and 
> weathercatimport.py.
>
> I am using "derived' for the import because I originally configured 
> WeatherCat with Adaptive interval set true so I have data logged at 
> varying  rates from 1 minute all the way up to 30+ minute intervals. 
>
> I think in this case, weathercatimport.py needs to use timezone aware 
> date-time objects during its construction of dateTime else data will become 
> misplaced and DST changes will fail as noted. Once this is accomplished, 
> then something needs to get added to deal with the apparent bug in 
> WeatherCat where transitioning from standard time to daylight saving time 
> in spring causes it to skip an extra hour forward sometimes. It looks to me 
> like this can all be isolated in weathercatimport.py's getRawData function. 
>
> I'm curious to know if anyone else has run into this problem, or if it is 
> something specific to my WeatherCat data. Or perhaps WeatherCat import has 
> been used in anger. 
>
> On Thursday, August 25, 2022 at 11:59:34 AM UTC+1 gjr80 wrote:
>
>> If you are using 'derive' to derive the archive interval from the 
>> imported data then the imported records must be in ascending 
>> timestamp/date-time order. This is because the derived archive interval is 
>> calculated by taking the difference between the timestamp of the current 
>> record and the timestamp of the previous record. If the calculated interval 
>> is negative it indicates the records being imported are not in timestamp 
>> order and the calculated archive interval cannot be trusted. The import 
>> will abort.
>>
>> This should not present a problem at the start of a daylight saving 
>> period where clocks jump forward (though the archive interval would be 
>> incorrectly calculated for the first record after daylight saving begins). 
>> It would certainly cause a problem at the end of a period of daylight 
>> saving where clocks jump back in time.
>>
>> So what to do? WeeWX date-time objects are timezone naive 
>>  
>> so managing daylight saving is near impossible. I would suggest removing 
>> the data to be imported with the duplicate date-times (which set of 
>> duplicates I leave up to you) and re-trying the import. You might also want 
>> to consider using a fixed interval (if your data does indeed have a fixed 
>> interval) rather than deriving it from the data. FWIW, the vantage driver 
>> has a similar issue in that each year when daylight saving ends the driver 
>> receives archive records with duplicate timestamps, in this case the 
>> situation is handled slightly more gracefully in that WeeWX ignores the 
>> duplicate timestamped records with a 'PRIMARY KEY must be unique' error. 
>> wee_import is somewhat less forgiving.
>>
>> Gary
>> On Thursday, 25 August 2022 at 00:45:13 UTC+10 kurand wrote:
>>
>>> Has anyone successfully imported data from WeatherCat for months where 
>>> Daylight Savings Time starts or ends? 
>>>
>>> When I have tried this, I get the following type of error (example for 
>>> Oct-2017):
>>>
>>> Cannot derive 'interval' for record timestamp: 2017-10-29 01:00:00 BST 
>>> (1509235200).
>>>
>>>  Raw data is not in ascending date time order.
>>>
>>> Looking at the 7 years of data I have logged by WeatherCat, it appears 
>>> that WeatherCat does not really handle DST transitions very well. Data is 
>>> generally logging in local time.
>>>
>>> For spring DST changes (in March for my UK based timezone), it seems 
>>> usually to jump forward by two hours at 0100 going to 0300 rather than just 
>>> one hour as you'd expect. It then tack on an hours worth of data for day 01 
>>> at the end of the month. This gives rise to the above error when the end of 
>>> the month's data is processed. However, for some reason, in my data for 
>>> 2014, it does only jump forward one hour as expected and there was no extra 
>>> data logged at the end of the month. 
>>>
>>> For autumn DST changes (in October in UK timezone), it seems to jump 
>>> back an hour at 0200, so it repeats log entries for hour 0100 again. This 
>>> gives rise to the above error.
>>>
>>> It's all a bit of a mess really; I'm not sure if its just my WeatherCat 
>>> data, or the way I configured it with Adaptive interval set true or 
>>> something else. 
>>>
>>> Does anyone have a fix for this, or do I need to work something out.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving 

[weewx-user] Re: WeatherCat import Vs Daylight Savings Time

[gjr80]

On Sunday, 28 August 2022 at 01:37:33 UTC+10 vhel...@gmail.com wrote:

> Hello,
>
> I have succesfully imported Weathercat data from about 7 years. That was 
> about year and half ago. I remember, that I didn't have any problems with 
> daylight saving. And In Finland we have summer time , which is +1 hour. 
>
> I have data logged in 1 minute intervals, and I used X minutes option in 
> import config.
>
 
In your case the fixed one minute interval effectively bypasses the 
ascending date-time order check. I suspect you will find an hours worth of 
import data will have been discarded each time daylight saving ended and 
clocks were wound back. This will have been recorded in the log, but 
unfortunately wee_import is not made aware of the issue (and hence does not 
report it) as the WeeWX archive management machinery is used for saving 
imported records to archive it that machinery only logs success or failure 
and does not otherwise report back. 

Gary

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/0a053f72-bc8d-4d3c-a671-985c1396d0f5n%40googlegroups.com.

[weewx-user] Re: WeatherCat import Vs Daylight Savings Time

[gjr80]

On Thursday, 25 August 2022 at 23:54:11 UTC+10 kurand wrote:

> Thanks for confirming what I understood from weeimport.py and 
> weathercatimport.py.
>
> I am using "derived' for the import because I originally configured 
> WeatherCat with Adaptive interval set true so I have data logged at 
> varying  rates from 1 minute all the way up to 30+ minute intervals. 
>
> I think in this case, weathercatimport.py needs to use timezone aware 
> date-time objects during its construction of dateTime else data will become 
> misplaced and DST changes will fail as noted. Once this is accomplished, 
> then something needs to get added to deal with the apparent bug in 
> WeatherCat where transitioning from standard time to daylight saving time 
> in spring causes it to skip an extra hour forward sometimes. It looks to me 
> like this can all be isolated in weathercatimport.py's getRawData function. 
>
> I'm curious to know if anyone else has run into this problem, or if it is 
> something specific to my WeatherCat data. Or perhaps WeatherCat import has 
> been used in anger. 
>

 Yes 'aware' datetime objects would be preferred; however, their use raises 
a number of other issues. Programatically, there is the potential for 
conflict with the rest of the WeeWX code base which uses naive datetime 
objects only. Also, we try to avoid external dependencies unless absolutely 
necessary so hence we need to work within the standard libraries shipped 
with python. Then there is the issue of daylight saving cutover times often 
being politically driven, so who knows what impact that may have - we may 
be solving one issue only to create others.

That being said there are two issues at play here. Firstly, an out of 
date-time order record caused the entire import to abort. The reasoning for 
this behaviour was the assumption that a randomly ordered batch of records 
was provided for import, this is probably too onerous and we could (should 
be able to) back this off to simply ignore the miscreant record. This 
should be a simple change. Secondly, the issue of daylight saving. I am 
happy to have a look at wee_import to see what can be done to support 
daylight saving using the existing included python libraries. To this end 
some relevant WeatherCat data files would help, would you be able to 
provide example WeatherCat files covering both the start and end of 
daylight saving? If you want you can reply privately to this post and 
include them. thanks.

Gary

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/b4a8ec79-e9f0-4ea4-a28f-5f3d1a2d3c5fn%40googlegroups.com.