Hello Gary, Do you think there has been a new change in the data flow protocol coming from Wunderground? I always receive erratic data with no relation to reality.
On my diagrams, I no longer have hatched areas but still discontinuous curved lines. Le mercredi 8 novembre 2023 à 19:31:12 UTC+1, Sylvain M a écrit : > Linux Opensuse made a big update that forced all python programs to use > the python3.11 version. > This way of doing things caused a big mess in programs like tuned, > firewalld or sendmail. > Weewx had become completely inoperable. > > I found a solution by making a link from python3.11 to python3 in the > /usr/bin/ folder. > It is effective and solves the problem. > If this solution can help. > > The solution to my blockage and some practical information about > Wunderground. > > My blockage was due to an update with pip which allowed me to receive the > weewx version 5.0.0b15. > At the same time, I updated the documentation with the command > weectl station upgrade --what docs examples util > which replaced my wu-example.conf file. He was a virgin. > > I made a copy/paste error filling this file with my API key information. > This explains the 401 error. > > The exchange with Wunderground Feedback taught me that we had a quota of > transactions which is not indicated anywhere on the page dedicated to the > API key. > It’s limited to 1500 calls per day and 30 calls per minute. > Always good to know. > > Since November 6 I have been receiving erratic data with negative > temperatures even though they are around 11°C. There is also the example of > "current conditions" which indicate 1.4°C and the "statistics" indicate > 21.1°C while my station indicates 11.2°C. > > I think strongly of Gary who previously explained that there were > unexpected changes in data flows. Is that the case ? > > There are also holes in my diagrams. > I don't know how to check the continuity of temporal data in my database? > > What mistake did I make to have hatched diagrams instead of standard > curves? > > I still can't convert my wind data from m/s to km/h as well as mbar to > hPa. However, I have read the documentation and forum threads which for > some no longer apply to version 5 of weewx. > I used the settings recommended in weewx.conf: > [[Defaults]] > [[[Units]]] > [[[Groups]]]] > group_altitude = meter # Options are 'foot' or 'meter' > group_pressure = hPa # Options are 'inHg', 'mmHg', > 'mbar', or 'hPa' > group_rain = mm # Options are 'inch', 'cm', or 'mm' > group_rainrate = mm_per_hour # Options are > 'inch_per_hour', 'cm_per_hour', or 'mm_per_hour' > group_temperature = degree_C # Options are 'degree_C', > 'degree_F', or 'degree_K' > group_speed = km_per_hour # Wind and gust speed > group_speed2 = km_per_hour2 > > Thank you for your help. > > [image: Screenshot_20231108_183740 weewx 08-11-2023.jpeg] > > Le mardi 31 octobre 2023 à 18:28:28 UTC+1, Sylvain M a écrit : > >> It's not joy. >> I no longer have access to my Wunderground data with a 401 error message >> like this: >> >> #* ./maj_meteo.sh* >> >> >> *Using WeeWX configuration file >> /home/sylvain/weewx-data/weewx.confStarting wee_import...**Observation >> history for Weather Underground station 'I*****' will be imported.* >> >> >> >> *Using database binding 'wx_binding', which is bound to database >> 'weewx'Destination table 'archive' unit system is '0x11' (METRICWX).Missing >> derived observations will be calculated.* >> *Observations timestamped after 2023-10-31 00:05:00 CET (1698707100) and >> up to andincluding 2023-10-31 12:15:00 CET (1698750900) will be imported.* >> >> >> *Starting import...Records covering multiple periods have been identified >> for import.* >> >> >> *Unable to open Weather Underground station I***** or HTTP Error 401: >> Unauthorized**** Unable to load config file.****HTTP Error 401: >> Unauthorized****Nothing done, exiting.* >> >> I sent an email to the Feeback service to ask the cause of this blockage >> since there is no quota imposed when the API key is provided to us. >> I also requested the restoration of access to my data, explaining that I >> was not a flow vacuum cleaner. Just an amateur tweaking his weather >> software. >> I am waiting for a response and action. >> >> Have there ever been flow blockages before? >> >> Furthermore, I still have this duplicate primary key error but in 2022, >> which is the first day of sending data from my weather station. So every 5 >> minutes I get these errors that I have absolutely no idea how to resolve. >> I'll only paste the last 10 lines: >> #* journalctl -u weewx.service -p err -b -f --since today* >> ..... >> >> >> >> >> >> >> >> >> *Oct 31 16:49:35 gandalf python3[1297]: weewx[1297] ERROR weewx.manager: >> Unable to add record 2022-07-12 05:45:00 CEST (1657597500) to database >> 'weewx': (1062, " Duplicate entry '1657597500' for key 'PRIMARY'")Oct 31 >> 16:54:35 gandalf python3[1297]: weewx[1297] ERROR weewx.manager: Unable to >> add record 2022-07-12 05:50:00 CEST (1657597800) to database 'weewx': >> (1062, " Duplicate entry '1657597800' for key 'PRIMARY'")Oct 31 16:59:36 >> gandalf python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add >> record 2022-07-12 05:55:00 CEST (1657598100) to database 'weewx': (1062, " >> Duplicate entry '1657598100' for key 'PRIMARY'")Oct 31 17:04:36 gandalf >> python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record >> 2022-07-12 06:00:00 CEST (1657598400) to database 'weewx': (1062, " >> Duplicate entry '1657598400' for key 'PRIMARY'")Oct 31 17:09:36 gandalf >> python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record >> 2022-07-12 06:05:00 CEST (1657598700) to database 'weewx': (1062, " >> Duplicate entry '1657598700' for key 'PRIMARY'")Oct 31 17:14:36 gandalf >> python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record >> 2022-07-12 06:10:00 CEST (1657599000) to database 'weewx': (1062, " >> Duplicate entry '1657599000' for key 'PRIMARY'")Oct 31 17:19:37 gandalf >> python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record >> 2022-07-12 06:15:00 CEST (1657599300) to database 'weewx': (1062, " >> Duplicate entry '1657599300' for key 'PRIMARY'")Oct 31 17:24:37 gandalf >> python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record >> 2022-07-12 06:20:00 CEST (1657599600) to database 'weewx': (1062, " >> Duplicate entry '1657599600' for key 'PRIMARY'")Oct 31 17:29:37 gandalf >> python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record >> 2022-07-12 06:25:00 CEST (1657599900) to database 'weewx': (1062, " >> Duplicate entry '1657599900' for key 'PRIMARY'")* >> Oct 31 17:34:37 gandalf python3[1297]: weewx[1297] ERROR weewx.manager: >> Unable to add record 2022-07-12 06:30:00 CEST (1657600200) to database >> 'weewx': (1062, " Duplicate entry '1657600200' for key 'PRIMARY'") >> >> If you have an explanation and a solution to resolve this error, it would >> be appreciated? >> >> Thank you for your help. >> >> Le lundi 30 octobre 2023 à 21:29:43 UTC+1, gjr80 a écrit : >> >>> When importing records with wee_import primary key duplicate entry log >>> entries indicate a record with the timestamp of the current imported record >>> already exists in the database. This most commonly occurs if the import >>> period overlaps with existing database data. This is normal and not >>> considered an error. In such cases the imported record is discarded and no >>> database changes are made. If the imported record is required rather than >>> the corresponding record in the database, the database record must be >>> deleted and the import performed again. If the database record is to be >>> retained then nothing need be done. >>> >>> Gary >>> >>> On Tuesday, 31 October 2023 at 04:50:45 UTC+10 sylvain...@gmail.com >>> wrote: >>> >>>> Hello Gary. >>>> >>>> Thank you very much for your work ! >>>> I updated the weeimport.py file at 6:50 p.m. >>>> >>>> I created a small bash script (I'm not a programmer) to download my WU >>>> data every hour (I didn't dare to be more greedy). The script is triggered >>>> by the crontab. >>>> My script retrieves the last timestamp from the mysql database and uses >>>> it as dateFrom=%Y-%m-%dT%H:%M. The recovery date is dateTo=%Y-%m-%dT%H:%M. >>>> In principle everything works well. >>>> >>>> If I understood correctly, the calculation error that you detected can >>>> explain my mysql errors of this type 1062, "Duplicate entry '1698663292' >>>> for key 'PRIMARY": >>>> >>>> #* journalctl -u weewx.service -p err -b -f --since today* >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> *...Oct 30 17:28:01 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 03:55:00 CEST (1657590900) >>>> to database 'weewx': (1062, " Duplicate entry '1657590900' for key >>>> 'PRIMARY'")Oct 30 17:33:02 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 04:00:00 CEST (1657591200) >>>> to database 'weewx': (1062, " Duplicate entry '1657591200' for key >>>> 'PRIMARY'")Oct 30 17:38:03 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 04:05:00 CEST (1657591500) >>>> to database 'weewx': (1062, " Duplicate entry '1657591500' for key >>>> 'PRIMARY'")Oct 30 17:43:04 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 04:10:00 CEST (1657591800) >>>> to database 'weewx': (1062, " Duplicate entry '1657591800' for key >>>> 'PRIMARY'")Oct 30 17:48:04 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 04:15:00 CEST (1657592100) >>>> to database 'weewx': (1062, " Duplicate entry '1657592100' for key >>>> 'PRIMARY'")Oct 30 17:53:04 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 04:20:00 CEST (1657592400) >>>> to database 'weewx': (1062, " Duplicate entry '1657592400' for key >>>> 'PRIMARY'")Oct 30 17:58:05 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 04:25:00 CEST (1657592700) >>>> to database 'weewx': (1062, " Duplicate entry '1657592700' for key >>>> 'PRIMARY'")Oct 30 18:03:05 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 04:30:00 CEST (1657593000) >>>> to database 'weewx': (1062, " Duplicate entry '1657593000' for key >>>> 'PRIMARY'")Oct 30 18:08:05 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 04:35:00 CEST (1657593300) >>>> to database 'weewx': (1062, " Duplicate entry '1657593300' for key >>>> 'PRIMARY'")Oct 30 18:13:05 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 04:40:00 CEST (1657593600) >>>> to database 'weewx': (1062, " Duplicate entry '1657593600' for key >>>> 'PRIMARY'")Oct 30 18:18:07 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 04:45:00 CEST (1657593900) >>>> to database 'weewx': (1062, " Duplicate entry '1657593900' for key >>>> 'PRIMARY'")Oct 30 18:23:07 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 04:50:00 CEST (1657594200) >>>> to database 'weewx': (1062, " Duplicate entry '1657594200' for key >>>> 'PRIMARY'")Oct 30 18:28:07 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 04:55:00 CEST (1657594500) >>>> to database 'weewx': (1062, " Duplicate entry '1657594500' for key >>>> 'PRIMARY'")Oct 30 18:33:07 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 05:00:00 CEST (1657594800) >>>> to database 'weewx': (1062, " Duplicate entry '1657594800' for key >>>> 'PRIMARY'")Oct 30 18:38:08 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 05:05:00 CEST (1657595100) >>>> to database 'weewx': (1062, " Duplicate entry '1657595100' for key >>>> 'PRIMARY'")Oct 30 18:43:08 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 05:10:00 CEST (1657595400) >>>> to database 'weewx': (1062, " Duplicate entry '1657595400' for key >>>> 'PRIMARY'")Oct 30 18:48:08 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 05:15:00 CEST (1657595700) >>>> to database 'weewx': (1062, " Duplicate entry '1657595700' for key >>>> 'PRIMARY'")Oct 30 18:53:08 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 05:20:00 CEST (1657596000) >>>> to database 'weewx': (1062, " Duplicate entry '1657596000' for key >>>> 'PRIMARY'")Oct 30 18:58:09 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 05:25:00 CEST (1657596300) >>>> to database 'weewx': (1062, " Duplicate entry '1657596300' for key >>>> 'PRIMARY'")Oct 30 19:03:09 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 05:30:00 CEST (1657596600) >>>> to database 'weewx': (1062, " Duplicate entry '1657596600' for key >>>> 'PRIMARY'")Oct 30 19:08:09 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 05:35:00 CEST (1657596900) >>>> to database 'weewx': (1062, "Duplicate entry '1657596900' for key >>>> 'PRIMARY'")Oct 30 19:13:09 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 05:40:00 CEST (1657597200) >>>> to database 'weewx': (1062, " Duplicate entry '1657597200' for key >>>> 'PRIMARY'")Oct 30 19:18:10 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 05:45:00 CEST (1657597500) >>>> to database 'weewx': (1062, " Duplicate entry '1657597500' for key >>>> 'PRIMARY'")Oct 30 19:23:10 gandalf python3[1261]: weewx[1261] ERROR >>>> weewx.manager: Unable to add record 2022-07-12 05:50:00 CEST (1657597800) >>>> to database 'weewx': (1062, " Duplicate entry '1657597800' for key >>>> 'PRIMARY'")* >>>> >>>> >>>> I don't understand why there are Duplicate entry errors on different >>>> keys? >>>> >>>> Do you have any idea what's going on? >>>> >>>> >>>> Le lundi 30 octobre 2023 à 13:42:32 UTC+1, gjr80 a écrit : >>>> >>>>> I have been able to reproduce your symptoms and I now believe I >>>>> understand what is happening. >>>>> >>>>> In terms of why wee_import wants to import from 27 and 28 October >>>>> Vince has covered the main point. So as far as WeeWX is concerned (and >>>>> assuming a five minute archive period) the first archive record of the 27 >>>>> October covers the five minute period from midnight at the start of 27 >>>>> October until 00:05, so the first archive period of the day is >>>>> timestamped >>>>> at five minutes after midnight. The last archive period covers the five >>>>> minute period from 23:55 until (and including) midnight at the end of 27 >>>>> October, so the last archive period of 27 October is timestamped at >>>>> midnight on 28 October. >>>>> >>>>> The other factor at play is how WU provides data via its API. The WU >>>>> API call used by wee_import provides one full day of data at a time, >>>>> the problem being that WU defines a day as starting at midnight at the >>>>> start of the day though until (but not including) midnight at the end of >>>>> the day. So when the WU API responds with data for 27 October the first >>>>> record provided is the archive record timestamped at midnight at the >>>>> start >>>>> of 27 October, this archive record actually contains data from 26 October >>>>> so it is discarded. The last record provided by WU is timestamped at >>>>> 23:55 >>>>> 27 October, which is only the second last archive record for the day. WU >>>>> does not provide the last archive record of the day, to get that archive >>>>> record we need to ask WU for the data for 28 October. The first record >>>>> from >>>>> the WU 28 October data is used and the rest are discarded. >>>>> >>>>> That is why when you attempt to import a given day of data from WU you >>>>> will actually see wee_import obtain two days data from WU. >>>>> >>>>> Now for your error message. To cut a long story short there was a flaw >>>>> in how wee_import calculated the total elapsed time for an import. I >>>>> believe I have fixed it. You can use the new code by downloading the >>>>> updated weeimport.py and replacing weeimport.py on your system with >>>>> the downloaded version. In an earlier post I gave you instructions for >>>>> replacing wuimport.py, this time you need to replace weeimport.py, it >>>>> is in the same directory as wuimport.py and the process is the same >>>>> just a different file. You can download the updated weeimport.py via >>>>> the following command: >>>>> >>>>> $ wget -P /var/tmp >>>>> https://raw.githubusercontent.com/weewx/weewx/V5/src/weeimport/weeimport.py >>>>> >>>>> Do not get weeimport.py confused with wee_import.py, they are in >>>>> different directories but if you replace the wrong file you will break >>>>> wee_import. >>>>> >>>>> Once you have replaced weeimport.py wee_import should work without >>>>> error. >>>>> >>>>> Gary >>>>> >>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/d31d4635-7d08-4eaf-ab84-9a52b0107de4n%40googlegroups.com.
