It's not joy. I no longer have access to my Wunderground data with a 401 error message like this:
#* ./maj_meteo.sh* *Using WeeWX configuration file /home/sylvain/weewx-data/weewx.confStarting wee_import...Observation history for Weather Underground station 'I*****' will be imported.Using database binding 'wx_binding', which is bound to database 'weewx'Destination table 'archive' unit system is '0x11' (METRICWX).Missing derived observations will be calculated.Observations timestamped after 2023-10-31 00:05:00 CET (1698707100) and up to andincluding 2023-10-31 12:15:00 CET (1698750900) will be imported.Starting import...Records covering multiple periods have been identified for import.Unable to open Weather Underground station I***** or HTTP Error 401: Unauthorized**** Unable to load config file.****HTTP Error 401: Unauthorized****Nothing done, exiting.* I sent an email to the Feeback service to ask the cause of this blockage since there is no quota imposed when the API key is provided to us. I also requested the restoration of access to my data, explaining that I was not a flow vacuum cleaner. Just an amateur tweaking his weather software. I am waiting for a response and action. Have there ever been flow blockages before? Furthermore, I still have this duplicate primary key error but in 2022, which is the first day of sending data from my weather station. So every 5 minutes I get these errors that I have absolutely no idea how to resolve. I'll only paste the last 10 lines: #* journalctl -u weewx.service -p err -b -f --since today* ..... *Oct 31 16:49:35 gandalf python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record 2022-07-12 05:45:00 CEST (1657597500) to database 'weewx': (1062, " Duplicate entry '1657597500' for key 'PRIMARY'")Oct 31 16:54:35 gandalf python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record 2022-07-12 05:50:00 CEST (1657597800) to database 'weewx': (1062, " Duplicate entry '1657597800' for key 'PRIMARY'")Oct 31 16:59:36 gandalf python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record 2022-07-12 05:55:00 CEST (1657598100) to database 'weewx': (1062, " Duplicate entry '1657598100' for key 'PRIMARY'")Oct 31 17:04:36 gandalf python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record 2022-07-12 06:00:00 CEST (1657598400) to database 'weewx': (1062, " Duplicate entry '1657598400' for key 'PRIMARY'")Oct 31 17:09:36 gandalf python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record 2022-07-12 06:05:00 CEST (1657598700) to database 'weewx': (1062, " Duplicate entry '1657598700' for key 'PRIMARY'")Oct 31 17:14:36 gandalf python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record 2022-07-12 06:10:00 CEST (1657599000) to database 'weewx': (1062, " Duplicate entry '1657599000' for key 'PRIMARY'")Oct 31 17:19:37 gandalf python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record 2022-07-12 06:15:00 CEST (1657599300) to database 'weewx': (1062, " Duplicate entry '1657599300' for key 'PRIMARY'")Oct 31 17:24:37 gandalf python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record 2022-07-12 06:20:00 CEST (1657599600) to database 'weewx': (1062, " Duplicate entry '1657599600' for key 'PRIMARY'")Oct 31 17:29:37 gandalf python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record 2022-07-12 06:25:00 CEST (1657599900) to database 'weewx': (1062, " Duplicate entry '1657599900' for key 'PRIMARY'")* Oct 31 17:34:37 gandalf python3[1297]: weewx[1297] ERROR weewx.manager: Unable to add record 2022-07-12 06:30:00 CEST (1657600200) to database 'weewx': (1062, " Duplicate entry '1657600200' for key 'PRIMARY'") If you have an explanation and a solution to resolve this error, it would be appreciated? Thank you for your help. Le lundi 30 octobre 2023 à 21:29:43 UTC+1, gjr80 a écrit : > When importing records with wee_import primary key duplicate entry log > entries indicate a record with the timestamp of the current imported record > already exists in the database. This most commonly occurs if the import > period overlaps with existing database data. This is normal and not > considered an error. In such cases the imported record is discarded and no > database changes are made. If the imported record is required rather than > the corresponding record in the database, the database record must be > deleted and the import performed again. If the database record is to be > retained then nothing need be done. > > Gary > > On Tuesday, 31 October 2023 at 04:50:45 UTC+10 sylvain...@gmail.com wrote: > >> Hello Gary. >> >> Thank you very much for your work ! >> I updated the weeimport.py file at 6:50 p.m. >> >> I created a small bash script (I'm not a programmer) to download my WU >> data every hour (I didn't dare to be more greedy). The script is triggered >> by the crontab. >> My script retrieves the last timestamp from the mysql database and uses >> it as dateFrom=%Y-%m-%dT%H:%M. The recovery date is dateTo=%Y-%m-%dT%H:%M. >> In principle everything works well. >> >> If I understood correctly, the calculation error that you detected can >> explain my mysql errors of this type 1062, "Duplicate entry '1698663292' >> for key 'PRIMARY": >> >> #* journalctl -u weewx.service -p err -b -f --since today* >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> *...Oct 30 17:28:01 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 03:55:00 CEST (1657590900) >> to database 'weewx': (1062, " Duplicate entry '1657590900' for key >> 'PRIMARY'")Oct 30 17:33:02 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 04:00:00 CEST (1657591200) >> to database 'weewx': (1062, " Duplicate entry '1657591200' for key >> 'PRIMARY'")Oct 30 17:38:03 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 04:05:00 CEST (1657591500) >> to database 'weewx': (1062, " Duplicate entry '1657591500' for key >> 'PRIMARY'")Oct 30 17:43:04 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 04:10:00 CEST (1657591800) >> to database 'weewx': (1062, " Duplicate entry '1657591800' for key >> 'PRIMARY'")Oct 30 17:48:04 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 04:15:00 CEST (1657592100) >> to database 'weewx': (1062, " Duplicate entry '1657592100' for key >> 'PRIMARY'")Oct 30 17:53:04 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 04:20:00 CEST (1657592400) >> to database 'weewx': (1062, " Duplicate entry '1657592400' for key >> 'PRIMARY'")Oct 30 17:58:05 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 04:25:00 CEST (1657592700) >> to database 'weewx': (1062, " Duplicate entry '1657592700' for key >> 'PRIMARY'")Oct 30 18:03:05 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 04:30:00 CEST (1657593000) >> to database 'weewx': (1062, " Duplicate entry '1657593000' for key >> 'PRIMARY'")Oct 30 18:08:05 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 04:35:00 CEST (1657593300) >> to database 'weewx': (1062, " Duplicate entry '1657593300' for key >> 'PRIMARY'")Oct 30 18:13:05 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 04:40:00 CEST (1657593600) >> to database 'weewx': (1062, " Duplicate entry '1657593600' for key >> 'PRIMARY'")Oct 30 18:18:07 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 04:45:00 CEST (1657593900) >> to database 'weewx': (1062, " Duplicate entry '1657593900' for key >> 'PRIMARY'")Oct 30 18:23:07 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 04:50:00 CEST (1657594200) >> to database 'weewx': (1062, " Duplicate entry '1657594200' for key >> 'PRIMARY'")Oct 30 18:28:07 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 04:55:00 CEST (1657594500) >> to database 'weewx': (1062, " Duplicate entry '1657594500' for key >> 'PRIMARY'")Oct 30 18:33:07 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 05:00:00 CEST (1657594800) >> to database 'weewx': (1062, " Duplicate entry '1657594800' for key >> 'PRIMARY'")Oct 30 18:38:08 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 05:05:00 CEST (1657595100) >> to database 'weewx': (1062, " Duplicate entry '1657595100' for key >> 'PRIMARY'")Oct 30 18:43:08 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 05:10:00 CEST (1657595400) >> to database 'weewx': (1062, " Duplicate entry '1657595400' for key >> 'PRIMARY'")Oct 30 18:48:08 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 05:15:00 CEST (1657595700) >> to database 'weewx': (1062, " Duplicate entry '1657595700' for key >> 'PRIMARY'")Oct 30 18:53:08 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 05:20:00 CEST (1657596000) >> to database 'weewx': (1062, " Duplicate entry '1657596000' for key >> 'PRIMARY'")Oct 30 18:58:09 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 05:25:00 CEST (1657596300) >> to database 'weewx': (1062, " Duplicate entry '1657596300' for key >> 'PRIMARY'")Oct 30 19:03:09 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 05:30:00 CEST (1657596600) >> to database 'weewx': (1062, " Duplicate entry '1657596600' for key >> 'PRIMARY'")Oct 30 19:08:09 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 05:35:00 CEST (1657596900) >> to database 'weewx': (1062, "Duplicate entry '1657596900' for key >> 'PRIMARY'")Oct 30 19:13:09 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 05:40:00 CEST (1657597200) >> to database 'weewx': (1062, " Duplicate entry '1657597200' for key >> 'PRIMARY'")Oct 30 19:18:10 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 05:45:00 CEST (1657597500) >> to database 'weewx': (1062, " Duplicate entry '1657597500' for key >> 'PRIMARY'")Oct 30 19:23:10 gandalf python3[1261]: weewx[1261] ERROR >> weewx.manager: Unable to add record 2022-07-12 05:50:00 CEST (1657597800) >> to database 'weewx': (1062, " Duplicate entry '1657597800' for key >> 'PRIMARY'")* >> >> >> I don't understand why there are Duplicate entry errors on different keys? >> >> Do you have any idea what's going on? >> >> >> Le lundi 30 octobre 2023 à 13:42:32 UTC+1, gjr80 a écrit : >> >>> I have been able to reproduce your symptoms and I now believe I >>> understand what is happening. >>> >>> In terms of why wee_import wants to import from 27 and 28 October Vince >>> has covered the main point. So as far as WeeWX is concerned (and assuming a >>> five minute archive period) the first archive record of the 27 October >>> covers the five minute period from midnight at the start of 27 October >>> until 00:05, so the first archive period of the day is timestamped at five >>> minutes after midnight. The last archive period covers the five minute >>> period from 23:55 until (and including) midnight at the end of 27 October, >>> so the last archive period of 27 October is timestamped at midnight on 28 >>> October. >>> >>> The other factor at play is how WU provides data via its API. The WU API >>> call used by wee_import provides one full day of data at a time, the >>> problem being that WU defines a day as starting at midnight at the start of >>> the day though until (but not including) midnight at the end of the day. So >>> when the WU API responds with data for 27 October the first record provided >>> is the archive record timestamped at midnight at the start of 27 October, >>> this archive record actually contains data from 26 October so it is >>> discarded. The last record provided by WU is timestamped at 23:55 27 >>> October, which is only the second last archive record for the day. WU does >>> not provide the last archive record of the day, to get that archive record >>> we need to ask WU for the data for 28 October. The first record from the WU >>> 28 October data is used and the rest are discarded. >>> >>> That is why when you attempt to import a given day of data from WU you >>> will actually see wee_import obtain two days data from WU. >>> >>> Now for your error message. To cut a long story short there was a flaw >>> in how wee_import calculated the total elapsed time for an import. I >>> believe I have fixed it. You can use the new code by downloading the >>> updated weeimport.py and replacing weeimport.py on your system with the >>> downloaded version. In an earlier post I gave you instructions for >>> replacing wuimport.py, this time you need to replace weeimport.py, it >>> is in the same directory as wuimport.py and the process is the same >>> just a different file. You can download the updated weeimport.py via >>> the following command: >>> >>> $ wget -P /var/tmp >>> https://raw.githubusercontent.com/weewx/weewx/V5/src/weeimport/weeimport.py >>> >>> Do not get weeimport.py confused with wee_import.py, they are in >>> different directories but if you replace the wrong file you will break >>> wee_import. >>> >>> Once you have replaced weeimport.py wee_import should work without >>> error. >>> >>> Gary >>> >>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/0a2feb3c-6b7d-451e-8019-fe8795008c8en%40googlegroups.com.
