I have renewed the API. Also changed the skin to standard. And retried many times to import without succes (both with conf and with derive). I can login normally to WU My website and WU are updated with my current data. It is so strange i could import 2021, but not the subsequent years of data from WU. I am posting the terminal output, but it does not give much info as you can see. And also the end of my many efforts to import the WU data.
Unique records processed: 96; Last timestamp: 2022-02-25 23:45:53 CET (1645829153) Period 57 ... Unique records processed: 96; Last timestamp: 2022-02-26 23:45:54 CET (1645915554) Period 58 ... Unique records processed: 96; Last timestamp: 2022-02-27 23:45:53 CET (1646001953) Period 59 ... Unique records processed: 96; Last timestamp: 2022-02-28 23:45:53 CET (1646088353) Period 60 ... Period 60 - no records identified for import. Calculating missing derived observations ... Processing record: 5648; Last record: 2022-03-01 00:00:00 CET (1646089200) Recalculating daily summaries... **** Daily summaries are not complete. Try again without from/to dates. **** Nothing done, exiting. (weewx-venv) avd@raspberrypi:~ $ source ~/weewx-venv/bin/activate (weewx-venv) avd@raspberrypi:~ $ weectl database rebuild-daily Using configuration file /home/avd/weewx-data/weewx.conf All daily summaries will be rebuilt. Rebuild the daily summaries in the database 'weewx.sdb' (y/n)? y Rebuilding daily summaries in database 'weewx.sdb' ... Daily summaries up to date in 'weewx.sdb'. Op dinsdag 18 februari 2025 om 03:37:24 UTC+1 schreef gjr80: > The speed of the rebuild is not the issue, the speed will vary depending > on the number of archive records that need to be read to do the rebuild. > Better to post the console output than try to describe it, the console > output will give details on the number of days rebuilt, the number of > records used and the time taken. > > As for the 401 error, that is indicative of an incorrect API key, though > if you completed a previous WU import using the same import config file > perhaps not. In any case you might like to check your API key in your > import config file against what is recorded on your WU account. The other > possibility is WU misbehaviour; WU has had a long history of random > dropouts, failures and generally not working at times. In that case all you > can do is wait and try again later. > > Gary > On Tuesday, 18 February 2025 at 07:19:13 UTC+10 Artvd wrote: > >> Thank for your reaction. I run the rebuild but it went in a slit >> second. Indeed the data was in the archive and i therefor tried an second >> import with only changing the dates. The first time it was confiirmed to be >> OK but there was no data in the archive. Again with only changing the dates >> i get the following message from the terminal. >> >> eectl import --import-config=/var/tmp/wu.conf --from=2023-01-01T00:00 >> --to=2023-01-06T00:00 >> Using configuration file /home/avd/weewx-data/weewx.conf >> Starting weectl import... >> Observation history for Weather Underground station 'IVAUXS11' will be >> imported. >> Using database binding 'wx_binding', which is bound to database >> 'weewx.sdb' >> Destination table 'archive' unit system is '0x01' (US). >> Imported records will not overwrite existing database records. >> Missing derived observations will be calculated. >> Observations timestamped after 2023-01-01 00:00:00 CET (1672527600) and >> up to and >> including 2023-01-06 00:00:00 CET (1672959600) will be imported. >> >> Starting import ... >> Records covering multiple periods have been identified for import. >> Period 1 ... >> Unable to open Weather Underground station IVAUXS11 or HTTP Error 401: >> Unauthorized >> **** Unable to load config file. >> **** HTTP Error 401: Unauthorized >> Traceback (most recent call last): >> File >> "/home/avd/weewx-venv/lib/python3.11/site-packages/weectllib/import_actions.py", >> >> line 58, in obs_import >> source_obj.run() >> File >> "/home/avd/weewx-venv/lib/python3.11/site-packages/weeimport/weeimport.py", >> line 367, in run >> _raw_data = self.get_raw_data(period) >> ^^^^^^^^^^^^^^^^^^^^^^^^^ >> File >> "/home/avd/weewx-venv/lib/python3.11/site-packages/weeimport/wuimport.py", >> line 322, in get_raw_data >> response = urllib.request.urlopen(request_obj) >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> File "/usr/lib/python3.11/urllib/request.py", line 216, in urlopen >> return opener.open(url, data, timeout) >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> File "/usr/lib/python3.11/urllib/request.py", line 525, in open >> response = meth(req, response) >> ^^^^^^^^^^^^^^^^^^^ >> File "/usr/lib/python3.11/urllib/request.py", line 634, in http_response >> response = self.parent.error( >> ^^^^^^^^^^^^^^^^^^ >> File "/usr/lib/python3.11/urllib/request.py", line 563, in error >> return self._call_chain(*args) >> ^^^^^^^^^^^^^^^^^^^^^^^ >> File "/usr/lib/python3.11/urllib/request.py", line 496, in _call_chain >> result = func(*args) >> ^^^^^^^^^^^ >> File "/usr/lib/python3.11/urllib/request.py", line 643, in >> http_error_default >> raise HTTPError(req.full_url, code, msg, hdrs, fp) >> urllib.error.HTTPError: HTTP Error 401: Unauthorized >> >> During handling of the above exception, another exception occurred: >> >> Traceback (most recent call last): >> File "/home/avd/weewx-venv/bin/weectl", line 8, in <module> >> sys.exit(main()) >> ^^^^^^ >> File "/home/avd/weewx-venv/lib/python3.11/site-packages/weectl.py", >> line 67, in main >> namespace.func(namespace) >> File >> "/home/avd/weewx-venv/lib/python3.11/site-packages/weectllib/__init__.py", >> line 90, in dispatch >> namespace.action_func(config_dict, namespace) >> File >> "/home/avd/weewx-venv/lib/python3.11/site-packages/weectllib/import_cmd.py", >> line 85, in import_func >> weectllib.import_actions.obs_import(config_dict, >> File >> "/home/avd/weewx-venv/lib/python3.11/site-packages/weectllib/import_actions.py", >> >> line 110, in obs_import >> log.info(f"**** " % e) >> ~~~~~~~~~^~~ >> TypeError: not all arguments converted during string formatting >> (weewx-venv) avd@raspberrypi:~ $ weectl import >> --import-config=/var/tmp/wu.conf --from=2021-01-01T00:00 >> --to=2022-01-06T00:00 >> Using configuration file /home/avd/weewx-data/weewx.conf >> Starting weectl import... >> Observation history for Weather Underground station 'IVAUXS11' will be >> imported. >> Using database binding 'wx_binding', which is bound to database >> 'weewx.sdb' >> Destination table 'archive' unit system is '0x01' (US). >> Imported records will not overwrite existing database records. >> Missing derived observations will be calculated. >> Observations timestamped after 2021-01-01 00:00:00 CET (1609455600) and >> up to and >> including 2022-01-06 00:00:00 CET (1641423600) will be imported. >> >> Starting import ... >> Records covering multiple periods have been identified for import. >> Period 1 ... >> Unable to open Weather Underground station IVAUXS11 or HTTP Error 401: >> Unauthorized >> **** Unable to load config file. >> **** HTTP Error 401: Unauthorized >> Traceback (most recent call last): >> File >> "/home/avd/weewx-venv/lib/python3.11/site-packages/weectllib/import_actions.py", >> >> line 58, in obs_import >> source_obj.run() >> File >> "/home/avd/weewx-venv/lib/python3.11/site-packages/weeimport/weeimport.py", >> line 367, in run >> _raw_data = self.get_raw_data(period) >> ^^^^^^^^^^^^^^^^^^^^^^^^^ >> File >> "/home/avd/weewx-venv/lib/python3.11/site-packages/weeimport/wuimport.py", >> line 322, in get_raw_data >> response = urllib.request.urlopen(request_obj) >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> File "/usr/lib/python3.11/urllib/request.py", line 216, in urlopen >> return opener.open(url, data, timeout) >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> File "/usr/lib/python3.11/urllib/request.py", line 525, in open >> response = meth(req, response) >> ^^^^^^^^^^^^^^^^^^^ >> File "/usr/lib/python3.11/urllib/request.py", line 634, in http_response >> response = self.parent.error( >> ^^^^^^^^^^^^^^^^^^ >> File "/usr/lib/python3.11/urllib/request.py", line 563, in error >> return self._call_chain(*args) >> ^^^^^^^^^^^^^^^^^^^^^^^ >> File "/usr/lib/python3.11/urllib/request.py", line 496, in _call_chain >> result = func(*args) >> ^^^^^^^^^^^ >> File "/usr/lib/python3.11/urllib/request.py", line 643, in >> http_error_default >> raise HTTPError(req.full_url, code, msg, hdrs, fp) >> urllib.error.HTTPError: HTTP Error 401: Unauthorized >> >> During handling of the above exception, another exception occurred: >> >> Traceback (most recent call last): >> File "/home/avd/weewx-venv/bin/weectl", line 8, in <module> >> sys.exit(main()) >> ^^^^^^ >> File "/home/avd/weewx-venv/lib/python3.11/site-packages/weectl.py", >> line 67, in main >> namespace.func(namespace) >> File >> "/home/avd/weewx-venv/lib/python3.11/site-packages/weectllib/__init__.py", >> line 90, in dispatch >> namespace.action_func(config_dict, namespace) >> File >> "/home/avd/weewx-venv/lib/python3.11/site-packages/weectllib/import_cmd.py", >> line 85, in import_func >> weectllib.import_actions.obs_import(config_dict, >> File >> "/home/avd/weewx-venv/lib/python3.11/site-packages/weectllib/import_actions.py", >> >> line 110, in obs_import >> log.info(f"**** " % e) >> ~~~~~~~~~^~~ >> TypeError: not all arguments converted during string formatting >> >> Op maandag 17 februari 2025 om 21:56:43 UTC+1 schreef gjr80: >> >>> It appears that your daily summaries have somehow ended up in a state >>> where they are out of sync with the archive records in the database. When >>> the daily summaries are in this state the partial update of the daily >>> summaries that occurs after missing derived records are calculated is >>> prevented from being run. This is what you are seeing. The solution is to >>> rebuild your daily summaries separately before re-importing the data >>> concerned. To do this try running: >>> >>> weectl database rebuild-daily >>> >>> This will rebuild the daily summaries and should see the daily summaries >>> back in sync with the archive. You should then be able to re-import your WU >>> data. Note the original non-dry-run import should have resulted in the >>> imported WU records being saved to archive (it was just the re-calculated >>> derived obs and daily summary rebuild that failed), so when you re-import >>> the same data you will likely see many 'duplicate record' warnings in the >>> log. These warnings can be ignored, the important thing is the import >>> completes and exits normally. >>> >>> Gary >>> >>> On Monday, 17 February 2025 at 22:55:27 UTC+10 Artvd wrote: >>> >>>> I am trying to import wunderground data with weectl import using the >>>> Weewx 5.1 guidance and standard settings in wu.config >>>> >>>> During dry run there were no problems. But in the " wet" run i get the >>>> message: >>>> "Daily summaries are not complete. Try again without from/to dates" >>>> I would like to import 4 years and hesitate to work without dates. Is >>>> there another possibility for resolve this problem? >>>> Below the relevant part from the terminal. >>>> >>>> $ weectl import --import-config=/var/tmp/wu.conf >>>> --from=2021-01-01T00:00 --to=2021-01-10T00:00 >>>> Using configuration file /home/avd/weewx-data/weewx.conf >>>> Starting weectl import... >>>> Observation history for Weather Underground station 'IVAUXS11' will be >>>> imported. >>>> Using database binding 'wx_binding', which is bound to database >>>> 'weewx.sdb' >>>> Destination table 'archive' unit system is '0x01' (US). >>>> Imported records will not overwrite existing database records. >>>> Missing derived observations will be calculated. >>>> Observations timestamped after 2021-01-01 00:00:00 CET (1609455600) and >>>> up to and >>>> including 2021-01-10 00:00:00 CET (1610233200) will be imported. >>>> Starting import ... >>>> Records covering multiple periods have been identified for import. >>>> Period 1 ... >>>> Proceeding will save all imported records in the WeeWX archive. >>>> Are you sure you want to proceed (y/n)? y >>>> Unique records processed: 74; Last timestamp: 2021-01-01 23:45:54 CET >>>> (1609541154) >>>> Period 2 ... >>>> Unique records processed: 74; Last timestamp: 2021-01-02 23:45:54 CET >>>> (1609627554) >>>> Period 3 ... >>>> Unique records processed: 94; Last timestamp: 2021-01-03 23:45:53 CET >>>> (1609713953) >>>> Period 4 ... >>>> Unique records processed: 96; Last timestamp: 2021-01-04 23:45:54 CET >>>> (1609800354) >>>> Period 5 ... >>>> Unique records processed: 96; Last timestamp: 2021-01-05 23:45:53 CET >>>> (1609886753) >>>> Period 6 ... >>>> Unique records processed: 84; Last timestamp: 2021-01-06 23:45:54 CET >>>> (1609973154) >>>> Period 7 ... >>>> Unique records processed: 70; Last timestamp: 2021-01-07 23:45:53 CET >>>> (1610059553) >>>> Period 8 ... >>>> Unique records processed: 71; Last timestamp: 2021-01-08 23:45:53 CET >>>> (1610145953) >>>> Period 9 ... >>>> Unique records processed: 71; Last timestamp: 2021-01-09 23:45:54 CET >>>> (1610232354) >>>> Period 10 ... >>>> Period 10 - no records identified for import. >>>> Calculating missing derived observations ... >>>> Processing record: 730; Last record: 2021-01-10 00:00:00 CET >>>> (1610233200) >>>> Recalculating daily summaries... >>>> **** Daily summaries are not complete. Try again without from/to dates. >>>> **** Nothing done, exiting. >>>> >>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/weewx-user/f23d33e9-e97a-4c95-b1c0-2b07ff6c626dn%40googlegroups.com.
