Tom, apologies for my misunderstanding. You were looking for the syslog I believe... Here it is a fresh one. Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Initializing weectl version 5.1.0 Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Command line: /usr/share/weewx/weectl.py database rebuild-daily Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Using Python: 3.10.12 (main, Nov 6 2024, 20:22:13) [GCC 11.4.0] Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Located at: /usr/bin/python3 Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Platform: Linux-6.8.0-47-generic-x86_64-with-glibc2.35 Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Locale: 'LC_CTYPE=en_US.UTF-8;LC_NUMERIC=sl_SI.UTF-8;LC_TIME=sl_SI.UTF-8;LC_COLLATE=en_US.UTF-8;LC_MONETARY=sl_SI.UTF-8;LC_MESSAGES=en_US.UTF-8;LC_PAPER=sl_SI.UTF-8;LC_NAME=sl_SI.UTF-8;LC_ADDRESS=sl_SI.UTF-8;LC_TELEPHONE=sl_SI.UTF-8;LC_MEASUREMENT=sl_SI.UTF-8;LC_IDENTIFICATION=sl_SI.UTF-8' Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Entry path: /usr/share/weewx/weectl.py Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: WEEWX_ROOT: /etc/weewx Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Config file: /etc/weewx/weewx.conf Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: User module: /etc/weewx/bin/user Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Debug: 0 Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: User: root Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Group: root Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Groups: root Dec 26 07:50:21 hostname weectl[309641]: INFO weectl-database: All daily summaries will be rebuilt. Dec 26 07:50:23 hostname weectl[309641]: INFO weectl-database: Rebuilding daily summaries in database 'weewx.sdb' ... Dec 26 07:50:23 hostname weectl[309641]: INFO weewx.manager: Daily summaries at V2.0. Patching to V4.0 Dec 26 07:50:23 hostname weectl[309641]: INFO weewx.manager: recalculate_weights: Using database 'weewx.sdb' Dec 26 07:51:34 hostname weectl[309641]: ERROR weewx.manager: No column error: no such column: inf Dec 26 07:51:34 hostname weectl[309641]: ERROR weewx.manager: *** sql statement: 'UPDATE archive_day_soilTemp1 SET sum=inf, count=1434, wsum=inf, sumtime=86040.0 WHERE dateTime = ?;' Dec 26 07:51:34 hostname weectl[309641]: ERROR weewx.manager: *** start time=1628719200.0
četrtek, 26. december 2024 ob 01:35:53 UTC+1 je oseba Tom Keffer napisala: > I am not seeing the log entries from the instrumented code that I > included. It will tell us much more about what is going on. Please include > it. > > For the record, I was also able to rebuild the daily summaries > uneventfully using your dumped archive table. > > But, it occurred to me, midnight here on the west coast of the USA (UTC-7) > is a different time zone from you, which I assume is CEST. I set up a VM > using CEST but, again, it rebuilt the summaries uneventfully. > > Really need to see the log. > > -tk > > > On Wed, Dec 25, 2024 at 2:40 PM mihec <pete...@gmail.com> wrote: > >> Thanks for checking. I ran the command again and monitored memfree but it >> was nothing to be worried about. >> Here is how I ran the command (note the from-to and where the processing >> was when I captured this status): >> >> username@hostname:~$ sudo weectl database rebuild-daily --from=2021-01-01 >> --to=2022-01-01 >> Using configuration file /etc/weewx/weewx.conf >> Daily summaries from 2021-01-01 through 2022-01-01, inclusive, will be >> rebuilt. >> Rebuild the daily summaries in the database 'weewx.sdb' (y/n)? y >> Rebuilding daily summaries in database 'weewx.sdb' ... >> Daily summaries at V2.0. Patching to V4.0 >> Processed through: 2020-08-28 00:00:00 CEST (1598565600) >> >> It again fails at the same point. I'm sorry to bother you so much with >> this. >> I just tried on a laptop where I run weewx 4.10. The 'wee_database >> --rebuild-daily' fails with same error at same point as on the other two >> systems. >> For the record, the RPi is 3B+ running bullseye 32-bit, my desktop is old >> i7-6700 3.4GHz with 8 GB, running Ubuntu 22.04.1. Laptop is i7-7500 2.70 >> GHz with 8 GB running Ubuntu 22.04.1. >> >> >> sreda, 25. december 2024 ob 23:12:39 UTC+1 je oseba Tom Keffer napisala: >> >>> I was thinking the same. >>> >>> On Wed, Dec 25, 2024 at 2:10 PM vince <vince...@gmail.com> wrote: >>> >>>> Thank you. I built a db from it and ran rebuild-daily and it worked >>>> just fine with no errors in 2.5 minutes. No errors running database >>>> check. No errors running database update. >>>> >>>> Computer was a m1 mac mini with 16 GB of RAM. I wonder if you're >>>> running out of memory and need to split your rebuild-daily up into pieces >>>> with the --from and --to options. Just a guess. >>>> >>>> >>>> On Wednesday, December 25, 2024 at 1:56:50 PM UTC-8 mihec wrote: >>>> >>>>> This is the dump: https://we.tl/t-X3r6gbimBx >>>>> >>>>> sreda, 25. december 2024 ob 22:48:59 UTC+1 je oseba mihec napisala: >>>>> >>>>>> I replaced the original manager.py with attached one. That file is in >>>>>> the /usr/share/weewx/weewx/ directory. >>>>>> I will prepare the dump of the database. >>>>>> >>>>>> username@hostname:~$ sudo weectl database rebuild-daily >>>>>> >>>>>> Using configuration file /etc/weewx/weewx.conf >>>>>> All daily summaries will be rebuilt. >>>>>> Rebuild the daily summaries in the database 'weewx.sdb' (y/n)? y >>>>>> Rebuilding daily summaries in database 'weewx.sdb' ... >>>>>> Daily summaries at V2.0. Patching to V4.0 >>>>>> Traceback (most recent call last):0:00 CEST (1628719200) >>>>>> File "/usr/share/weewx/weedb/sqlite.py", line 38, in guarded_fn >>>>>> return fn(*args, **kwargs) >>>>>> File "/usr/share/weewx/weedb/sqlite.py", line 233, in execute >>>>>> return sqlite3.Cursor.execute(self, *args, **kwargs) >>>>>> sqlite3.OperationalError: no such column: inf >>>>>> >>>>>> During handling of the above exception, another exception occurred: >>>>>> >>>>>> Traceback (most recent call last): >>>>>> File "/usr/share/weewx/weectl.py", line 75, in <module> >>>>>> main() >>>>>> File "/usr/share/weewx/weectl.py", line 67, in main >>>>>> namespace.func(namespace) >>>>>> File "/usr/share/weewx/weectllib/__init__.py", line 90, in dispatch >>>>>> namespace.action_func(config_dict, namespace) >>>>>> File "/usr/share/weewx/weectllib/database_cmd.py", line 329, in >>>>>> rebuild_daily >>>>>> weectllib.database_actions.rebuild_daily(config_dict, >>>>>> File "/usr/share/weewx/weectllib/database_actions.py", line 134, in >>>>>> rebuild_daily >>>>>> with weewx.manager.open_manager(manager_dict, initialize=True) as >>>>>> dbm: >>>>>> File "/usr/share/weewx/weewx/manager.py", line 924, in open_manager >>>>>> return manager_cls.open_with_create(manager_dict['database_dict'], >>>>>> File "/usr/share/weewx/weewx/manager.py", line 244, in >>>>>> open_with_create >>>>>> dbmanager = cls(connection, table_name=table_name, schema=schema) >>>>>> File "/usr/share/weewx/weewx/manager.py", line 1076, in __init__ >>>>>> self.patch_sums() >>>>>> File "/usr/share/weewx/weewx/manager.py", line 1537, in patch_sums >>>>>> >>>>>> self.recalculate_weights(start_d=datetime.date(2020, 6, 1)) >>>>>> File "/usr/share/weewx/weewx/manager.py", line 1451, in >>>>>> recalculate_weights >>>>>> self._do_tranche(mark_d, end_of_tranche_d, weight_fn, progress_fn) >>>>>> File "/usr/share/weewx/weewx/manager.py", line 1490, in _do_tranche >>>>>> self._set_day_sums(day_accum, cursor) >>>>>> File "/usr/share/weewx/weewx/manager.py", line 1516, in >>>>>> _set_day_sums >>>>>> >>>>>> cursor.execute(update_sql, (day_accum.timespan.start,)) >>>>>> File "/usr/share/weewx/weedb/sqlite.py", line 50, in guarded_fn >>>>>> raise weedb.NoColumnError(e) >>>>>> weedb.NoColumnError: no such column: inf >>>>>> >>>>>> >>>>>> sreda, 25. december 2024 ob 21:13:17 UTC+1 je oseba Tom Keffer >>>>>> napisala: >>>>>> >>>>>>> Thanks. That tells me what I needed to know to create a useful >>>>>>> instrumented version of manager.py. Replace the one you have with the >>>>>>> attached version. It will give more information about exactly what >>>>>>> created >>>>>>> the error. Post the log. >>>>>>> >>>>>>> -tk >>>>>>> >>>>>>> On Wed, Dec 25, 2024 at 9:29 AM mihec <pete...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi, first I want to wish you Merry Christmas and Happy Holidays! >>>>>>>> >>>>>>>> This is the output of the "weectl database rebuild-daily". Please >>>>>>>> note that I've copied the weewx.sdb database file to a desktop PC to >>>>>>>> make >>>>>>>> it run much faster. I've replaced the real username and hostname with >>>>>>>> something generic. The weewx's version is same as in RPi. >>>>>>>> >>>>>>>> When executed, the 'rebuild' process starts around June 2020 and >>>>>>>> goes on normally until it fails on August 12, 2021 (epoch >>>>>>>> 1628719200). In the output below, you only see the status when it >>>>>>>> fails already. I can provide with a screenshot if that helps. >>>>>>>> The thing is that any command on the database (update, check or >>>>>>>> rebuild) fails on the same date (record?). >>>>>>>> >>>>>>>> username@host:~$ ls -la weewx.sdb >>>>>>>> -rwxrwxr-x 1 username username 682086400 dec 25 18:18 weewx.sdb >>>>>>>> username@host:~$ weewxd -v >>>>>>>> 5.1.0 >>>>>>>> username@host:~$ weectl -v >>>>>>>> weectl 5.1.0 >>>>>>>> username@host:~$ sudo weectl database rebuild-daily >>>>>>>> [sudo] password for username: >>>>>>>> Using configuration file /etc/weewx/weewx.conf >>>>>>>> All daily summaries will be rebuilt. >>>>>>>> Rebuild the daily summaries in the database 'weewx.sdb' (y/n)? y >>>>>>>> Rebuilding daily summaries in database 'weewx.sdb' ... >>>>>>>> Daily summaries at V2.0. Patching to V4.0 >>>>>>>> Traceback (most recent call last):0:00 CEST (1628719200) >>>>>>>> File "/usr/share/weewx/weedb/sqlite.py", line 38, in guarded_fn >>>>>>>> return fn(*args, **kwargs) >>>>>>>> File "/usr/share/weewx/weedb/sqlite.py", line 233, in execute >>>>>>>> return sqlite3.Cursor.execute(self, *args, **kwargs) >>>>>>>> sqlite3.OperationalError: no such column: inf >>>>>>>> >>>>>>>> During handling of the above exception, another exception occurred: >>>>>>>> >>>>>>>> Traceback (most recent call last): >>>>>>>> File "/usr/share/weewx/weectl.py", line 75, in <module> >>>>>>>> main() >>>>>>>> File "/usr/share/weewx/weectl.py", line 67, in main >>>>>>>> namespace.func(namespace) >>>>>>>> File "/usr/share/weewx/weectllib/__init__.py", line 90, in >>>>>>>> dispatch >>>>>>>> namespace.action_func(config_dict, namespace) >>>>>>>> File "/usr/share/weewx/weectllib/database_cmd.py", line 329, in >>>>>>>> rebuild_daily >>>>>>>> weectllib.database_actions.rebuild_daily(config_dict, >>>>>>>> File "/usr/share/weewx/weectllib/database_actions.py", line 134, >>>>>>>> in rebuild_daily >>>>>>>> with weewx.manager.open_manager(manager_dict, initialize=True) >>>>>>>> as dbm: >>>>>>>> File "/usr/share/weewx/weewx/manager.py", line 924, in >>>>>>>> open_manager >>>>>>>> return >>>>>>>> manager_cls.open_with_create(manager_dict['database_dict'], >>>>>>>> File "/usr/share/weewx/weewx/manager.py", line 244, in >>>>>>>> open_with_create >>>>>>>> dbmanager = cls(connection, table_name=table_name, >>>>>>>> schema=schema) >>>>>>>> File "/usr/share/weewx/weewx/manager.py", line 1076, in __init__ >>>>>>>> self.patch_sums() >>>>>>>> File "/usr/share/weewx/weewx/manager.py", line 1531, in patch_sums >>>>>>>> self.recalculate_weights(start_d=datetime.date(2020, 6, 1)) >>>>>>>> File "/usr/share/weewx/weewx/manager.py", line 1451, in >>>>>>>> recalculate_weights >>>>>>>> self._do_tranche(mark_d, end_of_tranche_d, weight_fn, >>>>>>>> progress_fn) >>>>>>>> File "/usr/share/weewx/weewx/manager.py", line 1490, in >>>>>>>> _do_tranche >>>>>>>> self._set_day_sums(day_accum, cursor) >>>>>>>> File "/usr/share/weewx/weewx/manager.py", line 1515, in >>>>>>>> _set_day_sums >>>>>>>> cursor.execute(update_sql, (day_accum.timespan.start,)) >>>>>>>> File "/usr/share/weewx/weedb/sqlite.py", line 50, in guarded_fn >>>>>>>> raise weedb.NoColumnError(e) >>>>>>>> weedb.NoColumnError: no such column: inf >>>>>>>> username@host:~$ >>>>>>>> >>>>>>>> torek, 24. december 2024 ob 15:15:16 UTC+1 je oseba Tom Keffer >>>>>>>> napisala: >>>>>>>> >>>>>>>>> Let's put aside the database update and just focus on rebuilding >>>>>>>>> the daily summaries. >>>>>>>>> >>>>>>>>> Could you please include the stack trace when doing "weectl >>>>>>>>> database rebuild-daily"? Once I see where it fails, I can send you an >>>>>>>>> instrumented version of the database code so we can see exactly what >>>>>>>>> is >>>>>>>>> happening. >>>>>>>>> >>>>>>>>> -tk >>>>>>>>> >>>>>>>>> On Tue, Dec 24, 2024 at 3:39 AM mihec <pete...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Well, I've checked also the weectl database check which fails at >>>>>>>>>> the same point as the update. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> torek, 24. december 2024 ob 12:31:36 UTC+1 je oseba Auchtermuchty >>>>>>>>>> Weather napisala: >>>>>>>>>> >>>>>>>>>>> That's not what it does now. >>>>>>>>>>> >>>>>>>>>>> " Databases created earlier than 3.7.0 (released 11-March-2017) >>>>>>>>>>> have a flaw that prevents them from being used with archive >>>>>>>>>>> intervals that >>>>>>>>>>> change with time. This utility check whether your database is >>>>>>>>>>> affected." >>>>>>>>>>> >>>>>>>>>>> $ weectl database check >>>>>>>>>>> Using configuration file /etc/weewx/weewx.conf >>>>>>>>>>> Checking daily summary tables version... >>>>>>>>>>> Daily summary tables are at version 4.0. >>>>>>>>>>> Interval Weighting Fix is not required. >>>>>>>>>>> $ >>>>>>>>>>> >>>>>>>>>>> On Tuesday, 24 December 2024 at 11:13:48 UTC Graham Eddy wrote: >>>>>>>>>>> >>>>>>>>>>>> weewx used to include a tool for picking up things like a >>>>>>>>>>>> string where a number should be. maybe `weectl database check’ >>>>>>>>>>>> still does >>>>>>>>>>>> this >>>>>>>>>>>> *⊣GE⊢* >>>>>>>>>>>> >>>>>>>>>>>> On 24 Dec 2024, at 4:17 pm, mihec <pete...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>> I think there must be an invalid entry in my database (Aug 12, >>>>>>>>>>>> 2021). My SQLite database management knowledge is basically zero, >>>>>>>>>>>> so I'm >>>>>>>>>>>> looking for advice >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "weewx-user" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to weewx-user+...@googlegroups.com. >>>>>>>>>> >>>>>>>>> To view this discussion visit >>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/e60483cc-3910-45f8-aff4-7b4e95939cd9n%40googlegroups.com >>>>>>>>>> >>>>>>>>>> <https://groups.google.com/d/msgid/weewx-user/e60483cc-3910-45f8-aff4-7b4e95939cd9n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "weewx-user" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to weewx-user+...@googlegroups.com. >>>>>>>> >>>>>>> To view this discussion visit >>>>>>>> https://groups.google.com/d/msgid/weewx-user/e92cf44c-d06f-4de1-ba04-440bceed755fn%40googlegroups.com >>>>>>>> >>>>>>>> <https://groups.google.com/d/msgid/weewx-user/e92cf44c-d06f-4de1-ba04-440bceed755fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "weewx-user" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to weewx-user+...@googlegroups.com. >>>> >>> To view this discussion visit >>>> https://groups.google.com/d/msgid/weewx-user/124fd290-b296-4c56-a4d4-43e7b30ae2d7n%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/weewx-user/124fd290-b296-4c56-a4d4-43e7b30ae2d7n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to weewx-user+...@googlegroups.com. >> > To view this discussion visit >> https://groups.google.com/d/msgid/weewx-user/8fe396c2-d6b2-4aac-a8c8-fc014e9d2df8n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/weewx-user/8fe396c2-d6b2-4aac-a8c8-fc014e9d2df8n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/weewx-user/bb0c8c81-d7b4-401f-8758-5561660d001fn%40googlegroups.com.
