Was there anything after the last two lines? The 'calculate missing' part of the log is where there may be a clue. The 'UNIQUE constraint failed' error messages are expected and can be ignored; once WeeWX has written a given record to archive any attempt to write the same record again will result in such an error.
Gary On Tuesday 23 July 2024 at 19:16:04 UTC+10 sebastian...@googlemail.com wrote: > Hey, attached is an excerpt from journalctl. > > Greetings Sebastian > > gjr80 schrieb am Dienstag, 23. Juli 2024 um 10:41:51 UTC+2: > >> OK. At the moment my best guess is that when weectl import calculates >> missing derived obs one of the GTS derived obs is not being calculated >> cleanly/causing a process to remain in memory. Over time the overall memory >> use increases until the system hangs/crashes due to lack of free memory. A >> restart relieves the symptoms by clearing the old weectl import >> processes and the cycle repeats. weectl import is completing execution >> (the printing of the two lines starting 'Those records with a timestamp >> already in the archive will not have been' are the last thing weectl >> import does), but I suspect the thread launched by the GTS type is >> continues once weectl import has otherwise finished. >> >> You still haven't provided any log extracts, my suggestion is to edit >> weewx.conf, set debug = 1 then save weewx.conf. The next time your cron >> job runs weectl import it should use debug = 1 and there may be some >> clue in the log as to what is happening. If your system is using >> systemd-journald for logging you may need to poke around using journalctl >> to find the weectl import log output, this section >> <https://github.com/weewx/weewx/wiki/view-logs#journalctl> of the view >> logs wiki page <https://github.com/weewx/weewx/wiki/view-logs> may help. >> You might want to start with sudo journalctl -t weectl. >> >> Gary >> >> On Tuesday 23 July 2024 at 17:02:33 UTC+10 sebastian...@googlemail.com >> wrote: >> >>> Hello, first of all thank you very much for your help. >>> >>> Here is the content of htop. There I see that the import has started >>> several times. According to crontab it starts every 15 minutes. Because it >>> doesn't seem to finish, there are so many views. Just my guess. >>> >>> gjr80 schrieb am Dienstag, 23. Juli 2024 um 07:47:50 UTC+2: >>> >>>> So if ctl-C is being used to interrupt a script, the script must be >>>> being run from the command line; ctl-C will have no effect on scripts >>>> executed by cron. weectl import does not directly use threading, >>>> however, weectl import uses a number of WeeWX 'API' calls to do things >>>> (save data to database and calculate missing/derived obs are the two main >>>> ones). I would suggest this use of threading is possibly due to an xtype >>>> call when calculating missing/derived obs. weectl import does not >>>> 'load' any extensions per se; however, that is not to say that some other >>>> parts of WeeWX called by weectl import do not. Again an xtype being >>>> used to calculate missing/derived obs is an obvious candidate. I don't see >>>> the 'area 30 31' output being direct weectl import output, quite >>>> possibly it is output from some other extension. >>>> >>>> To start tracking this down we need to go back to basics with log and >>>> config. What does the WeeWX log shown when weectl import is run? And what >>>> is in weewx.conf (and to a lesser extent) the import config file >>>> /etc/weewx/import/wu-example.conf. >>>> >>>> Gary >>>> >>>> On Tuesday 23 July 2024 at 15:08:09 UTC+10 kk44...@gmail.com wrote: >>>> >>>>> Gary, >>>>> >>>>> may be, it is a little bit of a language problem. I heard of this case >>>>> before, so I may add some explanation, but I cannot explain all. >>>>> >>>>> The user seems to call weectl import from the command line or from >>>>> crontab to retrieve data from Wunderground. The command first runs as >>>>> expected. Data is imported and saved to the database. But then, the >>>>> command >>>>> does not end. It hangs. So I suggested to the user to press CTRL-C >>>>> several >>>>> times. This resulted in an error message saying that the command hang in >>>>> the threading module while waiting for a lock. >>>>> >>>>> First question: Where does weectl import use threading? Or does this >>>>> point to the problem that threading is involved at all? >>>>> >>>>> I had a short glance at the sources, but I did not see it. >>>>> >>>>> Then, there are some hints that third party extensions are involved in >>>>> the problem. Does weectl import load WeeWX extensions? I do not see a >>>>> reason to do so. But the output "area 30 31" at the end of the first >>>>> post is from an extension, not from the command itself. >>>>> >>>>> Karen >>>>> >>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/49adcf2b-9ce6-4a42-8def-fbb5f857ee83n%40googlegroups.com.
