I ran this, and did the recalc-missing, and it seems to be running better. Except there is only one record in those tables.
Not sure why that would cause that issue. weectl database add-column appTemp weectl database add-column cloudbase weectl database add-column visibility weectl database add-column windrun weectl database add-column cloud_cover weectl database add-column aqi weectl calc-missing On Tuesday, May 28, 2024 at 8:10:45 PM UTC-5 Mark Jenks wrote: > Seasons itself doesn't do it, and Belchertown with no settings on it is > doing it. > So, I'm adding those columns now and recalculating now. > > > On Tuesday, May 28, 2024 at 1:15:56 PM UTC-5 mh081...@gmail.com wrote: > >> I think this was the Problem with missing Database extended schema >> object. I had the same problem as i upgraded from weewx 4 to 5. I solved >> this with. >> >> weectl database add-column appTemp >> weectl database add-column cloudbase >> weectl database add-column visibility >> weectl database add-column windrun >> weectl database add-column cloud_cover >> weectl database add-column aqi >> weectl calc-missing >> weectl calc-missing appTemp >> >> vince schrieb am Dienstag, 28. Mai 2024 um 19:38:07 UTC+2: >> >>> The other test would be to disable belchertown, enable seasons, and see >>> if that’s ok. That would check the db vs. the v5 calculated aka synthetic >>> readings feature that slowed some folks down. >>> >>> On Tuesday, May 28, 2024 at 10:25:10 AM UTC-7 Tom Keffer wrote: >>> >>>> I thought of the appTemp angle, but those queries involve thousands of >>>> small aggregate intervals, not one query over 4 years. >>>> >>>> Still, it's worth a try. >>>> >>>> On Tue, May 28, 2024 at 9:07 AM vince <vince...@gmail.com> wrote: >>>> >>>>> You seem to be hacking randomly and praying. I'd suggest deleting >>>>> Belchertown at this point, verifying it is deleted, then add it back in. >>>>> Do not customize it yet. Just run the default setup, it works. Really. >>>>> >>>>> My guess is you are missing the appTemp element in the db. There are >>>>> a bunch of threads on this re: Belchertown and v5 weewx. >>>>> >>>>> Check your mariadb schema and see if it is missing appTemp. If you >>>>> have the old wview-compatible db you'd have about 50 elements in it. If >>>>> you later switched to the v4 and above bigger wview-extended schema, >>>>> you'd >>>>> have 114 elements in it. Sorry but I don't know the mariadb commands to >>>>> count them nor check. >>>>> >>>>> For sqlite3 it would be: 'echo ".schema" | sqlite3 mydbname | wc -l' >>>>> >>>>> On Tuesday, May 28, 2024 at 3:59:19 AM UTC-7 Mark Jenks wrote: >>>>> >>>>>> I removed the last two from skin.conf, and it still does the large >>>>>> query. I'll keep digging tonight. >>>>>> >>>>>> generator_list = weewx.cheetahgenerator.CheetahGenerator, >>>>>> weewx.reportengine.CopyGenerator, >>>>>> user.belchertown.HighchartsJsonGenerator >>>>>> >>>>>> On Monday, May 27, 2024 at 10:06:02 PM UTC-5 Mark Jenks wrote: >>>>>> >>>>>>> false alarm so far. mqtt is tied to belchertown. So, seems to be >>>>>>> belchertown like you suspected. >>>>>>> >>>>>>> Digging more tomorrow. >>>>>>> >>>>>>> >>>>>>> On Monday, May 27, 2024 at 10:00:15 PM UTC-5 Mark Jenks wrote: >>>>>>> >>>>>>>> I just ripped mqtt out also. I'll check it in the morning and see >>>>>>>> if that makes a difference. >>>>>>>> >>>>>>>> >>>>>>>> On Monday, May 27, 2024 at 9:52:21 PM UTC-5 Mark Jenks wrote: >>>>>>>> >>>>>>>>> Bad news? I uninstalled belchertown, and it's still hitting high >>>>>>>>> CPU on mariadb and python3. This is the hit to the DB. >>>>>>>>> >>>>>>>>> SELECT >>>>>>>>> <http://phpmyadmin.mjenks.net/url.php?url=https://dev.mysql.com/doc/refman/8.0/en/select.html> >>>>>>>>> * FROM archive WHERE dateTime > 1704088800 AND >>>>>>>>> <http://phpmyadmin.mjenks.net/url.php?url=https://dev.mysql.com/doc/refman/8.0/en/logical-operators.html%23operator_and> >>>>>>>>> dateTime <= 1735711200 ORDER BY dateTime ASC >>>>>>>>> >>>>>>>>> >>>>>>>>> On Monday, May 27, 2024 at 7:16:36 PM UTC-5 Tom Keffer wrote: >>>>>>>>> >>>>>>>>>> That query is asking for every single record over 4 years of data >>>>>>>>>> --- about 400,000+ records with a 5 minute archive interval. That's >>>>>>>>>> not the >>>>>>>>>> use pattern when an xtype is causing the problem >>>>>>>>>> <https://github.com/weewx/weewx/wiki/XTypes-performance>. >>>>>>>>>> >>>>>>>>>> It's hard to imagine why the Belchertown skin would need data at >>>>>>>>>> that density. Try isolating the problem by shutting off the >>>>>>>>>> imagegenerator, >>>>>>>>>> then the cheetahgenerator (you can do that by modifying >>>>>>>>>> generator_list in >>>>>>>>>> skin.conf). >>>>>>>>>> >>>>>>>>>> Then once you know which one is the culprit, then start trimming >>>>>>>>>> their respective sections in skin.conf until you isolate the plot or >>>>>>>>>> tag >>>>>>>>>> that is causing the problem. >>>>>>>>>> >>>>>>>>>> On Mon, May 27, 2024 at 4:47 PM Mark Jenks <mjenk...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Just built a new weewx on my Fedora 38 this morning, and >>>>>>>>>>> attached it to my mariadb. It has 13 years worth of data in it. >>>>>>>>>>> >>>>>>>>>>> I am running Belchertown and mqtt, and installed Windy. But I >>>>>>>>>>> just removed Windy to see if that was it, but still does it. >>>>>>>>>>> Python and/or >>>>>>>>>>> moriadb goes to 100% CPU. >>>>>>>>>>> Mariadb says the high query during that time is: SELECT * FROM >>>>>>>>>>> archive WHERE dateTime > 1320346500 AND dateTime <= 1716824400 >>>>>>>>>>> ORDER BY >>>>>>>>>>> dateTime ASC >>>>>>>>>>> But that makes no sense, since I can query my entire archive >>>>>>>>>>> table (select *) in about 3 seconds. >>>>>>>>>>> >>>>>>>>>>> Any thoughts on how to catch what is going on? I could enable >>>>>>>>>>> debug, but hoping for a better way other that looking at a ton of >>>>>>>>>>> logs. >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "weewx-user" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to weewx-user+...@googlegroups.com. >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/a882fb44-d36b-4bf4-8588-41199524b5f7n%40googlegroups.com >>>>>>>>>>> >>>>>>>>>>> <https://groups.google.com/d/msgid/weewx-user/a882fb44-d36b-4bf4-8588-41199524b5f7n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to weewx-user+...@googlegroups.com. >>>>> >>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/weewx-user/e42cdc71-32a2-4d36-9169-f2c195ccf28dn%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/weewx-user/e42cdc71-32a2-4d36-9169-f2c195ccf28dn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ae663ace-fe2d-4951-be0f-2fbe54c57944n%40googlegroups.com.
