Hmm, that is odd. Using WeeWX for 6 odd years on a number of machines I have always noticed a drop in times. I guess there is an exception to the rule. I guess I need to refactor the SLEs from my highcharts extension but that will not be a quick process. If Pat decides to adopt the revised code he will need to modify his code as he has made a number of changes to mine.
If you are running a 1 minute archive interval (I think you mentioned that earlier?) and have those sorts of execution time you will eventually grind to a halt. When a report cycle comes around WeeWX kicks off a new thread to run the report engine, if that takes longer than the archive interval that thread is still running when the next report thread is launched and the extra load slows things down. The process repeats and eventually it will grind to a halt with too many report threads open. Gary On Wednesday, 29 August 2018 10:37:18 UTC+10, David Hathaway wrote: > > Gary, > I appreciate your response, although much of it is lost on me. However, > the idea of a dramatic difference between first pass and subsequent passes > isn't happening here. You can tell by the pid that there was a restart, > but the times are about the same. > > Aug 28 17:58:35 hathaway weewx[1319]: cheetahgenerator: Generated 16 files > for report Belchertown in 137.68 seconds > Aug 28 18:01:35 hathaway weewx[1319]: cheetahgenerator: Generated 16 files > for report Belchertown in 137.63 seconds > Aug 28 18:04:38 hathaway weewx[1319]: cheetahgenerator: Generated 16 files > for report Belchertown in 137.25 seconds > Aug 28 18:08:35 hathaway weewx[25730]: cheetahgenerator: Generated 16 > files for report Belchertown in 136.85 seconds > Aug 28 18:11:34 hathaway weewx[25730]: cheetahgenerator: Generated 16 > files for report Belchertown in 137.22 seconds > Aug 28 18:14:35 hathaway weewx[25730]: cheetahgenerator: Generated 16 > files for report Belchertown in 138.26 seconds > > Dave > > > > On Tue, Aug 28, 2018 at 7:05 PM gjr80 <[email protected] <javascript:>> > wrote: > >> On Tuesday, 28 August 2018 04:11:36 UTC+10, Pat wrote: >>> >>> Most of the highcharts code is from @gjr80 so he may be able to speak >>> better to it, but looking at how it gets its data, it's using getSqlVectors >>> which looks like it gets its data from the archive table and not a day >>> summary table. >>> >>> >> I realise this is a bit dated in terms of this thread but I missed this >> post. >> >> The original 'highcharts' extension I shared with Pat used getSQLVectors >> throughout which queries the archive and not the daily summaries. From >> memory (and my current highcharts extension has morphed substantially from >> what I first shared with Pat) the day and week SLEs pulled data directly >> from the archive, month and year used an aggregate (hour and day >> respectively I think). This was done for much the same reason as the >> default WeeWX plots plot an aggregate for month and year plots - there is >> just too many data points otherwise. >> >> Also, on the topic of execution times. My original highcharts extension >> used 4 SLEs to produce 4 json format data files using 1 skin with 4 WeeWX >> reports; one report each for day, week, month and year. Each of the SLE >> made a number of calls to getSQLVectors to gather the data vectors for >> the period concerned, These were not done lazily rather they were executed >> in full each time the SLE was called. The way that SLEs work within WeeWX >> is that all SLEs listed in skin.conf are executed for each report in the >> skin. So even with my 4 reports and 4 SLEs there was quite a lot of >> unnecessary SLE execution (16 SLE calls in all when only 4 were actually >> required) going on each with numerous complex SQL queries of the archive. >> If you now magnify that with Pat's extension where there are quite a few >> more reports in the skin that use the highcharts SLEs, the underlying >> queries in the SLEs are being run many times each report cycle. When >> coupled with a database that has a very short archive interval (say sub-5 >> minutes) you magnify the time taken to execute these queries due to the >> extra data points in a given period. >> >> I guess the bottom line is that the highcharts SLEs do not scale well in >> a WeeWX report context, judicious placements of reports/SLEs in skins can >> help. The highcharts SLEs really need to rewritten to be more efficient (eg >> lazy evaluation) but that is a biggish task and has follow on consequences >> for the supported reports. >> >> One quick comment on report times, take note of the time taken for the >> second and subsequent report passes after a restart. If the total time >> taken for the reports is approaching your archive interval you will >> possibly have stability issues. The first report pass after a restart >> always takes longer, in my experience up to twice as long as second and >> subsequent passes. >> >> Gary >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "weewx-user" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/weewx-user/GSrKZEiPqsc/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
