Hi Andrew, I finally managed to find some spare time to try out your latest suggestion, sorry for the delay. Here's the result of running those two commands (please note that I used "sudo find" else I get lots of "permission denied" messages from numerous non-related RPi directories). Does this shed any new light on things ? Thanks again, Paul
pi@raspberrypi:~ $ sudo find / -name weewx.conf /etc/weewx/logwatch/conf/services/weewx.conf /etc/weewx/logwatch/conf/logfiles/weewx.conf /etc/weewx/rsyslog.d/weewx.conf /etc/weewx/apache/conf.d/weewx.conf /etc/weewx/weewx.conf find: ‘/run/user/1000/gvfs’: Permission denied pi@raspberrypi:~ $ sudo find / -name weewx.sdb /var/lib/weewx/weewx.sdb find: ‘/run/user/1000/gvfs’: Permission denied pi@raspberrypi:~ $ ==================================================================== On Wednesday, August 1, 2018 at 12:08:31 PM UTC+1, Paul_N wrote: > > Thanks for that suggestion Andrew, sounds very possible ! I'll check it > out later today when I next get a spare moment and will let you know how I > get on. > Thanks again, Paul > > On Wednesday, August 1, 2018 at 11:57:36 AM UTC+1, Andrew Milner wrote: >> >> sorry - last one should have been >> find / -name weewx.sdb >> >> >> >> On Wednesday, 1 August 2018 13:56:46 UTC+3, Andrew Milner wrote: >>> >>> My suspicion is that you have either got two weewx.conf files or two >>> .sdb files and SQLite is looking in one place whilst drop daly is dropping >>> tables from a different .sdb file. >>> >>> You can check by using find >>> find / -name weewx.conf >>> and >>> find / -name archive.sdb >>> >>> >>> >>> On Wednesday, 1 August 2018 13:26:04 UTC+3, Paul_N wrote: >>>> >>>> Hi Andrew, thanks for this - and I think we're getting somewhere : it >>>> would appear that the drop-daily command is not working. I followed your >>>> first four steps and attach below the output from my terminal. As you'll >>>> see the initial query found the rogue outTemp min values but they were >>>> still there when I repeated that query AFTER doing the drop-daily command. >>>> So, if drop-daily is supposed to clear these values then it has not >>>> worked. >>>> >>>> Any suggestions please ? >>>> Thanks again, Paul (ps please note that all the temperature values >>>> below are actually in Fahrenheit because that is how my db is set up, >>>> hence >>>> the rogue min values are actually below 5 deg F). >>>> >>>> [ ok ] Stopping weewx (via systemctl): weewx.service. >>>> weewx stopped >>>> >>>> pi@raspberrypi:/var/lib/weewx $ sqlite3 weewx.sdb >>>> SQLite version 3.16.2 2017-01-06 16:32:41 >>>> Enter ".help" for usage hints. >>>> sqlite> select dateTime, min from archive_day_outTemp where min < 5; >>>> 1523919600|-3.46 >>>> 1524006000|-3.64 >>>> 1524438000|-3.82 >>>> sqlite> >>>> >>>> pi@raspberrypi:/var/lib/weewx $ cd /etc/weewx >>>> pi@raspberrypi:/etc/weewx $ wee_database weewx.conf --drop-daily >>>> Using configuration file weewx.conf >>>> Using database binding 'wx_binding', which is bound to database >>>> 'archive_sqlite' >>>> Proceeding will delete all your daily summaries from database >>>> 'weewx.sdb' >>>> Are you sure you want to proceed (y/n)? y >>>> Dropping daily summary tables from 'weewx.sdb' ... >>>> Daily summary tables dropped from database 'weewx.sdb' in 0.04 seconds >>>> >>>> pi@raspberrypi:/etc/weewx $ cd /var/lib/weewx >>>> pi@raspberrypi:/var/lib/weewx $ sqlite3 weewx.sdb >>>> SQLite version 3.16.2 2017-01-06 16:32:41 >>>> Enter ".help" for usage hints. >>>> sqlite> select dateTime, min from archive_day_outTemp where min < 5; >>>> 1523919600|-3.46 >>>> 1524006000|-3.64 >>>> 1524438000|-3.82 >>>> sqlite> >>>> pi@raspberrypi:/var/lib/weewx $ >>>> >>>> =============================================================== >>>> >>>> On Wednesday, August 1, 2018 at 11:04:08 AM UTC+1, Andrew Milner wrote: >>>>> >>>>> 1. stop weewx >>>>> >>>>> 2. try and see if your rogue data is in the summaries: >>>>> >>>>> select dateTime, min from archive_day_outTemp where min < -19; >>>>> >>>>> hopefully you will find the rogue day!! >>>>> >>>>> >>>>> 3. try and drop daily again >>>>> >>>>> 4. check that they have dropped by repeating the command: >>>>> select dateTime,min from archive_day_outTemp where min < -19; >>>>> >>>>> Hopefully none will be found - in fact you may even get an error. >>>>> >>>>> 5. check there is no rogue value in the archive with >>>>> select * from archive where outTemp < -19; >>>>> >>>>> 6. rebuild daily - this will rebuild all the archive_day tables from >>>>> the archive data >>>>> >>>>> 7. check the summary table no longer contains the rogue data >>>>> select * from archive_day_outTemp where min < -19; >>>>> >>>>> Hopefully you will have solved it. >>>>> >>>>> How long rebuild takes depends on loads of things, but should take a >>>>> few seconds at least I would have thought and not been instantaneous. >>>>> >>>>> DO REMEMBER THE ; ON THE END OF SQL COMMANDS >>>>> >>>>> >>>>> On Wednesday, 1 August 2018 11:13:01 UTC+3, Paul_N wrote: >>>>>> >>>>>> Hi Andrew, thanks for that suggestion. I have checked the database >>>>>> with that query and can confirm that there are no rogue values left in >>>>>> there. So the problem appears to be in the re-calculation of the minimum >>>>>> outdoor temperature for that month. I do not really understand exactly >>>>>> what >>>>>> the "rebuild-daily" command does - is that when these values get >>>>>> calculated >>>>>> ? If so, then maybe my observation that this command completed instantly >>>>>> might be relevant. There is a about 8 months of data in the database so >>>>>> should it have taken some while to run, as suggested in Tom K's wiki ? >>>>>> >>>>>> Thanks again for any advice you could give me, >>>>>> Regards, Paul >>>>>> >>>>>> On Wednesday, August 1, 2018 at 2:55:37 AM UTC+1, Andrew Milner wrote: >>>>>>> >>>>>>> have you rechecked the database with >>>>>>> >>>>>>> select dateTime, outTemp from archive where outTemp < 19; >>>>>>> >>>>>>> to see if there are still any rogue temperature records in there? >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tuesday, 31 July 2018 21:24:15 UTC+3, Paul_N wrote: >>>>>>>> >>>>>>>> Hi, I'd appreciate some advice on the problem I've been having with >>>>>>>> removing some erroneous outside temperature readings from my weewx >>>>>>>> system >>>>>>>> which occurred very early on in my set-up when the sensor was not >>>>>>>> connected >>>>>>>> - and before I discovered the value of setting StdQC properly ! The >>>>>>>> erroneous values were very low outside temperatures (-19.9 deg C), >>>>>>>> causing >>>>>>>> the minimum values to be reported for that month and for the year to >>>>>>>> be >>>>>>>> incorrect. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I followed the steps described by Tom K in his wiki guide >>>>>>>> <https://github.com/weewx/weewx/wiki/Cleaning-up-old-'bad'-data> >>>>>>>> and all seemed to go well: I was able to locate and remove (set to >>>>>>>> null) >>>>>>>> the erroneous outTemp readings in the database and then drop and >>>>>>>> rebuild >>>>>>>> the daily summaries OK. As advised elsewhere on this forum I also >>>>>>>> deleted >>>>>>>> all the PNG,HTML and NOAA text files so they were re-created after I >>>>>>>> did >>>>>>>> the database editing. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> The good news is that the png chart which shows monthly Min/Max >>>>>>>> outside temperatures no longer shows these erroneous values and so I >>>>>>>> am >>>>>>>> encouraged to think that they really are gone from the database. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> However, the NOAA summary for that month (and also the yearly one) >>>>>>>> still shows this bad data so it seems that whatever calculates the >>>>>>>> minimum >>>>>>>> outside temperature has not recalculated it for some reason. Please >>>>>>>> note >>>>>>>> that these NOAA summaries have definitely been recreated since the >>>>>>>> database >>>>>>>> editing as I deleted them after doing it and before re-starting weewx. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> One thing I did note which could be indicative is that although the >>>>>>>> "rebuild-daily" command appeared to execute OK, it completed almost >>>>>>>> instantaneously whereas Tom's wiki guide led me to believe it would >>>>>>>> take >>>>>>>> some while. Could it be that this has not actually worked and hence is >>>>>>>> the >>>>>>>> cause of my problem ? How can I check to see if it has done what is >>>>>>>> expected ? >>>>>>>> >>>>>>>> >>>>>>>> Just in case it's relevant, here is what I saw on the terminal when >>>>>>>> dropping and rebuilding the daily summaries: >>>>>>>> >>>>>>>> >>>>>>>> pi@raspberrypi:/etc/weewx $ wee_database weewx.conf --drop-daily >>>>>>>> >>>>>>>> Using configuration file weewx.conf >>>>>>>> >>>>>>>> Using database binding 'wx_binding', which is bound to database >>>>>>>> 'archive_sqlite' >>>>>>>> >>>>>>>> Proceeding will delete all your daily summaries from database >>>>>>>> 'weewx.sdb' >>>>>>>> >>>>>>>> Are you sure you want to proceed (y/n)? y >>>>>>>> >>>>>>>> Dropping daily summary tables from 'weewx.sdb' ... >>>>>>>> >>>>>>>> Daily summary tables dropped from database 'weewx.sdb' in 0.04 >>>>>>>> seconds >>>>>>>> >>>>>>>> >>>>>>>> pi@raspberrypi:/etc/weewx $ wee_database weewx.conf --rebuild-daily >>>>>>>> >>>>>>>> Using configuration file weewx.conf >>>>>>>> >>>>>>>> Using database binding 'wx_binding', which is bound to database >>>>>>>> 'archive_sqlite' >>>>>>>> >>>>>>>> All daily summaries will be rebuilt. >>>>>>>> >>>>>>>> Proceed (y/n)? y >>>>>>>> >>>>>>>> Rebuilding daily summaries in database 'weewx.sdb' ... >>>>>>>> >>>>>>>> Daily summaries up to date in 'weewx.sdb' >>>>>>>> >>>>>>>> >>>>>>>> With thanks in advance for any help you could give to enable me to >>>>>>>> track down this problem, >>>>>>>> >>>>>>>> Regards, Paul >>>>>>>> >>>>>>>> >>>>>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
