That was it! Thank you so much for your help!!! On Tuesday, July 23, 2024 at 7:33:10 PM UTC-5 gjr80 wrote:
> Zach, > > Did you by chance copy and paste those two lines from my earlier post into > extensions.py? If so if you look closely I had a typo, lakeSurfaceLevel > was incorrectly capitalised. I've just run weectl import with your data > and import config file with the correct two lines in extensions.py and > the import worked just fine. > > Gary > > On Wednesday 24 July 2024 at 08:53:39 UTC+10 Zach wrote: > >> Unfortunately, adding it to extensions.py did not work. I'll sit tight >> until a fix is completed. But no hurry! >> >> Thanks Gary! >> >> On Tuesday, July 23, 2024 at 5:48:01 PM UTC-5 gjr80 wrote: >> >>> The problem is weectl import is a standalone utility run independent of >>> WeeWX and as such does not load the WeeWX services specified in the WeeWX >>> config file (well it actually does cause some of the services to be loaded, >>> but later on after the import has occurred when missing obs are/may be >>> calculated). I've had some second thoughts since my previous post and I am >>> not even sure that adding those lines to extensions.py will work. If it >>> is easy enough to try please try it, it will either work or not, nothing >>> will be broken/corrupted. >>> >>> Failing that there will be a small change required to weectl import >>> that I should be able to implement later today. >>> >>> Let me know how you get on. >>> >>> Gary >>> On Wednesday 24 July 2024 at 07:33:27 UTC+10 Zach wrote: >>> >>>> I have that line in my custom service for pulling the data from an API. >>>> Should I also put it in the extension.py? >>>> >>>> On Tuesday, July 23, 2024 at 4:17:36 PM UTC-5 gjr80 wrote: >>>> >>>>> OK, I suspect the problem is when you added field lakeSurfaceLevel to >>>>> WeeWX you didn't tell WeeWX what unit group the field belongs to. >>>>> Consequently, when you attempt to import data into lakeSurfaceLevel >>>>> weectl >>>>> import conducts some checks to see if any source field unit >>>>> conversion is required. That is what is causing the KeyError error. >>>>> >>>>> The simple fix is to tell WeeWX what unit group is used by the WeeWX >>>>> field lakeSurfaceLevel. There are a number of ways to do this, the >>>>> most common is to add a few lines to >>>>> ~/weewx-data/bin/user/extensions.py. You could try adding the >>>>> following to ~/weewx-data/bin/user/extensions.py: >>>>> >>>>> import weewx.units >>>>> weewx.units.obs_group_dict['lakeSurfacelevel'] = 'group_altitude' >>>>> >>>>> save extensions.py and try the import agin. >>>>> >>>>> Gary >>>>> >>>>> On Wednesday 24 July 2024 at 06:50:28 UTC+10 Zach wrote: >>>>> >>>>>> Sorry, I should have clicked reply all. ;) >>>>>> >>>>>> Here is the output >>>>>> >>>>>> Using configuration file */Users/zach/weewx-data/weewx.conf* >>>>>> >>>>>> This is a dry run. Nothing will actually be done. >>>>>> >>>>>> Starting weectl import... >>>>>> >>>>>> A CSV import from source file '/Users/zach/Downloads/Untitled.csv' >>>>>> has been requested. >>>>>> >>>>>> The following options will be used: >>>>>> >>>>>> config=/Users/zach/weewx-data/weewx.conf, >>>>>> import-config=/Users/zach/weewx-data/csv.conf >>>>>> >>>>>> source=/Users/zach/Downloads/Untitled.csv, from=None, to=None >>>>>> >>>>>> dry-run=True, calc_missing=False, ignore_invalid_data=True >>>>>> >>>>>> tranche=250, interval=derive, date/time_string_format=%Y-%m-%d >>>>>> %H:%M:%S >>>>>> >>>>>> delimiter=',', wind_direction=[-360.0, 360.0] >>>>>> >>>>>> UV=False, radiation=False >>>>>> >>>>>> Using database binding 'wx_binding', which is bound to database >>>>>> 'weewx.sdb' >>>>>> >>>>>> Destination table 'archive' unit system is '0x01' (US). >>>>>> >>>>>> The following imported field-to-WeeWX field map will be used: >>>>>> >>>>>> source field 'datetime' in units 'unix_epoch' --> WeeWX field >>>>>> 'dateTime' >>>>>> >>>>>> source field 'lakeSurfaceLevel' in units 'foot' --> WeeWX field >>>>>> 'lakeSurfaceLevel' >>>>>> >>>>>> Imported records will not overwrite existing database records. >>>>>> >>>>>> All WeeWX UV fields will be set to None. >>>>>> >>>>>> All WeeWX radiation fields will be set to None. >>>>>> >>>>>> This is a dry run, imported data will not be saved to archive. >>>>>> >>>>>> Starting dry run import ... >>>>>> >>>>>> Obtaining raw import data for period 1 ... >>>>>> >>>>>> Raw import data read successfully for period 1. >>>>>> >>>>>> Mapping raw import data for period 1 ... >>>>>> >>>>>> Traceback (most recent call last): >>>>>> >>>>>> File "/Users/zach/weewx-venv/bin/weectl", line 8, in <module> >>>>>> >>>>>> sys.exit(main()) >>>>>> >>>>>> File >>>>>> "/Users/zach/weewx-venv/lib/python3.9/site-packages/weectl.py", line 67, >>>>>> in >>>>>> main >>>>>> >>>>>> namespace.func(namespace) >>>>>> >>>>>> File >>>>>> "/Users/zach/weewx-venv/lib/python3.9/site-packages/weectllib/__init__.py", >>>>>> >>>>>> line 90, in dispatch >>>>>> >>>>>> namespace.action_func(config_dict, namespace) >>>>>> >>>>>> File >>>>>> "/Users/zach/weewx-venv/lib/python3.9/site-packages/weectllib/import_cmd.py", >>>>>> >>>>>> line 85, in import_func >>>>>> >>>>>> weectllib.import_actions.obs_import(config_dict, >>>>>> >>>>>> File >>>>>> "/Users/zach/weewx-venv/lib/python3.9/site-packages/weectllib/import_actions.py", >>>>>> >>>>>> line 58, in obs_import >>>>>> >>>>>> source_obj.run() >>>>>> >>>>>> File >>>>>> "/Users/zach/weewx-venv/lib/python3.9/site-packages/weeimport/weeimport.py", >>>>>> >>>>>> line 406, in run >>>>>> >>>>>> _mapped_data = self.map_raw_data(_raw_data, self.archive_unit_sys) >>>>>> >>>>>> File >>>>>> "/Users/zach/weewx-venv/lib/python3.9/site-packages/weeimport/weeimport.py", >>>>>> >>>>>> line 976, in map_raw_data >>>>>> >>>>>> weewx.units.obs_group_dict[_field]) >>>>>> >>>>>> File >>>>>> "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/collections/__init__.py", >>>>>> >>>>>> line 941, in __getitem__ >>>>>> >>>>>> return self.__missing__(key) # support subclasses that >>>>>> define __missing__ >>>>>> >>>>>> File >>>>>> "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/collections/__init__.py", >>>>>> >>>>>> line 933, in __missing__ >>>>>> >>>>>> raise KeyError(key) >>>>>> >>>>>> KeyError: 'lakeSurfaceLevel' >>>>>> >>>>>> >>>>>> On Tuesday, July 23, 2024 at 3:45:46 PM UTC-5 gjr80 wrote: >>>>>> >>>>>>> Thanks, can you post the complete and exact output to the console >>>>>>> when weectl import is run. Also, what does the weectl log show? >>>>>>> >>>>>>> Gary >>>>>>> On Wednesday 24 July 2024 at 06:40:53 UTC+10 Zach wrote: >>>>>>> >>>>>>>> I'm trying to track the water level of a lake so I added a new >>>>>>>> column to the database (lakeSurfaceLevel), created a service to >>>>>>>> populate >>>>>>>> the data, and modified my skin to display it. All of that works great, >>>>>>>> however when I try to import data for that new observation using >>>>>>>> weectl, I >>>>>>>> get a error "KeyError: 'lakeSurfaceLevel' >>>>>>>> >>>>>>>> Any help would be greatly appreciated! >>>>>>>> >>>>>>>> Attached are the csv.conf and a data file >>>>>>>> >>>>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/8d648d6f-a917-4990-af5e-d8c0bac96d7an%40googlegroups.com.
