Zach, Did you by chance copy and paste those two lines from my earlier post into extensions.py? If so if you look closely I had a typo, lakeSurfaceLevel was incorrectly capitalised. I've just run weectl import with your data and import config file with the correct two lines in extensions.py and the import worked just fine.
Gary On Wednesday 24 July 2024 at 08:53:39 UTC+10 Zach wrote: > Unfortunately, adding it to extensions.py did not work. I'll sit tight > until a fix is completed. But no hurry! > > Thanks Gary! > > On Tuesday, July 23, 2024 at 5:48:01 PM UTC-5 gjr80 wrote: > >> The problem is weectl import is a standalone utility run independent of >> WeeWX and as such does not load the WeeWX services specified in the WeeWX >> config file (well it actually does cause some of the services to be loaded, >> but later on after the import has occurred when missing obs are/may be >> calculated). I've had some second thoughts since my previous post and I am >> not even sure that adding those lines to extensions.py will work. If it >> is easy enough to try please try it, it will either work or not, nothing >> will be broken/corrupted. >> >> Failing that there will be a small change required to weectl import that >> I should be able to implement later today. >> >> Let me know how you get on. >> >> Gary >> On Wednesday 24 July 2024 at 07:33:27 UTC+10 Zach wrote: >> >>> I have that line in my custom service for pulling the data from an API. >>> Should I also put it in the extension.py? >>> >>> On Tuesday, July 23, 2024 at 4:17:36 PM UTC-5 gjr80 wrote: >>> >>>> OK, I suspect the problem is when you added field lakeSurfaceLevel to >>>> WeeWX you didn't tell WeeWX what unit group the field belongs to. >>>> Consequently, when you attempt to import data into lakeSurfaceLevel weectl >>>> import conducts some checks to see if any source field unit conversion >>>> is required. That is what is causing the KeyError error. >>>> >>>> The simple fix is to tell WeeWX what unit group is used by the WeeWX >>>> field lakeSurfaceLevel. There are a number of ways to do this, the >>>> most common is to add a few lines to >>>> ~/weewx-data/bin/user/extensions.py. You could try adding the >>>> following to ~/weewx-data/bin/user/extensions.py: >>>> >>>> import weewx.units >>>> weewx.units.obs_group_dict['lakeSurfacelevel'] = 'group_altitude' >>>> >>>> save extensions.py and try the import agin. >>>> >>>> Gary >>>> >>>> On Wednesday 24 July 2024 at 06:50:28 UTC+10 Zach wrote: >>>> >>>>> Sorry, I should have clicked reply all. ;) >>>>> >>>>> Here is the output >>>>> >>>>> Using configuration file */Users/zach/weewx-data/weewx.conf* >>>>> >>>>> This is a dry run. Nothing will actually be done. >>>>> >>>>> Starting weectl import... >>>>> >>>>> A CSV import from source file '/Users/zach/Downloads/Untitled.csv' has >>>>> been requested. >>>>> >>>>> The following options will be used: >>>>> >>>>> config=/Users/zach/weewx-data/weewx.conf, >>>>> import-config=/Users/zach/weewx-data/csv.conf >>>>> >>>>> source=/Users/zach/Downloads/Untitled.csv, from=None, to=None >>>>> >>>>> dry-run=True, calc_missing=False, ignore_invalid_data=True >>>>> >>>>> tranche=250, interval=derive, date/time_string_format=%Y-%m-%d >>>>> %H:%M:%S >>>>> >>>>> delimiter=',', wind_direction=[-360.0, 360.0] >>>>> >>>>> UV=False, radiation=False >>>>> >>>>> Using database binding 'wx_binding', which is bound to database >>>>> 'weewx.sdb' >>>>> >>>>> Destination table 'archive' unit system is '0x01' (US). >>>>> >>>>> The following imported field-to-WeeWX field map will be used: >>>>> >>>>> source field 'datetime' in units 'unix_epoch' --> WeeWX field >>>>> 'dateTime' >>>>> >>>>> source field 'lakeSurfaceLevel' in units 'foot' --> WeeWX field >>>>> 'lakeSurfaceLevel' >>>>> >>>>> Imported records will not overwrite existing database records. >>>>> >>>>> All WeeWX UV fields will be set to None. >>>>> >>>>> All WeeWX radiation fields will be set to None. >>>>> >>>>> This is a dry run, imported data will not be saved to archive. >>>>> >>>>> Starting dry run import ... >>>>> >>>>> Obtaining raw import data for period 1 ... >>>>> >>>>> Raw import data read successfully for period 1. >>>>> >>>>> Mapping raw import data for period 1 ... >>>>> >>>>> Traceback (most recent call last): >>>>> >>>>> File "/Users/zach/weewx-venv/bin/weectl", line 8, in <module> >>>>> >>>>> sys.exit(main()) >>>>> >>>>> File "/Users/zach/weewx-venv/lib/python3.9/site-packages/weectl.py", >>>>> line 67, in main >>>>> >>>>> namespace.func(namespace) >>>>> >>>>> File >>>>> "/Users/zach/weewx-venv/lib/python3.9/site-packages/weectllib/__init__.py", >>>>> >>>>> line 90, in dispatch >>>>> >>>>> namespace.action_func(config_dict, namespace) >>>>> >>>>> File >>>>> "/Users/zach/weewx-venv/lib/python3.9/site-packages/weectllib/import_cmd.py", >>>>> >>>>> line 85, in import_func >>>>> >>>>> weectllib.import_actions.obs_import(config_dict, >>>>> >>>>> File >>>>> "/Users/zach/weewx-venv/lib/python3.9/site-packages/weectllib/import_actions.py", >>>>> >>>>> line 58, in obs_import >>>>> >>>>> source_obj.run() >>>>> >>>>> File >>>>> "/Users/zach/weewx-venv/lib/python3.9/site-packages/weeimport/weeimport.py", >>>>> >>>>> line 406, in run >>>>> >>>>> _mapped_data = self.map_raw_data(_raw_data, self.archive_unit_sys) >>>>> >>>>> File >>>>> "/Users/zach/weewx-venv/lib/python3.9/site-packages/weeimport/weeimport.py", >>>>> >>>>> line 976, in map_raw_data >>>>> >>>>> weewx.units.obs_group_dict[_field]) >>>>> >>>>> File >>>>> "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/collections/__init__.py", >>>>> >>>>> line 941, in __getitem__ >>>>> >>>>> return self.__missing__(key) # support subclasses that >>>>> define __missing__ >>>>> >>>>> File >>>>> "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/collections/__init__.py", >>>>> >>>>> line 933, in __missing__ >>>>> >>>>> raise KeyError(key) >>>>> >>>>> KeyError: 'lakeSurfaceLevel' >>>>> >>>>> >>>>> On Tuesday, July 23, 2024 at 3:45:46 PM UTC-5 gjr80 wrote: >>>>> >>>>>> Thanks, can you post the complete and exact output to the console >>>>>> when weectl import is run. Also, what does the weectl log show? >>>>>> >>>>>> Gary >>>>>> On Wednesday 24 July 2024 at 06:40:53 UTC+10 Zach wrote: >>>>>> >>>>>>> I'm trying to track the water level of a lake so I added a new >>>>>>> column to the database (lakeSurfaceLevel), created a service to >>>>>>> populate >>>>>>> the data, and modified my skin to display it. All of that works great, >>>>>>> however when I try to import data for that new observation using >>>>>>> weectl, I >>>>>>> get a error "KeyError: 'lakeSurfaceLevel' >>>>>>> >>>>>>> Any help would be greatly appreciated! >>>>>>> >>>>>>> Attached are the csv.conf and a data file >>>>>>> >>>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/804eb2f2-f2ff-415e-838b-8913e7282ecan%40googlegroups.com.
