The error in the original post is due to one of the [[FieldMap]] mappings not having WeeWX unit name. You do not need to include mappings for all fields in the CSV source, only those you wish to import, but those you do include must be of the form:
WeeWX field = CSV field name, WeeWX unit name You are correct in that the raw_datetime_format setting determines how the CSV date-time field is decoded. For the data shown I would expect something like (untested): raw_datetime_format = "%m/%d/%Y %H:%M" this assumes the date is in the format month/day/year, just transpose %m and %d if it id day/month/year. Gary On Wednesday, 19 June 2019 12:33:03 UTC+10, rich T wrote: > > I could be wrong, but the raw_datetime_format is the datetime format in > the CSV file (i.e. %m/%d%Y). The dateTime = timestamp, unix_epoch in the > CSV Map basically means convert the timestamp column into unix_epoch time > format. > > On Tuesday, June 18, 2019 at 10:06:49 PM UTC-4, Andrew Milner wrote: >> >> shouldn't it just be dateTime = timestamp and the format of timestamp be >> specified in raw_datetime_format ?? >> >> >> >> On Wednesday, 19 June 2019 04:32:30 UTC+3, rich T wrote: >>> >>> I'm trying to import a large CSV file into the database and I'm >>> receiving the following error the the dry run. >>> >>> pi@raspberrypi:/home/weewx/bin $ sudo ./wee_import >>> --import-config=/home/pi/Desktop/clog/csv.conf --dry-run >>> Starting wee_import... >>> A CSV import from source file >>> '/home/pi/Desktop/clog/archive_updated1.csv' has been requested. >>> Using database binding 'wx_binding', which is bound to database >>> 'weewx.sdb' >>> Destination table 'archive' unit system is '0x01' (US). >>> Missing derived observations will be calculated. >>> All WeeWX UV fields will be set to None. >>> All WeeWX radiation fields will be set to None. >>> This is a dry run, imported data will not be saved to archive. >>> Traceback (most recent call last): >>> File "./wee_import", line 703, in <module> >>> main() >>> File "./wee_import", line 653, in main >>> source_obj.run() >>> File "/home/weewx/bin/weeimport/weeimport.py", line 350, in run >>> _mapped_data = self.mapRawData(_raw_data, self.archive_unit_sys) >>> File "/home/weewx/bin/weeimport/weeimport.py", line 721, in mapRawData >>> self.map[_field]['units'], >>> KeyError: 'units' >>> >>> Here is the mapping in the csv.conf: >>> >>> [[FieldMap]] >>> dateTime = timestamp, unix_epoch >>> usUnits = units >>> interval = interval >>> barometer = barometer, inHg >>> pressure = pressure >>> altimeter = altimeter >>> inTemp = inTemp >>> outTemp = outTemp, degree_F >>> inHumidity = inHumidity >>> outHumidity = outHumidity, percent >>> windSpeed = windSpeed, mile_per_hour >>> windDir = windDir, degree_compass >>> windGust = windGust, mile_per_hour >>> windGustDir = windGustDir, degree_compass >>> rainRate = rainRate, inch_per_hour >>> rain = rain, inch >>> dewpoint = dewpoint >>> windchill = windchill >>> heatindex = heatindex >>> ET = >>> radiation = >>> UV = >>> rxCheckPercent = rxCheckPercent >>> >>> Here is a small portion of the CSV file: >>> >>> >>> timestamp,units,interval,barometer,pressure,altimeter,inTemp,outTemp,inHumidity,outHumidity,windSpeed,windDir,windGust,windGustDir,rainRate,rain,dewpoint,windchill,heatindex,ET,radiation,UV,extraTemp1,extraTemp2,extraTemp3,soilTemp1,soilTemp2,soilTemp3,soilTemp4,leafTemp1,leafTemp2,extraHumid1,extraHumid2,soilMoist1,soilMoist2,soilMoist3,soilMoist4,leafWet1,leafWet2,rxCheckPercent,txBatteryStatus,consBatteryVoltage,hail,hailRate,heatingTemp,heatingVoltage,supplyVoltage,referenceVoltage,windBatteryStatus,rainBatteryStatus,outTempBatteryStatus,inTempBatteryStatus >>> 04/09/2018 >>> 00:05,1,5,30.101,,,77.2,24.2,20,62,0,,0,,0,0,13.08666521,24.2,24.2,0,,,,,,,,,,,,,,,,,,,,99.9375,,,,,,,,,,,, >>> 04/09/2018 >>> 00:10,1,5,30.102,,,77,24,20,63,0,,0,,0,0,13.25922371,24,24,0,,,,,,,,,,,,,,,,,,,,97.375,,,,,,,,,,,, >>> 04/09/2018 >>> 00:15,1,5,30.108,,,76.9,23.8,20,63,0,,0,,0,0,13.06932323,23.8,23.8,0,,,,,,,,,,,,,,,,,,,,96.52083333,,,,,,,,,,,, >>> 04/09/2018 >>> 00:20,1,5,30.11,,,76.7,23.7,20,63,0,,0,,0,0,12.97437126,23.7,23.7,0,,,,,,,,,,,,,,,,,,,,97.375,,,,,,,,,,,, >>> 04/09/2018 >>> 00:25,1,5,30.112,,,76.6,23.5,20,63,0,,0,,0,0,12.78446384,23.5,23.5,0,,,,,,,,,,,,,,,,,,,,77.72916667,,,,,,,,,,,, >>> 04/09/2018 >>> 00:30,1,5,30.11,,,76.4,23.5,21,63,0,,0,,0,0,12.78446384,23.5,23.5,0,,,,,,,,,,,,,,,,,,,,99.9375,,,,,,,,,,,, >>> 04/09/2018 >>> 00:35,1,5,30.111,,,76.7,23.3,20,64,0,,3,225,0,0,12.95043454,23.3,23.3,0,,,,,,,,,,,,,,,,,,,,99.08333333,,,,,,,,,,,, >>> 04/09/2018 >>> 00:40,1,5,30.106,,,76.9,23.3,20,64,0,,0,,0,0,12.95043454,23.3,23.3,0,,,,,,,,,,,,,,,,,,,,99.08333333,,,,,,,,,,,, >>> 04/09/2018 >>> 00:45,1,5,30.11,,,77,23.1,20,64,0,,0,,0,0,12.76018685,23.1,23.1,0,,,,,,,,,,,,,,,,,,,,99.08333333,,,,,,,,,,,, >>> 04/09/2018 >>> 00:50,1,5,30.114,,,77,23,20,65,0,,0,,0,0,13.01554258,23,23,0,,,,,,,,,,,,,,,,,,,,100,,,,,,,,,,,, >>> >>> >>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/311ac154-a724-4d73-a2b0-d7df9389d23c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
