Rich, Try deleting the mapping for usUnits from the field map. Long story short, it does not make much sense to be importing/converting a unit system code when units are specified/implied for all the fields. I know this isn't covered in the wee_import guide, will fix that up later.
Gary On Wednesday, 19 June 2019 13:10:13 UTC+10, rich T wrote: > > Gary > > I have the raw_datetime_format as "%m/%d/%Y %H:%M", but forgot to include > it in previous post. Now I receive the following error: > > pi@raspberrypi:/home/weewx/bin $ sudo ./wee_import > --import-config=/home/pi/Desktop/clog/csv.conf --dry-run > Starting wee_import... > A CSV import from source file '/home/pi/Desktop/clog/archive_updated1.csv' > has been requested. > Using database binding 'wx_binding', which is bound to database 'weewx.sdb' > Destination table 'archive' unit system is '0x01' (US). > Missing derived observations will be calculated. > All WeeWX UV fields will be set to None. > All WeeWX radiation fields will be set to None. > This is a dry run, imported data will not be saved to archive. > **** Unable to map source data. > **** Field 'units' not found in source data. > **** Nothing done, exiting. > > Thanks > Rich > > On Tuesday, June 18, 2019 at 10:46:20 PM UTC-4, gjr80 wrote: >> >> The error in the original post is due to one of the [[FieldMap]] >> mappings not having WeeWX unit name. You do not need to include mappings >> for all fields in the CSV source, only those you wish to import, but those >> you do include must be of the form: >> >> WeeWX field = CSV field name, WeeWX unit name >> >> You are correct in that the raw_datetime_format setting determines how >> the CSV date-time field is decoded. For the data shown I would expect >> something like (untested): >> >> raw_datetime_format = "%m/%d/%Y %H:%M" >> >> this assumes the date is in the format month/day/year, just transpose %m >> and %d if it id day/month/year. >> >> Gary >> >> On Wednesday, 19 June 2019 12:33:03 UTC+10, rich T wrote: >>> >>> I could be wrong, but the raw_datetime_format is the datetime format in >>> the CSV file (i.e. %m/%d%Y). The dateTime = timestamp, unix_epoch in the >>> CSV Map basically means convert the timestamp column into unix_epoch time >>> format. >>> >>> On Tuesday, June 18, 2019 at 10:06:49 PM UTC-4, Andrew Milner wrote: >>>> >>>> shouldn't it just be dateTime = timestamp and the format of timestamp >>>> be specified in raw_datetime_format ?? >>>> >>>> >>>> >>>> On Wednesday, 19 June 2019 04:32:30 UTC+3, rich T wrote: >>>>> >>>>> I'm trying to import a large CSV file into the database and I'm >>>>> receiving the following error the the dry run. >>>>> >>>>> pi@raspberrypi:/home/weewx/bin $ sudo ./wee_import >>>>> --import-config=/home/pi/Desktop/clog/csv.conf --dry-run >>>>> Starting wee_import... >>>>> A CSV import from source file >>>>> '/home/pi/Desktop/clog/archive_updated1.csv' has been requested. >>>>> Using database binding 'wx_binding', which is bound to database >>>>> 'weewx.sdb' >>>>> Destination table 'archive' unit system is '0x01' (US). >>>>> Missing derived observations will be calculated. >>>>> All WeeWX UV fields will be set to None. >>>>> All WeeWX radiation fields will be set to None. >>>>> This is a dry run, imported data will not be saved to archive. >>>>> Traceback (most recent call last): >>>>> File "./wee_import", line 703, in <module> >>>>> main() >>>>> File "./wee_import", line 653, in main >>>>> source_obj.run() >>>>> File "/home/weewx/bin/weeimport/weeimport.py", line 350, in run >>>>> _mapped_data = self.mapRawData(_raw_data, self.archive_unit_sys) >>>>> File "/home/weewx/bin/weeimport/weeimport.py", line 721, in >>>>> mapRawData >>>>> self.map[_field]['units'], >>>>> KeyError: 'units' >>>>> >>>>> Here is the mapping in the csv.conf: >>>>> >>>>> [[FieldMap]] >>>>> dateTime = timestamp, unix_epoch >>>>> usUnits = units >>>>> interval = interval >>>>> barometer = barometer, inHg >>>>> pressure = pressure >>>>> altimeter = altimeter >>>>> inTemp = inTemp >>>>> outTemp = outTemp, degree_F >>>>> inHumidity = inHumidity >>>>> outHumidity = outHumidity, percent >>>>> windSpeed = windSpeed, mile_per_hour >>>>> windDir = windDir, degree_compass >>>>> windGust = windGust, mile_per_hour >>>>> windGustDir = windGustDir, degree_compass >>>>> rainRate = rainRate, inch_per_hour >>>>> rain = rain, inch >>>>> dewpoint = dewpoint >>>>> windchill = windchill >>>>> heatindex = heatindex >>>>> ET = >>>>> radiation = >>>>> UV = >>>>> rxCheckPercent = rxCheckPercent >>>>> >>>>> Here is a small portion of the CSV file: >>>>> >>>>> >>>>> timestamp,units,interval,barometer,pressure,altimeter,inTemp,outTemp,inHumidity,outHumidity,windSpeed,windDir,windGust,windGustDir,rainRate,rain,dewpoint,windchill,heatindex,ET,radiation,UV,extraTemp1,extraTemp2,extraTemp3,soilTemp1,soilTemp2,soilTemp3,soilTemp4,leafTemp1,leafTemp2,extraHumid1,extraHumid2,soilMoist1,soilMoist2,soilMoist3,soilMoist4,leafWet1,leafWet2,rxCheckPercent,txBatteryStatus,consBatteryVoltage,hail,hailRate,heatingTemp,heatingVoltage,supplyVoltage,referenceVoltage,windBatteryStatus,rainBatteryStatus,outTempBatteryStatus,inTempBatteryStatus >>>>> 04/09/2018 >>>>> 00:05,1,5,30.101,,,77.2,24.2,20,62,0,,0,,0,0,13.08666521,24.2,24.2,0,,,,,,,,,,,,,,,,,,,,99.9375,,,,,,,,,,,, >>>>> 04/09/2018 >>>>> 00:10,1,5,30.102,,,77,24,20,63,0,,0,,0,0,13.25922371,24,24,0,,,,,,,,,,,,,,,,,,,,97.375,,,,,,,,,,,, >>>>> 04/09/2018 >>>>> 00:15,1,5,30.108,,,76.9,23.8,20,63,0,,0,,0,0,13.06932323,23.8,23.8,0,,,,,,,,,,,,,,,,,,,,96.52083333,,,,,,,,,,,, >>>>> 04/09/2018 >>>>> 00:20,1,5,30.11,,,76.7,23.7,20,63,0,,0,,0,0,12.97437126,23.7,23.7,0,,,,,,,,,,,,,,,,,,,,97.375,,,,,,,,,,,, >>>>> 04/09/2018 >>>>> 00:25,1,5,30.112,,,76.6,23.5,20,63,0,,0,,0,0,12.78446384,23.5,23.5,0,,,,,,,,,,,,,,,,,,,,77.72916667,,,,,,,,,,,, >>>>> 04/09/2018 >>>>> 00:30,1,5,30.11,,,76.4,23.5,21,63,0,,0,,0,0,12.78446384,23.5,23.5,0,,,,,,,,,,,,,,,,,,,,99.9375,,,,,,,,,,,, >>>>> 04/09/2018 >>>>> 00:35,1,5,30.111,,,76.7,23.3,20,64,0,,3,225,0,0,12.95043454,23.3,23.3,0,,,,,,,,,,,,,,,,,,,,99.08333333,,,,,,,,,,,, >>>>> 04/09/2018 >>>>> 00:40,1,5,30.106,,,76.9,23.3,20,64,0,,0,,0,0,12.95043454,23.3,23.3,0,,,,,,,,,,,,,,,,,,,,99.08333333,,,,,,,,,,,, >>>>> 04/09/2018 >>>>> 00:45,1,5,30.11,,,77,23.1,20,64,0,,0,,0,0,12.76018685,23.1,23.1,0,,,,,,,,,,,,,,,,,,,,99.08333333,,,,,,,,,,,, >>>>> 04/09/2018 >>>>> 00:50,1,5,30.114,,,77,23,20,65,0,,0,,0,0,13.01554258,23,23,0,,,,,,,,,,,,,,,,,,,,100,,,,,,,,,,,, >>>>> >>>>> >>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/498d3747-e55f-4f9e-a2e3-b6cf1e9e3e86%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
