> Various questions about missing values in the R ncdf package, and how they are > handled if the file lacks the standard "_FillValue" attribute.
Hi Andre, It sounds like the fundamental problem is that your data files are using a missval, but that fact is not recorded in the file's metadata as an attribute of the variable (although it should be). The "best" solution is to fix the problematic data files by putting in the correct missval. You can do this with R using the ncdf package. For example, to add this attribute to a file, you could do something like this (untested code, just writing off the top of my head): varname = 'Temperature' # for example new_missval = 9.96921e+36 ncid = open.ncdf( 'filename.nc', write=TRUE ) att.put.ncdf( ncid, varname, '_FillValue', new_missval ) close.ncdf( ncid ) Note that the more commonly accepted attribute name nowadays for a missing value is "_FillValue", not "missing_value". You can also do this outside of R by using the netcdf operators (nco), http://nco.sourceforge.net/. If fixing the files to be correct is not practical, you can always fix the data after you read it in: file_missval = 9.96921e+36 mvtol = abs(file_missval)*1.e-5 varname = 'Temperature' # for example ncid = open.ncdf( 'filename.nc' ) data = get.var.ncdf( ncid, varname ) is_missing = (abs( data - file_missval) < mvtol ) # floating comparison data[ is_missing ] = NA To use R to see if the missing_value of _FillValue attribute actually exists in the file: varname = 'Temperature' # for example ncid = open.ncdf( 'filename.nc' ) mv = att.get.ncdf( ncid, varname, '_FillValue" ) print(mv$hasatt) This will be TRUE if the variable has the attribute, and FALSE otherwise. Of course, you can repeat this for an attribute named "missing_value" instead of "_FillValue". See also the R documentation page for ncdf routine "set.missval.ncdf", which allows you to change the actual missing values that are written in the data file. Regards, --Dave On Thu, Feb 6, 2014 at 7:26 AM, Dusterhus, Andre <and...@noc.ac.uk> wrote: > Hi all, > > I got currently some trouble with missvals in netCDF files: > > I have a netCDF file, written with an unknown program, which shows via > ncdump no fillval or missval for its variables. Within the variables > obviously a missval of 9.96921e+36 is used. > > When I read it into R via the ncdf package it shows me a missval of 1e+30, > but the 9.96921e+36 are still in the dataset. > > Since my application is an automised processing of such data I have the > following questions: > > Is R showing a missval, even when it is not existent in the file itself (I > know that it uses the 1e+30 value as a default itself, when it writes > files)? > Could it be the case that R (ncdf) varies here from the "netCDF standard", > which assumes 9.96921e+36 instead of 1e+30 as the standard missval (at > least a lot of software on the market does this)? > How is it possible via R (ncdf) to find out, whether the actual given > missval is really given in the data? > > Many thanks in advance, > André Düsterhus > > This message (and any attachments) is for the recipien...{{dropped:21}}
______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
