cal_ncgen module

Script for creating FAAM calibration netCDF files.

$ python3 cal_ncgen.py SEA-WCM2000.cdl

creates a netCDF4 file, SEA-WCM2000.nc from the cdl source file. To update variables in the netCDF directly from the command line;

$ python3 cal_ncgen.py SEA-WCM2000.nc -u time 700 800
  -u applies_to C150- C180- -u TWC/r100 31.4473 31.5585
  -u TWC/dtdr 33.9276 34.0387 --user 'Graeme Nott <graeme.nott@faam.ac.uk>'
  --hist \<now\>\ Artificial\ update\ 1 '<today> Artificial update 2'

where the nc file is read in and four parameters are updated. Two entries are appended to the global variables time and applies_to, and to the TWC group variables TWC/r100 and TWC/dtdr. The same username is appended to the username global attribute for both entries. Different history strings are appended to the global history attribute however (note the two different ways to escape history strings) with <now> and <today> being converted to the current date.

An existing calibration netCDF may have values appended to to it from a cdl or nc file with;

$ python3 cal_ncgen.py SEA-WCM2000_cal_20180810.nc SEA-WCM2000_20190805.cdl

Explicitly giving a different output filename with the –output argument will leave WCM2000_cal_20180810.nc unaltered and create a new calibration netCDF;

$ python3 cal_ncgen.py SEA-WCM2000_cal_20180810.nc SEA-WCM2000_20190805.cdl
  --output WCM2000_cal_2018-19.nc

It is possible to update variables using one or more external files. This requires a custom parser to be part of the instrument processor class so that these files can be read and injested. A single external file may be added to an existing calibration netCDF file with;

$ python3 cal_ncgen.py PCASP_faam_20170701_v001_r000_cal.nc -u time 20170919
  -u applies_to C027-C055 -u parsefile testing/data/20170919_P1_cal_results_cs.csv

Additional metadata associated with the text file testing/data/20170919_P1_cal_results_cs.csv is given with -u arguments. The special –update key parsefile indicates that the following value needs to be parsed with the instrument-specific parser method.

It is also possible to add multiple external calibration files at the same time as the associated metadata with an external configuration file. This file uses the standard ascii format that is parsed with the configparser package. So if for example the config file PCASP1_CLARIFY_cals.cfg contained;

[pre-CLARIFY]
_group = bin_cal
time = 20170701
applied_to = C027-
user = Graeme Nott
traceability = List of PSL lot number information
comments = After realignment of inlet jet
cal_flag = 0
_parsefile = testing/data/20170801_P1_cal_results_cs.csv

[post-CLARIFY]
_group = bin_cal
time = 20170919
applied_to = C027-C055
user = Graeme Nott
traceability = List of PSL lot number information
cal_flag = 0
_parsefile = testing/data/20170919_P1_cal_results_cs.csv

this could be inserted into an existing netCDF file that has been created from the PCASP1 template cdl file as follows;

$ python3 cal_ncgen.py PCASP1_cal.cdl -u parsefile PCASP1_CLARIFY_cals.cfg
  -o PCASP_faam_20170701_v001_r000_cal.nc

Note that such a config file must have a recognisable format and the .cfg or .config extension to ensure that the instrument parser is not invoked on the config file directly. The special options in each section that start with an _ are not treated as netCDF attributes or variables but are used to assist in the processing of the options.

cal_ncgen.call(infile, args)[source]

Convenience function for cal_ncgen.py.

The main grunt work is actually done in nc_func.process_nc().

How this works:

  • Creates separate lists of nc and cdl files. Determines ‘master’

    file based on list order with nc files having priority over cdl.

  • netCDF files are created for any cdl files by calling ncgen. If no other arguments for nc then finish

  • Determine output filename

  • If no changes to master nc file required then exit.

  • Consolidate any arguments into an updates dictionary

  • Pass to nc_func.process_nc() for creation of complete cal-nc file.

Parameters

  • infile (list) – List of one or more cdl filesnames. If multiple files are given than these will be concatenated.

  • single cdl then ncgen to nc. (if) –

  • args (dict) – Arguments for adding to resultant nc file.