This page was created by the IDL library routine
mk_html_help2.
Last modified: Thu Aug 6 12:53:14 2020.
FUNCTION cdfi = cdf_load_vars2(file)
INPUT:
file = CDF filename(s)
OUTPUT:
CDFI = A structure containing pointers to the data and attributes
for the files, with tags:
CDFI.FILENAME = The filename(s)
CDFI.INQ = A structure with information about the file:
CDFI.INQ.NDIMS = CDF Dims attribute, for rVariables typically 0
(rVariables are rarely used anymore See http://cdf.gsfc.nasa.gov/html/FAQ.html#intro)
CDFI.INQ.DECODING = 'HOST_DECODING' (can be network or host)
CDFI.INQ.ENCODING = 'NETWORK_ENCODING' (can be network or host)
CDFI.INQ.MAJORITY = 'ROW_MAJOR' (can be row or column)
CDFI.INQ.MAXREC = Max number of records (Default is -1)
CDFI.INQ.NVARS = number of rVariables, usually 0
CDFI.INQ.NZVARS = number of zVariables, usually all of them
CDFI.INQ.NATTS = number of variable attributes
CDFI.INQ.DIM = dimensions of rVariables
CDFI.g_atttributes = CDF global attributes, structure varies
Here is a sample from THEMIS EFI:
PROJECT STRING 'THEMIS'
SOURCE_NAME STRING 'THA>Themis Probe A'
DISCIPLINE STRING 'Space Physics>Magnetospheric Science'
DATA_TYPE STRING 'EFI'
DESCRIPTOR STRING 'L2>L2 DATA'
DATA_VERSION STRING '1'
PI_NAME STRING 'V. Angelopoulos, J. Bonnell & F. Mozer'
PI_AFFILIATION STRING 'UCB, NASA NAS5-02099'
TITLE STRING 'Electric Field Instrument (EFI) Measurements'
TEXT STRING 'THEMIS-A: Electric Field Instrument (EFI) Electric field measurements. The L2 product is a 3D estimate of'...
INSTRUMENT_TYPE STRING 'Electric Fields (space)'
MISSION_GROUP STRING 'THEMIS'
LOGICAL_SOURCE STRING 'tha_l2_efi'
LOGICAL_FILE_ID STRING 'tha_l2_efi_20131001_v01'
LOGICAL_SOURCE_DESCRIPTION
STRING 'Spacecraft-collected (EFI) Electric field'
TIME_RESOLUTION STRING '3-1/8s'
RULES_OF_USE STRING 'Open Data for Scientific Use'
GENERATED_BY STRING 'THEMIS SOC'
GENERATION_DATE STRING 'Sun Oct 6 03:11:38 2013'
ACKNOWLEDGEMENT STRING 'NASA Contract NAS5-02099'
MODS STRING 'Rev- 2009-09-16'
ADID_REF STRING 'NSSD0110'
LINK_TEXT STRING Array[3]
LINK_TITLE STRING Array[3]
HTTP_LINK STRING Array[3]
FILE_NAMING_CONVENTION
STRING 'source_descriptor_datatype'
CAVEATS STRING 'See THEMIS website for caveats'
VALIDITY STRING 'to be validated'
VALIDATOR STRING 'tbd'
VALIDATE STRING 'Compatible with the ISTP CDF Standards'
INST_MOD STRING 'THM>xxxx'
PARENTS STRING 'xxxx'
INST_SETTINGS STRING 'Not used'
SOFTWARE_VERSION
STRING '13273'
CDFI.NV = Number of variables
CDFI.VARS = AN array of CDFI.NV structures, one for each zvariable:
CDFI.VARS.NAME = The variable name
CDFI.VARS.NUM = The index of the given variable in the cdfi.vars array
CDFI.VARS.IS_ZVAR = 1 for a zVariable
CDFI.VARS.DATATYPE = The data type, e.g.'CDF_FLOAT'
CDFI.VARS.TYPE = The nummerical IDL data type (float is 4, etc...)
CDFI.VARS.NUMATTR = -1, Not sure about this one, returned from CDF_VARGET
CDFI.VARS.NUMELEM = Number of elements in a record, returned from CDF_VARGET
CDFI.VARS.RECVARY = Set to 1 if variable varies from record to record
CDFI.VARS.NUMREC = the number of records input.
CDFI.VARS.NDIMEN = the number dimensions in the data
CDFI.VARS.D = A six-element array with the number of dimensions for
each index
CDFI.VARS.DATAPTR = A pointer to the data array:
CDFI.VARS.ATTRPTR = A pointer to the variable attributes
structure for each variable. Content varies,
here is a sample from THEMIS EFI Electric
field data:
CATDESC STRING 'EFF_DOT0 (fast-survey, 1/8 sec time resolution, using E dot B=0) electric field vector in GSM coordinates'...
FIELDNAM STRING 'EFF_DOT0 (fast-survey, 1/8 sec time resolution, using E dot B=0) electric field vector in GSM coordinates'...
FILLVAL FLOAT NaN
VALIDMIN FLOAT Array[3]
VALIDMAX FLOAT Array[3]
VAR_TYPE STRING 'data'
DISPLAY_TYPE STRING 'time_series'
FORMAT STRING 'E13.6'
LABL_PTR_1 STRING 'tha_eff_dot0_gsm_labl'
UNITS STRING 'mV/m'
DEPEND_TIME STRING 'tha_eff_dot0_time'
DEPEND_EPOCH0 STRING 'tha_eff_dot0_epoch0'
DEPEND_0 STRING 'tha_eff_dot0_epoch'
DEPEND_1 STRING 'tha_eff_dot0_gsm_compno'
VAR_NOTES STRING 'Units are in mV/m'
COORDINATE_SYSTEM
STRING 'GSM'
REPRESENTATION_1
STRING 'Rep_xyz_gsm'
TENSOR_ORDER STRING '1'
AVG_TYPE STRING 'standard'
PROPERTY STRING 'vector'
SC_ID STRING 'a'
SCALE_TYP STRING 'linear'
DICT_KEY STRING 'electric_field>vector_GSM'
SI_CONVERSION STRING '1e-3>V/m'
LABEL_1 STRING 'tha_eff_dot0_gsm_labl'
Each variable may have a different set of attributes, but this
example is a minimal structure that will be ISTP compliant.
KEYWORDS:
VARFORMAT = string or string array: a string or string array (which may contain wildcard
characters) that specifies the CDF variable names to load. Use
'VARFORMAT='*' to load all variables. NOTE
THAT VARFORMAT MUST BE SET IF YOU ACTUALLY
WANT TO READ DATA.
VARNAMES = named variable ;output variable for variable names that were loaded.
SPDF_DEPENDENCIES : Set to 1 to have SPDF defined dependent variables also loaded.
VAR_TYPE = string or string array; Variables that have a VAR_TYPE matching these strings will
be loaded.
CONVERT_INT1_TO_INT2 Set this keyword to convert signed one byte to signed 2 byte integers.
This is useful because IDL does not have the equivalent of INT1 (bytes are unsigned)
RECORD: Specify the record index where you want to start reading. By default, this option will read one record.
NUMBER_RECORDS: Specify the number of records that you want to read. By default, this option will begin at record zero.
Note: Record & Number_Records can be used together to specify a range of records to be read.
Author: Davin Larson - 2006
Side Effects:
Data is returned in pointer variables. Calling routine is responsible for freeing up heap memory - otherwise a memory leak will occur.
$LastChangedBy: $
$LastChangedDate: $
$LastChangedRevision: $
$URL: $
(See projects/maven/l2gen/cdf_load_vars2.pro)
Version of CDF_SAVE_VARS with some fixes, square brackets. Hopefully
temporary, need to reconcile with data_dimen issues in CDF_SAVE_VARS
FUNCTION: dummy = cdf_save_vars(cdf_structure,new_cdf_name)
PURPOSE: To dump data and metadata from an IDL structure into a CDF file.
The structure format is the structure produced by cdf_load_vars.pro
INPUTS: cdf_structure : IDL structure defined by cdf_load_vars.pro
new_cdf_name : a string to name the new CDF file with
OUTPUTS: CDF file named by the new_cdf_name input
KEYWORDS: SET_COMPRESSION
Set this keyword to the compression type to be used for the
single-file CDF file. Note that individual CDF variables may use
compression types different than the one for the rest of the CDF
file. Valid compression types are:
0 = No Compression
1 = Run-Length Encoding
2 = Huffman
3 = Adaptive Huffman
5 = GZIP (see the optional GZIP_LEVEL keyword)
SET_GZIP_LEVEL
This keyword is used to indicate the desired effort for the GZIP
compression. This effort must be expressed as a scalar in the range
(1-9). If GZIP_LEVEL is not specified upon entry then the default
effort level is taken to be 5. If the SET_GZIP_LEVEL keyword is set
to a valid value, and the keyword SET_COMPRESSION is not specified,
the SET_COMPRESSION is set to GZIP (5).
EXAMPLE: dummy = cdf_save_vars(cdfi,'newcdf.cdf')
Written by: Matt Davis
Note: To use this routine you must have the CDF_EPOCH/CDF_EPOCH16 bug patch on your IDL6.3
and if you are using Solaris you need to be in 32-bit mode NOT 64-bit
(ie, idl -32)
$LastChangedBy: jimm $
$LastChangedDate: 2016-10-05 16:07:18 -0700 (Wed, 05 Oct 2016) $
$LastChangedRevision: 22049 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_4_0/projects/maven/l2gen/cdf_save_vars2.pro $
(See projects/maven/l2gen/cdf_save_vars2.pro)
unction dummy_maven_l2_struct, vars, only_1_time_array = only_1_time_array, $
instrument = instrument, _extra=_extra
Takes an array of tplot variables, creates a dummy CDF master
structure, inserts data, returns structure suitable for output.
If you set the only_1_time_array keyword, then there will be only 1
time_array, the instrument keyword also needs to be set, so that the
time variable name is 'instrument_time' (e.g., instrument = 'swe_l2'
will give a time variable name of 'swe_l2_time'. Otherwise, the time
variable name will be 'L2_time').
(See projects/maven/l2gen/dummy_maven_l2_struct.pro)
NAME:
mvn_call_swe_l2gen
CALLING SEQUENCE:
mvn_call_swe_l2gen
INPUT:
None -- the default is to read in a file
/disks/data/maven/data/sci/swe/l2/most_recent_l0_processed.txt
and process all files after that time
KEYWORDS:
time_in = the time for which old files created *after* this date
will be processed. E.g., if you pass in '2017-11-07'
then all files created after 7-nov-2017/00:00:00 will
be reprocessed
before_time = if set, process all of the files created *before* the
input time
days_in = An array of dates, e.g., ['2009-01-30','2009-02-01'] to
process. This ignores the input time. This option
replicates the proceesing done by
thm_reprocess_l2gen_days.
out_dir = the directory in which you write the data, default defined
by the MAVEN directory structure, e.g.:
'/disks/data/maven/data/sci/swe/l2/YYYY/MM/'
use_file4time = if set, use filenames for time test instead of file
modified time, useful for reprocessing
search_time_range = if set, then use this time range to find files
to be processed, instead of just before or after
time.
use_l2_files = if set, then use L2 files as input, and not
L0's. Note that L0 is still used for file
searching, so you might want to use this with the
days_in option. (passed through to mvn_swe_l2gen.pro)
l2only = if set, then insist on using L2 MAG data for generating
L2 PAD data
no_reset_time = if set, then don't reset the time in
most_recent_l0_processed.txt (useful for repocessing
data without affecting the cron job)
max_l0_files = This is the maximum number of L0 files that can be
processed, the deault is 30. This is a fail-safe to
avoid mass reprocessing of data, if a bunch of file
ctime values are inadvertently changed. If more files
than this limit are tried, an error email will be sent.
HISTORY:
Hacked from mvn_call_sta_l2gen, 17-Apr-2014, jmm
$LastChangedBy: jimm $
$LastChangedDate: 2020-01-24 09:13:41 -0800 (Fri, 24 Jan 2020) $
$LastChangedRevision: 28229 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_4_0/projects/maven/l2gen/mvn_call_swe_l2gen.pro $
(See projects/maven/l2gen/mvn_call_swe_l2gen.pro)
NAME:
mvn_call_swe_resample_pad_daily
PURPOSE:
Wrapper for mvn_swe_resample_pad_daily.pro, checks time for most
recent mag L1 data, and processes the new days.
CALLING SEQUENCE:
mvn_call_swe_resample_pad_daily
INPUT:
None -- the default is to process all files after a given time, from
a file
/disks/data/maven/data/sci/swe/l2/most_recent_l1mag_processed.txt
and process all files after that time.
KEYWORDS:
time_in = the time for which old files created *after* this date
will be processed. E.g., if you pass in '2017-11-07'
then all files created after 7-nov-2017/00:00:00 will
be reprocessed.
before_time = if set, process all of the files created *before* the
input time
days_in = An array of dates, e.g., ['2009-01-30','2009-02-01'] to
process. This ignores the input time. This option
replicates the proceesing done by
thm_reprocess_l2gen_days.
use_file4time = if set, use filenames for time test instead of file
modified time, useful for reprocessing
search_time_range = if set, then use this time range to find files
to be processed, instead of just before or after
time.
out_rootdir = if set, reset the root_data_dir variable to this
value, for testing,
l2only = if set, then insist on using L2 MAG data for generating
L2 PAD data
no_reset_time = if set, then don't reset the time in
most_recent_l1mag_processed.txt (useful for repocessing
data without affecting the cron job)
HISTORY:
Hacked from mvn_call_swe_l2gen, 14-Oct-2015, jmm
$LastChangedBy: jimm $
$LastChangedDate: 2015-10-13 16:02:56 -0700 (Tue, 13 Oct 2015) $
$LastChangedRevision: 19066 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_4_0/projects/maven/l2gen/mvn_call_swe_resample_pad_daily.pro $
(See projects/maven/l2gen/mvn_call_swe_resample_pad_daily.pro)
NAME: mvn_l2file_compress PURPOSE: compresses an output L2 file and creates an md5 file CALLING SEQUENCE: mvn_l2file_compress, fullfile0 INPUT: fullfile0 = the full-path filename for the revisionless cdf file OUTPUT: No explicit output, the revisioned file is written, an md5 sum file is written in the same directory, and the revisionless file is linked to the revsioned file KEYWORDS: None, so far HISTORY: 26-nov-2014, jmm, jimm@ssl.berkeley.edu $LastChangedBy: jimmpc1 $ $LastChangedDate: 2019-10-06 14:57:53 -0700 (Sun, 06 Oct 2019) $ $LastChangedRevision: 27823 $ $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_4_0/projects/maven/l2gen/mvn_l2file_compress.pro $
(See projects/maven/l2gen/mvn_l2file_compress.pro)
ro mvn_l2gen_multiprocess_a, function_in, nproc, offset, time_range, $
proc_workdir, remove_lock = remove_lock
function_in is an IDL main program (LEAVE OUT THE .pro)
THis function needs to run on inputs read in from start and end time files.
nproc is the number of processes that are desired
time_range is a 2xN array of start and end times,
proc_workdir is the directory where the function_in program lives,
each subprocess will create a directory, copy the function_in
program into that directory, and run it there. The function_in
program must pick up its start_time from a file
'function_in_start_time.txt' in that directory, and not work
past 'function_in_end_time.txt'. These files are created for the
first process, and the start_time file gets updated each time the
program runs.
23-apr-2009, jmm, Now only sets up the process, processing is handled
be generic_multiprocess_b.sh
Changed name, removed the SSW calls, for use for overplotting. Note
also that this version overwrites start and end times if the
subdirectories exist, jmm, 21-nov-2011
Added offset to be able to use multiple computers, jmm, 6-oct-2013
(See projects/maven/l2gen/mvn_l2gen_multiprocess_a.pro)
Remove zero length static log files from log directory ro mvn_l2gen_remove_0logs, dir = dir, _extra = _extra
(See projects/maven/l2gen/mvn_l2gen_remove_0logs.pro)
NAME:
mvn_swe_kpgen
PURPOSE:
Loads L0 data, creates L2 files for 1 day
CALLING SEQUENCE:
mvn_swe_kpgen, date = date
INPUT:
date keyword
KEYWORDS:
date = If set, the input date. The default is today
directory = If set, output into this directory, for testing
purposes, don't forget a slash '/' at the end.
HISTORY:
Hacked from mvn_swe_l2gen, 2015-01-25, jmm, jimm@ssl.berkeley.edu
jimm@ssl.berkeley.edu
$LastChangedBy: jimm $
$LastChangedDate: 2015-02-24 18:45:38 -0800 (Tue, 24 Feb 2015) $
$LastChangedRevision: 17037 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_4_0/projects/maven/l2gen/mvn_swe_kpgen.pro $
(See projects/maven/l2gen/mvn_swe_kpgen.pro)
NAME:
mvn_swe_l2gen
PURPOSE:
Loads L0 data, creates L2 files for 1 day
CALLING SEQUENCE:
mvn_swe_l2gen, date=date
INPUT:
None.
KEYWORDS:
DATE: If set, the input date. The default is today.
DIRECTORY: If set, output into this directory, for testing
purposes, don't forget a slash '/' at the end.
L2ONLY: If set, only generate PAD L2 data if MAG L2 data are available.
NOKP: If set, do not generate SWEA KP data.
NOL2: If set, do not generate SWEA L2 data.
ABINS: Anode bin mask -> 16 elements (0 = off, 1 = on)
Default = replicate(1,16)
DBINS: Deflector bin mask -> 6 elements (0 = off, 1 = on)
Default = replicate(1,6)
OBINS: 3D solid angle bin mask -> 96 elements (0 = off, 1 = on)
Default = reform(ABINS # DBINS)
MASK_SC: Mask the spacecraft blockage. This is in addition to any
masking defined by the ABINS, DBINS, and OBINS.
Default = 1 (yes). Set this to 0 to disable and use the
above 3 keywords only (not recommended!).
HISTORY:
Hacked from Matt F's crib_l0_to_l2.txt, 2014-11-14: jmm
Better memory management and added keywords to control processing: dlm
$LastChangedBy: muser $
$LastChangedDate: 2019-11-12 15:57:15 -0800 (Tue, 12 Nov 2019) $
$LastChangedRevision: 28013 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_4_0/projects/maven/l2gen/mvn_swe_l2gen.pro $
(See projects/maven/l2gen/mvn_swe_l2gen.pro)
NAME: mvn_swe_l2gen_1day PURPOSE: reads in a start and end date, and reprocesses all of the days in the interval. This is a main program, designed to be called from a shell script. Processes 1 day at a time CALLING SEQUENCE: .run mvn_swe_l2gen_1day INPUT: start_time, end_time are input from files mvn_swe_l2gen_1day_start_time.txt and mvn_swe_l2gen_1day_end_time.txt. OUTPUT: Maven SWE L2 files HISTORY: 2014-11-05, jmm, jimm@ssl.berkeley.edu
(See projects/maven/l2gen/mvn_swe_l2gen_1day.pro)
PROCEDURE: mvn_swe_resample_pad_daily
PURPOSE:
Resamples SWEA pitch angle distributions for one or more UT days and saves
the results in a tplot save/restore file(s). These can then be loaded with
mvn_swe_pad_restore. This pre-calculation saves time.
USAGE:
mvn_swe_resample_pad, trange
INPUTS:
trange: One or more dates, in any format accepted by time_double.
Only full UT days are processed; any fractional part of
the day is ignored. When trange has more than one element,
all days between the earliest and the latest (inclusive)
are processed.
KEYWORDS:
L2ONLY: Use L2 MAG data only. Skip any date(s) where L2 data are
incomplete or not available.
$LastChangedBy: dmitchell $
$LastChangedDate: 2017-04-05 10:03:02 -0700 (Wed, 05 Apr 2017) $
$LastChangedRevision: 23109 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_4_0/projects/maven/l2gen/mvn_swe_resample_pad_daily.pro $
CREATED BY: David L. Mitchell
(See projects/maven/l2gen/mvn_swe_resample_pad_daily.pro)
NAME: run_swe_L2gen PURPOSE: Designed to run from a cronjob, sets up a lock file, and processes. It the lock file exists, no processing CALLING SEQUENCE: run_swe_l2gen, noffset_sec = noffset_sec INPUT: none OUTPUT: none KEYWORDS: none HISTORY: 25-jun-2014, jmm, jimm@ssl.berkeley.edu $LastChangedBy: $ $LastChangedDate: $ $LastChangedRevision: $ $URL: $
(See projects/maven/l2gen/run_swe_l2gen.pro)
NAME: run_swe_repad PURPOSE: Designed to run from a cronjob, sets up a lock file, and processes. It the lock file exists, no processing CALLING SEQUENCE: run_swe_repad INPUT: none OUTPUT: none KEYWORDS: none HISTORY: 13-Oct-2015, jmm, jimm@ssl.berkeley.edu $LastChangedBy: jimmpc1 $ $LastChangedDate: 2017-09-05 11:35:05 -0700 (Tue, 05 Sep 2017) $ $LastChangedRevision: 23885 $ $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_4_0/projects/maven/l2gen/run_swe_repad.pro $
(See projects/maven/l2gen/run_swe_repad.pro)