1) UPDATING IDL PATH: The muser SPEDAS working copy is under the directory /home/muser/export_socware/idl_socware/. To update, cd into that directory and run svn update, the SVN passphrase is the same as for muser login. 2) The following PFP L2/QL cronjobs are run by the muser user on mojo, deimos: MOJO JOBS: # once per day, manage L2 access 17 11 * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/private/mvn_manage_l2access.sh >/dev/null 2>&1 # Once every half-hour, STA L2 process from L0 files 22,52 * * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/muser_sta_l2gen.sh >/dev/null 2>&1 # run STA L2 reprocess from L2 files, should increment and only run once per day, 7 4,7,10,13,16,19,22 * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/muser_sta_l2reprocess.sh >/dev/null 2>&1 # Once an hour, remove 0 length STATIC L2 log files 40 * * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/mvn_l2gen_remove_0logs.sh >/dev/null 2>&1 # Once an hour MAG L2 process 32 * * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/run_mag_l2gen.sh >/dev/null 2>&1 # Once a day, process any new files for PFPL2 plots 55 11 * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/quicklook/run_pfpl2plot.sh >/dev/null 2>&1 # Once an hour, run SWE repad 14 * * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/run_swe_repad.sh >/dev/null 2>&1 # Once a day, update spice kernels 23 3 * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/muser_spice_update.sh >/dev/null 2>&1 DEIMOS JOBS: # MAVEN SWE L2 Processing, moved to mojo, temporarily, 2015-05-11, moved # back 2015-05-29 23,53 * * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/run_swe_l2gen.sh >/dev/null 2>&1 The shell scripts are in the IDL path, /home/muser/export_socware/idl_socware/projects/maven/ in the l2gen and quicklook directories. 3) JOB DESCRIPTIONS: # once per day, manage L2 access 17 11 * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/private/mvn_manage_l2access.sh >/dev/null 2>&1 The script mvn_manage_l2access.sh calls the procedure mvn_manage_l2access for the 'pfp', 'sta', 'swe', and 'mag' directories under /disks/data/maven/data/sci. This program sets up the .htaccess files for the level 2 (l2) and quicklook (ql) subdirectories to password protect unreleased data. # Once every half-hour, STA L2 process from L0 files 22,52 * * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/muser_sta_l2gen.sh >/dev/null 2>&1 The script muser_sta_l2gen.sh picks up any new L0 files that have not yet been processed. The file creation time for the most recent L0 files processed can be found in the file: /disks/data/maven/data/sci/sta/l2/most_recent_l0_processed.txt. This process will wait for any L2 to L2 processes to finish before starting. Process log files are found in the directory /disks/mojo/home/maven/stalog, caled run_sta_l2gen.txtYYYYMMDDhhmmss The STA L0 to L2 process also creates LPW tplot files in the directories under /disks/data/maven/data/sci/lpw/tplot_l0 # run STA L2 reprocess from L2 files, should increment and only run once per day, 7 4,7,10,13,16,19,22 * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/muser_sta_l2reprocess.sh >/dev/null 2>&1 The script muser_sta_l2reprocess.sh calls the program mvn_call_sta_l1l2.pro that reprocesses STATIC L2 files for 3, 10, 30, and 60 days ago. It is set up to run only once per day, and will not run if there is a current L0 to L2 process running. Process log files are found in the directory /disks/mojo/home/maven/stalog, caled run_sta_l2reprocess.txtYYYYMMDDhhmmss # Once an hour, remove 0 length STATIC L2 log files 40 * * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/mvn_l2gen_remove_0logs.sh >/dev/null 2>&1 If an L0 process does not find any new files, there is an empty log file. This process clears out empty log files. # Once an hour MAG L2 process 32 * * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/run_mag_l2gen.sh >/dev/null 2>&1 The script run_mag_l2gen.sh calls the mvn_mag_batch.pro program that generates the MAG L2 IDL sav files. # Once a day, process any new files for PFPL2 plots 55 11 * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/quicklook/run_pfpl2plot.sh >/dev/null 2>&1 The script run_pfpl2plot.sh calls the mvn_call_pfpl2plot.pro routine that produces the PFP L2 browser plots, for any new L2 files created in the last day. These include the full-day, single-orbit, periapsis, and long-term plots created in the directory /disks/data/maven/data/sci/pfp/l2/plots. (Note that plots in /disks/data/maven/data/sci/pfp/ql/ are created by a different process that is called when the L0 files are moved into the database.) # Once an hour, run SWE repad 14 * * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/run_swe_repad.sh >/dev/null 2>&1 The script run_swe_repad.sh calls the program run_swe_repad.pro which calls the program mvn_call_swe_repad_daily.pro, which checks for new L1 mag data, and reprocesses SWEA pitch angle distribution data. The date for the most recent L1 mag data processed is held in the file /dsks/data/maven/data/sci/swe/l2/most_recent_l1mag_processed.txt. # Once a day, update spice kernels 23 3 * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/muser_spice_update.sh >/dev/null 2>&1 The script muser_spice_update.sh calls mvn_load_spice.pro that downloads and loads the latest SPICE kernels. # MAVEN SWE L2 Processing, 23,53 * * * * /bin/csh /home/muser/export_socware/idl_socware/projects/maven/l2gen/run_swe_l2gen.sh >/dev/null 2>&1 The script run_swe_l2gen.sh calls the program mvn_call_swe_l2gen which runs the L0 to L2 process for new L0 files. The file creation time for the most recent L0 files processed can be found in the file: /disks/data/maven/data/sci/swe/l2/most_recent_l0_processed.txt. 4) IF JOBS CRASH OR HANG OR NEED RESTARTING: FOR STATIC L0 -> L2 Delete the file /disks/mojo/home/maven/muser/STAL2lock.txt and the process will repeat for files created after the time listed in /disks/data/maven/data/sci/sta/l2/most_recent_l0_processed.txt FOR STATIC L2 -> L2 Delete the file /disks/mojo/home/maven/muser/STAL2Rlock.txt And the process will run dates after the time listed in /disks/data/maven/data/sci/sta/l2/most_recent_l2_processed.txt FOR SWEA L0 -> L2 Delete (on deimos) /mydisks/home/maven/muser/SWEL2lock.txt and the process will repeat for files created after the time listed in /disks/data/maven/data/sci/swe/l2/most_recent_l0_processed.txt 5) REPROCESSING DATA: For some reprocessing jobs, the most_recent_l0_processed.txt file can be edited and the automated process will pick up all L0 files created after this time. To reprocess a single day, it is best to call idl and run the appropriate program, e.g. mvn_sta_l2gen, date = '2???-??-??' for the appropriate date. Other reprocess scripts exist for more complicated reprocesses. NEVER 'touch' Level 0 files, unless absolutely necessary, since this will trigger reprocessing of those files for all instruments. EXAMPLES: The best way to reprocess a small amount of data is to use the program that starts with "mvn_call_", and use the days_in keyword, e.g., To reprocess STATIC L2 data for multiple days: mvn_call_sta_l2gen, days_in = ['2019-12-11', '2019-12-21'] ;for two days Or SWEA data: mvn_call_swe_l2gen, days_in = '2019-12-'+['01','02','03'] Or PFP L2 plots; mvn_call_pfpl2plot, days_in = '2019-11-'+string(1+indgen(30), format='(i.2.2)') ;for a full month 6) ADDING NEW USER FOR MAVEN DATA ACCESS: See: /disks/data/maven/pfp/.hidden/README.html