README for download of Greenland data from TGO (as of May 2011)

NB: As of November 2011 we have moved to automated download and processing of Greenland data. The new system no longer uses template files. 
See the admin guide for details of the download and processing scripts.

DTU/TGO data is currently downloaded manually when need arises. 
The complete list of DTU/TGO sites THEMIS has data for can be found at: trunk/src/config/green_site_list_new.txt
It is this list that is used to download the files (list may need updating as more sites become available).
Configuration settings, including dates to check for data, can be found in trunk/src/config/wget_green_gmag.config
(The file to use as site list is set in call_wget_green_gmag.ksh)

To download new data
 - change the config file and/or site list as necessary (be sure to run thm_update_production_config.sh after commiting files (from astraea, login as thmsw))
 - run the script call_wget_green_gmag.ksh (from astraea). This downloads the html files from the TGO website to the directory greenMag2do (as specified in the config file)
 - run green_ascii2all_new.ksh. This creates txt and cdf files of the data.

Note: move existing files in greenMag2do to greenMag2do/tempdir before running green_ascii2all_new or they will all be reprocessed.
 
Note: the html files are read using a standard template greenmag_l2gen.template, for some reason files from the site TDC (TristanDaCunha) do not conform to the same template.
To process files from TDC change the template listed in thm_greenmag_l2gen_get_template_path.pro to greenmag_l2gen_tdc.template (recomit and run thm_update_production_idl.sh)

If other files do not process properly due to problems with the template, a new template can be created using:
templ = ascii_template(filename)
where the file is the html file the template is needed for.
And saved with:
save, templ, filename = ...
(Note, the variable must be called templ as it is referred by name in processing - the last letter is an L not a one.)
The name of the template file then needs to be updated in thm_greenmag_l2gen_get_template_path.pro.
Note that the template needs to be created from the html exactly as it will be read in processing - using 'save as' to save the html will add extra lines that will mess up the template.



