This page was created by the IDL library routine `mk_html_help2`.

Last modified: Mon Nov 5 01:15:46 2012.

# Directory Listing of Routines

# ssl_general/mini

- CALC
- CSSTACK
- EVALUATE
- EVALUATOR_ROUTINES
- GET_TOKEN
- IS_EQUAL
- MINI_PREDICATES
- MINI_ROUTINES
- PARSE_TABLE_ROUTINES
- PRODUCTIONS
- SAVE_CALC_TABLES
- SLR
- STRING_GLOB_PREPROCESS

# Routine Descriptions

## CALC

[Next Routine] [List of Routines]

     Procedure: calc

     Purpose:  This routine takes a string as input and interprets the string as a mini-language.
        This language can manipulate normal idl variables and tplot variables.  The idl variables
        that can be modified are non-system variables in the scope in which the procedure is called.

     Inputs: s : A string that will be interpreted as the mini language

     Keywords: error:
                  If an error occurs during processing this keyword will return a struct
                  that contains information about the error.  If error is set at time of input
                  this routine will set it to undefined, before it interprets 's' to prevent internal errors.

                  Note that setting this keyword will supress automatic printing of errors unless the verbose keyword is set.

               function_list: return an array of strings that lists of the names/syntax of
                              available functions in the mini_language.  Return without processing 's'
                              if an argument in the returned name is in brackets it is optional
                              for example:  min(x[,dim])
               operator_list:  return an array of strings that lists of the names of available
                               operators in the mini_language. Return without processing 's'

               verbose:  set this keyword if you want the routine to print errors to screen when you are using the error keyword.

               quiet: set this keyword to supress printing of errors to the screen at all times.

               gui_data_obj: If 'calc' is being used inside the gui, then the loaded_data object will
                             be passed in through this keyword.  NOTE: end users should not even set
                             this argument.

               interpolate:  Set this to the name of the tplot variable that you'd like to interpolate data to.  If you set this keyword to 1(ex /interpolate), but
               don't name a specific variable, it will interpolate to the left-most variable in each binary operation.  This second way of using this keyword should be used carefully in complicated expressions,
               as it may not be obvious which interpolation operations are being performed

               nan: Set nan=0 to disable the /nan keyword when calc calls IDL functions that accept the /nan keyword.
                 For example: "total","min","max","mean".
                 Default: nan=1, ie nans are treated as missing data by default.
                 Default is set to 1 to maintain backwards compatibility with calc code from before that this feature existed.

     Outputs: none, but it will modify various variables in your environment

     Examples:
        calc,'a = 5'
        calc,'"pos_re" = "tha_state_pos"/6374'
        calc,'a += 7',/v ;abbreviated verbose keyword
        calc,'"tvar" = "tvar" + var'
        calc,'"tvar" = ln("tvar")'
        calc,'"tvar" = total("tvar"+3,2)'
        calc,'"tvar" = -a + 5.43e-7 ^ ("thb_fgs_dsl_x" / total("thb_fgs_dsl_x"))
        calc,operator_list=o,function_list=f
        calc,'"th?_state_pos_re" = "th?_state_pos"/6374.4' ;globbing

     Notes:
        1. The language generally uses a fairly straightforward computational syntax.  The main
           difference from idl is that quoted strings are treated as tplot variables in this language
        2. A full specification of language syntax in backus-naur form can be found
           in the file bnf_formal.txt, the programmatic specification of this syntax
           can be found in productions.pro
        3. The language is parsed using an slr parser.  The tables required to do this parsing
           are generated and stored ahead of time in the file grammar.sav and parse_tables.sav
        4. The routines that describe the evaluation rules for the language can be found in the file
           mini_routines.pro
        5. If you want to modify the way the language works you'll probably need to modify productions.pro,
           regenerate the slr parse tables using save_calc_tables and modify/add routines to mini_routines.pro
        6. Arrays must have the same dimensions to be combines, and tplot variables must also have the same times.
        7. Procedures: min,max,mean,median,count,total  all take a second argument that allow you to select the
           dimension over which the operation is performed
        8. Calc supports globbing in tplot variable operands, but for it to work, the output variable also needs to be a tplot variable with the same number of glob characters.
           Correct: calc,'"th?_state_pos_re" = "th?_state_pos"/6374.4'
           Incorrect: calc,'tha_state_pos_re = "th?_state_pos"/6374.4'
           Incorrect: calc,'"th?_state_pos_re" = "th?_state_*"/6374.4'

     See Also:
       All routines in the ssl_general/mini directory
       The techniques used for this interpreter are based on two books:

       1. Compilers:Principles,Techniques,and Tools by Aho,Sethi,& Ullman 1986 (esp. Ch3(for lexical analysis) & Ch4(for SLR parser design))

       2. Structure & Interpretation of Computer Programs by Abelson & Sussman 1996 (esp. Ch4(evaluators & meta-circular evaluator))

       If you want to understand/modify this program it may help to use these books as
       a reference.

       Also see:  thm_crib_calc.pro for examples of usage

     ToDo: 1. Implement 0 argument functions
           2. Implement keywords for functions
           3. Implement procedures
           4. Implement control statements

     $LastChangedBy: pcruce $
     $LastChangedDate: 2012-08-14 14:08:36 -0700 (Tue, 14 Aug 2012) $
     $LastChangedRevision: 10817 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/ssl_general/trunk/mini/calc.pro $

(See ssl_general/mini/calc.pro)

## CSSTACK

[Previous Routine] [Next Routine] [List of Routines]

     Function: csstack

     Purpose:  This procedure implements the push,pop,& peek
               methods for a traditional computer science
               data structure: the stack.  It is a basic
               LIFO data structure.

               Advantages over array:
               1. store heterogenous elements of any type in a stack.
               2. stack can grow as large as memory and you don't
               need to know how big it will be in advance
               3. You don't need to worry about how the data is stored

               Disadvantages over array:
               1. You can't directly apply operations to the data
               structure
               2. You are forced to use abstraction

               Inputs: arg1:the meaning of the argument varies with syntax
                       arg2:the meaning of the argument varies with syntax

               Keywords: push(optional) : set this to add an item to the stack and return the modified stack
                         pop(optional) : set this to remove an item from the stack and return the modified stack
                         peek(optional) : set this to return the top element on the stack
                         length(optional): set this if you want
                                           to return the length
                         free(optional): set this if you want to free the
                                         vector's memory without
                                         creating a leak, it will return
                                         the number of elements free'd

                        If no keywords are set, default behavior is push

               Outputs: If push or pop are set it returns the modified stack
                        If peek is set it returns the top element on the stack
                        If length or free are set it returns a number of items

               Syntax(each method is followed by examples):

                       push
                        stk = csstack(item)
                        stk = csstack(item,stk)  ;stk can be defined or not
                        stk = csstack(item,stk,/push)
                       pop:
                        stk = csstack(stk,/pop) ;must have at least one element
                       peek:
                        item = csstack(stk,/peek) ;must have at least one element
                       length:
                         length = csvector(stk,/length)
                       free:
                         num = csvector(stk,/free)

           NOTES: in the event of overflow during add the vector.a
           component will double in size

           Push stores a copy of the element not the element itself

           If you want to do manual lengths and reads you can look
           at the code, but I would recommend against cause you are
           violating abstraction which means the internal representation
           could change and invalidate your code.

           This might be worth writing in O.O. idl as well

           To get type flexibility it uses a pointer for every object
           Thus if you aren't careful this function will eat your
           system memory for breakfast.  Use heap_gc to clean up if you
           are running out of memory.

     $LastChangedBy: pcruce $
     $LastChangedDate: 2008-09-12 16:21:16 -0700 (Fri, 12 Sep 2008) $
     $LastChangedRevision: 3487 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/ssl_general/trunk/mini/csstack.pro $

(See ssl_general/mini/csstack.pro)

## EVALUATE

[Previous Routine] [Next Routine] [List of Routines]

     Procedure: evaluate

     Purpose:  This routine performs the actual evaluation of an expression in the mini_language
               It basically combines an slr shift/reduce parser with an evaluator

     Description(of how it works, super-concise version):
       This routine is essentially a table-driven stack machine.
       It uses two tables action & goto.(both are 2-d arrays) to determine how to behave at any given step.  Prior states are pushed onto the stack.
       At each step, an action is performed, determined by the action table, and a new state is pushed onto the stack, determined by the goto table.
       Some actions may also pop a certain number of states off of the stack.

       Actions can be either shift actions or reduce actions(using reduction N).
         A shift action, means that it doesn't have enough tokens yet to determine the next operation, so it should process an additional token.
         A reduce action, means that it should perform some operation.  Replacing a complex expression with a simplified expression by using one of the production rules.

       One more note, the stack has flexible type.  Each push or pop operation generally pushes or pops two items.  One will be a numerical state(long int), another will be a data item.
         The data items are often implemented as structs but can represent various types of items. (tokens, intermediate processing results, outputs, operators, variable identifiers)
         Generally the data items on the stack are anything that might be required as an input to perform a reduction.

     Inputs:
        tk_list:  a list of token structures from the lex routine
        grammar:  a grammar description structure
        parse_tables: a parse table structure

     Keywords: error: On error this routine returns a structure that describes the error that occurred

     $LastChangedBy: pcruce $
     $LastChangedDate: 2012-07-12 15:50:21 -0700 (Thu, 12 Jul 2012) $
     $LastChangedRevision: 10702 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/ssl_general/trunk/mini/evaluate.pro $

(See ssl_general/mini/evaluate.pro)

## EVALUATOR_ROUTINES

[Previous Routine] [Next Routine] [List of Routines]

     Procedure: evaluator_routines

     Purpose: When called this routine compiles a library of helper routines
              for the evaluator of the mini_language

     $LastChangedBy: pcruce $
     $LastChangedDate: 2011-01-25 12:10:36 -0800 (Tue, 25 Jan 2011) $
     $LastChangedRevision: 8026 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/ssl_general/trunk/mini/evaluator_routines.pro $

(See ssl_general/mini/evaluator_routines.pro)

## GET_TOKEN

[Previous Routine] [Next Routine] [List of Routines]

     Function: get_token

     Purpose: this routine performs the meat of the work for the mini language lexical analyzer.
              It will identify and return the next token in its string input.  It uses a series
              of regular expressions to identify various tokens.

     Input: s: A string from which the token is taken

     Output: A struct describing a token or an error struct

     $LastChangedBy: lphilpott $
     $LastChangedDate: 2011-03-14 14:11:19 -0700 (Mon, 14 Mar 2011) $
     $LastChangedRevision: 8388 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/ssl_general/trunk/mini/get_token.pro $

(See ssl_general/mini/get_token.pro)

## IS_EQUAL

[Previous Routine] [Next Routine] [List of Routines]

    Function: is_equal

    Purpose: determines if two inputs are equal
             like array_equal, but inputs can be structs
             Will descend structs within structs recursive
             Requires names of fields within structs to be the same

             Does not descend pointers to verify targets, if two
             different pointers have the same destination, it will return
             false.  This may cause some problems, but it also prevents any infinite
             looping that can occur with pointer objects.  (Future versions should fix this)

             Will not work on objects.It will always return false. (It should be made to)

             This is designed to be similar to the equal operation in LISP or Scheme.  A
             high level equivalency operator that will test without halting regardless of input type.

             Does allow ambiguity between 0 element array and one element array

    Inputs: a,b: can be anything

    Outputs: 1:yes 0:no

    Keywords: array_strict: set this if you don't want it to allow conflation of 0 dimensional types and 1 element arrays

    NOTES:
            Right now this routine has only been tested to demonstrate its adequacy for use with the mini_language,
            It should be used with other routines with caution.  If it ever reaches general purpose maturity, it will
            be placed in a different ssl_general directory.

    TODO:  1. Pointer & Object support needs to be added
           2.  Also struct tag strictness needs supported

     $LASTCHANGEDBY: DAVIN-WIN $
     $LASTCHANGEDDATE: 2007-10-22 07:26:16 -0700 (MON, 22 OCT 2007) $
     $LASTCHANGEDREVISION: 1758 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/ssl_general/trunk/mini/is_equal.pro $

(See ssl_general/mini/is_equal.pro)

## MINI_PREDICATES

[Previous Routine] [Next Routine] [List of Routines]

    Procedure: mini_predicates

    Purpose:  mini_predicates compiles a library of type checking predicates for
              many of the different types used in the mini language.  Type predicates
              that are not defined here are defined in evaluator_routines.pro

     $LastChangedBy: pcruce $
     $LastChangedDate: 2009-07-08 17:53:08 -0700 (Wed, 08 Jul 2009) $
     $LastChangedRevision: 6401 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/ssl_general/trunk/mini/mini_predicates.pro $

(See ssl_general/mini/mini_predicates.pro)

## MINI_ROUTINES

[Previous Routine] [Next Routine] [List of Routines]

    Procedure: mini_routines

    Purpose:  Compiles a bunch of routines used to describe the evaluation rules
              used by the language.  productions.pro actually describes which syntactical
              rules use each of the routines in this file. Also see the routines 'function_list' in
              this file to see which names get bound to which routines in the language

    TODO: 1. need to include linear algebraic functions in the set of available routines
            (crossp,norm,normalize), also multivariable calculus functions(gradient,curl)
            /nan flag set whenever possible, & statistical routines, skew,kurtosis,variance,stddev

          2. consider putting function/operator list inside common block

     NOTES:
          these routines are intentionally designed to preserve type
          i.e. not upgrade float to double or short to long unless required
          It leaves decisions about type to the evaluator and/or user
          trigonometric routines will transform inputs into floating point,
          however

     $LastChangedBy: pcruce $
     $LastChangedDate: 2012-08-14 14:08:36 -0700 (Tue, 14 Aug 2012) $
     $LastChangedRevision: 10817 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/ssl_general/trunk/mini/mini_routines.pro $

(See ssl_general/mini/mini_routines.pro)

## PARSE_TABLE_ROUTINES

[Previous Routine] [Next Routine] [List of Routines]

    Procedure: parse_table_routines

    Purpose: Compiles a library of helper routines for the parse_table generator
              It contains general purpose routing routines
              anything table specific will be stored in slr.pro,
              lk1.pro & lalr.pro (right now there is only support for slr parse tables)

     $LastChangedBy: pcruce $
     $LastChangedDate: 2012-07-12 15:50:21 -0700 (Thu, 12 Jul 2012) $
     $LastChangedRevision: 10702 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/ssl_general/trunk/mini/parse_table_routines.pro $

(See ssl_general/mini/parse_table_routines.pro)

## PRODUCTIONS

[Previous Routine] [Next Routine] [List of Routines]

    Procedure: productions

    Purpose:  generates the grammar for the mini_language.  This file combined with the routines in mini_routines
              describes most of the structure of the mini_language.  By changing these one should be able to change
              the language with relatively few modifications to other files.

    Outputs: A structure that describes the grammar of the mini_language

     $LastChangedBy: pcruce $
     $LastChangedDate: 2012-07-12 15:26:32 -0700 (Thu, 12 Jul 2012) $
     $LastChangedRevision: 10700 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/ssl_general/trunk/mini/productions.pro $

(See ssl_general/mini/productions.pro)

## SAVE_CALC_TABLES

[Previous Routine] [Next Routine] [List of Routines]

    Procedure: save_calc_tables

    Purpose:  This simple routine calls the proper procedures to generate the files needed to run the mini language
              You should run this routine if you've made a change to the mini_language descriptions and you want that change to
              be reflected in the runtime behavior of calc.pro  This routine generates two files: grammar.sav and parse_tables.sav

     $LastChangedBy: pcruce $
     $LastChangedDate: 2012-07-12 15:26:32 -0700 (Thu, 12 Jul 2012) $
     $LastChangedRevision: 10700 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/ssl_general/trunk/mini/save_calc_tables.pro $

(See ssl_general/mini/save_calc_tables.pro)

## SLR

[Previous Routine] [Next Routine] [List of Routines]

    Procedure: slr

    Purpose: This routine will generate SLR(1) parse tables for a bottom-up shift/
             reduce parcer.  The entire algorithm is described in "Compilers:
             Principles, Tools, and Techniques" by Aho, Sethi, & Ullman 1986 Section
             4.7

             Helper functions in this file are specific to the SLR technique
             and uses a similar naming schema to that used in Aho,Sethi & Ullman.
             Different versions of closure, goto, & items are needed for LARL or LRK parse table generation methods.
             These have not been implemented

     Inputs: grammar:  A grammar description structure from which the parse_tables are generated
                       (see productions.pro)

     Keywords: parse_tables:  A structure storing the parse tables for the grammar are returned through this named variable

     $LastChangedBy: pcruce $
     $LastChangedDate: 2012-07-12 15:50:21 -0700 (Thu, 12 Jul 2012) $
     $LastChangedRevision: 10702 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/ssl_general/trunk/mini/slr.pro $

(See ssl_general/mini/slr.pro)

## STRING_GLOB_PREPROCESS

[Previous Routine] [List of Routines]

     Procedure: string_glob_preprocess

     Purpose: Preprocesses tokenized input from the mini-language to add globbing support
     Each time there is a '?' or '*' character in a tplot variable, it creates another copy of
     the token list with specific values filled in.  Because an output variable must be selected
     the output variable should contain globbing tokens to avoid errors.

     Inputs: l : The token list after lexical analyzer.

     Outputs: gl : The list of token lists after glob preprocessing.

     Keywords: error: Returns an error struct if illegal globbing is used

     $LastChangedBy: pcruce $
     $LastChangedDate: 2011-06-10 13:33:56 -0700 (Fri, 10 Jun 2011) $
     $LastChangedRevision: 8731 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/ssl_general/trunk/mini/string_glob_preprocess.pro $

(See ssl_general/mini/string_glob_preprocess.pro)