This page was created by the IDL library routine
mk_html_help2.
Last modified: Thu Oct 24 15:35:45 2019.
Procedure: calc
Purpose: This routine takes a string as input and interprets the string as a mini-language.
This language can manipulate normal idl variables and tplot variables. The idl variables
that can be modified are non-system variables in the scope in which the procedure is called.
Inputs: s : A string that will be interpreted as the mini language
Keywords: error:
If an error occurs during processing this keyword will return a struct
that contains information about the error. If error is set at time of input
this routine will set it to undefined, before it interprets 's' to prevent internal errors.
Note that setting this keyword will supress automatic printing of errors unless the verbose keyword is set.
function_list: return an array of strings that lists of the names/syntax of
available functions in the mini_language. Return without processing 's'
if an argument in the returned name is in brackets it is optional
for example: min(x[,dim])
operator_list: return an array of strings that lists of the names of available
operators in the mini_language. Return without processing 's'
verbose: set this keyword if you want the routine to print errors to screen when you are using the error keyword.
quiet: set this keyword to supress printing of errors to the screen at all times.
gui_data_obj: If 'calc' is being used inside the gui, then the loaded_data object will
be passed in through this keyword. NOTE: end users should not even set
this argument.
replay: if 'calc' is being used from the GUI, this keyword is set when the GUI is replaying a document.
Users shouldn't set this keyword manually
overwrite_selections: tracking the user's selections to overwrite tplot variables that already exist.
Users shouldn't set this keyword manually
statusbar: the status bar object will be passed through this keyword. Users shouldn't set this keyword manually
historywin: the history window object will be passed through this keyword. Users shouldn't set this keyword manually
gui_id: the top level GUI widget ID -- for prompting the user for overwrites in the GUI. Users shouldn't set this keyword manually
interpolate: Set this to the name of the tplot variable that you'd like to interpolate data to. If you set this keyword to 1(ex /interpolate), but
don't name a specific variable, it will interpolate to the left-most variable in each binary operation. This second way of using this keyword should be used carefully in complicated expressions,
as it may not be obvious which interpolation operations are being performed
nan: Deprecated. Use /nan keyword to functions
cumulative_total: Deprecated. Use /cumulative keyword to total function.
Outputs: none, but it will modify various variables in your environment
Examples:
calc,'a = 5'
calc,'"pos_re" = "tha_state_pos"/6371.2'
calc,'a += 7',/v ;abbreviated verbose keyword
calc,'"tvar" = "tvar" + var'
calc,'"tvar" = ln("tvar")'
calc,'"tvar" = total("tvar"+3,2)'
calc,'"tvar" = -a + 5.43e-7 ^ ("thb_fgs_dsl_x" / total("thb_fgs_dsl_x"))
calc,operator_list=o,function_list=f
calc,'"th?_state_pos_re" = "th?_state_pos"/6371.2' ;globbing
Notes:
1. The language generally uses a fairly straightforward computational syntax. The main
difference from idl is that quoted strings are treated as tplot variables in this language
2. A full specification of language syntax in backus-naur form can be found
in the file bnf_formal.txt, the programmatic specification of this syntax
can be found in productions.pro
3. The language is parsed using an slr parser. The tables required to do this parsing
are generated and stored ahead of time in the file grammar.sav and parse_tables.sav
4. The routines that describe the evaluation rules for the language can be found in the file
mini_routines.pro
5. If you want to modify the way the language works you'll probably need to modify productions.pro,
regenerate the slr parse tables using save_calc_tables and modify/add routines to mini_routines.pro
6. Arrays must have the same dimensions to be combines, and tplot variables must also have the same times.
7. Procedures: min,max,mean,median,count,total all take a second argument that allow you to select the
dimension over which the operation is performed
8. Calc supports globbing in tplot variable operands, but for it to work, the output variable also needs to be a tplot variable with the same number of glob characters.
Correct: calc,'"th?_state_pos_re" = "th?_state_pos"/6371.2'
Incorrect: calc,'tha_state_pos_re = "th?_state_pos"/6371.2'
Incorrect: calc,'"th?_state_pos_re" = "th?_state_*"/6371.2'
See Also:
All routines in the ssl_general/mini directory
The techniques used for this interpreter are based on two books:
1. Compilers:Principles,Techniques,and Tools by Aho,Sethi,& Ullman 1986 (esp. Ch3(for lexical analysis) & Ch4(for SLR parser design))
2. Structure & Interpretation of Computer Programs by Abelson & Sussman 1996 (esp. Ch4(evaluators & meta-circular evaluator))
If you want to understand/modify this program it may help to use these books as
a reference.
Also see: thm_crib_calc.pro for examples of usage
ToDo: 1. Implement 0 argument functions
2. Implement keywords for functions
3. Implement procedures
4. Implement control statements
$LastChangedBy: pcruce $
$LastChangedDate: 2016-06-27 14:12:31 -0700 (Mon, 27 Jun 2016) $
$LastChangedRevision: 21373 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_3_2/general/mini/calc.pro $
(See general/mini/calc.pro)
Function: csstack
Purpose: This procedure implements the push,pop,& peek
methods for a traditional computer science
data structure: the stack. It is a basic
LIFO data structure.
Advantages over array:
1. store heterogenous elements of any type in a stack.
2. stack can grow as large as memory and you don't
need to know how big it will be in advance
3. You don't need to worry about how the data is stored
Disadvantages over array:
1. You can't directly apply operations to the data
structure
2. You are forced to use abstraction
Inputs: arg1:the meaning of the argument varies with syntax
arg2:the meaning of the argument varies with syntax
Keywords: push(optional) : set this to add an item to the stack and return the modified stack
pop(optional) : set this to remove an item from the stack and return the modified stack
peek(optional) : set this to return the top element on the stack
length(optional): set this if you want
to return the length
free(optional): set this if you want to free the
vector's memory without
creating a leak, it will return
the number of elements free'd
If no keywords are set, default behavior is push
Outputs: If push or pop are set it returns the modified stack
If peek is set it returns the top element on the stack
If length or free are set it returns a number of items
Syntax(each method is followed by examples):
push
stk = csstack(item)
stk = csstack(item,stk) ;stk can be defined or not
stk = csstack(item,stk,/push)
pop:
stk = csstack(stk,/pop) ;must have at least one element
peek:
item = csstack(stk,/peek) ;must have at least one element
length:
length = csvector(stk,/length)
free:
num = csvector(stk,/free)
NOTES: in the event of overflow during add the vector.a
component will double in size
Push stores a copy of the element not the element itself
If you want to do manual lengths and reads you can look
at the code, but I would recommend against cause you are
violating abstraction which means the internal representation
could change and invalidate your code.
This might be worth writing in O.O. idl as well
To get type flexibility it uses a pointer for every object
Thus if you aren't careful this function will eat your
system memory for breakfast. Use heap_gc to clean up if you
are running out of memory.
$LastChangedBy: pcruce $
$LastChangedDate: 2008-09-12 16:21:16 -0700 (Fri, 12 Sep 2008) $
$LastChangedRevision: 3487 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_3_2/general/mini/csstack.pro $
(See general/mini/csstack.pro)
Procedure: evaluate
Purpose: This routine performs the actual evaluation of an expression in the mini_language
It basically combines an slr shift/reduce parser with an evaluator
Description(of how it works, super-concise version):
This routine is essentially a table-driven stack machine.
It uses two tables action & goto.(both are 2-d arrays) to determine how to behave at any given step. Prior states are pushed onto the stack.
At each step, an action is performed, determined by the action table, and a new state is pushed onto the stack, determined by the goto table.
Some actions may also pop a certain number of states off of the stack.
Actions can be either shift actions or reduce actions(using reduction N).
A shift action, means that it doesn't have enough tokens yet to determine the next operation, so it should process an additional token.
A reduce action, means that it should perform some operation. Replacing a complex expression with a simplified expression by using one of the production rules.
One more note, the stack has flexible type. Each push or pop operation generally pushes or pops two items. One will be a numerical state(long int), another will be a data item.
The data items are often implemented as structs but can represent various types of items. (tokens, intermediate processing results, outputs, operators, variable identifiers)
Generally the data items on the stack are anything that might be required as an input to perform a reduction.
Inputs:
tk_list: a list of token structures from the lex routine
grammar: a grammar description structure
parse_tables: a parse table structure
Keywords: error: On error this routine returns a structure that describes the error that occurred
$LastChangedBy: pcruce $
$LastChangedDate: 2013-05-10 17:04:22 -0700 (Fri, 10 May 2013) $
$LastChangedRevision: 12331 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_3_2/general/mini/evaluate.pro $
(See general/mini/evaluate.pro)
Procedure: evaluator_routines
Purpose: When called this routine compiles a library of helper routines
for the evaluator of the mini_language
$LastChangedBy: pcruce $
$LastChangedDate: 2016-06-16 16:20:59 -0700 (Thu, 16 Jun 2016) $
$LastChangedRevision: 21331 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_3_2/general/mini/evaluator_routines.pro $
(See general/mini/evaluator_routines.pro)
Function: get_token
Purpose: this routine performs the meat of the work for the mini language lexical analyzer.
It will identify and return the next token in its string input. It uses a series
of regular expressions to identify various tokens.
Input: s: A string from which the token is taken
Output: A struct describing a token or an error struct
$LastChangedBy: pcruce $
$LastChangedDate: 2016-10-25 17:48:17 -0700 (Tue, 25 Oct 2016) $
$LastChangedRevision: 22195 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_3_2/general/mini/get_token.pro $
(See general/mini/get_token.pro)
Function: is_equal
Purpose: determines if two inputs are equal
like array_equal, but inputs can be structs
Will descend structs within structs recursive
Requires names of fields within structs to be the same
Does not descend pointers to verify targets, if two
different pointers have the same destination, it will return
false. This may cause some problems, but it also prevents any infinite
looping that can occur with pointer objects. (Future versions should fix this)
Will not work on objects.It will always return false. (It should be made to)
This is designed to be similar to the equal operation in LISP or Scheme. A
high level equivalency operator that will test without halting regardless of input type.
Does allow ambiguity between 0 element array and one element array
Inputs: a,b: can be anything
Outputs: 1:yes 0:no
Keywords: array_strict: set this if you don't want it to allow conflation of 0 dimensional types and 1 element arrays
NOTES:
Right now this routine has only been tested to demonstrate its adequacy for use with the mini_language,
It should be used with other routines with caution. If it ever reaches general purpose maturity, it will
be placed in a different ssl_general directory.
TODO: 1. Pointer & Object support needs to be added
2. Also struct tag strictness needs supported
$LASTCHANGEDBY: DAVIN-WIN $
$LASTCHANGEDDATE: 2007-10-22 07:26:16 -0700 (MON, 22 OCT 2007) $
$LASTCHANGEDREVISION: 1758 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_3_2/general/mini/is_equal.pro $
(See general/mini/is_equal.pro)
Procedure: mini_predicates
Purpose: mini_predicates compiles a library of type checking predicates for
many of the different types used in the mini language. Type predicates
that are not defined here are defined in evaluator_routines.pro
$LastChangedBy: pcruce $
$LastChangedDate: 2013-05-10 17:04:22 -0700 (Fri, 10 May 2013) $
$LastChangedRevision: 12331 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_3_2/general/mini/mini_predicates.pro $
(See general/mini/mini_predicates.pro)
Procedure: mini_routines
Purpose: Compiles a bunch of routines used to describe the evaluation rules
used by the language.
productions.pro actually describes which syntactical
rules used to compose each of the routines in this file.
Routines in this file should be used to evaluate a production.
Helper routines should go in evaluator_routines.pro or mini_predicates.pro
Exceptions: function_list,operator_list,get_function,mini_routines
TODO: 1. need to include linear algebraic functions in the set of available routines
(crossp,norm,normalize), also multivariable calculus functions(gradient,curl)
/nan flag set whenever possible, & statistical routines, skew,kurtosis,variance,stddev
2. consider putting function/operator list inside common block
NOTES:
these routines are intentionally designed to preserve type
i.e. not upgrade float to double or short to long unless required
It leaves decisions about type to the evaluator and/or user
trigonometric routines will transform inputs into floating point,
however
$LastChangedBy: pcruce $
$LastChangedDate: 2016-08-02 16:55:11 -0700 (Tue, 02 Aug 2016) $
$LastChangedRevision: 21594 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_3_2/general/mini/mini_routines.pro $
(See general/mini/mini_routines.pro)
Procedure: parse_table_routines
Purpose: Compiles a library of helper routines for the parse_table generator
It contains general purpose routing routines
anything table specific will be stored in slr.pro,
lk1.pro & lalr.pro (right now there is only support for slr parse tables)
$LastChangedBy: pcruce $
$LastChangedDate: 2012-07-12 15:50:21 -0700 (Thu, 12 Jul 2012) $
$LastChangedRevision: 10702 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_3_2/general/mini/parse_table_routines.pro $
(See general/mini/parse_table_routines.pro)
Procedure: productions
Purpose: generates the grammar for the mini_language. This file combined with the routines in mini_routines
describes most of the structure of the mini_language. By changing these one should be able to change
the language with relatively few modifications to other files.
Outputs: A structure that describes the grammar of the mini_language
$LastChangedBy: pcruce $
$LastChangedDate: 2016-06-16 16:20:59 -0700 (Thu, 16 Jun 2016) $
$LastChangedRevision: 21331 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_3_2/general/mini/productions.pro $
(See general/mini/productions.pro)
Procedure: save_calc_tables
Purpose: This simple routine calls the proper procedures to generate the files needed to run the mini language
You should run this routine if you've made a change to the mini_language descriptions and you want that change to
be reflected in the runtime behavior of calc.pro This routine generates two files: grammar.sav and parse_tables.sav
$LastChangedBy: pcruce $
$LastChangedDate: 2012-07-12 15:26:32 -0700 (Thu, 12 Jul 2012) $
$LastChangedRevision: 10700 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_3_2/general/mini/save_calc_tables.pro $
(See general/mini/save_calc_tables.pro)
Procedure: slr
Purpose: This routine will generate SLR(1) parse tables for a bottom-up shift/
reduce parcer. The entire algorithm is described in "Compilers:
Principles, Tools, and Techniques" by Aho, Sethi, & Ullman 1986 Section
4.7
Helper functions in this file are specific to the SLR technique
and uses a similar naming schema to that used in Aho,Sethi & Ullman.
Different versions of closure, goto, & items are needed for LARL or LRK parse table generation methods.
These have not been implemented
Inputs: grammar: A grammar description structure from which the parse_tables are generated
(see productions.pro)
Keywords: parse_tables: A structure storing the parse tables for the grammar are returned through this named variable
$LastChangedBy: pcruce $
$LastChangedDate: 2013-05-10 17:04:22 -0700 (Fri, 10 May 2013) $
$LastChangedRevision: 12331 $
$URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_3_2/general/mini/slr.pro $
(See general/mini/slr.pro)
Procedure: string_glob_preprocess Purpose: Preprocesses tokenized input from the mini-language to add globbing support Each time there is a '?' or '*' character in a tplot variable, it creates another copy of the token list with specific values filled in. Because an output variable must be selected the output variable should contain globbing tokens to avoid errors. Inputs: l : The token list after lexical analyzer. Outputs: gl : The list of token lists after glob preprocessing. Keywords: error: Returns an error struct if illegal globbing is used $LastChangedBy: pcruce $ $LastChangedDate: 2015-09-12 11:37:42 -0700 (Sat, 12 Sep 2015) $ $LastChangedRevision: 18779 $ $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_3_2/general/mini/string_glob_preprocess.pro $
(See general/mini/string_glob_preprocess.pro)
Procedure: string_var_preprocess Purpose: Preprocesses tokenized input from the mini-language to implement string concatenation operator through a preprocessor stage Inputs: l : The token list after lexical analyzer. Outputs: sl : The list of token lists after string var preprocessing. Keywords: error: Returns an error struct if problem occurred. Verbose: Set to get more detailed output $LastChangedBy: pcruce $ $LastChangedDate: 2015-09-18 11:13:49 -0700 (Fri, 18 Sep 2015) $ $LastChangedRevision: 18839 $ $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/tags/spedas_3_2/general/mini/string_var_preprocess.pro $
(See general/mini/string_var_preprocess.pro)